2024-05-20T20:33:00.338 INFO:root:teuthology version: 0.0.1.dev276+g4fbe5e8 2024-05-20T20:33:00.338 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --name leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983 -- /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/orig.config.yaml 2024-05-20T20:33:00.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T20:33:00.482 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983 branch: wip-lusov-quiesce description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7715983' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 4ebd1d5361d457738957dc0455bba90102296634 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 4ebd1d5361d457738957dc0455bba90102296634 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce sha1: 4ebd1d5361d457738957dc0455bba90102296634 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 202 sha1: 4ebd1d5361d457738957dc0455bba90102296634 sleep_before_teardown: 0 subset: 941/111111 suite: fs suite_branch: wip-lusov-quiesce suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 4ebd1d5361d457738957dc0455bba90102296634 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH94DB/ArPbA9slEZyw7UsFPtn415LxD70Ohu53eSzEK2M4tmVNGzI0YTZCJSEbbJvF+paqSGOOu9/GBtbx8XTQ= smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOceghuAt2xCh7SAA4kXy89Vpu7EiGPRy1B8UX+5OL5Ni/AdkLR95xz6wQrjQ0WNt1TUcaPCdspHNMwKawvhso= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDe6kYwkIY8LgqrD4JJLzygwA2Ng83ubO4qAQSdBwB1ytf8aUDzGlU/eqTp06AvCPdyPSaoTV6httuOjRe7jCJk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: install-fix teuthology_sha1: 4fbe5e8dc899215f216853b8569a107d8be8861d timestamp: 2024-05-20_18:42:22 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-20T20:33:00.482 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa; will attempt to use it 2024-05-20T20:33:00.483 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks 2024-05-20T20:33:00.483 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-20T20:33:00.484 INFO:teuthology.task.internal:Checking packages... 2024-05-20T20:33:00.504 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4ebd1d5361d457738957dc0455bba90102296634' 2024-05-20T20:33:00.504 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-20T20:33:00.504 INFO:teuthology.packaging:ref: None 2024-05-20T20:33:00.504 INFO:teuthology.packaging:tag: None 2024-05-20T20:33:00.504 INFO:teuthology.packaging:branch: wip-lusov-quiesce 2024-05-20T20:33:00.504 INFO:teuthology.packaging:sha1: 4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:33:00.504 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce 2024-05-20T20:33:00.671 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3830.g4ebd1d53 2024-05-20T20:33:00.673 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-20T20:33:00.680 INFO:teuthology.task.internal:no buildpackages task found 2024-05-20T20:33:00.680 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-20T20:33:00.691 INFO:teuthology.task.internal:Saving configuration 2024-05-20T20:33:00.709 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-20T20:33:00.717 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T20:33:00.741 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 20:28:06.300152', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH94DB/ArPbA9slEZyw7UsFPtn415LxD70Ohu53eSzEK2M4tmVNGzI0YTZCJSEbbJvF+paqSGOOu9/GBtbx8XTQ='} 2024-05-20T20:33:00.764 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 20:28:06.297437', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOceghuAt2xCh7SAA4kXy89Vpu7EiGPRy1B8UX+5OL5Ni/AdkLR95xz6wQrjQ0WNt1TUcaPCdspHNMwKawvhso='} 2024-05-20T20:33:00.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 20:28:06.298894', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDe6kYwkIY8LgqrD4JJLzygwA2Ng83ubO4qAQSdBwB1ytf8aUDzGlU/eqTp06AvCPdyPSaoTV6httuOjRe7jCJk='} 2024-05-20T20:33:00.780 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-20T20:33:00.789 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-20T20:33:00.789 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-20T20:33:00.789 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-20T20:33:00.789 INFO:teuthology.run_tasks:Running task console_log... 2024-05-20T20:33:00.869 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbb442ebac0>, signals=[15]) 2024-05-20T20:33:00.870 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-20T20:33:00.883 INFO:teuthology.task.internal:Opening connections... 2024-05-20T20:33:00.883 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T20:33:00.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:33:00.961 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T20:33:00.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:33:01.038 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-05-20T20:33:01.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:33:01.113 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-20T20:33:01.118 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-05-20T20:33:01.137 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-05-20T20:33:01.137 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-05-20T20:33:01.195 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-05-20T20:33:01.195 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-05-20T20:33:01.196 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-05-20T20:33:01.197 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T20:33:01.197 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-05-20T20:33:01.197 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T20:33:01.197 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T20:33:01.197 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T20:33:01.197 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-05-20T20:33:01.219 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-05-20T20:33:01.237 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-05-20T20:33:01.237 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-05-20T20:33:01.295 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-05-20T20:33:01.295 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T20:33:01.296 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T20:33:01.297 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-05-20T20:33:01.327 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-05-20T20:33:01.345 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-05-20T20:33:01.345 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-05-20T20:33:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T20:33:01.403 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T20:33:01.403 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-05-20T20:33:01.427 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-20T20:33:01.436 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-20T20:33:01.441 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-20T20:33:01.441 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-05-20T20:33:01.443 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2024-05-20T20:33:01.446 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-05-20T20:33:01.462 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-20T20:33:01.469 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-20T20:33:01.469 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-05-20T20:33:01.501 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2024-05-20T20:33:01.503 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-05-20T20:33:01.591 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-20T20:33:01.657 INFO:teuthology.run_tasks:Running task kernel... 2024-05-20T20:33:01.671 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-20T20:33:01.671 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-20T20:33:01.671 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T20:33:01.673 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T20:33:01.674 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T20:33:01.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:33:01.690 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-05-20T20:33:01.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:33:01.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:33:01.692 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-05-20T20:33:01.692 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-05-20T20:33:01.749 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-437.el9.x86_64 2024-05-20T20:33:01.749 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-437.el9.x86_64 2024-05-20T20:33:01.749 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-05-20T20:33:01.750 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-437.el9.x86_64 2024-05-20T20:33:01.750 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-437.el9.x86_64 2024-05-20T20:33:01.751 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-05-20T20:33:01.752 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-437.el9.x86_64 2024-05-20T20:33:01.752 INFO:teuthology.task.kernel:Running kernel on smithi055: 5.14.0-437.el9.x86_64 2024-05-20T20:33:01.752 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum install -y kernel 2024-05-20T20:33:05.394 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:01 2024-05-20T20:33:05.413 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-20T20:33:05.518 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-20T20:33:07.557 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-20T20:33:08.368 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-05-20T20:33:08.629 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-05-20T20:33:13.222 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-20T20:33:13.756 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-20T20:33:14.041 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-20T20:33:14.809 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-20T20:33:15.251 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-20T20:33:15.662 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-05-20T20:33:16.196 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-20T20:33:16.852 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-20T20:33:18.568 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 21 MB 00:02 2024-05-20T20:33:21.569 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-05-20T20:33:22.111 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-05-20T20:33:23.600 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T20:33:23.600 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T20:33:23.698 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-05-20T20:33:23.703 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-05-20T20:33:23.703 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2024-05-20T20:33:23.703 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout:Install 4 Packages 2024-05-20T20:33:23.704 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:33:23.705 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 78 M 2024-05-20T20:33:23.705 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 123 M 2024-05-20T20:33:23.705 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-05-20T20:33:23.755 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-20T20:33:24.009 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T20:33:24.009 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T20:33:24.016 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:24.016 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:24.017 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:24.100 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-05-20T20:33:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-05-20T20:33:24.106 INFO:teuthology.orchestra.run.smithi026.stdout:Install 4 Packages 2024-05-20T20:33:24.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:33:24.106 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 78 M 2024-05-20T20:33:24.106 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 123 M 2024-05-20T20:33:24.106 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-05-20T20:33:24.175 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.177 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.177 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.276 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:24.278 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:24.278 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:24.438 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.438 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.441 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.524 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.565 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.576 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.754 INFO:teuthology.orchestra.run.smithi055.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 745 kB/s | 576 kB 00:00 2024-05-20T20:33:24.790 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.836 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.843 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:24.844 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:25.004 INFO:teuthology.orchestra.run.smithi055.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-05-20T20:33:25.007 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:25.037 INFO:teuthology.orchestra.run.smithi026.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 725 kB/s | 576 kB 00:00 2024-05-20T20:33:25.128 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:25.153 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:25.327 INFO:teuthology.orchestra.run.smithi026.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-05-20T20:33:25.446 INFO:teuthology.orchestra.run.smithi055.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 23 MB/s | 34 MB 00:01 2024-05-20T20:33:25.636 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T20:33:25.636 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T20:33:25.729 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:33:25.734 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-20T20:33:25.735 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-20T20:33:25.735 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-05-20T20:33:25.735 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:33:25.735 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 78 M 2024-05-20T20:33:25.735 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 123 M 2024-05-20T20:33:25.735 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-20T20:33:25.763 INFO:teuthology.orchestra.run.smithi055.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 28 MB/s | 28 MB 00:01 2024-05-20T20:33:25.769 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:33:25.769 INFO:teuthology.orchestra.run.smithi055.stdout:Total 38 MB/s | 78 MB 00:02 2024-05-20T20:33:25.837 INFO:teuthology.orchestra.run.smithi026.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 21 MB/s | 34 MB 00:01 2024-05-20T20:33:25.926 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:25.928 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:25.928 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:26.146 INFO:teuthology.orchestra.run.smithi026.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 25 MB/s | 28 MB 00:01 2024-05-20T20:33:26.153 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:33:26.153 INFO:teuthology.orchestra.run.smithi026.stdout:Total 38 MB/s | 78 MB 00:02 2024-05-20T20:33:26.271 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:26.298 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:26.306 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:26.345 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:26.345 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:26.385 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:33:26.483 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-05-20T20:33:26.498 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 789 kB/s | 576 kB 00:00 2024-05-20T20:33:26.581 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:26.748 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-05-20T20:33:26.751 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:33:26.867 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-05-20T20:33:26.898 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-05-20T20:33:26.898 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-05-20T20:33:27.198 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-05-20T20:33:27.303 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-05-20T20:33:27.303 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-05-20T20:33:27.465 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 29 MB/s | 28 MB 00:00 2024-05-20T20:33:27.470 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:33:27.470 INFO:teuthology.orchestra.run.smithi144.stdout:Total 45 MB/s | 78 MB 00:01 2024-05-20T20:33:27.839 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-05-20T20:33:27.839 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-05-20T20:33:28.150 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-20T20:33:28.246 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-05-20T20:33:28.247 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-05-20T20:33:28.494 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-20T20:33:28.494 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-20T20:33:29.422 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-20T20:33:29.423 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-20T20:33:30.386 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-05-20T20:33:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-05-20T20:33:31.034 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T20:33:31.113 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:33:31.431 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T20:33:31.512 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:33:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-20T20:33:32.458 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T20:33:32.549 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:33:33.887 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:33:34.026 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:33:34.293 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:33:34.431 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:33:35.193 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:33:35.330 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:33:39.104 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:33:39.146 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:33:39.505 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:33:39.551 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:33:39.998 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:33:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:33:44.238 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:33:44.649 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:33:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T20:34:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:34:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:34:15.319 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.319 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:34:15.319 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-20T20:34:15.319 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:15.320 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T20:34:15.320 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T20:34:15.320 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T20:34:15.320 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:34:15.320 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-20T20:34:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.534 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.574 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.695 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T20:34:15.775 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:15.775 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T20:34:15.775 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T20:34:15.775 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T20:34:15.775 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T20:34:15.776 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-20T20:34:15.776 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T20:34:15.776 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T20:34:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T20:34:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:34:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:34:15.825 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:15.826 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T20:34:15.826 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-05-20T20:34:15.859 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T20:34:15.860 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-05-20T20:34:15.861 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:34:15.862 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T20:34:15.863 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T20:34:15.863 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-447.el9.x86_64 2024-05-20T20:34:15.863 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T20:34:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T20:34:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T20:34:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T20:34:16.473 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:34:16.629 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-05-20T20:34:16.819 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T20:34:16.819 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:34:16.820 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-05-20T20:34:16.820 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.820 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.820 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.820 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T20:34:16.821 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:34:16.821 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-05-20T20:34:17.011 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T20:34:17.093 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:17.093 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.093 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.093 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T20:34:17.093 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T20:34:17.093 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-20T20:34:17.093 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T20:34:17.093 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T20:34:17.145 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:17.146 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2024-05-20T20:34:17.166 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T20:34:17.176 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2024-05-20T20:34:17.176 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.06 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 68.el9 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Group : Unspecified 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 8264254 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : builder@centos.org 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : CentOS 2024-05-20T20:34:17.177 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:34:17.178 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T20:34:17.179 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T20:34:17.179 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.179 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T20:34:17.245 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:17.245 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.246 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.246 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T20:34:17.246 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T20:34:17.246 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-20T20:34:17.247 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T20:34:17.247 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T20:34:17.298 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T20:34:17.299 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-05-20T20:34:17.331 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-05-20T20:34:17.332 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-05-20T20:34:17.332 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-05-20T20:34:17.332 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 68.el9 2024-05-20T20:34:17.332 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-05-20T20:34:17.332 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T20:34:17.332 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8264254 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T20:34:17.333 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-05-20T20:34:17.334 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-05-20T20:34:17.334 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T20:34:17.334 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-05-20T20:34:17.334 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-05-20T20:34:17.334 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:34:17.334 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T20:34:17.335 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T20:34:17.335 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T20:34:17.335 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-05-20T20:34:17.335 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:34:17.335 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T20:34:17.337 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T20:34:17.337 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 5.14.0-447.el9.x86_64 2024-05-20T20:34:17.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T20:34:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T20:34:17.798 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-05-20T20:34:17.802 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-05-20T20:34:17.814 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2024-05-20T20:34:17.818 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.tvPfyFg8LW 2024-05-20T20:34:17.818 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tvPfyFg8LW 2024-05-20T20:34:17.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.tvPfyFg8LW 2024-05-20T20:34:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-05-20T20:34:18.024 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.tvPfyFg8LW is 6KB 2024-05-20T20:34:18.075 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.tvPfyFg8LW 2024-05-20T20:34:18.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T20:34:18.152 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T20:34:18.152 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T20:34:18.152 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T20:34:18.153 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T20:34:18.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T20:34:18.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-05-20T20:34:19.247 INFO:teuthology.orchestra.run.smithi055.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T20:34:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:done 2024-05-20T20:34:19.287 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2024-05-20T20:34:19.302 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.sNTI0aTHko 2024-05-20T20:34:19.302 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.sNTI0aTHko 2024-05-20T20:34:19.380 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.sNTI0aTHko 2024-05-20T20:34:19.524 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.sNTI0aTHko is 6KB 2024-05-20T20:34:19.552 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T20:34:19.573 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.sNTI0aTHko 2024-05-20T20:34:19.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T20:34:19.597 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-05-20T20:34:19.599 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-05-20T20:34:19.621 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.00MBHt0Ldt 2024-05-20T20:34:19.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.00MBHt0Ldt 2024-05-20T20:34:19.654 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T20:34:19.654 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T20:34:19.654 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T20:34:19.654 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T20:34:19.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T20:34:19.708 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.00MBHt0Ldt 2024-05-20T20:34:19.809 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2024-05-20T20:34:19.810 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.00MBHt0Ldt is 6KB 2024-05-20T20:34:19.859 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.00MBHt0Ldt 2024-05-20T20:34:19.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T20:34:19.942 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T20:34:19.942 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T20:34:19.942 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T20:34:19.942 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T20:34:19.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T20:34:20.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-05-20T20:34:48.322 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-20T20:34:48.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-20T20:34:48.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:34:49.814 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-20T20:34:49.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T20:34:49.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:34:50.090 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-20T20:34:50.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-20T20:34:50.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:35:06.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-05-20T20:35:08.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-05-20T20:35:08.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-05-20T20:35:15.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-20T20:35:15.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:35:17.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T20:35:17.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:35:17.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-20T20:35:17.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:35:18.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-05-20T20:35:30.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-20T20:35:30.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:35:35.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-05-20T20:35:47.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-20T20:35:47.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:35:48.774 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-05-20T20:35:49.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-20T20:35:49.157 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-447.el9.x86_64"... 2024-05-20T20:35:49.157 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-05-20T20:35:49.173 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-447.el9.x86_64 2024-05-20T20:35:49.173 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T20:35:49.173 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T20:35:49.173 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T20:35:50.174 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-20T20:35:50.174 DEBUG:teuthology.orchestra.run.smithi026:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-20T20:35:50.267 INFO:teuthology.orchestra.run.smithi026.stdout:ttyS1 2024-05-20T20:35:50.309 DEBUG:teuthology.parallel:result is None 2024-05-20T20:36:02.458 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-05-20T20:36:02.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-20T20:36:02.900 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-447.el9.x86_64"... 2024-05-20T20:36:02.900 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-05-20T20:36:02.915 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-447.el9.x86_64 2024-05-20T20:36:02.915 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T20:36:02.915 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T20:36:02.915 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T20:36:03.917 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-20T20:36:03.917 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-20T20:36:04.005 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-05-20T20:36:04.047 DEBUG:teuthology.parallel:result is None 2024-05-20T20:36:17.196 DEBUG:teuthology.orchestra.remote:timed out 2024-05-20T20:36:29.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T20:36:29.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:36:29.471 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-05-20T20:36:29.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T20:36:29.841 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-447.el9.x86_64"... 2024-05-20T20:36:29.841 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-05-20T20:36:29.859 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-447.el9.x86_64 2024-05-20T20:36:29.859 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T20:36:29.859 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T20:36:29.859 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T20:36:30.860 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-20T20:36:30.860 DEBUG:teuthology.orchestra.run.smithi055:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-20T20:36:30.927 INFO:teuthology.orchestra.run.smithi055.stdout:ttyS1 2024-05-20T20:36:30.967 DEBUG:teuthology.parallel:result is None 2024-05-20T20:36:30.968 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-20T20:36:30.975 INFO:teuthology.task.internal:Creating test directory... 2024-05-20T20:36:30.976 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T20:36:30.979 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T20:36:30.981 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T20:36:31.000 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-20T20:36:31.006 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-20T20:36:31.013 INFO:teuthology.task.internal:Creating archive directory... 2024-05-20T20:36:31.013 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T20:36:31.038 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T20:36:31.040 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T20:36:31.079 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-20T20:36:31.085 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-20T20:36:31.086 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T20:36:31.109 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T20:36:31.115 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T20:36:31.139 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T20:36:31.144 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T20:36:31.151 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T20:36:31.152 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T20:36:31.156 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T20:36:31.163 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T20:36:31.165 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-20T20:36:31.172 INFO:teuthology.task.internal:Configuring sudo... 2024-05-20T20:36:31.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T20:36:31.196 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T20:36:31.200 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T20:36:31.232 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-20T20:36:31.241 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-20T20:36:31.241 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T20:36:31.264 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T20:36:31.269 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T20:36:31.289 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T20:36:31.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T20:36:31.415 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T20:36:31.500 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T20:36:31.541 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:36:31.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T20:36:31.607 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T20:36:31.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T20:36:31.703 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T20:36:31.782 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T20:36:31.831 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:36:31.832 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T20:36:31.899 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T20:36:31.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T20:36:31.998 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T20:36:32.077 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T20:36:32.127 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:36:32.127 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T20:36:32.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-05-20T20:36:32.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2024-05-20T20:36:32.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-05-20T20:36:32.242 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T20:36:32.254 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T20:36:32.279 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T20:36:32.738 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-20T20:36:32.746 INFO:teuthology.task.internal:Starting timer... 2024-05-20T20:36:32.746 INFO:teuthology.run_tasks:Running task pcp... 2024-05-20T20:36:32.759 INFO:teuthology.run_tasks:Running task selinux... 2024-05-20T20:36:32.769 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-20T20:36:32.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-05-20T20:36:32.838 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-05-20T20:36:32.841 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2024-05-20T20:36:32.898 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: 2024-05-20T20:36:32.900 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-05-20T20:36:32.973 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-05-20T20:36:32.975 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-20T20:36:32.975 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-05-20T20:36:33.017 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-05-20T20:36:33.017 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2024-05-20T20:36:33.046 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2024-05-20T20:36:33.046 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-05-20T20:36:33.096 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-05-20T20:36:33.096 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-05-20T20:36:33.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T20:36:33.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:36:33.126 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T20:36:33.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:36:33.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T20:36:33.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:36:33.184 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-20T20:36:33.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-05-20T20:36:33.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2024-05-20T20:36:33.241 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-05-20T20:36:33.270 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-20T20:36:33.279 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-20T20:36:33.291 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-20T20:36:33.292 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi055.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-20T20:40:55.995 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-05-20T20:40:55.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-20T20:40:55.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:40:56.077 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-05-20T20:40:56.164 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-20T20:40:56.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T20:40:56.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:40:56.242 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-05-20T20:40:56.323 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T20:40:56.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-20T20:40:56.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T20:40:56.399 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-05-20T20:40:56.478 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-20T20:40:56.479 INFO:teuthology.run_tasks:Running task clock... 2024-05-20T20:40:56.489 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-20T20:40:56.489 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T20:40:56.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T20:40:56.492 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T20:40:56.493 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T20:40:56.495 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T20:40:56.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T20:40:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T20:40:56.528 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T20:40:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T20:40:56.544 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T20:40:56.549 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T20:40:56.563 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T20:40:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-05-20T20:40:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2024-05-20T20:40:56.583 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-05-20T20:40:56.584 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2024-05-20T20:40:56.588 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-05-20T20:40:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T20:40:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T20:40:56.599 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-05-20T20:40:56.613 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T20:40:56.613 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T20:40:56.614 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T20:40:56.627 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T20:40:56.672 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-05-20T20:40:56.675 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T20:40:56.675 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-05-20T20:40:56.675 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.676 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.676 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.676 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-05-20T20:40:56.680 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-05-20T20:40:56.682 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T20:40:56.682 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-05-20T20:40:56.682 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.682 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.682 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.683 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.684 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T20:40:56.684 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-05-20T20:40:56.684 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.684 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.684 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.684 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T20:40:56.685 INFO:teuthology.run_tasks:Running task install... 2024-05-20T20:40:56.693 DEBUG:teuthology.task.install:project ceph 2024-05-20T20:40:56.693 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4ebd1d5361d457738957dc0455bba90102296634'}} 2024-05-20T20:40:56.693 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ebd1d5361d457738957dc0455bba90102296634'} 2024-05-20T20:40:56.693 INFO:teuthology.task.install:Using flavor: default 2024-05-20T20:40:56.699 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-20T20:40:56.699 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-20T20:40:56.700 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ebd1d5361d457738957dc0455bba90102296634', 'tag': None, 'wait_for_package': False} 2024-05-20T20:40:56.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:40:56.702 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ebd1d5361d457738957dc0455bba90102296634', 'tag': None, 'wait_for_package': False} 2024-05-20T20:40:56.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:40:56.703 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ebd1d5361d457738957dc0455bba90102296634', 'tag': None, 'wait_for_package': False} 2024-05-20T20:40:56.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:40:56.842 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/ 2024-05-20T20:40:56.842 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g4ebd1d53 2024-05-20T20:40:56.924 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/ 2024-05-20T20:40:56.925 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g4ebd1d53 2024-05-20T20:40:56.940 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/ 2024-05-20T20:40:56.940 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g4ebd1d53 2024-05-20T20:40:57.042 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T20:40:57.042 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:40:57.043 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T20:40:57.051 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T20:40:57.051 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:40:57.051 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T20:40:57.062 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T20:40:57.062 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:40:57.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T20:40:57.070 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T20:40:57.070 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ebd1d5361d457738957dc0455bba90102296634/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T20:40:57.077 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T20:40:57.077 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ebd1d5361d457738957dc0455bba90102296634/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T20:40:57.090 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T20:40:57.090 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ebd1d5361d457738957dc0455bba90102296634/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T20:40:57.137 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T20:40:57.143 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T20:40:57.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T20:40:57.217 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T20:40:57.218 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T20:40:57.235 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T20:40:57.282 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-05-20T20:40:57.283 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2024-05-20T20:40:57.284 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-05-20T20:40:57.285 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2024-05-20T20:40:57.301 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-05-20T20:40:57.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-05-20T20:40:57.525 INFO:teuthology.orchestra.run.smithi144.stdout:57 files removed 2024-05-20T20:40:57.530 INFO:teuthology.orchestra.run.smithi055.stdout:57 files removed 2024-05-20T20:40:57.544 INFO:teuthology.orchestra.run.smithi026.stdout:57 files removed 2024-05-20T20:40:57.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T20:40:57.557 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T20:40:57.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T20:40:58.144 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 265 kB/s | 84 kB 00:00 2024-05-20T20:40:58.258 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-05-20T20:40:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 175 kB/s | 84 kB 00:00 2024-05-20T20:40:58.510 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-20T20:40:58.651 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-20T20:40:58.810 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-05-20T20:40:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-05-20T20:40:59.142 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-20T20:40:59.252 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-20T20:41:00.715 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.1 MB 00:01 2024-05-20T20:41:00.973 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-05-20T20:41:01.014 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.1 MB 00:01 2024-05-20T20:41:03.632 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-05-20T20:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-05-20T20:41:04.522 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-05-20T20:41:08.819 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-20T20:41:09.892 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-20T20:41:09.969 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-20T20:41:10.390 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-05-20T20:41:11.380 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-05-20T20:41:12.289 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-20T20:41:12.365 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-05-20T20:41:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-05-20T20:41:17.807 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-20T20:41:18.616 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-20T20:41:20.042 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T20:41:20.043 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T20:41:20.054 INFO:teuthology.orchestra.run.smithi055.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T20:41:20.055 INFO:teuthology.orchestra.run.smithi055.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T20:41:20.056 INFO:teuthology.orchestra.run.smithi055.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T20:41:20.056 INFO:teuthology.orchestra.run.smithi055.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T20:41:20.058 INFO:teuthology.orchestra.run.smithi055.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T20:41:20.060 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T20:41:20.061 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T20:41:20.127 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 6.6 k 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 5.4 M 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 890 k 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 145 k 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 1.5 M 2024-05-20T20:41:20.147 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 143 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 3.5 M 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 7.4 M 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 49 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 11 M 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 49 M 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 273 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 764 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 225 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-shell x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 27 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 18 k 2024-05-20T20:41:20.148 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 32 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 747 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 127 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 161 k 2024-05-20T20:41:20.149 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 320 k 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 300 k 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 100 k 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 85 k 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.1 M 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 171 k 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T20:41:20.150 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.4 M 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.1 M 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T20:41:20.151 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 21 M 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 34 k 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 2.4 M 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 274 k 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 4.7 M 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 17 M 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 16 k 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 25 k 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T20:41:20.152 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 163 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T20:41:20.153 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 501 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 5.4 M 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T20:41:20.154 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 45 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 136 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T20:41:20.155 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T20:41:20.156 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T20:41:20.157 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T20:41:20.158 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T20:41:20.159 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T20:41:20.160 INFO:teuthology.orchestra.run.smithi055.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout:Installing weak dependencies: 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout:Install 148 Packages 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 11 Packages 2024-05-20T20:41:20.161 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:41:20.162 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 227 M 2024-05-20T20:41:20.162 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-05-20T20:41:20.884 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T20:41:20.885 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T20:41:20.896 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T20:41:20.897 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T20:41:20.897 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T20:41:20.898 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T20:41:20.900 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T20:41:20.902 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T20:41:20.903 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T20:41:20.969 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-05-20T20:41:20.991 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-05-20T20:41:20.991 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-05-20T20:41:20.991 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-05-20T20:41:20.991 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-05-20T20:41:20.991 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T20:41:20.991 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 6.6 k 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 5.4 M 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 890 k 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 145 k 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 1.5 M 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 143 k 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 3.5 M 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 7.4 M 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 49 k 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 11 M 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 49 M 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 273 k 2024-05-20T20:41:20.992 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 764 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 225 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 27 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 18 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 32 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 747 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 127 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T20:41:20.993 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 161 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 320 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 300 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 100 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 85 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.1 M 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 171 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T20:41:20.994 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.4 M 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.1 M 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-05-20T20:41:20.995 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 21 M 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 34 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 2.4 M 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 274 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 4.7 M 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 17 M 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 16 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 25 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T20:41:20.996 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 163 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 501 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 5.4 M 2024-05-20T20:41:20.997 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T20:41:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 45 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 136 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T20:41:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T20:41:21.000 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T20:41:21.001 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T20:41:21.002 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T20:41:21.003 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 11 Packages 2024-05-20T20:41:21.004 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:41:21.005 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 227 M 2024-05-20T20:41:21.005 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-05-20T20:41:21.025 INFO:teuthology.orchestra.run.smithi055.stdout:(1/159): ceph-19.0.0-3830.g4ebd1d53.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-05-20T20:41:22.200 INFO:teuthology.orchestra.run.smithi026.stdout:(1/159): ceph-19.0.0-3830.g4ebd1d53.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-05-20T20:41:22.317 INFO:teuthology.orchestra.run.smithi055.stdout:(2/159): ceph-fuse-19.0.0-3830.g4ebd1d53.el9.x8 689 kB/s | 890 kB 00:01 2024-05-20T20:41:22.509 INFO:teuthology.orchestra.run.smithi055.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 757 kB/s | 145 kB 00:00 2024-05-20T20:41:23.000 INFO:teuthology.orchestra.run.smithi026.stdout:(2/159): ceph-fuse-19.0.0-3830.g4ebd1d53.el9.x8 1.1 MB/s | 890 kB 00:00 2024-05-20T20:41:23.167 INFO:teuthology.orchestra.run.smithi055.stdout:(4/159): ceph-base-19.0.0-3830.g4ebd1d53.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-20T20:41:23.168 INFO:teuthology.orchestra.run.smithi026.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 868 kB/s | 145 kB 00:00 2024-05-20T20:41:23.517 INFO:teuthology.orchestra.run.smithi026.stdout:(4/159): ceph-base-19.0.0-3830.g4ebd1d53.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-05-20T20:41:23.643 INFO:teuthology.orchestra.run.smithi055.stdout:(5/159): ceph-common-19.0.0-3830.g4ebd1d53.el9. 7.3 MB/s | 21 MB 00:02 2024-05-20T20:41:24.018 INFO:teuthology.orchestra.run.smithi055.stdout:(6/159): ceph-mgr-19.0.0-3830.g4ebd1d53.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-05-20T20:41:24.193 INFO:teuthology.orchestra.run.smithi026.stdout:(5/159): ceph-mgr-19.0.0-3830.g4ebd1d53.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-20T20:41:24.518 INFO:teuthology.orchestra.run.smithi026.stdout:(6/159): ceph-mds-19.0.0-3830.g4ebd1d53.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-05-20T20:41:24.560 INFO:teuthology.orchestra.run.smithi055.stdout:(7/159): ceph-mon-19.0.0-3830.g4ebd1d53.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-05-20T20:41:24.627 INFO:teuthology.orchestra.run.smithi055.stdout:(8/159): ceph-mds-19.0.0-3830.g4ebd1d53.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-05-20T20:41:24.727 INFO:teuthology.orchestra.run.smithi055.stdout:(9/159): ceph-selinux-19.0.0-3830.g4ebd1d53.el9 252 kB/s | 25 kB 00:00 2024-05-20T20:41:24.935 INFO:teuthology.orchestra.run.smithi026.stdout:(7/159): ceph-mon-19.0.0-3830.g4ebd1d53.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-05-20T20:41:26.335 INFO:teuthology.orchestra.run.smithi055.stdout:(10/159): ceph-radosgw-19.0.0-3830.g4ebd1d53.el 6.0 MB/s | 11 MB 00:01 2024-05-20T20:41:26.402 INFO:teuthology.orchestra.run.smithi055.stdout:(11/159): cephfs-mirror-19.0.0-3830.g4ebd1d53.e 3.3 MB/s | 225 kB 00:00 2024-05-20T20:41:26.435 INFO:teuthology.orchestra.run.smithi026.stdout:(8/159): ceph-radosgw-19.0.0-3830.g4ebd1d53.el9 7.1 MB/s | 11 MB 00:01 2024-05-20T20:41:26.461 INFO:teuthology.orchestra.run.smithi055.stdout:(12/159): cephfs-shell-19.0.0-3830.g4ebd1d53.el 466 kB/s | 27 kB 00:00 2024-05-20T20:41:26.510 INFO:teuthology.orchestra.run.smithi026.stdout:(9/159): ceph-selinux-19.0.0-3830.g4ebd1d53.el9 337 kB/s | 25 kB 00:00 2024-05-20T20:41:26.520 INFO:teuthology.orchestra.run.smithi055.stdout:(13/159): libcephfs-devel-19.0.0-3830.g4ebd1d53 543 kB/s | 32 kB 00:00 2024-05-20T20:41:26.695 INFO:teuthology.orchestra.run.smithi055.stdout:(14/159): libcephfs2-19.0.0-3830.g4ebd1d53.el9. 4.2 MB/s | 747 kB 00:00 2024-05-20T20:41:26.795 INFO:teuthology.orchestra.run.smithi055.stdout:(15/159): libcephsqlite-19.0.0-3830.g4ebd1d53.e 1.6 MB/s | 163 kB 00:00 2024-05-20T20:41:26.879 INFO:teuthology.orchestra.run.smithi055.stdout:(16/159): librados-devel-19.0.0-3830.g4ebd1d53. 1.5 MB/s | 127 kB 00:00 2024-05-20T20:41:27.029 INFO:teuthology.orchestra.run.smithi055.stdout:(17/159): libradosstriper1-19.0.0-3830.g4ebd1d5 3.3 MB/s | 501 kB 00:00 2024-05-20T20:41:27.946 INFO:teuthology.orchestra.run.smithi055.stdout:(18/159): librgw2-19.0.0-3830.g4ebd1d53.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-05-20T20:41:28.021 INFO:teuthology.orchestra.run.smithi055.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g4e 605 kB/s | 45 kB 00:00 2024-05-20T20:41:28.097 INFO:teuthology.orchestra.run.smithi055.stdout:(20/159): python3-ceph-common-19.0.0-3830.g4ebd 1.8 MB/s | 136 kB 00:00 2024-05-20T20:41:28.172 INFO:teuthology.orchestra.run.smithi055.stdout:(21/159): python3-cephfs-19.0.0-3830.g4ebd1d53. 2.1 MB/s | 161 kB 00:00 2024-05-20T20:41:28.256 INFO:teuthology.orchestra.run.smithi055.stdout:(22/159): python3-rados-19.0.0-3830.g4ebd1d53.e 3.8 MB/s | 320 kB 00:00 2024-05-20T20:41:28.331 INFO:teuthology.orchestra.run.smithi055.stdout:(23/159): python3-rbd-19.0.0-3830.g4ebd1d53.el9 3.9 MB/s | 300 kB 00:00 2024-05-20T20:41:28.398 INFO:teuthology.orchestra.run.smithi055.stdout:(24/159): python3-rgw-19.0.0-3830.g4ebd1d53.el9 1.5 MB/s | 100 kB 00:00 2024-05-20T20:41:28.465 INFO:teuthology.orchestra.run.smithi055.stdout:(25/159): rbd-fuse-19.0.0-3830.g4ebd1d53.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-20T20:41:29.015 INFO:teuthology.orchestra.run.smithi055.stdout:(26/159): rbd-mirror-19.0.0-3830.g4ebd1d53.el9. 5.6 MB/s | 3.1 MB 00:00 2024-05-20T20:41:29.099 INFO:teuthology.orchestra.run.smithi055.stdout:(27/159): rbd-nbd-19.0.0-3830.g4ebd1d53.el9.x86 2.0 MB/s | 171 kB 00:00 2024-05-20T20:41:29.157 INFO:teuthology.orchestra.run.smithi055.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3830.g 589 kB/s | 34 kB 00:00 2024-05-20T20:41:29.224 INFO:teuthology.orchestra.run.smithi055.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3830.g4ebd1d5 2.1 MB/s | 143 kB 00:00 2024-05-20T20:41:29.850 INFO:teuthology.orchestra.run.smithi055.stdout:(30/159): ceph-mgr-dashboard-19.0.0-3830.g4ebd1 5.7 MB/s | 3.5 MB 00:00 2024-05-20T20:41:30.575 INFO:teuthology.orchestra.run.smithi055.stdout:(31/159): ceph-osd-19.0.0-3830.g4ebd1d53.el9.x8 2.6 MB/s | 17 MB 00:06 2024-05-20T20:41:30.609 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:20 2024-05-20T20:41:30.767 INFO:teuthology.orchestra.run.smithi055.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g4e 1.4 MB/s | 274 kB 00:00 2024-05-20T20:41:30.859 INFO:teuthology.orchestra.run.smithi055.stdout:(33/159): ceph-mgr-rook-19.0.0-3830.g4ebd1d53.e 539 kB/s | 49 kB 00:00 2024-05-20T20:41:30.942 INFO:teuthology.orchestra.run.smithi055.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3830.g4 198 kB/s | 16 kB 00:00 2024-05-20T20:41:31.076 INFO:teuthology.orchestra.run.smithi055.stdout:(35/159): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-05-20T20:41:31.118 INFO:teuthology.orchestra.run.smithi055.stdout:(36/159): ceph-volume-19.0.0-3830.g4ebd1d53.el9 1.5 MB/s | 273 kB 00:00 2024-05-20T20:41:31.202 INFO:teuthology.orchestra.run.smithi055.stdout:(37/159): cephfs-top-19.0.0-3830.g4ebd1d53.el9. 219 kB/s | 18 kB 00:00 2024-05-20T20:41:31.227 INFO:teuthology.orchestra.run.smithi055.stdout:(38/159): cephadm-19.0.0-3830.g4ebd1d53.el9.noa 5.0 MB/s | 764 kB 00:00 2024-05-20T20:41:31.319 INFO:teuthology.orchestra.run.smithi055.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 687 kB/s | 80 kB 00:00 2024-05-20T20:41:31.386 INFO:teuthology.orchestra.run.smithi055.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-20T20:41:31.445 INFO:teuthology.orchestra.run.smithi055.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-05-20T20:41:31.470 INFO:teuthology.orchestra.run.smithi055.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-20T20:41:31.520 INFO:teuthology.orchestra.run.smithi055.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-05-20T20:41:31.545 INFO:teuthology.orchestra.run.smithi055.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-20T20:41:31.571 INFO:teuthology.orchestra.run.smithi055.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-20T20:41:31.596 INFO:teuthology.orchestra.run.smithi055.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-20T20:41:31.622 INFO:teuthology.orchestra.run.smithi055.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-20T20:41:32.027 INFO:teuthology.orchestra.run.smithi026.stdout:(10/159): ceph-test-19.0.0-3830.g4ebd1d53.el9.x 8.9 MB/s | 49 MB 00:05 2024-05-20T20:41:32.244 INFO:teuthology.orchestra.run.smithi026.stdout:(11/159): ceph-osd-19.0.0-3830.g4ebd1d53.el9.x8 2.2 MB/s | 17 MB 00:07 2024-05-20T20:41:32.269 INFO:teuthology.orchestra.run.smithi026.stdout:(12/159): cephfs-mirror-19.0.0-3830.g4ebd1d53.e 930 kB/s | 225 kB 00:00 2024-05-20T20:41:32.352 INFO:teuthology.orchestra.run.smithi026.stdout:(13/159): cephfs-shell-19.0.0-3830.g4ebd1d53.el 250 kB/s | 27 kB 00:00 2024-05-20T20:41:32.378 INFO:teuthology.orchestra.run.smithi026.stdout:(14/159): libcephfs-devel-19.0.0-3830.g4ebd1d53 292 kB/s | 32 kB 00:00 2024-05-20T20:41:32.445 INFO:teuthology.orchestra.run.smithi026.stdout:(15/159): libcephsqlite-19.0.0-3830.g4ebd1d53.e 2.4 MB/s | 163 kB 00:00 2024-05-20T20:41:32.447 INFO:teuthology.orchestra.run.smithi055.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 810 kB/s | 689 kB 00:00 2024-05-20T20:41:32.489 INFO:teuthology.orchestra.run.smithi055.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-20T20:41:32.512 INFO:teuthology.orchestra.run.smithi026.stdout:(16/159): librados-devel-19.0.0-3830.g4ebd1d53. 1.8 MB/s | 127 kB 00:00 2024-05-20T20:41:32.556 INFO:teuthology.orchestra.run.smithi055.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 986 kB/s | 107 kB 00:00 2024-05-20T20:41:32.578 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:32.587 INFO:teuthology.orchestra.run.smithi026.stdout:(17/159): libcephfs2-19.0.0-3830.g4ebd1d53.el9. 3.1 MB/s | 747 kB 00:00 2024-05-20T20:41:32.662 INFO:teuthology.orchestra.run.smithi026.stdout:(18/159): libradosstriper1-19.0.0-3830.g4ebd1d5 3.3 MB/s | 501 kB 00:00 2024-05-20T20:41:32.681 INFO:teuthology.orchestra.run.smithi055.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-05-20T20:41:32.738 INFO:teuthology.orchestra.run.smithi026.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g4e 603 kB/s | 45 kB 00:00 2024-05-20T20:41:32.806 INFO:teuthology.orchestra.run.smithi026.stdout:(20/159): python3-ceph-common-19.0.0-3830.g4ebd 2.0 MB/s | 136 kB 00:00 2024-05-20T20:41:32.823 INFO:teuthology.orchestra.run.smithi055.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 292 kB/s | 33 kB 00:00 2024-05-20T20:41:32.880 INFO:teuthology.orchestra.run.smithi026.stdout:(21/159): python3-cephfs-19.0.0-3830.g4ebd1d53. 2.1 MB/s | 161 kB 00:00 2024-05-20T20:41:32.964 INFO:teuthology.orchestra.run.smithi026.stdout:(22/159): python3-rados-19.0.0-3830.g4ebd1d53.e 3.8 MB/s | 320 kB 00:00 2024-05-20T20:41:32.997 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:33.015 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:33.081 INFO:teuthology.orchestra.run.smithi026.stdout:(23/159): python3-rbd-19.0.0-3830.g4ebd1d53.el9 2.5 MB/s | 300 kB 00:00 2024-05-20T20:41:33.098 INFO:teuthology.orchestra.run.smithi055.stdout:(53/159): elfutils-libelf-devel-0.191-4.el9.x86 36 kB/s | 22 kB 00:00 2024-05-20T20:41:33.148 INFO:teuthology.orchestra.run.smithi026.stdout:(24/159): python3-rgw-19.0.0-3830.g4ebd1d53.el9 1.5 MB/s | 100 kB 00:00 2024-05-20T20:41:33.215 INFO:teuthology.orchestra.run.smithi055.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-05-20T20:41:33.231 INFO:teuthology.orchestra.run.smithi026.stdout:(25/159): rbd-fuse-19.0.0-3830.g4ebd1d53.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-20T20:41:33.266 INFO:teuthology.orchestra.run.smithi055.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.8 MB/s | 3.0 MB 00:00 2024-05-20T20:41:33.333 INFO:teuthology.orchestra.run.smithi055.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-20T20:41:33.400 INFO:teuthology.orchestra.run.smithi055.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-20T20:41:33.450 INFO:teuthology.orchestra.run.smithi055.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-05-20T20:41:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-05-20T20:41:33.650 INFO:teuthology.orchestra.run.smithi055.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-05-20T20:41:33.706 INFO:teuthology.orchestra.run.smithi026.stdout:(26/159): ceph-common-19.0.0-3830.g4ebd1d53.el9 1.8 MB/s | 21 MB 00:11 2024-05-20T20:41:33.751 INFO:teuthology.orchestra.run.smithi055.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-05-20T20:41:33.765 INFO:teuthology.orchestra.run.smithi026.stdout:(27/159): rbd-mirror-19.0.0-3830.g4ebd1d53.el9. 5.8 MB/s | 3.1 MB 00:00 2024-05-20T20:41:33.768 INFO:teuthology.orchestra.run.smithi055.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-05-20T20:41:33.815 INFO:teuthology.orchestra.run.smithi026.stdout:(28/159): rbd-nbd-19.0.0-3830.g4ebd1d53.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-20T20:41:33.841 INFO:teuthology.orchestra.run.smithi026.stdout:(29/159): ceph-grafana-dashboards-19.0.0-3830.g 455 kB/s | 34 kB 00:00 2024-05-20T20:41:33.860 INFO:teuthology.orchestra.run.smithi055.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-20T20:41:33.866 INFO:teuthology.orchestra.run.smithi026.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3830.g4ebd1d5 2.8 MB/s | 143 kB 00:00 2024-05-20T20:41:33.885 INFO:teuthology.orchestra.run.smithi055.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-20T20:41:33.960 INFO:teuthology.orchestra.run.smithi055.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-20T20:41:34.158 INFO:teuthology.orchestra.run.smithi026.stdout:(31/159): librgw2-19.0.0-3830.g4ebd1d53.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-05-20T20:41:34.250 INFO:teuthology.orchestra.run.smithi026.stdout:(32/159): ceph-mgr-dashboard-19.0.0-3830.g4ebd1 8.7 MB/s | 3.5 MB 00:00 2024-05-20T20:41:34.300 INFO:teuthology.orchestra.run.smithi026.stdout:(33/159): ceph-mgr-modules-core-19.0.0-3830.g4e 1.9 MB/s | 274 kB 00:00 2024-05-20T20:41:34.325 INFO:teuthology.orchestra.run.smithi026.stdout:(34/159): ceph-mgr-rook-19.0.0-3830.g4ebd1d53.e 656 kB/s | 49 kB 00:00 2024-05-20T20:41:34.359 INFO:teuthology.orchestra.run.smithi026.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3830.g4 281 kB/s | 16 kB 00:00 2024-05-20T20:41:34.393 INFO:teuthology.orchestra.run.smithi026.stdout:(36/159): ceph-volume-19.0.0-3830.g4ebd1d53.el9 4.0 MB/s | 273 kB 00:00 2024-05-20T20:41:34.460 INFO:teuthology.orchestra.run.smithi026.stdout:(37/159): cephfs-top-19.0.0-3830.g4ebd1d53.el9. 273 kB/s | 18 kB 00:00 2024-05-20T20:41:34.494 INFO:teuthology.orchestra.run.smithi055.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-05-20T20:41:34.590 INFO:teuthology.orchestra.run.smithi026.stdout:(38/159): cephadm-19.0.0-3830.g4ebd1d53.el9.noa 3.2 MB/s | 764 kB 00:00 2024-05-20T20:41:34.610 INFO:teuthology.orchestra.run.smithi026.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-05-20T20:41:34.627 INFO:teuthology.orchestra.run.smithi055.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-20T20:41:34.685 INFO:teuthology.orchestra.run.smithi026.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-05-20T20:41:34.735 INFO:teuthology.orchestra.run.smithi026.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-20T20:41:34.769 INFO:teuthology.orchestra.run.smithi026.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.4 MB/s | 803 kB 00:00 2024-05-20T20:41:34.811 INFO:teuthology.orchestra.run.smithi026.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-20T20:41:34.853 INFO:teuthology.orchestra.run.smithi026.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-05-20T20:41:34.879 INFO:teuthology.orchestra.run.smithi026.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-20T20:41:34.904 INFO:teuthology.orchestra.run.smithi026.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-20T20:41:34.937 INFO:teuthology.orchestra.run.smithi026.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-20T20:41:34.980 INFO:teuthology.orchestra.run.smithi026.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-05-20T20:41:35.021 INFO:teuthology.orchestra.run.smithi026.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-05-20T20:41:35.038 INFO:teuthology.orchestra.run.smithi026.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-20T20:41:35.039 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:35.080 INFO:teuthology.orchestra.run.smithi026.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-05-20T20:41:35.122 INFO:teuthology.orchestra.run.smithi026.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 786 kB/s | 33 kB 00:00 2024-05-20T20:41:35.214 INFO:teuthology.orchestra.run.smithi026.stdout:(53/159): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-05-20T20:41:35.215 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:35.273 INFO:teuthology.orchestra.run.smithi026.stdout:(54/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-05-20T20:41:35.298 INFO:teuthology.orchestra.run.smithi026.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-05-20T20:41:35.348 INFO:teuthology.orchestra.run.smithi026.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-05-20T20:41:35.365 INFO:teuthology.orchestra.run.smithi026.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.1 MB/s | 160 kB 00:00 2024-05-20T20:41:35.391 INFO:teuthology.orchestra.run.smithi026.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-20T20:41:35.424 INFO:teuthology.orchestra.run.smithi026.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-20T20:41:35.458 INFO:teuthology.orchestra.run.smithi026.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-20T20:41:35.483 INFO:teuthology.orchestra.run.smithi026.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 447 kB/s | 25 kB 00:00 2024-05-20T20:41:35.508 INFO:teuthology.orchestra.run.smithi026.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-20T20:41:35.534 INFO:teuthology.orchestra.run.smithi026.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-20T20:41:35.559 INFO:teuthology.orchestra.run.smithi026.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-20T20:41:35.576 INFO:teuthology.orchestra.run.smithi026.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 843 kB/s | 35 kB 00:00 2024-05-20T20:41:35.589 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:35.653 INFO:teuthology.orchestra.run.smithi055.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 2.7 MB/s | 4.7 MB 00:01 2024-05-20T20:41:35.668 INFO:teuthology.orchestra.run.smithi026.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-05-20T20:41:35.685 INFO:teuthology.orchestra.run.smithi026.stdout:(67/159): elfutils-libelf-devel-0.191-4.el9.x86 33 kB/s | 22 kB 00:00 2024-05-20T20:41:35.752 INFO:teuthology.orchestra.run.smithi026.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-20T20:41:35.785 INFO:teuthology.orchestra.run.smithi026.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-20T20:41:35.802 INFO:teuthology.orchestra.run.smithi026.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-05-20T20:41:35.853 INFO:teuthology.orchestra.run.smithi026.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-05-20T20:41:35.861 INFO:teuthology.orchestra.run.smithi055.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 738 kB/s | 911 kB 00:01 2024-05-20T20:41:35.914 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-20T20:41:35.936 INFO:teuthology.orchestra.run.smithi026.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-05-20T20:41:35.951 INFO:teuthology.orchestra.run.smithi055.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 447 kB/s | 133 kB 00:00 2024-05-20T20:41:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-05-20T20:41:35.996 INFO:teuthology.orchestra.run.smithi026.stdout:(74/159): postgresql-contrib-13.14-1.el9.x86_64 2.9 MB/s | 911 kB 00:00 2024-05-20T20:41:35.996 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T20:41:36.037 INFO:teuthology.orchestra.run.smithi026.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-05-20T20:41:36.121 INFO:teuthology.orchestra.run.smithi026.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-05-20T20:41:36.122 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:36.122 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T20:41:36.163 INFO:teuthology.orchestra.run.smithi026.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 619 kB/s | 26 kB 00:00 2024-05-20T20:41:36.205 INFO:teuthology.orchestra.run.smithi026.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-05-20T20:41:36.206 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:36.206 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:36.247 INFO:teuthology.orchestra.run.smithi026.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-05-20T20:41:36.289 INFO:teuthology.orchestra.run.smithi026.stdout:(80/159): python3-colorama-0.4.6-3.el9.noarch.r 187 kB/s | 61 kB 00:00 2024-05-20T20:41:36.289 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:36.314 INFO:teuthology.orchestra.run.smithi026.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-05-20T20:41:36.356 INFO:teuthology.orchestra.run.smithi026.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-20T20:41:36.381 INFO:teuthology.orchestra.run.smithi026.stdout:(83/159): python3-jinja2-2.11.3-6.el9.noarch.rp 722 kB/s | 248 kB 00:00 2024-05-20T20:41:36.407 INFO:teuthology.orchestra.run.smithi026.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-20T20:41:36.424 INFO:teuthology.orchestra.run.smithi026.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-20T20:41:36.446 INFO:teuthology.orchestra.run.smithi055.stdout:(71/159): ceph-test-19.0.0-3830.g4ebd1d53.el9.x 4.2 MB/s | 49 MB 00:11 2024-05-20T20:41:36.449 INFO:teuthology.orchestra.run.smithi026.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-20T20:41:36.532 INFO:teuthology.orchestra.run.smithi026.stdout:(87/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-05-20T20:41:36.558 INFO:teuthology.orchestra.run.smithi026.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-05-20T20:41:36.583 INFO:teuthology.orchestra.run.smithi026.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-20T20:41:36.661 INFO:teuthology.orchestra.run.smithi026.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 498 kB/s | 51 kB 00:00 2024-05-20T20:41:36.693 INFO:teuthology.orchestra.run.smithi026.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-05-20T20:41:36.717 INFO:teuthology.orchestra.run.smithi026.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-05-20T20:41:36.743 INFO:teuthology.orchestra.run.smithi026.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-05-20T20:41:36.768 INFO:teuthology.orchestra.run.smithi026.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-05-20T20:41:36.793 INFO:teuthology.orchestra.run.smithi026.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-20T20:41:36.818 INFO:teuthology.orchestra.run.smithi026.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-05-20T20:41:36.844 INFO:teuthology.orchestra.run.smithi026.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-20T20:41:36.869 INFO:teuthology.orchestra.run.smithi026.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-20T20:41:36.887 INFO:teuthology.orchestra.run.smithi055.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 198 kB/s | 87 kB 00:00 2024-05-20T20:41:36.894 INFO:teuthology.orchestra.run.smithi026.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-20T20:41:36.919 INFO:teuthology.orchestra.run.smithi026.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-20T20:41:36.945 INFO:teuthology.orchestra.run.smithi026.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-20T20:41:36.979 INFO:teuthology.orchestra.run.smithi026.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 761 kB/s | 45 kB 00:00 2024-05-20T20:41:37.004 INFO:teuthology.orchestra.run.smithi026.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 378 kB/s | 22 kB 00:00 2024-05-20T20:41:37.046 INFO:teuthology.orchestra.run.smithi026.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-05-20T20:41:37.280 INFO:teuthology.orchestra.run.smithi026.stdout:(105/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-20T20:41:37.522 INFO:teuthology.orchestra.run.smithi026.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-20T20:41:37.605 INFO:teuthology.orchestra.run.smithi026.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 912 kB/s | 548 kB 00:00 2024-05-20T20:41:37.613 INFO:teuthology.orchestra.run.smithi055.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 622 kB/s | 1.0 MB 00:01 2024-05-20T20:41:37.664 INFO:teuthology.orchestra.run.smithi026.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 500 kB/s | 309 kB 00:00 2024-05-20T20:41:37.696 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:37.706 INFO:teuthology.orchestra.run.smithi026.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-20T20:41:37.764 INFO:teuthology.orchestra.run.smithi026.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-05-20T20:41:37.776 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:37.793 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T20:41:37.806 INFO:teuthology.orchestra.run.smithi026.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-05-20T20:41:37.854 INFO:teuthology.orchestra.run.smithi055.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 253 kB/s | 61 kB 00:00 2024-05-20T20:41:37.873 INFO:teuthology.orchestra.run.smithi026.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-20T20:41:38.024 INFO:teuthology.orchestra.run.smithi026.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-05-20T20:41:38.065 INFO:teuthology.orchestra.run.smithi026.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-05-20T20:41:38.123 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T20:41:38.124 INFO:teuthology.orchestra.run.smithi026.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-05-20T20:41:38.124 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T20:41:38.135 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T20:41:38.136 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T20:41:38.137 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T20:41:38.138 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T20:41:38.139 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T20:41:38.141 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T20:41:38.142 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T20:41:38.205 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 6.6 k 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 5.4 M 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 890 k 2024-05-20T20:41:38.246 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 145 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 1.5 M 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 143 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 3.5 M 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 7.4 M 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 49 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 11 M 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 49 M 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 273 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 764 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 225 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 27 k 2024-05-20T20:41:38.247 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 18 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 32 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 747 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 127 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T20:41:38.248 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 161 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 320 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 300 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 100 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 85 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.1 M 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 171 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T20:41:38.249 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.4 M 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 3.1 M 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-20T20:41:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 21 M 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 34 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 2.4 M 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 274 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 4.7 M 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 17 M 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g4ebd1d53.el9 ceph-noarch 16 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 25 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T20:41:38.251 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 163 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T20:41:38.252 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 501 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 5.4 M 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T20:41:38.253 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 45 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g4ebd1d53.el9 ceph 136 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T20:41:38.254 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T20:41:38.255 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T20:41:38.256 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T20:41:38.257 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T20:41:38.258 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T20:41:38.259 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout:Install 148 Packages 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 11 Packages 2024-05-20T20:41:38.260 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:41:38.261 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 227 M 2024-05-20T20:41:38.261 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-20T20:41:38.283 INFO:teuthology.orchestra.run.smithi026.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 148 kB/s | 32 kB 00:00 2024-05-20T20:41:38.308 INFO:teuthology.orchestra.run.smithi026.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 81 kB/s | 15 kB 00:00 2024-05-20T20:41:38.383 INFO:teuthology.orchestra.run.smithi026.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-05-20T20:41:38.405 INFO:teuthology.orchestra.run.smithi055.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-05-20T20:41:38.408 INFO:teuthology.orchestra.run.smithi026.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-20T20:41:38.430 INFO:teuthology.orchestra.run.smithi055.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 424 kB/s | 244 kB 00:00 2024-05-20T20:41:38.484 INFO:teuthology.orchestra.run.smithi026.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-20T20:41:38.485 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:38.509 INFO:teuthology.orchestra.run.smithi026.stdout:(121/159): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-05-20T20:41:38.531 INFO:teuthology.orchestra.run.smithi055.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-20T20:41:38.534 INFO:teuthology.orchestra.run.smithi026.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-20T20:41:38.566 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:38.583 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T20:41:38.609 INFO:teuthology.orchestra.run.smithi026.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-20T20:41:38.627 INFO:teuthology.orchestra.run.smithi026.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-20T20:41:38.630 INFO:teuthology.orchestra.run.smithi055.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-05-20T20:41:38.656 INFO:teuthology.orchestra.run.smithi055.stdout:(79/159): python3-jinja2-2.11.3-6.el9.noarch.rp 990 kB/s | 248 kB 00:00 2024-05-20T20:41:38.694 INFO:teuthology.orchestra.run.smithi026.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-05-20T20:41:38.711 INFO:teuthology.orchestra.run.smithi026.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-05-20T20:41:38.739 INFO:teuthology.orchestra.run.smithi055.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-05-20T20:41:38.765 INFO:teuthology.orchestra.run.smithi055.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-20T20:41:38.777 INFO:teuthology.orchestra.run.smithi026.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-05-20T20:41:38.836 INFO:teuthology.orchestra.run.smithi026.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-05-20T20:41:38.895 INFO:teuthology.orchestra.run.smithi026.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-20T20:41:38.937 INFO:teuthology.orchestra.run.smithi026.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-05-20T20:41:38.957 INFO:teuthology.orchestra.run.smithi144.stdout:(1/159): ceph-19.0.0-3830.g4ebd1d53.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-05-20T20:41:38.987 INFO:teuthology.orchestra.run.smithi026.stdout:(131/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-05-20T20:41:39.012 INFO:teuthology.orchestra.run.smithi026.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 671 kB/s | 79 kB 00:00 2024-05-20T20:41:39.038 INFO:teuthology.orchestra.run.smithi026.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 571 kB/s | 58 kB 00:00 2024-05-20T20:41:39.088 INFO:teuthology.orchestra.run.smithi026.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-05-20T20:41:39.107 INFO:teuthology.orchestra.run.smithi055.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 733 kB/s | 250 kB 00:00 2024-05-20T20:41:39.113 INFO:teuthology.orchestra.run.smithi026.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-05-20T20:41:39.139 INFO:teuthology.orchestra.run.smithi026.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-05-20T20:41:39.189 INFO:teuthology.orchestra.run.smithi026.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-05-20T20:41:39.214 INFO:teuthology.orchestra.run.smithi026.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-20T20:41:39.239 INFO:teuthology.orchestra.run.smithi026.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-20T20:41:39.290 INFO:teuthology.orchestra.run.smithi026.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-05-20T20:41:39.315 INFO:teuthology.orchestra.run.smithi026.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-05-20T20:41:39.340 INFO:teuthology.orchestra.run.smithi026.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-05-20T20:41:39.382 INFO:teuthology.orchestra.run.smithi026.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-20T20:41:39.416 INFO:teuthology.orchestra.run.smithi026.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-05-20T20:41:39.441 INFO:teuthology.orchestra.run.smithi026.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-05-20T20:41:39.457 INFO:teuthology.orchestra.run.smithi055.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-05-20T20:41:39.475 INFO:teuthology.orchestra.run.smithi026.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-20T20:41:39.516 INFO:teuthology.orchestra.run.smithi026.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-20T20:41:39.632 INFO:teuthology.orchestra.run.smithi144.stdout:(2/159): ceph-fuse-19.0.0-3830.g4ebd1d53.el9.x8 1.3 MB/s | 890 kB 00:00 2024-05-20T20:41:39.632 INFO:teuthology.orchestra.run.smithi055.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 441 kB/s | 77 kB 00:00 2024-05-20T20:41:39.633 INFO:teuthology.orchestra.run.smithi026.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-20T20:41:39.724 INFO:teuthology.orchestra.run.smithi144.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 1.5 MB/s | 145 kB 00:00 2024-05-20T20:41:39.741 INFO:teuthology.orchestra.run.smithi055.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-05-20T20:41:39.817 INFO:teuthology.orchestra.run.smithi026.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-20T20:41:39.884 INFO:teuthology.orchestra.run.smithi026.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-20T20:41:39.899 INFO:teuthology.orchestra.run.smithi055.stdout:(86/159): postgresql-server-13.14-1.el9.x86_64. 1.4 MB/s | 5.8 MB 00:04 2024-05-20T20:41:39.926 INFO:teuthology.orchestra.run.smithi026.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-20T20:41:39.968 INFO:teuthology.orchestra.run.smithi026.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-20T20:41:40.026 INFO:teuthology.orchestra.run.smithi026.stdout:(153/159): librbd1-19.0.0-3830.g4ebd1d53.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-05-20T20:41:40.027 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:40.043 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:40.132 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:40.212 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:40.216 INFO:teuthology.orchestra.run.smithi055.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 566 kB/s | 269 kB 00:00 2024-05-20T20:41:40.221 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:40.242 INFO:teuthology.orchestra.run.smithi055.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 464 kB/s | 159 kB 00:00 2024-05-20T20:41:40.249 INFO:teuthology.orchestra.run.smithi144.stdout:(4/159): ceph-mds-19.0.0-3830.g4ebd1d53.el9.x86 4.5 MB/s | 2.4 MB 00:00 2024-05-20T20:41:40.302 INFO:teuthology.orchestra.run.smithi026.stdout:(154/159): librados2-19.0.0-3830.g4ebd1d53.el9. 4.1 MB/s | 3.4 MB 00:00 2024-05-20T20:41:40.303 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:40.350 INFO:teuthology.orchestra.run.smithi055.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-05-20T20:41:40.360 INFO:teuthology.orchestra.run.smithi026.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 94 kB/s | 37 kB 00:00 2024-05-20T20:41:40.394 INFO:teuthology.orchestra.run.smithi026.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 563 kB/s | 207 kB 00:00 2024-05-20T20:41:40.394 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:40.449 INFO:teuthology.orchestra.run.smithi144.stdout:(5/159): ceph-mgr-19.0.0-3830.g4ebd1d53.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-05-20T20:41:40.459 INFO:teuthology.orchestra.run.smithi055.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-05-20T20:41:40.474 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:40.490 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:40.544 INFO:teuthology.orchestra.run.smithi026.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.1 MB/s | 260 kB 00:00 2024-05-20T20:41:40.651 INFO:teuthology.orchestra.run.smithi055.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 642 kB/s | 279 kB 00:00 2024-05-20T20:41:40.760 INFO:teuthology.orchestra.run.smithi055.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-05-20T20:41:40.825 INFO:teuthology.orchestra.run.smithi144.stdout:(6/159): ceph-mon-19.0.0-3830.g4ebd1d53.el9.x86 13 MB/s | 4.7 MB 00:00 2024-05-20T20:41:40.860 INFO:teuthology.orchestra.run.smithi055.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-20T20:41:41.202 INFO:teuthology.orchestra.run.smithi055.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 646 kB/s | 221 kB 00:00 2024-05-20T20:41:41.261 INFO:teuthology.orchestra.run.smithi026.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-05-20T20:41:41.303 INFO:teuthology.orchestra.run.smithi055.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 360 kB/s | 36 kB 00:00 2024-05-20T20:41:41.378 INFO:teuthology.orchestra.run.smithi026.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-20T20:41:41.383 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:41:41.383 INFO:teuthology.orchestra.run.smithi026.stdout:Total 11 MB/s | 227 MB 00:20 2024-05-20T20:41:41.403 INFO:teuthology.orchestra.run.smithi055.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-20T20:41:41.666 INFO:teuthology.orchestra.run.smithi144.stdout:(7/159): ceph-base-19.0.0-3830.g4ebd1d53.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-05-20T20:41:41.720 INFO:teuthology.orchestra.run.smithi055.stdout:(97/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.7 MB/s | 5.1 MB 00:02 2024-05-20T20:41:41.820 INFO:teuthology.orchestra.run.smithi055.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-05-20T20:41:41.841 INFO:teuthology.orchestra.run.smithi144.stdout:(8/159): ceph-osd-19.0.0-3830.g4ebd1d53.el9.x86 17 MB/s | 17 MB 00:01 2024-05-20T20:41:41.871 INFO:teuthology.orchestra.run.smithi055.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 652 kB/s | 305 kB 00:00 2024-05-20T20:41:41.892 INFO:teuthology.orchestra.run.smithi144.stdout:(9/159): ceph-selinux-19.0.0-3830.g4ebd1d53.el9 505 kB/s | 25 kB 00:00 2024-05-20T20:41:42.004 INFO:teuthology.orchestra.run.smithi055.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 510 kB/s | 94 kB 00:00 2024-05-20T20:41:42.046 INFO:teuthology.orchestra.run.smithi055.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 363 kB/s | 64 kB 00:00 2024-05-20T20:41:42.096 INFO:teuthology.orchestra.run.smithi055.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-05-20T20:41:42.113 INFO:teuthology.orchestra.run.smithi055.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 411 kB/s | 45 kB 00:00 2024-05-20T20:41:42.214 INFO:teuthology.orchestra.run.smithi055.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-05-20T20:41:42.447 INFO:teuthology.orchestra.run.smithi055.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-05-20T20:41:42.539 INFO:teuthology.orchestra.run.smithi055.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-20T20:41:42.556 INFO:teuthology.orchestra.run.smithi055.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-05-20T20:41:42.648 INFO:teuthology.orchestra.run.smithi055.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-20T20:41:42.654 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-05-20T20:41:42.765 INFO:teuthology.orchestra.run.smithi055.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-05-20T20:41:42.834 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-05-20T20:41:42.834 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-05-20T20:41:42.907 INFO:teuthology.orchestra.run.smithi055.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-05-20T20:41:43.199 INFO:teuthology.orchestra.run.smithi055.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 519 kB/s | 151 kB 00:00 2024-05-20T20:41:43.333 INFO:teuthology.orchestra.run.smithi055.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-05-20T20:41:43.500 INFO:teuthology.orchestra.run.smithi055.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-05-20T20:41:43.625 INFO:teuthology.orchestra.run.smithi055.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-05-20T20:41:43.650 INFO:teuthology.orchestra.run.smithi055.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-05-20T20:41:43.750 INFO:teuthology.orchestra.run.smithi055.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-05-20T20:41:43.784 INFO:teuthology.orchestra.run.smithi055.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-05-20T20:41:43.825 INFO:teuthology.orchestra.run.smithi144.stdout:(10/159): ceph-radosgw-19.0.0-3830.g4ebd1d53.el 4.9 MB/s | 11 MB 00:02 2024-05-20T20:41:43.909 INFO:teuthology.orchestra.run.smithi144.stdout:(11/159): cephfs-mirror-19.0.0-3830.g4ebd1d53.e 2.6 MB/s | 225 kB 00:00 2024-05-20T20:41:43.951 INFO:teuthology.orchestra.run.smithi055.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 857 kB/s | 172 kB 00:00 2024-05-20T20:41:43.993 INFO:teuthology.orchestra.run.smithi144.stdout:(12/159): cephfs-shell-19.0.0-3830.g4ebd1d53.el 325 kB/s | 27 kB 00:00 2024-05-20T20:41:44.068 INFO:teuthology.orchestra.run.smithi144.stdout:(13/159): libcephfs-devel-19.0.0-3830.g4ebd1d53 424 kB/s | 32 kB 00:00 2024-05-20T20:41:44.101 INFO:teuthology.orchestra.run.smithi055.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-05-20T20:41:44.243 INFO:teuthology.orchestra.run.smithi055.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-05-20T20:41:44.244 INFO:teuthology.orchestra.run.smithi144.stdout:(14/159): libcephfs2-19.0.0-3830.g4ebd1d53.el9. 4.2 MB/s | 747 kB 00:00 2024-05-20T20:41:44.343 INFO:teuthology.orchestra.run.smithi144.stdout:(15/159): libcephsqlite-19.0.0-3830.g4ebd1d53.e 1.6 MB/s | 163 kB 00:00 2024-05-20T20:41:44.377 INFO:teuthology.orchestra.run.smithi055.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 746 kB/s | 206 kB 00:00 2024-05-20T20:41:44.435 INFO:teuthology.orchestra.run.smithi144.stdout:(16/159): librados-devel-19.0.0-3830.g4ebd1d53. 1.4 MB/s | 127 kB 00:00 2024-05-20T20:41:44.552 INFO:teuthology.orchestra.run.smithi055.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 826 kB/s | 144 kB 00:00 2024-05-20T20:41:44.653 INFO:teuthology.orchestra.run.smithi055.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-20T20:41:44.670 INFO:teuthology.orchestra.run.smithi144.stdout:(17/159): ceph-common-19.0.0-3830.g4ebd1d53.el9 3.6 MB/s | 21 MB 00:05 2024-05-20T20:41:44.703 INFO:teuthology.orchestra.run.smithi144.stdout:(18/159): libradosstriper1-19.0.0-3830.g4ebd1d5 1.8 MB/s | 501 kB 00:00 2024-05-20T20:41:44.761 INFO:teuthology.orchestra.run.smithi144.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g4e 776 kB/s | 45 kB 00:00 2024-05-20T20:41:44.795 INFO:teuthology.orchestra.run.smithi055.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-05-20T20:41:44.853 INFO:teuthology.orchestra.run.smithi144.stdout:(20/159): python3-ceph-common-19.0.0-3830.g4ebd 1.4 MB/s | 136 kB 00:00 2024-05-20T20:41:44.878 INFO:teuthology.orchestra.run.smithi055.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-05-20T20:41:45.029 INFO:teuthology.orchestra.run.smithi055.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 129 kB/s | 19 kB 00:00 2024-05-20T20:41:45.079 INFO:teuthology.orchestra.run.smithi055.stdout:(127/159): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-05-20T20:41:45.154 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-05-20T20:41:45.154 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-05-20T20:41:45.295 INFO:teuthology.orchestra.run.smithi144.stdout:(21/159): ceph-test-19.0.0-3830.g4ebd1d53.el9.x 14 MB/s | 49 MB 00:03 2024-05-20T20:41:45.320 INFO:teuthology.orchestra.run.smithi144.stdout:(22/159): python3-cephfs-19.0.0-3830.g4ebd1d53. 346 kB/s | 161 kB 00:00 2024-05-20T20:41:45.363 INFO:teuthology.orchestra.run.smithi144.stdout:(23/159): python3-rados-19.0.0-3830.g4ebd1d53.e 4.7 MB/s | 320 kB 00:00 2024-05-20T20:41:45.379 INFO:teuthology.orchestra.run.smithi055.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 56 kB/s | 20 kB 00:00 2024-05-20T20:41:45.413 INFO:teuthology.orchestra.run.smithi144.stdout:(24/159): python3-rgw-19.0.0-3830.g4ebd1d53.el9 2.0 MB/s | 100 kB 00:00 2024-05-20T20:41:45.438 INFO:teuthology.orchestra.run.smithi144.stdout:(25/159): python3-rbd-19.0.0-3830.g4ebd1d53.el9 2.5 MB/s | 300 kB 00:00 2024-05-20T20:41:45.464 INFO:teuthology.orchestra.run.smithi144.stdout:(26/159): rbd-fuse-19.0.0-3830.g4ebd1d53.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-20T20:41:45.522 INFO:teuthology.orchestra.run.smithi144.stdout:(27/159): rbd-nbd-19.0.0-3830.g4ebd1d53.el9.x86 2.9 MB/s | 171 kB 00:00 2024-05-20T20:41:45.571 INFO:teuthology.orchestra.run.smithi055.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 2.1 MB/s | 1.0 MB 00:00 2024-05-20T20:41:45.572 INFO:teuthology.orchestra.run.smithi144.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3830.g 684 kB/s | 34 kB 00:00 2024-05-20T20:41:45.588 INFO:teuthology.orchestra.run.smithi055.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-05-20T20:41:45.631 INFO:teuthology.orchestra.run.smithi144.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3830.g4ebd1d5 2.4 MB/s | 143 kB 00:00 2024-05-20T20:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:(30/159): librgw2-19.0.0-3830.g4ebd1d53.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-05-20T20:41:45.722 INFO:teuthology.orchestra.run.smithi055.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 433 kB/s | 58 kB 00:00 2024-05-20T20:41:45.747 INFO:teuthology.orchestra.run.smithi055.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 449 kB/s | 79 kB 00:00 2024-05-20T20:41:45.897 INFO:teuthology.orchestra.run.smithi055.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-05-20T20:41:45.923 INFO:teuthology.orchestra.run.smithi144.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3830.g4ebd1 12 MB/s | 3.5 MB 00:00 2024-05-20T20:41:45.939 INFO:teuthology.orchestra.run.smithi055.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-05-20T20:41:45.982 INFO:teuthology.orchestra.run.smithi144.stdout:(32/159): rbd-mirror-19.0.0-3830.g4ebd1d53.el9. 5.7 MB/s | 3.1 MB 00:00 2024-05-20T20:41:46.007 INFO:teuthology.orchestra.run.smithi144.stdout:(33/159): ceph-mgr-modules-core-19.0.0-3830.g4e 3.2 MB/s | 274 kB 00:00 2024-05-20T20:41:46.031 INFO:teuthology.orchestra.run.smithi055.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-05-20T20:41:46.032 INFO:teuthology.orchestra.run.smithi144.stdout:(34/159): ceph-mgr-rook-19.0.0-3830.g4ebd1d53.e 986 kB/s | 49 kB 00:00 2024-05-20T20:41:46.048 INFO:teuthology.orchestra.run.smithi055.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 597 kB/s | 90 kB 00:00 2024-05-20T20:41:46.049 INFO:teuthology.orchestra.run.smithi144.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3830.g4 394 kB/s | 16 kB 00:00 2024-05-20T20:41:46.100 INFO:teuthology.orchestra.run.smithi144.stdout:(36/159): ceph-volume-19.0.0-3830.g4ebd1d53.el9 4.0 MB/s | 273 kB 00:00 2024-05-20T20:41:46.142 INFO:teuthology.orchestra.run.smithi144.stdout:(37/159): cephadm-19.0.0-3830.g4ebd1d53.el9.noa 8.1 MB/s | 764 kB 00:00 2024-05-20T20:41:46.159 INFO:teuthology.orchestra.run.smithi144.stdout:(38/159): cephfs-top-19.0.0-3830.g4ebd1d53.el9. 313 kB/s | 18 kB 00:00 2024-05-20T20:41:46.165 INFO:teuthology.orchestra.run.smithi055.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-05-20T20:41:46.276 INFO:teuthology.orchestra.run.smithi144.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-05-20T20:41:46.282 INFO:teuthology.orchestra.run.smithi055.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2024-05-20T20:41:46.307 INFO:teuthology.orchestra.run.smithi055.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 728 kB/s | 188 kB 00:00 2024-05-20T20:41:46.343 INFO:teuthology.orchestra.run.smithi144.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-20T20:41:46.368 INFO:teuthology.orchestra.run.smithi144.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-20T20:41:46.391 INFO:teuthology.orchestra.run.smithi055.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-05-20T20:41:46.393 INFO:teuthology.orchestra.run.smithi144.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-20T20:41:46.435 INFO:teuthology.orchestra.run.smithi144.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-20T20:41:46.466 INFO:teuthology.orchestra.run.smithi055.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 321 kB/s | 51 kB 00:00 2024-05-20T20:41:46.477 INFO:teuthology.orchestra.run.smithi144.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-20T20:41:46.503 INFO:teuthology.orchestra.run.smithi144.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-20T20:41:46.528 INFO:teuthology.orchestra.run.smithi144.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-20T20:41:46.553 INFO:teuthology.orchestra.run.smithi144.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-20T20:41:46.595 INFO:teuthology.orchestra.run.smithi144.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-05-20T20:41:46.616 INFO:teuthology.orchestra.run.smithi055.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 598 kB/s | 90 kB 00:00 2024-05-20T20:41:46.637 INFO:teuthology.orchestra.run.smithi144.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-05-20T20:41:46.662 INFO:teuthology.orchestra.run.smithi144.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-20T20:41:46.663 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:46.684 INFO:teuthology.orchestra.run.smithi055.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 786 kB/s | 230 kB 00:00 2024-05-20T20:41:46.704 INFO:teuthology.orchestra.run.smithi144.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-05-20T20:41:46.746 INFO:teuthology.orchestra.run.smithi144.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-05-20T20:41:46.817 INFO:teuthology.orchestra.run.smithi055.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-05-20T20:41:46.838 INFO:teuthology.orchestra.run.smithi144.stdout:(53/159): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-05-20T20:41:46.839 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:46.914 INFO:teuthology.orchestra.run.smithi144.stdout:(54/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-05-20T20:41:46.931 INFO:teuthology.orchestra.run.smithi144.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-05-20T20:41:46.972 INFO:teuthology.orchestra.run.smithi144.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-05-20T20:41:46.998 INFO:teuthology.orchestra.run.smithi144.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.9 MB/s | 160 kB 00:00 2024-05-20T20:41:47.001 INFO:teuthology.orchestra.run.smithi055.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-05-20T20:41:47.014 INFO:teuthology.orchestra.run.smithi144.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-20T20:41:47.040 INFO:teuthology.orchestra.run.smithi144.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 662 kB 00:00 2024-05-20T20:41:47.074 INFO:teuthology.orchestra.run.smithi144.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-20T20:41:47.090 INFO:teuthology.orchestra.run.smithi144.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-20T20:41:47.107 INFO:teuthology.orchestra.run.smithi144.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-20T20:41:47.118 INFO:teuthology.orchestra.run.smithi055.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 638 kB/s | 192 kB 00:00 2024-05-20T20:41:47.133 INFO:teuthology.orchestra.run.smithi144.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-05-20T20:41:47.158 INFO:teuthology.orchestra.run.smithi144.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-20T20:41:47.175 INFO:teuthology.orchestra.run.smithi144.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 850 kB/s | 35 kB 00:00 2024-05-20T20:41:47.215 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:47.300 INFO:teuthology.orchestra.run.smithi144.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-05-20T20:41:47.318 INFO:teuthology.orchestra.run.smithi055.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-05-20T20:41:47.359 INFO:teuthology.orchestra.run.smithi144.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-05-20T20:41:47.376 INFO:teuthology.orchestra.run.smithi144.stdout:(68/159): elfutils-libelf-devel-0.191-4.el9.x86 29 kB/s | 22 kB 00:00 2024-05-20T20:41:47.410 INFO:teuthology.orchestra.run.smithi144.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-05-20T20:41:47.452 INFO:teuthology.orchestra.run.smithi144.stdout:(70/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-20T20:41:47.510 INFO:teuthology.orchestra.run.smithi144.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-20T20:41:47.528 INFO:teuthology.orchestra.run.smithi144.stdout:(72/159): postgresql-private-libs-13.14-1.el9.x 883 kB/s | 133 kB 00:00 2024-05-20T20:41:47.552 INFO:teuthology.orchestra.run.smithi144.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.0 MB/s | 87 kB 00:00 2024-05-20T20:41:47.560 INFO:teuthology.orchestra.run.smithi055.stdout:(148/159): librados2-19.0.0-3830.g4ebd1d53.el9. 7.7 MB/s | 3.4 MB 00:00 2024-05-20T20:41:47.628 INFO:teuthology.orchestra.run.smithi144.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-05-20T20:41:47.628 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T20:41:47.670 INFO:teuthology.orchestra.run.smithi144.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-20T20:41:47.686 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T20:41:47.709 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:47.744 INFO:teuthology.orchestra.run.smithi055.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-20T20:41:47.787 INFO:teuthology.orchestra.run.smithi144.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-05-20T20:41:47.789 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:47.811 INFO:teuthology.orchestra.run.smithi055.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-20T20:41:47.829 INFO:teuthology.orchestra.run.smithi144.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-05-20T20:41:47.853 INFO:teuthology.orchestra.run.smithi055.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 996 kB/s | 41 kB 00:00 2024-05-20T20:41:47.862 INFO:teuthology.orchestra.run.smithi144.stdout:(78/159): python3-colorama-0.4.6-3.el9.noarch.r 198 kB/s | 61 kB 00:00 2024-05-20T20:41:47.879 INFO:teuthology.orchestra.run.smithi144.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-20T20:41:47.880 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:47.895 INFO:teuthology.orchestra.run.smithi055.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-20T20:41:47.905 INFO:teuthology.orchestra.run.smithi144.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-20T20:41:47.911 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:47.922 INFO:teuthology.orchestra.run.smithi144.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-05-20T20:41:47.964 INFO:teuthology.orchestra.run.smithi144.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-20T20:41:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T20:41:48.006 INFO:teuthology.orchestra.run.smithi144.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-05-20T20:41:48.081 INFO:teuthology.orchestra.run.smithi144.stdout:(84/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-20T20:41:48.086 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:48.106 INFO:teuthology.orchestra.run.smithi144.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 769 kB/s | 77 kB 00:00 2024-05-20T20:41:48.132 INFO:teuthology.orchestra.run.smithi144.stdout:(86/159): python3-jinja2-2.11.3-6.el9.noarch.rp 538 kB/s | 248 kB 00:00 2024-05-20T20:41:48.157 INFO:teuthology.orchestra.run.smithi144.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-05-20T20:41:48.182 INFO:teuthology.orchestra.run.smithi144.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-20T20:41:48.207 INFO:teuthology.orchestra.run.smithi144.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-20T20:41:48.233 INFO:teuthology.orchestra.run.smithi144.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-20T20:41:48.258 INFO:teuthology.orchestra.run.smithi144.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-20T20:41:48.284 INFO:teuthology.orchestra.run.smithi144.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 703 kB/s | 54 kB 00:00 2024-05-20T20:41:48.326 INFO:teuthology.orchestra.run.smithi144.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 993 kB/s | 42 kB 00:00 2024-05-20T20:41:48.354 INFO:teuthology.orchestra.run.smithi144.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 656 kB/s | 44 kB 00:00 2024-05-20T20:41:48.377 INFO:teuthology.orchestra.run.smithi144.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-20T20:41:48.428 INFO:teuthology.orchestra.run.smithi144.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 473 kB/s | 36 kB 00:00 2024-05-20T20:41:48.439 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:48.453 INFO:teuthology.orchestra.run.smithi144.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 624 kB/s | 47 kB 00:00 2024-05-20T20:41:48.479 INFO:teuthology.orchestra.run.smithi144.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 305 kB 00:00 2024-05-20T20:41:48.504 INFO:teuthology.orchestra.run.smithi144.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-20T20:41:48.530 INFO:teuthology.orchestra.run.smithi144.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-20T20:41:48.555 INFO:teuthology.orchestra.run.smithi144.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-20T20:41:48.578 INFO:teuthology.orchestra.run.smithi055.stdout:(153/159): librbd1-19.0.0-3830.g4ebd1d53.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-05-20T20:41:48.580 INFO:teuthology.orchestra.run.smithi144.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 888 kB/s | 45 kB 00:00 2024-05-20T20:41:48.595 INFO:teuthology.orchestra.run.smithi055.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 52 kB/s | 37 kB 00:00 2024-05-20T20:41:48.675 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:48.691 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:48.756 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:48.771 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:48.799 INFO:teuthology.orchestra.run.smithi144.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-20T20:41:48.815 INFO:teuthology.orchestra.run.smithi144.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-20T20:41:48.845 INFO:teuthology.orchestra.run.smithi055.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 828 kB/s | 207 kB 00:00 2024-05-20T20:41:49.119 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:49.136 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:49.187 INFO:teuthology.orchestra.run.smithi055.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 439 kB/s | 260 kB 00:00 2024-05-20T20:41:49.341 INFO:teuthology.orchestra.run.smithi144.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-05-20T20:41:49.441 INFO:teuthology.orchestra.run.smithi144.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 494 kB/s | 309 kB 00:00 2024-05-20T20:41:49.525 INFO:teuthology.orchestra.run.smithi144.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 755 kB/s | 548 kB 00:00 2024-05-20T20:41:49.550 INFO:teuthology.orchestra.run.smithi144.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 93 kB/s | 19 kB 00:00 2024-05-20T20:41:49.750 INFO:teuthology.orchestra.run.smithi144.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 116 kB/s | 26 kB 00:00 2024-05-20T20:41:49.775 INFO:teuthology.orchestra.run.smithi144.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 215 kB/s | 49 kB 00:00 2024-05-20T20:41:49.851 INFO:teuthology.orchestra.run.smithi144.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-20T20:41:49.876 INFO:teuthology.orchestra.run.smithi144.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 535 kB/s | 67 kB 00:00 2024-05-20T20:41:49.902 INFO:teuthology.orchestra.run.smithi144.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-20T20:41:49.985 INFO:teuthology.orchestra.run.smithi144.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-20T20:41:50.010 INFO:teuthology.orchestra.run.smithi144.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-20T20:41:50.035 INFO:teuthology.orchestra.run.smithi144.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-05-20T20:41:50.086 INFO:teuthology.orchestra.run.smithi144.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-05-20T20:41:50.136 INFO:teuthology.orchestra.run.smithi144.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-20T20:41:50.153 INFO:teuthology.orchestra.run.smithi144.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-20T20:41:50.195 INFO:teuthology.orchestra.run.smithi144.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-20T20:41:50.246 INFO:teuthology.orchestra.run.smithi144.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-20T20:41:50.271 INFO:teuthology.orchestra.run.smithi144.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-20T20:41:50.355 INFO:teuthology.orchestra.run.smithi144.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-20T20:41:50.379 INFO:teuthology.orchestra.run.smithi144.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-20T20:41:50.396 INFO:teuthology.orchestra.run.smithi144.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-05-20T20:41:50.455 INFO:teuthology.orchestra.run.smithi144.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-05-20T20:41:50.489 INFO:teuthology.orchestra.run.smithi144.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-20T20:41:50.505 INFO:teuthology.orchestra.run.smithi144.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-05-20T20:41:50.547 INFO:teuthology.orchestra.run.smithi144.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-20T20:41:50.631 INFO:teuthology.orchestra.run.smithi144.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-20T20:41:50.648 INFO:teuthology.orchestra.run.smithi144.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 325 kB/s | 46 kB 00:00 2024-05-20T20:41:50.665 INFO:teuthology.orchestra.run.smithi144.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-05-20T20:41:50.723 INFO:teuthology.orchestra.run.smithi144.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-05-20T20:41:50.769 INFO:teuthology.orchestra.run.smithi055.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 729 kB/s | 1.3 MB 00:01 2024-05-20T20:41:50.774 INFO:teuthology.orchestra.run.smithi144.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-20T20:41:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-05-20T20:41:50.824 INFO:teuthology.orchestra.run.smithi144.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 893 kB/s | 90 kB 00:00 2024-05-20T20:41:50.875 INFO:teuthology.orchestra.run.smithi144.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-05-20T20:41:50.900 INFO:teuthology.orchestra.run.smithi144.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-05-20T20:41:50.933 INFO:teuthology.orchestra.run.smithi144.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-20T20:41:50.975 INFO:teuthology.orchestra.run.smithi144.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-05-20T20:41:51.009 INFO:teuthology.orchestra.run.smithi144.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-20T20:41:51.035 INFO:teuthology.orchestra.run.smithi144.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 505 kB/s | 51 kB 00:00 2024-05-20T20:41:51.085 INFO:teuthology.orchestra.run.smithi144.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-20T20:41:51.110 INFO:teuthology.orchestra.run.smithi144.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 890 kB/s | 90 kB 00:00 2024-05-20T20:41:51.144 INFO:teuthology.orchestra.run.smithi144.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-20T20:41:51.186 INFO:teuthology.orchestra.run.smithi144.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-05-20T20:41:51.219 INFO:teuthology.orchestra.run.smithi144.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-20T20:41:51.337 INFO:teuthology.orchestra.run.smithi144.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-20T20:41:51.520 INFO:teuthology.orchestra.run.smithi144.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-20T20:41:51.587 INFO:teuthology.orchestra.run.smithi144.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-20T20:41:51.629 INFO:teuthology.orchestra.run.smithi144.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 997 kB/s | 41 kB 00:00 2024-05-20T20:41:51.671 INFO:teuthology.orchestra.run.smithi144.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-20T20:41:51.687 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-05-20T20:41:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T20:41:51.861 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T20:41:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T20:41:51.954 INFO:teuthology.orchestra.run.smithi144.stdout:(153/159): librados2-19.0.0-3830.g4ebd1d53.el9. 4.4 MB/s | 3.4 MB 00:00 2024-05-20T20:41:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T20:41:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T20:41:52.004 INFO:teuthology.orchestra.run.smithi144.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 110 kB/s | 37 kB 00:00 2024-05-20T20:41:52.037 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T20:41:52.053 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:52.085 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:52.134 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:52.222 INFO:teuthology.orchestra.run.smithi144.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 779 kB/s | 207 kB 00:00 2024-05-20T20:41:52.371 INFO:teuthology.orchestra.run.smithi055.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 770 kB/s | 2.4 MB 00:03 2024-05-20T20:41:52.372 INFO:teuthology.orchestra.run.smithi144.stdout:(156/159): librbd1-19.0.0-3830.g4ebd1d53.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-05-20T20:41:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T20:41:52.478 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T20:41:52.494 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T20:41:52.509 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:41:52.547 INFO:teuthology.orchestra.run.smithi144.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 479 kB/s | 260 kB 00:00 2024-05-20T20:41:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:41:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:41:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:41:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 10/170 2024-05-20T20:41:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T20:41:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T20:41:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T20:41:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:41:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:41:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 15/170 2024-05-20T20:41:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 15/170 2024-05-20T20:41:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T20:41:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T20:41:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T20:41:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T20:41:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T20:41:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T20:41:54.022 INFO:teuthology.orchestra.run.smithi144.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 767 kB/s | 1.3 MB 00:01 2024-05-20T20:41:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:41:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:41:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:41:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T20:41:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T20:41:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-20T20:41:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-20T20:41:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-20T20:41:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-20T20:41:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-20T20:41:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-20T20:41:54.792 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-20T20:41:54.843 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-20T20:41:54.880 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-20T20:41:54.915 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-20T20:41:54.995 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-20T20:41:55.026 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9. 33/170 2024-05-20T20:41:55.095 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el 34/170 2024-05-20T20:41:55.123 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_6 35/170 2024-05-20T20:41:55.197 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-20T20:41:55.239 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-20T20:41:55.317 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-20T20:41:55.464 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-20T20:41:55.534 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-20T20:41:55.673 INFO:teuthology.orchestra.run.smithi144.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 743 kB/s | 2.4 MB 00:03 2024-05-20T20:41:55.681 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:41:55.681 INFO:teuthology.orchestra.run.smithi144.stdout:Total 13 MB/s | 227 MB 00:17 2024-05-20T20:41:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-20T20:41:56.181 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-20T20:41:56.224 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-20T20:41:56.269 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-20T20:41:56.318 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-20T20:41:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_6 46/170 2024-05-20T20:41:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-20T20:41:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-20T20:41:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-20T20:41:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-20T20:41:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-20T20:41:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T20:41:56.891 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-20T20:41:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T20:41:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T20:41:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T20:41:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T20:41:57.049 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-20T20:41:57.049 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-20T20:41:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T20:41:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T20:41:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T20:41:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T20:41:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T20:41:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T20:41:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T20:41:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T20:41:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T20:41:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T20:41:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T20:41:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T20:41:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T20:41:57.950 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T20:41:57.984 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T20:41:58.023 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T20:41:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T20:41:58.093 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T20:41:58.127 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T20:41:58.160 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T20:41:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T20:41:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T20:41:58.328 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T20:41:58.376 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T20:41:58.407 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T20:41:58.455 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T20:41:58.566 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T20:41:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T20:41:59.263 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-20T20:41:59.263 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-20T20:41:59.393 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T20:41:59.447 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T20:41:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T20:41:59.588 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T20:41:59.781 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T20:41:59.816 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T20:41:59.992 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T20:42:00.023 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T20:42:00.103 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T20:42:00.812 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T20:42:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T20:42:02.294 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T20:42:02.378 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T20:42:02.406 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T20:42:02.439 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T20:42:02.562 INFO:teuthology.orchestra.run.smithi055.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 753 kB/s | 16 MB 00:22 2024-05-20T20:42:02.568 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:42:02.569 INFO:teuthology.orchestra.run.smithi055.stdout:Total 5.3 MB/s | 227 MB 00:42 2024-05-20T20:42:02.775 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T20:42:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 103/170 2024-05-20T20:42:02.918 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 103/170 2024-05-20T20:42:02.948 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 104/170 2024-05-20T20:42:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T20:42:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T20:42:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 107/170 2024-05-20T20:42:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 107/170 2024-05-20T20:42:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 108/170 2024-05-20T20:42:03.840 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-05-20T20:42:04.018 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-05-20T20:42:04.018 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-05-20T20:42:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-20T20:42:05.586 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T20:42:05.631 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T20:42:05.688 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T20:42:05.728 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T20:42:05.753 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T20:42:05.807 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T20:42:06.000 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:06.036 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:06.149 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T20:42:06.262 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:42:06.303 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-05-20T20:42:06.304 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-05-20T20:42:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:42:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:42:06.521 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:42:06.552 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 10/170 2024-05-20T20:42:06.611 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T20:42:06.632 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:06.672 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 110/170 2024-05-20T20:42:06.766 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 110/170 2024-05-20T20:42:06.766 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T20:42:06.766 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T20:42:06.766 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:06.815 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 111/170 2024-05-20T20:42:06.829 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T20:42:06.883 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T20:42:06.911 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:42:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:42:07.037 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 15/170 2024-05-20T20:42:07.100 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 15/170 2024-05-20T20:42:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T20:42:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T20:42:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T20:42:07.487 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T20:42:07.666 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T20:42:07.729 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T20:42:07.836 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:42:07.914 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:42:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:42:08.228 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T20:42:08.260 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T20:42:08.295 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-20T20:42:08.376 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-20T20:42:08.411 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-20T20:42:08.454 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-20T20:42:08.488 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-20T20:42:08.515 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-20T20:42:08.591 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-20T20:42:08.645 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-20T20:42:08.676 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-20T20:42:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-20T20:42:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-20T20:42:08.831 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9. 33/170 2024-05-20T20:42:08.874 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el 34/170 2024-05-20T20:42:08.903 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_6 35/170 2024-05-20T20:42:08.972 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-20T20:42:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-20T20:42:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-20T20:42:09.224 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-20T20:42:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-20T20:42:09.804 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-20T20:42:09.882 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-20T20:42:09.912 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-20T20:42:09.954 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-20T20:42:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-20T20:42:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_6 46/170 2024-05-20T20:42:10.077 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-20T20:42:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-20T20:42:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-20T20:42:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-20T20:42:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-20T20:42:10.492 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T20:42:10.525 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T20:42:10.561 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T20:42:10.608 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T20:42:10.644 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T20:42:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T20:42:10.729 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T20:42:10.865 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T20:42:10.906 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T20:42:10.947 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T20:42:11.041 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T20:42:11.073 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T20:42:11.114 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T20:42:11.155 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T20:42:11.179 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T20:42:11.269 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T20:42:11.397 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T20:42:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T20:42:11.691 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T20:42:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T20:42:11.774 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T20:42:11.813 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T20:42:11.843 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T20:42:11.877 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T20:42:11.910 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T20:42:11.953 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T20:42:11.986 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T20:42:12.078 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T20:42:12.125 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T20:42:12.155 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T20:42:12.203 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T20:42:12.309 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T20:42:12.364 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T20:42:12.676 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-05-20T20:42:12.767 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T20:42:12.810 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T20:42:12.878 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T20:42:12.918 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T20:42:12.943 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T20:42:12.995 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T20:42:13.079 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T20:42:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T20:42:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T20:42:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T20:42:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T20:42:13.341 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T20:42:13.345 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T20:42:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T20:42:13.453 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:42:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T20:42:13.562 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:42:13.587 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:42:13.658 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T20:42:13.679 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:42:13.688 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T20:42:13.709 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 10/170 2024-05-20T20:42:13.760 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T20:42:13.766 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T20:42:13.984 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T20:42:14.032 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T20:42:14.060 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:42:14.166 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:42:14.186 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 15/170 2024-05-20T20:42:14.240 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 15/170 2024-05-20T20:42:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T20:42:14.288 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T20:42:14.415 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T20:42:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T20:42:14.611 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T20:42:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T20:42:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T20:42:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T20:42:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:42:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:42:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T20:42:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T20:42:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T20:42:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-20T20:42:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-20T20:42:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-20T20:42:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-20T20:42:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-20T20:42:15.781 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T20:42:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-20T20:42:15.883 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T20:42:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-20T20:42:15.910 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T20:42:15.943 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T20:42:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-20T20:42:15.974 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-20T20:42:16.018 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-20T20:42:16.097 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-20T20:42:16.254 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9. 33/170 2024-05-20T20:42:16.273 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T20:42:16.298 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el 34/170 2024-05-20T20:42:16.324 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 103/170 2024-05-20T20:42:16.326 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_6 35/170 2024-05-20T20:42:16.391 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-20T20:42:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 103/170 2024-05-20T20:42:16.441 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-20T20:42:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 104/170 2024-05-20T20:42:16.481 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T20:42:16.520 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-20T20:42:16.664 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-20T20:42:16.720 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-20T20:42:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T20:42:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 107/170 2024-05-20T20:42:17.065 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 107/170 2024-05-20T20:42:17.095 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 108/170 2024-05-20T20:42:17.288 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-20T20:42:17.375 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-20T20:42:17.410 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-20T20:42:17.447 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-20T20:42:17.495 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-20T20:42:17.533 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_6 46/170 2024-05-20T20:42:17.577 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-20T20:42:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-20T20:42:17.803 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-20T20:42:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-20T20:42:17.961 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-20T20:42:18.035 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T20:42:18.071 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T20:42:18.107 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T20:42:18.156 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T20:42:18.190 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T20:42:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T20:42:18.277 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T20:42:18.421 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T20:42:18.461 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T20:42:18.503 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T20:42:18.601 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T20:42:18.644 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T20:42:18.685 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T20:42:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T20:42:18.750 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T20:42:18.845 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 111/170 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-05-20T20:42:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:18.983 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T20:42:19.016 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T20:42:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 112/170 2024-05-20T20:42:19.050 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T20:42:19.056 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 112/170 2024-05-20T20:42:19.057 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:19.057 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T20:42:19.057 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T20:42:19.118 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T20:42:19.158 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T20:42:19.187 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T20:42:19.222 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T20:42:19.255 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T20:42:19.300 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T20:42:19.333 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T20:42:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:19.425 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T20:42:19.482 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T20:42:19.512 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T20:42:19.560 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T20:42:19.670 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T20:42:19.723 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T20:42:19.943 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 110/170 2024-05-20T20:42:20.073 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 110/170 2024-05-20T20:42:20.073 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T20:42:20.073 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T20:42:20.073 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:20.106 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 111/170 2024-05-20T20:42:20.484 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T20:42:20.537 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T20:42:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T20:42:20.599 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 113/170 2024-05-20T20:42:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 113/170 2024-05-20T20:42:20.627 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:20.627 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T20:42:20.627 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T20:42:20.627 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T20:42:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:20.673 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T20:42:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T20:42:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T20:42:20.857 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 114/170 2024-05-20T20:42:20.862 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T20:42:20.882 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 114/170 2024-05-20T20:42:20.882 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:20.882 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T20:42:20.882 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T20:42:20.882 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T20:42:20.882 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:20.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T20:42:21.083 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T20:42:21.121 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T20:42:21.203 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T20:42:21.350 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 115/170 2024-05-20T20:42:21.375 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 115/170 2024-05-20T20:42:21.375 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:21.375 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T20:42:21.375 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T20:42:21.375 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T20:42:21.375 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:21.489 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T20:42:21.547 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T20:42:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T20:42:21.806 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T20:42:21.868 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T20:42:21.888 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T20:42:21.941 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T20:42:22.016 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T20:42:22.038 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T20:42:22.182 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T20:42:22.226 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T20:42:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T20:42:22.285 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T20:42:22.768 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 127/170 2024-05-20T20:42:22.801 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 127/170 2024-05-20T20:42:22.833 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.e 128/170 2024-05-20T20:42:22.898 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53. 129/170 2024-05-20T20:42:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 130/170 2024-05-20T20:42:23.336 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T20:42:23.420 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T20:42:23.448 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T20:42:23.481 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T20:42:23.819 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T20:42:23.877 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 103/170 2024-05-20T20:42:23.969 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 103/170 2024-05-20T20:42:24.000 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 104/170 2024-05-20T20:42:24.025 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 130/170 2024-05-20T20:42:24.036 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T20:42:24.058 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 131/170 2024-05-20T20:42:24.543 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T20:42:24.572 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 107/170 2024-05-20T20:42:24.644 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 107/170 2024-05-20T20:42:24.674 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 108/170 2024-05-20T20:42:25.074 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 131/170 2024-05-20T20:42:25.100 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 132/170 2024-05-20T20:42:25.219 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 132/170 2024-05-20T20:42:25.386 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el 133/170 2024-05-20T20:42:25.419 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 134/170 2024-05-20T20:42:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 134/170 2024-05-20T20:42:25.444 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:25.444 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T20:42:25.445 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T20:42:25.445 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T20:42:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 135/170 2024-05-20T20:42:25.507 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 135/170 2024-05-20T20:42:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 136/170 2024-05-20T20:42:25.669 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 137/170 2024-05-20T20:42:26.645 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T20:42:26.672 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 139/170 2024-05-20T20:42:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 139/170 2024-05-20T20:42:26.698 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:26.698 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T20:42:26.698 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T20:42:26.698 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T20:42:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:26.740 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 140/170 2024-05-20T20:42:26.764 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 140/170 2024-05-20T20:42:26.764 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:26.764 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T20:42:26.764 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 141/170 2024-05-20T20:42:26.840 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 141/170 2024-05-20T20:42:26.840 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:26.840 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T20:42:26.840 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T20:42:26.841 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T20:42:26.841 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:27.129 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:27.157 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 142/170 2024-05-20T20:42:27.164 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:27.183 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 142/170 2024-05-20T20:42:27.183 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:27.183 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T20:42:27.183 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T20:42:27.183 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T20:42:27.183 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:27.741 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 109/170 2024-05-20T20:42:27.777 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 110/170 2024-05-20T20:42:27.868 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 110/170 2024-05-20T20:42:27.869 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T20:42:27.869 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T20:42:27.869 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 111/170 2024-05-20T20:42:31.518 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 111/170 2024-05-20T20:42:31.518 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-05-20T20:42:31.519 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:31.686 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 112/170 2024-05-20T20:42:31.702 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 112/170 2024-05-20T20:42:31.702 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:31.702 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T20:42:31.702 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:32.068 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 143/170 2024-05-20T20:42:32.118 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 144/170 2024-05-20T20:42:32.159 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 145/170 2024-05-20T20:42:32.190 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T20:42:32.226 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T20:42:32.437 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 148/170 2024-05-20T20:42:32.488 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T20:42:32.521 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_ 150/170 2024-05-20T20:42:32.570 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T20:42:32.599 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T20:42:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T20:42:33.181 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 113/170 2024-05-20T20:42:33.208 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 113/170 2024-05-20T20:42:33.208 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:33.208 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T20:42:33.208 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T20:42:33.208 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T20:42:33.208 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:33.451 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 114/170 2024-05-20T20:42:33.474 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 114/170 2024-05-20T20:42:33.474 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:33.474 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T20:42:33.474 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T20:42:33.474 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T20:42:33.474 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:33.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 115/170 2024-05-20T20:42:33.947 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 115/170 2024-05-20T20:42:33.947 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:33.947 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T20:42:33.947 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T20:42:33.947 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T20:42:33.948 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:34.045 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T20:42:34.103 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T20:42:34.159 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T20:42:34.213 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 155/170 2024-05-20T20:42:34.314 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T20:42:34.344 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T20:42:34.349 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T20:42:34.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T20:42:34.452 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T20:42:34.469 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T20:42:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T20:42:34.517 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T20:42:34.542 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T20:42:34.569 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T20:42:34.569 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T20:42:34.574 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T20:42:34.654 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T20:42:34.686 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T20:42:34.686 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T20:42:34.722 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T20:42:34.758 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T20:42:34.779 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T20:42:34.790 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T20:42:34.804 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T20:42:34.810 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:34.810 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:34.817 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T20:42:34.998 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:34.999 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T20:42:35.074 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T20:42:35.098 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T20:42:35.162 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 127/170 2024-05-20T20:42:35.168 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T20:42:35.191 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 127/170 2024-05-20T20:42:35.223 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.e 128/170 2024-05-20T20:42:35.285 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53. 129/170 2024-05-20T20:42:35.314 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 130/170 2024-05-20T20:42:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 130/170 2024-05-20T20:42:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 131/170 2024-05-20T20:42:37.357 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 131/170 2024-05-20T20:42:37.396 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 132/170 2024-05-20T20:42:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 132/170 2024-05-20T20:42:37.670 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el 133/170 2024-05-20T20:42:37.702 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 134/170 2024-05-20T20:42:37.725 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 134/170 2024-05-20T20:42:37.725 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:37.725 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T20:42:37.725 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T20:42:37.726 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T20:42:37.726 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 135/170 2024-05-20T20:42:37.780 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 135/170 2024-05-20T20:42:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 136/170 2024-05-20T20:42:37.934 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 137/170 2024-05-20T20:42:38.865 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T20:42:38.897 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 139/170 2024-05-20T20:42:38.920 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 139/170 2024-05-20T20:42:38.920 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:38.920 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T20:42:38.920 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T20:42:38.920 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T20:42:38.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:38.964 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 140/170 2024-05-20T20:42:38.986 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 140/170 2024-05-20T20:42:38.986 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:38.987 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T20:42:38.987 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:39.031 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 141/170 2024-05-20T20:42:39.054 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 141/170 2024-05-20T20:42:39.055 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:39.055 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T20:42:39.055 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T20:42:39.055 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T20:42:39.055 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:39.357 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 142/170 2024-05-20T20:42:39.380 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 142/170 2024-05-20T20:42:39.380 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:39.380 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T20:42:39.380 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T20:42:39.380 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T20:42:39.380 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:39.869 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 111/170 2024-05-20T20:42:39.870 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2024-05-20T20:42:39.870 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2024-05-20T20:42:39.870 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2024-05-20T20:42:39.870 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2024-05-20T20:42:39.870 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2024-05-20T20:42:39.870 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /root 2024-05-20T20:42:39.871 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2024-05-20T20:42:39.871 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:40.056 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 112/170 2024-05-20T20:42:40.074 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 112/170 2024-05-20T20:42:40.074 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:40.074 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T20:42:40.074 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:40.783 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T20:42:40.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 1/170 2024-05-20T20:42:40.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2/170 2024-05-20T20:42:40.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 3/170 2024-05-20T20:42:40.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 4/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 5/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 6/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 7/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 10/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 11/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 12/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 13/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_ 15/170 2024-05-20T20:42:40.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 16/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 17/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_6 18/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 19/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 20/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el 21/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9. 22/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_6 23/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 24/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 25/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 26/170 2024-05-20T20:42:40.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 27/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 28/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 29/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53. 30/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 31/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 32/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 33/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el 34/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 35/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.e 36/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 37/170 2024-05-20T20:42:40.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 38/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 39/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T20:42:40.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T20:42:40.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T20:42:40.789 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T20:42:40.790 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T20:42:40.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T20:42:40.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T20:42:40.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T20:42:40.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 149/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 151/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T20:42:40.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T20:42:40.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T20:42:40.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T20:42:40.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T20:42:41.626 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 113/170 2024-05-20T20:42:41.655 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 113/170 2024-05-20T20:42:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T20:42:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T20:42:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T20:42:41.656 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:41.894 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 114/170 2024-05-20T20:42:41.919 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 114/170 2024-05-20T20:42:41.919 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:41.919 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T20:42:41.919 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T20:42:41.919 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T20:42:41.919 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:42.386 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 115/170 2024-05-20T20:42:42.411 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 115/170 2024-05-20T20:42:42.412 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:42.412 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T20:42:42.412 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T20:42:42.412 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T20:42:42.412 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:42.526 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T20:42:42.593 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T20:42:42.656 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T20:42:42.852 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T20:42:42.914 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T20:42:42.995 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T20:42:43.084 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T20:42:43.228 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T20:42:43.272 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T20:42:43.313 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T20:42:43.340 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:43.810 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T20:42:43.811 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T20:42:43.813 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T20:42:43.814 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T20:42:43.815 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T20:42:43.816 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T20:42:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T20:42:43.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T20:42:43.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T20:42:43.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.821 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T20:42:43.822 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T20:42:43.823 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 127/170 2024-05-20T20:42:43.823 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T20:42:43.823 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:42:43.823 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-05-20T20:42:43.839 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 127/170 2024-05-20T20:42:43.871 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.e 128/170 2024-05-20T20:42:43.936 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53. 129/170 2024-05-20T20:42:43.962 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 130/170 2024-05-20T20:42:43.979 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 143/170 2024-05-20T20:42:44.020 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 144/170 2024-05-20T20:42:44.061 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 145/170 2024-05-20T20:42:44.093 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T20:42:44.120 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T20:42:44.235 DEBUG:teuthology.parallel:result is None 2024-05-20T20:42:44.328 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 148/170 2024-05-20T20:42:44.375 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T20:42:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_ 150/170 2024-05-20T20:42:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T20:42:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T20:42:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T20:42:45.063 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 130/170 2024-05-20T20:42:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 131/170 2024-05-20T20:42:45.144 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:45.208 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:46.032 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 155/170 2024-05-20T20:42:46.118 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T20:42:46.121 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 131/170 2024-05-20T20:42:46.156 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 132/170 2024-05-20T20:42:46.168 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T20:42:46.261 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T20:42:46.276 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 132/170 2024-05-20T20:42:46.297 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T20:42:46.322 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T20:42:46.347 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T20:42:46.373 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T20:42:46.373 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T20:42:46.442 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el 133/170 2024-05-20T20:42:46.458 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T20:42:46.475 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 134/170 2024-05-20T20:42:46.491 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T20:42:46.491 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T20:42:46.500 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 134/170 2024-05-20T20:42:46.501 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:46.501 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T20:42:46.501 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T20:42:46.501 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T20:42:46.501 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:46.551 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 135/170 2024-05-20T20:42:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 135/170 2024-05-20T20:42:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T20:42:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T20:42:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:46.664 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 136/170 2024-05-20T20:42:46.725 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 137/170 2024-05-20T20:42:46.786 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:46.786 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T20:42:46.875 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T20:42:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T20:42:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T20:42:47.702 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T20:42:47.729 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 139/170 2024-05-20T20:42:47.754 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 139/170 2024-05-20T20:42:47.754 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:47.755 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T20:42:47.755 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T20:42:47.755 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T20:42:47.755 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:47.796 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 140/170 2024-05-20T20:42:47.820 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 140/170 2024-05-20T20:42:47.820 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:47.821 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T20:42:47.821 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:47.872 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 141/170 2024-05-20T20:42:47.897 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 141/170 2024-05-20T20:42:47.897 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:47.897 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T20:42:47.897 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T20:42:47.897 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T20:42:47.897 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:48.223 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 142/170 2024-05-20T20:42:48.248 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 142/170 2024-05-20T20:42:48.248 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T20:42:48.248 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T20:42:48.249 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T20:42:48.249 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T20:42:48.249 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 1/170 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2/170 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 3/170 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 4/170 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 5/170 2024-05-20T20:42:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 6/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 7/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 10/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 11/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 12/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 13/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_ 15/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 16/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 17/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_6 18/170 2024-05-20T20:42:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 19/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 20/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el 21/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9. 22/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_6 23/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 24/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 25/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 26/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 27/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 28/170 2024-05-20T20:42:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 29/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53. 30/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 31/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 32/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 33/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el 34/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 35/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.e 36/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 37/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 38/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 39/170 2024-05-20T20:42:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T20:42:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T20:42:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T20:42:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T20:42:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T20:42:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T20:42:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T20:42:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 149/170 2024-05-20T20:42:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 151/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T20:42:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T20:42:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T20:42:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T20:42:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T20:42:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T20:42:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T20:42:53.107 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 143/170 2024-05-20T20:42:53.152 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 144/170 2024-05-20T20:42:53.193 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 145/170 2024-05-20T20:42:53.228 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T20:42:53.261 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T20:42:53.467 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 148/170 2024-05-20T20:42:53.515 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T20:42:53.547 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_ 150/170 2024-05-20T20:42:53.587 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T20:42:53.617 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T20:42:53.663 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T20:42:54.310 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:54.352 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:54.374 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T20:42:55.182 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.183 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.184 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T20:42:55.185 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T20:42:55.186 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T20:42:55.187 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T20:42:55.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T20:42:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T20:42:55.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T20:42:55.191 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T20:42:55.192 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T20:42:55.193 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T20:42:55.194 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T20:42:55.194 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:42:55.194 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-20T20:42:55.292 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 155/170 2024-05-20T20:42:55.383 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T20:42:55.425 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T20:42:55.532 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T20:42:55.569 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T20:42:55.594 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T20:42:55.619 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T20:42:55.647 DEBUG:teuthology.parallel:result is None 2024-05-20T20:42:55.648 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T20:42:55.648 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T20:42:55.747 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T20:42:55.780 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T20:42:55.780 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T20:42:55.856 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T20:42:55.881 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T20:42:55.887 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:55.887 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:56.075 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T20:42:56.075 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T20:42:56.167 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T20:42:56.192 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T20:42:56.261 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 1/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 3/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 4/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1 5/170 2024-05-20T20:43:02.075 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 6/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 7/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 8/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 9/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 10/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 11/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 12/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 13/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 14/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_ 15/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 16/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 17/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_6 18/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86 19/170 2024-05-20T20:43:02.076 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 20/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el 21/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9. 22/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_6 23/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 24/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 25/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 26/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 27/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 28/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 29/170 2024-05-20T20:43:02.077 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53. 30/170 2024-05-20T20:43:02.089 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noa 31/170 2024-05-20T20:43:02.089 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.n 32/170 2024-05-20T20:43:02.089 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g4eb 33/170 2024-05-20T20:43:02.089 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el 34/170 2024-05-20T20:43:02.089 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 35/170 2024-05-20T20:43:02.089 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.e 36/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 37/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 38/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 39/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T20:43:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T20:43:02.091 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T20:43:02.092 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T20:43:02.093 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T20:43:02.094 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T20:43:02.095 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T20:43:02.096 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T20:43:02.097 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 149/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 151/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T20:43:02.098 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T20:43:02.099 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T20:43:04.928 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-shell-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top-2:19.0.0-3830.g4ebd1d53.el9.noarch 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T20:43:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T20:43:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T20:43:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T20:43:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T20:43:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T20:43:04.937 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T20:43:04.938 INFO:teuthology.orchestra.run.smithi055.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T20:43:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T20:43:04.940 INFO:teuthology.orchestra.run.smithi055.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:19.0.0-3830.g4ebd1d53.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T20:43:04.941 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T20:43:04.942 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-05-20T20:43:05.405 DEBUG:teuthology.parallel:result is None 2024-05-20T20:43:05.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:43:05.623 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T20:43:05.651 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-3830.g4ebd1d53.el9 2024-05-20T20:43:05.651 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g4ebd1d53.el9 2024-05-20T20:43:05.652 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g4ebd1d53 is installed. 2024-05-20T20:43:05.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:43:05.788 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T20:43:05.813 INFO:teuthology.orchestra.run.smithi055.stdout:19.0.0-3830.g4ebd1d53.el9 2024-05-20T20:43:05.814 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g4ebd1d53.el9 2024-05-20T20:43:05.814 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g4ebd1d53 is installed. 2024-05-20T20:43:05.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:43:06.031 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T20:43:06.058 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-3830.g4ebd1d53.el9 2024-05-20T20:43:06.058 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g4ebd1d53.el9 2024-05-20T20:43:06.059 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g4ebd1d53 is installed. 2024-05-20T20:43:06.060 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-20T20:43:06.061 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:43:06.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T20:43:06.094 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:43:06.094 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T20:43:06.128 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:43:06.128 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T20:43:06.160 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-20T20:43:06.160 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:43:06.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T20:43:06.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T20:43:06.261 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:43:06.261 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T20:43:06.292 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T20:43:06.362 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:43:06.362 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T20:43:06.392 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T20:43:06.461 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-20T20:43:06.462 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:43:06.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T20:43:06.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T20:43:06.555 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:43:06.555 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T20:43:06.585 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T20:43:06.655 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:43:06.655 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T20:43:06.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T20:43:06.752 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-20T20:43:06.753 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:43:06.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T20:43:06.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T20:43:06.846 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:43:06.846 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T20:43:06.876 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T20:43:06.947 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:43:06.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T20:43:06.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T20:43:07.044 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-20T20:43:07.132 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '4ebd1d5361d457738957dc0455bba90102296634', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-20T20:43:07.132 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:43:07.132 INFO:tasks.cephadm:Cluster fsid is 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:43:07.133 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-20T20:43:07.133 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.26', 'mon.b': '172.21.15.55', 'mon.c': '172.21.15.144'} 2024-05-20T20:43:07.133 INFO:tasks.cephadm:First mon is mon.a on smithi026 2024-05-20T20:43:07.133 INFO:tasks.cephadm:First mgr is x 2024-05-20T20:43:07.133 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-20T20:43:07.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-05-20T20:43:07.160 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname $(hostname -s) 2024-05-20T20:43:07.191 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-05-20T20:43:07.258 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-20T20:43:07.258 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:43:07.475 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '4ebd1d5361d457738957dc0455bba90102296634', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79554/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3830-g4ebd1d53', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3830.g4ebd1d53'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-20 18:01:42.330101', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-20T20:43:07.689 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce, sha1 4ebd1d5361d457738957dc0455bba90102296634 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:43:07.691 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/x86_64/flavors/default/cephadm 2024-05-20T20:43:07.691 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/x86_64/flavors/default/cephadm 2024-05-20T20:43:07.691 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:08.578 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 782820 May 20 20:43 /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:08.578 DEBUG:teuthology.orchestra.run.smithi055:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:09.541 INFO:teuthology.orchestra.run.smithi055.stdout:-rw-r--r--. 1 ubuntu ubuntu 782820 May 20 20:43 /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:09.541 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/4ebd1d5361d457738957dc0455bba90102296634/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:10.664 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 782820 May 20 20:43 /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:10.664 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:10.685 DEBUG:teuthology.orchestra.run.smithi055:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:10.705 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-20T20:43:10.739 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 on all hosts... 2024-05-20T20:43:10.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 pull 2024-05-20T20:43:10.742 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 pull 2024-05-20T20:43:10.748 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 pull 2024-05-20T20:43:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634... 2024-05-20T20:43:11.620 INFO:teuthology.orchestra.run.smithi055.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634... 2024-05-20T20:43:11.730 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634... 2024-05-20T20:43:48.707 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-20T20:43:48.708 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev)", 2024-05-20T20:43:48.708 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489", 2024-05-20T20:43:48.708 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-05-20T20:43:48.708 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3" 2024-05-20T20:43:48.708 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-05-20T20:43:48.708 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev)", 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489", 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3" 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T20:44:24.286 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev)", 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout: "image_id": "559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489", 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout: "repo_digests": [ 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3" 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-20T20:44:24.748 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T20:44:24.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-05-20T20:44:24.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph 2024-05-20T20:44:24.833 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-05-20T20:44:24.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-05-20T20:44:24.891 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /etc/ceph 2024-05-20T20:44:24.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-05-20T20:44:24.946 INFO:tasks.cephadm:Writing seed config... 2024-05-20T20:44:24.947 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-20T20:44:24.947 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-20T20:44:24.947 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-20T20:44:24.947 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-20T20:44:24.948 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-20T20:44:24.949 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-20T20:44:24.950 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-20T20:44:24.951 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-20T20:44:24.951 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-20T20:44:24.951 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:44:24.951 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-20T20:44:24.967 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9006c62a-16e9-11ef-bc9a-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-20T20:44:24.967 DEBUG:teuthology.orchestra.run.smithi026:mon.a> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a.service 2024-05-20T20:44:25.010 DEBUG:teuthology.orchestra.run.smithi026:mgr.x> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-20T20:44:25.053 INFO:tasks.cephadm:Bootstrapping... 2024-05-20T20:44:25.053 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 -v bootstrap --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.26 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-20T20:44:25.250 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-05-20T20:44:25.250 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634', '-v', 'bootstrap', '--fsid', '9006c62a-16e9-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.26', '--skip-admin-label'] 2024-05-20T20:44:25.250 INFO:teuthology.orchestra.run.smithi026.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-20T20:44:25.251 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying podman|docker is present... 2024-05-20T20:44:25.277 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 5.0.2 2024-05-20T20:44:25.277 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying lvm2 is present... 2024-05-20T20:44:25.277 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying time synchronization is in place... 2024-05-20T20:44:25.285 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-20T20:44:25.285 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-20T20:44:25.291 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-20T20:44:25.291 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-05-20T20:44:25.298 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-05-20T20:44:25.305 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-05-20T20:44:25.305 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-05-20T20:44:25.305 INFO:teuthology.orchestra.run.smithi026.stdout:Repeating the final host check... 2024-05-20T20:44:25.329 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 5.0.2 2024-05-20T20:44:25.329 INFO:teuthology.orchestra.run.smithi026.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-20T20:44:25.329 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl is present 2024-05-20T20:44:25.330 INFO:teuthology.orchestra.run.smithi026.stdout:lvcreate is present 2024-05-20T20:44:25.336 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-20T20:44:25.336 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-20T20:44:25.342 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-20T20:44:25.342 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-05-20T20:44:25.349 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-05-20T20:44:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-05-20T20:44:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-05-20T20:44:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:Host looks OK 2024-05-20T20:44:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:Cluster fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:Acquiring lock 140414289825312 on /run/cephadm/9006c62a-16e9-11ef-bc9a-c7b262605968.lock 2024-05-20T20:44:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:Lock 140414289825312 acquired on /run/cephadm/9006c62a-16e9-11ef-bc9a-c7b262605968.lock 2024-05-20T20:44:25.357 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 3300 ... 2024-05-20T20:44:25.357 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 6789 ... 2024-05-20T20:44:25.357 INFO:teuthology.orchestra.run.smithi026.stdout:Base mon IP(s) is [172.21.15.26:3300, 172.21.15.26:6789], mon addrv is [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-05-20T20:44:25.360 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.26 metric 100 2024-05-20T20:44:25.360 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.26 metric 100 2024-05-20T20:44:25.363 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-20T20:44:25.363 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-20T20:44:25.363 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-20T20:44:25.365 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-20T20:44:25.366 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-20T20:44:25.366 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-20T20:44:25.366 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-20T20:44:25.366 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-05-20T20:44:25.366 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-20T20:44:25.366 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-05-20T20:44:25.367 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-05-20T20:44:25.367 INFO:teuthology.orchestra.run.smithi026.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-20T20:44:25.367 INFO:teuthology.orchestra.run.smithi026.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-20T20:44:25.368 INFO:teuthology.orchestra.run.smithi026.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634... 2024-05-20T20:44:25.837 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 2024-05-20T20:44:25.838 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634... 2024-05-20T20:44:25.838 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Getting image source signatures 2024-05-20T20:44:25.838 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:9af6dedc48c4fca9f79c2db09a94f41f0f515ee6ded78101f148d319f313a254 2024-05-20T20:44:25.838 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-20T20:44:25.838 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying config sha256:559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 2024-05-20T20:44:25.838 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-20T20:44:26.701 INFO:teuthology.orchestra.run.smithi026.stdout:ceph: stdout ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev) 2024-05-20T20:44:26.701 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph version: ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev) 2024-05-20T20:44:26.701 INFO:teuthology.orchestra.run.smithi026.stdout:Extracting ceph user uid/gid from container image... 2024-05-20T20:44:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:stat: stdout 167 167 2024-05-20T20:44:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial keys... 2024-05-20T20:44:28.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQArtktmF90+ORAAOka+XOF8srrmYzzEWihU0g== 2024-05-20T20:44:29.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQAstktmkgyqMxAA7BIT/5IMo2bRpZ0u4dnllQ== 2024-05-20T20:44:30.499 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQAttktmj7KEMxAAo4dAPbRJRWQ7bsSGGMyqBw== 2024-05-20T20:44:30.499 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial monmap... 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool for a [v2:172.21.15.26:3300,v1:172.21.15.26:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: set fsid to 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:44:31.314 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mon... 2024-05-20T20:44:32.203 INFO:teuthology.orchestra.run.smithi026.stdout:create mon.a on 2024-05-20T20:44:32.417 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-20T20:44:32.570 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-20T20:44:32.746 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968.target. 2024-05-20T20:44:32.746 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968.target. 2024-05-20T20:44:32.982 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a 2024-05-20T20:44:32.982 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-20T20:44:33.157 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968.target.wants/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@.service. 2024-05-20T20:44:33.702 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T20:44:33.703 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-05-20T20:44:33.709 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T20:44:33.709 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-05-20T20:44:33.709 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-05-20T20:44:33.710 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-05-20T20:44:33.710 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon to start... 2024-05-20T20:44:33.710 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon... 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout cluster: 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout id: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout services: 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.572576s) 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:34.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout data: 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pgs: 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.136+0000 7f9b5845c700 1 Processor -- start 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.137+0000 7f9b5845c700 1 -- start start 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.137+0000 7f9b5845c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5007d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.137+0000 7f9b5845c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5007d860 con 0x7f9b5007ee30 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.137+0000 7f9b561f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5007d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.137+0000 7f9b561f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5007d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35630/0 (socket says 172.21.15.26:35630) 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.137+0000 7f9b561f8700 1 -- 172.21.15.26:0/2925855044 learned_addr learned my addr 172.21.15.26:0/2925855044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.138+0000 7f9b561f8700 1 -- 172.21.15.26:0/2925855044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5007d9e0 con 0x7f9b5007ee30 2024-05-20T20:44:34.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.138+0000 7f9b561f8700 1 --2- 172.21.15.26:0/2925855044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5007d290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098a0 tx=0x7f9b44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309726426445cbdd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.138+0000 7f9b551f6700 1 -- 172.21.15.26:0/2925855044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44004160 con 0x7f9b5007ee30 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.138+0000 7f9b551f6700 1 -- 172.21.15.26:0/2925855044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b44004300 con 0x7f9b5007ee30 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.138+0000 7f9b551f6700 1 -- 172.21.15.26:0/2925855044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b440036a0 con 0x7f9b5007ee30 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 -- 172.21.15.26:0/2925855044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 msgr2=0x7f9b5007d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 --2- 172.21.15.26:0/2925855044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5007d290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098a0 tx=0x7f9b44009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 -- 172.21.15.26:0/2925855044 shutdown_connections 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 --2- 172.21.15.26:0/2925855044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5007d290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 -- 172.21.15.26:0/2925855044 >> 172.21.15.26:0/2925855044 conn(0x7f9b501042b0 msgr2=0x7f9b50106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 -- 172.21.15.26:0/2925855044 shutdown_connections 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 -- 172.21.15.26:0/2925855044 wait complete. 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 Processor -- start 2024-05-20T20:44:34.780 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.139+0000 7f9b5845c700 1 -- start start 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b5845c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b5845c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5007df10 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b561f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b561f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35636/0 (socket says 172.21.15.26:35636) 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b561f8700 1 -- 172.21.15.26:0/1993607572 learned_addr learned my addr 172.21.15.26:0/1993607572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b561f8700 1 -- 172.21.15.26:0/1993607572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5019eb50 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b561f8700 1 --2- 172.21.15.26:0/1993607572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5019e610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9b44000c00 tx=0x7f9b44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44003db0 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b44011630 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5019ede0 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.140+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44011950 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.141+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a1ad0 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.141+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b44011af0 con 0x7f9b5007ee30 2024-05-20T20:44:34.781 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.141+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b4401c450 con 0x7f9b5007ee30 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.141+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b50064ba0 con 0x7f9b5007ee30 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.144+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9b44032020 con 0x7f9b5007ee30 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.219+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9b5002e770 con 0x7f9b5007ee30 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.219+0000 7f9b437fe700 1 -- 172.21.15.26:0/1993607572 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9b44004260 con 0x7f9b5007ee30 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 msgr2=0x7f9b5019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 --2- 172.21.15.26:0/1993607572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5019e610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9b44000c00 tx=0x7f9b44004750 comp rx=0 tx=0).stop 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 shutdown_connections 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 --2- 172.21.15.26:0/1993607572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5007ee30 0x7f9b5019e610 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 >> 172.21.15.26:0/1993607572 conn(0x7f9b501042b0 msgr2=0x7f9b50106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 shutdown_connections 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:34.220+0000 7f9b5845c700 1 -- 172.21.15.26:0/1993607572 wait complete. 2024-05-20T20:44:34.782 INFO:teuthology.orchestra.run.smithi026.stdout:mon is available 2024-05-20T20:44:34.783 INFO:teuthology.orchestra.run.smithi026.stdout:Assimilating anything we can from ceph.conf... 2024-05-20T20:44:35.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-20T20:44:35.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.204+0000 7f25e1f9b700 1 Processor -- start 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.204+0000 7f25e1f9b700 1 -- start start 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.204+0000 7f25e1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.204+0000 7f25e1f9b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc108620 con 0x7f25dc107c30 2024-05-20T20:44:35.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35638/0 (socket says 172.21.15.26:35638) 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25db7fe700 1 -- 172.21.15.26:0/2641851137 learned_addr learned my addr 172.21.15.26:0/2641851137 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25db7fe700 1 -- 172.21.15.26:0/2641851137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc108e30 con 0x7f25dc107c30 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25db7fe700 1 --2- 172.21.15.26:0/2641851137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc108050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0098a0 tx=0x7f25cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4aa7bc8b6b47de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25da7fc700 1 -- 172.21.15.26:0/2641851137 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25cc004160 con 0x7f25dc107c30 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25da7fc700 1 -- 172.21.15.26:0/2641851137 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc004300 con 0x7f25dc107c30 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.205+0000 7f25da7fc700 1 -- 172.21.15.26:0/2641851137 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25cc0036a0 con 0x7f25dc107c30 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 -- 172.21.15.26:0/2641851137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 msgr2=0x7f25dc108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 --2- 172.21.15.26:0/2641851137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc108050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0098a0 tx=0x7f25cc009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 -- 172.21.15.26:0/2641851137 shutdown_connections 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 --2- 172.21.15.26:0/2641851137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc108050 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 -- 172.21.15.26:0/2641851137 >> 172.21.15.26:0/2641851137 conn(0x7f25dc1031f0 msgr2=0x7f25dc105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:35.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 -- 172.21.15.26:0/2641851137 shutdown_connections 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.206+0000 7f25e1f9b700 1 -- 172.21.15.26:0/2641851137 wait complete. 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25e1f9b700 1 Processor -- start 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25e1f9b700 1 -- start start 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25e1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25e1f9b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc109360 con 0x7f25dc107c30 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35640/0 (socket says 172.21.15.26:35640) 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25db7fe700 1 -- 172.21.15.26:0/4264312741 learned_addr learned my addr 172.21.15.26:0/4264312741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.207+0000 7f25db7fe700 1 -- 172.21.15.26:0/4264312741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc19c9d0 con 0x7f25dc107c30 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25db7fe700 1 --2- 172.21.15.26:0/4264312741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc19c490 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f25cc000c00 tx=0x7f25cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25cc003db0 con 0x7f25dc107c30 2024-05-20T20:44:35.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc011630 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc19cc60 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25cc011950 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc19f950 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f25cc01a460 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.208+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25cc01b780 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.209+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc064ba0 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.211+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f25cc031020 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.283+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f25dc02e860 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.286+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f25cc01b9d0 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.287+0000 7f25d8ff9700 1 -- 172.21.15.26:0/4264312741 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f25cc01ad00 con 0x7f25dc107c30 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.287+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 msgr2=0x7f25dc19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:35.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.287+0000 7f25e1f9b700 1 --2- 172.21.15.26:0/4264312741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc19c490 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f25cc000c00 tx=0x7f25cc004750 comp rx=0 tx=0).stop 2024-05-20T20:44:35.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.288+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 shutdown_connections 2024-05-20T20:44:35.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.288+0000 7f25e1f9b700 1 --2- 172.21.15.26:0/4264312741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25dc107c30 0x7f25dc19c490 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:35.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.288+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 >> 172.21.15.26:0/4264312741 conn(0x7f25dc1031f0 msgr2=0x7f25dc193070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:35.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.288+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 shutdown_connections 2024-05-20T20:44:35.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:35.288+0000 7f25e1f9b700 1 -- 172.21.15.26:0/4264312741 wait complete. 2024-05-20T20:44:35.823 INFO:teuthology.orchestra.run.smithi026.stdout:Generating new minimal ceph.conf... 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.265+0000 7f007d560700 1 Processor -- start 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.265+0000 7f007d560700 1 -- start start 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f007d560700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f007807d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f007d560700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f007807d8a0 con 0x7f007807ee70 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0076ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f007807d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0076ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f007807d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35650/0 (socket says 172.21.15.26:35650) 2024-05-20T20:44:36.881 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0076ffd700 1 -- 172.21.15.26:0/236319009 learned_addr learned my addr 172.21.15.26:0/236319009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0076ffd700 1 -- 172.21.15.26:0/236319009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007807da20 con 0x7f007807ee70 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0076ffd700 1 --2- 172.21.15.26:0/236319009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f007807d2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f00680098a0 tx=0x7f0068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efa3e18621edac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0075ffb700 1 -- 172.21.15.26:0/236319009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068004160 con 0x7f007807ee70 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.266+0000 7f0075ffb700 1 -- 172.21.15.26:0/236319009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0068004300 con 0x7f007807ee70 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f0075ffb700 1 -- 172.21.15.26:0/236319009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00680119e0 con 0x7f007807ee70 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 -- 172.21.15.26:0/236319009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 msgr2=0x7f007807d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 --2- 172.21.15.26:0/236319009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f007807d2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f00680098a0 tx=0x7f0068009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 -- 172.21.15.26:0/236319009 shutdown_connections 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 --2- 172.21.15.26:0/236319009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f007807d2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 -- 172.21.15.26:0/236319009 >> 172.21.15.26:0/236319009 conn(0x7f0078104310 msgr2=0x7f0078106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 -- 172.21.15.26:0/236319009 shutdown_connections 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.267+0000 7f007d560700 1 -- 172.21.15.26:0/236319009 wait complete. 2024-05-20T20:44:36.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f007d560700 1 Processor -- start 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f007d560700 1 -- start start 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f007d560700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f00781a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f007d560700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f007810aea0 con 0x7f007807ee70 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f0076ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f00781a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f0076ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f00781a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35662/0 (socket says 172.21.15.26:35662) 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.268+0000 7f0076ffd700 1 -- 172.21.15.26:0/864628517 learned_addr learned my addr 172.21.15.26:0/864628517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f0076ffd700 1 -- 172.21.15.26:0/864628517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00781a36b0 con 0x7f007807ee70 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f0076ffd700 1 --2- 172.21.15.26:0/864628517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f00781a3170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00680119b0 tx=0x7f0068011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068003af0 con 0x7f007807ee70 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00781a3940 con 0x7f007807ee70 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f006801a8a0 con 0x7f007807ee70 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068024830 con 0x7f007807ee70 2024-05-20T20:44:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00781a6630 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0068022020 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.269+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f006802ddf0 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.270+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0078064ba0 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.272+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0068042ec0 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.344+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f00780008d0 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.344+0000 7f005ffff700 1 -- 172.21.15.26:0/864628517 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0068031020 con 0x7f007807ee70 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 msgr2=0x7f00781a3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 --2- 172.21.15.26:0/864628517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f00781a3170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00680119b0 tx=0x7f0068011a90 comp rx=0 tx=0).stop 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 shutdown_connections 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 --2- 172.21.15.26:0/864628517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f007807ee70 0x7f00781a3170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 >> 172.21.15.26:0/864628517 conn(0x7f0078104310 msgr2=0x7f007819a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 shutdown_connections 2024-05-20T20:44:36.885 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:36.345+0000 7f007d560700 1 -- 172.21.15.26:0/864628517 wait complete. 2024-05-20T20:44:36.885 INFO:teuthology.orchestra.run.smithi026.stdout:Restarting the monitor... 2024-05-20T20:44:36.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:36 smithi026 systemd[1]: Stopping Ceph mon.a for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:44:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a[31613]: 2024-05-20T20:44:37.152+0000 7fc2ddf54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T20:44:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a[31613]: 2024-05-20T20:44:37.152+0000 7fc2ddf54700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-20T20:44:37.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:37 smithi026 podman[31865]: 2024-05-20 20:44:37.337040922 +0000 UTC m=+0.286744931 container died 4bfe46463762eaec7dc853113f8f152c150eda487c47c03fd714909b5dd208da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T20:44:37.861 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:37 smithi026 podman[31865]: 2024-05-20 20:44:37.677242672 +0000 UTC m=+0.626946682 container cleanup 4bfe46463762eaec7dc853113f8f152c150eda487c47c03fd714909b5dd208da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-20T20:44:37.861 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:37 smithi026 bash[31865]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a 2024-05-20T20:44:38.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:37 smithi026 podman[31879]: 2024-05-20 20:44:37.861418275 +0000 UTC m=+0.514218377 container remove 4bfe46463762eaec7dc853113f8f152c150eda487c47c03fd714909b5dd208da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-20T20:44:38.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:38 smithi026 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-20T20:44:38.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:38 smithi026 systemd[1]: Stopped Ceph mon.a for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:44:38.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:38 smithi026 systemd[1]: Starting Ceph mon.a for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:44:38.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:38 smithi026 podman[31971]: 2024-05-20 20:44:38.312168395 +0000 UTC m=+0.100640291 container create 1dab537c29f14448c9792f303cfb52938d4dedd19b6662773eeb19f02f193ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD) 2024-05-20T20:44:38.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:38 smithi026 podman[31971]: 2024-05-20 20:44:38.228784463 +0000 UTC m=+0.017256376 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:44:39.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 podman[31971]: 2024-05-20 20:44:39.652441141 +0000 UTC m=+1.440913039 container init 1dab537c29f14448c9792f303cfb52938d4dedd19b6662773eeb19f02f193ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 podman[31971]: 2024-05-20 20:44:39.656869476 +0000 UTC m=+1.445341373 container start 1dab537c29f14448c9792f303cfb52938d4dedd19b6662773eeb19f02f193ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: set uid:gid to 167:167 (ceph:ceph) 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev), process ceph-mon, pid 2 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: pidfile_write: ignore empty --pid-file 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: load: jerasure load: lrc 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: RocksDB version: 7.9.2 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Git sha 0 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Compile date 2024-05-20 17:11:59 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: DB SUMMARY 2024-05-20T20:44:39.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: DB Session ID: HYHJJMB63MGXK6VXGNWJ 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: CURRENT file: CURRENT 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: IDENTITY file: IDENTITY 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104132 ; 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.error_if_exists: 0 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.create_if_missing: 0 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.paranoid_checks: 1 2024-05-20T20:44:39.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.env: 0x56124004bcc0 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.fs: PosixFileSystem 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.info_log: 0x561241735da0 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_file_opening_threads: 16 2024-05-20T20:44:39.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.statistics: (nil) 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.use_fsync: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_log_file_size: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.keep_log_file_num: 1000 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.recycle_log_file_num: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.allow_fallocate: 1 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.allow_mmap_reads: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.allow_mmap_writes: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.use_direct_reads: 0 2024-05-20T20:44:39.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.create_missing_column_families: 0 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.db_log_dir: 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.wal_dir: 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-20T20:44:39.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.advise_random_on_open: 1 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.db_write_buffer_size: 0 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.write_buffer_manager: 0x5612417070e0 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.rate_limiter: (nil) 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.wal_recovery_mode: 2 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.enable_thread_tracking: 0 2024-05-20T20:44:39.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.enable_pipelined_write: 0 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.unordered_write: 0 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.row_cache: None 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.wal_filter: None 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.allow_ingest_behind: 0 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.two_write_queues: 0 2024-05-20T20:44:39.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.manual_wal_flush: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.wal_compression: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.atomic_flush: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.log_readahead_size: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.best_efforts_recovery: 0 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-20T20:44:39.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.allow_data_in_errors: 0 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.db_host_id: __hostname__ 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_background_jobs: 2 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_background_compactions: -1 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_subcompactions: 1 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_total_wal_size: 0 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-20T20:44:39.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_open_files: -1 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bytes_per_sync: 0 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_readahead_size: 0 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_background_flushes: -1 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Compression algorithms supported: 2024-05-20T20:44:39.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kZSTD supported: 0 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kXpressCompression supported: 0 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kLZ4HCCompression supported: 1 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kZlibCompression supported: 1 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kSnappyCompression supported: 1 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kLZ4Compression supported: 1 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: kBZip2Compression supported: 0 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-20T20:44:39.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.merge_operator: 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_filter: None 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_filter_factory: None 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.sst_partitioner_factory: None 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561241735f60) 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks: 1 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_top_level_index_and_filter: 1 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_type: 0 2024-05-20T20:44:39.965 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_index_type: 0 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_shortening: 1 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: checksum: 4 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: no_block_cache: 0 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache: 0x5612417391f0 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_name: BinnedLRUCache 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_options: 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: capacity : 536870912 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_shard_bits : 4 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: strict_capacity_limit : 0 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: high_pri_pool_ratio: 0.000 2024-05-20T20:44:39.966 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_compressed: (nil) 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: persistent_cache: (nil) 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size: 4096 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size_deviation: 10 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_restart_interval: 16 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_block_restart_interval: 1 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: metadata_block_size: 4096 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: partition_filters: 0 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: use_delta_encoding: 1 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: filter_policy: bloomfilter 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: whole_key_filtering: 1 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: verify_compression: 0 2024-05-20T20:44:39.967 INFO:journalctl@ceph.mon.a.smithi026.stdout: read_amp_bytes_per_bit: 0 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: format_version: 5 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_index_compression: 1 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_align: 0 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: max_auto_readahead_size: 262144 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: prepopulate_block_cache: 0 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: initial_auto_readahead_size: 8192 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.write_buffer_size: 33554432 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_write_buffer_number: 2 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression: NoCompression 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression: Disabled 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.prefix_extractor: nullptr 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-20T20:44:39.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.num_levels: 7 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-20T20:44:39.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.level: 32767 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.strategy: 0 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.enabled: false 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-20T20:44:39.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.target_file_size_base: 67108864 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-20T20:44:39.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.arena_block_size: 1048576 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.disable_auto_compactions: 0 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-20T20:44:39.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.inplace_update_support: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.bloom_locality: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.max_successive_merges: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.paranoid_file_checks: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.force_consistency_checks: 1 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.report_bg_io_stats: 0 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.ttl: 2592000 2024-05-20T20:44:39.973 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.enable_blob_files: false 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.min_blob_size: 0 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.blob_file_size: 268435456 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-20T20:44:39.974 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.blob_file_starting_level: 0 2024-05-20T20:44:39.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-20T20:44:39.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-20T20:44:39.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-20T20:44:39.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2cf81537-c163-41cc-a7b3-c05f14c53775 2024-05-20T20:44:39.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716237879692214, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-20T20:44:39.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-20T20:44:39.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716237879693503, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97314, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13923, "raw_average_key_size": 49, "raw_value_size": 89694, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716237879, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2cf81537-c163-41cc-a7b3-c05f14c53775", "db_session_id": "HYHJJMB63MGXK6VXGNWJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-20T20:44:39.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716237879693607, "job": 1, "event": "recovery_finished"} 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561241856000 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: DB pointer 0x561241842000 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** DB Stats ** 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-20T20:44:39.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: L0 2/0 98.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: Sum 2/0 98.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-05-20T20:44:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-20T20:44:39.980 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative compaction: 0.00 GB write, 24.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval compaction: 0.00 GB write, 24.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache BinnedLRUCache@0x5612417391f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: starting mon.a rank 0 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???) e1 preinit fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).mds e1 new map 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).mds e1 print_map 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: e1 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T20:44:39.981 INFO:journalctl@ceph.mon.a.smithi026.stdout: legacy client fscid: -1 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout: No filesystems configured 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: monmap epoch 1 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:39.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: last_changed 2024-05-20T20:44:30.783372+0000 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: created 2024-05-20T20:44:30.783372+0000 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: min_mon_release 19 (squid) 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: election_strategy: 1 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: fsmap 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: osdmap e1: 0 total, 0 up, 0 in 2024-05-20T20:44:39.983 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:39 smithi026 ceph-mon[31990]: mgrmap e1: no daemons active 2024-05-20T20:44:40.688 INFO:teuthology.orchestra.run.smithi026.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-20T20:44:40.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:40 smithi026 bash[31971]: 1dab537c29f14448c9792f303cfb52938d4dedd19b6662773eeb19f02f193ea3 2024-05-20T20:44:40.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:40 smithi026 systemd[1]: Started Ceph mon.a for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:44:44.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:44 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1834945190' entity='client.admin' 2024-05-20T20:44:48.348 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.614+0000 7fee61895700 1 Processor -- start 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.614+0000 7fee61895700 1 -- start start 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.614+0000 7fee61895700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c10bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.614+0000 7fee61895700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c07ce00 con 0x7fee5c1097f0 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee5affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c10bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee5affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c10bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44852/0 (socket says 172.21.15.26:44852) 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee5affd700 1 -- 172.21.15.26:0/3131548697 learned_addr learned my addr 172.21.15.26:0/3131548697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee5affd700 1 -- 172.21.15.26:0/3131548697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c10c120 con 0x7fee5c1097f0 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee5affd700 1 --2- 172.21.15.26:0/3131548697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c10bbe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0098a0 tx=0x7fee4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8951a8cc7afdc91 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee59ffb700 1 -- 172.21.15.26:0/3131548697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee4c004160 con 0x7fee5c1097f0 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee59ffb700 1 -- 172.21.15.26:0/3131548697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fee4c004300 con 0x7fee5c1097f0 2024-05-20T20:44:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.615+0000 7fee59ffb700 1 -- 172.21.15.26:0/3131548697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee4c0119e0 con 0x7fee5c1097f0 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 -- 172.21.15.26:0/3131548697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 msgr2=0x7fee5c10bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 --2- 172.21.15.26:0/3131548697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c10bbe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0098a0 tx=0x7fee4c009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 -- 172.21.15.26:0/3131548697 shutdown_connections 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 --2- 172.21.15.26:0/3131548697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c10bbe0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 -- 172.21.15.26:0/3131548697 >> 172.21.15.26:0/3131548697 conn(0x7fee5c1031a0 msgr2=0x7fee5c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 -- 172.21.15.26:0/3131548697 shutdown_connections 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.616+0000 7fee61895700 1 -- 172.21.15.26:0/3131548697 wait complete. 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee61895700 1 Processor -- start 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee61895700 1 -- start start 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee61895700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee61895700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c10d080 con 0x7fee5c1097f0 2024-05-20T20:44:48.350 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee5affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee5affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c19cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44856/0 (socket says 172.21.15.26:44856) 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee5affd700 1 -- 172.21.15.26:0/1834945190 learned_addr learned my addr 172.21.15.26:0/1834945190 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.617+0000 7fee5affd700 1 -- 172.21.15.26:0/1834945190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c19d080 con 0x7fee5c1097f0 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee5affd700 1 --2- 172.21.15.26:0/1834945190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c19cb40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009130 tx=0x7fee4c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee4c003b20 con 0x7fee5c1097f0 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c19d310 con 0x7fee5c1097f0 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fee4c01a8a0 con 0x7fee5c1097f0 2024-05-20T20:44:48.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee4c024810 con 0x7fee5c1097f0 2024-05-20T20:44:48.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1a0000 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee4c022020 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.618+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee4c0193c0 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.619+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c064ba0 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.621+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fee4c02dde0 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.691+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fee5c1a01d0 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.693+0000 7fee60893700 1 -- 172.21.15.26:0/1834945190 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fee4c01ad70 con 0x7fee5c1097f0 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 msgr2=0x7fee5c19cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 --2- 172.21.15.26:0/1834945190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c19cb40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009130 tx=0x7fee4c011a60 comp rx=0 tx=0).stop 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 shutdown_connections 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 --2- 172.21.15.26:0/1834945190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1097f0 0x7fee5c19cb40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 >> 172.21.15.26:0/1834945190 conn(0x7fee5c1031a0 msgr2=0x7fee5c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:48.354 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 shutdown_connections 2024-05-20T20:44:48.355 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:43.694+0000 7fee61895700 1 -- 172.21.15.26:0/1834945190 wait complete. 2024-05-20T20:44:48.355 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-20T20:44:48.377 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-20T20:44:48.377 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mgr... 2024-05-20T20:44:48.378 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-20T20:44:48.378 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-20T20:44:48.645 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x 2024-05-20T20:44:48.646 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-20T20:44:48.802 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968.target.wants/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@.service. 2024-05-20T20:44:48.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:48 smithi026 systemd[1]: Starting Ceph mgr.x for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:44:49.295 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 podman[32205]: 2024-05-20 20:44:49.018620936 +0000 UTC m=+0.092717132 container create 42c0ddaaf231c8bc23c510eb995f5bc23414f72c677e3264a77c2c8a318bdd89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T20:44:49.295 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 podman[32205]: 2024-05-20 20:44:48.943568576 +0000 UTC m=+0.017664730 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:44:49.295 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 podman[32205]: 2024-05-20 20:44:49.21264468 +0000 UTC m=+0.286740827 container init 42c0ddaaf231c8bc23c510eb995f5bc23414f72c677e3264a77c2c8a318bdd89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-20T20:44:49.295 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 podman[32205]: 2024-05-20 20:44:49.215505876 +0000 UTC m=+0.289602033 container start 42c0ddaaf231c8bc23c510eb995f5bc23414f72c677e3264a77c2c8a318bdd89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=) 2024-05-20T20:44:49.295 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 bash[32205]: 42c0ddaaf231c8bc23c510eb995f5bc23414f72c677e3264a77c2c8a318bdd89 2024-05-20T20:44:49.309 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T20:44:49.309 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-05-20T20:44:49.317 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T20:44:49.317 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-05-20T20:44:49.317 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-05-20T20:44:49.317 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-05-20T20:44:49.325 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T20:44:49.325 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-05-20T20:44:49.333 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T20:44:49.334 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-05-20T20:44:49.334 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-05-20T20:44:49.334 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-20T20:44:49.334 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr to start... 2024-05-20T20:44:49.334 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr... 2024-05-20T20:44:49.570 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 systemd[1]: Started Ceph mgr.x for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:44:49.570 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:49.376+0000 7f091c0b2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T20:44:49.869 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:49.573+0000 7f091c0b2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:44:49.869 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:49.683+0000 7f091c0b2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T20:44:50.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:49 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2701992742' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-20T20:44:50.135 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:50 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:50.020+0000 7f091c0b2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T20:44:50.135 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:50 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:50.112+0000 7f091c0b2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T20:44:50.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-05-20T20:44:50.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-20T20:44:50.416 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-20T20:44:50.417 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-20T20:44:50.424 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-20T20:44:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-05-20T20:44:33.648660+0000", 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-20T20:44:50.426 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.785+0000 7fb8e381a700 1 Processor -- start 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.785+0000 7fb8e381a700 1 -- start start 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.785+0000 7fb8e381a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc109610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.785+0000 7fb8e381a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc109be0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.786+0000 7fb8e15b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.786+0000 7fb8e15b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59018/0 (socket says 172.21.15.26:59018) 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.786+0000 7fb8e15b6700 1 -- 172.21.15.26:0/503781188 learned_addr learned my addr 172.21.15.26:0/503781188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.786+0000 7fb8e15b6700 1 -- 172.21.15.26:0/503781188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc107e50 con 0x7fb8dc1091f0 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.786+0000 7fb8e15b6700 1 --2- 172.21.15.26:0/503781188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc109610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80098a0 tx=0x7fb8d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936c437183d0f612 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8cffff700 1 -- 172.21.15.26:0/503781188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8d8004160 con 0x7fb8dc1091f0 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8cffff700 1 -- 172.21.15.26:0/503781188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb8d8004300 con 0x7fb8dc1091f0 2024-05-20T20:44:50.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8cffff700 1 -- 172.21.15.26:0/503781188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8d80119e0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8e381a700 1 -- 172.21.15.26:0/503781188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 msgr2=0x7fb8dc109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8e381a700 1 --2- 172.21.15.26:0/503781188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc109610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80098a0 tx=0x7fb8d8009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8e381a700 1 -- 172.21.15.26:0/503781188 shutdown_connections 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8e381a700 1 --2- 172.21.15.26:0/503781188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc109610 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.787+0000 7fb8e381a700 1 -- 172.21.15.26:0/503781188 >> 172.21.15.26:0/503781188 conn(0x7fb8dc102960 msgr2=0x7fb8dc104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.788+0000 7fb8e381a700 1 -- 172.21.15.26:0/503781188 shutdown_connections 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.788+0000 7fb8e381a700 1 -- 172.21.15.26:0/503781188 wait complete. 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.788+0000 7fb8e381a700 1 Processor -- start 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.788+0000 7fb8e381a700 1 -- start start 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.788+0000 7fb8e381a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.788+0000 7fb8e381a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc10c090 con 0x7fb8dc1091f0 2024-05-20T20:44:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e15b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e15b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59032/0 (socket says 172.21.15.26:59032) 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e15b6700 1 -- 172.21.15.26:0/2701992742 learned_addr learned my addr 172.21.15.26:0/2701992742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e15b6700 1 -- 172.21.15.26:0/2701992742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc19d140 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e15b6700 1 --2- 172.21.15.26:0/2701992742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc19cc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80119e0 tx=0x7fb8d80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8d8003bf0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb8d801a8a0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc19d3d0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8d80248c0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.789+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1a00c0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.790+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb8d8022020 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.790+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb8d802c3a0 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.790+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc10c210 con 0x7fb8dc1091f0 2024-05-20T20:44:50.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.792+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb8d8004300 con 0x7fb8dc1091f0 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.868+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb8dc02e760 con 0x7fb8dc1091f0 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.868+0000 7fb8ce7fc700 1 -- 172.21.15.26:0/2701992742 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fb8d801a430 con 0x7fb8dc1091f0 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.869+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 msgr2=0x7fb8dc19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.869+0000 7fb8e381a700 1 --2- 172.21.15.26:0/2701992742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc19cc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80119e0 tx=0x7fb8d80046a0 comp rx=0 tx=0).stop 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.870+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 shutdown_connections 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.870+0000 7fb8e381a700 1 --2- 172.21.15.26:0/2701992742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc1091f0 0x7fb8dc19cc00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.870+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 >> 172.21.15.26:0/2701992742 conn(0x7fb8dc102960 msgr2=0x7fb8dc1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.870+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 shutdown_connections 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:49.870+0000 7fb8e381a700 1 -- 172.21.15.26:0/2701992742 wait complete. 2024-05-20T20:44:50.430 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (1/15)... 2024-05-20T20:44:51.702 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:51.386+0000 7f091c0b2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T20:44:51.702 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:51.563+0000 7f091c0b2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T20:44:52.041 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:51.740+0000 7f091c0b2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T20:44:52.041 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:52.041+0000 7f091c0b2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T20:44:52.349 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:52.137+0000 7f091c0b2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T20:44:52.349 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:52.349+0000 7f091c0b2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T20:44:52.661 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:52.561+0000 7f091c0b2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:44:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3128422289' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-20T20:44:53.546 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:53.217+0000 7f091c0b2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T20:44:53.546 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:53.447+0000 7f091c0b2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-20T20:44:54.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:53.890+0000 7f091c0b2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T20:44:54.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:53.990+0000 7f091c0b2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T20:44:54.490 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:54.381+0000 7f091c0b2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T20:44:55.123 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:54.767+0000 7f091c0b2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T20:44:55.123 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:54.865+0000 7f091c0b2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-20T20:44:55.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:55.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-20T20:44:55.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-20T20:44:55.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-20T20:44:55.138 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-20T20:44:55.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-20T20:44:55.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-05-20T20:44:33.648660+0000", 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.848+0000 7fd95722a700 1 Processor -- start 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.848+0000 7fd95722a700 1 -- start start 2024-05-20T20:44:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd95722a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd950107a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd95722a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd95007cde0 con 0x7fd9501075f0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd954fc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd950107a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd954fc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd950107a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59034/0 (socket says 172.21.15.26:59034) 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd954fc6700 1 -- 172.21.15.26:0/3490179887 learned_addr learned my addr 172.21.15.26:0/3490179887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd954fc6700 1 -- 172.21.15.26:0/3490179887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd950107f50 con 0x7fd9501075f0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd954fc6700 1 --2- 172.21.15.26:0/3490179887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd950107a10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9440098a0 tx=0x7fd944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eec98e102d4cab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.849+0000 7fd94f7fe700 1 -- 172.21.15.26:0/3490179887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd944004160 con 0x7fd9501075f0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd94f7fe700 1 -- 172.21.15.26:0/3490179887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd944004300 con 0x7fd9501075f0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd94f7fe700 1 -- 172.21.15.26:0/3490179887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9440119e0 con 0x7fd9501075f0 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 -- 172.21.15.26:0/3490179887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 msgr2=0x7fd950107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 --2- 172.21.15.26:0/3490179887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd950107a10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9440098a0 tx=0x7fd944009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 -- 172.21.15.26:0/3490179887 shutdown_connections 2024-05-20T20:44:55.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 --2- 172.21.15.26:0/3490179887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd950107a10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 -- 172.21.15.26:0/3490179887 >> 172.21.15.26:0/3490179887 conn(0x7fd9501031f0 msgr2=0x7fd950105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 -- 172.21.15.26:0/3490179887 shutdown_connections 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.850+0000 7fd95722a700 1 -- 172.21.15.26:0/3490179887 wait complete. 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd95722a700 1 Processor -- start 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd95722a700 1 -- start start 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd95722a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd95019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd95722a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd950108f20 con 0x7fd9501075f0 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd954fc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd95019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd954fc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd95019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59036/0 (socket says 172.21.15.26:59036) 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd954fc6700 1 -- 172.21.15.26:0/3128422289 learned_addr learned my addr 172.21.15.26:0/3128422289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.851+0000 7fd954fc6700 1 -- 172.21.15.26:0/3128422289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95019f310 con 0x7fd9501075f0 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd954fc6700 1 --2- 172.21.15.26:0/3128422289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd95019edd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9440116a0 tx=0x7fd9440046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:55.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd944003b70 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd94401a8a0 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd944024890 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95019f5a0 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a2290 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd944022020 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.852+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd9440193c0 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.853+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950064ba0 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.855+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd9440094f0 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.931+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd9500008d0 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.931+0000 7fd94dffb700 1 -- 172.21.15.26:0/3128422289 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd94401aa50 con 0x7fd9501075f0 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 msgr2=0x7fd95019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 --2- 172.21.15.26:0/3128422289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd95019edd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9440116a0 tx=0x7fd9440046a0 comp rx=0 tx=0).stop 2024-05-20T20:44:55.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 shutdown_connections 2024-05-20T20:44:55.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 --2- 172.21.15.26:0/3128422289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9501075f0 0x7fd95019edd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:55.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 >> 172.21.15.26:0/3128422289 conn(0x7fd9501031f0 msgr2=0x7fd950195a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:55.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 shutdown_connections 2024-05-20T20:44:55.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:52.932+0000 7fd95722a700 1 -- 172.21.15.26:0/3128422289 wait complete. 2024-05-20T20:44:55.145 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (2/15)... 2024-05-20T20:44:55.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:55.122+0000 7f091c0b2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T20:44:55.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:55.267+0000 7f091c0b2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T20:44:55.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:55.368+0000 7f091c0b2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T20:44:55.807 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:55.711+0000 7f091c0b2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T20:44:55.807 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:55.807+0000 7f091c0b2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T20:44:56.702 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:56.442+0000 7f091c0b2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T20:44:56.702 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:44:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:44:56.667+0000 7f091c0b2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T20:44:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: Activating manager daemon x 2024-05-20T20:44:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: mgrmap e2: x(active, starting, since 0.0033065s) 2024-05-20T20:44:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-20T20:44:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: Manager daemon x is now available 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' 2024-05-20T20:44:57.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:56 smithi026 ceph-mon[31990]: from='mgr.14100 172.21.15.26:0/432826367' entity='mgr.x' 2024-05-20T20:44:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-05-20T20:44:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-20T20:44:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-20T20:44:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-20T20:44:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-20T20:44:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-05-20T20:44:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-05-20T20:44:33.648660+0000", 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.595+0000 7f52efd27700 1 Processor -- start 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.595+0000 7f52efd27700 1 -- start start 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.595+0000 7f52efd27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e8109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.595+0000 7f52efd27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52e8109760 con 0x7f52e8108d70 2024-05-20T20:44:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.596+0000 7f52edac3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e8109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.596+0000 7f52edac3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e8109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59114/0 (socket says 172.21.15.26:59114) 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.596+0000 7f52edac3700 1 -- 172.21.15.26:0/2493643425 learned_addr learned my addr 172.21.15.26:0/2493643425 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.596+0000 7f52edac3700 1 -- 172.21.15.26:0/2493643425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e8109f20 con 0x7f52e8108d70 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.596+0000 7f52edac3700 1 --2- 172.21.15.26:0/2493643425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e8109190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009e80 tx=0x7f52e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813c8a0456224143 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52ecac1700 1 -- 172.21.15.26:0/2493643425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e4017070 con 0x7f52e8108d70 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52ecac1700 1 -- 172.21.15.26:0/2493643425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f52e400b8c0 con 0x7f52e8108d70 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52ecac1700 1 -- 172.21.15.26:0/2493643425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e4004420 con 0x7f52e8108d70 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 -- 172.21.15.26:0/2493643425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 msgr2=0x7f52e8109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 --2- 172.21.15.26:0/2493643425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e8109190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009e80 tx=0x7f52e400b330 comp rx=0 tx=0).stop 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 -- 172.21.15.26:0/2493643425 shutdown_connections 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 --2- 172.21.15.26:0/2493643425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e8109190 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 -- 172.21.15.26:0/2493643425 >> 172.21.15.26:0/2493643425 conn(0x7f52e81042d0 msgr2=0x7f52e8106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 -- 172.21.15.26:0/2493643425 shutdown_connections 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.597+0000 7f52efd27700 1 -- 172.21.15.26:0/2493643425 wait complete. 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52efd27700 1 Processor -- start 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52efd27700 1 -- start start 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52efd27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52efd27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52e810d150 con 0x7f52e8108d70 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52edac3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52edac3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59126/0 (socket says 172.21.15.26:59126) 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.598+0000 7f52edac3700 1 -- 172.21.15.26:0/2309490955 learned_addr learned my addr 172.21.15.26:0/2309490955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52edac3700 1 -- 172.21.15.26:0/2309490955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e819d120 con 0x7f52e8108d70 2024-05-20T20:44:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52edac3700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e819cbe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009e50 tx=0x7f52e400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e4020020 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f52e4012430 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e4017040 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e819d3b0 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.599+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e81a00a0 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.600+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 2) ==== 49978+0+0 (secure 0 0 0) 0x7f52e4023020 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.600+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e8064ba0 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.600+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52e400f050 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.602+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f52e402a020 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.673+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7f52e40125e0 con 0x7f52e8108d70 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.673+0000 7f52deffd700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f52d403eef0 0x7f52d40412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.673+0000 7f52ed2c2700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f52d403eef0 0x7f52d40412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.675+0000 7f52ed2c2700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f52d403eef0 0x7f52d40412e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f52d8006fd0 tx=0x7f52d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.680+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52e81a0270 con 0x7f52e8108d70 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.680+0000 7f52deffd700 1 -- 172.21.15.26:0/2309490955 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f52e81a0270 con 0x7f52e8108d70 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f52d403eef0 msgr2=0x7f52d40412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f52d403eef0 0x7f52d40412e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f52d8006fd0 tx=0x7f52d8006e40 comp rx=0 tx=0).stop 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 msgr2=0x7f52e819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e819cbe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009e50 tx=0x7f52e400bd50 comp rx=0 tx=0).stop 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 shutdown_connections 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f52d403eef0 0x7f52d40412e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 --2- 172.21.15.26:0/2309490955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e8108d70 0x7f52e819cbe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 >> 172.21.15.26:0/2309490955 conn(0x7f52e81042d0 msgr2=0x7f52e8104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.681+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 shutdown_connections 2024-05-20T20:44:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:57.682+0000 7f52efd27700 1 -- 172.21.15.26:0/2309490955 wait complete. 2024-05-20T20:44:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:mgr is available 2024-05-20T20:44:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:58 smithi026 ceph-mon[31990]: mgrmap e3: x(active, since 1.00684s) 2024-05-20T20:44:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:58 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2309490955' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-20T20:44:59.553 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f554dc57700 1 Processor -- start 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f554dc57700 1 -- start start 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f554dc57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554807d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f554dc57700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f554807d860 con 0x7f554807ee30 2024-05-20T20:44:59.554 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f55477fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f55477fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554807d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59130/0 (socket says 172.21.15.26:59130) 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f55477fe700 1 -- 172.21.15.26:0/70374542 learned_addr learned my addr 172.21.15.26:0/70374542 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.790+0000 7f55477fe700 1 -- 172.21.15.26:0/70374542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554807d9e0 con 0x7f554807ee30 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f55477fe700 1 --2- 172.21.15.26:0/70374542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554807d290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55380098a0 tx=0x7f5538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c460c87bca94801f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f5546ffd700 1 -- 172.21.15.26:0/70374542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5538004160 con 0x7f554807ee30 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f5546ffd700 1 -- 172.21.15.26:0/70374542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5538004300 con 0x7f554807ee30 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f5546ffd700 1 -- 172.21.15.26:0/70374542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55380119e0 con 0x7f554807ee30 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 -- 172.21.15.26:0/70374542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 msgr2=0x7f554807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 --2- 172.21.15.26:0/70374542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554807d290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55380098a0 tx=0x7f5538009bb0 comp rx=0 tx=0).stop 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 -- 172.21.15.26:0/70374542 shutdown_connections 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 --2- 172.21.15.26:0/70374542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554807d290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 -- 172.21.15.26:0/70374542 >> 172.21.15.26:0/70374542 conn(0x7f55481042b0 msgr2=0x7f55481066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 -- 172.21.15.26:0/70374542 shutdown_connections 2024-05-20T20:44:59.555 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.791+0000 7f554dc57700 1 -- 172.21.15.26:0/70374542 wait complete. 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.792+0000 7f554dc57700 1 Processor -- start 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.792+0000 7f554dc57700 1 -- start start 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.792+0000 7f554dc57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.792+0000 7f554dc57700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f554810f260 con 0x7f554807ee30 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.792+0000 7f55477fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55477fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59132/0 (socket says 172.21.15.26:59132) 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55477fe700 1 -- 172.21.15.26:0/4136450100 learned_addr learned my addr 172.21.15.26:0/4136450100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55477fe700 1 -- 172.21.15.26:0/4136450100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554819f2a0 con 0x7f554807ee30 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55477fe700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554819ed60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f55380116a0 tx=0x7f5538011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55380039f0 con 0x7f554807ee30 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554819f530 con 0x7f554807ee30 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5538003b90 con 0x7f554807ee30 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f553801a600 con 0x7f554807ee30 2024-05-20T20:44:59.556 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.793+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55481a2220 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.794+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f553801a900 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.794+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5548064ba0 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.794+0000 7f55457fa700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f553403e3d0 0x7f5534040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.794+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5538053b90 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.794+0000 7f553effd700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f553403e3d0 0x7f5534040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.795+0000 7f553effd700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f553403e3d0 0x7f5534040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:58.800+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f553802c350 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.016+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f55481a23f0 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.017+0000 7f55457fa700 1 -- 172.21.15.26:0/4136450100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f55481a23f0 con 0x7f554807ee30 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f553403e3d0 msgr2=0x7f5534040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f553403e3d0 0x7f5534040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530006e40 comp rx=0 tx=0).stop 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 msgr2=0x7f554819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:44:59.557 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554819ed60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f55380116a0 tx=0x7f5538011a60 comp rx=0 tx=0).stop 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 shutdown_connections 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f553403e3d0 0x7f5534040890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 --2- 172.21.15.26:0/4136450100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f554807ee30 0x7f554819ed60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 >> 172.21.15.26:0/4136450100 conn(0x7f55481042b0 msgr2=0x7f5548104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.019+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 shutdown_connections 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.020+0000 7f554dc57700 1 -- 172.21.15.26:0/4136450100 wait complete. 2024-05-20T20:44:59.558 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling cephadm module... 2024-05-20T20:44:59.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:59 smithi026 ceph-mon[31990]: mgrmap e4: x(active, since 2s) 2024-05-20T20:44:59.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:44:59 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4136450100' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-20T20:45:00.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:00 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4240558426' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-20T20:45:00.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: ignoring --setuser ceph since I am not root 2024-05-20T20:45:00.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: ignoring --setgroup ceph since I am not root 2024-05-20T20:45:00.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:00.854+0000 7f06ca45c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T20:45:01.267 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:01.052+0000 7f06ca45c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:45:01.267 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:01.159+0000 7f06ca45c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T20:45:01.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b624dc700 1 Processor -- start 2024-05-20T20:45:01.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b624dc700 1 -- start start 2024-05-20T20:45:01.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b624dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c07d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:01.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b624dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c07d860 con 0x7f0b5c07ee30 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b5bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b5bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c07d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52216/0 (socket says 172.21.15.26:52216) 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:44:59.999+0000 7f0b5bfff700 1 -- 172.21.15.26:0/2936590072 learned_addr learned my addr 172.21.15.26:0/2936590072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.000+0000 7f0b5bfff700 1 -- 172.21.15.26:0/2936590072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c07d9e0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.000+0000 7f0b5bfff700 1 --2- 172.21.15.26:0/2936590072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c07d290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0098a0 tx=0x7f0b4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d562c52fb977fb3f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.000+0000 7f0b5affd700 1 -- 172.21.15.26:0/2936590072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b4c004160 con 0x7f0b5c07ee30 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.000+0000 7f0b5affd700 1 -- 172.21.15.26:0/2936590072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0b4c004300 con 0x7f0b5c07ee30 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.000+0000 7f0b5affd700 1 -- 172.21.15.26:0/2936590072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b4c0119e0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 -- 172.21.15.26:0/2936590072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 msgr2=0x7f0b5c07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 --2- 172.21.15.26:0/2936590072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c07d290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0098a0 tx=0x7f0b4c009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 -- 172.21.15.26:0/2936590072 shutdown_connections 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 --2- 172.21.15.26:0/2936590072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c07d290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:01.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 -- 172.21.15.26:0/2936590072 >> 172.21.15.26:0/2936590072 conn(0x7f0b5c104310 msgr2=0x7f0b5c106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 -- 172.21.15.26:0/2936590072 shutdown_connections 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 -- 172.21.15.26:0/2936590072 wait complete. 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 Processor -- start 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.001+0000 7f0b624dc700 1 -- start start 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b624dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b624dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c10f2a0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b5bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b5bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52228/0 (socket says 172.21.15.26:52228) 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b5bfff700 1 -- 172.21.15.26:0/4240558426 learned_addr learned my addr 172.21.15.26:0/4240558426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b5bfff700 1 -- 172.21.15.26:0/4240558426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c19f2d0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.002+0000 7f0b5bfff700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c19ed90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0119b0 tx=0x7f0b4c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.003+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b4c003af0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.003+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c19f560 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.003+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0b4c01a8a0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.003+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b4c024830 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.003+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c1a2210 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.004+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f0b4c022020 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.004+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c064ba0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.004+0000 7f0b597fa700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f0b4403e420 0x7f0b440408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.004+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b4c0536a0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.004+0000 7f0b5b7fe700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f0b4403e420 0x7f0b440408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.004+0000 7f0b5b7fe700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f0b4403e420 0x7f0b440408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0b50006fd0 tx=0x7f0b50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.009+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b4c00b3e0 con 0x7f0b5c07ee30 2024-05-20T20:45:01.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.286+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f0b5c1a2430 con 0x7f0b5c07ee30 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.679+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0b5c1a2430 con 0x7f0b5c07ee30 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.680+0000 7f0b597fa700 1 -- 172.21.15.26:0/4240558426 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f0b4c02dc90 con 0x7f0b5c07ee30 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f0b4403e420 msgr2=0x7f0b440408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f0b4403e420 0x7f0b440408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0b50006fd0 tx=0x7f0b50006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 msgr2=0x7f0b5c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c19ed90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0119b0 tx=0x7f0b4c011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 shutdown_connections 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f0b4403e420 0x7f0b440408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 --2- 172.21.15.26:0/4240558426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5c07ee30 0x7f0b5c19ed90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 >> 172.21.15.26:0/4240558426 conn(0x7f0b5c104310 msgr2=0x7f0b5c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 shutdown_connections 2024-05-20T20:45:01.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:00.682+0000 7f0b624dc700 1 -- 172.21.15.26:0/4240558426 wait complete. 2024-05-20T20:45:01.578 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:01.491+0000 7f06ca45c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T20:45:01.888 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:01.581+0000 7f06ca45c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T20:45:01.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:01 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4240558426' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-20T20:45:01.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:01 smithi026 ceph-mon[31990]: mgrmap e5: x(active, since 4s) 2024-05-20T20:45:02.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.682+0000 7f1d0e8cc700 1 Processor -- start 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.682+0000 7f1d0e8cc700 1 -- start start 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d0e8cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0810bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d0e8cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0807cde0 con 0x7f1d08109830 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0810bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0810bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52248/0 (socket says 172.21.15.26:52248) 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d07fff700 1 -- 172.21.15.26:0/1508303462 learned_addr learned my addr 172.21.15.26:0/1508303462 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:02.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d07fff700 1 -- 172.21.15.26:0/1508303462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0810c160 con 0x7f1d08109830 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d07fff700 1 --2- 172.21.15.26:0/1508303462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0810bc20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098a0 tx=0x7f1cf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12fb07e9f1c8301 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d06ffd700 1 -- 172.21.15.26:0/1508303462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf8004160 con 0x7f1d08109830 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d06ffd700 1 -- 172.21.15.26:0/1508303462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1cf8004300 con 0x7f1d08109830 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.683+0000 7f1d06ffd700 1 -- 172.21.15.26:0/1508303462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf80119e0 con 0x7f1d08109830 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/1508303462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 msgr2=0x7f1d0810bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 --2- 172.21.15.26:0/1508303462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0810bc20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098a0 tx=0x7f1cf8009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/1508303462 shutdown_connections 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 --2- 172.21.15.26:0/1508303462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0810bc20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/1508303462 >> 172.21.15.26:0/1508303462 conn(0x7f1d08103200 msgr2=0x7f1d08105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/1508303462 shutdown_connections 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.684+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/1508303462 wait complete. 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d0e8cc700 1 Processor -- start 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d0e8cc700 1 -- start start 2024-05-20T20:45:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d0e8cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d0e8cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081081d0 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52260/0 (socket says 172.21.15.26:52260) 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.685+0000 7f1d07fff700 1 -- 172.21.15.26:0/226727223 learned_addr learned my addr 172.21.15.26:0/226727223 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d07fff700 1 -- 172.21.15.26:0/226727223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0819aef0 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d07fff700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0819a9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80119b0 tx=0x7f1cf8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf80039f0 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0819b180 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1cf8003b90 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf801a620 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.686+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0819de70 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.687+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f1cf801a920 con 0x7f1d08109830 2024-05-20T20:45:02.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.687+0000 7f1d057fa700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 0x7f1cf0044cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.687+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1cf8053c20 con 0x7f1d08109830 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.687+0000 7f1d077fe700 1 -- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 msgr2=0x7f1cf0044cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.687+0000 7f1d077fe700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 0x7f1cf0044cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.687+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d08064ba0 con 0x7f1d08109830 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.697+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cf800e260 con 0x7f1d08109830 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.888+0000 7f1d077fe700 1 -- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 msgr2=0x7f1cf0044cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:01.888+0000 7f1d077fe700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 0x7f1cf0044cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.054+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1d0802e7b0 con 0x7f1d08109830 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.054+0000 7f1d057fa700 1 -- 172.21.15.26:0/226727223 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1cf801adf0 con 0x7f1d08109830 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.058+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 msgr2=0x7f1cf0044cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.058+0000 7f1d0e8cc700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 0x7f1cf0044cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.058+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 msgr2=0x7f1d0819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.058+0000 7f1d0e8cc700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0819a9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80119b0 tx=0x7f1cf8011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:02.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.059+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 shutdown_connections 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.059+0000 7f1d0e8cc700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7f1cf0042830 0x7f1cf0044cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.059+0000 7f1d0e8cc700 1 --2- 172.21.15.26:0/226727223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d08109830 0x7f1d0819a9b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.059+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 >> 172.21.15.26:0/226727223 conn(0x7f1d08103200 msgr2=0x7f1d08103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.059+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 shutdown_connections 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.059+0000 7f1d0e8cc700 1 -- 172.21.15.26:0/226727223 wait complete. 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-05-20T20:45:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 5... 2024-05-20T20:45:02.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:02 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/226727223' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-20T20:45:03.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:02.904+0000 7f06ca45c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T20:45:03.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:03.083+0000 7f06ca45c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T20:45:03.539 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:03.261+0000 7f06ca45c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T20:45:03.871 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:03.564+0000 7f06ca45c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T20:45:03.872 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:03.660+0000 7f06ca45c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T20:45:03.872 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:03.871+0000 7f06ca45c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T20:45:04.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:04 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:04.072+0000 7f06ca45c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:45:05.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:04 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:04.710+0000 7f06ca45c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T20:45:05.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:04 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:04.942+0000 7f06ca45c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-20T20:45:05.702 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:05.388+0000 7f06ca45c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T20:45:05.702 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:05.488+0000 7f06ca45c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T20:45:06.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:05.878+0000 7f06ca45c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T20:45:06.605 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:06 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:06.254+0000 7f06ca45c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T20:45:06.606 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:06 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:06.350+0000 7f06ca45c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T20:45:06.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:06 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:06.605+0000 7f06ca45c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T20:45:06.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:06 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:06.745+0000 7f06ca45c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T20:45:06.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:06 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:06.846+0000 7f06ca45c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T20:45:07.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:07.186+0000 7f06ca45c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T20:45:07.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:07.283+0000 7f06ca45c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T20:45:08.200 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:07.919+0000 7f06ca45c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T20:45:08.200 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:08.147+0000 7f06ca45c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T20:45:08.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: Active manager daemon x restarted 2024-05-20T20:45:08.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: Activating manager daemon x 2024-05-20T20:45:08.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: osdmap e2: 0 total, 0 up, 0 in 2024-05-20T20:45:08.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: mgrmap e6: x(active, starting, since 0.00377277s) 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: Manager daemon x is now available 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:08 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:09 smithi026 ceph-mon[31990]: Found migration_current of "None". Setting to last migration. 2024-05-20T20:45:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:09 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-20T20:45:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:09 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-20T20:45:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:09 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:09 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:09 smithi026 ceph-mon[31990]: mgrmap e7: x(active, since 1.00558s) 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.931+0000 7ffab0e30700 1 Processor -- start 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.931+0000 7ffab0e30700 1 -- start start 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffab0e30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac10bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffab0e30700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac07ce40 con 0x7ffaac1097b0 2024-05-20T20:45:09.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac10bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac10bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52264/0 (socket says 172.21.15.26:52264) 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffaaa59c700 1 -- 172.21.15.26:0/887696150 learned_addr learned my addr 172.21.15.26:0/887696150 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffaaa59c700 1 -- 172.21.15.26:0/887696150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac10c0e0 con 0x7ffaac1097b0 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.932+0000 7ffaaa59c700 1 --2- 172.21.15.26:0/887696150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac10bba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0098a0 tx=0x7ffa9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1353570b3ba2614d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffaa959a700 1 -- 172.21.15.26:0/887696150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c004160 con 0x7ffaac1097b0 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffaa959a700 1 -- 172.21.15.26:0/887696150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffa9c004300 con 0x7ffaac1097b0 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffaa959a700 1 -- 172.21.15.26:0/887696150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c0119e0 con 0x7ffaac1097b0 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 -- 172.21.15.26:0/887696150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 msgr2=0x7ffaac10bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 --2- 172.21.15.26:0/887696150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac10bba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0098a0 tx=0x7ffa9c009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 -- 172.21.15.26:0/887696150 shutdown_connections 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 --2- 172.21.15.26:0/887696150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac10bba0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:09.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 -- 172.21.15.26:0/887696150 >> 172.21.15.26:0/887696150 conn(0x7ffaac1031a0 msgr2=0x7ffaac105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 -- 172.21.15.26:0/887696150 shutdown_connections 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.933+0000 7ffab0e30700 1 -- 172.21.15.26:0/887696150 wait complete. 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.934+0000 7ffab0e30700 1 Processor -- start 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.934+0000 7ffab0e30700 1 -- start start 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.934+0000 7ffab0e30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.934+0000 7ffab0e30700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac10d050 con 0x7ffaac1097b0 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52268/0 (socket says 172.21.15.26:52268) 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffaaa59c700 1 -- 172.21.15.26:0/3551967879 learned_addr learned my addr 172.21.15.26:0/3551967879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffaaa59c700 1 -- 172.21.15.26:0/3551967879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac19d070 con 0x7ffaac1097b0 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffaaa59c700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac19cb30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0119b0 tx=0x7ffa9c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:09.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c0039f0 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffa9c003b90 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac19d300 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c01a620 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.935+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac19fff0 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.936+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7ffa9c01a920 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.937+0000 7ffa9b7fe700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.937+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 --> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa94040ff0 con 0x7ffa9403e420 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.937+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa9c053c30 con 0x7ffaac1097b0 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.937+0000 7ffaa9d9b700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 msgr2=0x7ffa940408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:02.937+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:03.138+0000 7ffaa9d9b700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 msgr2=0x7ffa940408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:03.138+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T20:45:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:03.539+0000 7ffaa9d9b700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 msgr2=0x7ffa940408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:03.539+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:04.340+0000 7ffaa9d9b700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 msgr2=0x7ffa940408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:04.340+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:05.941+0000 7ffaa9d9b700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 msgr2=0x7ffa940408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/1825381346 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:05.941+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:08.151+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7ffa9c01ac10 con 0x7ffaac1097b0 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:08.151+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 msgr2=0x7ffa940408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:08.151+0000 7ffa9b7fe700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.152+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7ffa9c018350 con 0x7ffaac1097b0 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.152+0000 7ffa9b7fe700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7ffa94041d10 0x7ffa94044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.152+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa94040ff0 con 0x7ffa94041d10 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.152+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7ffa94041d10 0x7ffa94044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:09.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.153+0000 7ffaa9d9b700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7ffa94041d10 0x7ffa94044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0003a10 tx=0x7ffaa0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.154+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ffa94040ff0 con 0x7ffa94041d10 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.162+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ffaac064ba0 con 0x7ffa94041d10 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffa9b7fe700 1 -- 172.21.15.26:0/3551967879 <== mgr.14118 v2:172.21.15.26:6800/2532144649 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ffaac064ba0 con 0x7ffa94041d10 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7ffa94041d10 msgr2=0x7ffa94044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7ffa94041d10 0x7ffa94044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0003a10 tx=0x7ffaa0009290 comp rx=0 tx=0).stop 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 msgr2=0x7ffaac19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac19cb30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0119b0 tx=0x7ffa9c011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 shutdown_connections 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaac1097b0 0x7ffaac19cb30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7ffa94041d10 0x7ffa94044100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 --2- 172.21.15.26:0/3551967879 >> [v2:172.21.15.26:6800/1825381346,v1:172.21.15.26:6801/1825381346] conn(0x7ffa9403e420 0x7ffa940408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 >> 172.21.15.26:0/3551967879 conn(0x7ffaac1031a0 msgr2=0x7ffaac103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 shutdown_connections 2024-05-20T20:45:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:09.163+0000 7ffab0e30700 1 -- 172.21.15.26:0/3551967879 wait complete. 2024-05-20T20:45:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 5 is available 2024-05-20T20:45:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:Setting orchestrator backend to cephadm... 2024-05-20T20:45:10.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:10 smithi026 ceph-mon[31990]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-20T20:45:10.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:10 smithi026 ceph-mon[31990]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-20T20:45:11.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.245+0000 7f368849a700 1 Processor -- start 2024-05-20T20:45:11.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.245+0000 7f368849a700 1 -- start start 2024-05-20T20:45:11.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f368849a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f368007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:11.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f368849a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f368007d8a0 con 0x7f368007ee20 2024-05-20T20:45:11.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3686236700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f368007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3686236700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f368007d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57834/0 (socket says 172.21.15.26:57834) 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3686236700 1 -- 172.21.15.26:0/181608100 learned_addr learned my addr 172.21.15.26:0/181608100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3686236700 1 -- 172.21.15.26:0/181608100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368007da20 con 0x7f368007ee20 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3686236700 1 --2- 172.21.15.26:0/181608100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f368007d2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f367c0098a0 tx=0x7f367c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dbb7b24c85e4b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3685234700 1 -- 172.21.15.26:0/181608100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f367c004160 con 0x7f368007ee20 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3685234700 1 -- 172.21.15.26:0/181608100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f367c004300 con 0x7f368007ee20 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.246+0000 7f3685234700 1 -- 172.21.15.26:0/181608100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f367c0119e0 con 0x7f368007ee20 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 -- 172.21.15.26:0/181608100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 msgr2=0x7f368007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 --2- 172.21.15.26:0/181608100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f368007d2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f367c0098a0 tx=0x7f367c009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:11.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 -- 172.21.15.26:0/181608100 shutdown_connections 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 --2- 172.21.15.26:0/181608100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f368007d2d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 -- 172.21.15.26:0/181608100 >> 172.21.15.26:0/181608100 conn(0x7f36801042d0 msgr2=0x7f3680106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 -- 172.21.15.26:0/181608100 shutdown_connections 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.247+0000 7f368849a700 1 -- 172.21.15.26:0/181608100 wait complete. 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f368849a700 1 Processor -- start 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f368849a700 1 -- start start 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f368849a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f36801a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f368849a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f368010ae70 con 0x7f368007ee20 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f3686236700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f36801a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f3686236700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f36801a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57836/0 (socket says 172.21.15.26:57836) 2024-05-20T20:45:11.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.248+0000 7f3686236700 1 -- 172.21.15.26:0/3571562727 learned_addr learned my addr 172.21.15.26:0/3571562727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f3686236700 1 -- 172.21.15.26:0/3571562727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36801a36d0 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f3686236700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f36801a3190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f367c0119b0 tx=0x7f367c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f367c0039f0 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f367c003b90 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36801a3960 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f367c01a620 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.249+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36801a6650 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.250+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3680064ba0 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.250+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f367c024a50 con 0x7f368007ee20 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.250+0000 7f36777fe700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f366c03e470 0x7f366c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:11.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.251+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f367c053bc0 con 0x7f368007ee20 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.251+0000 7f3685a35700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f366c03e470 0x7f366c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.251+0000 7f3685a35700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f366c03e470 0x7f366c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3670006fd0 tx=0x7f3670006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.256+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f367c019c70 con 0x7f368007ee20 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.487+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f36800008d0 con 0x7f366c03e470 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.492+0000 7f36777fe700 1 -- 172.21.15.26:0/3571562727 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36800008d0 con 0x7f366c03e470 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f366c03e470 msgr2=0x7f366c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f366c03e470 0x7f366c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3670006fd0 tx=0x7f3670006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 msgr2=0x7f36801a3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f36801a3190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f367c0119b0 tx=0x7f367c011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 shutdown_connections 2024-05-20T20:45:11.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f366c03e470 0x7f366c040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:11.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 --2- 172.21.15.26:0/3571562727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368007ee20 0x7f36801a3190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:11.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.494+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 >> 172.21.15.26:0/3571562727 conn(0x7f36801042d0 msgr2=0x7f368019a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:11.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.495+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 shutdown_connections 2024-05-20T20:45:11.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:10.495+0000 7f368849a700 1 -- 172.21.15.26:0/3571562727 wait complete. 2024-05-20T20:45:11.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: mgrmap e8: x(active, since 2s) 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: [20/May/2024:20:45:10] ENGINE Bus STARTING 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: [20/May/2024:20:45:10] ENGINE Serving on https://172.21.15.26:7150 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: [20/May/2024:20:45:10] ENGINE Serving on http://172.21.15.26:8765 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: [20/May/2024:20:45:10] ENGINE Bus STARTED 2024-05-20T20:45:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:11 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:12.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-20T20:45:12.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.505+0000 7fc322d34700 1 Processor -- start 2024-05-20T20:45:12.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.505+0000 7fc322d34700 1 -- start start 2024-05-20T20:45:12.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.505+0000 7fc322d34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:12.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.505+0000 7fc322d34700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c108630 con 0x7fc31c107c40 2024-05-20T20:45:12.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc320ad0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc320ad0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57840/0 (socket says 172.21.15.26:57840) 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc320ad0700 1 -- 172.21.15.26:0/4239162272 learned_addr learned my addr 172.21.15.26:0/4239162272 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc320ad0700 1 -- 172.21.15.26:0/4239162272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c108e40 con 0x7fc31c107c40 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc320ad0700 1 --2- 172.21.15.26:0/4239162272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c108060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098a0 tx=0x7fc310009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e282603230ed1eee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc31b7fe700 1 -- 172.21.15.26:0/4239162272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc310004160 con 0x7fc31c107c40 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc31b7fe700 1 -- 172.21.15.26:0/4239162272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc310004300 con 0x7fc31c107c40 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.506+0000 7fc31b7fe700 1 -- 172.21.15.26:0/4239162272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3100119e0 con 0x7fc31c107c40 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 -- 172.21.15.26:0/4239162272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 msgr2=0x7fc31c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 --2- 172.21.15.26:0/4239162272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c108060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098a0 tx=0x7fc310009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 -- 172.21.15.26:0/4239162272 shutdown_connections 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 --2- 172.21.15.26:0/4239162272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c108060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:12.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 -- 172.21.15.26:0/4239162272 >> 172.21.15.26:0/4239162272 conn(0x7fc31c1031e0 msgr2=0x7fc31c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 -- 172.21.15.26:0/4239162272 shutdown_connections 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.507+0000 7fc322d34700 1 -- 172.21.15.26:0/4239162272 wait complete. 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc322d34700 1 Processor -- start 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc322d34700 1 -- start start 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc322d34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc322d34700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c07bf60 con 0x7fc31c107c40 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc320ad0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc320ad0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57854/0 (socket says 172.21.15.26:57854) 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc320ad0700 1 -- 172.21.15.26:0/2752867725 learned_addr learned my addr 172.21.15.26:0/2752867725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc320ad0700 1 -- 172.21.15.26:0/2752867725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c19d180 con 0x7fc31c107c40 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.508+0000 7fc320ad0700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c19cc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc3100116a0 tx=0x7fc310011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.509+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3100039f0 con 0x7fc31c107c40 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.509+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc310003b90 con 0x7fc31c107c40 2024-05-20T20:45:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.509+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c19d410 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.509+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31001a600 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.509+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c1a00b0 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.510+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc31001a8b0 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.510+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c064ba0 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.510+0000 7fc319ffb700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fc30c03e470 0x7fc30c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.510+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc310053bb0 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.510+0000 7fc31bfff700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fc30c03e470 0x7fc30c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.511+0000 7fc31bfff700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fc30c03e470 0x7fc30c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc308006fd0 tx=0x7fc308006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.515+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc31002c350 con 0x7fc31c107c40 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.733+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fc31c0008d0 con 0x7fc30c03e470 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.734+0000 7fc319ffb700 1 -- 172.21.15.26:0/2752867725 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fc31c0008d0 con 0x7fc30c03e470 2024-05-20T20:45:12.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fc30c03e470 msgr2=0x7fc30c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fc30c03e470 0x7fc30c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc308006fd0 tx=0x7fc308006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 msgr2=0x7fc31c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c19cc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc3100116a0 tx=0x7fc310011a60 comp rx=0 tx=0).stop 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 shutdown_connections 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fc30c03e470 0x7fc30c040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 --2- 172.21.15.26:0/2752867725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c107c40 0x7fc31c19cc40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.736+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 >> 172.21.15.26:0/2752867725 conn(0x7fc31c1031e0 msgr2=0x7fc31c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.737+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 shutdown_connections 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:11.737+0000 7fc322d34700 1 -- 172.21.15.26:0/2752867725 wait complete. 2024-05-20T20:45:12.297 INFO:teuthology.orchestra.run.smithi026.stdout:Generating ssh key... 2024-05-20T20:45:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:12 smithi026 ceph-mon[31990]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: Generating public/private rsa key pair. 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: Your identification has been saved in /tmp/tmp0qtzjzh1/key. 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: Your public key has been saved in /tmp/tmp0qtzjzh1/key.pub. 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: The key fingerprint is: 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: SHA256:KaxRgdvWYUqgjCu2J+Z8+srEh4n1833ULYaaKmRtRKE ceph-9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: The key's randomart image is: 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: +---[RSA 3072]----+ 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: | .oo. | 2024-05-20T20:45:13.453 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: | o ...o.o | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: |. o E=.+ . | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: | . .o= .. | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: |o.. .+o S o . | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: |=.+.ooo. o + . | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: |.O =+. + . . | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: |* +..o .o . | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: | *=. .o... | 2024-05-20T20:45:13.454 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:13 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: +----[SHA256]-----+ 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd07a8bb700 1 Processor -- start 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd07a8bb700 1 -- start start 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd07a8bb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd074108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd07a8bb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd074108630 con 0x7fd074107c40 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd073fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd074108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd073fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd074108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57864/0 (socket says 172.21.15.26:57864) 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd073fff700 1 -- 172.21.15.26:0/3460587417 learned_addr learned my addr 172.21.15.26:0/3460587417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.725+0000 7fd073fff700 1 -- 172.21.15.26:0/3460587417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074108e40 con 0x7fd074107c40 2024-05-20T20:45:13.710 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd073fff700 1 --2- 172.21.15.26:0/3460587417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd074108060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0640098a0 tx=0x7fd064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cdeb4bb4f105d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd072ffd700 1 -- 172.21.15.26:0/3460587417 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd064004160 con 0x7fd074107c40 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd072ffd700 1 -- 172.21.15.26:0/3460587417 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd064004300 con 0x7fd074107c40 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd072ffd700 1 -- 172.21.15.26:0/3460587417 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0640119e0 con 0x7fd074107c40 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd07a8bb700 1 -- 172.21.15.26:0/3460587417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 msgr2=0x7fd074108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd07a8bb700 1 --2- 172.21.15.26:0/3460587417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd074108060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0640098a0 tx=0x7fd064009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd07a8bb700 1 -- 172.21.15.26:0/3460587417 shutdown_connections 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd07a8bb700 1 --2- 172.21.15.26:0/3460587417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd074108060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd07a8bb700 1 -- 172.21.15.26:0/3460587417 >> 172.21.15.26:0/3460587417 conn(0x7fd0741031a0 msgr2=0x7fd074105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.726+0000 7fd07a8bb700 1 -- 172.21.15.26:0/3460587417 shutdown_connections 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.727+0000 7fd07a8bb700 1 -- 172.21.15.26:0/3460587417 wait complete. 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.727+0000 7fd07a8bb700 1 Processor -- start 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.727+0000 7fd07a8bb700 1 -- start start 2024-05-20T20:45:13.711 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.727+0000 7fd07a8bb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd07419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.727+0000 7fd07a8bb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410c070 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd073fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd07419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd073fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd07419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57878/0 (socket says 172.21.15.26:57878) 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd073fff700 1 -- 172.21.15.26:0/975233853 learned_addr learned my addr 172.21.15.26:0/975233853 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd073fff700 1 -- 172.21.15.26:0/975233853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07419d0f0 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd073fff700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd07419cbb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0640116a0 tx=0x7fd064011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0640039f0 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd064003b90 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07419d380 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd06401a600 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.728+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0741a0070 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.729+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd06401a900 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.729+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd074064ba0 con 0x7fd074107c40 2024-05-20T20:45:13.712 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.730+0000 7fd0717fa700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fd05c03e420 0x7fd05c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.730+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd064053de0 con 0x7fd074107c40 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.730+0000 7fd0737fe700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fd05c03e420 0x7fd05c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.730+0000 7fd0737fe700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fd05c03e420 0x7fd05c0408e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd068006fd0 tx=0x7fd068006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.735+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd06402c350 con 0x7fd074107c40 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:12.956+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd0740008d0 con 0x7fd05c03e420 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.182+0000 7fd0717fa700 1 -- 172.21.15.26:0/975233853 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd0740008d0 con 0x7fd05c03e420 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fd05c03e420 msgr2=0x7fd05c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fd05c03e420 0x7fd05c0408e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd068006fd0 tx=0x7fd068006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 msgr2=0x7fd07419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd07419cbb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0640116a0 tx=0x7fd064011a60 comp rx=0 tx=0).stop 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 shutdown_connections 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fd05c03e420 0x7fd05c0408e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 --2- 172.21.15.26:0/975233853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd074107c40 0x7fd07419cbb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:13.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 >> 172.21.15.26:0/975233853 conn(0x7fd0741031a0 msgr2=0x7fd074193770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:13.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 shutdown_connections 2024-05-20T20:45:13.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:13.184+0000 7fd07a8bb700 1 -- 172.21.15.26:0/975233853 wait complete. 2024-05-20T20:45:14.359 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:14 smithi026 ceph-mon[31990]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:14.359 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:14 smithi026 ceph-mon[31990]: Generating ssh key... 2024-05-20T20:45:14.360 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:14 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:14.360 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:14 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:14.923 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:45:14.923 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.124+0000 7fe2ddec6700 1 Processor -- start 2024-05-20T20:45:14.923 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.124+0000 7fe2ddec6700 1 -- start start 2024-05-20T20:45:14.923 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.124+0000 7fe2ddec6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d810a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.124+0000 7fe2ddec6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d810ad10 con 0x7fe2d810a320 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.124+0000 7fe2d77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d810a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d810a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57888/0 (socket says 172.21.15.26:57888) 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d77fe700 1 -- 172.21.15.26:0/2197399021 learned_addr learned my addr 172.21.15.26:0/2197399021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d77fe700 1 -- 172.21.15.26:0/2197399021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d810ae90 con 0x7fe2d810a320 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d77fe700 1 --2- 172.21.15.26:0/2197399021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d810a740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80098a0 tx=0x7fe2c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb859c6813d37e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d67fc700 1 -- 172.21.15.26:0/2197399021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2c8004160 con 0x7fe2d810a320 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d67fc700 1 -- 172.21.15.26:0/2197399021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe2c8004300 con 0x7fe2d810a320 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.125+0000 7fe2d67fc700 1 -- 172.21.15.26:0/2197399021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2c80119e0 con 0x7fe2d810a320 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2197399021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 msgr2=0x7fe2d810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 --2- 172.21.15.26:0/2197399021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d810a740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80098a0 tx=0x7fe2c8009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2197399021 shutdown_connections 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 --2- 172.21.15.26:0/2197399021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d810a740 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:14.924 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2197399021 >> 172.21.15.26:0/2197399021 conn(0x7fe2d8102960 msgr2=0x7fe2d8104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2197399021 shutdown_connections 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2197399021 wait complete. 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.126+0000 7fe2ddec6700 1 Processor -- start 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2ddec6700 1 -- start start 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2ddec6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d8109940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2ddec6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8107ad0 con 0x7fe2d810a320 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2d77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d8109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2d77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d8109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57894/0 (socket says 172.21.15.26:57894) 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2d77fe700 1 -- 172.21.15.26:0/2023137995 learned_addr learned my addr 172.21.15.26:0/2023137995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2d77fe700 1 -- 172.21.15.26:0/2023137995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d8109e80 con 0x7fe2d810a320 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.127+0000 7fe2d77fe700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d8109940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80119b0 tx=0x7fe2c8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.128+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2c8003af0 con 0x7fe2d810a320 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.128+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe2c801a8a0 con 0x7fe2d810a320 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.128+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d8108070 con 0x7fe2d810a320 2024-05-20T20:45:14.925 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.128+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2c8024830 con 0x7fe2d810a320 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.128+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d8108530 con 0x7fe2d810a320 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.129+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe2c8022020 con 0x7fe2d810a320 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.129+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d8064ba0 con 0x7fe2d810a320 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.129+0000 7fe2d4ff9700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fe2c003e470 0x7fe2c0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.129+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe2c8053b30 con 0x7fe2d810a320 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.129+0000 7fe2d6ffd700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fe2c003e470 0x7fe2c0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.130+0000 7fe2d6ffd700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fe2c003e470 0x7fe2c0040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc006fd0 tx=0x7fe2cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.134+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2c800b3e0 con 0x7fe2d810a320 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.359+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe2d8103b10 con 0x7fe2c003e470 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.360+0000 7fe2d4ff9700 1 -- 172.21.15.26:0/2023137995 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe2d8103b10 con 0x7fe2c003e470 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fe2c003e470 msgr2=0x7fe2c0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:14.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fe2c003e470 0x7fe2c0040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc006fd0 tx=0x7fe2cc006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 msgr2=0x7fe2d8109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d8109940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80119b0 tx=0x7fe2c8011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 shutdown_connections 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fe2c003e470 0x7fe2c0040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 --2- 172.21.15.26:0/2023137995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d810a320 0x7fe2d8109940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 >> 172.21.15.26:0/2023137995 conn(0x7fe2d8102960 msgr2=0x7fe2d8103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 shutdown_connections 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:14.363+0000 7fe2ddec6700 1 -- 172.21.15.26:0/2023137995 wait complete. 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:Adding key to root@localhost authorized_keys... 2024-05-20T20:45:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:Adding host smithi026... 2024-05-20T20:45:15.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:15 smithi026 ceph-mon[31990]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:16 smithi026 ceph-mon[31990]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:17 smithi026 ceph-mon[31990]: Deploying cephadm binary to smithi026 2024-05-20T20:45:20.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Added host 'smithi026' with addr '172.21.15.26' 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.286+0000 7f60ef17d700 1 Processor -- start 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.286+0000 7f60ef17d700 1 -- start start 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ef17d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e8107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ef17d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60e807ce00 con 0x7f60e81075e0 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ecf19700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e8107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ecf19700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e8107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57906/0 (socket says 172.21.15.26:57906) 2024-05-20T20:45:20.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ecf19700 1 -- 172.21.15.26:0/1424408033 learned_addr learned my addr 172.21.15.26:0/1424408033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ecf19700 1 -- 172.21.15.26:0/1424408033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e8107f40 con 0x7f60e81075e0 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60ecf19700 1 --2- 172.21.15.26:0/1424408033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e8107a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098a0 tx=0x7f60dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d16e87b01ae8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.287+0000 7f60e77fe700 1 -- 172.21.15.26:0/1424408033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60dc004160 con 0x7f60e81075e0 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60e77fe700 1 -- 172.21.15.26:0/1424408033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f60dc004300 con 0x7f60e81075e0 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60e77fe700 1 -- 172.21.15.26:0/1424408033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60dc0119e0 con 0x7f60e81075e0 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 -- 172.21.15.26:0/1424408033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 msgr2=0x7f60e8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 --2- 172.21.15.26:0/1424408033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e8107a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098a0 tx=0x7f60dc009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 -- 172.21.15.26:0/1424408033 shutdown_connections 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 --2- 172.21.15.26:0/1424408033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e8107a00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:20.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 -- 172.21.15.26:0/1424408033 >> 172.21.15.26:0/1424408033 conn(0x7f60e81031a0 msgr2=0x7f60e8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 -- 172.21.15.26:0/1424408033 shutdown_connections 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.288+0000 7f60ef17d700 1 -- 172.21.15.26:0/1424408033 wait complete. 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ef17d700 1 Processor -- start 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ef17d700 1 -- start start 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ef17d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ef17d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8108f10 con 0x7f60e81075e0 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ecf19700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ecf19700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57920/0 (socket says 172.21.15.26:57920) 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ecf19700 1 -- 172.21.15.26:0/3007550803 learned_addr learned my addr 172.21.15.26:0/3007550803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.289+0000 7f60ecf19700 1 -- 172.21.15.26:0/3007550803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e819f280 con 0x7f60e81075e0 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.290+0000 7f60ecf19700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e819ed40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0119b0 tx=0x7f60dc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.290+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60dc0039f0 con 0x7f60e81075e0 2024-05-20T20:45:20.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.290+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f60dc003b90 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.290+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e819f510 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.290+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60dc01a620 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.290+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81a2200 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.291+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f60dc024430 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.291+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e8064ba0 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.291+0000 7f60e5ffb700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f60d803e420 0x7f60d80408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.291+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f60dc053cb0 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.291+0000 7f60e7fff700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f60d803e420 0x7f60d80408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.292+0000 7f60e7fff700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f60d803e420 0x7f60d80408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60d4006fd0 tx=0x7f60d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.297+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60dc0172b0 con 0x7f60e81075e0 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:15.515+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}) -- 0x7f60e80008d0 con 0x7f60d803e420 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.775+0000 7f60e5ffb700 1 -- 172.21.15.26:0/3007550803 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f60e80008d0 con 0x7f60d803e420 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f60d803e420 msgr2=0x7f60d80408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:20.376 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f60d803e420 0x7f60d80408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60d4006fd0 tx=0x7f60d4006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 msgr2=0x7f60e819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e819ed40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0119b0 tx=0x7f60dc011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 shutdown_connections 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f60d803e420 0x7f60d80408e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 --2- 172.21.15.26:0/3007550803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60e81075e0 0x7f60e819ed40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.778+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 >> 172.21.15.26:0/3007550803 conn(0x7f60e81031a0 msgr2=0x7f60e8195a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.779+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 shutdown_connections 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:19.779+0000 7f60ef17d700 1 -- 172.21.15.26:0/3007550803 wait complete. 2024-05-20T20:45:20.377 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mon service... 2024-05-20T20:45:21.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:20 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:21.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:20 smithi026 ceph-mon[31990]: Added host smithi026 2024-05-20T20:45:21.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:20 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.843+0000 7fcf36222700 1 Processor -- start 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.843+0000 7fcf36222700 1 -- start start 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.843+0000 7fcf36222700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3010bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.843+0000 7fcf36222700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3007cd60 con 0x7fcf301098b0 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3010bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3010bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39068/0 (socket says 172.21.15.26:39068) 2024-05-20T20:45:21.926 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2f7fe700 1 -- 172.21.15.26:0/1915308508 learned_addr learned my addr 172.21.15.26:0/1915308508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2f7fe700 1 -- 172.21.15.26:0/1915308508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3010c1e0 con 0x7fcf301098b0 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2f7fe700 1 --2- 172.21.15.26:0/1915308508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3010bca0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf200098a0 tx=0x7fcf20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a2336d6d35a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2e7fc700 1 -- 172.21.15.26:0/1915308508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf20004160 con 0x7fcf301098b0 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2e7fc700 1 -- 172.21.15.26:0/1915308508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcf20004300 con 0x7fcf301098b0 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.844+0000 7fcf2e7fc700 1 -- 172.21.15.26:0/1915308508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf200119e0 con 0x7fcf301098b0 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 -- 172.21.15.26:0/1915308508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 msgr2=0x7fcf3010bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 --2- 172.21.15.26:0/1915308508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3010bca0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf200098a0 tx=0x7fcf20009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 -- 172.21.15.26:0/1915308508 shutdown_connections 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 --2- 172.21.15.26:0/1915308508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3010bca0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 -- 172.21.15.26:0/1915308508 >> 172.21.15.26:0/1915308508 conn(0x7fcf30103220 msgr2=0x7fcf30105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 -- 172.21.15.26:0/1915308508 shutdown_connections 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 -- 172.21.15.26:0/1915308508 wait complete. 2024-05-20T20:45:21.927 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.845+0000 7fcf36222700 1 Processor -- start 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf36222700 1 -- start start 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf36222700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf36222700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3010d080 con 0x7fcf301098b0 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf2f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf2f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39074/0 (socket says 172.21.15.26:39074) 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf2f7fe700 1 -- 172.21.15.26:0/3772262149 learned_addr learned my addr 172.21.15.26:0/3772262149 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf2f7fe700 1 -- 172.21.15.26:0/3772262149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3019d0a0 con 0x7fcf301098b0 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf2f7fe700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3019cb60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf200119b0 tx=0x7fcf20011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.846+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf200039f0 con 0x7fcf301098b0 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.847+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcf20003b90 con 0x7fcf301098b0 2024-05-20T20:45:21.928 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.847+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3019d330 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.847+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf2001a620 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.847+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301a0020 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.847+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcf2001a8d0 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.848+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf30064ba0 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.848+0000 7fcf2cff9700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf1c03e470 0x7fcf1c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.848+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf20053cf0 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.848+0000 7fcf2effd700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf1c03e470 0x7fcf1c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.848+0000 7fcf2effd700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf1c03e470 0x7fcf1c040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf18006fd0 tx=0x7fcf18006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:20.853+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf2002c350 con 0x7fcf301098b0 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.097+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcf30104330 con 0x7fcf1c03e470 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.102+0000 7fcf2cff9700 1 -- 172.21.15.26:0/3772262149 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcf30104330 con 0x7fcf1c03e470 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf1c03e470 msgr2=0x7fcf1c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:21.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf1c03e470 0x7fcf1c040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf18006fd0 tx=0x7fcf18006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 msgr2=0x7fcf3019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3019cb60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf200119b0 tx=0x7fcf20011a90 comp rx=0 tx=0).stop 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 shutdown_connections 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf1c03e470 0x7fcf1c040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 --2- 172.21.15.26:0/3772262149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf301098b0 0x7fcf3019cb60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.104+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 >> 172.21.15.26:0/3772262149 conn(0x7fcf30103220 msgr2=0x7fcf30103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.105+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 shutdown_connections 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:21.105+0000 7fcf36222700 1 -- 172.21.15.26:0/3772262149 wait complete. 2024-05-20T20:45:21.930 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mgr service... 2024-05-20T20:45:22.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:22 smithi026 ceph-mon[31990]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:22.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:22 smithi026 ceph-mon[31990]: Saving service mon spec with placement count:5 2024-05-20T20:45:22.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:22 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:23.485 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-20T20:45:23.485 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.356+0000 7fcb53201700 1 Processor -- start 2024-05-20T20:45:23.485 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.356+0000 7fcb53201700 1 -- start start 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb53201700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb53201700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c109740 con 0x7fcb4c108d50 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb50f9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb50f9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39078/0 (socket says 172.21.15.26:39078) 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb50f9d700 1 -- 172.21.15.26:0/1446430697 learned_addr learned my addr 172.21.15.26:0/1446430697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb50f9d700 1 -- 172.21.15.26:0/1446430697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c109f50 con 0x7fcb4c108d50 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb50f9d700 1 --2- 172.21.15.26:0/1446430697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c109170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb400098a0 tx=0x7fcb40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd1f639efb337c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb4b7fe700 1 -- 172.21.15.26:0/1446430697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40004160 con 0x7fcb4c108d50 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb4b7fe700 1 -- 172.21.15.26:0/1446430697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcb40004300 con 0x7fcb4c108d50 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.357+0000 7fcb4b7fe700 1 -- 172.21.15.26:0/1446430697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb400119e0 con 0x7fcb4c108d50 2024-05-20T20:45:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 -- 172.21.15.26:0/1446430697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 msgr2=0x7fcb4c109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 --2- 172.21.15.26:0/1446430697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c109170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb400098a0 tx=0x7fcb40009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 -- 172.21.15.26:0/1446430697 shutdown_connections 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 --2- 172.21.15.26:0/1446430697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c109170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 -- 172.21.15.26:0/1446430697 >> 172.21.15.26:0/1446430697 conn(0x7fcb4c104310 msgr2=0x7fcb4c106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 -- 172.21.15.26:0/1446430697 shutdown_connections 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.358+0000 7fcb53201700 1 -- 172.21.15.26:0/1446430697 wait complete. 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb53201700 1 Processor -- start 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb53201700 1 -- start start 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb53201700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c07eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:23.487 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb53201700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c10d180 con 0x7fcb4c108d50 2024-05-20T20:45:23.488 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb50f9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c07eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:23.488 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb50f9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c07eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39090/0 (socket says 172.21.15.26:39090) 2024-05-20T20:45:23.488 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.359+0000 7fcb50f9d700 1 -- 172.21.15.26:0/2804269081 learned_addr learned my addr 172.21.15.26:0/2804269081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:23.488 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb50f9d700 1 -- 172.21.15.26:0/2804269081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c07f040 con 0x7fcb4c108d50 2024-05-20T20:45:23.488 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb50f9d700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c07eb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb400116a0 tx=0x7fcb40011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb400039f0 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcb40003b90 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c07d230 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb4001a600 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.360+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c07d770 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.361+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcb40024430 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.361+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c064ba0 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.361+0000 7fcb49ffb700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcb3c03e420 0x7fcb3c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.361+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb40053bb0 con 0x7fcb4c108d50 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.361+0000 7fcb4bfff700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcb3c03e420 0x7fcb3c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.362+0000 7fcb4bfff700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcb3c03e420 0x7fcb3c0408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcb38006fd0 tx=0x7fcb38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:23.489 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.366+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb40018a40 con 0x7fcb4c108d50 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.597+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcb4c07dfd0 con 0x7fcb3c03e420 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.602+0000 7fcb49ffb700 1 -- 172.21.15.26:0/2804269081 <== mgr.14118 v2:172.21.15.26:6800/2532144649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcb4c07dfd0 con 0x7fcb3c03e420 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcb3c03e420 msgr2=0x7fcb3c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcb3c03e420 0x7fcb3c0408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcb38006fd0 tx=0x7fcb38006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 msgr2=0x7fcb4c07eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c07eb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb400116a0 tx=0x7fcb40011a60 comp rx=0 tx=0).stop 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 shutdown_connections 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcb3c03e420 0x7fcb3c0408e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 --2- 172.21.15.26:0/2804269081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c108d50 0x7fcb4c07eb00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 >> 172.21.15.26:0/2804269081 conn(0x7fcb4c104310 msgr2=0x7fcb4c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 shutdown_connections 2024-05-20T20:45:23.490 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:22.604+0000 7fcb53201700 1 -- 172.21.15.26:0/2804269081 wait complete. 2024-05-20T20:45:23.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:23 smithi026 ceph-mon[31990]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:23.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:23 smithi026 ceph-mon[31990]: Saving service mgr spec with placement count:2 2024-05-20T20:45:23.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:23 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:23.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:23 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:23.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:23 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:25.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:25 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1888063636' entity='client.admin' 2024-05-20T20:45:25.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:25 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.953+0000 7faaf222e700 1 Processor -- start 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.953+0000 7faaf222e700 1 -- start start 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.953+0000 7faaf222e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec1095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.953+0000 7faaf222e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaec109bc0 con 0x7faaec1091f0 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaeb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec1095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaeb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec1095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39106/0 (socket says 172.21.15.26:39106) 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaeb7fe700 1 -- 172.21.15.26:0/3485754344 learned_addr learned my addr 172.21.15.26:0/3485754344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaeb7fe700 1 -- 172.21.15.26:0/3485754344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec109d40 con 0x7faaec1091f0 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaeb7fe700 1 --2- 172.21.15.26:0/3485754344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec1095f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faadc0098a0 tx=0x7faadc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1b8e8d63eaf318 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaea7fc700 1 -- 172.21.15.26:0/3485754344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faadc004160 con 0x7faaec1091f0 2024-05-20T20:45:25.405 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaea7fc700 1 -- 172.21.15.26:0/3485754344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faadc004300 con 0x7faaec1091f0 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.954+0000 7faaea7fc700 1 -- 172.21.15.26:0/3485754344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faadc011960 con 0x7faaec1091f0 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 -- 172.21.15.26:0/3485754344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 msgr2=0x7faaec1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 --2- 172.21.15.26:0/3485754344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec1095f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faadc0098a0 tx=0x7faadc009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 -- 172.21.15.26:0/3485754344 shutdown_connections 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 --2- 172.21.15.26:0/3485754344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec1095f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 -- 172.21.15.26:0/3485754344 >> 172.21.15.26:0/3485754344 conn(0x7faaec102960 msgr2=0x7faaec104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 -- 172.21.15.26:0/3485754344 shutdown_connections 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.955+0000 7faaf222e700 1 -- 172.21.15.26:0/3485754344 wait complete. 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaf222e700 1 Processor -- start 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaf222e700 1 -- start start 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaf222e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:25.406 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaf222e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaec107ba0 con 0x7faaec1091f0 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaeb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaeb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39110/0 (socket says 172.21.15.26:39110) 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.956+0000 7faaeb7fe700 1 -- 172.21.15.26:0/1888063636 learned_addr learned my addr 172.21.15.26:0/1888063636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faaeb7fe700 1 -- 172.21.15.26:0/1888063636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec19f390 con 0x7faaec1091f0 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faaeb7fe700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec19ee50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faadc011670 tx=0x7faadc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faadc003aa0 con 0x7faaec1091f0 2024-05-20T20:45:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faadc01b8d0 con 0x7faaec1091f0 2024-05-20T20:45:25.408 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec19f620 con 0x7faaec1091f0 2024-05-20T20:45:25.408 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faadc0197f0 con 0x7faaec1091f0 2024-05-20T20:45:25.408 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.957+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a22c0 con 0x7faaec1091f0 2024-05-20T20:45:25.408 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.958+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7faadc019990 con 0x7faaec1091f0 2024-05-20T20:45:25.408 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.958+0000 7faae8ff9700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7faad403e470 0x7faad4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:25.408 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.958+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faadc053e60 con 0x7faaec1091f0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.958+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec064ba0 con 0x7faaec1091f0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.958+0000 7faaeaffd700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7faad403e470 0x7faad4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.959+0000 7faaeaffd700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7faad403e470 0x7faad4040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faae0006fd0 tx=0x7faae0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:23.964+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faadc01f030 con 0x7faaec1091f0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.200+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7faaec1983b0 con 0x7faaec1091f0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.203+0000 7faae8ff9700 1 -- 172.21.15.26:0/1888063636 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7faaec1983b0 con 0x7faaec1091f0 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7faad403e470 msgr2=0x7faad4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7faad403e470 0x7faad4040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faae0006fd0 tx=0x7faae0006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 msgr2=0x7faaec19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec19ee50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faadc011670 tx=0x7faadc011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:25.409 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 shutdown_connections 2024-05-20T20:45:25.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7faad403e470 0x7faad4040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:25.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 --2- 172.21.15.26:0/1888063636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec1091f0 0x7faaec19ee50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:25.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 >> 172.21.15.26:0/1888063636 conn(0x7faaec102960 msgr2=0x7faaec195470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:25.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 shutdown_connections 2024-05-20T20:45:25.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:24.206+0000 7faaf222e700 1 -- 172.21.15.26:0/1888063636 wait complete. 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.885+0000 7f409beb7700 1 Processor -- start 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.885+0000 7f409beb7700 1 -- start start 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.885+0000 7f409beb7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f4094108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.885+0000 7f409beb7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4094108630 con 0x7f4094107c40 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4099c53700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f4094108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4099c53700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f4094108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39126/0 (socket says 172.21.15.26:39126) 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4099c53700 1 -- 172.21.15.26:0/3073269862 learned_addr learned my addr 172.21.15.26:0/3073269862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4099c53700 1 -- 172.21.15.26:0/3073269862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4094108e40 con 0x7f4094107c40 2024-05-20T20:45:26.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4099c53700 1 --2- 172.21.15.26:0/3073269862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f4094108060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40900098a0 tx=0x7f4090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e18574407b362f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4098c51700 1 -- 172.21.15.26:0/3073269862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090004160 con 0x7f4094107c40 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4098c51700 1 -- 172.21.15.26:0/3073269862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4090004300 con 0x7f4094107c40 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.886+0000 7f4098c51700 1 -- 172.21.15.26:0/3073269862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090011960 con 0x7f4094107c40 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 -- 172.21.15.26:0/3073269862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 msgr2=0x7f4094108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 --2- 172.21.15.26:0/3073269862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f4094108060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40900098a0 tx=0x7f4090009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 -- 172.21.15.26:0/3073269862 shutdown_connections 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 --2- 172.21.15.26:0/3073269862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f4094108060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 -- 172.21.15.26:0/3073269862 >> 172.21.15.26:0/3073269862 conn(0x7f4094103200 msgr2=0x7f4094105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 -- 172.21.15.26:0/3073269862 shutdown_connections 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.887+0000 7f409beb7700 1 -- 172.21.15.26:0/3073269862 wait complete. 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f409beb7700 1 Processor -- start 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f409beb7700 1 -- start start 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f409beb7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f409419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:26.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f409beb7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f409410c070 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f4099c53700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f409419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f4099c53700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f409419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39132/0 (socket says 172.21.15.26:39132) 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f4099c53700 1 -- 172.21.15.26:0/2676567820 learned_addr learned my addr 172.21.15.26:0/2676567820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f4099c53700 1 -- 172.21.15.26:0/2676567820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409419d140 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.888+0000 7f4099c53700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f409419cc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4090011670 tx=0x7f4090011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.889+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090003a60 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.889+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409419d3d0 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.889+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f409001b8d0 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.889+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090023750 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.889+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a00c0 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.890+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4090023a50 con 0x7f4094107c40 2024-05-20T20:45:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.890+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409410c1f0 con 0x7f4094107c40 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.890+0000 7f408affd700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f408003e420 0x7f40800408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.890+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4090053f90 con 0x7f4094107c40 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.890+0000 7f4099452700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f408003e420 0x7f40800408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.891+0000 7f4099452700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f408003e420 0x7f40800408e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4084006fd0 tx=0x7f4084006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:25.895+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f409001f030 con 0x7f4094107c40 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.111+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f409402e890 con 0x7f4094107c40 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.115+0000 7f408affd700 1 -- 172.21.15.26:0/2676567820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f40900202b0 con 0x7f4094107c40 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.117+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f408003e420 msgr2=0x7f40800408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.117+0000 7f409beb7700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f408003e420 0x7f40800408e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4084006fd0 tx=0x7f4084006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.117+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 msgr2=0x7f409419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.117+0000 7f409beb7700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f409419cc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4090011670 tx=0x7f4090011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.118+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 shutdown_connections 2024-05-20T20:45:26.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.118+0000 7f409beb7700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f408003e420 0x7f40800408e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:26.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.118+0000 7f409beb7700 1 --2- 172.21.15.26:0/2676567820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4094107c40 0x7f409419cc00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:26.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.118+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 >> 172.21.15.26:0/2676567820 conn(0x7f4094103200 msgr2=0x7f4094104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:26.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.118+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 shutdown_connections 2024-05-20T20:45:26.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:26.118+0000 7f409beb7700 1 -- 172.21.15.26:0/2676567820 wait complete. 2024-05-20T20:45:26.707 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling the dashboard module... 2024-05-20T20:45:27.315 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:27 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2676567820' entity='client.admin' 2024-05-20T20:45:27.315 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:27 smithi026 ceph-mon[31990]: from='mgr.14118 172.21.15.26:0/4251422378' entity='mgr.x' 2024-05-20T20:45:28.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:28 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2559950847' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-20T20:45:28.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: ignoring --setuser ceph since I am not root 2024-05-20T20:45:28.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: ignoring --setgroup ceph since I am not root 2024-05-20T20:45:28.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:28.315+0000 7f603140d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T20:45:28.832 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:28.509+0000 7f603140d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:45:28.833 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:28.618+0000 7f603140d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T20:45:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.127+0000 7f8046542700 1 Processor -- start 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.127+0000 7f8046542700 1 -- start start 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.127+0000 7f8046542700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804007d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.127+0000 7f8046542700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f804007d880 con 0x7f804007ee00 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804007d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804007d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39148/0 (socket says 172.21.15.26:39148) 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803ffff700 1 -- 172.21.15.26:0/1882183505 learned_addr learned my addr 172.21.15.26:0/1882183505 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803ffff700 1 -- 172.21.15.26:0/1882183505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804007da00 con 0x7f804007ee00 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803ffff700 1 --2- 172.21.15.26:0/1882183505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804007d2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80300098a0 tx=0x7f8030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c789f8654ac5a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803effd700 1 -- 172.21.15.26:0/1882183505 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8030004160 con 0x7f804007ee00 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803effd700 1 -- 172.21.15.26:0/1882183505 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8030004300 con 0x7f804007ee00 2024-05-20T20:45:28.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.128+0000 7f803effd700 1 -- 172.21.15.26:0/1882183505 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8030011960 con 0x7f804007ee00 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 -- 172.21.15.26:0/1882183505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 msgr2=0x7f804007d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 --2- 172.21.15.26:0/1882183505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804007d2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80300098a0 tx=0x7f8030009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 -- 172.21.15.26:0/1882183505 shutdown_connections 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 --2- 172.21.15.26:0/1882183505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804007d2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 -- 172.21.15.26:0/1882183505 >> 172.21.15.26:0/1882183505 conn(0x7f80401042d0 msgr2=0x7f8040106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 -- 172.21.15.26:0/1882183505 shutdown_connections 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.129+0000 7f8046542700 1 -- 172.21.15.26:0/1882183505 wait complete. 2024-05-20T20:45:28.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f8046542700 1 Processor -- start 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f8046542700 1 -- start start 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f8046542700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f8046542700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f804010f290 con 0x7f804007ee00 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f803ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f803ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39154/0 (socket says 172.21.15.26:39154) 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.130+0000 7f803ffff700 1 -- 172.21.15.26:0/2559950847 learned_addr learned my addr 172.21.15.26:0/2559950847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:28.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.131+0000 7f803ffff700 1 -- 172.21.15.26:0/2559950847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804019f2a0 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.131+0000 7f803ffff700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804019ed60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8030003e50 tx=0x7f8030011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.131+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8030003a60 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.131+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804019f530 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.131+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f803001b8d0 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.131+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80300237d0 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.132+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80401a2220 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.133+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f803001b440 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.133+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8040064ba0 con 0x7f804007ee00 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.133+0000 7f803d7fa700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f802c03e470 0x7f802c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:28.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.133+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8030054020 con 0x7f804007ee00 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.133+0000 7f803f7fe700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f802c03e470 0x7f802c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.134+0000 7f803f7fe700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f802c03e470 0x7f802c040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8028006fd0 tx=0x7f8028006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.138+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f803001f030 con 0x7f804007ee00 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:27.426+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f80401a27d0 con 0x7f804007ee00 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.116+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8030017e90 con 0x7f804007ee00 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.118+0000 7f803d7fa700 1 -- 172.21.15.26:0/2559950847 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f803000b3e0 con 0x7f804007ee00 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.120+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f802c03e470 msgr2=0x7f802c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.120+0000 7f8046542700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f802c03e470 0x7f802c040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8028006fd0 tx=0x7f8028006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.120+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 msgr2=0x7f804019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:28.844 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.120+0000 7f8046542700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804019ed60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8030003e50 tx=0x7f8030011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:28.845 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.121+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 shutdown_connections 2024-05-20T20:45:28.845 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.121+0000 7f8046542700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7f802c03e470 0x7f802c040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:28.845 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.121+0000 7f8046542700 1 --2- 172.21.15.26:0/2559950847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804007ee00 0x7f804019ed60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:28.845 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.121+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 >> 172.21.15.26:0/2559950847 conn(0x7f80401042d0 msgr2=0x7f8040104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:28.845 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.121+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 shutdown_connections 2024-05-20T20:45:28.845 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:28.121+0000 7f8046542700 1 -- 172.21.15.26:0/2559950847 wait complete. 2024-05-20T20:45:29.117 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:28.945+0000 7f603140d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T20:45:29.117 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:29.037+0000 7f603140d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T20:45:29.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:29 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2559950847' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-20T20:45:29.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:29 smithi026 ceph-mon[31990]: mgrmap e9: x(active, since 19s) 2024-05-20T20:45:30.189 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:30 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2822026000' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:45:30.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.265+0000 7fb718e30700 1 Processor -- start 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.265+0000 7fb718e30700 1 -- start start 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.265+0000 7fb718e30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb714109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.265+0000 7fb718e30700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb714109760 con 0x7fb714108d70 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb714109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb714109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39178/0 (socket says 172.21.15.26:39178) 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71259c700 1 -- 172.21.15.26:0/1731374537 learned_addr learned my addr 172.21.15.26:0/1731374537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71259c700 1 -- 172.21.15.26:0/1731374537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb714109f20 con 0x7fb714108d70 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71259c700 1 --2- 172.21.15.26:0/1731374537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb714109190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7080098a0 tx=0x7fb708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185dd1680d471ab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71159a700 1 -- 172.21.15.26:0/1731374537 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708004160 con 0x7fb714108d70 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71159a700 1 -- 172.21.15.26:0/1731374537 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb708004300 con 0x7fb714108d70 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.266+0000 7fb71159a700 1 -- 172.21.15.26:0/1731374537 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708011960 con 0x7fb714108d70 2024-05-20T20:45:30.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 -- 172.21.15.26:0/1731374537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 msgr2=0x7fb714109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 --2- 172.21.15.26:0/1731374537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb714109190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7080098a0 tx=0x7fb708009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 -- 172.21.15.26:0/1731374537 shutdown_connections 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 --2- 172.21.15.26:0/1731374537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb714109190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 -- 172.21.15.26:0/1731374537 >> 172.21.15.26:0/1731374537 conn(0x7fb7141042d0 msgr2=0x7fb714106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 -- 172.21.15.26:0/1731374537 shutdown_connections 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.267+0000 7fb718e30700 1 -- 172.21.15.26:0/1731374537 wait complete. 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb718e30700 1 Processor -- start 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb718e30700 1 -- start start 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb718e30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb71407eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb718e30700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb71410d150 con 0x7fb714108d70 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb71259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb71407eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:30.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb71259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb71407eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39182/0 (socket says 172.21.15.26:39182) 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.268+0000 7fb71259c700 1 -- 172.21.15.26:0/2822026000 learned_addr learned my addr 172.21.15.26:0/2822026000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb71259c700 1 -- 172.21.15.26:0/2822026000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71407f040 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb71259c700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb71407eb00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb708011670 tx=0x7fb708011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708003a60 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb70801b8d0 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71407d230 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708023750 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.269+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71407d770 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.270+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fb708023a90 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.270+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb714064ba0 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.270+0000 7fb7037fe700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 0x7fb6fc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.271+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb7080542b0 con 0x7fb714108d70 2024-05-20T20:45:30.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.271+0000 7fb711d9b700 1 -- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 msgr2=0x7fb6fc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.271+0000 7fb711d9b700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 0x7fb6fc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.276+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb70801f030 con 0x7fb714108d70 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.472+0000 7fb711d9b700 1 -- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 msgr2=0x7fb6fc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.472+0000 7fb711d9b700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 0x7fb6fc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.598+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb71407e080 con 0x7fb714108d70 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.598+0000 7fb7037fe700 1 -- 172.21.15.26:0/2822026000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb70800b3e0 con 0x7fb714108d70 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 msgr2=0x7fb6fc040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 0x7fb6fc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 msgr2=0x7fb71407eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb71407eb00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb708011670 tx=0x7fb708011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 shutdown_connections 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fb6fc03e470 0x7fb6fc040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:30.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 --2- 172.21.15.26:0/2822026000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb714108d70 0x7fb71407eb00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:30.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 >> 172.21.15.26:0/2822026000 conn(0x7fb7141042d0 msgr2=0x7fb714104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:30.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 shutdown_connections 2024-05-20T20:45:30.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:29.602+0000 7fb718e30700 1 -- 172.21.15.26:0/2822026000 wait complete. 2024-05-20T20:45:30.202 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-05-20T20:45:30.202 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 9... 2024-05-20T20:45:30.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:30.315+0000 7f603140d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T20:45:30.847 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:30.499+0000 7f603140d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T20:45:30.847 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:30.682+0000 7f603140d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T20:45:31.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:30.986+0000 7f603140d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T20:45:31.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:31.084+0000 7f603140d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T20:45:31.500 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:31.297+0000 7f603140d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T20:45:31.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:31.501+0000 7f603140d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:45:32.364 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:32.133+0000 7f603140d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T20:45:32.364 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:32.364+0000 7f603140d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-20T20:45:33.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:32.811+0000 7f603140d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T20:45:33.202 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:32.913+0000 7f603140d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T20:45:33.651 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:33.301+0000 7f603140d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T20:45:33.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:33.676+0000 7f603140d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T20:45:33.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:33.774+0000 7f603140d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T20:45:34.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:34.033+0000 7f603140d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T20:45:34.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:34.173+0000 7f603140d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T20:45:34.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:34.275+0000 7f603140d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T20:45:34.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:34.622+0000 7f603140d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T20:45:34.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:34.720+0000 7f603140d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T20:45:35.648 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:35 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:35.359+0000 7f603140d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T20:45:35.711 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:45:35 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:45:35.596+0000 7f603140d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T20:45:35.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: Active manager daemon x restarted 2024-05-20T20:45:35.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: Activating manager daemon x 2024-05-20T20:45:35.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: osdmap e3: 0 total, 0 up, 0 in 2024-05-20T20:45:35.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: mgrmap e10: x(active, starting, since 0.00353908s) 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: Manager daemon x is now available 2024-05-20T20:45:35.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:36.831 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-20T20:45:36.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-20T20:45:36.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: [20/May/2024:20:45:35] ENGINE Bus STARTING 2024-05-20T20:45:36.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: [20/May/2024:20:45:36] ENGINE Serving on https://172.21.15.26:7150 2024-05-20T20:45:36.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: [20/May/2024:20:45:36] ENGINE Serving on http://172.21.15.26:8765 2024-05-20T20:45:36.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: [20/May/2024:20:45:36] ENGINE Bus STARTED 2024-05-20T20:45:36.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:36 smithi026 ceph-mon[31990]: mgrmap e11: x(active, since 1.0063s) 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcfa4f46700 1 Processor -- start 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcfa4f46700 1 -- start start 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcfa4f46700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa0109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:37.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcfa4f46700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0109760 con 0x7fcfa0108d70 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcf9e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa0109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcf9e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa0109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55004/0 (socket says 172.21.15.26:55004) 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.641+0000 7fcf9e59c700 1 -- 172.21.15.26:0/1406685281 learned_addr learned my addr 172.21.15.26:0/1406685281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.642+0000 7fcf9e59c700 1 -- 172.21.15.26:0/1406685281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa0109f20 con 0x7fcfa0108d70 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.642+0000 7fcf9e59c700 1 --2- 172.21.15.26:0/1406685281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa0109190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf900098a0 tx=0x7fcf90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4201b86c75dfdd85 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.642+0000 7fcf9d59a700 1 -- 172.21.15.26:0/1406685281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90004160 con 0x7fcfa0108d70 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.642+0000 7fcf9d59a700 1 -- 172.21.15.26:0/1406685281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf90004300 con 0x7fcfa0108d70 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.642+0000 7fcf9d59a700 1 -- 172.21.15.26:0/1406685281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90011960 con 0x7fcfa0108d70 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1406685281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 msgr2=0x7fcfa0109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1406685281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa0109190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf900098a0 tx=0x7fcf90009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1406685281 shutdown_connections 2024-05-20T20:45:37.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1406685281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa0109190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1406685281 >> 172.21.15.26:0/1406685281 conn(0x7fcfa01042d0 msgr2=0x7fcfa0106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1406685281 shutdown_connections 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.643+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1406685281 wait complete. 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.644+0000 7fcfa4f46700 1 Processor -- start 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.644+0000 7fcfa4f46700 1 -- start start 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.644+0000 7fcfa4f46700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.644+0000 7fcfa4f46700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa010d150 con 0x7fcfa0108d70 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf9e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf9e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55012/0 (socket says 172.21.15.26:55012) 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf9e59c700 1 -- 172.21.15.26:0/1380694589 learned_addr learned my addr 172.21.15.26:0/1380694589 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf9e59c700 1 -- 172.21.15.26:0/1380694589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa019d110 con 0x7fcfa0108d70 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf9e59c700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa019cbd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf90011670 tx=0x7fcf90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:37.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90003a60 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf9001b8d0 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90023750 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa019d3a0 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.645+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa01a0090 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.646+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fcf90023a00 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.646+0000 7fcf8f7fe700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.646+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 --> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcf880410a0 con 0x7fcf8803e4d0 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.646+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf90053fc0 con 0x7fcfa0108d70 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.646+0000 7fcf9dd9b700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 msgr2=0x7fcf88040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:37.470 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.646+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.847+0000 7fcf9dd9b700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 msgr2=0x7fcf88040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:30.847+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:31.248+0000 7fcf9dd9b700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 msgr2=0x7fcf88040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:31.248+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:32.050+0000 7fcf9dd9b700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 msgr2=0x7fcf88040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:32.050+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:33.651+0000 7fcf9dd9b700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 msgr2=0x7fcf88040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2532144649 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:33.651+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:35.599+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fcf9002c3f0 con 0x7fcfa0108d70 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:35.599+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 msgr2=0x7fcf88040990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:35.599+0000 7fcf8f7fe700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.601+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fcf90055b50 con 0x7fcfa0108d70 2024-05-20T20:45:37.471 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.602+0000 7fcf8f7fe700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcf88041f50 0x7fcf88044340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.602+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcf90008e50 con 0x7fcf88041f50 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.602+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcf88041f50 0x7fcf88044340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.603+0000 7fcf9dd9b700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcf88041f50 0x7fcf88044340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcf94003a10 tx=0x7fcf94009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.603+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fcf90008e50 con 0x7fcf88041f50 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.612+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcfa010d2d0 con 0x7fcf88041f50 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.612+0000 7fcf8f7fe700 1 -- 172.21.15.26:0/1380694589 <== mgr.14150 v2:172.21.15.26:6800/2583575856 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fcfa010d2d0 con 0x7fcf88041f50 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcf88041f50 msgr2=0x7fcf88044340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcf88041f50 0x7fcf88044340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcf94003a10 tx=0x7fcf94009270 comp rx=0 tx=0).stop 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 msgr2=0x7fcfa019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa019cbd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf90011670 tx=0x7fcf90011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:37.472 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 shutdown_connections 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcf88041f50 0x7fcf88044340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfa0108d70 0x7fcfa019cbd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 --2- 172.21.15.26:0/1380694589 >> [v2:172.21.15.26:6800/2532144649,v1:172.21.15.26:6801/2532144649] conn(0x7fcf8803e4d0 0x7fcf88040990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 >> 172.21.15.26:0/1380694589 conn(0x7fcfa01042d0 msgr2=0x7fcfa0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 shutdown_connections 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:36.613+0000 7fcfa4f46700 1 -- 172.21.15.26:0/1380694589 wait complete. 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 9 is available 2024-05-20T20:45:37.473 INFO:teuthology.orchestra.run.smithi026.stdout:Generating a dashboard self-signed certificate... 2024-05-20T20:45:37.843 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:37 smithi026 ceph-mon[31990]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-20T20:45:37.843 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:37 smithi026 ceph-mon[31990]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-20T20:45:37.843 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:38.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-20T20:45:38.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.900+0000 7f49d9a7b700 1 Processor -- start 2024-05-20T20:45:38.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.900+0000 7f49d9a7b700 1 -- start start 2024-05-20T20:45:38.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.900+0000 7f49d9a7b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d410bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.900+0000 7f49d9a7b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49d407ce40 con 0x7f49d41097a0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d410bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d410bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55086/0 (socket says 172.21.15.26:55086) 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d37fe700 1 -- 172.21.15.26:0/2593198113 learned_addr learned my addr 172.21.15.26:0/2593198113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d37fe700 1 -- 172.21.15.26:0/2593198113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d410c0d0 con 0x7f49d41097a0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d37fe700 1 --2- 172.21.15.26:0/2593198113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d410bb90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49c40098a0 tx=0x7f49c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f30baae33a9904 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d27fc700 1 -- 172.21.15.26:0/2593198113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4004160 con 0x7f49d41097a0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d27fc700 1 -- 172.21.15.26:0/2593198113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49c4004300 con 0x7f49d41097a0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.901+0000 7f49d27fc700 1 -- 172.21.15.26:0/2593198113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4011960 con 0x7f49d41097a0 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 -- 172.21.15.26:0/2593198113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 msgr2=0x7f49d410bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 --2- 172.21.15.26:0/2593198113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d410bb90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49c40098a0 tx=0x7f49c4009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:38.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 -- 172.21.15.26:0/2593198113 shutdown_connections 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 --2- 172.21.15.26:0/2593198113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d410bb90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 -- 172.21.15.26:0/2593198113 >> 172.21.15.26:0/2593198113 conn(0x7f49d41031f0 msgr2=0x7f49d4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 -- 172.21.15.26:0/2593198113 shutdown_connections 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.902+0000 7f49d9a7b700 1 -- 172.21.15.26:0/2593198113 wait complete. 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d9a7b700 1 Processor -- start 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d9a7b700 1 -- start start 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d9a7b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d9a7b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410d080 con 0x7f49d41097a0 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55096/0 (socket says 172.21.15.26:55096) 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.903+0000 7f49d37fe700 1 -- 172.21.15.26:0/906455472 learned_addr learned my addr 172.21.15.26:0/906455472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d37fe700 1 -- 172.21.15.26:0/906455472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d419d140 con 0x7f49d41097a0 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d37fe700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d419cc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49c4011670 tx=0x7f49c4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c401f070 con 0x7f49d41097a0 2024-05-20T20:45:38.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49c4003b20 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d419d3d0 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c40195f0 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.904+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a00c0 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.905+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f49c4022070 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.905+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d4064ba0 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.905+0000 7f49d0ff9700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49c003e3a0 0x7f49c0040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.905+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f49c4053a10 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.906+0000 7f49d2ffd700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49c003e3a0 0x7f49c0040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.906+0000 7f49d2ffd700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49c003e3a0 0x7f49c0040860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49bc006fd0 tx=0x7f49bc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:37.911+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49c40172b0 con 0x7f49d41097a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.131+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f49d4104410 con 0x7f49c003e3a0 2024-05-20T20:45:38.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.193+0000 7f49d0ff9700 1 -- 172.21.15.26:0/906455472 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f49d4104410 con 0x7f49c003e3a0 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49c003e3a0 msgr2=0x7f49c0040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49c003e3a0 0x7f49c0040860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49bc006fd0 tx=0x7f49bc006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 msgr2=0x7f49d419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d419cc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49c4011670 tx=0x7f49c4011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 shutdown_connections 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49c003e3a0 0x7f49c0040860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 --2- 172.21.15.26:0/906455472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d41097a0 0x7f49d419cc00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 >> 172.21.15.26:0/906455472 conn(0x7f49d41031f0 msgr2=0x7f49d4103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 shutdown_connections 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:38.195+0000 7f49d9a7b700 1 -- 172.21.15.26:0/906455472 wait complete. 2024-05-20T20:45:38.775 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial admin user... 2024-05-20T20:45:39.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:39 smithi026 ceph-mon[31990]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:39.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:39.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:39.403 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:39 smithi026 ceph-mon[31990]: mgrmap e12: x(active, since 2s) 2024-05-20T20:45:39.403 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:40.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$v5Rg9TckHhW1vN.T2n1KbeLfXE/8PA.n3X4G6dHyEI4dATO6NMguG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716237939, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-20T20:45:40.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.185+0000 7f24e497f700 1 Processor -- start 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.185+0000 7f24e497f700 1 -- start start 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24e497f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e0108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24e497f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24e0108620 con 0x7f24e0107c30 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e0108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e0108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55112/0 (socket says 172.21.15.26:55112) 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24de59c700 1 -- 172.21.15.26:0/799385566 learned_addr learned my addr 172.21.15.26:0/799385566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24de59c700 1 -- 172.21.15.26:0/799385566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e0108e30 con 0x7f24e0107c30 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.186+0000 7f24de59c700 1 --2- 172.21.15.26:0/799385566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e0108050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009e80 tx=0x7f24d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78266fea668ba14 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24dd59a700 1 -- 172.21.15.26:0/799385566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d0017070 con 0x7f24e0107c30 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24dd59a700 1 -- 172.21.15.26:0/799385566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d000b8c0 con 0x7f24e0107c30 2024-05-20T20:45:40.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24dd59a700 1 -- 172.21.15.26:0/799385566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d0004420 con 0x7f24e0107c30 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 -- 172.21.15.26:0/799385566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 msgr2=0x7f24e0108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 --2- 172.21.15.26:0/799385566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e0108050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009e80 tx=0x7f24d000b330 comp rx=0 tx=0).stop 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 -- 172.21.15.26:0/799385566 shutdown_connections 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 --2- 172.21.15.26:0/799385566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e0108050 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 -- 172.21.15.26:0/799385566 >> 172.21.15.26:0/799385566 conn(0x7f24e01031f0 msgr2=0x7f24e0105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 -- 172.21.15.26:0/799385566 shutdown_connections 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.187+0000 7f24e497f700 1 -- 172.21.15.26:0/799385566 wait complete. 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.188+0000 7f24e497f700 1 Processor -- start 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.188+0000 7f24e497f700 1 -- start start 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24e497f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24e497f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010c060 con 0x7f24e0107c30 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:40.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55122/0 (socket says 172.21.15.26:55122) 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24de59c700 1 -- 172.21.15.26:0/3531060884 learned_addr learned my addr 172.21.15.26:0/3531060884 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24de59c700 1 -- 172.21.15.26:0/3531060884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e019d1f0 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.189+0000 7f24de59c700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e019ccb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f24d000bca0 tx=0x7f24d000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.192+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d0020020 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.192+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e019d480 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.192+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d00128c0 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.192+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d0017040 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.192+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01a0170 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.193+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f24d0023020 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.193+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e010c1e0 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.193+0000 7f24d77fe700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc03e4c0 0x7f24cc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.193+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24d000f050 con 0x7f24e0107c30 2024-05-20T20:45:40.276 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.193+0000 7f24ddd9b700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc03e4c0 0x7f24cc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.194+0000 7f24ddd9b700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc03e4c0 0x7f24cc040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006fd0 tx=0x7f24c8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.198+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24d002a020 con 0x7f24e0107c30 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.434+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f24e002e7b0 con 0x7f24cc03e4c0 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.681+0000 7f24d77fe700 1 -- 172.21.15.26:0/3531060884 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f24e002e7b0 con 0x7f24cc03e4c0 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc03e4c0 msgr2=0x7f24cc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc03e4c0 0x7f24cc040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006fd0 tx=0x7f24c8006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 msgr2=0x7f24e019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e019ccb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f24d000bca0 tx=0x7f24d000bd80 comp rx=0 tx=0).stop 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 shutdown_connections 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc03e4c0 0x7f24cc040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 --2- 172.21.15.26:0/3531060884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0107c30 0x7f24e019ccb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 >> 172.21.15.26:0/3531060884 conn(0x7f24e01031f0 msgr2=0x7f24e01040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:40.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 shutdown_connections 2024-05-20T20:45:40.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:39.683+0000 7f24e497f700 1 -- 172.21.15.26:0/3531060884 wait complete. 2024-05-20T20:45:40.278 INFO:teuthology.orchestra.run.smithi026.stdout:Fetching dashboard port number... 2024-05-20T20:45:40.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:40 smithi026 ceph-mon[31990]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:40.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:41.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 8443 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.729+0000 7f1b596d4700 1 Processor -- start 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.729+0000 7f1b596d4700 1 -- start start 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b596d4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b54109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b596d4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54109740 con 0x7f1b54108d50 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b54109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b54109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48778/0 (socket says 172.21.15.26:48778) 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b52ffd700 1 -- 172.21.15.26:0/1003690342 learned_addr learned my addr 172.21.15.26:0/1003690342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b52ffd700 1 -- 172.21.15.26:0/1003690342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b54109f50 con 0x7f1b54108d50 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.730+0000 7f1b52ffd700 1 --2- 172.21.15.26:0/1003690342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b54109170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b440098a0 tx=0x7f1b44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdfb27cc7004fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b51ffb700 1 -- 172.21.15.26:0/1003690342 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b44004160 con 0x7f1b54108d50 2024-05-20T20:45:41.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b51ffb700 1 -- 172.21.15.26:0/1003690342 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b44004300 con 0x7f1b54108d50 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b51ffb700 1 -- 172.21.15.26:0/1003690342 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b44011960 con 0x7f1b54108d50 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b596d4700 1 -- 172.21.15.26:0/1003690342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 msgr2=0x7f1b54109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b596d4700 1 --2- 172.21.15.26:0/1003690342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b54109170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b440098a0 tx=0x7f1b44009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b596d4700 1 -- 172.21.15.26:0/1003690342 shutdown_connections 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b596d4700 1 --2- 172.21.15.26:0/1003690342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b54109170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.731+0000 7f1b596d4700 1 -- 172.21.15.26:0/1003690342 >> 172.21.15.26:0/1003690342 conn(0x7f1b54104310 msgr2=0x7f1b54106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.732+0000 7f1b596d4700 1 -- 172.21.15.26:0/1003690342 shutdown_connections 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.732+0000 7f1b596d4700 1 -- 172.21.15.26:0/1003690342 wait complete. 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.732+0000 7f1b596d4700 1 Processor -- start 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b596d4700 1 -- start start 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b596d4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b5419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b596d4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410d180 con 0x7f1b54108d50 2024-05-20T20:45:41.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b5419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b5419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48788/0 (socket says 172.21.15.26:48788) 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b52ffd700 1 -- 172.21.15.26:0/3562848987 learned_addr learned my addr 172.21.15.26:0/3562848987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b52ffd700 1 -- 172.21.15.26:0/3562848987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5419d0b0 con 0x7f1b54108d50 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.733+0000 7f1b52ffd700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b5419cb70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b44011670 tx=0x7f1b44011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.734+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b44003a60 con 0x7f1b54108d50 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.734+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5419d340 con 0x7f1b54108d50 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.734+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b4401b8d0 con 0x7f1b54108d50 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.734+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b44023790 con 0x7f1b54108d50 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.734+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b541a0030 con 0x7f1b54108d50 2024-05-20T20:45:41.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.735+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f1b4401b440 con 0x7f1b54108d50 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.735+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5410d300 con 0x7f1b54108d50 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.735+0000 7f1b3bfff700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1b3c03e520 0x7f1b3c0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.735+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1b44053fd0 con 0x7f1b54108d50 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.735+0000 7f1b527fc700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1b3c03e520 0x7f1b3c0409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.736+0000 7f1b527fc700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1b3c03e520 0x7f1b3c0409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b48006fd0 tx=0x7f1b48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.740+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b4401f030 con 0x7f1b54108d50 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.957+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f1b541a0200 con 0x7f1b54108d50 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.957+0000 7f1b3bfff700 1 -- 172.21.15.26:0/3562848987 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1b440202b0 con 0x7f1b54108d50 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.959+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1b3c03e520 msgr2=0x7f1b3c0409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:41.637 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.959+0000 7f1b596d4700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1b3c03e520 0x7f1b3c0409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b48006fd0 tx=0x7f1b48006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 msgr2=0x7f1b5419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b5419cb70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b44011670 tx=0x7f1b44011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 shutdown_connections 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1b3c03e520 0x7f1b3c0409e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 --2- 172.21.15.26:0/3562848987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b54108d50 0x7f1b5419cb70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 >> 172.21.15.26:0/3562848987 conn(0x7f1b54104310 msgr2=0x7f1b54104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 shutdown_connections 2024-05-20T20:45:41.638 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:40.960+0000 7f1b596d4700 1 -- 172.21.15.26:0/3562848987 wait complete. 2024-05-20T20:45:41.642 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T20:45:41.642 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-05-20T20:45:41.650 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T20:45:41.650 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-05-20T20:45:41.650 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-05-20T20:45:41.650 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-20T20:45:41.652 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph Dashboard is now available at: 2024-05-20T20:45:41.652 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:41.652 INFO:teuthology.orchestra.run.smithi026.stdout: URL: https://smithi026.front.sepia.ceph.com:8443/ 2024-05-20T20:45:41.652 INFO:teuthology.orchestra.run.smithi026.stdout: User: admin 2024-05-20T20:45:41.652 INFO:teuthology.orchestra.run.smithi026.stdout: Password: rekgfmfivi 2024-05-20T20:45:41.652 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:41.653 INFO:teuthology.orchestra.run.smithi026.stdout:Saving cluster configuration to /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config directory 2024-05-20T20:45:41.653 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling autotune for osd_memory_target 2024-05-20T20:45:41.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:41 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3562848987' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-20T20:45:42.764 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff65e113700 1 Processor -- start 2024-05-20T20:45:42.764 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff65e113700 1 -- start start 2024-05-20T20:45:42.764 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff65e113700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff658109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:42.764 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff65e113700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff658109700 con 0x7ff658108d10 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff6577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff658109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff6577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff658109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48802/0 (socket says 172.21.15.26:48802) 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.078+0000 7ff6577fe700 1 -- 172.21.15.26:0/1153092490 learned_addr learned my addr 172.21.15.26:0/1153092490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.079+0000 7ff6577fe700 1 -- 172.21.15.26:0/1153092490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff658109f10 con 0x7ff658108d10 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.079+0000 7ff6577fe700 1 --2- 172.21.15.26:0/1153092490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff658109130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6480098a0 tx=0x7ff648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429174c52ccddad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.079+0000 7ff6567fc700 1 -- 172.21.15.26:0/1153092490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff648004160 con 0x7ff658108d10 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.079+0000 7ff6567fc700 1 -- 172.21.15.26:0/1153092490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff648004300 con 0x7ff658108d10 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.079+0000 7ff6567fc700 1 -- 172.21.15.26:0/1153092490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff648011960 con 0x7ff658108d10 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 -- 172.21.15.26:0/1153092490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 msgr2=0x7ff658109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 --2- 172.21.15.26:0/1153092490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff658109130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6480098a0 tx=0x7ff648009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 -- 172.21.15.26:0/1153092490 shutdown_connections 2024-05-20T20:45:42.765 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 --2- 172.21.15.26:0/1153092490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff658109130 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 -- 172.21.15.26:0/1153092490 >> 172.21.15.26:0/1153092490 conn(0x7ff6581042b0 msgr2=0x7ff6581066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 -- 172.21.15.26:0/1153092490 shutdown_connections 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 -- 172.21.15.26:0/1153092490 wait complete. 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.080+0000 7ff65e113700 1 Processor -- start 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff65e113700 1 -- start start 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff65e113700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff65807eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff65e113700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810d140 con 0x7ff658108d10 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff6577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff65807eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff6577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff65807eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48808/0 (socket says 172.21.15.26:48808) 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff6577fe700 1 -- 172.21.15.26:0/108119341 learned_addr learned my addr 172.21.15.26:0/108119341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff6577fe700 1 -- 172.21.15.26:0/108119341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65807f040 con 0x7ff658108d10 2024-05-20T20:45:42.766 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff6577fe700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff65807eb00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff648011670 tx=0x7ff648011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff648003a60 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff64801b8d0 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.081+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65807d230 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.082+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff648023750 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.082+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65807d770 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.083+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff64801b440 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.083+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65810d2c0 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.083+0000 7ff654ff9700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff64003e570 0x7ff640040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.083+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff648054140 con 0x7ff658108d10 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.083+0000 7ff656ffd700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff64003e570 0x7ff640040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.083+0000 7ff656ffd700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff64003e570 0x7ff640040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.088+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff64801f030 con 0x7ff658108d10 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.298+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff658064ba0 con 0x7ff658108d10 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.298+0000 7ff654ff9700 1 -- 172.21.15.26:0/108119341 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff6480187b0 con 0x7ff658108d10 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.300+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff64003e570 msgr2=0x7ff640040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff64003e570 0x7ff640040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 msgr2=0x7ff65807eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff65807eb00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff648011670 tx=0x7ff648011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 shutdown_connections 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff64003e570 0x7ff640040a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 --2- 172.21.15.26:0/108119341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff658108d10 0x7ff65807eb00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 >> 172.21.15.26:0/108119341 conn(0x7ff6581042b0 msgr2=0x7ff658104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 shutdown_connections 2024-05-20T20:45:42.768 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:42.301+0000 7ff65e113700 1 -- 172.21.15.26:0/108119341 wait complete. 2024-05-20T20:45:42.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:42 smithi026 ceph-mon[31990]: mgrmap e13: x(active, since 6s) 2024-05-20T20:45:44.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa450f1700 1 Processor -- start 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa450f1700 1 -- start start 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa450f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa40109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa450f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa40109700 con 0x7faa40108d10 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa3ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa40109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa3ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa40109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48824/0 (socket says 172.21.15.26:48824) 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa3ed9d700 1 -- 172.21.15.26:0/3948705717 learned_addr learned my addr 172.21.15.26:0/3948705717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:44.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.157+0000 7faa3ed9d700 1 -- 172.21.15.26:0/3948705717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa40109f10 con 0x7faa40108d10 2024-05-20T20:45:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.158+0000 7faa3ed9d700 1 --2- 172.21.15.26:0/3948705717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa40109130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa300098a0 tx=0x7faa30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6887781389cea6c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.158+0000 7faa3dd9b700 1 -- 172.21.15.26:0/3948705717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa30004160 con 0x7faa40108d10 2024-05-20T20:45:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.158+0000 7faa3dd9b700 1 -- 172.21.15.26:0/3948705717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa30004300 con 0x7faa40108d10 2024-05-20T20:45:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.158+0000 7faa3dd9b700 1 -- 172.21.15.26:0/3948705717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa30011960 con 0x7faa40108d10 2024-05-20T20:45:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.158+0000 7faa450f1700 1 -- 172.21.15.26:0/3948705717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 msgr2=0x7faa40109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.158+0000 7faa450f1700 1 --2- 172.21.15.26:0/3948705717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa40109130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa300098a0 tx=0x7faa30009bb0 comp rx=0 tx=0).stop 2024-05-20T20:45:44.077 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.159+0000 7faa450f1700 1 -- 172.21.15.26:0/3948705717 shutdown_connections 2024-05-20T20:45:44.077 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.159+0000 7faa450f1700 1 --2- 172.21.15.26:0/3948705717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa40109130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:44.077 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.159+0000 7faa450f1700 1 -- 172.21.15.26:0/3948705717 >> 172.21.15.26:0/3948705717 conn(0x7faa401042b0 msgr2=0x7faa401066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:44.077 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.159+0000 7faa450f1700 1 -- 172.21.15.26:0/3948705717 shutdown_connections 2024-05-20T20:45:44.077 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.159+0000 7faa450f1700 1 -- 172.21.15.26:0/3948705717 wait complete. 2024-05-20T20:45:44.077 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.159+0000 7faa450f1700 1 Processor -- start 2024-05-20T20:45:44.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa450f1700 1 -- start start 2024-05-20T20:45:44.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa450f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa4019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:44.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa450f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa4010d140 con 0x7faa40108d10 2024-05-20T20:45:44.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa3ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa4019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:44.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa3ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa4019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48832/0 (socket says 172.21.15.26:48832) 2024-05-20T20:45:44.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa3ed9d700 1 -- 172.21.15.26:0/602022841 learned_addr learned my addr 172.21.15.26:0/602022841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:44.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.160+0000 7faa3ed9d700 1 -- 172.21.15.26:0/602022841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4019d0a0 con 0x7faa40108d10 2024-05-20T20:45:44.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.161+0000 7faa3ed9d700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa4019cb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa30011670 tx=0x7faa30011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:44.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.161+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa30003a60 con 0x7faa40108d10 2024-05-20T20:45:44.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.161+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa3001b8d0 con 0x7faa40108d10 2024-05-20T20:45:44.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.161+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4019d330 con 0x7faa40108d10 2024-05-20T20:45:44.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.161+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa30023750 con 0x7faa40108d10 2024-05-20T20:45:44.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.161+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa401a0020 con 0x7faa40108d10 2024-05-20T20:45:44.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.162+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4010d2c0 con 0x7faa40108d10 2024-05-20T20:45:44.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.162+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7faa30023a20 con 0x7faa40108d10 2024-05-20T20:45:44.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.162+0000 7faa27fff700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faa2803e570 0x7faa28040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:44.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.162+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa30054190 con 0x7faa40108d10 2024-05-20T20:45:44.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.162+0000 7faa3e59c700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faa2803e570 0x7faa28040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:44.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.163+0000 7faa3e59c700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faa2803e570 0x7faa28040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa34006fd0 tx=0x7faa34006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:44.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.167+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa3001f030 con 0x7faa40108d10 2024-05-20T20:45:44.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.501+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7faa401a01f0 con 0x7faa40108d10 2024-05-20T20:45:44.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.504+0000 7faa27fff700 1 -- 172.21.15.26:0/602022841 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7faa3000b3e0 con 0x7faa40108d10 2024-05-20T20:45:44.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faa2803e570 msgr2=0x7faa28040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:44.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faa2803e570 0x7faa28040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa34006fd0 tx=0x7faa34006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:44.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 msgr2=0x7faa4019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:44.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa4019cb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa30011670 tx=0x7faa30011a30 comp rx=0 tx=0).stop 2024-05-20T20:45:44.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 shutdown_connections 2024-05-20T20:45:44.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faa2803e570 0x7faa28040a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:44.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 --2- 172.21.15.26:0/602022841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa40108d10 0x7faa4019cb60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:44.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 >> 172.21.15.26:0/602022841 conn(0x7faa401042b0 msgr2=0x7faa40104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 shutdown_connections 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-05-20T20:45:43.506+0000 7faa450f1700 1 -- 172.21.15.26:0/602022841 wait complete. 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout:Or, if you are only running a single cluster on this host: 2024-05-20T20:45:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: ceph telemetry on 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout:For more information see: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:44.084 INFO:teuthology.orchestra.run.smithi026.stdout:Bootstrap complete. 2024-05-20T20:45:44.114 INFO:tasks.cephadm:Fetching config... 2024-05-20T20:45:44.114 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:45:44.114 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-20T20:45:44.132 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-20T20:45:44.133 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:45:44.133 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-20T20:45:44.191 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-20T20:45:44.191 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:45:44.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-20T20:45:44.259 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-20T20:45:44.259 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:45:44.259 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-20T20:45:44.315 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-20T20:45:44.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-20T20:45:44.406 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:45:44.418 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-20T20:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:45:44.475 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-20T20:45:44.516 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCesoh+S62hfaAJbznQ2ol4J7h3+WGHmsw+DOv3uQrouTeUZq+GPDNI85C/yUabZOYWPlbeYA3CwHnnZA9lE0rfRli3lNWhTTJszufB7tiPDfjMMRoxQ/vVabVi9nf0Q5nJabLaT2EhtsgctPx48zUtl/spKi+ZXrmMedPaUEzCaR6pUAQQMo66pBewo24rJpayhwyf6Nrpvg1nNl2vcSwsVV4uO0xcDcuegfnKu/Kd0ov+QcT52k41KR05nR3RXHXy2YKEFq1UFQBBh0gD9lumo3DVNroBHGq3aHGVvR3IJu/bnyqPDgWEV8CHtUh4nkgUN9nNcAXdNxZhTWibPtKKNN1aZNOBa7Z2XE+QkyHqGV5V5BdP82Qq8K/8l9SXvMia2+Tf//PZNNUfUZjrTIpYkyzMrpzBEJ4cS8BxvTYAn3p/3mKS0QTY7Hr5RkFdq5bAK/BGdp8iNFsQqE33sICCjBxQb38FehYnt/gDlx1KZI0k20i1Bo5eHGNbL0juzuk= ceph-9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:45:44.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-20T20:45:44.775 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:45:44.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:44 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/602022841' entity='client.admin' 2024-05-20T20:45:47.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:47.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:47.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:45:47.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:47.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:47.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:47.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 -- 172.21.15.26:0/1593535664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 msgr2=0x7f48bc102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:47.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 --2- 172.21.15.26:0/1593535664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc102270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b0a0 tx=0x7f48b800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:45:47.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 -- 172.21.15.26:0/1593535664 shutdown_connections 2024-05-20T20:45:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 --2- 172.21.15.26:0/1593535664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc102270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 -- 172.21.15.26:0/1593535664 >> 172.21.15.26:0/1593535664 conn(0x7f48bc0fd440 msgr2=0x7f48bc0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 -- 172.21.15.26:0/1593535664 shutdown_connections 2024-05-20T20:45:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.226+0000 7f48c3c80700 1 -- 172.21.15.26:0/1593535664 wait complete. 2024-05-20T20:45:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c3c80700 1 Processor -- start 2024-05-20T20:45:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c3c80700 1 -- start start 2024-05-20T20:45:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c3c80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c3c80700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc107c00 con 0x7f48bc101e50 2024-05-20T20:45:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c1a1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c1a1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48856/0 (socket says 172.21.15.26:48856) 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c1a1c700 1 -- 172.21.15.26:0/4226903663 learned_addr learned my addr 172.21.15.26:0/4226903663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c1a1c700 1 -- 172.21.15.26:0/4226903663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc198a50 con 0x7f48bc101e50 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.227+0000 7f48c1a1c700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc198510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48b800bc90 tx=0x7f48b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.228+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b8017040 con 0x7f48bc101e50 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.228+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48b8022070 con 0x7f48bc101e50 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.228+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc198ce0 con 0x7f48bc101e50 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.228+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b80125c0 con 0x7f48bc101e50 2024-05-20T20:45:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.228+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc19b9d0 con 0x7f48bc101e50 2024-05-20T20:45:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.229+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f48b8012870 con 0x7f48bc101e50 2024-05-20T20:45:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.229+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc108380 con 0x7f48bc101e50 2024-05-20T20:45:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.229+0000 7f48b2ffd700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48a803e570 0x7f48a8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.229+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48b800f050 con 0x7f48bc101e50 2024-05-20T20:45:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.229+0000 7f48c121b700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48a803e570 0x7f48a8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.229+0000 7f48c121b700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48a803e570 0x7f48a8040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006fd0 tx=0x7f48ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.234+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48b801a350 con 0x7f48bc101e50 2024-05-20T20:45:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.448+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f48bc02e670 con 0x7f48bc101e50 2024-05-20T20:45:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.450+0000 7f48b2ffd700 1 -- 172.21.15.26:0/4226903663 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f48b802d250 con 0x7f48bc101e50 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.453+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48a803e570 msgr2=0x7f48a8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.453+0000 7f48c3c80700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48a803e570 0x7f48a8040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006fd0 tx=0x7f48ac006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.453+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 msgr2=0x7f48bc198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.453+0000 7f48c3c80700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc198510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48b800bc90 tx=0x7f48b800bd70 comp rx=0 tx=0).stop 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.454+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 shutdown_connections 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.454+0000 7f48c3c80700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48a803e570 0x7f48a8040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.454+0000 7f48c3c80700 1 --2- 172.21.15.26:0/4226903663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48bc101e50 0x7f48bc198510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.454+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 >> 172.21.15.26:0/4226903663 conn(0x7f48bc0fd440 msgr2=0x7f48bc06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.454+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 shutdown_connections 2024-05-20T20:45:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:47.454+0000 7f48c3c80700 1 -- 172.21.15.26:0/4226903663 wait complete. 2024-05-20T20:45:48.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4226903663' entity='client.admin' 2024-05-20T20:45:48.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:48.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:48.553 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-20T20:45:48.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-20T20:45:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:45:50.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- 172.21.15.26:0/2328532473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 msgr2=0x7fcff81022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 --2- 172.21.15.26:0/2328532473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff81022c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009910 tx=0x7fcfe8009c20 comp rx=0 tx=0).stop 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- 172.21.15.26:0/2328532473 shutdown_connections 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 --2- 172.21.15.26:0/2328532473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff81022c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- 172.21.15.26:0/2328532473 >> 172.21.15.26:0/2328532473 conn(0x7fcff80fd450 msgr2=0x7fcff80ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- 172.21.15.26:0/2328532473 shutdown_connections 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- 172.21.15.26:0/2328532473 wait complete. 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 Processor -- start 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- start start 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff80ff350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.173+0000 7fcffdc18700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8107c50 con 0x7fcff8101ea0 2024-05-20T20:45:50.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff80ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff80ff350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44808/0 (socket says 172.21.15.26:44808) 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff77fe700 1 -- 172.21.15.26:0/506691343 learned_addr learned my addr 172.21.15.26:0/506691343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff77fe700 1 -- 172.21.15.26:0/506691343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff80ff890 con 0x7fcff8101ea0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff77fe700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff80ff350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005790 tx=0x7fcfe80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfe8027070 con 0x7fcff8101ea0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfe800bdf0 con 0x7fcff8101ea0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff80fda80 con 0x7fcff8101ea0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfe800fe50 con 0x7fcff8101ea0 2024-05-20T20:45:50.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.174+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff80fdfc0 con 0x7fcff8101ea0 2024-05-20T20:45:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.175+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcfe802a070 con 0x7fcff8101ea0 2024-05-20T20:45:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.176+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff81083d0 con 0x7fcff8101ea0 2024-05-20T20:45:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.176+0000 7fcff57fa700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfe403e6b0 0x7fcfe4040b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.176+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcfe8053940 con 0x7fcff8101ea0 2024-05-20T20:45:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.176+0000 7fcfeffff700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfe403e6b0 0x7fcfe4040b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:50.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.176+0000 7fcfeffff700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfe403e6b0 0x7fcfe4040b70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0006fd0 tx=0x7fcfe0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.181+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcfe8013070 con 0x7fcff8101ea0 2024-05-20T20:45:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.400+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fcff81919d0 con 0x7fcfe403e6b0 2024-05-20T20:45:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.404+0000 7fcff57fa700 1 -- 172.21.15.26:0/506691343 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcff81919d0 con 0x7fcfe403e6b0 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfe403e6b0 msgr2=0x7fcfe4040b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfe403e6b0 0x7fcfe4040b70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0006fd0 tx=0x7fcfe0006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 msgr2=0x7fcff80ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff80ff350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005790 tx=0x7fcfe80057c0 comp rx=0 tx=0).stop 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 shutdown_connections 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfe403e6b0 0x7fcfe4040b70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 --2- 172.21.15.26:0/506691343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcff8101ea0 0x7fcff80ff350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 >> 172.21.15.26:0/506691343 conn(0x7fcff80fd450 msgr2=0x7fcff806b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 shutdown_connections 2024-05-20T20:45:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:50.406+0000 7fcffdc18700 1 -- 172.21.15.26:0/506691343 wait complete. 2024-05-20T20:45:51.052 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi055 2024-05-20T20:45:51.052 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:45:51.052 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/etc/ceph/ceph.conf 2024-05-20T20:45:51.072 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:45:51.072 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:45:51.131 INFO:tasks.cephadm:Adding host smithi055 to orchestrator... 2024-05-20T20:45:51.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch host add smithi055 2024-05-20T20:45:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: Updating smithi026:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:45:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:51 smithi026 ceph-mon[31990]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:45:52.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:52 smithi026 ceph-mon[31990]: Updating smithi026:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T20:45:52.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:52.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:52.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 -- 172.21.15.26:0/3535184553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 msgr2=0x7f4818103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 --2- 172.21.15.26:0/3535184553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f4818103590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f480800b0a0 tx=0x7f480800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:45:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 -- 172.21.15.26:0/3535184553 shutdown_connections 2024-05-20T20:45:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 --2- 172.21.15.26:0/3535184553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f4818103590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 -- 172.21.15.26:0/3535184553 >> 172.21.15.26:0/3535184553 conn(0x7f48180fea80 msgr2=0x7f4818100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 -- 172.21.15.26:0/3535184553 shutdown_connections 2024-05-20T20:45:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.802+0000 7f481cfc0700 1 -- 172.21.15.26:0/3535184553 wait complete. 2024-05-20T20:45:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481cfc0700 1 Processor -- start 2024-05-20T20:45:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481cfc0700 1 -- start start 2024-05-20T20:45:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481cfc0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f481819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481cfc0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f481810d2e0 con 0x7f48181031b0 2024-05-20T20:45:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f481819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f481819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44834/0 (socket says 172.21.15.26:44834) 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481659c700 1 -- 172.21.15.26:0/2455920164 learned_addr learned my addr 172.21.15.26:0/2455920164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.803+0000 7f481659c700 1 -- 172.21.15.26:0/2455920164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481819eb90 con 0x7f48181031b0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.804+0000 7f481659c700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f481819e650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f480800b830 tx=0x7f4808007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.804+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4808017040 con 0x7f48181031b0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.804+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4808022070 con 0x7f48181031b0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.804+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481819ee20 con 0x7f48181031b0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.804+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4808004870 con 0x7f48181031b0 2024-05-20T20:45:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.804+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181987d0 con 0x7f48181031b0 2024-05-20T20:45:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.805+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4808027460 con 0x7f48181031b0 2024-05-20T20:45:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.805+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4818104690 con 0x7f48181031b0 2024-05-20T20:45:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.805+0000 7f480f7fe700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f480403e570 0x7f4804040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.805+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f480800f050 con 0x7f48181031b0 2024-05-20T20:45:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.805+0000 7f4815d9b700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f480403e570 0x7f4804040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.806+0000 7f4815d9b700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f480403e570 0x7f4804040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4800006fd0 tx=0x7f4800006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:52.810+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f480801cd50 con 0x7f48181031b0 2024-05-20T20:45:53.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:53.031+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi055", "target": ["mon-mgr", ""]}) -- 0x7f4818064ba0 con 0x7f480403e570 2024-05-20T20:45:53.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:53 smithi026 ceph-mon[31990]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi055", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:45:54.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:54 smithi026 ceph-mon[31990]: Deploying cephadm binary to smithi055 2024-05-20T20:45:56.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:56 smithi026 ceph-mon[31990]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:45:57.345 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi055' with addr '172.21.15.55' 2024-05-20T20:45:57.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.345+0000 7f480f7fe700 1 -- 172.21.15.26:0/2455920164 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4818064ba0 con 0x7f480403e570 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f480403e570 msgr2=0x7f4804040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f480403e570 0x7f4804040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4800006fd0 tx=0x7f4800006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 msgr2=0x7f481819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f481819e650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f480800b830 tx=0x7f4808007940 comp rx=0 tx=0).stop 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 shutdown_connections 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f480403e570 0x7f4804040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 --2- 172.21.15.26:0/2455920164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48181031b0 0x7f481819e650 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 >> 172.21.15.26:0/2455920164 conn(0x7f48180fea80 msgr2=0x7f481810ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.348+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 shutdown_connections 2024-05-20T20:45:57.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:57.349+0000 7f481cfc0700 1 -- 172.21.15.26:0/2455920164 wait complete. 2024-05-20T20:45:57.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-20T20:45:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:45:58.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:58.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:58 smithi026 ceph-mon[31990]: Added host smithi055 2024-05-20T20:45:58.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:45:58.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:58 smithi026 ceph-mon[31990]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:45:58.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:59.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:45:59 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:45:59.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 -- 172.21.15.26:0/3381237522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 msgr2=0x7fb388104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:59.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 --2- 172.21.15.26:0/3381237522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb388104530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb37800b0a0 tx=0x7fb37800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 -- 172.21.15.26:0/3381237522 shutdown_connections 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 --2- 172.21.15.26:0/3381237522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb388104530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 -- 172.21.15.26:0/3381237522 >> 172.21.15.26:0/3381237522 conn(0x7fb3880fd9a0 msgr2=0x7fb3880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 -- 172.21.15.26:0/3381237522 shutdown_connections 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.560+0000 7fb38c9a8700 1 -- 172.21.15.26:0/3381237522 wait complete. 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38c9a8700 1 Processor -- start 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38c9a8700 1 -- start start 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38c9a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb38819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38c9a8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810d380 con 0x7fb388104150 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb38819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb38819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44846/0 (socket says 172.21.15.26:44846) 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38659c700 1 -- 172.21.15.26:0/2065728485 learned_addr learned my addr 172.21.15.26:0/2065728485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38659c700 1 -- 172.21.15.26:0/2065728485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38819ebc0 con 0x7fb388104150 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb38659c700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb38819e680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3780062a0 tx=0x7fb378009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.561+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb378017040 con 0x7fb388104150 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.562+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38819ee50 con 0x7fb388104150 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.562+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb378022070 con 0x7fb388104150 2024-05-20T20:45:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.562+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb378004850 con 0x7fb388104150 2024-05-20T20:45:59.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.562+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb388198750 con 0x7fb388104150 2024-05-20T20:45:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.563+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb3780040d0 con 0x7fb388104150 2024-05-20T20:45:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.563+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38806bc90 con 0x7fb388104150 2024-05-20T20:45:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.563+0000 7fb37f7fe700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb37403e510 0x7fb3740409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:45:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.563+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb37800f050 con 0x7fb388104150 2024-05-20T20:45:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.563+0000 7fb385d9b700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb37403e510 0x7fb3740409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:45:59.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.563+0000 7fb385d9b700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb37403e510 0x7fb3740409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb370006fd0 tx=0x7fb370006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:45:59.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.568+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3780093e0 con 0x7fb388104150 2024-05-20T20:45:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.790+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb388064ba0 con 0x7fb37403e510 2024-05-20T20:45:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.792+0000 7fb37f7fe700 1 -- 172.21.15.26:0/2065728485 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fb388064ba0 con 0x7fb37403e510 2024-05-20T20:45:59.792 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:45:59.792 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}] 2024-05-20T20:45:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb37403e510 msgr2=0x7fb3740409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb37403e510 0x7fb3740409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb370006fd0 tx=0x7fb370006e40 comp rx=0 tx=0).stop 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 msgr2=0x7fb38819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb38819e680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3780062a0 tx=0x7fb378009810 comp rx=0 tx=0).stop 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 shutdown_connections 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb37403e510 0x7fb3740409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 --2- 172.21.15.26:0/2065728485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388104150 0x7fb38819e680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 >> 172.21.15.26:0/2065728485 conn(0x7fb3880fd9a0 msgr2=0x7fb388104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 shutdown_connections 2024-05-20T20:45:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:45:59.794+0000 7fb38c9a8700 1 -- 172.21.15.26:0/2065728485 wait complete. 2024-05-20T20:46:00.359 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-05-20T20:46:00.360 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:46:00.360 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-05-20T20:46:00.378 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:46:00.379 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:46:00.436 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-05-20T20:46:00.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch host add smithi144 2024-05-20T20:46:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:46:01.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:01 smithi026 ceph-mon[31990]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:01.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:01 smithi026 ceph-mon[31990]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:46:01.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:01 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:02.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 -- 172.21.15.26:0/885386550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 msgr2=0x7f19bc103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 --2- 172.21.15.26:0/885386550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc103590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00b0a0 tx=0x7f19ac00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:46:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 -- 172.21.15.26:0/885386550 shutdown_connections 2024-05-20T20:46:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 --2- 172.21.15.26:0/885386550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc103590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 -- 172.21.15.26:0/885386550 >> 172.21.15.26:0/885386550 conn(0x7f19bc0fea80 msgr2=0x7f19bc100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 -- 172.21.15.26:0/885386550 shutdown_connections 2024-05-20T20:46:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.229+0000 7f19c3606700 1 -- 172.21.15.26:0/885386550 wait complete. 2024-05-20T20:46:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c3606700 1 Processor -- start 2024-05-20T20:46:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c3606700 1 -- start start 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c3606700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c3606700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc10d2e0 con 0x7f19bc1031b0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c13a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c13a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34140/0 (socket says 172.21.15.26:34140) 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.230+0000 7f19c13a2700 1 -- 172.21.15.26:0/1340660248 learned_addr learned my addr 172.21.15.26:0/1340660248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19c13a2700 1 -- 172.21.15.26:0/1340660248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc19eb50 con 0x7f19bc1031b0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19c13a2700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc19e610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00b830 tx=0x7f19ac007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ac017040 con 0x7f19bc1031b0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac022070 con 0x7f19bc1031b0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ac004870 con 0x7f19bc1031b0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc19ede0 con 0x7f19bc1031b0 2024-05-20T20:46:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.231+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc1986e0 con 0x7f19bc1031b0 2024-05-20T20:46:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.232+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc104690 con 0x7f19bc1031b0 2024-05-20T20:46:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.232+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f19ac004040 con 0x7f19bc1031b0 2024-05-20T20:46:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.232+0000 7f19ba7fc700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19a403e570 0x7f19a4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.232+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f19ac00f050 con 0x7f19bc1031b0 2024-05-20T20:46:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.232+0000 7f19c0ba1700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19a403e570 0x7f19a4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.233+0000 7f19c0ba1700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19a403e570 0x7f19a4040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f19b0006fd0 tx=0x7f19b0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.237+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19ac027460 con 0x7f19bc1031b0 2024-05-20T20:46:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:02.459+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7f19bc199c10 con 0x7f19a403e570 2024-05-20T20:46:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:03 smithi026 ceph-mon[31990]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: Deploying cephadm binary to smithi144 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi055", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:46:04.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:04.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:05 smithi026 ceph-mon[31990]: Updating smithi055:/etc/ceph/ceph.conf 2024-05-20T20:46:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:05 smithi026 ceph-mon[31990]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:05 smithi026 ceph-mon[31990]: Updating smithi055:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.832+0000 7f19ba7fc700 1 -- 172.21.15.26:0/1340660248 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f19bc199c10 con 0x7f19a403e570 2024-05-20T20:46:06.833 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-05-20T20:46:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19a403e570 msgr2=0x7f19a4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19a403e570 0x7f19a4040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f19b0006fd0 tx=0x7f19b0006e40 comp rx=0 tx=0).stop 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 msgr2=0x7f19bc19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc19e610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00b830 tx=0x7f19ac007940 comp rx=0 tx=0).stop 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 shutdown_connections 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19a403e570 0x7f19a4040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 --2- 172.21.15.26:0/1340660248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19bc1031b0 0x7f19bc19e610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 >> 172.21.15.26:0/1340660248 conn(0x7f19bc0fea80 msgr2=0x7f19bc104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 shutdown_connections 2024-05-20T20:46:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:06.835+0000 7f19c3606700 1 -- 172.21.15.26:0/1340660248 wait complete. 2024-05-20T20:46:07.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:07 smithi026 ceph-mon[31990]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:07.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:07.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:07.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:07.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:07.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:07.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-20T20:46:07.692 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:46:08.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:08 smithi026 ceph-mon[31990]: Updating smithi055:/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:46:08.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:08 smithi026 ceph-mon[31990]: Updating smithi055:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T20:46:08.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:08 smithi026 ceph-mon[31990]: Added host smithi144 2024-05-20T20:46:08.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:08.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:09.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:09 smithi026 ceph-mon[31990]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/413841292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 msgr2=0x7f2bec075cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 --2- 172.21.15.26:0/413841292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec075cc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009910 tx=0x7f2bdc009c20 comp rx=0 tx=0).stop 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/413841292 shutdown_connections 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 --2- 172.21.15.26:0/413841292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec075cc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/413841292 >> 172.21.15.26:0/413841292 conn(0x7f2bec0fea70 msgr2=0x7f2bec100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/413841292 shutdown_connections 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/413841292 wait complete. 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.223+0000 7f2bf0bbb700 1 Processor -- start 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bf0bbb700 1 -- start start 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bf0bbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bf0bbb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec113980 con 0x7f2bec0758e0 2024-05-20T20:46:09.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34156/0 (socket says 172.21.15.26:34156) 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bea59c700 1 -- 172.21.15.26:0/2667640 learned_addr learned my addr 172.21.15.26:0/2667640 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bea59c700 1 -- 172.21.15.26:0/2667640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec19f470 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bea59c700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec19ef30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc006010 tx=0x7f2bdc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bdc026070 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bdc00b750 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec19f700 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.224+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bdc0038a0 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.225+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec19fb60 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.226+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec077020 con 0x7f2bec0758e0 2024-05-20T20:46:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.226+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2bdc029070 con 0x7f2bec0758e0 2024-05-20T20:46:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.226+0000 7f2bdb7fe700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2bd403e570 0x7f2bd4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.226+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2bdc01e070 con 0x7f2bec0758e0 2024-05-20T20:46:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.226+0000 7f2be9d9b700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2bd403e570 0x7f2bd4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.226+0000 7f2be9d9b700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2bd403e570 0x7f2bd4040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:09.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.231+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2bdc013070 con 0x7f2bec0758e0 2024-05-20T20:46:09.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.451+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bec1a0300 con 0x7f2bd403e570 2024-05-20T20:46:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.452+0000 7f2bdb7fe700 1 -- 172.21.15.26:0/2667640 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f2bec1a0300 con 0x7f2bd403e570 2024-05-20T20:46:09.453 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:46:09.453 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-05-20T20:46:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2bd403e570 msgr2=0x7f2bd4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2bd403e570 0x7f2bd4040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0006e40 comp rx=0 tx=0).stop 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 msgr2=0x7f2bec19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec19ef30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc006010 tx=0x7f2bdc005710 comp rx=0 tx=0).stop 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 shutdown_connections 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2bd403e570 0x7f2bd4040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 --2- 172.21.15.26:0/2667640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bec0758e0 0x7f2bec19ef30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 >> 172.21.15.26:0/2667640 conn(0x7f2bec0fea70 msgr2=0x7f2bec0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 shutdown_connections 2024-05-20T20:46:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:09.455+0000 7f2bf0bbb700 1 -- 172.21.15.26:0/2667640 wait complete. 2024-05-20T20:46:10.075 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-20T20:46:10.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-20T20:46:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:46:10.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:10 smithi026 ceph-mon[31990]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:46:10.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:10.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:10 smithi026 ceph-mon[31990]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 -- 172.21.15.26:0/2143398205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 msgr2=0x7f2f8c109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 --2- 172.21.15.26:0/2143398205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c109500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009910 tx=0x7f2f88009c20 comp rx=0 tx=0).stop 2024-05-20T20:46:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 -- 172.21.15.26:0/2143398205 shutdown_connections 2024-05-20T20:46:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 --2- 172.21.15.26:0/2143398205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c109500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 -- 172.21.15.26:0/2143398205 >> 172.21.15.26:0/2143398205 conn(0x7f2f8c079470 msgr2=0x7f2f8c079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 -- 172.21.15.26:0/2143398205 shutdown_connections 2024-05-20T20:46:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.602+0000 7f2f938cd700 1 -- 172.21.15.26:0/2143398205 wait complete. 2024-05-20T20:46:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f938cd700 1 Processor -- start 2024-05-20T20:46:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f938cd700 1 -- start start 2024-05-20T20:46:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f938cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f938cd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c10d460 con 0x7f2f8c109120 2024-05-20T20:46:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f91669700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f91669700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48826/0 (socket says 172.21.15.26:48826) 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f91669700 1 -- 172.21.15.26:0/1055730327 learned_addr learned my addr 172.21.15.26:0/1055730327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f91669700 1 -- 172.21.15.26:0/1055730327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c104720 con 0x7f2f8c109120 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f91669700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c102830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005660 tx=0x7f2f88005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.603+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f88026070 con 0x7f2f8c109120 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.604+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f8800b750 con 0x7f2f8c109120 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.604+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c102d70 con 0x7f2f8c109120 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.604+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f880037b0 con 0x7f2f8c109120 2024-05-20T20:46:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.604+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c103230 con 0x7f2f8c109120 2024-05-20T20:46:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.604+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2f88029070 con 0x7f2f8c109120 2024-05-20T20:46:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.605+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c06b130 con 0x7f2f8c109120 2024-05-20T20:46:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.605+0000 7f2f827fc700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f7803e570 0x7f2f78040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.605+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f8801e070 con 0x7f2f8c109120 2024-05-20T20:46:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.605+0000 7f2f90e68700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f7803e570 0x7f2f78040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.605+0000 7f2f90e68700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f7803e570 0x7f2f78040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c006fd0 tx=0x7f2f7c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.610+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f88013070 con 0x7f2f8c109120 2024-05-20T20:46:11.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:11.833+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2f8c103d60 con 0x7f2f8c109120 2024-05-20T20:46:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.664+0000 7f2f827fc700 1 -- 172.21.15.26:0/1055730327 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2f8800d340 con 0x7f2f8c109120 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f7803e570 msgr2=0x7f2f78040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f7803e570 0x7f2f78040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c006fd0 tx=0x7f2f7c006e40 comp rx=0 tx=0).stop 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 msgr2=0x7f2f8c102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c102830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005660 tx=0x7f2f88005740 comp rx=0 tx=0).stop 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 shutdown_connections 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f7803e570 0x7f2f78040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 --2- 172.21.15.26:0/1055730327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f8c109120 0x7f2f8c102830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 >> 172.21.15.26:0/1055730327 conn(0x7f2f8c079470 msgr2=0x7f2f8c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.668+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 shutdown_connections 2024-05-20T20:46:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:12.669+0000 7f2f938cd700 1 -- 172.21.15.26:0/1055730327 wait complete. 2024-05-20T20:46:12.670 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1055730327' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:46:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:12.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:13.228 INFO:tasks.cephadm:Adding mon.a on smithi026 2024-05-20T20:46:13.228 INFO:tasks.cephadm:Adding mon.b on smithi055 2024-05-20T20:46:13.228 INFO:tasks.cephadm:Adding mon.c on smithi144 2024-05-20T20:46:13.228 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi026:172.21.15.26=a;smithi055:172.21.15.55=b;smithi144:172.21.15.144=c' 2024-05-20T20:46:13.461 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:13.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-20T20:46:13.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1055730327' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-20T20:46:13.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: Updating smithi144:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:13.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T20:46:13.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:46:13.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: Updating smithi144:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T20:46:13.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:13.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:13.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:14.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:14 smithi026 ceph-mon[31990]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 -- 172.21.15.144:0/2894008572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 msgr2=0x7fa148102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 --2- 172.21.15.144:0/2894008572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa148102490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa130009910 tx=0x7fa130009c20 comp rx=0 tx=0).stop 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 -- 172.21.15.144:0/2894008572 shutdown_connections 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 --2- 172.21.15.144:0/2894008572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa148102490 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 -- 172.21.15.144:0/2894008572 >> 172.21.15.144:0/2894008572 conn(0x7fa1480fd960 msgr2=0x7fa1480ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 -- 172.21.15.144:0/2894008572 shutdown_connections 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.987+0000 7fa14cb35700 1 -- 172.21.15.144:0/2894008572 wait complete. 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14cb35700 1 Processor -- start 2024-05-20T20:46:14.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14cb35700 1 -- start start 2024-05-20T20:46:14.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14cb35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa14819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:14.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14cb35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa14810c1e0 con 0x7fa1481020b0 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa14819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa14819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.144:55432/0 (socket says 172.21.15.144:55432) 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.988+0000 7fa14659c700 1 -- 172.21.15.144:0/4230623046 learned_addr learned my addr 172.21.15.144:0/4230623046 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa14659c700 1 -- 172.21.15.144:0/4230623046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14819eb90 con 0x7fa1481020b0 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa14659c700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa14819e650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa130005660 tx=0x7fa130005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa130026070 con 0x7fa1481020b0 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa13000b6f0 con 0x7fa1481020b0 2024-05-20T20:46:14.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14819ee80 con 0x7fa1481020b0 2024-05-20T20:46:14.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1300037b0 con 0x7fa1481020b0 2024-05-20T20:46:14.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.989+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481988f0 con 0x7fa1481020b0 2024-05-20T20:46:14.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.990+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148103590 con 0x7fa1481020b0 2024-05-20T20:46:14.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.990+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa130029070 con 0x7fa1481020b0 2024-05-20T20:46:14.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.991+0000 7fa13f7fe700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa13403e570 0x7fa134040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:14.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.991+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa13001e070 con 0x7fa1481020b0 2024-05-20T20:46:14.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.991+0000 7fa145d9b700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa13403e570 0x7fa134040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:14.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.991+0000 7fa145d9b700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa13403e570 0x7fa134040a30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa138006fd0 tx=0x7fa138006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:14.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:14.996+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa130013070 con 0x7fa1481020b0 2024-05-20T20:46:15.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.209+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi055:172.21.15.55=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}) -- 0x7fa1480008d0 con 0x7fa13403e570 2024-05-20T20:46:15.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.215+0000 7fa13f7fe700 1 -- 172.21.15.144:0/4230623046 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa1480008d0 con 0x7fa13403e570 2024-05-20T20:46:15.215 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mon update... 2024-05-20T20:46:15.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa13403e570 msgr2=0x7fa134040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa13403e570 0x7fa134040a30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa138006fd0 tx=0x7fa138006e40 comp rx=0 tx=0).stop 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 msgr2=0x7fa14819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa14819e650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa130005660 tx=0x7fa130005740 comp rx=0 tx=0).stop 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 shutdown_connections 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa13403e570 0x7fa134040a30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 --2- 172.21.15.144:0/4230623046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1481020b0 0x7fa14819e650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 >> 172.21.15.144:0/4230623046 conn(0x7fa1480fd960 msgr2=0x7fa148105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 shutdown_connections 2024-05-20T20:46:15.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:15.218+0000 7fa14cb35700 1 -- 172.21.15.144:0/4230623046 wait complete. 2024-05-20T20:46:15.802 DEBUG:teuthology.orchestra.run.smithi055:mon.b> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.b.service 2024-05-20T20:46:15.805 DEBUG:teuthology.orchestra.run.smithi144:mon.c> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.c.service 2024-05-20T20:46:15.807 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-20T20:46:15.807 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-20T20:46:16.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi055:172.21.15.55=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:16.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi055:172.21.15.55=b;smithi144:172.21.15.144=c;count:3 2024-05-20T20:46:16.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:16.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:16.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:16.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:16.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:16.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:16.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:16.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: Deploying daemon mon.c on smithi144 2024-05-20T20:46:16.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:16 smithi026 ceph-mon[31990]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:16.857 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.799+0000 7f73dcc33700 1 -- 172.21.15.144:0/2866997959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 msgr2=0x7f73d8103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 --2- 172.21.15.144:0/2866997959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d8103580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009910 tx=0x7f73c8009c20 comp rx=0 tx=0).stop 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 -- 172.21.15.144:0/2866997959 shutdown_connections 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 --2- 172.21.15.144:0/2866997959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d8103580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 -- 172.21.15.144:0/2866997959 >> 172.21.15.144:0/2866997959 conn(0x7f73d80fea30 msgr2=0x7f73d8100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 -- 172.21.15.144:0/2866997959 shutdown_connections 2024-05-20T20:46:18.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 -- 172.21.15.144:0/2866997959 wait complete. 2024-05-20T20:46:18.802 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 Processor -- start 2024-05-20T20:46:18.802 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.800+0000 7f73dcc33700 1 -- start start 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73dcc33700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73dcc33700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f73d810d2d0 con 0x7f73d81031a0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.144:55460/0 (socket says 172.21.15.144:55460) 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73d659c700 1 -- 172.21.15.144:0/1020986312 learned_addr learned my addr 172.21.15.144:0/1020986312 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73d659c700 1 -- 172.21.15.144:0/1020986312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d819eb20 con 0x7f73d81031a0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.801+0000 7f73d659c700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d819e5e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005660 tx=0x7f73c8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.802+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c8026070 con 0x7f73d81031a0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.802+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73c800b6f0 con 0x7f73d81031a0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.802+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c80037b0 con 0x7f73d81031a0 2024-05-20T20:46:18.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.802+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d819ee10 con 0x7f73d81031a0 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.802+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d8198880 con 0x7f73d81031a0 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.803+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d8104680 con 0x7f73d81031a0 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.803+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f73c8029070 con 0x7f73d81031a0 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.803+0000 7f73c77fe700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73c003e510 0x7f73c00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.803+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f73c801e070 con 0x7f73d81031a0 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.803+0000 7f73d5d9b700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73c003e510 0x7f73c00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:18.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.804+0000 7f73d5d9b700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73c003e510 0x7f73c00409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006fd0 tx=0x7f73cc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:18.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:18.808+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73c8013070 con 0x7f73d81031a0 2024-05-20T20:46:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:18 smithi026 ceph-mon[31990]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:19.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.097+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f73d8199be0 con 0x7f73d81031a0 2024-05-20T20:46:19.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.098+0000 7f73c77fe700 1 -- 172.21.15.144:0/1020986312 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f73c8030090 con 0x7f73d81031a0 2024-05-20T20:46:19.098 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:46:19.098 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","modified":"2024-05-20T20:44:30.783372Z","created":"2024-05-20T20:44:30.783372Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-20T20:46:19.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.100+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73c003e510 msgr2=0x7f73c00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:19.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.100+0000 7f73dcc33700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73c003e510 0x7f73c00409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006fd0 tx=0x7f73cc006e40 comp rx=0 tx=0).stop 2024-05-20T20:46:19.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.100+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 msgr2=0x7f73d819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.100+0000 7f73dcc33700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d819e5e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005660 tx=0x7f73c8005740 comp rx=0 tx=0).stop 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.101+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 shutdown_connections 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.101+0000 7f73dcc33700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73c003e510 0x7f73c00409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.101+0000 7f73dcc33700 1 --2- 172.21.15.144:0/1020986312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d81031a0 0x7f73d819e5e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.101+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 >> 172.21.15.144:0/1020986312 conn(0x7f73d80fea30 msgr2=0x7f73d8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.101+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 shutdown_connections 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:19.101+0000 7f73dcc33700 1 -- 172.21.15.144:0/1020986312 wait complete. 2024-05-20T20:46:19.101 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-20T20:46:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:19 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/1020986312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T20:46:20.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:20 smithi144 ceph-mon[35660]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-20T20:46:20.767 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-20T20:46:20.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-20T20:46:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:20 smithi026 ceph-mon[31990]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:46:21.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:20 smithi144 bash[35641]: 29b7074c07fe226242a44c8dbc1d5343ce89cc51599f5563276361d06bb7abd5 2024-05-20T20:46:21.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:20 smithi144 systemd[1]: Started Ceph mon.c for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.982+0000 7f21dc1ce700 1 -- 172.21.15.144:0/3944243784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c00035c0 msgr2=0x7f21c0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.982+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/3944243784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c00035c0 0x7f21c0005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f21d4102f90 tx=0x7f21d0014ec0 comp rx=0 tx=0).stop 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.983+0000 7f21dc1ce700 1 -- 172.21.15.144:0/3944243784 shutdown_connections 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.983+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/3944243784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c00035c0 0x7f21c0005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.983+0000 7f21dc1ce700 1 -- 172.21.15.144:0/3944243784 >> 172.21.15.144:0/3944243784 conn(0x7f21d40feb50 msgr2=0x7f21d4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.983+0000 7f21dc1ce700 1 -- 172.21.15.144:0/3944243784 shutdown_connections 2024-05-20T20:46:25.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.983+0000 7f21dc1ce700 1 -- 172.21.15.144:0/3944243784 wait complete. 2024-05-20T20:46:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.983+0000 7f21dc1ce700 1 Processor -- start 2024-05-20T20:46:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21dc1ce700 1 -- start start 2024-05-20T20:46:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21dc1ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21dc1ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d4075d80 0x7f21d4076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21dc1ce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21d410ae60 con 0x7f21d4075d80 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21dc1ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21d410ace0 con 0x7f21c00035c0 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60214/0 (socket says 172.21.15.144:60214) 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 -- 172.21.15.144:0/1961551298 learned_addr learned my addr 172.21.15.144:0/1961551298 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 -- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 msgr2=0x7f21d4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 -- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 msgr2=0x7f21d4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9f6a700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.984+0000 7f21d9769700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d4075d80 0x7f21d4076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21d9769700 1 -- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 msgr2=0x7f21d4075840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21d9769700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21d9769700 1 -- 172.21.15.144:0/1961551298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d4077760 con 0x7f21d4075d80 2024-05-20T20:46:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21d9769700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d4075d80 0x7f21d4076200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21c800b4d0 tx=0x7f21c800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f21c8011630 con 0x7f21d4075d80 2024-05-20T20:46:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21c8011c60 con 0x7f21d4075d80 2024-05-20T20:46:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d41a4d20 con 0x7f21d4075d80 2024-05-20T20:46:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f21c8010c90 con 0x7f21d4075d80 2024-05-20T20:46:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.985+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d41a5260 con 0x7f21d4075d80 2024-05-20T20:46:25.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.986+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f21c8018020 con 0x7f21d4075d80 2024-05-20T20:46:25.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.986+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d41047a0 con 0x7f21d4075d80 2024-05-20T20:46:25.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.987+0000 7f21c6ffd700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21b803e2d0 0x7f21b8040790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:25.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.987+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f21c8052060 con 0x7f21d4075d80 2024-05-20T20:46:25.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.987+0000 7f21d9f6a700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21b803e2d0 0x7f21b8040790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:25.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.987+0000 7f21d9f6a700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21b803e2d0 0x7f21b8040790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21d0014a70 tx=0x7f21d0021040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:25.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:25.992+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21c801f020 con 0x7f21d4075d80 2024-05-20T20:46:26.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: mon.a calling monitor election 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: mon.c calling monitor election 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: monmap epoch 2 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: last_changed 2024-05-20T20:46:20.687305+0000 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: created 2024-05-20T20:44:30.783372+0000 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: min_mon_release 19 (squid) 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: election_strategy: 1 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: fsmap 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: mgrmap e13: x(active, since 50s) 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: overall HEALTH_OK 2024-05-20T20:46:26.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:26.053 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:26.053 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:26.053 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:26.053 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:26.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: mon.a calling monitor election 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: mon.c calling monitor election 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: monmap epoch 2 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: last_changed 2024-05-20T20:46:20.687305+0000 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: created 2024-05-20T20:44:30.783372+0000 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: min_mon_release 19 (squid) 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: election_strategy: 1 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: fsmap 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: mgrmap e13: x(active, since 50s) 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: overall HEALTH_OK 2024-05-20T20:46:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:26.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:26.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:26.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:26.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:26.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.286+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f21d41a55e0 con 0x7f21d4075d80 2024-05-20T20:46:26.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.287+0000 7f21c6ffd700 1 -- 172.21.15.144:0/1961551298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f21c80187b0 con 0x7f21d4075d80 2024-05-20T20:46:26.287 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:46:26.287 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":2,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","modified":"2024-05-20T20:46:20.687305Z","created":"2024-05-20T20:44:30.783372Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21b803e2d0 msgr2=0x7f21b8040790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21b803e2d0 0x7f21b8040790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21d0014a70 tx=0x7f21d0021040 comp rx=0 tx=0).stop 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d4075d80 msgr2=0x7f21d4076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d4075d80 0x7f21d4076200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21c800b4d0 tx=0x7f21c800b890 comp rx=0 tx=0).stop 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 shutdown_connections 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c00035c0 0x7f21d4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21b803e2d0 0x7f21b8040790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:26.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 --2- 172.21.15.144:0/1961551298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d4075d80 0x7f21d4076200 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:26.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 >> 172.21.15.144:0/1961551298 conn(0x7f21d40feb50 msgr2=0x7f21d40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:26.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 shutdown_connections 2024-05-20T20:46:26.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:26.289+0000 7f21dc1ce700 1 -- 172.21.15.144:0/1961551298 wait complete. 2024-05-20T20:46:26.290 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 2 2024-05-20T20:46:26.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:26 smithi144 ceph-mon[35660]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:26.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:26 smithi144 ceph-mon[35660]: Deploying daemon mon.b on smithi055 2024-05-20T20:46:26.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:26 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/1961551298' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T20:46:26.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:26 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:27.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:26 smithi026 ceph-mon[31990]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:27.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:26 smithi026 ceph-mon[31990]: Deploying daemon mon.b on smithi055 2024-05-20T20:46:27.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:26 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/1961551298' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T20:46:27.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:26 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:27.892 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-20T20:46:27.893 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-20T20:46:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:46:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:28 smithi144 ceph-mon[35660]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:29.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:28 smithi026 ceph-mon[31990]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 -- 172.21.15.144:0/385655872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 msgr2=0x7f23481036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 --2- 172.21.15.144:0/385655872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f23481036b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2338009910 tx=0x7f2338009c20 comp rx=0 tx=0).stop 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 -- 172.21.15.144:0/385655872 shutdown_connections 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 --2- 172.21.15.144:0/385655872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f23481036b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 -- 172.21.15.144:0/385655872 >> 172.21.15.144:0/385655872 conn(0x7f23480feb60 msgr2=0x7f2348100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 -- 172.21.15.144:0/385655872 shutdown_connections 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 -- 172.21.15.144:0/385655872 wait complete. 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 Processor -- start 2024-05-20T20:46:29.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.592+0000 7f234d8d9700 1 -- start start 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f234d8d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f234819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f234d8d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 0x7f2348198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f234d8d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f234810d400 con 0x7f23481032d0 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f234d8d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f234810d280 con 0x7f234819ed00 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f234819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f234819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.144:47128/0 (socket says 172.21.15.144:47128) 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 -- 172.21.15.144:0/1853092153 learned_addr learned my addr 172.21.15.144:0/1853092153 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f23467fc700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 0x7f2348198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f23467fc700 1 -- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 msgr2=0x7f2348198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f23467fc700 1 -- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 msgr2=0x7f2348198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f23467fc700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 0x7f2348198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T20:46:29.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 -- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 msgr2=0x7f2348198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f23467fc700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 0x7f2348198980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 0x7f2348198980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 -- 172.21.15.144:0/1853092153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234819f340 con 0x7f23481032d0 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.593+0000 7f2346ffd700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f234819e7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2338005b40 tx=0x7f2338005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.594+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2338005dc0 con 0x7f23481032d0 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.594+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2348198ec0 con 0x7f23481032d0 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.594+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f233800bdf0 con 0x7f23481032d0 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.594+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2338022cf0 con 0x7f23481032d0 2024-05-20T20:46:29.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.594+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2348199450 con 0x7f23481032d0 2024-05-20T20:46:29.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.595+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23481997c0 con 0x7f23481032d0 2024-05-20T20:46:29.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.595+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f233802a070 con 0x7f23481032d0 2024-05-20T20:46:29.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.595+0000 7f234c8d7700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f233403e2b0 0x7f2334040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:29.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.595+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f233801e030 con 0x7f23481032d0 2024-05-20T20:46:29.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.595+0000 7f23467fc700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f233403e2b0 0x7f2334040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:29.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.596+0000 7f23467fc700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f233403e2b0 0x7f2334040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2330009850 tx=0x7f2330008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:29.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.601+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2338013070 con 0x7f23481032d0 2024-05-20T20:46:29.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.785+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338022470 con 0x7f23481032d0 2024-05-20T20:46:29.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:29.892+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2348199f50 con 0x7f23481032d0 2024-05-20T20:46:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:29 smithi055 systemd[1]: Started Ceph mon.b for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:46:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.811+0000 7f234c8d7700 1 -- 172.21.15.144:0/1853092153 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f233800d340 con 0x7f23481032d0 2024-05-20T20:46:34.811 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-20T20:46:34.811 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":3,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","modified":"2024-05-20T20:46:29.786536Z","created":"2024-05-20T20:44:30.783372Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f233403e2b0 msgr2=0x7f2334040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f233403e2b0 0x7f2334040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2330009850 tx=0x7f2330008040 comp rx=0 tx=0).stop 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 msgr2=0x7f234819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f234819e7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2338005b40 tx=0x7f2338005780 comp rx=0 tx=0).stop 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 shutdown_connections 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f233403e2b0 0x7f2334040770 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23481032d0 0x7f234819e7c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 --2- 172.21.15.144:0/1853092153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f234819ed00 0x7f2348198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 >> 172.21.15.144:0/1853092153 conn(0x7f23480feb60 msgr2=0x7f2348104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 shutdown_connections 2024-05-20T20:46:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:34.814+0000 7f234d8d9700 1 -- 172.21.15.144:0/1853092153 wait complete. 2024-05-20T20:46:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 3 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: mon.a calling monitor election 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: mon.c calling monitor election 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/1853092153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.155 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: mon.b calling monitor election 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: monmap epoch 3 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: last_changed 2024-05-20T20:46:29.786536+0000 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: created 2024-05-20T20:44:30.783372+0000 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: min_mon_release 19 (squid) 2024-05-20T20:46:35.156 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: election_strategy: 1 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: 2: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.b 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: fsmap 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: mgrmap e13: x(active, since 59s) 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: overall HEALTH_OK 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:35.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: mon.a calling monitor election 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: mon.c calling monitor election 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/1853092153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.203 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.203 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: mon.b calling monitor election 2024-05-20T20:46:35.203 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.203 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.203 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-20T20:46:35.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: monmap epoch 3 2024-05-20T20:46:35.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:35.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: last_changed 2024-05-20T20:46:29.786536+0000 2024-05-20T20:46:35.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: created 2024-05-20T20:44:30.783372+0000 2024-05-20T20:46:35.205 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: min_mon_release 19 (squid) 2024-05-20T20:46:35.205 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: election_strategy: 1 2024-05-20T20:46:35.205 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-05-20T20:46:35.205 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-05-20T20:46:35.205 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: 2: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.b 2024-05-20T20:46:35.205 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: fsmap 2024-05-20T20:46:35.206 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T20:46:35.206 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: mgrmap e13: x(active, since 59s) 2024-05-20T20:46:35.206 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: overall HEALTH_OK 2024-05-20T20:46:35.206 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:35.206 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: mon.a calling monitor election 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: mon.c calling monitor election 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/1853092153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: mon.b calling monitor election 2024-05-20T20:46:35.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: monmap epoch 3 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: last_changed 2024-05-20T20:46:29.786536+0000 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: created 2024-05-20T20:44:30.783372+0000 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: min_mon_release 19 (squid) 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: election_strategy: 1 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-05-20T20:46:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: 2: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.b 2024-05-20T20:46:35.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: fsmap 2024-05-20T20:46:35.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T20:46:35.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: mgrmap e13: x(active, since 59s) 2024-05-20T20:46:35.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: overall HEALTH_OK 2024-05-20T20:46:35.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:35.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:35.476 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-20T20:46:35.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-20T20:46:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:46:36.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:36.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:36.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:46:37.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:37.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.191 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:37.191 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 -- 172.21.15.26:0/91514168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301020f0 msgr2=0x7f85301024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 --2- 172.21.15.26:0/91514168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301020f0 0x7f85301024d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8520009910 tx=0x7f8520009c20 comp rx=0 tx=0).stop 2024-05-20T20:46:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 -- 172.21.15.26:0/91514168 shutdown_connections 2024-05-20T20:46:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 --2- 172.21.15.26:0/91514168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301020f0 0x7f85301024d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 -- 172.21.15.26:0/91514168 >> 172.21.15.26:0/91514168 conn(0x7f85300779d0 msgr2=0x7f8530077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 -- 172.21.15.26:0/91514168 shutdown_connections 2024-05-20T20:46:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.426+0000 7f853499f700 1 -- 172.21.15.26:0/91514168 wait complete. 2024-05-20T20:46:37.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 Processor -- start 2024-05-20T20:46:37.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 -- start start 2024-05-20T20:46:37.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85301020f0 0x7f853019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853019ed90 0x7f8530198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 0x7f8530199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f853010c200 con 0x7f8530198f50 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f853010c080 con 0x7f85301020f0 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f853499f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f853010c380 con 0x7f853019ed90 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f852ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 0x7f8530199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f852ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 0x7f8530199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57560/0 (socket says 172.21.15.26:57560) 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f852ed9d700 1 -- 172.21.15.26:0/1902326395 learned_addr learned my addr 172.21.15.26:0/1902326395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f852e59c700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85301020f0 0x7f853019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.427+0000 7f852dd9b700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853019ed90 0x7f8530198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852ed9d700 1 -- 172.21.15.26:0/1902326395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853019ed90 msgr2=0x7f8530198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852ed9d700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853019ed90 0x7f8530198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852ed9d700 1 -- 172.21.15.26:0/1902326395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85301020f0 msgr2=0x7f853019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852ed9d700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85301020f0 0x7f853019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852ed9d700 1 -- 172.21.15.26:0/1902326395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85301a5720 con 0x7f8530198f50 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852e59c700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85301020f0 0x7f853019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852dd9b700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853019ed90 0x7f8530198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f852ed9d700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 0x7f8530199400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f852800ba50 tx=0x7f852800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528019070 con 0x7f8530198f50 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f852800ce40 con 0x7f8530198f50 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85301a5950 con 0x7f8530198f50 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528007350 con 0x7f8530198f50 2024-05-20T20:46:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.428+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85301a5e90 con 0x7f8530198f50 2024-05-20T20:46:37.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.429+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f852801c070 con 0x7f8530198f50 2024-05-20T20:46:37.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.429+0000 7f851f7fe700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f851803e9a0 0x7f8518040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:37.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.429+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8530064ba0 con 0x7f8530198f50 2024-05-20T20:46:37.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.429+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8528014440 con 0x7f8530198f50 2024-05-20T20:46:37.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.429+0000 7f852e59c700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f851803e9a0 0x7f8518040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:37.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.430+0000 7f852e59c700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f851803e9a0 0x7f8518040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8520005920 tx=0x7f852001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:37.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.435+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f852801cc60 con 0x7f8530198f50 2024-05-20T20:46:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.652+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f85301a6120 con 0x7f8530198f50 2024-05-20T20:46:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.652+0000 7f851f7fe700 1 -- 172.21.15.26:0/1902326395 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f85280522d0 con 0x7f8530198f50 2024-05-20T20:46:37.652 INFO:teuthology.orchestra.run.smithi026.stdout:# minimal ceph.conf for 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:37.652 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-05-20T20:46:37.652 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:46:37.652 INFO:teuthology.orchestra.run.smithi026.stdout: mon_host = [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-05-20T20:46:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f851803e9a0 msgr2=0x7f8518040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f851803e9a0 0x7f8518040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8520005920 tx=0x7f852001a040 comp rx=0 tx=0).stop 2024-05-20T20:46:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 msgr2=0x7f8530199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 0x7f8530199400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f852800ba50 tx=0x7f852800bd60 comp rx=0 tx=0).stop 2024-05-20T20:46:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 shutdown_connections 2024-05-20T20:46:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f851803e9a0 0x7f8518040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8530198f50 0x7f8530199400 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85301020f0 0x7f853019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 --2- 172.21.15.26:0/1902326395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853019ed90 0x7f8530198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 >> 172.21.15.26:0/1902326395 conn(0x7f85300779d0 msgr2=0x7f853006aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 shutdown_connections 2024-05-20T20:46:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:37.655+0000 7f853499f700 1 -- 172.21.15.26:0/1902326395 wait complete. 2024-05-20T20:46:38.202 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-20T20:46:38.202 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:46:38.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: Updating smithi055:/etc/ceph/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: Updating smithi055:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: Updating smithi026:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: Updating smithi144:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:38.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:37 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1902326395' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:38.233 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:46:38.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:46:38.300 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:46:38.300 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.conf 2024-05-20T20:46:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-20T20:46:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: Updating smithi055:/etc/ceph/ceph.conf 2024-05-20T20:46:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: Updating smithi055:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: Updating smithi026:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: Updating smithi144:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:38.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:38.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:38.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:37 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1902326395' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: Updating smithi055:/etc/ceph/ceph.conf 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: Updating smithi055:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: Updating smithi026:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: Updating smithi144:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:38.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:37 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1902326395' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:38.339 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:46:38.339 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:46:38.411 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:46:38.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-05-20T20:46:38.444 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:46:38.444 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T20:46:38.513 INFO:tasks.cephadm:Adding mgr.x on smithi026 2024-05-20T20:46:38.513 INFO:tasks.cephadm:Adding mgr.y on smithi055 2024-05-20T20:46:38.514 INFO:tasks.cephadm:Adding mgr.z on smithi144 2024-05-20T20:46:38.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi026=x;smithi055=y;smithi144=z' 2024-05-20T20:46:38.791 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:46:39.194 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:38 smithi144 ceph-mon[35660]: Reconfiguring mon.a (unknown last config time)... 2024-05-20T20:46:39.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:38 smithi144 ceph-mon[35660]: Reconfiguring daemon mon.a on smithi026 2024-05-20T20:46:39.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:38 smithi144 ceph-mon[35660]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:39.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:38 smithi026 ceph-mon[31990]: Reconfiguring mon.a (unknown last config time)... 2024-05-20T20:46:39.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:38 smithi026 ceph-mon[31990]: Reconfiguring daemon mon.a on smithi026 2024-05-20T20:46:39.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:38 smithi026 ceph-mon[31990]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:38 smithi055 ceph-mon[35213]: Reconfiguring mon.a (unknown last config time)... 2024-05-20T20:46:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:38 smithi055 ceph-mon[35213]: Reconfiguring daemon mon.a on smithi026 2024-05-20T20:46:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:38 smithi055 ceph-mon[35213]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:40.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.097+0000 7f37d3a2e700 1 -- 172.21.15.144:0/3348923845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc101fa0 msgr2=0x7f37cc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:40.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.097+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/3348923845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc101fa0 0x7f37cc1023c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b0a0 tx=0x7f37c800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.097+0000 7f37d3a2e700 1 -- 172.21.15.144:0/3348923845 shutdown_connections 2024-05-20T20:46:40.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.097+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/3348923845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc101fa0 0x7f37cc1023c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.097+0000 7f37d3a2e700 1 -- 172.21.15.144:0/3348923845 >> 172.21.15.144:0/3348923845 conn(0x7f37cc0fd540 msgr2=0x7f37cc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.097+0000 7f37d3a2e700 1 -- 172.21.15.144:0/3348923845 shutdown_connections 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 -- 172.21.15.144:0/3348923845 wait complete. 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 Processor -- start 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 -- start start 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 0x7f37cc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc198c10 0x7f37cc19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc199090 0x7f37cc1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc107d40 con 0x7f37cc199090 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc107bc0 con 0x7f37cc198c10 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.098+0000 7f37d3a2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc107ec0 con 0x7f37cc101fa0 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 0x7f37cc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 0x7f37cc1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39258/0 (socket says 172.21.15.144:39258) 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 -- 172.21.15.144:0/755487538 learned_addr learned my addr 172.21.15.144:0/755487538 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d1fcb700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc199090 0x7f37cc1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d0fc9700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc198c10 0x7f37cc19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 -- 172.21.15.144:0/755487538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc198c10 msgr2=0x7f37cc19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc198c10 0x7f37cc19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 -- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc199090 msgr2=0x7f37cc1a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc199090 0x7f37cc1a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 -- 172.21.15.144:0/755487538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1a0ae0 con 0x7f37cc101fa0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d17ca700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 0x7f37cc1986d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b830 tx=0x7f37c800bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c801f020 con 0x7f37cc101fa0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1a0d70 con 0x7f37cc101fa0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37c8005700 con 0x7f37cc101fa0 2024-05-20T20:46:40.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8017040 con 0x7f37cc101fa0 2024-05-20T20:46:40.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.099+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1a1290 con 0x7f37cc101fa0 2024-05-20T20:46:40.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.100+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f37c8022020 con 0x7f37cc101fa0 2024-05-20T20:46:40.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.100+0000 7f37c27fc700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b803e9a0 0x7f37b8040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:40.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.100+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc064ba0 con 0x7f37cc101fa0 2024-05-20T20:46:40.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.100+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f37c800f020 con 0x7f37cc101fa0 2024-05-20T20:46:40.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.101+0000 7f37d0fc9700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b803e9a0 0x7f37b8040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:40.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.101+0000 7f37d0fc9700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b803e9a0 0x7f37b8040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009ad0 tx=0x7f37bc009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:40.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.106+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37c8005270 con 0x7f37cc101fa0 2024-05-20T20:46:40.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.320+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi055=y;smithi144=z", "target": ["mon-mgr", ""]}) -- 0x7f37cc063980 con 0x7f37b803e9a0 2024-05-20T20:46:40.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.327+0000 7f37c27fc700 1 -- 172.21.15.144:0/755487538 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f37cc063980 con 0x7f37b803e9a0 2024-05-20T20:46:40.327 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mgr update... 2024-05-20T20:46:40.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b803e9a0 msgr2=0x7f37b8040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:40.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b803e9a0 0x7f37b8040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009ad0 tx=0x7f37bc009210 comp rx=0 tx=0).stop 2024-05-20T20:46:40.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 msgr2=0x7f37cc1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:40.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 0x7f37cc1986d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b830 tx=0x7f37c800bd10 comp rx=0 tx=0).stop 2024-05-20T20:46:40.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 shutdown_connections 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b803e9a0 0x7f37b8040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc199090 0x7f37cc1a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc101fa0 0x7f37cc1986d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 --2- 172.21.15.144:0/755487538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc198c10 0x7f37cc19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 >> 172.21.15.144:0/755487538 conn(0x7f37cc0fd540 msgr2=0x7f37cc06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 shutdown_connections 2024-05-20T20:46:40.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:46:40.329+0000 7f37d3a2e700 1 -- 172.21.15.144:0/755487538 wait complete. 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: Reconfiguring mon.b (monmap changed)... 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: Reconfiguring daemon mon.b on smithi055 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: Reconfiguring mon.b (monmap changed)... 2024-05-20T20:46:40.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: Reconfiguring daemon mon.b on smithi055 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: Reconfiguring mon.b (monmap changed)... 2024-05-20T20:46:40.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:40.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:40.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:40.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: Reconfiguring daemon mon.b on smithi055 2024-05-20T20:46:40.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:40.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:40.926 DEBUG:teuthology.orchestra.run.smithi055:mgr.y> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-20T20:46:40.928 DEBUG:teuthology.orchestra.run.smithi144:mgr.z> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-20T20:46:40.931 INFO:tasks.cephadm:Deploying OSDs... 2024-05-20T20:46:40.931 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:46:40.931 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T20:46:40.952 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T20:46:40.952 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-20 20:45:47.446034450 +0000 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-20 20:40:15.473893634 +0000 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-20 20:40:15.473893634 +0000 2024-05-20T20:46:41.012 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-20 20:40:15.473893634 +0000 2024-05-20T20:46:41.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T20:46:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-20T20:46:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-20T20:46:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000123185 s, 4.2 MB/s 2024-05-20T20:46:41.083 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T20:46:41.143 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-05-20T20:46:41.201 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T20:46:41.201 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:41.201 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-20T20:46:41.201 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:41.202 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:41.202 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-20 20:45:47.483033881 +0000 2024-05-20T20:46:41.202 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-20 20:40:15.809888422 +0000 2024-05-20T20:46:41.202 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-20 20:40:15.809888422 +0000 2024-05-20T20:46:41.202 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-20 20:40:15.809888422 +0000 2024-05-20T20:46:41.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T20:46:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-20T20:46:41.270 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-20T20:46:41.270 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000122527 s, 4.2 MB/s 2024-05-20T20:46:41.271 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T20:46:41.330 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-20 20:45:47.516033373 +0000 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-20 20:40:16.153883086 +0000 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-20 20:40:16.153883086 +0000 2024-05-20T20:46:41.388 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-20 20:40:16.153883086 +0000 2024-05-20T20:46:41.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T20:46:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-20T20:46:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-20T20:46:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000114408 s, 4.5 MB/s 2024-05-20T20:46:41.456 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T20:46:41.515 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-05-20T20:46:41.573 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T20:46:41.573 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:41.573 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-20T20:46:41.574 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:41.574 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:41.574 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-20 20:45:47.548032882 +0000 2024-05-20T20:46:41.574 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-20 20:40:16.483877967 +0000 2024-05-20T20:46:41.574 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-20 20:40:16.483877967 +0000 2024-05-20T20:46:41.574 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-20 20:40:16.483877967 +0000 2024-05-20T20:46:41.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T20:46:41.641 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-20T20:46:41.666 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-20T20:46:41.666 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.00012383 s, 4.1 MB/s 2024-05-20T20:46:41.667 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T20:46:41.726 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T20:46:41.727 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T20:46:41.747 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T20:46:41.747 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2024-05-20T20:46:41.807 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 20:46:29.235301987 +0000 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 20:40:16.828832548 +0000 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 20:40:16.828832548 +0000 2024-05-20T20:46:41.808 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 20:40:16.828832548 +0000 2024-05-20T20:46:41.808 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T20:46:41.879 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T20:46:41.879 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T20:46:41.879 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000312093 s, 1.6 MB/s 2024-05-20T20:46:41.880 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T20:46:41.940 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 20:46:29.235301987 +0000 2024-05-20T20:46:41.999 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 20:40:17.207825278 +0000 2024-05-20T20:46:42.000 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 20:40:17.207825278 +0000 2024-05-20T20:46:42.000 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 20:40:17.207825278 +0000 2024-05-20T20:46:42.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T20:46:42.067 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T20:46:42.067 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T20:46:42.067 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000309561 s, 1.7 MB/s 2024-05-20T20:46:42.069 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T20:46:42.128 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2024-05-20T20:46:42.186 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T20:46:42.186 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:42.186 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-20T20:46:42.186 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:42.186 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:42.187 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 20:46:29.235301987 +0000 2024-05-20T20:46:42.187 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 20:40:17.568818353 +0000 2024-05-20T20:46:42.187 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 20:40:17.568818353 +0000 2024-05-20T20:46:42.187 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 20:40:17.568818353 +0000 2024-05-20T20:46:42.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi055=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: Saving service mgr spec with placement smithi026=x;smithi055=y;smithi144=z;count:3 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: Reconfiguring mon.c (monmap changed)... 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:42.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:41 smithi055 ceph-mon[35213]: Reconfiguring daemon mon.c on smithi144 2024-05-20T20:46:42.227 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T20:46:42.227 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T20:46:42.227 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000443951 s, 1.2 MB/s 2024-05-20T20:46:42.228 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T20:46:42.287 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi055=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: Saving service mgr spec with placement smithi026=x;smithi055=y;smithi144=z;count:3 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: Reconfiguring mon.c (monmap changed)... 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:42.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:42.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:41 smithi144 ceph-mon[35660]: Reconfiguring daemon mon.c on smithi144 2024-05-20T20:46:42.344 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T20:46:42.344 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 20:46:29.236301968 +0000 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 20:40:17.917811659 +0000 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 20:40:17.917811659 +0000 2024-05-20T20:46:42.345 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 20:40:17.917811659 +0000 2024-05-20T20:46:42.345 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T20:46:42.411 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T20:46:42.411 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T20:46:42.411 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.00029793 s, 1.7 MB/s 2024-05-20T20:46:42.413 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T20:46:42.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi055=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:42.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: Saving service mgr spec with placement smithi026=x;smithi055=y;smithi144=z;count:3 2024-05-20T20:46:42.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:42.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:42.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: Reconfiguring mon.c (monmap changed)... 2024-05-20T20:46:42.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T20:46:42.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T20:46:42.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:42.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:41 smithi026 ceph-mon[31990]: Reconfiguring daemon mon.c on smithi144 2024-05-20T20:46:42.471 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-20T20:46:42.472 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T20:46:42.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T20:46:42.488 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-05-20T20:46:42.543 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-20 20:46:20.136110025 +0000 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-20 20:40:09.121304459 +0000 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-20 20:40:09.121304459 +0000 2024-05-20T20:46:42.544 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-20 20:40:09.121304459 +0000 2024-05-20T20:46:42.544 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T20:46:42.608 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-20T20:46:42.608 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-20T20:46:42.608 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000216701 s, 2.4 MB/s 2024-05-20T20:46:42.609 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T20:46:42.666 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-05-20T20:46:42.722 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T20:46:42.722 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:42.722 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-20T20:46:42.722 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:42.723 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:42.723 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-20 20:46:20.137110008 +0000 2024-05-20T20:46:42.723 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-20 20:40:09.460298450 +0000 2024-05-20T20:46:42.723 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-20 20:40:09.460298450 +0000 2024-05-20T20:46:42.723 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-20 20:40:09.460298450 +0000 2024-05-20T20:46:42.723 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T20:46:42.786 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-20T20:46:42.786 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-20T20:46:42.786 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000171911 s, 3.0 MB/s 2024-05-20T20:46:42.787 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T20:46:42.843 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-05-20T20:46:42.898 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-20 20:46:20.137110008 +0000 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-20 20:40:09.785292689 +0000 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-20 20:40:09.785292689 +0000 2024-05-20T20:46:42.899 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-20 20:40:09.785292689 +0000 2024-05-20T20:46:42.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T20:46:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-20T20:46:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-20T20:46:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.00019487 s, 2.6 MB/s 2024-05-20T20:46:42.964 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T20:46:43.020 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-20 20:46:20.137110008 +0000 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-20 20:40:10.139286413 +0000 2024-05-20T20:46:43.076 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-20 20:40:10.139286413 +0000 2024-05-20T20:46:43.077 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-20 20:40:10.139286413 +0000 2024-05-20T20:46:43.077 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T20:46:43.140 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-20T20:46:43.141 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-20T20:46:43.141 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000202797 s, 2.5 MB/s 2024-05-20T20:46:43.142 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T20:46:43.199 INFO:tasks.cephadm:Deploying osd.0 on smithi026 with /dev/vg_nvme/lv_4... 2024-05-20T20:46:43.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-20T20:46:43.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:42 smithi055 ceph-mon[35213]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:42 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:42 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:42 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:42 smithi144 ceph-mon[35660]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:42 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:42 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:42 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:43.438 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:46:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:42 smithi026 ceph-mon[31990]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:42 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:42 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:42 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:44.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:44.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:43 smithi026 ceph-mon[31990]: Deploying daemon mgr.z on smithi144 2024-05-20T20:46:44.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:44.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:44.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:44.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:44.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T20:46:44.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:44.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:44.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:43 smithi144 ceph-mon[35660]: Deploying daemon mgr.z on smithi144 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:43 smithi055 ceph-mon[35213]: Deploying daemon mgr.z on smithi144 2024-05-20T20:46:45.182 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-20T20:46:45.182 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-20T20:46:45.182 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-20T20:46:45.182 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-20T20:46:45.182 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00824239 s, 1.3 GB/s 2024-05-20T20:46:45.182 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-20T20:46:45.243 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:45 smithi144 ceph-mon[35660]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:45 smithi055 ceph-mon[35213]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:45.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:45 smithi026 ceph-mon[31990]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:45.511 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:45 smithi144 systemd[1]: Started Ceph mgr.z for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:46:45.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-05-20T20:46:45.800 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:45.528+0000 7f7628e20200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T20:46:45.800 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:45.726+0000 7f7628e20200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:46:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:46:46.157 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:45.834+0000 7f7628e20200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T20:46:46.157 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:46.157+0000 7f7628e20200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T20:46:46.481 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:46.245+0000 7f7628e20200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T20:46:46.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: Deploying daemon mgr.y on smithi055 2024-05-20T20:46:46.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:46 smithi026 ceph-mon[31990]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:46.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: Deploying daemon mgr.y on smithi055 2024-05-20T20:46:46.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:46 smithi144 ceph-mon[35660]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:46.805 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: Deploying daemon mgr.y on smithi055 2024-05-20T20:46:46.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:46 smithi055 ceph-mon[35213]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- 172.21.15.26:0/1858139128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141047a0 msgr2=0x7f7314104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- 172.21.15.26:0/1858139128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141047a0 0x7f7314104c00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73000099c0 tx=0x7f7300009cd0 comp rx=0 tx=0).stop 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- 172.21.15.26:0/1858139128 shutdown_connections 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- 172.21.15.26:0/1858139128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314105140 0x7f731410b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- 172.21.15.26:0/1858139128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141035a0 0x7f73141039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- 172.21.15.26:0/1858139128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141047a0 0x7f7314104c00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- 172.21.15.26:0/1858139128 >> 172.21.15.26:0/1858139128 conn(0x7f73140feb40 msgr2=0x7f7314100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- 172.21.15.26:0/1858139128 shutdown_connections 2024-05-20T20:46:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- 172.21.15.26:0/1858139128 wait complete. 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 Processor -- start 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- start start 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 0x7f7314077290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141047a0 0x7f7314075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314105140 0x7f7314075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.490+0000 7f731b239700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f731410ef90 con 0x7f73141035a0 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f731b239700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f731410ee10 con 0x7f73141047a0 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f731b239700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f731410f110 con 0x7f7314105140 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 0x7f7314077290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7313fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141047a0 0x7f7314075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 0x7f7314077290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53084/0 (socket says 172.21.15.26:53084) 2024-05-20T20:46:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 -- 172.21.15.26:0/2907081655 learned_addr learned my addr 172.21.15.26:0/2907081655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f73197d6700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314105140 0x7f7314075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 -- 172.21.15.26:0/2907081655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314105140 msgr2=0x7f7314075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314105140 0x7f7314075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 -- 172.21.15.26:0/2907081655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141047a0 msgr2=0x7f7314075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141047a0 0x7f7314075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 -- 172.21.15.26:0/2907081655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7314076740 con 0x7f73141035a0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7313fff700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141047a0 0x7f7314075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7318fd5700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 0x7f7314077290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f730800b500 tx=0x7f730800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308015020 con 0x7f73141035a0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308011cc0 con 0x7f73141035a0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.491+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73141a3800 con 0x7f73141035a0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.492+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308010c10 con 0x7f73141035a0 2024-05-20T20:46:47.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.492+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141a3d60 con 0x7f73141035a0 2024-05-20T20:46:47.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.492+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7308018020 con 0x7f73141035a0 2024-05-20T20:46:47.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.493+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731406a230 con 0x7f73141035a0 2024-05-20T20:46:47.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.493+0000 7f7311ffb700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f730403e750 0x7f7304040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:46:47.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.493+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7308052120 con 0x7f73141035a0 2024-05-20T20:46:47.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.493+0000 7f7313fff700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f730403e750 0x7f7304040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:46:47.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.493+0000 7f7313fff700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f730403e750 0x7f7304040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7300000c00 tx=0x7f730001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:46:47.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.498+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f730801f020 con 0x7f73141035a0 2024-05-20T20:46:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:47.722+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7314063980 con 0x7f730403e750 2024-05-20T20:46:47.760 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:47 smithi055 podman[36575]: 2024-05-20 20:46:47.761322554 +0000 UTC m=+0.157383070 container create b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True) 2024-05-20T20:46:47.785 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:47.449+0000 7f7628e20200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T20:46:47.785 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:47.615+0000 7f7628e20200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T20:46:47.785 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:47.785+0000 7f7628e20200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T20:46:48.062 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:47 smithi055 podman[36575]: 2024-05-20 20:46:47.971914859 +0000 UTC m=+0.367975374 container init b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-20T20:46:48.063 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:47 smithi055 podman[36575]: 2024-05-20 20:46:47.974673292 +0000 UTC m=+0.370733805 container start b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-20T20:46:48.063 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mgr[36596]: -- 172.21.15.55:0/837419288 <== mon.2 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604cb0665a0 con 0x5604cb03a800 2024-05-20T20:46:48.331 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 bash[36575]: b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 2024-05-20T20:46:48.332 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 systemd[1]: Started Ceph mgr.y for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:46:48.332 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:48.135+0000 7fe6982ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T20:46:48.332 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:48.331+0000 7fe6982ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:46:48.366 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:48.069+0000 7f7628e20200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T20:46:48.366 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:48.160+0000 7f7628e20200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T20:46:48.366 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:48.366+0000 7f7628e20200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T20:46:48.665 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:48.567+0000 7f7628e20200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:46:48.669 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:48.438+0000 7fe6982ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:48.774+0000 7fe6982ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:48.868+0000 7fe6982ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:49.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:49.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:49.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:49.499 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:49.237+0000 7f7628e20200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T20:46:49.800 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:49.463+0000 7f7628e20200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-20T20:46:50.300 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:49.922+0000 7f7628e20200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T20:46:50.300 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:50.021+0000 7f7628e20200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T20:46:50.451 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:50.192+0000 7fe6982ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T20:46:50.451 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:50.371+0000 7fe6982ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T20:46:50.663 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:50.396+0000 7f7628e20200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T20:46:50.852 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:50.548+0000 7fe6982ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T20:46:50.852 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:50.851+0000 7fe6982ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:50.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: Reconfiguring mgr.x (unknown last config time)... 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: Reconfiguring daemon mgr.x on smithi026 2024-05-20T20:46:50.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3983239758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2c397ed-d333-4e7b-ba99-70756dd5fe29"}]: dispatch 2024-05-20T20:46:50.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3983239758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2c397ed-d333-4e7b-ba99-70756dd5fe29"}]': finished 2024-05-20T20:46:50.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: osdmap e5: 1 total, 0 up, 1 in 2024-05-20T20:46:50.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:46:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: Reconfiguring mgr.x (unknown last config time)... 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: Reconfiguring daemon mgr.x on smithi026 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3983239758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2c397ed-d333-4e7b-ba99-70756dd5fe29"}]: dispatch 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3983239758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2c397ed-d333-4e7b-ba99-70756dd5fe29"}]': finished 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: osdmap e5: 1 total, 0 up, 1 in 2024-05-20T20:46:50.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:46:51.050 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:50.751+0000 7f7628e20200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T20:46:51.050 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:50.842+0000 7f7628e20200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: Reconfiguring mgr.x (unknown last config time)... 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: Reconfiguring daemon mgr.x on smithi026 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3983239758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2c397ed-d333-4e7b-ba99-70756dd5fe29"}]: dispatch 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3983239758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2c397ed-d333-4e7b-ba99-70756dd5fe29"}]': finished 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: osdmap e5: 1 total, 0 up, 1 in 2024-05-20T20:46:51.052 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:46:51.159 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:50.947+0000 7fe6982ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T20:46:51.159 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:51.158+0000 7fe6982ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T20:46:51.451 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:51.360+0000 7fe6982ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T20:46:51.550 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:51.084+0000 7f7628e20200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T20:46:51.550 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:51.218+0000 7f7628e20200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T20:46:51.550 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:51.313+0000 7f7628e20200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T20:46:51.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:51 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3284221112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:46:51.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:51.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:51 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3284221112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:46:52.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:52.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:51 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3284221112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:46:52.050 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:51.642+0000 7f7628e20200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T20:46:52.051 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:51.734+0000 7f7628e20200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T20:46:52.451 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:51.996+0000 7fe6982ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T20:46:52.451 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:52.226+0000 7fe6982ca200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-20T20:46:52.666 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:52.332+0000 7f7628e20200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T20:46:52.666 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 20:46:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z[37495]: 2024-05-20T20:46:52.548+0000 7f7628e20200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T20:46:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:52.678+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f730801aca0 con 0x7f73141035a0 2024-05-20T20:46:52.951 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:52.670+0000 7fe6982ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T20:46:52.951 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:52.771+0000 7fe6982ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T20:46:52.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:52 smithi055 ceph-mon[35213]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:52.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:52 smithi055 ceph-mon[35213]: Standby manager daemon z started 2024-05-20T20:46:52.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:52 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:52 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:52 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:52 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:52 smithi026 ceph-mon[31990]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:52 smithi026 ceph-mon[31990]: Standby manager daemon z started 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:52 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:52 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:52 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-20T20:46:52.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:52 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T20:46:53.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:52 smithi144 ceph-mon[35660]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:53.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:52 smithi144 ceph-mon[35660]: Standby manager daemon z started 2024-05-20T20:46:53.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:52 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-20T20:46:53.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:52 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T20:46:53.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:52 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-20T20:46:53.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:52 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.144:0/2918762686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T20:46:53.450 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:53.172+0000 7fe6982ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T20:46:53.901 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:53 smithi055 ceph-mon[35213]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-20T20:46:53.901 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-20T20:46:53.901 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:53.547+0000 7fe6982ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T20:46:53.901 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:53.643+0000 7fe6982ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T20:46:53.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:53 smithi026 ceph-mon[31990]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-20T20:46:53.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-20T20:46:54.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:53 smithi144 ceph-mon[35660]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-20T20:46:54.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-20T20:46:54.201 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:53.900+0000 7fe6982ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T20:46:54.201 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:54.041+0000 7fe6982ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T20:46:54.201 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:54.142+0000 7fe6982ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T20:46:54.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:54 smithi026 ceph-mon[31990]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:54.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:54 smithi055 ceph-mon[35213]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:54.951 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:54.511+0000 7fe6982ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T20:46:54.951 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:54.607+0000 7fe6982ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T20:46:55.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:54 smithi144 ceph-mon[35660]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:55.686 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:55.247+0000 7fe6982ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T20:46:55.686 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 20:46:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y[36592]: 2024-05-20T20:46:55.474+0000 7fe6982ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T20:46:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:46:55.696+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f730800fbb0 con 0x7f73141035a0 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: Standby manager daemon y started 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: Standby manager daemon y started 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T20:46:55.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:55.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:55.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:55.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.965 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: Standby manager daemon y started 2024-05-20T20:46:55.965 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-20T20:46:55.965 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T20:46:55.965 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.? 172.21.15.55:0/2884287036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:46:55.966 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:46:56.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:56 smithi026 ceph-mon[31990]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:56.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:56 smithi026 ceph-mon[31990]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-20T20:46:56.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-20T20:46:56.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:56 smithi055 ceph-mon[35213]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:56.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:56 smithi055 ceph-mon[35213]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-20T20:46:56.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-20T20:46:57.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:56 smithi144 ceph-mon[35660]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:57.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:56 smithi144 ceph-mon[35660]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-20T20:46:57.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-20T20:46:58.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:58 smithi026 ceph-mon[31990]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:58.758 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-20T20:46:58.758 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:58.758 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:46:58 smithi026 ceph-mon[31990]: Deploying daemon osd.0 on smithi026 2024-05-20T20:46:58.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:58 smithi055 ceph-mon[35213]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:58.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-20T20:46:58.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:58.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:46:58 smithi055 ceph-mon[35213]: Deploying daemon osd.0 on smithi026 2024-05-20T20:46:59.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:58 smithi144 ceph-mon[35660]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:46:59.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-20T20:46:59.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:46:59.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:46:58 smithi144 ceph-mon[35660]: Deploying daemon osd.0 on smithi026 2024-05-20T20:47:00.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:00 smithi026 ceph-mon[31990]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:01.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:00 smithi144 ceph-mon[35660]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:01.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:00 smithi055 ceph-mon[35213]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:02.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:02 smithi026 ceph-mon[31990]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:03.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:02 smithi144 ceph-mon[35660]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:03.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:02 smithi055 ceph-mon[35213]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:04.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:04 smithi144 ceph-mon[35660]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:04.651 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:04.651 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:04.651 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:04.651 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:04 smithi026 ceph-mon[31990]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:04 smithi055 ceph-mon[35213]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:06.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.378+0000 7f7311ffb700 1 -- 172.21.15.26:0/2907081655 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7314063980 con 0x7f730403e750 2024-05-20T20:47:06.379 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 0 on host 'smithi026' 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f730403e750 msgr2=0x7f7304040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f730403e750 0x7f7304040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7300000c00 tx=0x7f730001a040 comp rx=0 tx=0).stop 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 msgr2=0x7f7314077290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 0x7f7314077290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f730800b500 tx=0x7f730800b810 comp rx=0 tx=0).stop 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 shutdown_connections 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f730403e750 0x7f7304040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73141035a0 0x7f7314077290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314105140 0x7f7314075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 --2- 172.21.15.26:0/2907081655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73141047a0 0x7f7314075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 >> 172.21.15.26:0/2907081655 conn(0x7f73140feb40 msgr2=0x7f7314107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 shutdown_connections 2024-05-20T20:47:06.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:06.382+0000 7f731b239700 1 -- 172.21.15.26:0/2907081655 wait complete. 2024-05-20T20:47:06.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:06 smithi055 ceph-mon[35213]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:06.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:06 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:06 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:06 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:06 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:06 smithi026 ceph-mon[31990]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:06.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:06 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:06 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:06 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:06.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:06 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:06 smithi144 ceph-mon[35660]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:06 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:06 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:06 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:07.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:06 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:07.080 DEBUG:teuthology.orchestra.run.smithi026:osd.0> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.0.service 2024-05-20T20:47:07.082 INFO:tasks.cephadm:Deploying osd.1 on smithi055 with /dev/vg_nvme/lv_4... 2024-05-20T20:47:07.083 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-20T20:47:07.341 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:47:07.355 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0[46809]: 2024-05-20T20:47:07.159+0000 7f16c39417c0 -1 osd.0 0 log_to_monitors true 2024-05-20T20:47:07.735 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:07 smithi055 ceph-mon[35213]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-20T20:47:07.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:07 smithi026 ceph-mon[31990]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-20T20:47:08.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:07 smithi144 ceph-mon[35660]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-20T20:47:08.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:08 smithi055 ceph-mon[35213]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:08.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:08 smithi055 ceph-mon[35213]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-20T20:47:08.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:08 smithi055 ceph-mon[35213]: osdmap e6: 1 total, 0 up, 1 in 2024-05-20T20:47:08.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:08 smithi055 ceph-mon[35213]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:47:08.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:08.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:08 smithi026 ceph-mon[31990]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:08.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:08 smithi026 ceph-mon[31990]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-20T20:47:08.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:08 smithi026 ceph-mon[31990]: osdmap e6: 1 total, 0 up, 1 in 2024-05-20T20:47:08.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:08 smithi026 ceph-mon[31990]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:47:08.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:08 smithi144 ceph-mon[35660]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:08 smithi144 ceph-mon[35660]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-20T20:47:09.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:08 smithi144 ceph-mon[35660]: osdmap e6: 1 total, 0 up, 1 in 2024-05-20T20:47:09.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:08 smithi144 ceph-mon[35660]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:47:09.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.397 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-20T20:47:09.397 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-20T20:47:09.397 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-05-20T20:47:09.397 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-05-20T20:47:09.397 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0158002 s, 664 MB/s 2024-05-20T20:47:09.397 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-05-20T20:47:09.593 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:09 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0[46809]: 2024-05-20T20:47:09.227+0000 7f16b8fa5700 -1 osd.0 0 waiting for initial osdmap 2024-05-20T20:47:09.593 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:09 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0[46809]: 2024-05-20T20:47:09.230+0000 7f16b0d90700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:47:09.693 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:09 smithi055 ceph-mon[35213]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:47:09.693 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:09 smithi055 ceph-mon[35213]: osdmap e7: 1 total, 0 up, 1 in 2024-05-20T20:47:09.693 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.693 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.693 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:09 smithi055 ceph-mon[35213]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' 2024-05-20T20:47:09.693 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:09 smithi026 ceph-mon[31990]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:47:09.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:09 smithi026 ceph-mon[31990]: osdmap e7: 1 total, 0 up, 1 in 2024-05-20T20:47:09.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:09 smithi026 ceph-mon[31990]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' 2024-05-20T20:47:09.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:09.984 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_4 2024-05-20T20:47:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:09 smithi144 ceph-mon[35660]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:47:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:09 smithi144 ceph-mon[35660]: osdmap e7: 1 total, 0 up, 1 in 2024-05-20T20:47:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:10.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:09 smithi144 ceph-mon[35660]: from='osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364]' entity='osd.0' 2024-05-20T20:47:10.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:10.231 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:47:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:47:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:47:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi026 2024-05-20T20:47:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi026 to 17093M 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] boot 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: osdmap e8: 1 total, 1 up, 1 in 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi026 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi026 to 17093M 2024-05-20T20:47:11.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] boot 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: osdmap e8: 1 total, 1 up, 1 in 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:47:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:47:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi026 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi026 to 17093M 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: osd.0 [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] boot 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: osdmap e8: 1 total, 1 up, 1 in 2024-05-20T20:47:11.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T20:47:11.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:11.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:11.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:11.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.746+0000 7f5538704700 1 -- 172.21.15.55:0/493222963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5530105140 msgr2=0x7f553010b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:11.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 --2- 172.21.15.55:0/493222963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5530105140 0x7f553010b9f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f552800b0d0 tx=0x7f552800b3e0 comp rx=0 tx=0).stop 2024-05-20T20:47:11.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 -- 172.21.15.55:0/493222963 shutdown_connections 2024-05-20T20:47:11.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 --2- 172.21.15.55:0/493222963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55301035a0 0x7f55301039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:11.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 --2- 172.21.15.55:0/493222963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5530105140 0x7f553010b9f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:11.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 --2- 172.21.15.55:0/493222963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55301047a0 0x7f5530104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 -- 172.21.15.55:0/493222963 >> 172.21.15.55:0/493222963 conn(0x7f55300feb40 msgr2=0x7f5530100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 -- 172.21.15.55:0/493222963 shutdown_connections 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 -- 172.21.15.55:0/493222963 wait complete. 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 Processor -- start 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 -- start start 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55301035a0 0x7f55300772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.747+0000 7f5538704700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 0x7f5530075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5538704700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5530105140 0x7f5530075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5538704700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f553010eed0 con 0x7f5530105140 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5538704700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f553010ed50 con 0x7f55301047a0 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5538704700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f553010f050 con 0x7f55301035a0 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 0x7f5530075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:11.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 0x7f5530075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46222/0 (socket says 172.21.15.55:46222) 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 -- 172.21.15.55:0/3446255962 learned_addr learned my addr 172.21.15.55:0/3446255962 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f55364a0700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55301035a0 0x7f55300772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 -- 172.21.15.55:0/3446255962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55301035a0 msgr2=0x7f55300772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55301035a0 0x7f55300772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 -- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5530105140 msgr2=0x7f5530075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5530105140 0x7f5530075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 -- 172.21.15.55:0/3446255962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5530076720 con 0x7f55301047a0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5535c9f700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 0x7f5530075930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f552000c920 tx=0x7f552000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520007b00 con 0x7f55301047a0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5520004cc0 con 0x7f55301047a0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55301a3730 con 0x7f55301047a0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520005440 con 0x7f55301047a0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.748+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55301a3c70 con 0x7f55301047a0 2024-05-20T20:47:11.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.749+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553006a230 con 0x7f55301047a0 2024-05-20T20:47:11.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.751+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f5520004750 con 0x7f55301047a0 2024-05-20T20:47:11.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.752+0000 7f55277fe700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f551c0b2870 0x7f551c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:11.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.752+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f55200e01e0 con 0x7f55301047a0 2024-05-20T20:47:11.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.752+0000 7f55364a0700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f551c0b2870 0x7f551c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:11.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.753+0000 7f55364a0700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f551c0b2870 0x7f551c0b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5530104600 tx=0x7f552c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:47:11.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.757+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5520014030 con 0x7f55301047a0 2024-05-20T20:47:11.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:11.981+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5530063980 con 0x7f551c0b2870 2024-05-20T20:47:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:12 smithi055 ceph-mon[35213]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:12 smithi055 ceph-mon[35213]: osdmap e9: 1 total, 1 up, 1 in 2024-05-20T20:47:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:12 smithi055 ceph-mon[35213]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:47:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:47:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:47:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:12 smithi026 ceph-mon[31990]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:12 smithi026 ceph-mon[31990]: osdmap e9: 1 total, 1 up, 1 in 2024-05-20T20:47:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:12 smithi026 ceph-mon[31990]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:47:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:47:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:47:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:12 smithi144 ceph-mon[35660]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:12 smithi144 ceph-mon[35660]: osdmap e9: 1 total, 1 up, 1 in 2024-05-20T20:47:13.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:12 smithi144 ceph-mon[35660]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:47:13.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:47:13.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:47:13.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:14 smithi026 ceph-mon[31990]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:14 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/3417820529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]: dispatch 2024-05-20T20:47:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:14 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]: dispatch 2024-05-20T20:47:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:14 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]': finished 2024-05-20T20:47:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:14 smithi026 ceph-mon[31990]: osdmap e10: 2 total, 1 up, 2 in 2024-05-20T20:47:15.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:14 smithi144 ceph-mon[35660]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:14 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/3417820529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]: dispatch 2024-05-20T20:47:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:14 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]: dispatch 2024-05-20T20:47:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:14 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]': finished 2024-05-20T20:47:15.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:14 smithi144 ceph-mon[35660]: osdmap e10: 2 total, 1 up, 2 in 2024-05-20T20:47:15.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:14 smithi055 ceph-mon[35213]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:14 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/3417820529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]: dispatch 2024-05-20T20:47:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:14 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]: dispatch 2024-05-20T20:47:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:14 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d138e804-8e04-4eaf-a954-9ff798e9c4f0"}]': finished 2024-05-20T20:47:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:14 smithi055 ceph-mon[35213]: osdmap e10: 2 total, 1 up, 2 in 2024-05-20T20:47:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:16.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:15 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/1200323381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:47:16.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:15 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/1200323381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:47:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:15 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/1200323381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:47:17.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:16 smithi144 ceph-mon[35660]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:16 smithi055 ceph-mon[35213]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:16 smithi026 ceph-mon[31990]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:18 smithi055 ceph-mon[35213]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:18 smithi144 ceph-mon[35660]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:19.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:18 smithi026 ceph-mon[31990]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:21.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:20 smithi055 ceph-mon[35213]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:21.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:20 smithi144 ceph-mon[35660]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:20 smithi026 ceph-mon[31990]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:22.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-20T20:47:22.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:22.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-20T20:47:22.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:22.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-20T20:47:22.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:23.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:22 smithi055 ceph-mon[35213]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:23.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:22 smithi055 ceph-mon[35213]: Deploying daemon osd.1 on smithi055 2024-05-20T20:47:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:22 smithi144 ceph-mon[35660]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:22 smithi144 ceph-mon[35660]: Deploying daemon osd.1 on smithi055 2024-05-20T20:47:23.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:22 smithi026 ceph-mon[31990]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:23.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:22 smithi026 ceph-mon[31990]: Deploying daemon osd.1 on smithi055 2024-05-20T20:47:25.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:24 smithi055 ceph-mon[35213]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:24 smithi144 ceph-mon[35660]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:24 smithi026 ceph-mon[31990]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:27.100 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:26 smithi055 ceph-mon[35213]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:27.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:26 smithi026 ceph-mon[31990]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:26 smithi144 ceph-mon[35660]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:28.550 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:28.551 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:28.551 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:28.551 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:28 smithi026 ceph-mon[31990]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:28.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:28.603 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:28.603 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:28.603 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:28 smithi055 ceph-mon[35213]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:28 smithi144 ceph-mon[35660]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:30.744 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 1 on host 'smithi055' 2024-05-20T20:47:30.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.744+0000 7f55277fe700 1 -- 172.21.15.55:0/3446255962 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5530063980 con 0x7f551c0b2870 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f551c0b2870 msgr2=0x7f551c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f551c0b2870 0x7f551c0b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5530104600 tx=0x7f552c007400 comp rx=0 tx=0).stop 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 msgr2=0x7f5530075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 0x7f5530075930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f552000c920 tx=0x7f552000cc30 comp rx=0 tx=0).stop 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 shutdown_connections 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f551c0b2870 0x7f551c0b4d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5530105140 0x7f5530075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55301035a0 0x7f55300772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 --2- 172.21.15.55:0/3446255962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301047a0 0x7f5530075930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 >> 172.21.15.55:0/3446255962 conn(0x7f55300feb40 msgr2=0x7f5530107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 shutdown_connections 2024-05-20T20:47:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:47:30.747+0000 7f5538704700 1 -- 172.21.15.55:0/3446255962 wait complete. 2024-05-20T20:47:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:31.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:31.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:31.359 DEBUG:teuthology.orchestra.run.smithi055:osd.1> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.1.service 2024-05-20T20:47:31.361 INFO:tasks.cephadm:Deploying osd.2 on smithi144 with /dev/vg_nvme/lv_4... 2024-05-20T20:47:31.361 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-20T20:47:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:31.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:31.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:47:32.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:31 smithi144 ceph-mon[35660]: from='osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T20:47:32.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:31 smithi144 ceph-mon[35660]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T20:47:32.426 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T20:47:32.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:31 smithi055 ceph-mon[35213]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T20:47:32.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:31 smithi026 ceph-mon[31990]: from='osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T20:47:32.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:31 smithi026 ceph-mon[31990]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T20:47:33.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:33 smithi144 ceph-mon[35660]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:33.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:33 smithi144 ceph-mon[35660]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-20T20:47:33.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:33 smithi144 ceph-mon[35660]: osdmap e11: 2 total, 1 up, 2 in 2024-05-20T20:47:33.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:33.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:33 smithi144 ceph-mon[35660]: from='osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:47:33.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:33 smithi144 ceph-mon[35660]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:47:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:33 smithi055 ceph-mon[35213]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:33 smithi055 ceph-mon[35213]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-20T20:47:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:33 smithi055 ceph-mon[35213]: osdmap e11: 2 total, 1 up, 2 in 2024-05-20T20:47:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:33.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:33 smithi055 ceph-mon[35213]: from='osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:47:33.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:33 smithi055 ceph-mon[35213]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:47:33.452 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:33 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1[39775]: 2024-05-20T20:47:33.021+0000 7f29571f0700 -1 osd.1 0 waiting for initial osdmap 2024-05-20T20:47:33.452 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:33 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1[39775]: 2024-05-20T20:47:33.025+0000 7f2952fe3700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:47:33.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:33 smithi026 ceph-mon[31990]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:33.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:33 smithi026 ceph-mon[31990]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-20T20:47:33.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:33 smithi026 ceph-mon[31990]: osdmap e11: 2 total, 1 up, 2 in 2024-05-20T20:47:33.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:33.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:33 smithi026 ceph-mon[31990]: from='osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:47:33.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:33 smithi026 ceph-mon[31990]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:47:33.503 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-20T20:47:33.503 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-20T20:47:33.503 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-20T20:47:33.503 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-20T20:47:33.504 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111898 s, 937 MB/s 2024-05-20T20:47:33.504 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-20T20:47:34.006 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-05-20T20:47:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:34 smithi055 ceph-mon[35213]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:47:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:34 smithi055 ceph-mon[35213]: osdmap e12: 2 total, 1 up, 2 in 2024-05-20T20:47:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:34.252 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:47:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:34 smithi144 ceph-mon[35660]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:47:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:34 smithi144 ceph-mon[35660]: osdmap e12: 2 total, 1 up, 2 in 2024-05-20T20:47:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:34.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:34.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:34 smithi026 ceph-mon[31990]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:47:34.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:34 smithi026 ceph-mon[31990]: osdmap e12: 2 total, 1 up, 2 in 2024-05-20T20:47:34.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:34.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] boot 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: osdmap e13: 2 total, 2 up, 2 in 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:35.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] boot 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: osdmap e13: 2 total, 2 up, 2 in 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-20T20:47:35.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: osd.1 [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] boot 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: osdmap e13: 2 total, 2 up, 2 in 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:35.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:35.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.656+0000 7fd8bac1e700 1 -- 172.21.15.144:0/781985573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 msgr2=0x7fd8b410a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.656+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/781985573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b410a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00b0a0 tx=0x7fd8ac00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- 172.21.15.144:0/781985573 shutdown_connections 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/781985573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b410a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/781985573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b41024a0 0x7fd8b41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/781985573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b41036a0 0x7fd8b4103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- 172.21.15.144:0/781985573 >> 172.21.15.144:0/781985573 conn(0x7fd8b40fda40 msgr2=0x7fd8b40ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- 172.21.15.144:0/781985573 shutdown_connections 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- 172.21.15.144:0/781985573 wait complete. 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 Processor -- start 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- start start 2024-05-20T20:47:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b41024a0 0x7fd8b4199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b41036a0 0x7fd8b4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b41a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b410ddf0 con 0x7fd8b41036a0 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b410dc70 con 0x7fd8b41024a0 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.657+0000 7fd8bac1e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b410df70 con 0x7fd8b4104040 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b41a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b41a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46138/0 (socket says 172.21.15.144:46138) 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 -- 172.21.15.144:0/654769370 learned_addr learned my addr 172.21.15.144:0/654769370 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b89ba700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b41024a0 0x7fd8b4199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b3fff700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b41036a0 0x7fd8b4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 -- 172.21.15.144:0/654769370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b41024a0 msgr2=0x7fd8b4199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b41024a0 0x7fd8b4199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 -- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b41036a0 msgr2=0x7fd8b4199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b41036a0 0x7fd8b4199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 -- 172.21.15.144:0/654769370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41a11c0 con 0x7fd8b4104040 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b91bb700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b41a0b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac006070 tx=0x7fd8ac004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac021070 con 0x7fd8b4104040 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8ac007b80 con 0x7fd8b4104040 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41a1450 con 0x7fd8b4104040 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac004700 con 0x7fd8b4104040 2024-05-20T20:47:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.658+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a1900 con 0x7fd8b4104040 2024-05-20T20:47:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.659+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b406a230 con 0x7fd8b4104040 2024-05-20T20:47:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.661+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fd8ac003a00 con 0x7fd8b4104040 2024-05-20T20:47:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.661+0000 7fd8b1ffb700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a40b27f0 0x7fd8a40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.662+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd8ac00f020 con 0x7fd8b4104040 2024-05-20T20:47:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.662+0000 7fd8b89ba700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a40b27f0 0x7fd8a40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.662+0000 7fd8b89ba700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a40b27f0 0x7fd8a40b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005950 tx=0x7fd8a800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:47:35.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.665+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8ac02a020 con 0x7fd8b4104040 2024-05-20T20:47:35.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:35.877+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd8b41a1ad0 con 0x7fd8a40b27f0 2024-05-20T20:47:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:36 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi055 2024-05-20T20:47:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:36 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi055 to 17093M 2024-05-20T20:47:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:36 smithi144 ceph-mon[35660]: osdmap e14: 2 total, 2 up, 2 in 2024-05-20T20:47:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:47:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:47:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:36 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi055 2024-05-20T20:47:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:36 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi055 to 17093M 2024-05-20T20:47:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:36 smithi055 ceph-mon[35213]: osdmap e14: 2 total, 2 up, 2 in 2024-05-20T20:47:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:47:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:47:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:36.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:36 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi055 2024-05-20T20:47:36.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:36 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi055 to 17093M 2024-05-20T20:47:36.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:36 smithi026 ceph-mon[31990]: osdmap e14: 2 total, 2 up, 2 in 2024-05-20T20:47:36.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:47:36.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:47:36.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:37 smithi144 ceph-mon[35660]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:37 smithi144 ceph-mon[35660]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:47:37.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:37 smithi055 ceph-mon[35213]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:37.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:37 smithi055 ceph-mon[35213]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:47:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:37 smithi026 ceph-mon[31990]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:37 smithi026 ceph-mon[31990]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:47:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/111415606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]: dispatch 2024-05-20T20:47:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]: dispatch 2024-05-20T20:47:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]': finished 2024-05-20T20:47:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: osdmap e15: 3 total, 2 up, 3 in 2024-05-20T20:47:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:39 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/2539528196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/111415606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]: dispatch 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]: dispatch 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]': finished 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: osdmap e15: 3 total, 2 up, 3 in 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:39 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/2539528196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:47:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/111415606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]: dispatch 2024-05-20T20:47:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]: dispatch 2024-05-20T20:47:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7569bb5-2603-43fa-8f88-5bbba4b6e930"}]': finished 2024-05-20T20:47:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: osdmap e15: 3 total, 2 up, 3 in 2024-05-20T20:47:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:39 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/2539528196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:47:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:41 smithi055 ceph-mon[35213]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:41.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:41 smithi026 ceph-mon[31990]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:41 smithi144 ceph-mon[35660]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:43.310 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:43 smithi144 ceph-mon[35660]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:43 smithi055 ceph-mon[35213]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:43 smithi026 ceph-mon[31990]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:45 smithi055 ceph-mon[35213]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:45.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:45 smithi026 ceph-mon[31990]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:45 smithi144 ceph-mon[35660]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:46.092 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-20T20:47:46.092 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:46.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-20T20:47:46.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:46.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-20T20:47:46.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:47.149 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:47 smithi144 ceph-mon[35660]: Deploying daemon osd.2 on smithi144 2024-05-20T20:47:47.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:47 smithi144 ceph-mon[35660]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:47 smithi055 ceph-mon[35213]: Deploying daemon osd.2 on smithi144 2024-05-20T20:47:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:47 smithi055 ceph-mon[35213]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:47 smithi026 ceph-mon[31990]: Deploying daemon osd.2 on smithi144 2024-05-20T20:47:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:47 smithi026 ceph-mon[31990]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:49.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:49 smithi144 ceph-mon[35660]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:49.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:49 smithi055 ceph-mon[35213]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:49.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:49 smithi026 ceph-mon[31990]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:51 smithi026 ceph-mon[31990]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:51 smithi144 ceph-mon[35660]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:51 smithi055 ceph-mon[35213]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:53.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:53 smithi055 ceph-mon[35213]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:53 smithi026 ceph-mon[31990]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:53.463 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:53 smithi144 ceph-mon[35660]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:53.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.842+0000 7fd8b1ffb700 1 -- 172.21.15.144:0/654769370 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd8a800f910 con 0x7fd8a40b27f0 2024-05-20T20:47:53.842 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 2 on host 'smithi144' 2024-05-20T20:47:53.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a40b27f0 msgr2=0x7fd8a40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:53.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a40b27f0 0x7fd8a40b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005950 tx=0x7fd8a800a300 comp rx=0 tx=0).stop 2024-05-20T20:47:53.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 msgr2=0x7fd8b41a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:53.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b41a0b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac006070 tx=0x7fd8ac004d00 comp rx=0 tx=0).stop 2024-05-20T20:47:53.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 shutdown_connections 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b4104040 0x7fd8b41a0b20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b41024a0 0x7fd8b4199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a40b27f0 0x7fd8a40b4cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 --2- 172.21.15.144:0/654769370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b41036a0 0x7fd8b4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 >> 172.21.15.144:0/654769370 conn(0x7fd8b40fda40 msgr2=0x7fd8b406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 shutdown_connections 2024-05-20T20:47:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:47:53.844+0000 7fd8bac1e700 1 -- 172.21.15.144:0/654769370 wait complete. 2024-05-20T20:47:54.435 DEBUG:teuthology.orchestra.run.smithi144:osd.2> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.2.service 2024-05-20T20:47:54.437 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_3... 2024-05-20T20:47:54.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-20T20:47:54.695 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:54 smithi055 ceph-mon[35213]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:54.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:54.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T20:47:54.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:54 smithi026 ceph-mon[31990]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T20:47:54.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:54 smithi144 ceph-mon[35660]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T20:47:56.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:55 smithi026 ceph-mon[31990]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-20T20:47:56.169 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:55 smithi026 ceph-mon[31990]: osdmap e16: 3 total, 2 up, 3 in 2024-05-20T20:47:56.169 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:56.169 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:55 smithi026 ceph-mon[31990]: from='osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:47:56.169 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:55 smithi026 ceph-mon[31990]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:47:56.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:55 smithi055 ceph-mon[35213]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-20T20:47:56.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:55 smithi055 ceph-mon[35213]: osdmap e16: 3 total, 2 up, 3 in 2024-05-20T20:47:56.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:56.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:55 smithi055 ceph-mon[35213]: from='osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:47:56.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:55 smithi055 ceph-mon[35213]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:47:56.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:55 smithi144 ceph-mon[35660]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-20T20:47:56.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:55 smithi144 ceph-mon[35660]: osdmap e16: 3 total, 2 up, 3 in 2024-05-20T20:47:56.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:56.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:55 smithi144 ceph-mon[35660]: from='osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:47:56.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:55 smithi144 ceph-mon[35660]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:47:56.301 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2[40788]: 2024-05-20T20:47:55.813+0000 7fe27211a700 -1 osd.2 0 waiting for initial osdmap 2024-05-20T20:47:56.301 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2[40788]: 2024-05-20T20:47:55.818+0000 7fe26df0d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:47:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-20T20:47:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-20T20:47:56.779 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-20T20:47:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-20T20:47:56.779 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834869 s, 1.3 GB/s 2024-05-20T20:47:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-20T20:47:56.921 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:56 smithi144 ceph-mon[35660]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:56.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:56 smithi144 ceph-mon[35660]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:47:56.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:56 smithi144 ceph-mon[35660]: osdmap e17: 3 total, 2 up, 3 in 2024-05-20T20:47:56.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:56.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:56.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:56 smithi026 ceph-mon[31990]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:57.093 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:56 smithi026 ceph-mon[31990]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:47:57.093 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:56 smithi026 ceph-mon[31990]: osdmap e17: 3 total, 2 up, 3 in 2024-05-20T20:47:57.093 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.093 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.093 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.098 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:56 smithi055 ceph-mon[35213]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:57.098 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:56 smithi055 ceph-mon[35213]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:47:57.099 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:56 smithi055 ceph-mon[35213]: osdmap e17: 3 total, 2 up, 3 in 2024-05-20T20:47:57.099 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.099 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.099 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-05-20T20:47:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] boot 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: osdmap e18: 3 total, 3 up, 3 in 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:57.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:57.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:57.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:57.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:57.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:57.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] boot 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: osdmap e18: 3 total, 3 up, 3 in 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:58.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:58.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-20T20:47:58.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:47:58.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: osd.2 [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] boot 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: osdmap e18: 3 total, 3 up, 3 in 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:47:58.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-20T20:47:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:58 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi144 2024-05-20T20:47:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:58 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi144 to 17088M 2024-05-20T20:47:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:58 smithi026 ceph-mon[31990]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-20T20:47:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:58 smithi026 ceph-mon[31990]: osdmap e19: 3 total, 3 up, 3 in 2024-05-20T20:47:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T20:47:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.178+0000 7fcc000bc700 1 -- 172.21.15.26:0/1853888481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf8104060 msgr2=0x7fcbf810a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.178+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1853888481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf8104060 0x7fcbf810a910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000b0d0 tx=0x7fcbf000b3e0 comp rx=0 tx=0).stop 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.178+0000 7fcc000bc700 1 -- 172.21.15.26:0/1853888481 shutdown_connections 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.178+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1853888481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf8104060 0x7fcbf810a910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.178+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1853888481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf81024c0 0x7fcbf81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.178+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1853888481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf81036c0 0x7fcbf8103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- 172.21.15.26:0/1853888481 >> 172.21.15.26:0/1853888481 conn(0x7fcbf80fdaa0 msgr2=0x7fcbf80ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- 172.21.15.26:0/1853888481 shutdown_connections 2024-05-20T20:47:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- 172.21.15.26:0/1853888481 wait complete. 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 Processor -- start 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- start start 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf81024c0 0x7fcbf8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 0x7fcbf8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf8104060 0x7fcbf81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810de80 con 0x7fcbf81036c0 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810dd00 con 0x7fcbf8104060 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.179+0000 7fcc000bc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810e000 con 0x7fcbf81024c0 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 0x7fcbf8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 0x7fcbf8199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52010/0 (socket says 172.21.15.26:52010) 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 -- 172.21.15.26:0/1920518380 learned_addr learned my addr 172.21.15.26:0/1920518380 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:47:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfde58700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf81024c0 0x7fcbf8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfe659700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf8104060 0x7fcbf81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 -- 172.21.15.26:0/1920518380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf81024c0 msgr2=0x7fcbf8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf81024c0 0x7fcbf8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 -- 172.21.15.26:0/1920518380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf8104060 msgr2=0x7fcbf81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf8104060 0x7fcbf81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 -- 172.21.15.26:0/1920518380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf81a1290 con 0x7fcbf81036c0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbfd657700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 0x7fcbf8199b70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00b560 tx=0x7fcbec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec015020 con 0x7fcbf81036c0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbec011cc0 con 0x7fcbf81036c0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec010ca0 con 0x7fcbf81036c0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf81a1580 con 0x7fcbf81036c0 2024-05-20T20:47:59.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.180+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf81a1b90 con 0x7fcbf81036c0 2024-05-20T20:47:59.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.181+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf806a230 con 0x7fcbf81036c0 2024-05-20T20:47:59.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.183+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fcbec018020 con 0x7fcbf81036c0 2024-05-20T20:47:59.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.183+0000 7fcbeaffd700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbe40b2870 0x7fcbe40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:47:59.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.184+0000 7fcbfde58700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbe40b2870 0x7fcbe40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:47:59.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.184+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fcbec0e0900 con 0x7fcbf81036c0 2024-05-20T20:47:59.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.184+0000 7fcbfde58700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbe40b2870 0x7fcbe40b4d30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4005950 tx=0x7fcbf400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:47:59.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.187+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbec01f020 con 0x7fcbf81036c0 2024-05-20T20:47:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:58 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi144 2024-05-20T20:47:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:58 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi144 to 17088M 2024-05-20T20:47:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:58 smithi055 ceph-mon[35213]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-20T20:47:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:58 smithi055 ceph-mon[35213]: osdmap e19: 3 total, 3 up, 3 in 2024-05-20T20:47:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T20:47:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:58 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi144 2024-05-20T20:47:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:58 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi144 to 17088M 2024-05-20T20:47:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:58 smithi144 ceph-mon[35660]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-20T20:47:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-20T20:47:59.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:58 smithi144 ceph-mon[35660]: osdmap e19: 3 total, 3 up, 3 in 2024-05-20T20:47:59.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T20:47:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:47:59.427+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcbf80ffc90 con 0x7fcbe40b2870 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43433]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43433]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43433]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43436]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43436]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43436]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43436]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43427]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43427]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43427]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43430]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43430]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43430]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.702 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 20:47:59 smithi055 sudo[43430]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51482]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51485]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.703 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51485]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51555]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51555]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51555]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51555]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51558]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:47:59 smithi026 sudo[51558]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44363]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T20:47:59.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44363]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44363]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44363]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44366]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T20:47:59.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44366]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44366]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44366]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.801 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44357]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T20:47:59.801 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44357]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.801 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44357]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.801 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44357]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:47:59.802 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44360]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T20:47:59.802 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44360]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T20:47:59.802 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44360]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T20:47:59.802 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 20:47:59 smithi144 sudo[44360]: pam_unix(sudo:session): session closed for user root 2024-05-20T20:48:00.340 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-20T20:48:00.340 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: osdmap e20: 3 total, 3 up, 3 in 2024-05-20T20:48:00.340 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.341 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:00 smithi026 ceph-mon[31990]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:00.583+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcbec0a5670 con 0x7fcbf81036c0 2024-05-20T20:48:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-20T20:48:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: osdmap e20: 3 total, 3 up, 3 in 2024-05-20T20:48:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:00 smithi055 ceph-mon[35213]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-20T20:48:00.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: osdmap e20: 3 total, 3 up, 3 in 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T20:48:00.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:00 smithi144 ceph-mon[35660]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T20:48:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:01 smithi026 ceph-mon[31990]: osdmap e21: 3 total, 3 up, 3 in 2024-05-20T20:48:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:01 smithi026 ceph-mon[31990]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-20T20:48:01.651 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:01 smithi144 ceph-mon[35660]: osdmap e21: 3 total, 3 up, 3 in 2024-05-20T20:48:01.651 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:01 smithi144 ceph-mon[35660]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-20T20:48:01.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:01 smithi055 ceph-mon[35213]: osdmap e21: 3 total, 3 up, 3 in 2024-05-20T20:48:01.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:01 smithi055 ceph-mon[35213]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-20T20:48:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:02 smithi055 ceph-mon[35213]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:02 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4186769676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda9b415-a29a-489c-8d1c-2d893c0170e0"}]: dispatch 2024-05-20T20:48:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:02 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4186769676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eda9b415-a29a-489c-8d1c-2d893c0170e0"}]': finished 2024-05-20T20:48:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:02 smithi055 ceph-mon[35213]: osdmap e22: 4 total, 3 up, 4 in 2024-05-20T20:48:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:02 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:02.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:02 smithi026 ceph-mon[31990]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:02.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:02 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4186769676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda9b415-a29a-489c-8d1c-2d893c0170e0"}]: dispatch 2024-05-20T20:48:02.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:02 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4186769676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eda9b415-a29a-489c-8d1c-2d893c0170e0"}]': finished 2024-05-20T20:48:02.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:02 smithi026 ceph-mon[31990]: osdmap e22: 4 total, 3 up, 4 in 2024-05-20T20:48:02.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:02 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:02 smithi144 ceph-mon[35660]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:02 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4186769676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda9b415-a29a-489c-8d1c-2d893c0170e0"}]: dispatch 2024-05-20T20:48:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:02 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4186769676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eda9b415-a29a-489c-8d1c-2d893c0170e0"}]': finished 2024-05-20T20:48:02.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:02 smithi144 ceph-mon[35660]: osdmap e22: 4 total, 3 up, 4 in 2024-05-20T20:48:02.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:02 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:03 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3708450899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:03.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:03 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3708450899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:03.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:03 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3708450899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:04 smithi055 ceph-mon[35213]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:04.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:04 smithi026 ceph-mon[31990]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:04.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:04 smithi144 ceph-mon[35660]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:06.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:06 smithi026 ceph-mon[31990]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:06.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:06 smithi055 ceph-mon[35213]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:06 smithi144 ceph-mon[35660]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:08.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:08 smithi026 ceph-mon[31990]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:08.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:08 smithi055 ceph-mon[35213]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:08 smithi144 ceph-mon[35660]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:09.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-20T20:48:09.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:09.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:09 smithi055 ceph-mon[35213]: Deploying daemon osd.3 on smithi026 2024-05-20T20:48:09.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-20T20:48:09.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:09.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:09 smithi026 ceph-mon[31990]: Deploying daemon osd.3 on smithi026 2024-05-20T20:48:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-20T20:48:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:10.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:09 smithi144 ceph-mon[35660]: Deploying daemon osd.3 on smithi026 2024-05-20T20:48:10.747 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:10 smithi026 ceph-mon[31990]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:10.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:10 smithi055 ceph-mon[35213]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:10 smithi144 ceph-mon[35660]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:12.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:12 smithi055 ceph-mon[35213]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:12 smithi026 ceph-mon[31990]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:12 smithi144 ceph-mon[35660]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:14.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:14 smithi026 ceph-mon[31990]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:14 smithi144 ceph-mon[35660]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:15.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:14 smithi055 ceph-mon[35213]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:16.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:16.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:16.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:16 smithi144 ceph-mon[35660]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:17.117 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:16 smithi055 ceph-mon[35213]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:17.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:16 smithi026 ceph-mon[31990]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:18.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:18.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:18.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:18 smithi026 ceph-mon[31990]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.538+0000 7fcbeaffd700 1 -- 172.21.15.26:0/1920518380 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcbf80ffc90 con 0x7fcbe40b2870 2024-05-20T20:48:18.539 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.540+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbe40b2870 msgr2=0x7fcbe40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.540+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbe40b2870 0x7fcbe40b4d30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4005950 tx=0x7fcbf400b410 comp rx=0 tx=0).stop 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.540+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 msgr2=0x7fcbf8199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.540+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 0x7fcbf8199b70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00b560 tx=0x7fcbec00b870 comp rx=0 tx=0).stop 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 shutdown_connections 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf8104060 0x7fcbf81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf81024c0 0x7fcbf8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbe40b2870 0x7fcbe40b4d30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 --2- 172.21.15.26:0/1920518380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf81036c0 0x7fcbf8199b70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 >> 172.21.15.26:0/1920518380 conn(0x7fcbf80fdaa0 msgr2=0x7fcbf80ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:48:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 shutdown_connections 2024-05-20T20:48:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:48:18.541+0000 7fcc000bc700 1 -- 172.21.15.26:0/1920518380 wait complete. 2024-05-20T20:48:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:18 smithi055 ceph-mon[35213]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:18.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:18.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:18.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:18.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:18 smithi144 ceph-mon[35660]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:19.168 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.3.service 2024-05-20T20:48:19.171 INFO:tasks.cephadm:Deploying osd.4 on smithi055 with /dev/vg_nvme/lv_3... 2024-05-20T20:48:19.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-20T20:48:19.432 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:48:19.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:19.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:19.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:19.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:19 smithi144 ceph-mon[35660]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-20T20:48:19.804 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:19.805 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:19.805 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:19.805 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:19 smithi055 ceph-mon[35213]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-20T20:48:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:19 smithi026 ceph-mon[31990]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-20T20:48:20.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:20 smithi026 ceph-mon[31990]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-20T20:48:20.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:20 smithi026 ceph-mon[31990]: osdmap e23: 4 total, 3 up, 4 in 2024-05-20T20:48:20.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:20 smithi026 ceph-mon[31990]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:48:20.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:20 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:20.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:20 smithi026 ceph-mon[31990]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:20.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:20 smithi144 ceph-mon[35660]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-20T20:48:20.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:20 smithi144 ceph-mon[35660]: osdmap e23: 4 total, 3 up, 4 in 2024-05-20T20:48:20.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:20 smithi144 ceph-mon[35660]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:48:20.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:20 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:20.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:20 smithi144 ceph-mon[35660]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:20.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:20 smithi055 ceph-mon[35213]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-20T20:48:20.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:20 smithi055 ceph-mon[35213]: osdmap e23: 4 total, 3 up, 4 in 2024-05-20T20:48:20.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:20 smithi055 ceph-mon[35213]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:48:20.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:20 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:20.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:20 smithi055 ceph-mon[35213]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-20T20:48:21.402 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-20T20:48:21.402 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-20T20:48:21.403 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-05-20T20:48:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-05-20T20:48:21.403 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012725 s, 824 MB/s 2024-05-20T20:48:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-05-20T20:48:21.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 20:48:21 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T20:48:21.048+0000 7f7ce628a700 -1 osd.3 0 waiting for initial osdmap 2024-05-20T20:48:21.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 20:48:21 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T20:48:21.053+0000 7f7cde075700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:48:21.717 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:21 smithi055 ceph-mon[35213]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:48:21.717 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:21 smithi055 ceph-mon[35213]: osdmap e24: 4 total, 3 up, 4 in 2024-05-20T20:48:21.717 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:21.718 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:21.718 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:21 smithi055 ceph-mon[35213]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' 2024-05-20T20:48:21.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:21 smithi026 ceph-mon[31990]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:48:21.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:21 smithi026 ceph-mon[31990]: osdmap e24: 4 total, 3 up, 4 in 2024-05-20T20:48:21.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:21.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:21.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:21 smithi026 ceph-mon[31990]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' 2024-05-20T20:48:21.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:21 smithi144 ceph-mon[35660]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:48:21.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:21 smithi144 ceph-mon[35660]: osdmap e24: 4 total, 3 up, 4 in 2024-05-20T20:48:21.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:21.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:21.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:21 smithi144 ceph-mon[35660]: from='osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040]' entity='osd.3' 2024-05-20T20:48:22.112 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_3 2024-05-20T20:48:22.367 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:48:22.749 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:48:22.749 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] boot 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: osdmap e25: 4 total, 4 up, 4 in 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi026 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi026 to 8546M 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:22.751 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:22.751 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:48:22.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:48:22.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] boot 2024-05-20T20:48:22.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: osdmap e25: 4 total, 4 up, 4 in 2024-05-20T20:48:22.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi026 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi026 to 8546M 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:22.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:48:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:48:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: osd.3 [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] boot 2024-05-20T20:48:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: osdmap e25: 4 total, 4 up, 4 in 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi026 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi026 to 8546M 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:23.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:23 smithi144 ceph-mon[35660]: osdmap e26: 4 total, 4 up, 4 in 2024-05-20T20:48:23.804 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:23 smithi055 ceph-mon[35213]: osdmap e26: 4 total, 4 up, 4 in 2024-05-20T20:48:23.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:23 smithi026 ceph-mon[31990]: osdmap e26: 4 total, 4 up, 4 in 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.008+0000 7f9d26b3c700 1 -- 172.21.15.55:0/3147895618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 msgr2=0x7f9d20104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.008+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/3147895618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20104c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0099c0 tx=0x7f9d0c009cd0 comp rx=0 tx=0).stop 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.008+0000 7f9d26b3c700 1 -- 172.21.15.55:0/3147895618 shutdown_connections 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.008+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/3147895618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d201035c0 0x7f9d201039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.008+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/3147895618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d20105160 0x7f9d2010ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.008+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/3147895618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20104c20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- 172.21.15.55:0/3147895618 >> 172.21.15.55:0/3147895618 conn(0x7f9d200feb60 msgr2=0x7f9d20100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:48:24.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- 172.21.15.55:0/3147895618 shutdown_connections 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- 172.21.15.55:0/3147895618 wait complete. 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 Processor -- start 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- start start 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d201035c0 0x7f9d20199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d20105160 0x7f9d201a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010ef60 con 0x7f9d201035c0 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010ede0 con 0x7f9d201047c0 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d26b3c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010f0e0 con 0x7f9d20105160 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.009+0000 7f9d1ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44334/0 (socket says 172.21.15.55:44334) 2024-05-20T20:48:24.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 -- 172.21.15.55:0/1635580383 learned_addr learned my addr 172.21.15.55:0/1635580383 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d248d8700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d201035c0 0x7f9d20199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d250d9700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d20105160 0x7f9d201a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 -- 172.21.15.55:0/1635580383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d20105160 msgr2=0x7f9d201a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d20105160 0x7f9d201a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 -- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d201035c0 msgr2=0x7f9d20199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d201035c0 0x7f9d20199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 -- 172.21.15.55:0/1635580383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d201a1290 con 0x7f9d201047c0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1ffff700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20199b70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00b580 tx=0x7f9d0c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c027070 con 0x7f9d201047c0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d0c022470 con 0x7f9d201047c0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d201a1520 con 0x7f9d201047c0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c003a50 con 0x7f9d201047c0 2024-05-20T20:48:24.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.010+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d201a1a00 con 0x7f9d201047c0 2024-05-20T20:48:24.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.011+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2006a230 con 0x7f9d201047c0 2024-05-20T20:48:24.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.013+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9d0c02a070 con 0x7f9d201047c0 2024-05-20T20:48:24.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.014+0000 7f9d1dffb700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d100b2900 0x7f9d100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:24.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.014+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f9d0c0e23b0 con 0x7f9d201047c0 2024-05-20T20:48:24.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.014+0000 7f9d248d8700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d100b2900 0x7f9d100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:24.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.014+0000 7f9d248d8700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d100b2900 0x7f9d100b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d20104620 tx=0x7f9d1400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:48:24.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.017+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d0c013070 con 0x7f9d201047c0 2024-05-20T20:48:24.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:24.240+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9d20063a40 con 0x7f9d100b2900 2024-05-20T20:48:24.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:24 smithi144 ceph-mon[35660]: osdmap e27: 4 total, 4 up, 4 in 2024-05-20T20:48:24.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:24 smithi144 ceph-mon[35660]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:24.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:24.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:24.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:24.882 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:24 smithi055 ceph-mon[35213]: osdmap e27: 4 total, 4 up, 4 in 2024-05-20T20:48:24.882 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:24 smithi055 ceph-mon[35213]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:24.882 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:24.882 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:24.883 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:24 smithi026 ceph-mon[31990]: osdmap e27: 4 total, 4 up, 4 in 2024-05-20T20:48:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:24 smithi026 ceph-mon[31990]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:25.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:25 smithi144 ceph-mon[35660]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:25.829 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:25 smithi055 ceph-mon[35213]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:25.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:25 smithi026 ceph-mon[31990]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:26.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:26 smithi144 ceph-mon[35660]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-20T20:48:26.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:26 smithi055 ceph-mon[35213]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-20T20:48:26.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:26 smithi026 ceph-mon[31990]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-20T20:48:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:27 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/167339947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]: dispatch 2024-05-20T20:48:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:27 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]: dispatch 2024-05-20T20:48:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:27 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]': finished 2024-05-20T20:48:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:27 smithi055 ceph-mon[35213]: osdmap e28: 5 total, 4 up, 5 in 2024-05-20T20:48:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:27 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:27 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/167339947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]: dispatch 2024-05-20T20:48:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:27 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]: dispatch 2024-05-20T20:48:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:27 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]': finished 2024-05-20T20:48:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:27 smithi144 ceph-mon[35660]: osdmap e28: 5 total, 4 up, 5 in 2024-05-20T20:48:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:27 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:27.853 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:27 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/167339947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]: dispatch 2024-05-20T20:48:27.853 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:27 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]: dispatch 2024-05-20T20:48:27.853 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:27 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ca8decb-66db-4743-bcb4-ca3efeae40dd"}]': finished 2024-05-20T20:48:27.854 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:27 smithi026 ceph-mon[31990]: osdmap e28: 5 total, 4 up, 5 in 2024-05-20T20:48:27.854 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:27 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:28 smithi144 ceph-mon[35660]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:48:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:28 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/8438841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:28 smithi055 ceph-mon[35213]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:48:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:28 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/8438841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:28.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:28 smithi026 ceph-mon[31990]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:48:28.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:28 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/8438841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:30.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:30 smithi055 ceph-mon[35213]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-20T20:48:30.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:30 smithi026 ceph-mon[31990]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-20T20:48:31.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:30 smithi144 ceph-mon[35660]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-20T20:48:32.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:32 smithi055 ceph-mon[35213]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-20T20:48:32.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:32 smithi026 ceph-mon[31990]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-20T20:48:33.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:32 smithi144 ceph-mon[35660]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-20T20:48:34.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:34 smithi055 ceph-mon[35213]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:48:34.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-20T20:48:34.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:34 smithi026 ceph-mon[31990]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:48:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-20T20:48:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:35.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:34 smithi144 ceph-mon[35660]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:48:35.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-20T20:48:35.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:35.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:35 smithi055 ceph-mon[35213]: Deploying daemon osd.4 on smithi055 2024-05-20T20:48:35.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:35 smithi026 ceph-mon[31990]: Deploying daemon osd.4 on smithi055 2024-05-20T20:48:36.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:35 smithi144 ceph-mon[35660]: Deploying daemon osd.4 on smithi055 2024-05-20T20:48:36.878 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:36 smithi055 ceph-mon[35213]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:36.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:36 smithi026 ceph-mon[31990]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:36 smithi144 ceph-mon[35660]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:38.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:38 smithi055 ceph-mon[35213]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:38 smithi026 ceph-mon[31990]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:39.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:38 smithi144 ceph-mon[35660]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:41 smithi055 ceph-mon[35213]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:41 smithi144 ceph-mon[35660]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:41 smithi026 ceph-mon[31990]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:43 smithi144 ceph-mon[35660]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:43.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:43.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:43 smithi055 ceph-mon[35213]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:43.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:43.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:43.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:43 smithi026 ceph-mon[31990]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.161 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 4 on host 'smithi055' 2024-05-20T20:48:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.160+0000 7f9d1dffb700 1 -- 172.21.15.55:0/1635580383 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9d20063a40 con 0x7f9d100b2900 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d100b2900 msgr2=0x7f9d100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d100b2900 0x7f9d100b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d20104620 tx=0x7f9d1400a380 comp rx=0 tx=0).stop 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 msgr2=0x7f9d20199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20199b70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00b580 tx=0x7f9d0c00f690 comp rx=0 tx=0).stop 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 shutdown_connections 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d100b2900 0x7f9d100b4dc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d201035c0 0x7f9d20199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d20105160 0x7f9d201a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 --2- 172.21.15.55:0/1635580383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d201047c0 0x7f9d20199b70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 >> 172.21.15.55:0/1635580383 conn(0x7f9d200feb60 msgr2=0x7f9d20107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:48:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 shutdown_connections 2024-05-20T20:48:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:48:44.163+0000 7f9d26b3c700 1 -- 172.21.15.55:0/1635580383 wait complete. 2024-05-20T20:48:44.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:44 smithi055 ceph-mon[35213]: from='osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T20:48:44.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:44 smithi055 ceph-mon[35213]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T20:48:44.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:44.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:44 smithi144 ceph-mon[35660]: from='osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T20:48:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:44 smithi144 ceph-mon[35660]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T20:48:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:44.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:44 smithi026 ceph-mon[31990]: from='osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T20:48:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:44 smithi026 ceph-mon[31990]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T20:48:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:48:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:44.812 DEBUG:teuthology.orchestra.run.smithi055:osd.4> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.4.service 2024-05-20T20:48:44.818 INFO:tasks.cephadm:Deploying osd.5 on smithi144 with /dev/vg_nvme/lv_3... 2024-05-20T20:48:44.818 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-20T20:48:45.064 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:48:45.439 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:45 smithi144 ceph-mon[35660]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:45.439 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:45 smithi144 ceph-mon[35660]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-20T20:48:45.439 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:45 smithi144 ceph-mon[35660]: osdmap e29: 5 total, 4 up, 5 in 2024-05-20T20:48:45.439 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:45 smithi144 ceph-mon[35660]: from='osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:48:45.439 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:45 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:45.439 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:45 smithi144 ceph-mon[35660]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:48:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:45 smithi055 ceph-mon[35213]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:45 smithi055 ceph-mon[35213]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-20T20:48:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:45 smithi055 ceph-mon[35213]: osdmap e29: 5 total, 4 up, 5 in 2024-05-20T20:48:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:45 smithi055 ceph-mon[35213]: from='osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:48:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:45 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:45 smithi055 ceph-mon[35213]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:48:45.451 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 20:48:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T20:48:45.225+0000 7fcdd58fb700 -1 osd.4 0 waiting for initial osdmap 2024-05-20T20:48:45.451 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 20:48:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T20:48:45.231+0000 7fcdd16ee700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:48:45.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:45 smithi026 ceph-mon[31990]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:45.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:45 smithi026 ceph-mon[31990]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-20T20:48:45.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:45 smithi026 ceph-mon[31990]: osdmap e29: 5 total, 4 up, 5 in 2024-05-20T20:48:45.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:45 smithi026 ceph-mon[31990]: from='osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:48:45.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:45 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:45.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:45 smithi026 ceph-mon[31990]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:48:46.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:46 smithi055 ceph-mon[35213]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:48:46.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:46 smithi055 ceph-mon[35213]: osdmap e30: 5 total, 4 up, 5 in 2024-05-20T20:48:46.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:46.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:46.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:46 smithi055 ceph-mon[35213]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:46.335 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:46 smithi144 ceph-mon[35660]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:48:46.335 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:46 smithi144 ceph-mon[35660]: osdmap e30: 5 total, 4 up, 5 in 2024-05-20T20:48:46.335 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:46.335 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:46.335 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:46 smithi144 ceph-mon[35660]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:46.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:46 smithi026 ceph-mon[31990]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:48:46.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:46 smithi026 ceph-mon[31990]: osdmap e30: 5 total, 4 up, 5 in 2024-05-20T20:48:46.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:46.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:46.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:46 smithi026 ceph-mon[31990]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-20T20:48:46.913 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-20T20:48:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-20T20:48:46.914 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-20T20:48:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-20T20:48:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111931 s, 937 MB/s 2024-05-20T20:48:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-20T20:48:47.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:47 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:48:47.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:47 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:48:47.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:47 smithi055 ceph-mon[35213]: osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] boot 2024-05-20T20:48:47.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:47 smithi055 ceph-mon[35213]: osdmap e31: 5 total, 5 up, 5 in 2024-05-20T20:48:47.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:47 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:47.478 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-05-20T20:48:47.493 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:47 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:48:47.494 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:47 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:48:47.494 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:47 smithi144 ceph-mon[35660]: osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] boot 2024-05-20T20:48:47.494 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:47 smithi144 ceph-mon[35660]: osdmap e31: 5 total, 5 up, 5 in 2024-05-20T20:48:47.494 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:47 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:47.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:47 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:48:47.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:47 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:48:47.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:47 smithi026 ceph-mon[31990]: osd.4 [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] boot 2024-05-20T20:48:47.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:47 smithi026 ceph-mon[31990]: osdmap e31: 5 total, 5 up, 5 in 2024-05-20T20:48:47.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T20:48:47.727 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: osdmap e32: 5 total, 5 up, 5 in 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi055 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:48.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:48.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi055 to 8546M 2024-05-20T20:48:48.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:48.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:48.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:48 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: osdmap e32: 5 total, 5 up, 5 in 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi055 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:48.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi055 to 8546M 2024-05-20T20:48:48.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:48.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:48.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:48 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: osdmap e32: 5 total, 5 up, 5 in 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi055 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi055 to 8546M 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:48:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:48 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:48:49.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 -- 172.21.15.144:0/1939663074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1047a0 msgr2=0x7ff77c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:49.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 --2- 172.21.15.144:0/1939663074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1047a0 0x7ff77c104c00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff770009990 tx=0x7ff770009ca0 comp rx=0 tx=0).stop 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 -- 172.21.15.144:0/1939663074 shutdown_connections 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 --2- 172.21.15.144:0/1939663074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c1035a0 0x7ff77c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 --2- 172.21.15.144:0/1939663074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff77c105140 0x7ff77c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 --2- 172.21.15.144:0/1939663074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1047a0 0x7ff77c104c00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 -- 172.21.15.144:0/1939663074 >> 172.21.15.144:0/1939663074 conn(0x7ff77c0feb40 msgr2=0x7ff77c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.440+0000 7ff780a88700 1 -- 172.21.15.144:0/1939663074 shutdown_connections 2024-05-20T20:48:49.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 -- 172.21.15.144:0/1939663074 wait complete. 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 Processor -- start 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 -- start start 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 0x7ff77c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c1047a0 0x7ff77c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff77c105140 0x7ff77c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10eec0 con 0x7ff77c1047a0 2024-05-20T20:48:49.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10ed40 con 0x7ff77c105140 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.441+0000 7ff780a88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10f040 con 0x7ff77c1035a0 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 0x7ff77c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 0x7ff77c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58604/0 (socket says 172.21.15.144:58604) 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 -- 172.21.15.144:0/336530859 learned_addr learned my addr 172.21.15.144:0/336530859 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77ad9d700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff77c105140 0x7ff77c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff779d9b700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c1047a0 0x7ff77c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 -- 172.21.15.144:0/336530859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff77c105140 msgr2=0x7ff77c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff77c105140 0x7ff77c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:49.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 -- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c1047a0 msgr2=0x7ff77c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c1047a0 0x7ff77c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 -- 172.21.15.144:0/336530859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c1a1260 con 0x7ff77c1035a0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff77a59c700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 0x7ff77c1996a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00d9b0 tx=0x7ff76c00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c016070 con 0x7ff77c1035a0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff76c010460 con 0x7ff77c1035a0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c1a1550 con 0x7ff77c1035a0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007e60 con 0x7ff77c1035a0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.442+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1a1a90 con 0x7ff77c1035a0 2024-05-20T20:48:49.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.443+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c06a230 con 0x7ff77c1035a0 2024-05-20T20:48:49.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.445+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff76c019070 con 0x7ff77c1035a0 2024-05-20T20:48:49.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.445+0000 7ff76b7fe700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff7640b2880 0x7ff7640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:48:49.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.445+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7ff76c0e1650 con 0x7ff77c1035a0 2024-05-20T20:48:49.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.446+0000 7ff779d9b700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff7640b2880 0x7ff7640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:48:49.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.446+0000 7ff779d9b700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff7640b2880 0x7ff7640b4d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b5c0 tx=0x7ff770005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:48:49.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.449+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff76c020030 con 0x7ff77c1035a0 2024-05-20T20:48:49.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:48:49.660+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff77c063980 con 0x7ff7640b2880 2024-05-20T20:48:50.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:49 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:50.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:49 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:50.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:49 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:50.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:49 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:50.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:49 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:50.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:49 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:49 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:48:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:49 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:48:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:49 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:48:50.799 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:50 smithi144 ceph-mon[35660]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:50.799 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:50 smithi144 ceph-mon[35660]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:51.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:50 smithi055 ceph-mon[35213]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:51.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:50 smithi055 ceph-mon[35213]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:50 smithi026 ceph-mon[31990]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:50 smithi026 ceph-mon[31990]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:48:52.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:52 smithi144 ceph-mon[35660]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:52.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:52 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/276962787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]: dispatch 2024-05-20T20:48:52.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:52 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]: dispatch 2024-05-20T20:48:52.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:52 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]': finished 2024-05-20T20:48:52.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:52 smithi144 ceph-mon[35660]: osdmap e33: 6 total, 5 up, 6 in 2024-05-20T20:48:52.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:48:53.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:52 smithi055 ceph-mon[35213]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:53.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:52 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/276962787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]: dispatch 2024-05-20T20:48:53.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:52 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]: dispatch 2024-05-20T20:48:53.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:52 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]': finished 2024-05-20T20:48:53.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:52 smithi055 ceph-mon[35213]: osdmap e33: 6 total, 5 up, 6 in 2024-05-20T20:48:53.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:48:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:52 smithi026 ceph-mon[31990]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/276962787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]: dispatch 2024-05-20T20:48:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:52 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]: dispatch 2024-05-20T20:48:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:52 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "090b604a-f758-4fa3-b06f-dc15da11739f"}]': finished 2024-05-20T20:48:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:52 smithi026 ceph-mon[31990]: osdmap e33: 6 total, 5 up, 6 in 2024-05-20T20:48:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:48:54.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:53 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/871392006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:54.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:53 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/871392006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:54.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:53 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/871392006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:48:55.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:54 smithi144 ceph-mon[35660]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:55.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:54 smithi055 ceph-mon[35213]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:54 smithi026 ceph-mon[31990]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:57.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:56 smithi026 ceph-mon[31990]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:57.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:56 smithi144 ceph-mon[35660]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:57.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:56 smithi055 ceph-mon[35213]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:58.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:58 smithi055 ceph-mon[35213]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:59.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:58 smithi144 ceph-mon[35660]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:48:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:58 smithi026 ceph-mon[31990]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:00.034 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:59 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-20T20:49:00.034 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:48:59 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:00.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:59 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-20T20:49:00.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:48:59 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:00.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:59 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-20T20:49:00.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:48:59 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:01.019 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:00 smithi144 ceph-mon[35660]: Deploying daemon osd.5 on smithi144 2024-05-20T20:49:01.019 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:00 smithi144 ceph-mon[35660]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:01.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:00 smithi055 ceph-mon[35213]: Deploying daemon osd.5 on smithi144 2024-05-20T20:49:01.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:00 smithi055 ceph-mon[35213]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:01.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:00 smithi026 ceph-mon[31990]: Deploying daemon osd.5 on smithi144 2024-05-20T20:49:01.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:00 smithi026 ceph-mon[31990]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:03.032 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:02 smithi144 ceph-mon[35660]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:03.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:02 smithi055 ceph-mon[35213]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:03.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:02 smithi026 ceph-mon[31990]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:05.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:04 smithi144 ceph-mon[35660]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:05.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:04 smithi055 ceph-mon[35213]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:05.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:04 smithi026 ceph-mon[31990]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:06.013 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:05 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:06.013 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:05 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:06.013 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:05 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:05 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:05 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:05 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:05 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:05 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:05 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:07.025 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:06 smithi144 ceph-mon[35660]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:07.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:06 smithi055 ceph-mon[35213]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:07.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:06 smithi026 ceph-mon[31990]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:08.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.276+0000 7ff76b7fe700 1 -- 172.21.15.144:0/336530859 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff77c063980 con 0x7ff7640b2880 2024-05-20T20:49:08.277 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 5 on host 'smithi144' 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff7640b2880 msgr2=0x7ff7640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff7640b2880 0x7ff7640b4d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b5c0 tx=0x7ff770005fb0 comp rx=0 tx=0).stop 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 msgr2=0x7ff77c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 0x7ff77c1996a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00d9b0 tx=0x7ff76c00dd70 comp rx=0 tx=0).stop 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 shutdown_connections 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff77c105140 0x7ff77c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77c1035a0 0x7ff77c1996a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff7640b2880 0x7ff7640b4d40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:08.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 --2- 172.21.15.144:0/336530859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c1047a0 0x7ff77c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:08.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 >> 172.21.15.144:0/336530859 conn(0x7ff77c0feb40 msgr2=0x7ff77c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:49:08.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 shutdown_connections 2024-05-20T20:49:08.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:49:08.279+0000 7ff780a88700 1 -- 172.21.15.144:0/336530859 wait complete. 2024-05-20T20:49:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:08.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:08.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:08.897 DEBUG:teuthology.orchestra.run.smithi144:osd.5> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.5.service 2024-05-20T20:49:08.939 INFO:tasks.cephadm:Deploying osd.6 on smithi026 with /dev/vg_nvme/lv_2... 2024-05-20T20:49:08.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-20T20:49:09.194 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:49:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:09 smithi144 ceph-mon[35660]: from='osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T20:49:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:09 smithi144 ceph-mon[35660]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T20:49:09.645 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:09 smithi026 ceph-mon[31990]: from='osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T20:49:09.645 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:09 smithi026 ceph-mon[31990]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T20:49:09.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:09 smithi055 ceph-mon[35213]: from='osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T20:49:09.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:09 smithi055 ceph-mon[35213]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T20:49:10.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:10 smithi144 ceph-mon[35660]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-20T20:49:10.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:10 smithi144 ceph-mon[35660]: osdmap e34: 6 total, 5 up, 6 in 2024-05-20T20:49:10.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:10.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:10 smithi144 ceph-mon[35660]: from='osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:49:10.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:10 smithi144 ceph-mon[35660]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:49:10.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:10 smithi144 ceph-mon[35660]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:10.551 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 20:49:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T20:49:10.425+0000 7ff0214d6700 -1 osd.5 0 waiting for initial osdmap 2024-05-20T20:49:10.551 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 20:49:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T20:49:10.430+0000 7ff01d2c9700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:49:10.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:10 smithi026 ceph-mon[31990]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-20T20:49:10.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:10 smithi026 ceph-mon[31990]: osdmap e34: 6 total, 5 up, 6 in 2024-05-20T20:49:10.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:10.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:10 smithi026 ceph-mon[31990]: from='osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:49:10.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:10 smithi026 ceph-mon[31990]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:49:10.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:10 smithi026 ceph-mon[31990]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:10.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:10 smithi055 ceph-mon[35213]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-20T20:49:10.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:10 smithi055 ceph-mon[35213]: osdmap e34: 6 total, 5 up, 6 in 2024-05-20T20:49:10.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:10.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:10 smithi055 ceph-mon[35213]: from='osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:49:10.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:10 smithi055 ceph-mon[35213]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:49:10.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:10 smithi055 ceph-mon[35213]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-20T20:49:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-20T20:49:11.178 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-20T20:49:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-20T20:49:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00843111 s, 1.2 GB/s 2024-05-20T20:49:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-20T20:49:11.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:11 smithi026 ceph-mon[31990]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:49:11.491 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:11 smithi026 ceph-mon[31990]: osdmap e35: 6 total, 5 up, 6 in 2024-05-20T20:49:11.491 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:11.491 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:11 smithi055 ceph-mon[35213]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:49:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:11 smithi055 ceph-mon[35213]: osdmap e35: 6 total, 5 up, 6 in 2024-05-20T20:49:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:11.719 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:11 smithi144 ceph-mon[35660]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:49:11.719 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:11 smithi144 ceph-mon[35660]: osdmap e35: 6 total, 5 up, 6 in 2024-05-20T20:49:11.719 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:11.719 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:11.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-05-20T20:49:12.075 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:49:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:49:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:49:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] boot 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: osdmap e36: 6 total, 6 up, 6 in 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi144 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi144 to 8544M 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] boot 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: osdmap e36: 6 total, 6 up, 6 in 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi144 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi144 to 8544M 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:12.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:49:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:49:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: osd.5 [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] boot 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: osdmap e36: 6 total, 6 up, 6 in 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi144 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:12.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi144 to 8544M 2024-05-20T20:49:12.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:12.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:12.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:12.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 -- 172.21.15.26:0/2758687329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc4104830 msgr2=0x7fefc4104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 --2- 172.21.15.26:0/2758687329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc4104830 0x7fefc4104c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009990 tx=0x7fefb8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 -- 172.21.15.26:0/2758687329 shutdown_connections 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 --2- 172.21.15.26:0/2758687329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc41051d0 0x7fefc410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 --2- 172.21.15.26:0/2758687329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4101850 0x7fefc4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 --2- 172.21.15.26:0/2758687329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc4104830 0x7fefc4104c90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 -- 172.21.15.26:0/2758687329 >> 172.21.15.26:0/2758687329 conn(0x7fefc40fd1d0 msgr2=0x7fefc40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.677+0000 7fefca75d700 1 -- 172.21.15.26:0/2758687329 shutdown_connections 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.678+0000 7fefca75d700 1 -- 172.21.15.26:0/2758687329 wait complete. 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.678+0000 7fefca75d700 1 Processor -- start 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.678+0000 7fefca75d700 1 -- start start 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.678+0000 7fefca75d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc4101850 0x7fefc4104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.678+0000 7fefca75d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4104830 0x7fefc41027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.678+0000 7fefca75d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 0x7fefc4102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefca75d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410ef40 con 0x7fefc41051d0 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefca75d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410edc0 con 0x7fefc4101850 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefca75d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410f0c0 con 0x7fefc4104830 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 0x7fefc4102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 0x7fefc4102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52896/0 (socket says 172.21.15.26:52896) 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 -- 172.21.15.26:0/3812134330 learned_addr learned my addr 172.21.15.26:0/3812134330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc3fff700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc4101850 0x7fefc4104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc37fe700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4104830 0x7fefc41027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 -- 172.21.15.26:0/3812134330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4104830 msgr2=0x7fefc41027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4104830 0x7fefc41027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 -- 172.21.15.26:0/3812134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc4101850 msgr2=0x7fefc4104190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc4101850 0x7fefc4104190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 -- 172.21.15.26:0/3812134330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41035d0 con 0x7fefc41051d0 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc3fff700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc4101850 0x7fefc4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc37fe700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4104830 0x7fefc41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc8cfa700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 0x7fefc4102d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefbc007ee0 tx=0x7fefbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00dbd0 con 0x7fefc41051d0 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41a37e0 con 0x7fefc41051d0 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefbc00dd70 con 0x7fefc41051d0 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0058d0 con 0x7fefc41051d0 2024-05-20T20:49:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.679+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a3d20 con 0x7fefc41051d0 2024-05-20T20:49:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.680+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc406a230 con 0x7fefc41051d0 2024-05-20T20:49:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.682+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fefbc01a020 con 0x7fefc41051d0 2024-05-20T20:49:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.683+0000 7fefc17fa700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fefac0b2880 0x7fefac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.683+0000 7fefc3fff700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fefac0b2880 0x7fefac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.683+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fefbc0e12f0 con 0x7fefc41051d0 2024-05-20T20:49:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.683+0000 7fefc3fff700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fefac0b2880 0x7fefac0b4d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fefc406b020 tx=0x7fefb4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:49:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.687+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fefbc010070 con 0x7fefc41051d0 2024-05-20T20:49:13.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:13 smithi055 ceph-mon[35213]: osdmap e37: 6 total, 6 up, 6 in 2024-05-20T20:49:13.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:13 smithi026 ceph-mon[31990]: osdmap e37: 6 total, 6 up, 6 in 2024-05-20T20:49:13.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:13 smithi144 ceph-mon[35660]: osdmap e37: 6 total, 6 up, 6 in 2024-05-20T20:49:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:13.915+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fefc4063980 con 0x7fefac0b2880 2024-05-20T20:49:14.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:14 smithi144 ceph-mon[35660]: osdmap e38: 6 total, 6 up, 6 in 2024-05-20T20:49:14.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:14 smithi144 ceph-mon[35660]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:14.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:14 smithi144 ceph-mon[35660]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:49:14.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:49:14.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:49:14.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:14.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:14 smithi026 ceph-mon[31990]: osdmap e38: 6 total, 6 up, 6 in 2024-05-20T20:49:14.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:14 smithi026 ceph-mon[31990]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:14.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:14 smithi026 ceph-mon[31990]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:49:14.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:49:14.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:49:14.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:14.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:14 smithi055 ceph-mon[35213]: osdmap e38: 6 total, 6 up, 6 in 2024-05-20T20:49:14.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:14 smithi055 ceph-mon[35213]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:14.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:14 smithi055 ceph-mon[35213]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:49:14.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:49:14.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:49:14.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:16.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:16 smithi055 ceph-mon[35213]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-20T20:49:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:16 smithi026 ceph-mon[31990]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-20T20:49:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:16 smithi144 ceph-mon[35660]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-20T20:49:17.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:17 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3786059333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54095128-6ffb-408b-8ac6-e8728acec871"}]: dispatch 2024-05-20T20:49:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:17 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3786059333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54095128-6ffb-408b-8ac6-e8728acec871"}]': finished 2024-05-20T20:49:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:17 smithi055 ceph-mon[35213]: osdmap e39: 7 total, 6 up, 7 in 2024-05-20T20:49:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:17 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2978934968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:49:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:17 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3786059333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54095128-6ffb-408b-8ac6-e8728acec871"}]: dispatch 2024-05-20T20:49:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:17 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3786059333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54095128-6ffb-408b-8ac6-e8728acec871"}]': finished 2024-05-20T20:49:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:17 smithi026 ceph-mon[31990]: osdmap e39: 7 total, 6 up, 7 in 2024-05-20T20:49:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:17 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2978934968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:49:18.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:17 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3786059333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54095128-6ffb-408b-8ac6-e8728acec871"}]: dispatch 2024-05-20T20:49:18.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:17 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3786059333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54095128-6ffb-408b-8ac6-e8728acec871"}]': finished 2024-05-20T20:49:18.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:17 smithi144 ceph-mon[35660]: osdmap e39: 7 total, 6 up, 7 in 2024-05-20T20:49:18.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:18.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:17 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2978934968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:49:19.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:18 smithi144 ceph-mon[35660]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:49:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:18 smithi055 ceph-mon[35213]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:49:19.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:18 smithi026 ceph-mon[31990]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:49:21.145 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:20 smithi026 ceph-mon[31990]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-20T20:49:21.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:20 smithi055 ceph-mon[35213]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-20T20:49:21.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:20 smithi144 ceph-mon[35660]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-20T20:49:22.838 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:22 smithi026 ceph-mon[31990]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-20T20:49:23.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:22 smithi055 ceph-mon[35213]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-20T20:49:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:22 smithi144 ceph-mon[35660]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-20T20:49:25.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:24 smithi026 ceph-mon[31990]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:49:25.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-20T20:49:25.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:25.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:24 smithi055 ceph-mon[35213]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:49:25.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-20T20:49:25.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:24 smithi144 ceph-mon[35660]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:49:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-20T20:49:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:26.032 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:25 smithi026 ceph-mon[31990]: Deploying daemon osd.6 on smithi026 2024-05-20T20:49:26.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:25 smithi055 ceph-mon[35213]: Deploying daemon osd.6 on smithi026 2024-05-20T20:49:26.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:25 smithi144 ceph-mon[35660]: Deploying daemon osd.6 on smithi026 2024-05-20T20:49:27.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:26 smithi026 ceph-mon[31990]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:27.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:26 smithi055 ceph-mon[35213]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:26 smithi144 ceph-mon[35660]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:29.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:28 smithi144 ceph-mon[35660]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:29.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:28 smithi055 ceph-mon[35213]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:29.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:28 smithi026 ceph-mon[31990]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:30.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:30.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:31.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:30 smithi055 ceph-mon[35213]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:31.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:30 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:31.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:30 smithi026 ceph-mon[31990]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:31.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:30 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:30 smithi144 ceph-mon[35660]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:30 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:33.096 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:32 smithi026 ceph-mon[31990]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:32 smithi055 ceph-mon[35213]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:32 smithi144 ceph-mon[35660]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:33.740 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 6 on host 'smithi026' 2024-05-20T20:49:33.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.739+0000 7fefc17fa700 1 -- 172.21.15.26:0/3812134330 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fefc4063980 con 0x7fefac0b2880 2024-05-20T20:49:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fefac0b2880 msgr2=0x7fefac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fefac0b2880 0x7fefac0b4d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fefc406b020 tx=0x7fefb4008040 comp rx=0 tx=0).stop 2024-05-20T20:49:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 msgr2=0x7fefc4102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 0x7fefc4102d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefbc007ee0 tx=0x7fefbc00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 shutdown_connections 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fefac0b2880 0x7fefac0b4d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc41051d0 0x7fefc4102d20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc4101850 0x7fefc4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 --2- 172.21.15.26:0/3812134330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc4104830 0x7fefc41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 >> 172.21.15.26:0/3812134330 conn(0x7fefc40fd1d0 msgr2=0x7fefc4107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 shutdown_connections 2024-05-20T20:49:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:49:33.742+0000 7fefca75d700 1 -- 172.21.15.26:0/3812134330 wait complete. 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:33.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:34.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:34.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-20T20:49:34.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:34.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:34.454 DEBUG:teuthology.orchestra.run.smithi026:osd.6> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.6.service 2024-05-20T20:49:34.457 INFO:tasks.cephadm:Deploying osd.7 on smithi055 with /dev/vg_nvme/lv_2... 2024-05-20T20:49:34.457 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-20T20:49:34.717 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:49:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:34 smithi026 ceph-mon[31990]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:34 smithi026 ceph-mon[31990]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-20T20:49:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:34 smithi026 ceph-mon[31990]: osdmap e40: 7 total, 6 up, 7 in 2024-05-20T20:49:34.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:34 smithi026 ceph-mon[31990]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:49:34.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:34 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:34.953 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 20:49:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T20:49:34.878+0000 7f8c8df89700 -1 osd.6 0 waiting for initial osdmap 2024-05-20T20:49:34.953 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 20:49:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T20:49:34.886+0000 7f8c89d7c700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:34 smithi055 ceph-mon[35213]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:34 smithi055 ceph-mon[35213]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-20T20:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:34 smithi055 ceph-mon[35213]: osdmap e40: 7 total, 6 up, 7 in 2024-05-20T20:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:34 smithi055 ceph-mon[35213]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:34 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:34 smithi144 ceph-mon[35660]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:34 smithi144 ceph-mon[35660]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-20T20:49:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:34 smithi144 ceph-mon[35660]: osdmap e40: 7 total, 6 up, 7 in 2024-05-20T20:49:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:34 smithi144 ceph-mon[35660]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:49:35.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:34 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:35 smithi026 ceph-mon[31990]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:49:36.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:35 smithi026 ceph-mon[31990]: osdmap e41: 7 total, 6 up, 7 in 2024-05-20T20:49:36.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:35 smithi055 ceph-mon[35213]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:49:36.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:35 smithi055 ceph-mon[35213]: osdmap e41: 7 total, 6 up, 7 in 2024-05-20T20:49:36.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:35 smithi144 ceph-mon[35660]: from='osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:49:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:35 smithi144 ceph-mon[35660]: osdmap e41: 7 total, 6 up, 7 in 2024-05-20T20:49:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:36.864 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-20T20:49:36.865 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-20T20:49:36.865 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-05-20T20:49:36.865 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-05-20T20:49:36.865 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0163505 s, 641 MB/s 2024-05-20T20:49:36.865 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-05-20T20:49:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:36 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:49:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:36 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:49:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:36 smithi055 ceph-mon[35213]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:36 smithi055 ceph-mon[35213]: osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] boot 2024-05-20T20:49:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:36 smithi055 ceph-mon[35213]: osdmap e42: 7 total, 7 up, 7 in 2024-05-20T20:49:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:37.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:36 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:49:37.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:36 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:49:37.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:36 smithi026 ceph-mon[31990]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:37.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:36 smithi026 ceph-mon[31990]: osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] boot 2024-05-20T20:49:37.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:36 smithi026 ceph-mon[31990]: osdmap e42: 7 total, 7 up, 7 in 2024-05-20T20:49:37.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:36 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:49:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:36 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:49:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:36 smithi144 ceph-mon[35660]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-20T20:49:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:36 smithi144 ceph-mon[35660]: osd.6 [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] boot 2024-05-20T20:49:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:36 smithi144 ceph-mon[35660]: osdmap e42: 7 total, 7 up, 7 in 2024-05-20T20:49:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T20:49:37.601 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_2 2024-05-20T20:49:37.851 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:49:38.174 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: osdmap e43: 7 total, 7 up, 7 in 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:38.175 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: osdmap e43: 7 total, 7 up, 7 in 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:38.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: osdmap e43: 7 total, 7 up, 7 in 2024-05-20T20:49:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:49:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:49:38.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:38 smithi055 ceph-mon[35213]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:38 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi026 2024-05-20T20:49:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:38 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi026 to 5697M 2024-05-20T20:49:39.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:38 smithi026 ceph-mon[31990]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:39.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:38 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi026 2024-05-20T20:49:39.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:38 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi026 to 5697M 2024-05-20T20:49:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:38 smithi144 ceph-mon[35660]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:38 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi026 2024-05-20T20:49:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:38 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi026 to 5697M 2024-05-20T20:49:39.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 -- 172.21.15.55:0/2921201338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c104040 msgr2=0x7f4d5c10a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:39.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 --2- 172.21.15.55:0/2921201338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c104040 0x7f4d5c10a8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400b0d0 tx=0x7f4d5400b3e0 comp rx=0 tx=0).stop 2024-05-20T20:49:39.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 -- 172.21.15.55:0/2921201338 shutdown_connections 2024-05-20T20:49:39.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 --2- 172.21.15.55:0/2921201338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c104040 0x7f4d5c10a8f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 --2- 172.21.15.55:0/2921201338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c1024a0 0x7f4d5c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 --2- 172.21.15.55:0/2921201338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 -- 172.21.15.55:0/2921201338 >> 172.21.15.55:0/2921201338 conn(0x7f4d5c0fda40 msgr2=0x7f4d5c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:49:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.478+0000 7f4d6255a700 1 -- 172.21.15.55:0/2921201338 shutdown_connections 2024-05-20T20:49:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 -- 172.21.15.55:0/2921201338 wait complete. 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 Processor -- start 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 -- start start 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1024a0 0x7f4d5c106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c104040 0x7f4d5c104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10ddd0 con 0x7f4d5c1024a0 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10dc50 con 0x7f4d5c1036a0 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d6255a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10df50 con 0x7f4d5c104040 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d53fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:39.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d53fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54046/0 (socket says 172.21.15.55:54046) 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.479+0000 7f4d53fff700 1 -- 172.21.15.55:0/3473721557 learned_addr learned my addr 172.21.15.55:0/3473721557 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d60af7700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c104040 0x7f4d5c104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d5bfff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1024a0 0x7f4d5c106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d53fff700 1 -- 172.21.15.55:0/3473721557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c104040 msgr2=0x7f4d5c104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d53fff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c104040 0x7f4d5c104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d53fff700 1 -- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1024a0 msgr2=0x7f4d5c106430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d53fff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1024a0 0x7f4d5c106430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d53fff700 1 -- 172.21.15.55:0/3473721557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c105870 con 0x7f4d5c1036a0 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d5bfff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1024a0 0x7f4d5c106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d53fff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c104a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c920 tx=0x7f4d4400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44007b00 con 0x7f4d5c1036a0 2024-05-20T20:49:39.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d44004cc0 con 0x7f4d5c1036a0 2024-05-20T20:49:39.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c1a36f0 con 0x7f4d5c1036a0 2024-05-20T20:49:39.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d440054f0 con 0x7f4d5c1036a0 2024-05-20T20:49:39.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.480+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c1a3c00 con 0x7f4d5c1036a0 2024-05-20T20:49:39.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.481+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5c06a230 con 0x7f4d5c1036a0 2024-05-20T20:49:39.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.483+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4d44007ca0 con 0x7f4d5c1036a0 2024-05-20T20:49:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.484+0000 7f4d59ffb700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d480b2880 0x7f4d480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:49:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.484+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f4d440e1550 con 0x7f4d5c1036a0 2024-05-20T20:49:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.484+0000 7f4d5bfff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d480b2880 0x7f4d480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:49:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.485+0000 7f4d5bfff700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d480b2880 0x7f4d480b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:49:39.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.488+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d44014030 con 0x7f4d5c1036a0 2024-05-20T20:49:39.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:39.715+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4d5c1a3ec0 con 0x7f4d480b2880 2024-05-20T20:49:40.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:49:40.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:49:40.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:40.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:49:40.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:49:40.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:40.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:49:40.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:49:40.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:40 smithi055 ceph-mon[35213]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:40 smithi055 ceph-mon[35213]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:49:41.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:40 smithi026 ceph-mon[31990]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:41.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:40 smithi026 ceph-mon[31990]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:49:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:40 smithi144 ceph-mon[35660]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:40 smithi144 ceph-mon[35660]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:49:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:42 smithi055 ceph-mon[35213]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:42 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/3338633430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]: dispatch 2024-05-20T20:49:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:42 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]: dispatch 2024-05-20T20:49:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:42 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]': finished 2024-05-20T20:49:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:42 smithi055 ceph-mon[35213]: osdmap e44: 8 total, 7 up, 8 in 2024-05-20T20:49:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:42 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:49:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:42 smithi026 ceph-mon[31990]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:42 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/3338633430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]: dispatch 2024-05-20T20:49:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:42 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]: dispatch 2024-05-20T20:49:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:42 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]': finished 2024-05-20T20:49:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:42 smithi026 ceph-mon[31990]: osdmap e44: 8 total, 7 up, 8 in 2024-05-20T20:49:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:42 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:49:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:42 smithi144 ceph-mon[35660]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:42 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/3338633430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]: dispatch 2024-05-20T20:49:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:42 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]: dispatch 2024-05-20T20:49:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:42 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9320773-2918-438a-8cf7-f9171c836fd1"}]': finished 2024-05-20T20:49:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:42 smithi144 ceph-mon[35660]: osdmap e44: 8 total, 7 up, 8 in 2024-05-20T20:49:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:42 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:49:44.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:43 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/1741588901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:49:44.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:43 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/1741588901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:49:44.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:43 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/1741588901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:49:45.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:44 smithi055 ceph-mon[35213]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:45.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:44 smithi026 ceph-mon[31990]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:45.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:44 smithi144 ceph-mon[35660]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:47.111 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:46 smithi055 ceph-mon[35213]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:46 smithi026 ceph-mon[31990]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:46 smithi144 ceph-mon[35660]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:49.107 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:48 smithi055 ceph-mon[35213]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:49.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:48 smithi026 ceph-mon[31990]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:49.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:48 smithi144 ceph-mon[35660]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:51.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:50 smithi055 ceph-mon[35213]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:51.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-20T20:49:51.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:50 smithi026 ceph-mon[31990]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-20T20:49:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:50 smithi144 ceph-mon[35660]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-20T20:49:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:49:52.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:51 smithi055 ceph-mon[35213]: Deploying daemon osd.7 on smithi055 2024-05-20T20:49:52.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:51 smithi026 ceph-mon[31990]: Deploying daemon osd.7 on smithi055 2024-05-20T20:49:52.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:51 smithi144 ceph-mon[35660]: Deploying daemon osd.7 on smithi055 2024-05-20T20:49:52.943 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:52 smithi055 ceph-mon[35213]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:52 smithi026 ceph-mon[31990]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:53.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:52 smithi144 ceph-mon[35660]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:55.148 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:54 smithi055 ceph-mon[35213]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:54 smithi026 ceph-mon[31990]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:54 smithi144 ceph-mon[35660]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:57.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:57 smithi055 ceph-mon[35213]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:57.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:57.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:57.269 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:57 smithi144 ceph-mon[35660]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:57 smithi026 ceph-mon[31990]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:49:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:49:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:49:59 smithi144 ceph-mon[35660]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:59.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:49:59 smithi055 ceph-mon[35213]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:59.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:49:59 smithi026 ceph-mon[31990]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:49:59.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.484+0000 7f4d59ffb700 1 -- 172.21.15.55:0/3473721557 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d5c1a3ec0 con 0x7f4d480b2880 2024-05-20T20:49:59.484 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 7 on host 'smithi055' 2024-05-20T20:49:59.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d480b2880 msgr2=0x7f4d480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:59.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d480b2880 0x7f4d480b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c005e40 comp rx=0 tx=0).stop 2024-05-20T20:49:59.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 msgr2=0x7f4d5c104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c104a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c920 tx=0x7f4d4400cce0 comp rx=0 tx=0).stop 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 shutdown_connections 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d480b2880 0x7f4d480b4d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1024a0 0x7f4d5c106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c104040 0x7f4d5c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 --2- 172.21.15.55:0/3473721557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d5c1036a0 0x7f4d5c104a80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 >> 172.21.15.55:0/3473721557 conn(0x7f4d5c0fda40 msgr2=0x7f4d5c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 shutdown_connections 2024-05-20T20:49:59.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:49:59.487+0000 7f4d6255a700 1 -- 172.21.15.55:0/3473721557 wait complete. 2024-05-20T20:50:00.065 DEBUG:teuthology.orchestra.run.smithi055:osd.7> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.7.service 2024-05-20T20:50:00.067 INFO:tasks.cephadm:Deploying osd.8 on smithi144 with /dev/vg_nvme/lv_2... 2024-05-20T20:50:00.068 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-20T20:50:00.319 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:50:00.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: from='osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: overall HEALTH_OK 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: from='osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: overall HEALTH_OK 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: from='osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: overall HEALTH_OK 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:00.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:01.616 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:01 smithi144 ceph-mon[35660]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-20T20:50:01.616 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:01 smithi144 ceph-mon[35660]: osdmap e45: 8 total, 7 up, 8 in 2024-05-20T20:50:01.616 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:01 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:01.616 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:01 smithi144 ceph-mon[35660]: from='osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:50:01.616 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:01 smithi144 ceph-mon[35660]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:50:01.824 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:01 smithi055 ceph-mon[35213]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-20T20:50:01.824 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:01 smithi055 ceph-mon[35213]: osdmap e45: 8 total, 7 up, 8 in 2024-05-20T20:50:01.824 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:01 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:01.824 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:01 smithi055 ceph-mon[35213]: from='osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:50:01.824 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:01 smithi055 ceph-mon[35213]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:50:01.824 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 20:50:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T20:50:01.469+0000 7fa53435f700 -1 osd.7 0 waiting for initial osdmap 2024-05-20T20:50:01.824 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 20:50:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T20:50:01.481+0000 7fa530152700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:50:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:01 smithi026 ceph-mon[31990]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-20T20:50:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:01 smithi026 ceph-mon[31990]: osdmap e45: 8 total, 7 up, 8 in 2024-05-20T20:50:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:01 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:01 smithi026 ceph-mon[31990]: from='osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:50:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:01 smithi026 ceph-mon[31990]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:50:02.185 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-20T20:50:02.185 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-20T20:50:02.185 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-20T20:50:02.185 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-20T20:50:02.186 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113237 s, 926 MB/s 2024-05-20T20:50:02.186 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-20T20:50:02.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:02 smithi144 ceph-mon[35660]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:50:02.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:02 smithi144 ceph-mon[35660]: osdmap e46: 8 total, 7 up, 8 in 2024-05-20T20:50:02.764 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:02 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:02.764 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:02 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:02.764 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:02 smithi144 ceph-mon[35660]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:50:02.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-05-20T20:50:02.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:02 smithi055 ceph-mon[35213]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:50:02.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:02 smithi055 ceph-mon[35213]: osdmap e46: 8 total, 7 up, 8 in 2024-05-20T20:50:02.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:02 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:02.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:02 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:02.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:02 smithi055 ceph-mon[35213]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:50:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:02 smithi026 ceph-mon[31990]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:50:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:02 smithi026 ceph-mon[31990]: osdmap e46: 8 total, 7 up, 8 in 2024-05-20T20:50:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:02 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:02 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:02 smithi026 ceph-mon[31990]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-20T20:50:03.031 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:50:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:03 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:50:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:03 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:50:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:03 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:03 smithi055 ceph-mon[35213]: osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] boot 2024-05-20T20:50:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:03 smithi055 ceph-mon[35213]: osdmap e47: 8 total, 8 up, 8 in 2024-05-20T20:50:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:03 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:03.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:03 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:50:03.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:03 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:50:03.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:03 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:03.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:03 smithi144 ceph-mon[35660]: osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] boot 2024-05-20T20:50:03.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:03 smithi144 ceph-mon[35660]: osdmap e47: 8 total, 8 up, 8 in 2024-05-20T20:50:03.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:03 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:03.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:03 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:50:03.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:03 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:50:03.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:03 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:03.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:03 smithi026 ceph-mon[31990]: osd.7 [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] boot 2024-05-20T20:50:03.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:03 smithi026 ceph-mon[31990]: osdmap e47: 8 total, 8 up, 8 in 2024-05-20T20:50:03.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:03 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T20:50:04.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 -- 172.21.15.144:0/473059719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa664104090 msgr2=0x7fa66410a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:04.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 --2- 172.21.15.144:0/473059719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa664104090 0x7fa66410a920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b0a0 tx=0x7fa65c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:50:04.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 -- 172.21.15.144:0/473059719 shutdown_connections 2024-05-20T20:50:04.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 --2- 172.21.15.144:0/473059719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa664104090 0x7fa66410a920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:04.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 --2- 172.21.15.144:0/473059719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641029b0 0x7fa664102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 --2- 172.21.15.144:0/473059719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6641036d0 0x7fa664103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 -- 172.21.15.144:0/473059719 >> 172.21.15.144:0/473059719 conn(0x7fa6640fd1d0 msgr2=0x7fa6640ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 -- 172.21.15.144:0/473059719 shutdown_connections 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.629+0000 7fa6697a5700 1 -- 172.21.15.144:0/473059719 wait complete. 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 Processor -- start 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 -- start start 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 0x7fa664102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641036d0 0x7fa66419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa664104090 0x7fa6641a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa66410dd40 con 0x7fa664104090 2024-05-20T20:50:04.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa66410dbc0 con 0x7fa6641036d0 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6697a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa66410dec0 con 0x7fa6641029b0 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa662ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 0x7fa664102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa662ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 0x7fa664102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35628/0 (socket says 172.21.15.144:35628) 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa662ffd700 1 -- 172.21.15.144:0/1196304267 learned_addr learned my addr 172.21.15.144:0/1196304267 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa6637fe700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa664104090 0x7fa6641a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.630+0000 7fa65bfff700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641036d0 0x7fa66419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa662ffd700 1 -- 172.21.15.144:0/1196304267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641036d0 msgr2=0x7fa66419e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa662ffd700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641036d0 0x7fa66419e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa662ffd700 1 -- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa664104090 msgr2=0x7fa6641a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa662ffd700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa664104090 0x7fa6641a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa662ffd700 1 -- 172.21.15.144:0/1196304267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6641a1140 con 0x7fa6641029b0 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa6637fe700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa664104090 0x7fa6641a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:50:04.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa65bfff700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641036d0 0x7fa66419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa662ffd700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 0x7fa664102090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c8f0 tx=0x7fa65400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654004220 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa654007d90 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6641a1430 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654003ca0 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.631+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641a1920 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.632+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66406a230 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.634+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa654007900 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.635+0000 7fa660ff9700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6500b2880 0x7fa6500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.635+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5397+0+0 (secure 0 0 0) 0x7fa6540e17a0 con 0x7fa6641029b0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.635+0000 7fa65bfff700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6500b2880 0x7fa6500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.635+0000 7fa65bfff700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6500b2880 0x7fa6500b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005fd0 tx=0x7fa64c005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:50:04.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.639+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa654014030 con 0x7fa6641029b0 2024-05-20T20:50:04.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: osdmap e48: 8 total, 8 up, 8 in 2024-05-20T20:50:04.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:04.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi055 2024-05-20T20:50:04.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi055 to 5697M 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:04.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:04.853+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa664063980 con 0x7fa6500b2880 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: osdmap e48: 8 total, 8 up, 8 in 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi055 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi055 to 5697M 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: osdmap e48: 8 total, 8 up, 8 in 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi055 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi055 to 5697M 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:04.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:04.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:04.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:06.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:05 smithi144 ceph-mon[35660]: osdmap e49: 8 total, 8 up, 8 in 2024-05-20T20:50:06.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:05 smithi144 ceph-mon[35660]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:06.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:05 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:06.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:05 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:06.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:05 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:05 smithi055 ceph-mon[35213]: osdmap e49: 8 total, 8 up, 8 in 2024-05-20T20:50:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:05 smithi055 ceph-mon[35213]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:05 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:05 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:06.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:05 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:05 smithi026 ceph-mon[31990]: osdmap e49: 8 total, 8 up, 8 in 2024-05-20T20:50:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:05 smithi026 ceph-mon[31990]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:05 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:05 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:06.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:05 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:06 smithi144 ceph-mon[35660]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:07.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:06 smithi055 ceph-mon[35213]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:07.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:06 smithi026 ceph-mon[31990]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:08.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:07 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/1544893957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]: dispatch 2024-05-20T20:50:08.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:07 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]: dispatch 2024-05-20T20:50:08.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:07 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]': finished 2024-05-20T20:50:08.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:07 smithi144 ceph-mon[35660]: osdmap e50: 9 total, 8 up, 9 in 2024-05-20T20:50:08.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:07 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:08.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:07 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/1544893957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]: dispatch 2024-05-20T20:50:08.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:07 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]: dispatch 2024-05-20T20:50:08.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:07 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]': finished 2024-05-20T20:50:08.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:07 smithi055 ceph-mon[35213]: osdmap e50: 9 total, 8 up, 9 in 2024-05-20T20:50:08.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:07 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:08.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:07 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/1544893957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]: dispatch 2024-05-20T20:50:08.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:07 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]: dispatch 2024-05-20T20:50:08.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:07 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c564749e-8c86-4759-9818-af2408d1b735"}]': finished 2024-05-20T20:50:08.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:07 smithi026 ceph-mon[31990]: osdmap e50: 9 total, 8 up, 9 in 2024-05-20T20:50:08.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:08 smithi144 ceph-mon[35660]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:08 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/2331328531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:50:09.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:08 smithi055 ceph-mon[35213]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:09.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:08 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/2331328531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:50:09.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:08 smithi026 ceph-mon[31990]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:09.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:08 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/2331328531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:50:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:10 smithi144 ceph-mon[35660]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 187 KiB/s, 0 objects/s recovering 2024-05-20T20:50:11.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:10 smithi055 ceph-mon[35213]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 187 KiB/s, 0 objects/s recovering 2024-05-20T20:50:11.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:10 smithi026 ceph-mon[31990]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 187 KiB/s, 0 objects/s recovering 2024-05-20T20:50:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:12 smithi144 ceph-mon[35660]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T20:50:13.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:12 smithi055 ceph-mon[35213]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T20:50:13.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:12 smithi026 ceph-mon[31990]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T20:50:14.888 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:14 smithi144 ceph-mon[35660]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-05-20T20:50:15.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:14 smithi055 ceph-mon[35213]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-05-20T20:50:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:14 smithi026 ceph-mon[31990]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-05-20T20:50:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-20T20:50:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:15 smithi144 ceph-mon[35660]: Deploying daemon osd.8 on smithi144 2024-05-20T20:50:16.101 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-20T20:50:16.101 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:16.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:15 smithi055 ceph-mon[35213]: Deploying daemon osd.8 on smithi144 2024-05-20T20:50:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-20T20:50:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:15 smithi026 ceph-mon[31990]: Deploying daemon osd.8 on smithi144 2024-05-20T20:50:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:16 smithi144 ceph-mon[35660]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-20T20:50:17.121 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:16 smithi055 ceph-mon[35213]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-20T20:50:17.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:16 smithi026 ceph-mon[31990]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-20T20:50:19.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:18 smithi144 ceph-mon[35660]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-20T20:50:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:18 smithi055 ceph-mon[35213]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-20T20:50:19.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:18 smithi026 ceph-mon[31990]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-20T20:50:21.514 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:21 smithi144 ceph-mon[35660]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:50:21.515 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:21.515 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:21.515 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:21 smithi055 ceph-mon[35213]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:50:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:21 smithi026 ceph-mon[31990]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:50:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:23 smithi144 ceph-mon[35660]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:23.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.672+0000 7fa660ff9700 1 -- 172.21.15.144:0/1196304267 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa664063980 con 0x7fa6500b2880 2024-05-20T20:50:23.672 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 8 on host 'smithi144' 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6500b2880 msgr2=0x7fa6500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6500b2880 0x7fa6500b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005fd0 tx=0x7fa64c005f00 comp rx=0 tx=0).stop 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 msgr2=0x7fa664102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 0x7fa664102090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c8f0 tx=0x7fa65400ccb0 comp rx=0 tx=0).stop 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 shutdown_connections 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6500b2880 0x7fa6500b4d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa664104090 0x7fa6641a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6641029b0 0x7fa664102090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 --2- 172.21.15.144:0/1196304267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6641036d0 0x7fa66419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 >> 172.21.15.144:0/1196304267 conn(0x7fa6640fd1d0 msgr2=0x7fa664106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:50:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 shutdown_connections 2024-05-20T20:50:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:50:23.674+0000 7fa6697a5700 1 -- 172.21.15.144:0/1196304267 wait complete. 2024-05-20T20:50:23.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:23 smithi055 ceph-mon[35213]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:23.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:23 smithi026 ceph-mon[31990]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:24.266 DEBUG:teuthology.orchestra.run.smithi144:osd.8> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.8.service 2024-05-20T20:50:24.268 INFO:tasks.cephadm:Deploying osd.9 on smithi026 with /dev/vg_nvme/lv_1... 2024-05-20T20:50:24.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-20T20:50:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: from='osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: from='osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T20:50:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T20:50:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: from='osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:24.957 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:25.889 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:25 smithi144 ceph-mon[35660]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-20T20:50:25.889 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:25 smithi144 ceph-mon[35660]: osdmap e51: 9 total, 8 up, 9 in 2024-05-20T20:50:25.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:25.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:25 smithi144 ceph-mon[35660]: from='osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:50:25.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:25 smithi144 ceph-mon[35660]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:50:25.890 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 20:50:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T20:50:25.674+0000 7f897d78f700 -1 osd.8 0 waiting for initial osdmap 2024-05-20T20:50:25.890 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 20:50:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T20:50:25.686+0000 7f8979582700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:50:25.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:25 smithi055 ceph-mon[35213]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-20T20:50:25.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:25 smithi055 ceph-mon[35213]: osdmap e51: 9 total, 8 up, 9 in 2024-05-20T20:50:25.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:25.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:25 smithi055 ceph-mon[35213]: from='osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:50:25.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:25 smithi055 ceph-mon[35213]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:50:25.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:25 smithi026 ceph-mon[31990]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-20T20:50:25.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:25 smithi026 ceph-mon[31990]: osdmap e51: 9 total, 8 up, 9 in 2024-05-20T20:50:25.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:25.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:25 smithi026 ceph-mon[31990]: from='osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:50:25.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:25 smithi026 ceph-mon[31990]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:50:26.613 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-20T20:50:26.613 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-20T20:50:26.613 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-20T20:50:26.614 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-20T20:50:26.614 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00809022 s, 1.3 GB/s 2024-05-20T20:50:26.614 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 2024-05-20T20:50:26.614 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-20T20:50:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:26 smithi026 ceph-mon[31990]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:26 smithi026 ceph-mon[31990]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:50:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:26 smithi026 ceph-mon[31990]: osdmap e52: 9 total, 8 up, 9 in 2024-05-20T20:50:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:26 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:26 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:26.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:26 smithi055 ceph-mon[35213]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:26.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:26 smithi055 ceph-mon[35213]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:50:26.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:26 smithi055 ceph-mon[35213]: osdmap e52: 9 total, 8 up, 9 in 2024-05-20T20:50:26.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:26 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:26.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:26 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:26.961 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:26 smithi144 ceph-mon[35660]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:26.961 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:26 smithi144 ceph-mon[35660]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:50:26.961 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:26 smithi144 ceph-mon[35660]: osdmap e52: 9 total, 8 up, 9 in 2024-05-20T20:50:26.961 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:26 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:26.961 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:26 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:27.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2024-05-20T20:50:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:27 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:27 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:27 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:27 smithi055 ceph-mon[35213]: osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] boot 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:27 smithi055 ceph-mon[35213]: osdmap e53: 9 total, 9 up, 9 in 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:27 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:27.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:27 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:50:27.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:27 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:50:27.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:27 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:27.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:27 smithi026 ceph-mon[31990]: osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] boot 2024-05-20T20:50:27.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:27 smithi026 ceph-mon[31990]: osdmap e53: 9 total, 9 up, 9 in 2024-05-20T20:50:27.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:27 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:28.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:27 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:50:28.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:27 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:50:28.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:27 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:28.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:27 smithi144 ceph-mon[35660]: osd.8 [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] boot 2024-05-20T20:50:28.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:27 smithi144 ceph-mon[35660]: osdmap e53: 9 total, 9 up, 9 in 2024-05-20T20:50:28.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:27 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T20:50:28.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi144 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: osdmap e54: 9 total, 9 up, 9 in 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi144 to 5696M 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:28.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:28.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:28.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:28 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi144 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: osdmap e54: 9 total, 9 up, 9 in 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi144 to 5696M 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:28.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:28 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi144 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: osdmap e54: 9 total, 9 up, 9 in 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi144 to 5696M 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:28 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:29.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/533082156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8105130 msgr2=0x7fe0c810b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:29.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/533082156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8105130 0x7fe0c810b9e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b0d0 tx=0x7fe0c000b3e0 comp rx=0 tx=0).stop 2024-05-20T20:50:29.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/533082156 shutdown_connections 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/533082156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8105130 0x7fe0c810b9e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/533082156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c8103590 0x7fe0c81039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/533082156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 0x7fe0c8104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/533082156 >> 172.21.15.26:0/533082156 conn(0x7fe0c80feb30 msgr2=0x7fe0c8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/533082156 shutdown_connections 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.396+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/533082156 wait complete. 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 Processor -- start 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 -- start start 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 0x7fe0c80772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 0x7fe0c8075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:29.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c8105130 0x7fe0c8075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c810eee0 con 0x7fe0c8103590 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c810ed60 con 0x7fe0c8104790 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cfb8d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c810f060 con 0x7fe0c8105130 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 0x7fe0c80772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 0x7fe0c80772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37866/0 (socket says 172.21.15.26:37866) 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 -- 172.21.15.26:0/1041954006 learned_addr learned my addr 172.21.15.26:0/1041954006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd128700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 0x7fe0c8075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0ce12a700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c8105130 0x7fe0c8075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 -- 172.21.15.26:0/1041954006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c8105130 msgr2=0x7fe0c8075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c8105130 0x7fe0c8075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 -- 172.21.15.26:0/1041954006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 msgr2=0x7fe0c8075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 0x7fe0c8075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd929700 1 -- 172.21.15.26:0/1041954006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8076720 con 0x7fe0c8103590 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.397+0000 7fe0cd128700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 0x7fe0c8075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.398+0000 7fe0cd929700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 0x7fe0c80772e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400e950 tx=0x7fe0c400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.398+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4004230 con 0x7fe0c8103590 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.398+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c81a3740 con 0x7fe0c8103590 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.398+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0c40043d0 con 0x7fe0c8103590 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.398+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4003c90 con 0x7fe0c8103590 2024-05-20T20:50:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.398+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c81a3c00 con 0x7fe0c8103590 2024-05-20T20:50:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.399+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c806a230 con 0x7fe0c8103590 2024-05-20T20:50:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.400+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe0c400c7e0 con 0x7fe0c8103590 2024-05-20T20:50:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.401+0000 7fe0beffd700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0b40b2880 0x7fe0b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.401+0000 7fe0cd128700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0b40b2880 0x7fe0b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.402+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7fe0c40e1c00 con 0x7fe0c8103590 2024-05-20T20:50:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.402+0000 7fe0cd128700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0b40b2880 0x7fe0b40b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011490 tx=0x7fe0b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:50:29.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.405+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0c40aa230 con 0x7fe0c8103590 2024-05-20T20:50:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:29.630+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe0c8063980 con 0x7fe0b40b2880 2024-05-20T20:50:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:29 smithi026 ceph-mon[31990]: osdmap e55: 9 total, 9 up, 9 in 2024-05-20T20:50:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:30.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:29 smithi144 ceph-mon[35660]: osdmap e55: 9 total, 9 up, 9 in 2024-05-20T20:50:30.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:30.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:30.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:29 smithi055 ceph-mon[35213]: osdmap e55: 9 total, 9 up, 9 in 2024-05-20T20:50:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:30.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:30 smithi026 ceph-mon[31990]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:30.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:30 smithi026 ceph-mon[31990]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:31.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:30 smithi144 ceph-mon[35660]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:31.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:30 smithi144 ceph-mon[35660]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:31.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:30 smithi055 ceph-mon[35213]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:31.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:30 smithi055 ceph-mon[35213]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:33.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:32 smithi144 ceph-mon[35660]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:33.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:32 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1272738973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258768d8-6e31-4618-9ec1-12e184f159a1"}]: dispatch 2024-05-20T20:50:33.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:32 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1272738973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "258768d8-6e31-4618-9ec1-12e184f159a1"}]': finished 2024-05-20T20:50:33.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:32 smithi144 ceph-mon[35660]: osdmap e56: 10 total, 9 up, 10 in 2024-05-20T20:50:33.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:32 smithi055 ceph-mon[35213]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:32 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1272738973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258768d8-6e31-4618-9ec1-12e184f159a1"}]: dispatch 2024-05-20T20:50:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:32 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1272738973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "258768d8-6e31-4618-9ec1-12e184f159a1"}]': finished 2024-05-20T20:50:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:32 smithi055 ceph-mon[35213]: osdmap e56: 10 total, 9 up, 10 in 2024-05-20T20:50:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:32 smithi026 ceph-mon[31990]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:32 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1272738973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258768d8-6e31-4618-9ec1-12e184f159a1"}]: dispatch 2024-05-20T20:50:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:32 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1272738973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "258768d8-6e31-4618-9ec1-12e184f159a1"}]': finished 2024-05-20T20:50:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:32 smithi026 ceph-mon[31990]: osdmap e56: 10 total, 9 up, 10 in 2024-05-20T20:50:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:34.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:33 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/185823501' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:50:34.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:33 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/185823501' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:50:34.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:33 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/185823501' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:50:35.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:34 smithi144 ceph-mon[35660]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:50:35.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:34 smithi055 ceph-mon[35213]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:50:35.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:34 smithi026 ceph-mon[31990]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:50:36.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:36 smithi026 ceph-mon[31990]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-20T20:50:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:36 smithi144 ceph-mon[35660]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-20T20:50:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:36 smithi055 ceph-mon[35213]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-20T20:50:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:38 smithi026 ceph-mon[31990]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-20T20:50:39.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:38 smithi144 ceph-mon[35660]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-20T20:50:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:38 smithi055 ceph-mon[35213]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-20T20:50:41.018 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:40 smithi026 ceph-mon[31990]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:50:41.018 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-20T20:50:41.018 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:40 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:41.018 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:40 smithi026 ceph-mon[31990]: Deploying daemon osd.9 on smithi026 2024-05-20T20:50:41.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:40 smithi144 ceph-mon[35660]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:50:41.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-20T20:50:41.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:40 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:41.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:40 smithi144 ceph-mon[35660]: Deploying daemon osd.9 on smithi026 2024-05-20T20:50:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:40 smithi055 ceph-mon[35213]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:50:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-20T20:50:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:40 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:40 smithi055 ceph-mon[35213]: Deploying daemon osd.9 on smithi026 2024-05-20T20:50:42.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:42 smithi026 ceph-mon[31990]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:50:43.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:42 smithi144 ceph-mon[35660]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:50:43.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:42 smithi055 ceph-mon[35213]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:50:45.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:44 smithi144 ceph-mon[35660]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-20T20:50:45.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:44 smithi026 ceph-mon[31990]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-20T20:50:45.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:44 smithi055 ceph-mon[35213]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-20T20:50:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:47 smithi026 ceph-mon[31990]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:47 smithi055 ceph-mon[35213]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:47 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:47 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:47 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:47.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:47 smithi144 ceph-mon[35660]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:47.709 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:47 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:47.709 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:47 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:47.709 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:47 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:49.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:49 smithi026 ceph-mon[31990]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:49 smithi055 ceph-mon[35213]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:49 smithi144 ceph-mon[35660]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:50.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.075+0000 7fe0beffd700 1 -- 172.21.15.26:0/1041954006 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe0c8063980 con 0x7fe0b40b2880 2024-05-20T20:50:50.075 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 9 on host 'smithi026' 2024-05-20T20:50:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0b40b2880 msgr2=0x7fe0b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0b40b2880 0x7fe0b40b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011490 tx=0x7fe0b8009260 comp rx=0 tx=0).stop 2024-05-20T20:50:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 msgr2=0x7fe0c80772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 0x7fe0c80772e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400e950 tx=0x7fe0c400ec60 comp rx=0 tx=0).stop 2024-05-20T20:50:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 shutdown_connections 2024-05-20T20:50:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0b40b2880 0x7fe0b40b4d40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c8103590 0x7fe0c80772e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c8105130 0x7fe0c8075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 --2- 172.21.15.26:0/1041954006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c8104790 0x7fe0c8075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.077+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 >> 172.21.15.26:0/1041954006 conn(0x7fe0c80feb30 msgr2=0x7fe0c8107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:50:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.078+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 shutdown_connections 2024-05-20T20:50:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:50:50.078+0000 7fe0cfb8d700 1 -- 172.21.15.26:0/1041954006 wait complete. 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:50.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-20T20:50:50.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:50:50.825 DEBUG:teuthology.orchestra.run.smithi026:osd.9> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.9.service 2024-05-20T20:50:50.828 INFO:tasks.cephadm:Deploying osd.10 on smithi055 with /dev/vg_nvme/lv_1... 2024-05-20T20:50:50.828 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-20T20:50:51.090 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:50:51.359 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:51 smithi026 ceph-mon[31990]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:51.359 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:51 smithi026 ceph-mon[31990]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-20T20:50:51.360 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:51 smithi026 ceph-mon[31990]: osdmap e57: 10 total, 9 up, 10 in 2024-05-20T20:50:51.360 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:51 smithi026 ceph-mon[31990]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:50:51.360 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:51.360 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 20:50:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T20:50:51.235+0000 7fd89e2c0700 -1 osd.9 0 waiting for initial osdmap 2024-05-20T20:50:51.360 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 20:50:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T20:50:51.249+0000 7fd89a0b3700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:50:51.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:51 smithi055 ceph-mon[35213]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:51.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:51 smithi055 ceph-mon[35213]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-20T20:50:51.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:51 smithi055 ceph-mon[35213]: osdmap e57: 10 total, 9 up, 10 in 2024-05-20T20:50:51.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:51 smithi055 ceph-mon[35213]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:50:51.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:51 smithi144 ceph-mon[35660]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:51 smithi144 ceph-mon[35660]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-20T20:50:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:51 smithi144 ceph-mon[35660]: osdmap e57: 10 total, 9 up, 10 in 2024-05-20T20:50:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:51 smithi144 ceph-mon[35660]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-20T20:50:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:52 smithi055 ceph-mon[35213]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:50:52.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:52 smithi055 ceph-mon[35213]: osdmap e58: 10 total, 9 up, 10 in 2024-05-20T20:50:52.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:52 smithi055 ceph-mon[35213]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:52.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.442 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:52 smithi026 ceph-mon[31990]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:50:52.442 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:52 smithi026 ceph-mon[31990]: osdmap e58: 10 total, 9 up, 10 in 2024-05-20T20:50:52.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:52 smithi026 ceph-mon[31990]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:52.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:52 smithi144 ceph-mon[35660]: from='osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-20T20:50:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:52 smithi144 ceph-mon[35660]: osdmap e58: 10 total, 9 up, 10 in 2024-05-20T20:50:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:52 smithi144 ceph-mon[35660]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-05-20T20:50:52.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:53.032 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-20T20:50:53.033 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-20T20:50:53.033 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-05-20T20:50:53.033 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-05-20T20:50:53.033 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148804 s, 705 MB/s 2024-05-20T20:50:53.033 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-05-20T20:50:53.296 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:53 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:50:53.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:53 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:50:53.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:53 smithi055 ceph-mon[35213]: osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] boot 2024-05-20T20:50:53.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:53 smithi055 ceph-mon[35213]: osdmap e59: 10 total, 10 up, 10 in 2024-05-20T20:50:53.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:53 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:50:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:53 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:50:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:53 smithi144 ceph-mon[35660]: osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] boot 2024-05-20T20:50:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:53 smithi144 ceph-mon[35660]: osdmap e59: 10 total, 10 up, 10 in 2024-05-20T20:50:53.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:53.607 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_1 2024-05-20T20:50:53.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:53 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:50:53.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:53 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:50:53.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:53 smithi026 ceph-mon[31990]: osd.9 [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] boot 2024-05-20T20:50:53.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:53 smithi026 ceph-mon[31990]: osdmap e59: 10 total, 10 up, 10 in 2024-05-20T20:50:53.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T20:50:53.854 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T20:50:54.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: osdmap e60: 10 total, 10 up, 10 in 2024-05-20T20:50:54.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi026 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi026 to 4273M 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:54.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: osdmap e60: 10 total, 10 up, 10 in 2024-05-20T20:50:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi026 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi026 to 4273M 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:54.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:54.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.646 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: osdmap e60: 10 total, 10 up, 10 in 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi026 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi026 to 4273M 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:54.647 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:50:54.648 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:50:55.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 -- 172.21.15.55:0/1230596814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5510106970 msgr2=0x7f5510108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 --2- 172.21.15.55:0/1230596814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5510106970 0x7f5510108d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5504009990 tx=0x7f5504009ca0 comp rx=0 tx=0).stop 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 -- 172.21.15.55:0/1230596814 shutdown_connections 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 --2- 172.21.15.55:0/1230596814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 0x7f5510106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 --2- 172.21.15.55:0/1230596814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55101092a0 0x7f551010b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 --2- 172.21.15.55:0/1230596814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5510106970 0x7f5510108d60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 -- 172.21.15.55:0/1230596814 >> 172.21.15.55:0/1230596814 conn(0x7f55100fda70 msgr2=0x7f55100ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 -- 172.21.15.55:0/1230596814 shutdown_connections 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.447+0000 7f5518029700 1 -- 172.21.15.55:0/1230596814 wait complete. 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 Processor -- start 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 -- start start 2024-05-20T20:50:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 0x7f551019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5510106970 0x7f551019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 0x7f55101a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f551010ef70 con 0x7f5510104040 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f551010edf0 con 0x7f55101092a0 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5518029700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f551010f0f0 con 0x7f5510106970 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 0x7f55101a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 0x7f55101a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37184/0 (socket says 172.21.15.55:37184) 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 -- 172.21.15.55:0/1815513006 learned_addr learned my addr 172.21.15.55:0/1815513006 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55155c4700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5510106970 0x7f551019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5515dc5700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 0x7f551019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 -- 172.21.15.55:0/1815513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5510106970 msgr2=0x7f551019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:55.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5510106970 0x7f551019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 -- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 msgr2=0x7f551019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 0x7f551019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f55165c6700 1 -- 172.21.15.55:0/1815513006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55101a3520 con 0x7f55101092a0 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.448+0000 7f5515dc5700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 0x7f551019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.449+0000 7f55165c6700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 0x7f55101a2e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5508007ec0 tx=0x7f550800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.449+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550800dbd0 con 0x7f55101092a0 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.449+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55101a3810 con 0x7f55101092a0 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.449+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f550800dd70 con 0x7f55101092a0 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.449+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508004480 con 0x7f55101092a0 2024-05-20T20:50:55.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.449+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55101a3d00 con 0x7f55101092a0 2024-05-20T20:50:55.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.450+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551006a230 con 0x7f55101092a0 2024-05-20T20:50:55.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.451+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f550801a030 con 0x7f55101092a0 2024-05-20T20:50:55.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.452+0000 7f5502ffd700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b2880 0x7f54fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:50:55.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.452+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f55080e15d0 con 0x7f55101092a0 2024-05-20T20:50:55.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.452+0000 7f5515dc5700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b2880 0x7f54fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:50:55.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.453+0000 7f5515dc5700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b2880 0x7f54fc0b4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f550c005950 tx=0x7f550c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:50:55.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.456+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5508021020 con 0x7f55101092a0 2024-05-20T20:50:55.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:50:55.679+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5510063a40 con 0x7f54fc0b2880 2024-05-20T20:50:55.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:55.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:56.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:56.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:56.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:56.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:50:56.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:50:56.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:50:57.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:56 smithi144 ceph-mon[35660]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:57.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:56 smithi144 ceph-mon[35660]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:57.103 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:56 smithi026 ceph-mon[31990]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:57.103 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:56 smithi026 ceph-mon[31990]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:57.190 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:56 smithi055 ceph-mon[35213]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:57.190 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:56 smithi055 ceph-mon[35213]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:50:59.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:58 smithi144 ceph-mon[35660]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:59.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:58 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/3242750966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]: dispatch 2024-05-20T20:50:59.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:58 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]: dispatch 2024-05-20T20:50:59.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:58 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]': finished 2024-05-20T20:50:59.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:58 smithi144 ceph-mon[35660]: osdmap e61: 11 total, 10 up, 11 in 2024-05-20T20:50:59.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:50:59.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:58 smithi055 ceph-mon[35213]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:59.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:58 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/3242750966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]: dispatch 2024-05-20T20:50:59.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:58 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]: dispatch 2024-05-20T20:50:59.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:58 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]': finished 2024-05-20T20:50:59.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:58 smithi055 ceph-mon[35213]: osdmap e61: 11 total, 10 up, 11 in 2024-05-20T20:50:59.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:50:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:58 smithi026 ceph-mon[31990]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:50:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:58 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/3242750966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]: dispatch 2024-05-20T20:50:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:58 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]: dispatch 2024-05-20T20:50:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:58 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea416660-9136-4ac1-a30e-f449d892b3cc"}]': finished 2024-05-20T20:50:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:58 smithi026 ceph-mon[31990]: osdmap e61: 11 total, 10 up, 11 in 2024-05-20T20:50:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:00.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:50:59 smithi144 ceph-mon[35660]: from='client.? 172.21.15.55:0/2924927406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:51:00.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:50:59 smithi055 ceph-mon[35213]: from='client.? 172.21.15.55:0/2924927406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:51:00.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:50:59 smithi026 ceph-mon[31990]: from='client.? 172.21.15.55:0/2924927406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:51:01.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:00 smithi144 ceph-mon[35660]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:01.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:00 smithi055 ceph-mon[35213]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:01.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:00 smithi026 ceph-mon[31990]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:03.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:02 smithi144 ceph-mon[35660]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:03.125 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:02 smithi055 ceph-mon[35213]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:03.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:02 smithi026 ceph-mon[31990]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:05.023 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:04 smithi055 ceph-mon[35213]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:05.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:04 smithi144 ceph-mon[35660]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:05.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:04 smithi026 ceph-mon[31990]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:06 smithi144 ceph-mon[35660]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:06 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-20T20:51:07.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:06 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:07.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:06 smithi144 ceph-mon[35660]: Deploying daemon osd.10 on smithi055 2024-05-20T20:51:07.055 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:06 smithi055 ceph-mon[35213]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:07.055 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:06 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-20T20:51:07.055 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:06 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:07.055 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:06 smithi055 ceph-mon[35213]: Deploying daemon osd.10 on smithi055 2024-05-20T20:51:07.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:06 smithi026 ceph-mon[31990]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:07.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:06 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-20T20:51:07.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:06 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:07.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:06 smithi026 ceph-mon[31990]: Deploying daemon osd.10 on smithi055 2024-05-20T20:51:09.009 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:08 smithi055 ceph-mon[35213]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:08 smithi144 ceph-mon[35660]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:09.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:08 smithi026 ceph-mon[31990]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:10.868 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:10 smithi055 ceph-mon[35213]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:10 smithi144 ceph-mon[35660]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:11.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:10 smithi026 ceph-mon[31990]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:12.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:12 smithi055 ceph-mon[35213]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:12.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:12.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:12.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:12 smithi144 ceph-mon[35660]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:12 smithi026 ceph-mon[31990]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:13.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:14.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:14 smithi055 ceph-mon[35213]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:14.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:14.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:14.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:14.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:14.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:14 smithi144 ceph-mon[35660]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:14 smithi026 ceph-mon[31990]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:15.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.648+0000 7f5502ffd700 1 -- 172.21.15.55:0/1815513006 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5510063a40 con 0x7f54fc0b2880 2024-05-20T20:51:15.649 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 10 on host 'smithi055' 2024-05-20T20:51:15.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b2880 msgr2=0x7f54fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:15.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b2880 0x7f54fc0b4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f550c005950 tx=0x7f550c0058e0 comp rx=0 tx=0).stop 2024-05-20T20:51:15.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 msgr2=0x7f55101a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:15.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 0x7f55101a2e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5508007ec0 tx=0x7f550800d3b0 comp rx=0 tx=0).stop 2024-05-20T20:51:15.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 shutdown_connections 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b2880 0x7f54fc0b4d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5510104040 0x7f551019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55101092a0 0x7f55101a2e80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 --2- 172.21.15.55:0/1815513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5510106970 0x7f551019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.650+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 >> 172.21.15.55:0/1815513006 conn(0x7f55100fda70 msgr2=0x7f55101037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.651+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 shutdown_connections 2024-05-20T20:51:15.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T20:51:15.651+0000 7f5518029700 1 -- 172.21.15.55:0/1815513006 wait complete. 2024-05-20T20:51:16.210 DEBUG:teuthology.orchestra.run.smithi055:osd.10> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.10.service 2024-05-20T20:51:16.212 INFO:tasks.cephadm:Deploying osd.11 on smithi144 with /dev/vg_nvme/lv_1... 2024-05-20T20:51:16.212 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-20T20:51:16.463 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:51:16.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:16.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:16.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:16.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:16 smithi144 ceph-mon[35660]: from='osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T20:51:16.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:16 smithi144 ceph-mon[35660]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T20:51:16.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:16 smithi144 ceph-mon[35660]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:16.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:16.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:16.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:16.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:16 smithi055 ceph-mon[35213]: from='osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T20:51:16.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:16 smithi055 ceph-mon[35213]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T20:51:16.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:16 smithi055 ceph-mon[35213]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:16 smithi026 ceph-mon[31990]: from='osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T20:51:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:16 smithi026 ceph-mon[31990]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T20:51:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:16 smithi026 ceph-mon[31990]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:17 smithi055 ceph-mon[35213]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-20T20:51:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:17 smithi055 ceph-mon[35213]: from='osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:51:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:17 smithi055 ceph-mon[35213]: osdmap e62: 11 total, 10 up, 11 in 2024-05-20T20:51:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:17.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:17 smithi055 ceph-mon[35213]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:51:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:17 smithi026 ceph-mon[31990]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-20T20:51:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:17 smithi026 ceph-mon[31990]: from='osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:51:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:17 smithi026 ceph-mon[31990]: osdmap e62: 11 total, 10 up, 11 in 2024-05-20T20:51:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:17.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:17 smithi026 ceph-mon[31990]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:51:17.988 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:17 smithi144 ceph-mon[35660]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-20T20:51:17.989 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:17 smithi144 ceph-mon[35660]: from='osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:51:17.989 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:17 smithi144 ceph-mon[35660]: osdmap e62: 11 total, 10 up, 11 in 2024-05-20T20:51:17.989 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:17.989 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:17 smithi144 ceph-mon[35660]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-05-20T20:51:18.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 20:51:18 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T20:51:18.083+0000 7f1f41bd7700 -1 osd.10 0 waiting for initial osdmap 2024-05-20T20:51:18.517 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 20:51:18 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T20:51:18.095+0000 7f1f399c2700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:51:18.581 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-20T20:51:18.581 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-20T20:51:18.582 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-20T20:51:18.582 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-20T20:51:18.582 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011309 s, 927 MB/s 2024-05-20T20:51:18.582 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-20T20:51:18.892 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:18 smithi144 ceph-mon[35660]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:51:18.892 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:18 smithi144 ceph-mon[35660]: osdmap e63: 11 total, 10 up, 11 in 2024-05-20T20:51:18.892 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:18.892 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:18.892 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:18 smithi144 ceph-mon[35660]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:18.892 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:18 smithi144 ceph-mon[35660]: from='osd.10 ' entity='osd.10' 2024-05-20T20:51:18.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:18 smithi055 ceph-mon[35213]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:51:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:18 smithi055 ceph-mon[35213]: osdmap e63: 11 total, 10 up, 11 in 2024-05-20T20:51:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:18 smithi055 ceph-mon[35213]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:18 smithi055 ceph-mon[35213]: from='osd.10 ' entity='osd.10' 2024-05-20T20:51:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:18 smithi026 ceph-mon[31990]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-05-20T20:51:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:18 smithi026 ceph-mon[31990]: osdmap e63: 11 total, 10 up, 11 in 2024-05-20T20:51:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:18 smithi026 ceph-mon[31990]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-05-20T20:51:18.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:18 smithi026 ceph-mon[31990]: from='osd.10 ' entity='osd.10' 2024-05-20T20:51:19.384 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-05-20T20:51:19.627 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:51:19.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:51:19.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:51:19.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:19.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] boot 2024-05-20T20:51:19.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: osdmap e64: 11 total, 11 up, 11 in 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:19.950 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] boot 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: osdmap e64: 11 total, 11 up, 11 in 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:51:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: osd.10 [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] boot 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: osdmap e64: 11 total, 11 up, 11 in 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:20.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:20 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi055 2024-05-20T20:51:20.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:20 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi055 to 4273M 2024-05-20T20:51:20.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:20 smithi055 ceph-mon[35213]: osdmap e65: 11 total, 11 up, 11 in 2024-05-20T20:51:20.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:20 smithi055 ceph-mon[35213]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:20 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi055 2024-05-20T20:51:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:20 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi055 to 4273M 2024-05-20T20:51:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:20 smithi026 ceph-mon[31990]: osdmap e65: 11 total, 11 up, 11 in 2024-05-20T20:51:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:20 smithi026 ceph-mon[31990]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:21.007 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:20 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi055 2024-05-20T20:51:21.007 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:20 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi055 to 4273M 2024-05-20T20:51:21.007 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:20 smithi144 ceph-mon[35660]: osdmap e65: 11 total, 11 up, 11 in 2024-05-20T20:51:21.007 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:20 smithi144 ceph-mon[35660]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:21.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 -- 172.21.15.144:0/334063246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01024a0 msgr2=0x7fb2f01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:21.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 --2- 172.21.15.144:0/334063246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01024a0 0x7fb2f01028c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009990 tx=0x7fb2e0009ca0 comp rx=0 tx=0).stop 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 -- 172.21.15.144:0/334063246 shutdown_connections 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 --2- 172.21.15.144:0/334063246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0104040 0x7fb2f010a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 --2- 172.21.15.144:0/334063246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01024a0 0x7fb2f01028c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 --2- 172.21.15.144:0/334063246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f01036a0 0x7fb2f0103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 -- 172.21.15.144:0/334063246 >> 172.21.15.144:0/334063246 conn(0x7fb2f00fda40 msgr2=0x7fb2f00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 -- 172.21.15.144:0/334063246 shutdown_connections 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.249+0000 7fb2f6104700 1 -- 172.21.15.144:0/334063246 wait complete. 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 Processor -- start 2024-05-20T20:51:21.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 -- start start 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f01024a0 0x7fb2f006bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 0x7fb2f019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0104040 0x7fb2f01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010de50 con 0x7fb2f01024a0 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010dcd0 con 0x7fb2f0104040 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2f6104700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010dfd0 con 0x7fb2f01036a0 2024-05-20T20:51:21.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2e6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 0x7fb2f019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2e6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 0x7fb2f019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42440/0 (socket says 172.21.15.144:42440) 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2e6ffd700 1 -- 172.21.15.144:0/153105769 learned_addr learned my addr 172.21.15.144:0/153105769 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.250+0000 7fb2ef7fe700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f01024a0 0x7fb2f006bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2effff700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0104040 0x7fb2f01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2e6ffd700 1 -- 172.21.15.144:0/153105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0104040 msgr2=0x7fb2f01a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2e6ffd700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0104040 0x7fb2f01a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2e6ffd700 1 -- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f01024a0 msgr2=0x7fb2f006bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2e6ffd700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f01024a0 0x7fb2f006bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2e6ffd700 1 -- 172.21.15.144:0/153105769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f01a1380 con 0x7fb2f01036a0 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2e6ffd700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 0x7fb2f019e470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b530 tx=0x7fb2d800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8015020 con 0x7fb2f01036a0 2024-05-20T20:51:21.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2d8011cc0 con 0x7fb2f01036a0 2024-05-20T20:51:21.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8010dc0 con 0x7fb2f01036a0 2024-05-20T20:51:21.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a1670 con 0x7fb2f01036a0 2024-05-20T20:51:21.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.251+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a1c80 con 0x7fb2f01036a0 2024-05-20T20:51:21.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.252+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f006a230 con 0x7fb2f01036a0 2024-05-20T20:51:21.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.253+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb2d8018020 con 0x7fb2f01036a0 2024-05-20T20:51:21.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.254+0000 7fb2ed7fa700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b2900 0x7fb2dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:21.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.254+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7fb2d80e1950 con 0x7fb2f01036a0 2024-05-20T20:51:21.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.254+0000 7fb2ef7fe700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b2900 0x7fb2dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:21.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.255+0000 7fb2ef7fe700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b2900 0x7fb2dc0b4dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0006010 tx=0x7fb2e00037c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:21.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.258+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2d801f020 con 0x7fb2f01036a0 2024-05-20T20:51:21.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:21.470+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb2f0063980 con 0x7fb2dc0b2900 2024-05-20T20:51:21.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:21 smithi055 ceph-mon[35213]: osdmap e66: 11 total, 11 up, 11 in 2024-05-20T20:51:21.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:51:21.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:51:21.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:21.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:21 smithi026 ceph-mon[31990]: osdmap e66: 11 total, 11 up, 11 in 2024-05-20T20:51:21.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:51:21.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:51:21.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:22.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:21 smithi144 ceph-mon[35660]: osdmap e66: 11 total, 11 up, 11 in 2024-05-20T20:51:22.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T20:51:22.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T20:51:22.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:22.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:22 smithi055 ceph-mon[35213]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:51:22.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:22 smithi055 ceph-mon[35213]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:22 smithi026 ceph-mon[31990]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:51:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:22 smithi026 ceph-mon[31990]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:23.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:22 smithi144 ceph-mon[35660]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:51:23.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:22 smithi144 ceph-mon[35660]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:24 smithi055 ceph-mon[35213]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:51:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:24 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/1754477442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]: dispatch 2024-05-20T20:51:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:24 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]: dispatch 2024-05-20T20:51:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:24 smithi055 ceph-mon[35213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]': finished 2024-05-20T20:51:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:24 smithi055 ceph-mon[35213]: osdmap e67: 12 total, 11 up, 12 in 2024-05-20T20:51:24.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:24 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:24 smithi026 ceph-mon[31990]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:51:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:24 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/1754477442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]: dispatch 2024-05-20T20:51:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:24 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]: dispatch 2024-05-20T20:51:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:24 smithi026 ceph-mon[31990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]': finished 2024-05-20T20:51:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:24 smithi026 ceph-mon[31990]: osdmap e67: 12 total, 11 up, 12 in 2024-05-20T20:51:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:24 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:25.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:24 smithi144 ceph-mon[35660]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:51:25.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:24 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/1754477442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]: dispatch 2024-05-20T20:51:25.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:24 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]: dispatch 2024-05-20T20:51:25.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:24 smithi144 ceph-mon[35660]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19"}]': finished 2024-05-20T20:51:25.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:24 smithi144 ceph-mon[35660]: osdmap e67: 12 total, 11 up, 12 in 2024-05-20T20:51:25.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:24 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:25.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:25 smithi055 ceph-mon[35213]: from='client.? 172.21.15.144:0/3403328523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:51:25.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:25 smithi026 ceph-mon[31990]: from='client.? 172.21.15.144:0/3403328523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:51:26.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:25 smithi144 ceph-mon[35660]: from='client.? 172.21.15.144:0/3403328523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T20:51:26.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:26 smithi055 ceph-mon[35213]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:51:26.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:26 smithi026 ceph-mon[31990]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:51:27.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:26 smithi144 ceph-mon[35660]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-20T20:51:28.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:28 smithi055 ceph-mon[35213]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-20T20:51:28.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:28 smithi026 ceph-mon[31990]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-20T20:51:29.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:28 smithi144 ceph-mon[35660]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-20T20:51:30.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:30 smithi055 ceph-mon[35213]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-20T20:51:30.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:30 smithi026 ceph-mon[31990]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-20T20:51:31.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:30 smithi144 ceph-mon[35660]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-20T20:51:32.794 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:32 smithi144 ceph-mon[35660]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:51:32.794 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-20T20:51:32.794 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:32.794 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:32 smithi144 ceph-mon[35660]: Deploying daemon osd.11 on smithi144 2024-05-20T20:51:32.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:32 smithi055 ceph-mon[35213]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:51:32.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-20T20:51:32.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:32.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:32 smithi055 ceph-mon[35213]: Deploying daemon osd.11 on smithi144 2024-05-20T20:51:32.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:32 smithi026 ceph-mon[31990]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T20:51:32.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-20T20:51:32.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:32.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:32 smithi026 ceph-mon[31990]: Deploying daemon osd.11 on smithi144 2024-05-20T20:51:34.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:34 smithi144 ceph-mon[35660]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:35.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:34 smithi055 ceph-mon[35213]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:35.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:34 smithi026 ceph-mon[31990]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:36 smithi144 ceph-mon[35660]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:36 smithi055 ceph-mon[35213]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:37.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:36 smithi026 ceph-mon[31990]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:38.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:38 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:38.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:38 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:38.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:38 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:38.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:38 smithi144 ceph-mon[35660]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:38.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:38 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:38.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:38 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:38.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:38 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:38.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:38 smithi055 ceph-mon[35213]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:38 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:38 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:38 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:38 smithi026 ceph-mon[31990]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:41.023 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:40 smithi144 ceph-mon[35660]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:41.096 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 11 on host 'smithi144' 2024-05-20T20:51:41.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.096+0000 7fb2ed7fa700 1 -- 172.21.15.144:0/153105769 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb2f0063980 con 0x7fb2dc0b2900 2024-05-20T20:51:41.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.098+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b2900 msgr2=0x7fb2dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:41.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.098+0000 7fb2f6104700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b2900 0x7fb2dc0b4dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0006010 tx=0x7fb2e00037c0 comp rx=0 tx=0).stop 2024-05-20T20:51:41.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.098+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 msgr2=0x7fb2f019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.098+0000 7fb2f6104700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 0x7fb2f019e470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b530 tx=0x7fb2d800b840 comp rx=0 tx=0).stop 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 shutdown_connections 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b2900 0x7fb2dc0b4dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f01024a0 0x7fb2f006bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0104040 0x7fb2f01a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 --2- 172.21.15.144:0/153105769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f01036a0 0x7fb2f019e470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 >> 172.21.15.144:0/153105769 conn(0x7fb2f00fda40 msgr2=0x7fb2f0106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 shutdown_connections 2024-05-20T20:51:41.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:41.099+0000 7fb2f6104700 1 -- 172.21.15.144:0/153105769 wait complete. 2024-05-20T20:51:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:40 smithi055 ceph-mon[35213]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:41.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:40 smithi026 ceph-mon[31990]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:41.864 DEBUG:teuthology.orchestra.run.smithi144:osd.11> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.11.service 2024-05-20T20:51:41.866 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-20T20:51:41.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-20T20:51:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:51:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:42 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:42 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:42 smithi026 ceph-mon[31990]: from='osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T20:51:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:42 smithi026 ceph-mon[31990]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T20:51:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:42 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:42 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:42 smithi144 ceph-mon[35660]: from='osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T20:51:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:42 smithi144 ceph-mon[35660]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T20:51:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:42 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:42 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:42 smithi055 ceph-mon[35213]: from='osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T20:51:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:42 smithi055 ceph-mon[35213]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T20:51:43.300 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 20:51:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T20:51:43.081+0000 7f1cc2a17700 -1 osd.11 0 waiting for initial osdmap 2024-05-20T20:51:43.300 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 20:51:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T20:51:43.093+0000 7f1cbe80a700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T20:51:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: from='osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: osdmap e68: 12 total, 11 up, 12 in 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:43.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:43 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:43.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:43.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-20T20:51:43.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: from='osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:51:43.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: osdmap e68: 12 total, 11 up, 12 in 2024-05-20T20:51:43.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:43.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:51:43.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:43.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:43 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: from='osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: osdmap e68: 12 total, 11 up, 12 in 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:43 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 -- 172.21.15.26:0/145249856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 msgr2=0x7fbf6410b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/145249856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf6410b720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b0a0 tx=0x7fbf5c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 -- 172.21.15.26:0/145249856 shutdown_connections 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/145249856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf6410b720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/145249856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf64104050 0x7fbf64106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/145249856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf64106980 0x7fbf64108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 -- 172.21.15.26:0/145249856 >> 172.21.15.26:0/145249856 conn(0x7fbf640fda60 msgr2=0x7fbf640ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 -- 172.21.15.26:0/145249856 shutdown_connections 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.992+0000 7fbf6ad26700 1 -- 172.21.15.26:0/145249856 wait complete. 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 Processor -- start 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 -- start start 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf64104050 0x7fbf641995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf64106980 0x7fbf64199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf641a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6410ef30 con 0x7fbf641092b0 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6410edb0 con 0x7fbf64106980 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf6ad26700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6410f0b0 con 0x7fbf64104050 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf641a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf641a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55466/0 (socket says 172.21.15.26:55466) 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 -- 172.21.15.26:0/3667967039 learned_addr learned my addr 172.21.15.26:0/3667967039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:51:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf63fff700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf64106980 0x7fbf64199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf68ac2700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf64104050 0x7fbf641995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 -- 172.21.15.26:0/3667967039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf64104050 msgr2=0x7fbf641995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf64104050 0x7fbf641995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 -- 172.21.15.26:0/3667967039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf64106980 msgr2=0x7fbf64199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf64106980 0x7fbf64199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 -- 172.21.15.26:0/3667967039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf641a1200 con 0x7fbf641092b0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.993+0000 7fbf692c3700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf641a0b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c006070 tx=0x7fbf5c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.994+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c021070 con 0x7fbf641092b0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.994+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf641a1490 con 0x7fbf641092b0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.994+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf5c007c00 con 0x7fbf641092b0 2024-05-20T20:51:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.994+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c004860 con 0x7fbf641092b0 2024-05-20T20:51:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.994+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641a18e0 con 0x7fbf641092b0 2024-05-20T20:51:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.995+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6406a230 con 0x7fbf641092b0 2024-05-20T20:51:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.996+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbf5c003f90 con 0x7fbf641092b0 2024-05-20T20:51:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.997+0000 7fbf61ffb700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf540b2880 0x7fbf540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.997+0000 7fbf68ac2700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf540b2880 0x7fbf540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.997+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7fbf5c00f020 con 0x7fbf641092b0 2024-05-20T20:51:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:43.998+0000 7fbf68ac2700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf540b2880 0x7fbf540b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005950 tx=0x7fbf580058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.001+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf5c02a020 con 0x7fbf641092b0 2024-05-20T20:51:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.219+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbf6402e630 con 0x7fbf641092b0 2024-05-20T20:51:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.219+0000 7fbf61ffb700 1 -- 172.21.15.26:0/3667967039 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fbf5c0ab650 con 0x7fbf641092b0 2024-05-20T20:51:44.219 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf540b2880 msgr2=0x7fbf540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf540b2880 0x7fbf540b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005950 tx=0x7fbf580058e0 comp rx=0 tx=0).stop 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 msgr2=0x7fbf641a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf641a0b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c006070 tx=0x7fbf5c009bf0 comp rx=0 tx=0).stop 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 shutdown_connections 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf540b2880 0x7fbf540b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf641092b0 0x7fbf641a0b60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf64104050 0x7fbf641995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 --2- 172.21.15.26:0/3667967039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf64106980 0x7fbf64199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 >> 172.21.15.26:0/3667967039 conn(0x7fbf640fda60 msgr2=0x7fbf641049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:44.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 shutdown_connections 2024-05-20T20:51:44.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:44.221+0000 7fbf6ad26700 1 -- 172.21.15.26:0/3667967039 wait complete. 2024-05-20T20:51:44.262 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:44 smithi026 ceph-mon[31990]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:51:44.263 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:44 smithi026 ceph-mon[31990]: osdmap e69: 12 total, 11 up, 12 in 2024-05-20T20:51:44.263 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.263 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.263 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:44 smithi055 ceph-mon[35213]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:51:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:44 smithi055 ceph-mon[35213]: osdmap e69: 12 total, 11 up, 12 in 2024-05-20T20:51:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:44 smithi144 ceph-mon[35660]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-20T20:51:44.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:44 smithi144 ceph-mon[35660]: osdmap e69: 12 total, 11 up, 12 in 2024-05-20T20:51:44.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:44.860 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1716238304,"num_in_osds":12,"osd_in_since":1716238284,"num_remapped_pgs":0} 2024-05-20T20:51:44.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:51:45.130 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:51:45.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: purged_snaps scrub starts 2024-05-20T20:51:45.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: purged_snaps scrub ok 2024-05-20T20:51:45.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] boot 2024-05-20T20:51:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: osdmap e70: 12 total, 12 up, 12 in 2024-05-20T20:51:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:45 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3667967039' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-20T20:51:45.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: purged_snaps scrub starts 2024-05-20T20:51:45.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: purged_snaps scrub ok 2024-05-20T20:51:45.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:45.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] boot 2024-05-20T20:51:45.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: osdmap e70: 12 total, 12 up, 12 in 2024-05-20T20:51:45.380 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:45.380 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:45 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3667967039' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: purged_snaps scrub starts 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: purged_snaps scrub ok 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: osd.11 [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] boot 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: osdmap e70: 12 total, 12 up, 12 in 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T20:51:45.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:45 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3667967039' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-20T20:51:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: osdmap e71: 12 total, 12 up, 12 in 2024-05-20T20:51:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:46.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: osdmap e71: 12 total, 12 up, 12 in 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:46.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:46.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:46.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:46.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: osdmap e71: 12 total, 12 up, 12 in 2024-05-20T20:51:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 -- 172.21.15.26:0/2046757842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 msgr2=0x7f4e2c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 --2- 172.21.15.26:0/2046757842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c1081c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e140099c0 tx=0x7f4e14009cd0 comp rx=0 tx=0).stop 2024-05-20T20:51:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 -- 172.21.15.26:0/2046757842 shutdown_connections 2024-05-20T20:51:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 --2- 172.21.15.26:0/2046757842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2c1088f0 0x7f4e2c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 --2- 172.21.15.26:0/2046757842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1037f0 0x7f4e2c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 --2- 172.21.15.26:0/2046757842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c1081c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 -- 172.21.15.26:0/2046757842 >> 172.21.15.26:0/2046757842 conn(0x7f4e2c07a6a0 msgr2=0x7f4e2c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 -- 172.21.15.26:0/2046757842 shutdown_connections 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.967+0000 7f4e323c7700 1 -- 172.21.15.26:0/2046757842 wait complete. 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 Processor -- start 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 -- start start 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1037f0 0x7f4e2c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2c1088f0 0x7f4e2c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1135d0 con 0x7f4e2c1041a0 2024-05-20T20:51:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c113450 con 0x7f4e2c1037f0 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e323c7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c113750 con 0x7f4e2c1088f0 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55484/0 (socket says 172.21.15.26:55484) 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e23fff700 1 -- 172.21.15.26:0/795392854 learned_addr learned my addr 172.21.15.26:0/795392854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e2bfff700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1037f0 0x7f4e2c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.968+0000 7f4e30964700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2c1088f0 0x7f4e2c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e23fff700 1 -- 172.21.15.26:0/795392854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2c1088f0 msgr2=0x7f4e2c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e23fff700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2c1088f0 0x7f4e2c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e23fff700 1 -- 172.21.15.26:0/795392854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1037f0 msgr2=0x7f4e2c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e23fff700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1037f0 0x7f4e2c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e23fff700 1 -- 172.21.15.26:0/795392854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1a8010 con 0x7f4e2c1041a0 2024-05-20T20:51:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e23fff700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c075d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b580 tx=0x7f4e140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14027070 con 0x7f4e2c1041a0 2024-05-20T20:51:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1400fc80 con 0x7f4e2c1041a0 2024-05-20T20:51:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a82a0 con 0x7f4e2c1041a0 2024-05-20T20:51:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14022d30 con 0x7f4e2c1041a0 2024-05-20T20:51:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.969+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a86f0 con 0x7f4e2c1041a0 2024-05-20T20:51:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.970+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c06a230 con 0x7f4e2c1041a0 2024-05-20T20:51:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.971+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4e1402a070 con 0x7f4e2c1041a0 2024-05-20T20:51:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.972+0000 7f4e29ffb700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e180b2880 0x7f4e180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.973+0000 7f4e2bfff700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e180b2880 0x7f4e180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.973+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f4e140e3280 con 0x7f4e2c1041a0 2024-05-20T20:51:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.973+0000 7f4e2bfff700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e180b2880 0x7f4e180b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005950 tx=0x7f4e1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:46.976+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e14031070 con 0x7f4e2c1041a0 2024-05-20T20:51:47.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.197+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e2c02e7c0 con 0x7f4e2c1041a0 2024-05-20T20:51:47.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.198+0000 7f4e29ffb700 1 -- 172.21.15.26:0/795392854 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18409 (secure 0 0 0) 0x7f4e14013070 con 0x7f4e2c1041a0 2024-05-20T20:51:47.199 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:51:47.199 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:51:45.097838+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e180b2880 msgr2=0x7f4e180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e180b2880 0x7f4e180b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005950 tx=0x7f4e1c00b410 comp rx=0 tx=0).stop 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 msgr2=0x7f4e2c075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c075d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b580 tx=0x7f4e140047b0 comp rx=0 tx=0).stop 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 shutdown_connections 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2c1088f0 0x7f4e2c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1037f0 0x7f4e2c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e180b2880 0x7f4e180b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 --2- 172.21.15.26:0/795392854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2c1041a0 0x7f4e2c075d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 >> 172.21.15.26:0/795392854 conn(0x7f4e2c07a6a0 msgr2=0x7f4e2c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 shutdown_connections 2024-05-20T20:51:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:47.201+0000 7f4e323c7700 1 -- 172.21.15.26:0/795392854 wait complete. 2024-05-20T20:51:47.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:47 smithi026 ceph-mon[31990]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:47.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:47 smithi026 ceph-mon[31990]: Detected new or changed devices on smithi144 2024-05-20T20:51:47.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:47 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi144 to 4272M 2024-05-20T20:51:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:47 smithi055 ceph-mon[35213]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:47 smithi055 ceph-mon[35213]: Detected new or changed devices on smithi144 2024-05-20T20:51:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:47 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi144 to 4272M 2024-05-20T20:51:47.454 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:47 smithi144 ceph-mon[35660]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:47.454 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:47 smithi144 ceph-mon[35660]: Detected new or changed devices on smithi144 2024-05-20T20:51:47.454 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:47 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi144 to 4272M 2024-05-20T20:51:47.840 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T20:47:57.693552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T20:51:47.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-20T20:51:48.089 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:51:48.377 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/795392854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:51:48.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/795392854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:51:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/795392854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:51:49.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:49 smithi026 ceph-mon[31990]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:49 smithi055 ceph-mon[35213]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:49 smithi144 ceph-mon[35660]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 -- 172.21.15.26:0/4125781037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102e30 msgr2=0x7f751410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 --2- 172.21.15.26:0/4125781037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102e30 0x7f751410f2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f750c00b0a0 tx=0x7f750c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 -- 172.21.15.26:0/4125781037 shutdown_connections 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 --2- 172.21.15.26:0/4125781037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102e30 0x7f751410f2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 --2- 172.21.15.26:0/4125781037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75141084b0 0x7f7514108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 --2- 172.21.15.26:0/4125781037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514102470 0x7f75141028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 -- 172.21.15.26:0/4125781037 >> 172.21.15.26:0/4125781037 conn(0x7f75140793f0 msgr2=0x7f7514079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 -- 172.21.15.26:0/4125781037 shutdown_connections 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.965+0000 7f751bd4d700 1 -- 172.21.15.26:0/4125781037 wait complete. 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 Processor -- start 2024-05-20T20:51:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 -- start start 2024-05-20T20:51:49.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 0x7f7514104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:49.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7514102e30 0x7f75141045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:49.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75141084b0 0x7f7514104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:49.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75141124f0 con 0x7f7514102470 2024-05-20T20:51:49.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7514112370 con 0x7f7514102e30 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751bd4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7514112670 con 0x7f75141084b0 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 0x7f7514104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 0x7f7514104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35624/0 (socket says 172.21.15.26:35624) 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 -- 172.21.15.26:0/3547005212 learned_addr learned my addr 172.21.15.26:0/3547005212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f75192e8700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7514102e30 0x7f75141045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751a2ea700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75141084b0 0x7f7514104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 -- 172.21.15.26:0/3547005212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75141084b0 msgr2=0x7f7514104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75141084b0 0x7f7514104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 -- 172.21.15.26:0/3547005212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7514102e30 msgr2=0x7f75141045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7514102e30 0x7f75141045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f7519ae9700 1 -- 172.21.15.26:0/3547005212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75141a8100 con 0x7f7514102470 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f751a2ea700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75141084b0 0x7f7514104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:51:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.966+0000 7f75192e8700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7514102e30 0x7f75141045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.967+0000 7f7519ae9700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 0x7f7514104080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f751000d950 tx=0x7f751000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.967+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510016070 con 0x7f7514102470 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.967+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7510009dd0 con 0x7f7514102470 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.967+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75141a8390 con 0x7f7514102470 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.967+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751000f3c0 con 0x7f7514102470 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.967+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141a8920 con 0x7f7514102470 2024-05-20T20:51:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.968+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751406a230 con 0x7f7514102470 2024-05-20T20:51:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.970+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7510019070 con 0x7f7514102470 2024-05-20T20:51:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.971+0000 7f750affd700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75000b2900 0x7f75000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.971+0000 7f75192e8700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75000b2900 0x7f75000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.971+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f75100e19d0 con 0x7f7514102470 2024-05-20T20:51:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.971+0000 7f75192e8700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75000b2900 0x7f75000b4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7514079b40 tx=0x7f7504009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:49.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:49.974+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75100a9af0 con 0x7f7514102470 2024-05-20T20:51:50.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.185+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75141055b0 con 0x7f7514102470 2024-05-20T20:51:50.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.186+0000 7f750affd700 1 -- 172.21.15.26:0/3547005212 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f75100a93f0 con 0x7f7514102470 2024-05-20T20:51:50.186 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-05-20T20:51:50.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75000b2900 msgr2=0x7f75000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:50.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75000b2900 0x7f75000b4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7514079b40 tx=0x7f7504009190 comp rx=0 tx=0).stop 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 msgr2=0x7f7514104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 0x7f7514104080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f751000d950 tx=0x7f751000dc60 comp rx=0 tx=0).stop 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 shutdown_connections 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75000b2900 0x7f75000b4dc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7514102470 0x7f7514104080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75141084b0 0x7f7514104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 --2- 172.21.15.26:0/3547005212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7514102e30 0x7f75141045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 >> 172.21.15.26:0/3547005212 conn(0x7f75140793f0 msgr2=0x7f751406c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 shutdown_connections 2024-05-20T20:51:50.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:50.188+0000 7f751bd4d700 1 -- 172.21.15.26:0/3547005212 wait complete. 2024-05-20T20:51:50.831 INFO:tasks.cephadm:Adding mds.a on smithi026 2024-05-20T20:51:50.831 INFO:tasks.cephadm:Adding mds.d on smithi026 2024-05-20T20:51:50.831 INFO:tasks.cephadm:Adding mds.g on smithi026 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.j on smithi026 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.b on smithi055 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.e on smithi055 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.h on smithi055 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.k on smithi055 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.c on smithi144 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.f on smithi144 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.i on smithi144 2024-05-20T20:51:50.832 INFO:tasks.cephadm:Adding mds.l on smithi144 2024-05-20T20:51:50.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l' 2024-05-20T20:51:51.084 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T20:51:51.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:51 smithi144 ceph-mon[35660]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:51.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:51 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3547005212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T20:51:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:51 smithi055 ceph-mon[35213]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:51 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3547005212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T20:51:51.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:51 smithi026 ceph-mon[31990]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:51.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:51 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3547005212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/2202200840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a01024b0 msgr2=0x7fd1a01028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/2202200840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a01024b0 0x7fd1a01028d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd198009990 tx=0x7fd198009ca0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/2202200840 shutdown_connections 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/2202200840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1a0104050 0x7fd1a010a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/2202200840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a01024b0 0x7fd1a01028d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/2202200840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 0x7fd1a0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/2202200840 >> 172.21.15.144:0/2202200840 conn(0x7fd1a00fda70 msgr2=0x7fd1a00ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:52.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/2202200840 shutdown_connections 2024-05-20T20:51:52.758 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/2202200840 wait complete. 2024-05-20T20:51:52.758 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 Processor -- start 2024-05-20T20:51:52.758 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.757+0000 7fd1a6bd2700 1 -- start start 2024-05-20T20:51:52.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1a01024b0 0x7fd1a0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:52.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 0x7fd1a0199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:52.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 0x7fd1a01a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:52.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a010de60 con 0x7fd1a01036b0 2024-05-20T20:51:52.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a010dce0 con 0x7fd1a01024b0 2024-05-20T20:51:52.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a010dfe0 con 0x7fd1a0104050 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 0x7fd1a01a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 0x7fd1a01a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:49086/0 (socket says 172.21.15.144:49086) 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 -- 172.21.15.144:0/3678083046 learned_addr learned my addr 172.21.15.144:0/3678083046 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd19ffff700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 0x7fd1a0199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 -- 172.21.15.144:0/3678083046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1a01024b0 msgr2=0x7fd1a0199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1a01024b0 0x7fd1a0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 -- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 msgr2=0x7fd1a0199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 0x7fd1a0199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 -- 172.21.15.144:0/3678083046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a01a12c0 con 0x7fd1a0104050 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd19ffff700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 0x7fd1a0199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a516f700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 0x7fd1a01a0c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd18800b580 tx=0x7fd18800b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:52.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188011830 con 0x7fd1a0104050 2024-05-20T20:51:52.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a01a15b0 con 0x7fd1a0104050 2024-05-20T20:51:52.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd188016070 con 0x7fd1a0104050 2024-05-20T20:51:52.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.758+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188010bf0 con 0x7fd1a0104050 2024-05-20T20:51:52.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.759+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a01a1bc0 con 0x7fd1a0104050 2024-05-20T20:51:52.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.760+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a006a230 con 0x7fd1a0104050 2024-05-20T20:51:52.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.761+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd1880119d0 con 0x7fd1a0104050 2024-05-20T20:51:52.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.761+0000 7fd19dffb700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd18c0b2900 0x7fd18c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:52.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.762+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd1880e1290 con 0x7fd1a0104050 2024-05-20T20:51:52.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.762+0000 7fd1a496e700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd18c0b2900 0x7fd18c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:52.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.762+0000 7fd1a496e700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd18c0b2900 0x7fd18c0b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b5c0 tx=0x7fd198005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:52.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.766+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1880a9450 con 0x7fd1a0104050 2024-05-20T20:51:52.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.978+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}) -- 0x7fd1a006c4a0 con 0x7fd18c0b2900 2024-05-20T20:51:52.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.987+0000 7fd19dffb700 1 -- 172.21.15.144:0/3678083046 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd1a006c4a0 con 0x7fd18c0b2900 2024-05-20T20:51:52.986 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mds.all update... 2024-05-20T20:51:52.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.989+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd18c0b2900 msgr2=0x7fd18c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:52.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.989+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd18c0b2900 0x7fd18c0b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b5c0 tx=0x7fd198005fb0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.989+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 msgr2=0x7fd1a01a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:52.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.989+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 0x7fd1a01a0c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd18800b580 tx=0x7fd18800b940 comp rx=0 tx=0).stop 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 shutdown_connections 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a0104050 0x7fd1a01a0c20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1a01024b0 0x7fd1a0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd18c0b2900 0x7fd18c0b4dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 --2- 172.21.15.144:0/3678083046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a01036b0 0x7fd1a0199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 >> 172.21.15.144:0/3678083046 conn(0x7fd1a00fda70 msgr2=0x7fd1a006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 shutdown_connections 2024-05-20T20:51:52.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-20T20:51:52.990+0000 7fd1a6bd2700 1 -- 172.21.15.144:0/3678083046 wait complete. 2024-05-20T20:51:53.408 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:53 smithi144 ceph-mon[35660]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:53.408 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:53.408 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:53 smithi055 ceph-mon[35213]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:53.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:53.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:53 smithi026 ceph-mon[31990]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:51:53.622 DEBUG:teuthology.orchestra.run.smithi026:mds.a> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.a.service 2024-05-20T20:51:53.625 DEBUG:teuthology.orchestra.run.smithi026:mds.d> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.d.service 2024-05-20T20:51:53.668 DEBUG:teuthology.orchestra.run.smithi026:mds.g> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.g.service 2024-05-20T20:51:53.711 DEBUG:teuthology.orchestra.run.smithi026:mds.j> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.j.service 2024-05-20T20:51:53.754 DEBUG:teuthology.orchestra.run.smithi055:mds.b> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.b.service 2024-05-20T20:51:53.756 DEBUG:teuthology.orchestra.run.smithi055:mds.e> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.e.service 2024-05-20T20:51:53.800 DEBUG:teuthology.orchestra.run.smithi055:mds.h> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.h.service 2024-05-20T20:51:53.843 DEBUG:teuthology.orchestra.run.smithi055:mds.k> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.k.service 2024-05-20T20:51:53.886 DEBUG:teuthology.orchestra.run.smithi144:mds.c> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.c.service 2024-05-20T20:51:53.889 DEBUG:teuthology.orchestra.run.smithi144:mds.f> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.f.service 2024-05-20T20:51:53.931 DEBUG:teuthology.orchestra.run.smithi144:mds.i> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.i.service 2024-05-20T20:51:53.974 DEBUG:teuthology.orchestra.run.smithi144:mds.l> sudo journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.l.service 2024-05-20T20:51:54.017 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-20T20:51:54.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:51:54.266 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:51:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:51:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-05-20T20:51:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:51:54.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:51:54.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:54.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:51:54.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-05-20T20:51:54.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:54.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:54.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:54.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:51:54.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:51:54.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:54.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:51:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:55.435 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:55 smithi144 ceph-mon[35660]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:55.435 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:55 smithi144 ceph-mon[35660]: Deploying daemon mds.l on smithi144 2024-05-20T20:51:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:55 smithi055 ceph-mon[35213]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:55 smithi055 ceph-mon[35213]: Deploying daemon mds.l on smithi144 2024-05-20T20:51:55.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:55 smithi026 ceph-mon[31990]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:55.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:55 smithi026 ceph-mon[31990]: Deploying daemon mds.l on smithi144 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 -- 172.21.15.26:0/63927036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c109620 msgr2=0x7f193c109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 --2- 172.21.15.26:0/63927036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c109620 0x7f193c109a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1938009990 tx=0x7f1938009ca0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 -- 172.21.15.26:0/63927036 shutdown_connections 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 --2- 172.21.15.26:0/63927036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c109620 0x7f193c109a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 --2- 172.21.15.26:0/63927036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c06c670 0x7f193c110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 --2- 172.21.15.26:0/63927036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06bcb0 0x7f193c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 -- 172.21.15.26:0/63927036 >> 172.21.15.26:0/63927036 conn(0x7f193c078490 msgr2=0x7f193c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 -- 172.21.15.26:0/63927036 shutdown_connections 2024-05-20T20:51:56.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 -- 172.21.15.26:0/63927036 wait complete. 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 Processor -- start 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.012+0000 7f1943ac2700 1 -- start start 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1943ac2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 0x7f193c1025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1943ac2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06c670 0x7f193c102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1943ac2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c109620 0x7f193c103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1943ac2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f193c113630 con 0x7f193c06bcb0 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1943ac2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1134b0 con 0x7f193c109620 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1943ac2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1137b0 con 0x7f193c06c670 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 0x7f193c1025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 0x7f193c1025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35632/0 (socket says 172.21.15.26:35632) 2024-05-20T20:51:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 -- 172.21.15.26:0/4033292475 learned_addr learned my addr 172.21.15.26:0/4033292475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194105d700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06c670 0x7f193c102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194205f700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c109620 0x7f193c103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 -- 172.21.15.26:0/4033292475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06c670 msgr2=0x7f193c102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06c670 0x7f193c102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 -- 172.21.15.26:0/4033292475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c109620 msgr2=0x7f193c103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c109620 0x7f193c103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 -- 172.21.15.26:0/4033292475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a81b0 con 0x7f193c06bcb0 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194105d700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06c670 0x7f193c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194205f700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c109620 0x7f193c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f194185e700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 0x7f193c1025a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1938000c00 tx=0x7f19380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.013+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938027070 con 0x7f193c06bcb0 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.014+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f193800fdb0 con 0x7f193c06bcb0 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.014+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a8440 con 0x7f193c06bcb0 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.014+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19380173f0 con 0x7f193c06bcb0 2024-05-20T20:51:56.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.014+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1a8890 con 0x7f193c06bcb0 2024-05-20T20:51:56.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.015+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c10bad0 con 0x7f193c06bcb0 2024-05-20T20:51:56.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.016+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f193802a070 con 0x7f193c06bcb0 2024-05-20T20:51:56.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.017+0000 7f1932ffd700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19280b2880 0x7f19280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:51:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.017+0000 7f194105d700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19280b2880 0x7f19280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:51:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.017+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f19380e3350 con 0x7f193c06bcb0 2024-05-20T20:51:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.017+0000 7f194105d700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19280b2880 0x7f19280b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f193c078bc0 tx=0x7f192c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:51:56.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.020+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1938031070 con 0x7f193c06bcb0 2024-05-20T20:51:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.283+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193c06a230 con 0x7f193c06bcb0 2024-05-20T20:51:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.283+0000 7f1932ffd700 1 -- 172.21.15.26:0/4033292475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f1938013070 con 0x7f193c06bcb0 2024-05-20T20:51:56.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:51:56.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-20T20:51:56.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19280b2880 msgr2=0x7f19280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19280b2880 0x7f19280b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f193c078bc0 tx=0x7f192c008040 comp rx=0 tx=0).stop 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 msgr2=0x7f193c1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 0x7f193c1025a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1938000c00 tx=0x7f19380047b0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 shutdown_connections 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19280b2880 0x7f19280b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c06bcb0 0x7f193c1025a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f193c109620 0x7f193c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 --2- 172.21.15.26:0/4033292475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c06c670 0x7f193c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.285+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 >> 172.21.15.26:0/4033292475 conn(0x7f193c078490 msgr2=0x7f193c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.286+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 shutdown_connections 2024-05-20T20:51:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:51:56.286+0000 7f1943ac2700 1 -- 172.21.15.26:0/4033292475 wait complete. 2024-05-20T20:51:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 1 2024-05-20T20:51:56.800 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 20:51:56 smithi144 systemd[1]: Started Ceph mds.l for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:51:57.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:57.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4033292475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6832/3730419260,v1:172.21.15.144:6833/3730419260] up:boot 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: fsmap 1 up:standby 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:51:57.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4033292475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6832/3730419260,v1:172.21.15.144:6833/3730419260] up:boot 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: fsmap 1 up:standby 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:51:57.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:51:57.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4033292475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:51:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6832/3730419260,v1:172.21.15.144:6833/3730419260] up:boot 2024-05-20T20:51:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: fsmap 1 up:standby 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:51:57.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:51:58.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:58 smithi055 ceph-mon[35213]: Deploying daemon mds.k on smithi055 2024-05-20T20:51:58.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:58 smithi026 ceph-mon[31990]: Deploying daemon mds.k on smithi055 2024-05-20T20:51:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:58 smithi144 ceph-mon[35660]: Deploying daemon mds.k on smithi055 2024-05-20T20:51:58.923 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:58 smithi055 systemd[1]: Starting Ceph mds.k for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:51:59.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:51:59.190 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:51:59 smithi055 ceph-mon[35213]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:58 smithi055 podman[64812]: 2024-05-20 20:51:58.826408644 +0000 UTC m=+0.017984161 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:58 smithi055 podman[64812]: 2024-05-20 20:51:58.930246206 +0000 UTC m=+0.121821706 container create dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:59 smithi055 podman[64812]: 2024-05-20 20:51:59.133380981 +0000 UTC m=+0.324956493 container init dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:59 smithi055 podman[64812]: 2024-05-20 20:51:59.136302807 +0000 UTC m=+0.327878305 container start dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:59 smithi055 ceph-mds[64831]: -- 172.21.15.55:0/1593216423 <== mon.2 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623bd897a40 con 0x5623bd86a800 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k[64827]: starting mds.k at 2024-05-20T20:51:59.191 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:59 smithi055 bash[64812]: dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 2024-05-20T20:51:59.288 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:51:59.412 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:51:59 smithi026 ceph-mon[31990]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:51:59.451 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 20:51:59 smithi055 systemd[1]: Started Ceph mds.k for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:51:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:51:59 smithi144 ceph-mon[35660]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6832/2396855261,v1:172.21.15.55:6833/2396855261] up:boot 2024-05-20T20:52:00.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: fsmap 2 up:standby 2024-05-20T20:52:00.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-20T20:52:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6832/2396855261,v1:172.21.15.55:6833/2396855261] up:boot 2024-05-20T20:52:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: fsmap 2 up:standby 2024-05-20T20:52:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-20T20:52:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6832/2396855261,v1:172.21.15.55:6833/2396855261] up:boot 2024-05-20T20:52:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: fsmap 2 up:standby 2024-05-20T20:52:00.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-20T20:52:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:01 smithi026 ceph-mon[31990]: Deploying daemon mds.j on smithi026 2024-05-20T20:52:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:01 smithi026 ceph-mon[31990]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:01 smithi144 ceph-mon[35660]: Deploying daemon mds.j on smithi026 2024-05-20T20:52:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:01 smithi144 ceph-mon[35660]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:01.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:01 smithi055 ceph-mon[35213]: Deploying daemon mds.j on smithi026 2024-05-20T20:52:01.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:01 smithi055 ceph-mon[35213]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 -- 172.21.15.26:0/2609011169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 msgr2=0x7fee28108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/2609011169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28108900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee24009990 tx=0x7fee24009ca0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 -- 172.21.15.26:0/2609011169 shutdown_connections 2024-05-20T20:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/2609011169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28108900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/2609011169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee28102e00 0x7fee2810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/2609011169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee28102440 0x7fee281028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 -- 172.21.15.26:0/2609011169 >> 172.21.15.26:0/2609011169 conn(0x7fee280793f0 msgr2=0x7fee28079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 -- 172.21.15.26:0/2609011169 shutdown_connections 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.003+0000 7fee2fe3e700 1 -- 172.21.15.26:0/2609011169 wait complete. 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.004+0000 7fee2fe3e700 1 Processor -- start 2024-05-20T20:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.004+0000 7fee2fe3e700 1 -- start start 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.004+0000 7fee2fe3e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee28102440 0x7fee28104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.004+0000 7fee2fe3e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee28102e00 0x7fee281045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2fe3e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2fe3e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee28112480 con 0x7fee28108520 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2fe3e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fee28112300 con 0x7fee28102e00 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2fe3e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee28112600 con 0x7fee28102440 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52972/0 (socket says 172.21.15.26:52972) 2024-05-20T20:52:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 -- 172.21.15.26:0/1349919933 learned_addr learned my addr 172.21.15.26:0/1349919933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2dbda700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee28102440 0x7fee28104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2d3d9700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee28102e00 0x7fee281045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 -- 172.21.15.26:0/1349919933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee28102440 msgr2=0x7fee28104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee28102440 0x7fee28104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 -- 172.21.15.26:0/1349919933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee28102e00 msgr2=0x7fee281045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee28102e00 0x7fee281045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 -- 172.21.15.26:0/1349919933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee281a8000 con 0x7fee28108520 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2e3db700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28104b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee2000ef30 tx=0x7fee2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200092e0 con 0x7fee28108520 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee20004bb0 con 0x7fee28108520 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.005+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281a8290 con 0x7fee28108520 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.006+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20004020 con 0x7fee28108520 2024-05-20T20:52:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.006+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281a8780 con 0x7fee28108520 2024-05-20T20:52:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.006+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2806a230 con 0x7fee28108520 2024-05-20T20:52:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.008+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fee2001b030 con 0x7fee28108520 2024-05-20T20:52:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.009+0000 7fee1effd700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee140b2880 0x7fee140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.009+0000 7fee2dbda700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee140b2880 0x7fee140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.009+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fee200e21e0 con 0x7fee28108520 2024-05-20T20:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.009+0000 7fee2dbda700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee140b2880 0x7fee140b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b2e0 tx=0x7fee24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.013+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee20022020 con 0x7fee28108520 2024-05-20T20:52:02.297 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:02.297 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T20:52:02.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.297+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee281a8a90 con 0x7fee28108520 2024-05-20T20:52:02.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.297+0000 7fee1effd700 1 -- 172.21.15.26:0/1349919933 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fee20010040 con 0x7fee28108520 2024-05-20T20:52:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee140b2880 msgr2=0x7fee140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee140b2880 0x7fee140b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b2e0 tx=0x7fee24005fb0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 msgr2=0x7fee28104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28104b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee2000ef30 tx=0x7fee2000c3a0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 shutdown_connections 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee140b2880 0x7fee140b4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28108520 0x7fee28104b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee28102440 0x7fee28104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 --2- 172.21.15.26:0/1349919933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee28102e00 0x7fee281045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 >> 172.21.15.26:0/1349919933 conn(0x7fee280793f0 msgr2=0x7fee2806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 shutdown_connections 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:02.300+0000 7fee2fe3e700 1 -- 172.21.15.26:0/1349919933 wait complete. 2024-05-20T20:52:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-05-20T20:52:03.365 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:03 smithi026 ceph-mon[31990]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:03.365 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:03 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1349919933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:03 smithi144 ceph-mon[35660]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:03 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1349919933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:03 smithi055 ceph-mon[35213]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:03 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1349919933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:03.952 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 20:52:03 smithi026 systemd[1]: Started Ceph mds.j for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:04.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: Deploying daemon mds.d on smithi026 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.26:6834/1873553840,v1:172.21.15.26:6835/1873553840] up:boot 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: fsmap 3 up:standby 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:04 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: Deploying daemon mds.d on smithi026 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.26:6834/1873553840,v1:172.21.15.26:6835/1873553840] up:boot 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: fsmap 3 up:standby 2024-05-20T20:52:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:04 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-20T20:52:05.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:52:05.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:05.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:05.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: Deploying daemon mds.d on smithi026 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.26:6834/1873553840,v1:172.21.15.26:6835/1873553840] up:boot 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: fsmap 3 up:standby 2024-05-20T20:52:05.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:04 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-20T20:52:05.427 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:05.952 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:05 smithi026 systemd[1]: Starting Ceph mds.d for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:06.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:06 smithi026 ceph-mon[31990]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:06.941 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:06 smithi026 podman[73300]: 2024-05-20 20:52:06.593651759 +0000 UTC m=+0.027064967 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:06.941 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:06 smithi026 podman[73300]: 2024-05-20 20:52:06.703677393 +0000 UTC m=+0.137090579 container create e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5) 2024-05-20T20:52:06.941 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:06 smithi026 podman[73300]: 2024-05-20 20:52:06.93812865 +0000 UTC m=+0.371541828 container init e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-20T20:52:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:06 smithi144 ceph-mon[35660]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:07.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:06 smithi055 ceph-mon[35213]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:07.202 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:06 smithi026 podman[73300]: 2024-05-20 20:52:06.941729684 +0000 UTC m=+0.375142871 container start e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T20:52:07.202 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:06 smithi026 ceph-mds[73334]: -- 172.21.15.26:0/611135451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56417e23ba40 con 0x56417e20f000 2024-05-20T20:52:07.202 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:06 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d[73330]: starting mds.d at 2024-05-20T20:52:07.202 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:07 smithi026 bash[73300]: e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 2024-05-20T20:52:07.202 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 20:52:07 smithi026 systemd[1]: Started Ceph mds.d for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 -- 172.21.15.26:0/783210660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 msgr2=0x7f75ec110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/783210660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec110430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f75e400b0a0 tx=0x7f75e400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 -- 172.21.15.26:0/783210660 shutdown_connections 2024-05-20T20:52:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/783210660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec110430 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/783210660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec109720 0x7f75ec109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/783210660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ec101f70 0x7f75ec1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 -- 172.21.15.26:0/783210660 >> 172.21.15.26:0/783210660 conn(0x7f75ec0776b0 msgr2=0x7f75ec077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 -- 172.21.15.26:0/783210660 shutdown_connections 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.548+0000 7f75f2f3f700 1 -- 172.21.15.26:0/783210660 wait complete. 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 Processor -- start 2024-05-20T20:52:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 -- start start 2024-05-20T20:52:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 0x7f75ec103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ec109720 0x7f75ec103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec113600 con 0x7f75ec102930 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec113480 con 0x7f75ec101f70 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.549+0000 7f75f2f3f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec113780 con 0x7f75ec109720 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 0x7f75ec103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75ebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 0x7f75ec103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41622/0 (socket says 172.21.15.26:41622) 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75ebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53038/0 (socket says 172.21.15.26:53038) 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 -- 172.21.15.26:0/952104758 learned_addr learned my addr 172.21.15.26:0/952104758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f14dc700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ec109720 0x7f75ec103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 -- 172.21.15.26:0/952104758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ec109720 msgr2=0x7f75ec103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ec109720 0x7f75ec103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 -- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 msgr2=0x7f75ec103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 -- 172.21.15.26:0/952104758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec1a7ff0 con 0x7f75ec101f70 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75ebfff700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f0cdb700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 0x7f75ec103150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00d980 tx=0x7f75dc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc016070 con 0x7f75ec101f70 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75dc004830 con 0x7f75ec101f70 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec1a82e0 con 0x7f75ec101f70 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.550+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc005440 con 0x7f75ec101f70 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.551+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec1a8820 con 0x7f75ec101f70 2024-05-20T20:52:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.552+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec06a230 con 0x7f75ec101f70 2024-05-20T20:52:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.553+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f75dc0055e0 con 0x7f75ec101f70 2024-05-20T20:52:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.554+0000 7f75e9ffb700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75d40b2880 0x7f75d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.554+0000 7f75ebfff700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75d40b2880 0x7f75d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.554+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f75dc0e2140 con 0x7f75ec101f70 2024-05-20T20:52:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.554+0000 7f75ebfff700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75d40b2880 0x7f75d40b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75ec077cb0 tx=0x7f75e000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.557+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75dc01f020 con 0x7f75ec101f70 2024-05-20T20:52:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.819+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75ec02e720 con 0x7f75ec101f70 2024-05-20T20:52:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.820+0000 7f75e9ffb700 1 -- 172.21.15.26:0/952104758 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f75dc0aa260 con 0x7f75ec101f70 2024-05-20T20:52:07.819 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:07.819 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T20:52:07.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75d40b2880 msgr2=0x7f75d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75d40b2880 0x7f75d40b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75ec077cb0 tx=0x7f75e000b3c0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 msgr2=0x7f75ec103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 0x7f75ec103150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00d980 tx=0x7f75dc00dc90 comp rx=0 tx=0).stop 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 shutdown_connections 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ec109720 0x7f75ec103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ec101f70 0x7f75ec103150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75d40b2880 0x7f75d40b4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 --2- 172.21.15.26:0/952104758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec102930 0x7f75ec103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 >> 172.21.15.26:0/952104758 conn(0x7f75ec0776b0 msgr2=0x7f75ec100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 shutdown_connections 2024-05-20T20:52:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:07.822+0000 7f75f2f3f700 1 -- 172.21.15.26:0/952104758 wait complete. 2024-05-20T20:52:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 5 2024-05-20T20:52:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:08.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: Deploying daemon mds.e on smithi055 2024-05-20T20:52:08.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.26:6836/1148458694,v1:172.21.15.26:6837/1148458694] up:boot 2024-05-20T20:52:08.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: fsmap 4 up:standby 2024-05-20T20:52:08.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-20T20:52:08.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:08 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/952104758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: Deploying daemon mds.e on smithi055 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.26:6836/1148458694,v1:172.21.15.26:6837/1148458694] up:boot 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: fsmap 4 up:standby 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-20T20:52:08.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:08 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/952104758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: Deploying daemon mds.e on smithi055 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.26:6836/1148458694,v1:172.21.15.26:6837/1148458694] up:boot 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: fsmap 4 up:standby 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-20T20:52:08.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:08 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/952104758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:09.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:09 smithi055 ceph-mon[35213]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:09.451 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 systemd[1]: Starting Ceph mds.e for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:09 smithi026 ceph-mon[31990]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:09 smithi144 ceph-mon[35660]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:09.778 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 podman[65122]: 2024-05-20 20:52:09.475455675 +0000 UTC m=+0.104461453 container create 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-05-20T20:52:09.778 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 podman[65122]: 2024-05-20 20:52:09.388763769 +0000 UTC m=+0.017769524 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:09.778 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 podman[65122]: 2024-05-20 20:52:09.695412638 +0000 UTC m=+0.324418375 container init 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-20T20:52:09.778 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 podman[65122]: 2024-05-20 20:52:09.698272803 +0000 UTC m=+0.327278543 container start 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-20T20:52:09.778 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 ceph-mds[65141]: -- 172.21.15.55:0/1950152781 <== mon.2 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5de8b5a40 con 0x55b5de889000 2024-05-20T20:52:09.779 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e[65137]: starting mds.e at 2024-05-20T20:52:09.779 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 bash[65122]: 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 2024-05-20T20:52:10.201 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 20:52:09 smithi055 systemd[1]: Started Ceph mds.e for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:10.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:52:10.984 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: Deploying daemon mds.f on smithi144 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6834/2325650145,v1:172.21.15.55:6835/2325650145] up:boot 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: fsmap 5 up:standby 2024-05-20T20:52:11.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: Deploying daemon mds.f on smithi144 2024-05-20T20:52:11.107 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6834/2325650145,v1:172.21.15.55:6835/2325650145] up:boot 2024-05-20T20:52:11.108 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: fsmap 5 up:standby 2024-05-20T20:52:11.108 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: Deploying daemon mds.f on smithi144 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6834/2325650145,v1:172.21.15.55:6835/2325650145] up:boot 2024-05-20T20:52:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: fsmap 5 up:standby 2024-05-20T20:52:11.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T20:52:12.050 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:11 smithi144 systemd[1]: Starting Ceph mds.f for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:12.323 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 podman[65638]: 2024-05-20 20:52:11.973368476 +0000 UTC m=+0.017367574 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:12.324 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 podman[65638]: 2024-05-20 20:52:12.088680977 +0000 UTC m=+0.132680103 container create f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-20T20:52:12.324 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 podman[65638]: 2024-05-20 20:52:12.324651617 +0000 UTC m=+0.368650709 container init f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-20T20:52:12.800 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 podman[65638]: 2024-05-20 20:52:12.327408373 +0000 UTC m=+0.371407467 container start f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T20:52:12.800 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mds[65658]: -- 172.21.15.144:0/3658729355 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591789d9a40 con 0x5591789acc00 2024-05-20T20:52:12.800 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f[65654]: starting mds.f at 2024-05-20T20:52:12.800 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 bash[65638]: f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e 2024-05-20T20:52:12.801 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 20:52:12 smithi144 systemd[1]: Started Ceph mds.f for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/3319340831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 msgr2=0x7fcc541103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/3319340831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc541103c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b0a0 tx=0x7fcc4c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/3319340831 shutdown_connections 2024-05-20T20:52:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/3319340831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc541103c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/3319340831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc541095a0 0x7fcc54109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/3319340831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5406bab0 0x7fcc5406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/3319340831 >> 172.21.15.26:0/3319340831 conn(0x7fcc540793b0 msgr2=0x7fcc540797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/3319340831 shutdown_connections 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.842+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/3319340831 wait complete. 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 Processor -- start 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 -- start start 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5406bab0 0x7fcc5419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc5419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc541095a0 0x7fcc541998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54113570 con 0x7fcc5406c470 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541133f0 con 0x7fcc541095a0 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5bfe1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541136f0 con 0x7fcc5406bab0 2024-05-20T20:52:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc5419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc59d7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5406bab0 0x7fcc5419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc5419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56724/0 (socket says 172.21.15.26:56724) 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 -- 172.21.15.26:0/4280362659 learned_addr learned my addr 172.21.15.26:0/4280362659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5a57e700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc541095a0 0x7fcc541998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 -- 172.21.15.26:0/4280362659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5406bab0 msgr2=0x7fcc5419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5406bab0 0x7fcc5419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 -- 172.21.15.26:0/4280362659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc541095a0 msgr2=0x7fcc541998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc541095a0 0x7fcc541998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.843+0000 7fcc5957c700 1 -- 172.21.15.26:0/4280362659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5419a050 con 0x7fcc5406c470 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.844+0000 7fcc5957c700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc5419fc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400d980 tx=0x7fcc4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.844+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44016070 con 0x7fcc5406c470 2024-05-20T20:52:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.844+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc44004830 con 0x7fcc5406c470 2024-05-20T20:52:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.844+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5419a2e0 con 0x7fcc5406c470 2024-05-20T20:52:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.844+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440054c0 con 0x7fcc5406c470 2024-05-20T20:52:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.844+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a6600 con 0x7fcc5406c470 2024-05-20T20:52:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.845+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5406a230 con 0x7fcc5406c470 2024-05-20T20:52:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.846+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcc44004220 con 0x7fcc5406c470 2024-05-20T20:52:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.847+0000 7fcc4affd700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcc400b2880 0x7fcc400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.847+0000 7fcc59d7d700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcc400b2880 0x7fcc400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.847+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fcc440e1fd0 con 0x7fcc5406c470 2024-05-20T20:52:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.847+0000 7fcc59d7d700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcc400b2880 0x7fcc400b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcc5419aa30 tx=0x7fcc5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:12.850+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc4401f020 con 0x7fcc5406c470 2024-05-20T20:52:13.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.112+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc5402e560 con 0x7fcc5406c470 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:13.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:13.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.113+0000 7fcc4affd700 1 -- 172.21.15.26:0/4280362659 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fcc440a5020 con 0x7fcc5406c470 2024-05-20T20:52:13.113 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-20T20:52:13.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcc400b2880 msgr2=0x7fcc400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:13.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcc400b2880 0x7fcc400b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcc5419aa30 tx=0x7fcc5000b410 comp rx=0 tx=0).stop 2024-05-20T20:52:13.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 msgr2=0x7fcc5419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc5419fc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400d980 tx=0x7fcc4400dc90 comp rx=0 tx=0).stop 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 shutdown_connections 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc541095a0 0x7fcc541998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5406bab0 0x7fcc5419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcc400b2880 0x7fcc400b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 --2- 172.21.15.26:0/4280362659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5406c470 0x7fcc5419fc70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 >> 172.21.15.26:0/4280362659 conn(0x7fcc540793b0 msgr2=0x7fcc54101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 shutdown_connections 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:13.116+0000 7fcc5bfe1700 1 -- 172.21.15.26:0/4280362659 wait complete. 2024-05-20T20:52:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-05-20T20:52:13.149 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:13.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:13.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:13.150 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:14.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:13 smithi144 ceph-mon[35660]: Deploying daemon mds.c on smithi144 2024-05-20T20:52:14.162 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:13 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6834/1712709614,v1:172.21.15.144:6835/1712709614] up:boot 2024-05-20T20:52:14.162 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:13 smithi144 ceph-mon[35660]: fsmap 6 up:standby 2024-05-20T20:52:14.162 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:13 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T20:52:14.162 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:13 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4280362659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:14.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:13 smithi055 ceph-mon[35213]: Deploying daemon mds.c on smithi144 2024-05-20T20:52:14.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:13 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6834/1712709614,v1:172.21.15.144:6835/1712709614] up:boot 2024-05-20T20:52:14.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:13 smithi055 ceph-mon[35213]: fsmap 6 up:standby 2024-05-20T20:52:14.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:13 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T20:52:14.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:13 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4280362659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:14.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:13 smithi026 ceph-mon[31990]: Deploying daemon mds.c on smithi144 2024-05-20T20:52:14.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:13 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6834/1712709614,v1:172.21.15.144:6835/1712709614] up:boot 2024-05-20T20:52:14.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:13 smithi026 ceph-mon[31990]: fsmap 6 up:standby 2024-05-20T20:52:14.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T20:52:14.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:13 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4280362659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:14.800 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:14 smithi144 systemd[1]: Starting Ceph mds.c for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:14 smithi026 ceph-mon[31990]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:15.211 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:14 smithi055 ceph-mon[35213]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:15.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:14 smithi144 ceph-mon[35660]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:15.263 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:14 smithi144 podman[65938]: 2024-05-20 20:52:14.755842848 +0000 UTC m=+0.017537881 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:15.263 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:14 smithi144 podman[65938]: 2024-05-20 20:52:14.88747877 +0000 UTC m=+0.149173782 container create a4e344c6d0bd7b067dc8f07eda983f63a25d3fd9ed49cb5e33f6e9c1715d6b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T20:52:15.263 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:15 smithi144 podman[65938]: 2024-05-20 20:52:15.249755106 +0000 UTC m=+0.511450126 container init a4e344c6d0bd7b067dc8f07eda983f63a25d3fd9ed49cb5e33f6e9c1715d6b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-20T20:52:15.550 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:15 smithi144 podman[65938]: 2024-05-20 20:52:15.252802796 +0000 UTC m=+0.514497827 container start a4e344c6d0bd7b067dc8f07eda983f63a25d3fd9ed49cb5e33f6e9c1715d6b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T20:52:15.551 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:15 smithi144 ceph-mds[65957]: -- 172.21.15.144:0/3561741313 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56211edf7a40 con 0x56211edcb000 2024-05-20T20:52:15.551 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: starting mds.c at 2024-05-20T20:52:15.551 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:15 smithi144 bash[65938]: a4e344c6d0bd7b067dc8f07eda983f63a25d3fd9ed49cb5e33f6e9c1715d6b27 2024-05-20T20:52:15.551 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:52:15 smithi144 systemd[1]: Started Ceph mds.c for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:15.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:52:16.047 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: Deploying daemon mds.h on smithi055 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6836/539511953,v1:172.21.15.144:6837/539511953] up:boot 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: fsmap 7 up:standby 2024-05-20T20:52:16.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: Deploying daemon mds.h on smithi055 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6836/539511953,v1:172.21.15.144:6837/539511953] up:boot 2024-05-20T20:52:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: fsmap 7 up:standby 2024-05-20T20:52:16.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T20:52:16.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:16.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:16.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:16.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:16.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: Deploying daemon mds.h on smithi055 2024-05-20T20:52:16.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:16.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6836/539511953,v1:172.21.15.144:6837/539511953] up:boot 2024-05-20T20:52:16.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: fsmap 7 up:standby 2024-05-20T20:52:16.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 -- 172.21.15.26:0/255013470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06b250 msgr2=0x7f2ffc06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 --2- 172.21.15.26:0/255013470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06b250 0x7f2ffc06b630 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b0a0 tx=0x7f2fec00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 -- 172.21.15.26:0/255013470 shutdown_connections 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 --2- 172.21.15.26:0/255013470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06b250 0x7f2ffc06b630 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 --2- 172.21.15.26:0/255013470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ffc10fff0 0x7f2ffc1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 --2- 172.21.15.26:0/255013470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 -- 172.21.15.26:0/255013470 >> 172.21.15.26:0/255013470 conn(0x7f2ffc0793f0 msgr2=0x7f2ffc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 -- 172.21.15.26:0/255013470 shutdown_connections 2024-05-20T20:52:17.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.694+0000 7f3001445700 1 -- 172.21.15.26:0/255013470 wait complete. 2024-05-20T20:52:17.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 Processor -- start 2024-05-20T20:52:17.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 -- start start 2024-05-20T20:52:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06b250 0x7f2ffc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ffc10fff0 0x7f2ffc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc115820 con 0x7f2ffc06bb70 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1156a0 con 0x7f2ffc06b250 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f3001445700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1159a0 con 0x7f2ffc10fff0 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f2ffa7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f2ffa7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56738/0 (socket says 172.21.15.26:56738) 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f2ffa7fc700 1 -- 172.21.15.26:0/398187346 learned_addr learned my addr 172.21.15.26:0/398187346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.695+0000 7f2ffaffd700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06b250 0x7f2ffc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffb7fe700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ffc10fff0 0x7f2ffc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffaffd700 1 -- 172.21.15.26:0/398187346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ffc10fff0 msgr2=0x7f2ffc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffaffd700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ffc10fff0 0x7f2ffc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffaffd700 1 -- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 msgr2=0x7f2ffc19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffaffd700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffaffd700 1 -- 172.21.15.26:0/398187346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1a0d20 con 0x7f2ffc06b250 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffa7fc700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2ffaffd700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06b250 0x7f2ffc19bd10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b070 tx=0x7f2fec004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec021070 con 0x7f2ffc06b250 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc1a0fb0 con 0x7f2ffc06b250 2024-05-20T20:52:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fec007c80 con 0x7f2ffc06b250 2024-05-20T20:52:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0052a0 con 0x7f2ffc06b250 2024-05-20T20:52:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.696+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a8890 con 0x7f2ffc06b250 2024-05-20T20:52:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.697+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc06a230 con 0x7f2ffc06b250 2024-05-20T20:52:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.699+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2fec024070 con 0x7f2ffc06b250 2024-05-20T20:52:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.699+0000 7f2fe3fff700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2fe40b2880 0x7f2fe40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.699+0000 7f2ffa7fc700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2fe40b2880 0x7f2fe40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.699+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2fec00f020 con 0x7f2ffc06b250 2024-05-20T20:52:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.700+0000 7f2ffa7fc700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2fe40b2880 0x7f2fe40b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc1039a0 tx=0x7f2ff0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:17.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.703+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fec02b020 con 0x7f2ffc06b250 2024-05-20T20:52:17.886 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:17 smithi055 systemd[1]: Starting Ceph mds.h for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.966+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ffc02e560 con 0x7f2ffc06b250 2024-05-20T20:52:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.967+0000 7f2fe3fff700 1 -- 172.21.15.26:0/398187346 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f2fec0abb00 con 0x7f2ffc06b250 2024-05-20T20:52:17.967 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:17.967 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2fe40b2880 msgr2=0x7f2fe40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2fe40b2880 0x7f2fe40b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc1039a0 tx=0x7f2ff0005f90 comp rx=0 tx=0).stop 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06b250 msgr2=0x7f2ffc19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06b250 0x7f2ffc19bd10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b070 tx=0x7f2fec004e40 comp rx=0 tx=0).stop 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 shutdown_connections 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ffc10fff0 0x7f2ffc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ffc06b250 0x7f2ffc19bd10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2fe40b2880 0x7f2fe40b4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 --2- 172.21.15.26:0/398187346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ffc06bb70 0x7f2ffc19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 >> 172.21.15.26:0/398187346 conn(0x7f2ffc0793f0 msgr2=0x7f2ffc1114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 shutdown_connections 2024-05-20T20:52:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:17.970+0000 7f3001445700 1 -- 172.21.15.26:0/398187346 wait complete. 2024-05-20T20:52:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-05-20T20:52:18.161 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:17 smithi055 podman[65427]: 2024-05-20 20:52:17.789504936 +0000 UTC m=+0.017959832 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:18.161 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:17 smithi055 podman[65427]: 2024-05-20 20:52:17.925077808 +0000 UTC m=+0.153532745 container create bacb3afa4c4ed7dcf666bb29c407dd2f07e3d801efb47cca2252088e1c825e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T20:52:18.161 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:18 smithi055 podman[65427]: 2024-05-20 20:52:18.162115276 +0000 UTC m=+0.390570183 container init bacb3afa4c4ed7dcf666bb29c407dd2f07e3d801efb47cca2252088e1c825e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-20T20:52:18.451 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:18 smithi055 podman[65427]: 2024-05-20 20:52:18.166753616 +0000 UTC m=+0.395208507 container start bacb3afa4c4ed7dcf666bb29c407dd2f07e3d801efb47cca2252088e1c825e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0) 2024-05-20T20:52:18.451 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mds[65447]: -- 172.21.15.55:0/3203521708 <== mon.2 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606bf2d9a40 con 0x5606bf2ac800 2024-05-20T20:52:18.451 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:18 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: starting mds.h at 2024-05-20T20:52:18.451 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:18 smithi055 bash[65427]: bacb3afa4c4ed7dcf666bb29c407dd2f07e3d801efb47cca2252088e1c825e16 2024-05-20T20:52:18.451 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 20:52:18 smithi055 systemd[1]: Started Ceph mds.h for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:18.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:18.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/398187346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:18.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:18.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:18.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:18.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:18.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:18.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:18 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/398187346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:18.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:18 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:19.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/398187346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:19.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:18 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:20.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:19 smithi144 ceph-mon[35660]: Deploying daemon mds.i on smithi144 2024-05-20T20:52:20.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:19 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:boot 2024-05-20T20:52:20.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:19 smithi144 ceph-mon[35660]: fsmap 8 up:standby 2024-05-20T20:52:20.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T20:52:20.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:19 smithi055 ceph-mon[35213]: Deploying daemon mds.i on smithi144 2024-05-20T20:52:20.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:19 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:boot 2024-05-20T20:52:20.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:19 smithi055 ceph-mon[35213]: fsmap 8 up:standby 2024-05-20T20:52:20.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T20:52:20.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:19 smithi026 ceph-mon[31990]: Deploying daemon mds.i on smithi144 2024-05-20T20:52:20.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:19 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:boot 2024-05-20T20:52:20.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:19 smithi026 ceph-mon[31990]: fsmap 8 up:standby 2024-05-20T20:52:20.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T20:52:20.550 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:20 smithi144 systemd[1]: Starting Ceph mds.i for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:20.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:52:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:20.924 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:20 smithi144 ceph-mon[35660]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:20.924 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:20 smithi144 podman[66255]: 2024-05-20 20:52:20.542369549 +0000 UTC m=+0.017187617 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:20.925 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:20 smithi144 podman[66255]: 2024-05-20 20:52:20.661497017 +0000 UTC m=+0.136315076 container create 12ed66c231dd8dbc8729f2a6b642bea2e795c0d9b351f826de1b02e900dfe8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-20T20:52:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:20 smithi026 ceph-mon[31990]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:21.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:20 smithi055 ceph-mon[35213]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:21.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:21 smithi144 podman[66255]: 2024-05-20 20:52:21.048329206 +0000 UTC m=+0.523147259 container init 12ed66c231dd8dbc8729f2a6b642bea2e795c0d9b351f826de1b02e900dfe8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T20:52:21.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:21 smithi144 podman[66255]: 2024-05-20 20:52:21.051277134 +0000 UTC m=+0.526095197 container start 12ed66c231dd8dbc8729f2a6b642bea2e795c0d9b351f826de1b02e900dfe8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-20T20:52:21.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:21 smithi144 ceph-mds[66274]: -- 172.21.15.144:0/209807885 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ba86a7a40 con 0x561ba867b000 2024-05-20T20:52:21.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i[66270]: starting mds.i at 2024-05-20T20:52:21.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:21 smithi144 bash[66255]: 12ed66c231dd8dbc8729f2a6b642bea2e795c0d9b351f826de1b02e900dfe8cc 2024-05-20T20:52:21.301 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 20:52:21 smithi144 systemd[1]: Started Ceph mds.i for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:22.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: Deploying daemon mds.g on smithi026 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] up:boot 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: fsmap 9 up:standby 2024-05-20T20:52:22.395 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:22 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: Deploying daemon mds.g on smithi026 2024-05-20T20:52:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] up:boot 2024-05-20T20:52:22.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: fsmap 9 up:standby 2024-05-20T20:52:22.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:22 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T20:52:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:22.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:22.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:22.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: Deploying daemon mds.g on smithi026 2024-05-20T20:52:22.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] up:boot 2024-05-20T20:52:22.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: fsmap 9 up:standby 2024-05-20T20:52:22.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:22 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T20:52:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 -- 172.21.15.26:0/3444624642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c8102470 msgr2=0x7fd8c81028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 --2- 172.21.15.26:0/3444624642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c8102470 0x7fd8c81028f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009990 tx=0x7fd8bc009ca0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 -- 172.21.15.26:0/3444624642 shutdown_connections 2024-05-20T20:52:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 --2- 172.21.15.26:0/3444624642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c8102e30 0x7fd8c810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 --2- 172.21.15.26:0/3444624642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c81084b0 0x7fd8c8108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 --2- 172.21.15.26:0/3444624642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c8102470 0x7fd8c81028f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 -- 172.21.15.26:0/3444624642 >> 172.21.15.26:0/3444624642 conn(0x7fd8c80793f0 msgr2=0x7fd8c8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 -- 172.21.15.26:0/3444624642 shutdown_connections 2024-05-20T20:52:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.651+0000 7fd8cceca700 1 -- 172.21.15.26:0/3444624642 wait complete. 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 Processor -- start 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 -- start start 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c8102470 0x7fd8c810fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c8102e30 0x7fd8c8110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 0x7fd8c8110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81135f0 con 0x7fd8c81084b0 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8113470 con 0x7fd8c8102e30 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8cceca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8113770 con 0x7fd8c8102470 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8c6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 0x7fd8c8110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8c6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 0x7fd8c8110680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52064/0 (socket says 172.21.15.26:52064) 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8c6d9d700 1 -- 172.21.15.26:0/2989505807 learned_addr learned my addr 172.21.15.26:0/2989505807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8c5d9b700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c8102e30 0x7fd8c8110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.652+0000 7fd8c659c700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c8102470 0x7fd8c810fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c6d9d700 1 -- 172.21.15.26:0/2989505807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c8102470 msgr2=0x7fd8c810fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c6d9d700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c8102470 0x7fd8c810fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c6d9d700 1 -- 172.21.15.26:0/2989505807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c8102e30 msgr2=0x7fd8c8110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c6d9d700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c8102e30 0x7fd8c8110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c6d9d700 1 -- 172.21.15.26:0/2989505807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c81a8090 con 0x7fd8c81084b0 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c659c700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c8102470 0x7fd8c810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8c6d9d700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 0x7fd8c8110680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007ee0 tx=0x7fd8c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000dbd0 con 0x7fd8c81084b0 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8c000dd70 con 0x7fd8c81084b0 2024-05-20T20:52:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00058d0 con 0x7fd8c81084b0 2024-05-20T20:52:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c81a8380 con 0x7fd8c81084b0 2024-05-20T20:52:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.653+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81a8840 con 0x7fd8c81084b0 2024-05-20T20:52:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.654+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c806a230 con 0x7fd8c81084b0 2024-05-20T20:52:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.656+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd8c001a020 con 0x7fd8c81084b0 2024-05-20T20:52:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.656+0000 7fd8b77fe700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8b00b2880 0x7fd8b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.657+0000 7fd8c659c700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8b00b2880 0x7fd8b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.657+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd8c00e1bc0 con 0x7fd8c81084b0 2024-05-20T20:52:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.657+0000 7fd8c659c700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8b00b2880 0x7fd8b00b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005fd0 tx=0x7fd8b8005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.660+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8c0010070 con 0x7fd8c81084b0 2024-05-20T20:52:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.932+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8c802e720 con 0x7fd8c81084b0 2024-05-20T20:52:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.933+0000 7fd8b77fe700 1 -- 172.21.15.26:0/2989505807 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fd8c00a9d80 con 0x7fd8c81084b0 2024-05-20T20:52:22.933 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:22.933 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-20T20:52:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8b00b2880 msgr2=0x7fd8b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8b00b2880 0x7fd8b00b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005fd0 tx=0x7fd8b8005d90 comp rx=0 tx=0).stop 2024-05-20T20:52:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 msgr2=0x7fd8c8110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 0x7fd8c8110680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007ee0 tx=0x7fd8c000d3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 shutdown_connections 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8b00b2880 0x7fd8b00b4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8c81084b0 0x7fd8c8110680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c8102470 0x7fd8c810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 --2- 172.21.15.26:0/2989505807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c8102e30 0x7fd8c8110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 >> 172.21.15.26:0/2989505807 conn(0x7fd8c80793f0 msgr2=0x7fd8c8100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 shutdown_connections 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:22.935+0000 7fd8cceca700 1 -- 172.21.15.26:0/2989505807 wait complete. 2024-05-20T20:52:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 10 2024-05-20T20:52:23.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:23 smithi026 ceph-mon[31990]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:23.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:23 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2989505807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:23 smithi144 ceph-mon[35660]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:23 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2989505807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:23.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:23 smithi055 ceph-mon[35213]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:23.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:23 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2989505807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:24.434 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 systemd[1]: Starting Ceph mds.g for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:24.702 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 podman[74147]: 2024-05-20 20:52:24.543615184 +0000 UTC m=+0.018009699 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:24.702 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 podman[74147]: 2024-05-20 20:52:24.661649566 +0000 UTC m=+0.136044142 container create 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-20T20:52:25.047 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 podman[74147]: 2024-05-20 20:52:24.941304235 +0000 UTC m=+0.415698743 container init 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-20T20:52:25.047 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 podman[74147]: 2024-05-20 20:52:24.944293418 +0000 UTC m=+0.418687935 container start 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T20:52:25.047 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 ceph-mds[74166]: -- 172.21.15.26:0/1376075620 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd3533da40 con 0x55cd35310800 2024-05-20T20:52:25.047 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:24 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g[74162]: starting mds.g at 2024-05-20T20:52:25.047 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:25 smithi026 bash[74147]: 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 20:52:25 smithi026 systemd[1]: Started Ceph mds.g for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:25 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:25.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:25.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:25 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:25 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:26.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:26 smithi026 ceph-mon[31990]: Deploying daemon mds.a on smithi026 2024-05-20T20:52:26.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:26 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.26:6838/841729682,v1:172.21.15.26:6839/841729682] up:boot 2024-05-20T20:52:26.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:26 smithi026 ceph-mon[31990]: fsmap 10 up:standby 2024-05-20T20:52:26.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:26 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-20T20:52:26.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:52:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:26 smithi144 ceph-mon[35660]: Deploying daemon mds.a on smithi026 2024-05-20T20:52:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:26 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.26:6838/841729682,v1:172.21.15.26:6839/841729682] up:boot 2024-05-20T20:52:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:26 smithi144 ceph-mon[35660]: fsmap 10 up:standby 2024-05-20T20:52:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:26 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-20T20:52:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:26 smithi055 ceph-mon[35213]: Deploying daemon mds.a on smithi026 2024-05-20T20:52:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:26 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.26:6838/841729682,v1:172.21.15.26:6839/841729682] up:boot 2024-05-20T20:52:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:26 smithi055 ceph-mon[35213]: fsmap 10 up:standby 2024-05-20T20:52:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:26 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-20T20:52:26.778 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:27 smithi144 ceph-mon[35660]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:27 smithi055 ceph-mon[35213]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:27.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:27 smithi026 ceph-mon[31990]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:28.170 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 20:52:28 smithi026 podman[74542]: 2024-05-20 20:52:28.145191512 +0000 UTC m=+0.361827373 container start f0eb87a4777e22387ccff7fdb06962e1c2462c839819d6f1bc1f9067c3c7037e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-20T20:52:28.170 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 20:52:28 smithi026 ceph-mds[74563]: -- 172.21.15.26:0/2336417161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df34ad9a40 con 0x55df34aac800 2024-05-20T20:52:28.452 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 20:52:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a[74558]: starting mds.a at 2024-05-20T20:52:28.452 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 20:52:28 smithi026 bash[74542]: f0eb87a4777e22387ccff7fdb06962e1c2462c839819d6f1bc1f9067c3c7037e 2024-05-20T20:52:28.452 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 20:52:28 smithi026 systemd[1]: Started Ceph mds.a for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 -- 172.21.15.26:0/99816888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4581095b0 msgr2=0x7fe458109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 --2- 172.21.15.26:0/99816888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4581095b0 0x7fe458109990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe44800b0a0 tx=0x7fe44800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 -- 172.21.15.26:0/99816888 shutdown_connections 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 --2- 172.21.15.26:0/99816888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4581095b0 0x7fe458109990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 --2- 172.21.15.26:0/99816888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 0x7fe4581103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 --2- 172.21.15.26:0/99816888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe45806b220 0x7fe45806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 -- 172.21.15.26:0/99816888 >> 172.21.15.26:0/99816888 conn(0x7fe4580793f0 msgr2=0x7fe458079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 -- 172.21.15.26:0/99816888 shutdown_connections 2024-05-20T20:52:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.112+0000 7fe45da6e700 1 -- 172.21.15.26:0/99816888 wait complete. 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 Processor -- start 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 -- start start 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 0x7fe45819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 0x7fe45819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4581095b0 0x7fe458199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe458113570 con 0x7fe45806b220 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581133f0 con 0x7fe4581095b0 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe45da6e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581136f0 con 0x7fe45806bbe0 2024-05-20T20:52:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe4577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 0x7fe45819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe4577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 0x7fe45819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52100/0 (socket says 172.21.15.26:52100) 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.113+0000 7fe4577fe700 1 -- 172.21.15.26:0/4283964370 learned_addr learned my addr 172.21.15.26:0/4283964370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe456ffd700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 0x7fe45819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe457fff700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4581095b0 0x7fe458199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe4577fe700 1 -- 172.21.15.26:0/4283964370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 msgr2=0x7fe45819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe4577fe700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 0x7fe45819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe4577fe700 1 -- 172.21.15.26:0/4283964370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4581095b0 msgr2=0x7fe458199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe4577fe700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4581095b0 0x7fe458199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe4577fe700 1 -- 172.21.15.26:0/4283964370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45819a090 con 0x7fe45806b220 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe456ffd700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 0x7fe45819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe4577fe700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 0x7fe45819f770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe4480062a0 tx=0x7fe448004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448021070 con 0x7fe45806b220 2024-05-20T20:52:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45819a320 con 0x7fe45806b220 2024-05-20T20:52:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe448007c80 con 0x7fe45806b220 2024-05-20T20:52:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4480052a0 con 0x7fe45806b220 2024-05-20T20:52:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.114+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581a6640 con 0x7fe45806b220 2024-05-20T20:52:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.115+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45810ba90 con 0x7fe45806b220 2024-05-20T20:52:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.117+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe448024070 con 0x7fe45806b220 2024-05-20T20:52:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.118+0000 7fe454ff9700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe4440b2900 0x7fe4440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.118+0000 7fe456ffd700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe4440b2900 0x7fe4440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.118+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe44800f020 con 0x7fe45806b220 2024-05-20T20:52:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.118+0000 7fe456ffd700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe4440b2900 0x7fe4440b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe458079b30 tx=0x7fe440005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.122+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe44802b030 con 0x7fe45806b220 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.26:6840/3119421551,v1:172.21.15.26:6841/3119421551] up:boot 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: fsmap 11 up:standby 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:29.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:29.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:29 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.26:6840/3119421551,v1:172.21.15.26:6841/3119421551] up:boot 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: fsmap 11 up:standby 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:29.373 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:29.373 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:29 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:29.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.397+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe45819ae60 con 0x7fe45806b220 2024-05-20T20:52:29.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.398+0000 7fe454ff9700 1 -- 172.21.15.26:0/4283964370 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7fe4480ab700 con 0x7fe45806b220 2024-05-20T20:52:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:29.398 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-20T20:52:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe4440b2900 msgr2=0x7fe4440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe4440b2900 0x7fe4440b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe458079b30 tx=0x7fe440005f50 comp rx=0 tx=0).stop 2024-05-20T20:52:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 msgr2=0x7fe45819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 0x7fe45819f770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe4480062a0 tx=0x7fe448004e80 comp rx=0 tx=0).stop 2024-05-20T20:52:29.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 shutdown_connections 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe4440b2900 0x7fe4440b4dc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe45806b220 0x7fe45819f770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4581095b0 0x7fe458199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 --2- 172.21.15.26:0/4283964370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45806bbe0 0x7fe45819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 >> 172.21.15.26:0/4283964370 conn(0x7fe4580793f0 msgr2=0x7fe458101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 shutdown_connections 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:29.400+0000 7fe45da6e700 1 -- 172.21.15.26:0/4283964370 wait complete. 2024-05-20T20:52:29.401 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2024-05-20T20:52:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.26:6840/3119421551,v1:172.21.15.26:6841/3119421551] up:boot 2024-05-20T20:52:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: fsmap 11 up:standby 2024-05-20T20:52:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-20T20:52:29.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:29.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T20:52:29.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T20:52:29.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:29 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:30.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:30 smithi055 ceph-mon[35213]: Deploying daemon mds.b on smithi055 2024-05-20T20:52:30.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:30 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4283964370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:30.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:30 smithi026 ceph-mon[31990]: Deploying daemon mds.b on smithi055 2024-05-20T20:52:30.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:30 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4283964370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:30.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:30 smithi144 ceph-mon[35660]: Deploying daemon mds.b on smithi055 2024-05-20T20:52:30.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:30 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4283964370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:30.701 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:30 smithi055 systemd[1]: Starting Ceph mds.b for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T20:52:31.035 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:30 smithi055 podman[65744]: 2024-05-20 20:52:30.686977973 +0000 UTC m=+0.017963141 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T20:52:31.036 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:30 smithi055 podman[65744]: 2024-05-20 20:52:30.808329374 +0000 UTC m=+0.139314527 container create cf6c4f297a7567ca4f8e928d10b6074f70ec3bd68d8655096271aef399ccc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-20T20:52:31.036 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:31 smithi055 podman[65744]: 2024-05-20 20:52:31.036390627 +0000 UTC m=+0.367375773 container init cf6c4f297a7567ca4f8e928d10b6074f70ec3bd68d8655096271aef399ccc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:31 smithi055 ceph-mon[35213]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:31 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:31 smithi055 podman[65744]: 2024-05-20 20:52:31.039864541 +0000 UTC m=+0.370849695 container start cf6c4f297a7567ca4f8e928d10b6074f70ec3bd68d8655096271aef399ccc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:31 smithi055 ceph-mds[65763]: -- 172.21.15.55:0/1797799926 <== mon.2 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627c34e3a40 con 0x5627c34b6c00 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:31 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b[65759]: starting mds.b at 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:31 smithi055 bash[65744]: cf6c4f297a7567ca4f8e928d10b6074f70ec3bd68d8655096271aef399ccc0fb 2024-05-20T20:52:31.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:31 smithi055 systemd[1]: Started Ceph mds.b for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T20:52:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:31 smithi144 ceph-mon[35660]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:31 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:31 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:31 smithi026 ceph-mon[31990]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:31 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:31 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] up:boot 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: fsmap 12 up:standby 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:32 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] up:boot 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: fsmap 12 up:standby 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:32 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:52:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] up:boot 2024-05-20T20:52:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: fsmap 12 up:standby 2024-05-20T20:52:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T20:52:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:32.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:32 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:52:32.786 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:33 smithi144 ceph-mon[35660]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:33 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:33 smithi026 ceph-mon[31990]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:33.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:33 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:33 smithi055 ceph-mon[35213]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:33 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 -- 172.21.15.26:0/3202278319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 msgr2=0x7ff9501089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 --2- 172.21.15.26:0/3202278319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff9501089e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff938009990 tx=0x7ff938009ca0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 -- 172.21.15.26:0/3202278319 shutdown_connections 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 --2- 172.21.15.26:0/3202278319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff9501089e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 --2- 172.21.15.26:0/3202278319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9501028a0 0x7ff950110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 --2- 172.21.15.26:0/3202278319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff950101ee0 0x7ff950102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 -- 172.21.15.26:0/3202278319 >> 172.21.15.26:0/3202278319 conn(0x7ff950077760 msgr2=0x7ff950077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 -- 172.21.15.26:0/3202278319 shutdown_connections 2024-05-20T20:52:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.460+0000 7ff956bd3700 1 -- 172.21.15.26:0/3202278319 wait complete. 2024-05-20T20:52:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 Processor -- start 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 -- start start 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff950101ee0 0x7ff95019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9501028a0 0x7ff95019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff950199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501135e0 con 0x7ff950108600 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff950113460 con 0x7ff9501028a0 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff956bd3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff950113760 con 0x7ff950101ee0 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff955170700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff950199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff955170700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff950199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42334/0 (socket says 172.21.15.26:42334) 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9501028a0 0x7ff95019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff955170700 1 -- 172.21.15.26:0/4217106218 learned_addr learned my addr 172.21.15.26:0/4217106218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.461+0000 7ff95496f700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff950101ee0 0x7ff95019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff955170700 1 -- 172.21.15.26:0/4217106218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff950101ee0 msgr2=0x7ff95019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff955170700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff950101ee0 0x7ff95019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff955170700 1 -- 172.21.15.26:0/4217106218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9501028a0 msgr2=0x7ff95019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff955170700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9501028a0 0x7ff95019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff955170700 1 -- 172.21.15.26:0/4217106218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff950199fe0 con 0x7ff950108600 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff955170700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff950199880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff94800d980 tx=0x7ff94800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948017070 con 0x7ff950108600 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff948005020 con 0x7ff950108600 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95019a2d0 con 0x7ff950108600 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94800f980 con 0x7ff950108600 2024-05-20T20:52:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.462+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a6590 con 0x7ff950108600 2024-05-20T20:52:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.463+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95006a230 con 0x7ff950108600 2024-05-20T20:52:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.464+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff948004a30 con 0x7ff950108600 2024-05-20T20:52:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.465+0000 7ff94dffb700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff93c0b2900 0x7ff93c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.465+0000 7ff95496f700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff93c0b2900 0x7ff93c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.465+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff9480e1b90 con 0x7ff950108600 2024-05-20T20:52:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.466+0000 7ff95496f700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff93c0b2900 0x7ff93c0b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b5c0 tx=0x7ff938005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.470+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff948020020 con 0x7ff950108600 2024-05-20T20:52:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.770+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff95002e560 con 0x7ff950108600 2024-05-20T20:52:34.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.771+0000 7ff94dffb700 1 -- 172.21.15.26:0/4217106218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7ff9480a9cb0 con 0x7ff950108600 2024-05-20T20:52:34.771 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:34.771 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-20T20:52:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff93c0b2900 msgr2=0x7ff93c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff93c0b2900 0x7ff93c0b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b5c0 tx=0x7ff938005960 comp rx=0 tx=0).stop 2024-05-20T20:52:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 msgr2=0x7ff950199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff950199880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff94800d980 tx=0x7ff94800dc90 comp rx=0 tx=0).stop 2024-05-20T20:52:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 shutdown_connections 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff93c0b2900 0x7ff93c0b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff950108600 0x7ff950199880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff950101ee0 0x7ff95019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 --2- 172.21.15.26:0/4217106218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9501028a0 0x7ff95019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 >> 172.21.15.26:0/4217106218 conn(0x7ff950077760 msgr2=0x7ff950100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 shutdown_connections 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:34.773+0000 7ff956bd3700 1 -- 172.21.15.26:0/4217106218 wait complete. 2024-05-20T20:52:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 13 2024-05-20T20:52:35.426 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4217106218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:35.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:52:35.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-20T20:52:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:52:35.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4217106218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:35.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4217106218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:52:35.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:52:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:36.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:36 smithi026 ceph-mon[31990]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:36.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:36 smithi055 ceph-mon[35213]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:36.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:36.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:36 smithi144 ceph-mon[35660]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:37.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2319882033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103590 msgr2=0x7fdb18103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2319882033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103590 0x7fdb18103a10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb080099c0 tx=0x7fdb08009cd0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2319882033 shutdown_connections 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2319882033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103f50 0x7fdb18110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2319882033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb18109640 0x7fdb18109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2319882033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103590 0x7fdb18103a10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2319882033 >> 172.21.15.26:0/2319882033 conn(0x7fdb1807a6f0 msgr2=0x7fdb1807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2319882033 shutdown_connections 2024-05-20T20:52:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.000+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2319882033 wait complete. 2024-05-20T20:52:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 Processor -- start 2024-05-20T20:52:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 -- start start 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103590 0x7fdb1819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 0x7fdb1819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb18109640 0x7fdb18199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18113600 con 0x7fdb18103f50 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18113480 con 0x7fdb18109640 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1f45a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18113780 con 0x7fdb18103590 2024-05-20T20:52:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.001+0000 7fdb1c9f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 0x7fdb1819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 0x7fdb1819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42360/0 (socket says 172.21.15.26:42360) 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 -- 172.21.15.26:0/2973825631 learned_addr learned my addr 172.21.15.26:0/2973825631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1d1f6700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103590 0x7fdb1819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1d9f7700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb18109640 0x7fdb18199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 -- 172.21.15.26:0/2973825631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103590 msgr2=0x7fdb1819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103590 0x7fdb1819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 -- 172.21.15.26:0/2973825631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb18109640 msgr2=0x7fdb18199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb18109640 0x7fdb18199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 -- 172.21.15.26:0/2973825631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1819a060 con 0x7fdb18103f50 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1c9f5700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 0x7fdb1819fc80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000c00 tx=0x7fdb080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb1d1f6700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103590 0x7fdb1819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.002+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004830 con 0x7fdb18103f50 2024-05-20T20:52:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.003+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb0800f7f0 con 0x7fdb18103f50 2024-05-20T20:52:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.003+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1819a2f0 con 0x7fdb18103f50 2024-05-20T20:52:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.003+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08005310 con 0x7fdb18103f50 2024-05-20T20:52:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.003+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb18075770 con 0x7fdb18103f50 2024-05-20T20:52:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.004+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb18104cb0 con 0x7fdb18103f50 2024-05-20T20:52:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.006+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdb08029060 con 0x7fdb18103f50 2024-05-20T20:52:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.007+0000 7fdb0e7fc700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdb040b2900 0x7fdb040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.007+0000 7fdb1d1f6700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdb040b2900 0x7fdb040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.007+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdb080e2950 con 0x7fdb18103f50 2024-05-20T20:52:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.007+0000 7fdb1d1f6700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdb040b2900 0x7fdb040b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005950 tx=0x7fdb1400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.010+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb08013070 con 0x7fdb18103f50 2024-05-20T20:52:38.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.249+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdb1802e560 con 0x7fdb18103f50 2024-05-20T20:52:38.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.249+0000 7fdb0e7fc700 1 -- 172.21.15.26:0/2973825631 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fdb080aab00 con 0x7fdb18103f50 2024-05-20T20:52:38.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:38.249 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-05-20T20:52:38.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T20:52:38.250 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-05-20T20:52:38.250 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-05-20T20:52:38.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:52:38.250 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdb040b2900 msgr2=0x7fdb040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdb040b2900 0x7fdb040b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005950 tx=0x7fdb1400a300 comp rx=0 tx=0).stop 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 msgr2=0x7fdb1819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 0x7fdb1819fc80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000c00 tx=0x7fdb080047b0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 shutdown_connections 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb18109640 0x7fdb18199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18103590 0x7fdb1819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdb040b2900 0x7fdb040b4dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 --2- 172.21.15.26:0/2973825631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb18103f50 0x7fdb1819fc80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:38.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 >> 172.21.15.26:0/2973825631 conn(0x7fdb1807a6f0 msgr2=0x7fdb181022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:38.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 shutdown_connections 2024-05-20T20:52:38.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:38.252+0000 7fdb1f45a700 1 -- 172.21.15.26:0/2973825631 wait complete. 2024-05-20T20:52:38.900 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T20:52:38.901 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T20:52:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2973825631' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:52:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2973825631' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:52:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2973825631' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-20T20:52:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:52:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:52:39.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:52:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 -- 172.21.15.26:0/3694846953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 msgr2=0x7fef94109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 --2- 172.21.15.26:0/3694846953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef94109990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef8400b0a0 tx=0x7fef8400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 -- 172.21.15.26:0/3694846953 shutdown_connections 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 --2- 172.21.15.26:0/3694846953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef94109990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 --2- 172.21.15.26:0/3694846953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9406c630 0x7fef941103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 --2- 172.21.15.26:0/3694846953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406bc70 0x7fef9406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 -- 172.21.15.26:0/3694846953 >> 172.21.15.26:0/3694846953 conn(0x7fef940793f0 msgr2=0x7fef94079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 -- 172.21.15.26:0/3694846953 shutdown_connections 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.862+0000 7fef9a086700 1 -- 172.21.15.26:0/3694846953 wait complete. 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 Processor -- start 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 -- start start 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9406bc70 0x7fef9419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406c630 0x7fef9419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef941a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef94113670 con 0x7fef941095b0 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef941134f0 con 0x7fef9406c630 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.863+0000 7fef9a086700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef941137f0 con 0x7fef9406bc70 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef941a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef941a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51864/0 (socket says 172.21.15.26:51864) 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 -- 172.21.15.26:0/1775643788 learned_addr learned my addr 172.21.15.26:0/1775643788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef937fe700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9406bc70 0x7fef9419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406c630 0x7fef9419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 -- 172.21.15.26:0/1775643788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9406bc70 msgr2=0x7fef9419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9406bc70 0x7fef9419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 -- 172.21.15.26:0/1775643788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406c630 msgr2=0x7fef9419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406c630 0x7fef9419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 -- 172.21.15.26:0/1775643788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef941a0f00 con 0x7fef941095b0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406c630 0x7fef9419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef93fff700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef941a07c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b550 tx=0x7fef8c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c015020 con 0x7fef941095b0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef941a1190 con 0x7fef941095b0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef8c011cc0 con 0x7fef941095b0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c010c80 con 0x7fef941095b0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.864+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941a8a70 con 0x7fef941095b0 2024-05-20T20:52:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.865+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9406a230 con 0x7fef941095b0 2024-05-20T20:52:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.867+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fef8c018020 con 0x7fef941095b0 2024-05-20T20:52:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.867+0000 7fef90ff9700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef800b2880 0x7fef800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.868+0000 7fef937fe700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef800b2880 0x7fef800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.868+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fef8c0e1a60 con 0x7fef941095b0 2024-05-20T20:52:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.868+0000 7fef937fe700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef800b2880 0x7fef800b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef84009e60 tx=0x7fef8400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:40.871+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef8c01f020 con 0x7fef941095b0 2024-05-20T20:52:41.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.086+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fef941a8c40 con 0x7fef941095b0 2024-05-20T20:52:41.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.174+0000 7fef90ff9700 1 -- 172.21.15.26:0/1775643788 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fef8c0a9c20 con 0x7fef941095b0 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef800b2880 msgr2=0x7fef800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef800b2880 0x7fef800b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef84009e60 tx=0x7fef8400b7b0 comp rx=0 tx=0).stop 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 msgr2=0x7fef941a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef941a07c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b550 tx=0x7fef8c00b860 comp rx=0 tx=0).stop 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 shutdown_connections 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef800b2880 0x7fef800b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef941095b0 0x7fef941a07c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9406bc70 0x7fef9419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 --2- 172.21.15.26:0/1775643788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef9406c630 0x7fef9419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 >> 172.21.15.26:0/1775643788 conn(0x7fef940793f0 msgr2=0x7fef94101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.177+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 shutdown_connections 2024-05-20T20:52:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:41.178+0000 7fef9a086700 1 -- 172.21.15.26:0/1775643788 wait complete. 2024-05-20T20:52:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-05-20T20:52:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:41 smithi055 ceph-mon[35213]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:41 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1775643788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-20T20:52:41.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:41 smithi026 ceph-mon[31990]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:41.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:41 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1775643788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-20T20:52:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:41 smithi144 ceph-mon[35660]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:41 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1775643788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-20T20:52:41.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T20:52:42.042 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:42.444 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:42 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1775643788' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-20T20:52:42.444 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:42 smithi026 ceph-mon[31990]: osdmap e72: 12 total, 12 up, 12 in 2024-05-20T20:52:42.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:42 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1775643788' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-20T20:52:42.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:42 smithi055 ceph-mon[35213]: osdmap e72: 12 total, 12 up, 12 in 2024-05-20T20:52:42.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:42 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1775643788' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-20T20:52:42.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:42 smithi144 ceph-mon[35660]: osdmap e72: 12 total, 12 up, 12 in 2024-05-20T20:52:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:43 smithi055 ceph-mon[35213]: pgmap v275: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:43 smithi055 ceph-mon[35213]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-20T20:52:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:43 smithi055 ceph-mon[35213]: osdmap e73: 12 total, 12 up, 12 in 2024-05-20T20:52:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:43 smithi026 ceph-mon[31990]: pgmap v275: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:43 smithi026 ceph-mon[31990]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-20T20:52:43.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:43 smithi026 ceph-mon[31990]: osdmap e73: 12 total, 12 up, 12 in 2024-05-20T20:52:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:43 smithi144 ceph-mon[35660]: pgmap v275: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:43 smithi144 ceph-mon[35660]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-20T20:52:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:43 smithi144 ceph-mon[35660]: osdmap e73: 12 total, 12 up, 12 in 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.728+0000 7efe60fac700 1 -- 172.21.15.26:0/3312303608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c101f00 msgr2=0x7efe5c102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.728+0000 7efe60fac700 1 --2- 172.21.15.26:0/3312303608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c101f00 0x7efe5c102360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe440099c0 tx=0x7efe44009cd0 comp rx=0 tx=0).stop 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 -- 172.21.15.26:0/3312303608 shutdown_connections 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 --2- 172.21.15.26:0/3312303608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1028a0 0x7efe5c112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 --2- 172.21.15.26:0/3312303608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c1043b0 0x7efe5c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 --2- 172.21.15.26:0/3312303608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c101f00 0x7efe5c102360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 -- 172.21.15.26:0/3312303608 >> 172.21.15.26:0/3312303608 conn(0x7efe5c077780 msgr2=0x7efe5c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 -- 172.21.15.26:0/3312303608 shutdown_connections 2024-05-20T20:52:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 -- 172.21.15.26:0/3312303608 wait complete. 2024-05-20T20:52:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.729+0000 7efe60fac700 1 Processor -- start 2024-05-20T20:52:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 -- start start 2024-05-20T20:52:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c101f00 0x7efe5c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 0x7efe5c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1043b0 0x7efe5c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1158f0 con 0x7efe5c1028a0 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c115770 con 0x7efe5c101f00 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe60fac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c115a70 con 0x7efe5c1043b0 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 0x7efe5c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 0x7efe5c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51882/0 (socket says 172.21.15.26:51882) 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 -- 172.21.15.26:0/1950322641 learned_addr learned my addr 172.21.15.26:0/1950322641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe5ad9d700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1043b0 0x7efe5c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe5a59c700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c101f00 0x7efe5c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 -- 172.21.15.26:0/1950322641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1043b0 msgr2=0x7efe5c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1043b0 0x7efe5c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 -- 172.21.15.26:0/1950322641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c101f00 msgr2=0x7efe5c19be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c101f00 0x7efe5c19be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe59d9b700 1 -- 172.21.15.26:0/1950322641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1a0e10 con 0x7efe5c1028a0 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.730+0000 7efe5ad9d700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1043b0 0x7efe5c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.731+0000 7efe59d9b700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 0x7efe5c19c340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe44000c00 tx=0x7efe4400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.731+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44026070 con 0x7efe5c1028a0 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.731+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1a10a0 con 0x7efe5c1028a0 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.731+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a8980 con 0x7efe5c1028a0 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.731+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe440054c0 con 0x7efe5c1028a0 2024-05-20T20:52:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.731+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44004c20 con 0x7efe5c1028a0 2024-05-20T20:52:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.732+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c005330 con 0x7efe5c1028a0 2024-05-20T20:52:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.733+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efe44029070 con 0x7efe5c1028a0 2024-05-20T20:52:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.734+0000 7efe537fe700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe480b2900 0x7efe480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.734+0000 7efe5a59c700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe480b2900 0x7efe480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.734+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7efe440e2c10 con 0x7efe5c1028a0 2024-05-20T20:52:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.735+0000 7efe5a59c700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe480b2900 0x7efe480b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe4c007720 tx=0x7efe4c0076b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.738+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe44030020 con 0x7efe5c1028a0 2024-05-20T20:52:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:43.956+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efe3c005620 con 0x7efe5c1028a0 2024-05-20T20:52:44.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.196+0000 7efe537fe700 1 -- 172.21.15.26:0/1950322641 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7efe44013070 con 0x7efe5c1028a0 2024-05-20T20:52:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.199+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe480b2900 msgr2=0x7efe480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.199+0000 7efe60fac700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe480b2900 0x7efe480b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe4c007720 tx=0x7efe4c0076b0 comp rx=0 tx=0).stop 2024-05-20T20:52:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 msgr2=0x7efe5c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 0x7efe5c19c340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe44000c00 tx=0x7efe4400f740 comp rx=0 tx=0).stop 2024-05-20T20:52:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 shutdown_connections 2024-05-20T20:52:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1043b0 0x7efe5c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c101f00 0x7efe5c19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe480b2900 0x7efe480b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 --2- 172.21.15.26:0/1950322641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c1028a0 0x7efe5c19c340 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe44000c00 tx=0x7efe4400f740 comp rx=0 tx=0).stop 2024-05-20T20:52:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 >> 172.21.15.26:0/1950322641 conn(0x7efe5c077780 msgr2=0x7efe5c0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 shutdown_connections 2024-05-20T20:52:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:44.200+0000 7efe60fac700 1 -- 172.21.15.26:0/1950322641 wait complete. 2024-05-20T20:52:44.202 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-05-20T20:52:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:44 smithi055 ceph-mon[35213]: osdmap e74: 12 total, 12 up, 12 in 2024-05-20T20:52:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:44 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1950322641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-20T20:52:44.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:44 smithi026 ceph-mon[31990]: osdmap e74: 12 total, 12 up, 12 in 2024-05-20T20:52:44.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:44 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1950322641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-20T20:52:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:44 smithi144 ceph-mon[35660]: osdmap e74: 12 total, 12 up, 12 in 2024-05-20T20:52:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:44 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1950322641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-20T20:52:44.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T20:52:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:45.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:45 smithi026 ceph-mon[31990]: pgmap v278: 33 pgs: 5 creating+activating, 5 creating+peering, 19 active+clean, 4 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:45.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:45 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1950322641' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-20T20:52:45.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:45 smithi026 ceph-mon[31990]: osdmap e75: 12 total, 12 up, 12 in 2024-05-20T20:52:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:45 smithi144 ceph-mon[35660]: pgmap v278: 33 pgs: 5 creating+activating, 5 creating+peering, 19 active+clean, 4 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:45 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1950322641' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-20T20:52:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:45 smithi144 ceph-mon[35660]: osdmap e75: 12 total, 12 up, 12 in 2024-05-20T20:52:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:45 smithi055 ceph-mon[35213]: pgmap v278: 33 pgs: 5 creating+activating, 5 creating+peering, 19 active+clean, 4 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:45 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1950322641' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-20T20:52:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:45 smithi055 ceph-mon[35213]: osdmap e75: 12 total, 12 up, 12 in 2024-05-20T20:52:46.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:46 smithi026 ceph-mon[31990]: osdmap e76: 12 total, 12 up, 12 in 2024-05-20T20:52:46.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:46 smithi026 ceph-mon[31990]: osdmap e77: 12 total, 12 up, 12 in 2024-05-20T20:52:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:46 smithi144 ceph-mon[35660]: osdmap e76: 12 total, 12 up, 12 in 2024-05-20T20:52:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:46 smithi144 ceph-mon[35660]: osdmap e77: 12 total, 12 up, 12 in 2024-05-20T20:52:46.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:46 smithi055 ceph-mon[35213]: osdmap e76: 12 total, 12 up, 12 in 2024-05-20T20:52:46.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:46 smithi055 ceph-mon[35213]: osdmap e77: 12 total, 12 up, 12 in 2024-05-20T20:52:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 -- 172.21.15.26:0/2593388493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be006bd10 msgr2=0x7f3be006c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 --2- 172.21.15.26:0/2593388493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be006bd10 0x7f3be006c190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00099c0 tx=0x7f3bd0009cd0 comp rx=0 tx=0).stop 2024-05-20T20:52:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 -- 172.21.15.26:0/2593388493 shutdown_connections 2024-05-20T20:52:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 --2- 172.21.15.26:0/2593388493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be006c6d0 0x7f3be0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 --2- 172.21.15.26:0/2593388493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be0109630 0x7f3be0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 --2- 172.21.15.26:0/2593388493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be006bd10 0x7f3be006c190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 -- 172.21.15.26:0/2593388493 >> 172.21.15.26:0/2593388493 conn(0x7f3be00794e0 msgr2=0x7f3be00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 -- 172.21.15.26:0/2593388493 shutdown_connections 2024-05-20T20:52:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.915+0000 7f3be7872700 1 -- 172.21.15.26:0/2593388493 wait complete. 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 Processor -- start 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 -- start start 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bd10 0x7f3be019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be006c6d0 0x7f3be019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 0x7f3be0199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0113640 con 0x7f3be0109630 2024-05-20T20:52:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01134c0 con 0x7f3be006bd10 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be7872700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01137c0 con 0x7f3be006c6d0 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be5e0f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 0x7f3be0199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be5e0f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 0x7f3be0199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51900/0 (socket says 172.21.15.26:51900) 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be560e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bd10 0x7f3be019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.916+0000 7f3be5e0f700 1 -- 172.21.15.26:0/3357248614 learned_addr learned my addr 172.21.15.26:0/3357248614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be4e0d700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be006c6d0 0x7f3be019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be5e0f700 1 -- 172.21.15.26:0/3357248614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be006c6d0 msgr2=0x7f3be019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be5e0f700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be006c6d0 0x7f3be019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be5e0f700 1 -- 172.21.15.26:0/3357248614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bd10 msgr2=0x7f3be019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be5e0f700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bd10 0x7f3be019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be5e0f700 1 -- 172.21.15.26:0/3357248614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be019a020 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be5e0f700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 0x7f3be0199840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8007d70 tx=0x7f3bd800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8016020 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd800ad40 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be019a310 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80115e0 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.917+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be01a6520 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.918+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0104ca0 con 0x7f3be0109630 2024-05-20T20:52:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.920+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3bd8019020 con 0x7f3be0109630 2024-05-20T20:52:46.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.921+0000 7f3bd67fc700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3bcc0b2880 0x7f3bcc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:46.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.921+0000 7f3be560e700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3bcc0b2880 0x7f3bcc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:46.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.921+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f3bd80e25a0 con 0x7f3be0109630 2024-05-20T20:52:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.922+0000 7f3be560e700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3bcc0b2880 0x7f3bcc0b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00ac50 tx=0x7f3bdc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:46.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:46.925+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3bd8020020 con 0x7f3be0109630 2024-05-20T20:52:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.189+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3be010b4a0 con 0x7f3be0109630 2024-05-20T20:52:47.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.225+0000 7f3bd67fc700 1 -- 172.21.15.26:0/3357248614 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f3bd80aa3d0 con 0x7f3be0109630 2024-05-20T20:52:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3bcc0b2880 msgr2=0x7f3bcc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3bcc0b2880 0x7f3bcc0b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00ac50 tx=0x7f3bdc00a380 comp rx=0 tx=0).stop 2024-05-20T20:52:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 msgr2=0x7f3be0199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 0x7f3be0199840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8007d70 tx=0x7f3bd800a340 comp rx=0 tx=0).stop 2024-05-20T20:52:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 shutdown_connections 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3bcc0b2880 0x7f3bcc0b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be0109630 0x7f3be0199840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bd10 0x7f3be019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 --2- 172.21.15.26:0/3357248614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be006c6d0 0x7f3be019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 >> 172.21.15.26:0/3357248614 conn(0x7f3be00794e0 msgr2=0x7f3be0101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 shutdown_connections 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:47.228+0000 7f3be7872700 1 -- 172.21.15.26:0/3357248614 wait complete. 2024-05-20T20:52:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-05-20T20:52:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:47 smithi026 ceph-mon[31990]: pgmap v281: 97 pgs: 5 creating+activating, 22 creating+peering, 23 active+clean, 47 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:47 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3357248614' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-20T20:52:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:47 smithi026 ceph-mon[31990]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-20T20:52:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:47 smithi144 ceph-mon[35660]: pgmap v281: 97 pgs: 5 creating+activating, 22 creating+peering, 23 active+clean, 47 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:47 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3357248614' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-20T20:52:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:47 smithi144 ceph-mon[35660]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-20T20:52:47.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:47 smithi055 ceph-mon[35213]: pgmap v281: 97 pgs: 5 creating+activating, 22 creating+peering, 23 active+clean, 47 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T20:52:47.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:47 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3357248614' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-20T20:52:47.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:47 smithi055 ceph-mon[35213]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-20T20:52:47.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:52:48.136 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:48.497 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3357248614' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-20T20:52:48.497 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:48 smithi026 ceph-mon[31990]: osdmap e78: 12 total, 12 up, 12 in 2024-05-20T20:52:48.497 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:48 smithi026 ceph-mon[31990]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-20T20:52:48.497 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:48 smithi026 ceph-mon[31990]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-20T20:52:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3357248614' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-20T20:52:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:48 smithi144 ceph-mon[35660]: osdmap e78: 12 total, 12 up, 12 in 2024-05-20T20:52:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:48 smithi144 ceph-mon[35660]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-20T20:52:48.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:48 smithi144 ceph-mon[35660]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-20T20:52:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3357248614' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-20T20:52:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:48 smithi055 ceph-mon[35213]: osdmap e78: 12 total, 12 up, 12 in 2024-05-20T20:52:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:48 smithi055 ceph-mon[35213]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-20T20:52:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:48 smithi055 ceph-mon[35213]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-20T20:52:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:49 smithi144 ceph-mon[35660]: pgmap v284: 97 pgs: 3 creating+activating, 17 creating+peering, 59 active+clean, 18 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-20T20:52:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:49 smithi144 ceph-mon[35660]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-20T20:52:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:49 smithi144 ceph-mon[35660]: Cluster is now healthy 2024-05-20T20:52:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:49 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] up:active 2024-05-20T20:52:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:49 smithi144 ceph-mon[35660]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:49 smithi144 ceph-mon[35660]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:49.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:49 smithi026 ceph-mon[31990]: pgmap v284: 97 pgs: 3 creating+activating, 17 creating+peering, 59 active+clean, 18 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-20T20:52:49.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:49 smithi026 ceph-mon[31990]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-20T20:52:49.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:49 smithi026 ceph-mon[31990]: Cluster is now healthy 2024-05-20T20:52:49.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:49 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] up:active 2024-05-20T20:52:49.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:49 smithi026 ceph-mon[31990]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:49.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:49 smithi026 ceph-mon[31990]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:49 smithi055 ceph-mon[35213]: pgmap v284: 97 pgs: 3 creating+activating, 17 creating+peering, 59 active+clean, 18 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-20T20:52:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:49 smithi055 ceph-mon[35213]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-20T20:52:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:49 smithi055 ceph-mon[35213]: Cluster is now healthy 2024-05-20T20:52:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:49 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] up:active 2024-05-20T20:52:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:49 smithi055 ceph-mon[35213]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:49 smithi055 ceph-mon[35213]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 -- 172.21.15.26:0/935474178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d406bab0 msgr2=0x7fc3d406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/935474178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d406bab0 0x7fc3d406bf30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009990 tx=0x7fc3c8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:52:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 -- 172.21.15.26:0/935474178 shutdown_connections 2024-05-20T20:52:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/935474178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d406c470 0x7fc3d41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/935474178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41095a0 0x7fc3d4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/935474178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d406bab0 0x7fc3d406bf30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 -- 172.21.15.26:0/935474178 >> 172.21.15.26:0/935474178 conn(0x7fc3d40793b0 msgr2=0x7fc3d40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 -- 172.21.15.26:0/935474178 shutdown_connections 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.824+0000 7fc3d95f7700 1 -- 172.21.15.26:0/935474178 wait complete. 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 Processor -- start 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 -- start start 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d406bab0 0x7fc3d419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d406c470 0x7fc3d419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 0x7fc3d4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d4113560 con 0x7fc3d41095a0 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d41133e0 con 0x7fc3d406bab0 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d95f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d41136e0 con 0x7fc3d406c470 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 0x7fc3d4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 0x7fc3d4199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47500/0 (socket says 172.21.15.26:47500) 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d37fe700 1 -- 172.21.15.26:0/1969711833 learned_addr learned my addr 172.21.15.26:0/1969711833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d27fc700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d406c470 0x7fc3d419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d2ffd700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d406bab0 0x7fc3d419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.825+0000 7fc3d37fe700 1 -- 172.21.15.26:0/1969711833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d406c470 msgr2=0x7fc3d419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d37fe700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d406c470 0x7fc3d419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d37fe700 1 -- 172.21.15.26:0/1969711833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d406bab0 msgr2=0x7fc3d419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d37fe700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d406bab0 0x7fc3d419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d37fe700 1 -- 172.21.15.26:0/1969711833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d419a1b0 con 0x7fc3d41095a0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d37fe700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 0x7fc3d4199a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007ee0 tx=0x7fc3cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00ddd0 con 0x7fc3d41095a0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3cc004020 con 0x7fc3d41095a0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d419a4a0 con 0x7fc3d41095a0 2024-05-20T20:52:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc015660 con 0x7fc3d41095a0 2024-05-20T20:52:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.826+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d41a66c0 con 0x7fc3d41095a0 2024-05-20T20:52:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.827+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d406a230 con 0x7fc3d41095a0 2024-05-20T20:52:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.829+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc3cc0041d0 con 0x7fc3d41095a0 2024-05-20T20:52:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.829+0000 7fc3bbfff700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3bc0b2880 0x7fc3bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.829+0000 7fc3d2ffd700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3bc0b2880 0x7fc3bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.829+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc3cc0e23d0 con 0x7fc3d41095a0 2024-05-20T20:52:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.830+0000 7fc3d2ffd700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3bc0b2880 0x7fc3bc0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3d419ab90 tx=0x7fc3c400b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:49.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:49.833+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3cc010070 con 0x7fc3d41095a0 2024-05-20T20:52:50.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.049+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3d402e560 con 0x7fc3d41095a0 2024-05-20T20:52:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.051+0000 7fc3bbfff700 1 -- 172.21.15.26:0/1969711833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22172 (secure 0 0 0) 0x7fc3cc0aa1a0 con 0x7fc3d41095a0 2024-05-20T20:52:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:50.051 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:52:47.191576+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:52:50.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3bc0b2880 msgr2=0x7fc3bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3bc0b2880 0x7fc3bc0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3d419ab90 tx=0x7fc3c400b5b0 comp rx=0 tx=0).stop 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 msgr2=0x7fc3d4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 0x7fc3d4199a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007ee0 tx=0x7fc3cc00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 shutdown_connections 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3bc0b2880 0x7fc3bc0b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d41095a0 0x7fc3d4199a50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d406bab0 0x7fc3d419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 --2- 172.21.15.26:0/1969711833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d406c470 0x7fc3d419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.053+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 >> 172.21.15.26:0/1969711833 conn(0x7fc3d40793b0 msgr2=0x7fc3d4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.054+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 shutdown_connections 2024-05-20T20:52:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:50.054+0000 7fc3d95f7700 1 -- 172.21.15.26:0/1969711833 wait complete. 2024-05-20T20:52:50.533 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:50 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1969711833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:52:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:50 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1969711833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:52:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:50 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1969711833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:52:50.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:52:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:51.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:51 smithi026 ceph-mon[31990]: pgmap v285: 97 pgs: 13 creating+peering, 84 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-05-20T20:52:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:51 smithi144 ceph-mon[35660]: pgmap v285: 97 pgs: 13 creating+peering, 84 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-05-20T20:52:51.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:51 smithi055 ceph-mon[35213]: pgmap v285: 97 pgs: 13 creating+peering, 84 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-05-20T20:52:51.701 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:52:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b[65759]: 2024-05-20T20:52:51.254+0000 7efc2b874700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T20:52:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 -- 172.21.15.26:0/3047316001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 msgr2=0x7f25b0109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 --2- 172.21.15.26:0/3047316001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b0109a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25a00099c0 tx=0x7f25a0009cd0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 -- 172.21.15.26:0/3047316001 shutdown_connections 2024-05-20T20:52:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 --2- 172.21.15.26:0/3047316001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b0109a10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 --2- 172.21.15.26:0/3047316001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b006c6d0 0x7f25b0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 --2- 172.21.15.26:0/3047316001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006bd10 0x7f25b006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 -- 172.21.15.26:0/3047316001 >> 172.21.15.26:0/3047316001 conn(0x7f25b00794e0 msgr2=0x7f25b00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 -- 172.21.15.26:0/3047316001 shutdown_connections 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.706+0000 7f25b4df5700 1 -- 172.21.15.26:0/3047316001 wait complete. 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 Processor -- start 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 -- start start 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b006bd10 0x7f25b019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006c6d0 0x7f25b019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b01998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0113640 con 0x7f25b0109630 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01134c0 con 0x7f25b006c6d0 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25b4df5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01137c0 con 0x7f25b006bd10 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25aed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b01998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25add9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006c6d0 0x7f25b019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25aed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b01998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47512/0 (socket says 172.21.15.26:47512) 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.707+0000 7f25aed9d700 1 -- 172.21.15.26:0/502117856 learned_addr learned my addr 172.21.15.26:0/502117856 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25ae59c700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b006bd10 0x7f25b019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25aed9d700 1 -- 172.21.15.26:0/502117856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b006bd10 msgr2=0x7f25b019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25aed9d700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b006bd10 0x7f25b019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25aed9d700 1 -- 172.21.15.26:0/502117856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006c6d0 msgr2=0x7f25b019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25aed9d700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006c6d0 0x7f25b019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25aed9d700 1 -- 172.21.15.26:0/502117856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b019a020 con 0x7f25b0109630 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25add9b700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006c6d0 0x7f25b019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25aed9d700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b01998c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f25a800b970 tx=0x7f25a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a800d860 con 0x7f25b0109630 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b019a310 con 0x7f25b0109630 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25a800e430 con 0x7f25b0109630 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8017660 con 0x7f25b0109630 2024-05-20T20:52:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.708+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b01a65d0 con 0x7f25b0109630 2024-05-20T20:52:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.710+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b010bac0 con 0x7f25b0109630 2024-05-20T20:52:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.711+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f25a800da00 con 0x7f25b0109630 2024-05-20T20:52:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.712+0000 7f259f7fe700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25980b2880 0x7f25980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.712+0000 7f25ae59c700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25980b2880 0x7f25980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.712+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f25a80e1c50 con 0x7f25b0109630 2024-05-20T20:52:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.712+0000 7f25ae59c700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25980b2880 0x7f25980b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009990 tx=0x7f25a001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.716+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25a80e7020 con 0x7f25b0109630 2024-05-20T20:52:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.936+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25b010b4a0 con 0x7f25b0109630 2024-05-20T20:52:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.937+0000 7f259f7fe700 1 -- 172.21.15.26:0/502117856 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22172 (secure 0 0 0) 0x7f25a8013030 con 0x7f25b0109630 2024-05-20T20:52:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:52:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:52:47.191576+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:52:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25980b2880 msgr2=0x7f25980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25980b2880 0x7f25980b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009990 tx=0x7f25a001a040 comp rx=0 tx=0).stop 2024-05-20T20:52:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 msgr2=0x7f25b01998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b01998c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f25a800b970 tx=0x7f25a800bc80 comp rx=0 tx=0).stop 2024-05-20T20:52:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 shutdown_connections 2024-05-20T20:52:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25980b2880 0x7f25980b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b0109630 0x7f25b01998c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b006bd10 0x7f25b019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 --2- 172.21.15.26:0/502117856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25b006c6d0 0x7f25b019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 >> 172.21.15.26:0/502117856 conn(0x7f25b00794e0 msgr2=0x7f25b0100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 shutdown_connections 2024-05-20T20:52:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:52.940+0000 7f25b4df5700 1 -- 172.21.15.26:0/502117856 wait complete. 2024-05-20T20:52:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:53 smithi144 ceph-mon[35660]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-20T20:52:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:53 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/502117856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:52:53.553 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:53 smithi026 ceph-mon[31990]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-20T20:52:53.554 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:53 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/502117856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:52:53.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-20T20:52:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:53 smithi055 ceph-mon[35213]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-20T20:52:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:53 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/502117856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:52:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 -- 172.21.15.26:0/3255021740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e41029e0 msgr2=0x7f30e4102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 --2- 172.21.15.26:0/3255021740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e41029e0 0x7f30e4102dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009990 tx=0x7f30d4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:52:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 -- 172.21.15.26:0/3255021740 shutdown_connections 2024-05-20T20:52:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 --2- 172.21.15.26:0/3255021740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e41029e0 0x7f30e4102dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 --2- 172.21.15.26:0/3255021740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e4107a00 0x7f30e410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 --2- 172.21.15.26:0/3255021740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4107060 0x7f30e41074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:55.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 -- 172.21.15.26:0/3255021740 >> 172.21.15.26:0/3255021740 conn(0x7f30e4079330 msgr2=0x7f30e4079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:55.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 -- 172.21.15.26:0/3255021740 shutdown_connections 2024-05-20T20:52:55.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.504+0000 7f30ea39e700 1 -- 172.21.15.26:0/3255021740 wait complete. 2024-05-20T20:52:55.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 Processor -- start 2024-05-20T20:52:55.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 -- start start 2024-05-20T20:52:55.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e41029e0 0x7f30e4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:55.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4107060 0x7f30e4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:55.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 0x7f30e419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4112490 con 0x7f30e4107a00 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4112310 con 0x7f30e41029e0 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30ea39e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4112610 con 0x7f30e4107060 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30e893b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 0x7f30e419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30e893b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 0x7f30e419d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47528/0 (socket says 172.21.15.26:47528) 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.505+0000 7f30e893b700 1 -- 172.21.15.26:0/1153786719 learned_addr learned my addr 172.21.15.26:0/1153786719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e37fe700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4107060 0x7f30e4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e3fff700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e41029e0 0x7f30e4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e893b700 1 -- 172.21.15.26:0/1153786719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4107060 msgr2=0x7f30e4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e893b700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4107060 0x7f30e4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:55.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e893b700 1 -- 172.21.15.26:0/1153786719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e41029e0 msgr2=0x7f30e4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e893b700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e41029e0 0x7f30e4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e893b700 1 -- 172.21.15.26:0/1153786719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e419dcd0 con 0x7f30e4107a00 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e893b700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 0x7f30e419d790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30dc007ee0 tx=0x7f30dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00dd20 con 0x7f30e4107a00 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30dc004020 con 0x7f30e4107a00 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc015660 con 0x7f30e4107a00 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e419dfc0 con 0x7f30e4107a00 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.506+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e419e500 con 0x7f30e4107a00 2024-05-20T20:52:55.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.507+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e410a9d0 con 0x7f30e4107a00 2024-05-20T20:52:55.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.509+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f30dc0041d0 con 0x7f30e4107a00 2024-05-20T20:52:55.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:55 smithi026 ceph-mon[31990]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-20T20:52:55.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.510+0000 7f30e17fa700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30cc0b2900 0x7f30cc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:55.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.510+0000 7f30e3fff700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30cc0b2900 0x7f30cc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:55.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.510+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f30dc0e2170 con 0x7f30e4107a00 2024-05-20T20:52:55.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.510+0000 7f30e3fff700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30cc0b2900 0x7f30cc0b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b5c0 tx=0x7f30d4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:55.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.514+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30dc010070 con 0x7f30e4107a00 2024-05-20T20:52:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:55 smithi144 ceph-mon[35660]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-20T20:52:55.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:55 smithi055 ceph-mon[35213]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-20T20:52:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:55.777+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f30e4103bb0 con 0x7f30e4107a00 2024-05-20T20:52:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.274+0000 7f30e17fa700 1 -- 172.21.15.26:0/1153786719 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f30dc0a9fe0 con 0x7f30e4107a00 2024-05-20T20:52:56.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.276+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30cc0b2900 msgr2=0x7f30cc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:56.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.276+0000 7f30ea39e700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30cc0b2900 0x7f30cc0b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b5c0 tx=0x7f30d4005960 comp rx=0 tx=0).stop 2024-05-20T20:52:56.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 msgr2=0x7f30e419d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:56.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 0x7f30e419d790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30dc007ee0 tx=0x7f30dc00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 shutdown_connections 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30cc0b2900 0x7f30cc0b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4107a00 0x7f30e419d790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e41029e0 0x7f30e4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 --2- 172.21.15.26:0/1153786719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4107060 0x7f30e4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 >> 172.21.15.26:0/1153786719 conn(0x7f30e4079330 msgr2=0x7f30e41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 shutdown_connections 2024-05-20T20:52:56.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:56.277+0000 7f30ea39e700 1 -- 172.21.15.26:0/1153786719 wait complete. 2024-05-20T20:52:56.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:56 smithi144 ceph-mon[35660]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-20T20:52:56.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:56 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1153786719' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-20T20:52:56.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:56 smithi026 ceph-mon[31990]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-20T20:52:56.613 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:56 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1153786719' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-20T20:52:56.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:56 smithi055 ceph-mon[35213]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-20T20:52:56.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:56 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1153786719' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-20T20:52:56.971 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-20T20:52:56.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-20T20:52:57.230 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:52:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:57 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1153786719' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-20T20:52:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:57 smithi144 ceph-mon[35660]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:57.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:57 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1153786719' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-20T20:52:57.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:57 smithi026 ceph-mon[31990]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:57.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:57 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1153786719' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-20T20:52:57.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:57 smithi055 ceph-mon[35213]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:52:58.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:58 smithi026 ceph-mon[31990]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-20T20:52:58.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:58 smithi144 ceph-mon[35660]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-20T20:52:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:58 smithi055 ceph-mon[35213]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-20T20:52:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 -- 172.21.15.26:0/200822780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4101f00 msgr2=0x7f26e4102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/200822780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4101f00 0x7f26e4102380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009990 tx=0x7f26d8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:52:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 -- 172.21.15.26:0/200822780 shutdown_connections 2024-05-20T20:52:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/200822780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e41028c0 0x7f26e4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/200822780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e4108620 0x7f26e4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/200822780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4101f00 0x7f26e4102380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 -- 172.21.15.26:0/200822780 >> 172.21.15.26:0/200822780 conn(0x7f26e4077780 msgr2=0x7f26e4077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 -- 172.21.15.26:0/200822780 shutdown_connections 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.980+0000 7f26eb3a8700 1 -- 172.21.15.26:0/200822780 wait complete. 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 Processor -- start 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 -- start start 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e4101f00 0x7f26e41030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e41028c0 0x7f26e4103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 0x7f26e4103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41136e0 con 0x7f26e4108620 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4113560 con 0x7f26e41028c0 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26eb3a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4113860 con 0x7f26e4101f00 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26e9945700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 0x7f26e4103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26e9945700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 0x7f26e4103b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47540/0 (socket says 172.21.15.26:47540) 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26e9945700 1 -- 172.21.15.26:0/1353767157 learned_addr learned my addr 172.21.15.26:0/1353767157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26e9144700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e4101f00 0x7f26e41030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.981+0000 7f26e8943700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e41028c0 0x7f26e4103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26e9945700 1 -- 172.21.15.26:0/1353767157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e4101f00 msgr2=0x7f26e41030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26e9945700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e4101f00 0x7f26e41030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26e9945700 1 -- 172.21.15.26:0/1353767157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e41028c0 msgr2=0x7f26e4103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26e9945700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e41028c0 0x7f26e4103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26e9945700 1 -- 172.21.15.26:0/1353767157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e41a81e0 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26e9945700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 0x7f26e4103b60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00b970 tx=0x7f26dc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00d860 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26dc00e430 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e41a84d0 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc017660 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.982+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e41a89e0 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.983+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e406a230 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.984+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f26dc00da00 con 0x7f26e4108620 2024-05-20T20:52:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.985+0000 7f26d67fc700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26d00b2900 0x7f26d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:52:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.985+0000 7f26e9144700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26d00b2900 0x7f26d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:52:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.986+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f26dc0e1de0 con 0x7f26e4108620 2024-05-20T20:52:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.986+0000 7f26e9144700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26d00b2900 0x7f26d00b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26e4078ce0 tx=0x7f26e0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:52:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:58.989+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26dc013030 con 0x7f26e4108620 2024-05-20T20:52:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.258+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f26e402e560 con 0x7f26e4108620 2024-05-20T20:52:59.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.288+0000 7f26d67fc700 1 -- 172.21.15.26:0/1353767157 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f26dc0a9cc0 con 0x7f26e4108620 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26d00b2900 msgr2=0x7f26d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26d00b2900 0x7f26d00b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26e4078ce0 tx=0x7f26e0009190 comp rx=0 tx=0).stop 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 msgr2=0x7f26e4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 0x7f26e4103b60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00b970 tx=0x7f26dc00bc80 comp rx=0 tx=0).stop 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 shutdown_connections 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26d00b2900 0x7f26d00b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26e4108620 0x7f26e4103b60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e4101f00 0x7f26e41030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 --2- 172.21.15.26:0/1353767157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e41028c0 0x7f26e4103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 >> 172.21.15.26:0/1353767157 conn(0x7f26e4077780 msgr2=0x7f26e4100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 shutdown_connections 2024-05-20T20:52:59.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:52:59.291+0000 7f26eb3a8700 1 -- 172.21.15.26:0/1353767157 wait complete. 2024-05-20T20:52:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:52:59 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1353767157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-20T20:52:59.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:52:59 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1353767157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-20T20:52:59.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:52:59 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1353767157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-20T20:52:59.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-20T20:53:00.199 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-20T20:53:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1353767157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-20T20:53:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-20T20:53:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:53:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-20T20:53:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-20T20:53:00.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:00 smithi144 ceph-mon[35660]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1353767157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-20T20:53:00.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:00 smithi026 ceph-mon[31990]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1353767157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-20T20:53:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:00 smithi055 ceph-mon[35213]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-20T20:53:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:active 2024-05-20T20:53:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-20T20:53:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-20T20:53:01.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: Cluster is now healthy 2024-05-20T20:53:01.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-20T20:53:01.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-20T20:53:01.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:01 smithi144 ceph-mon[35660]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-20T20:53:01.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:active 2024-05-20T20:53:01.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-20T20:53:01.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-20T20:53:01.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: Cluster is now healthy 2024-05-20T20:53:01.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-20T20:53:01.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-20T20:53:01.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:01 smithi026 ceph-mon[31990]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-20T20:53:01.701 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b[65759]: 2024-05-20T20:53:01.255+0000 7efc2b874700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-20T20:53:01.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:active 2024-05-20T20:53:01.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-20T20:53:01.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-20T20:53:01.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: Cluster is now healthy 2024-05-20T20:53:01.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-20T20:53:01.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-20T20:53:01.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:01 smithi055 ceph-mon[35213]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-20T20:53:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 -- 172.21.15.26:0/18914940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102e20 msgr2=0x7ff5a410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 --2- 172.21.15.26:0/18914940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102e20 0x7ff5a410f330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b0a0 tx=0x7ff59c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 -- 172.21.15.26:0/18914940 shutdown_connections 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 --2- 172.21.15.26:0/18914940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102e20 0x7ff5a410f330 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 --2- 172.21.15.26:0/18914940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41084f0 0x7ff5a41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 --2- 172.21.15.26:0/18914940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5a4102460 0x7ff5a41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 -- 172.21.15.26:0/18914940 >> 172.21.15.26:0/18914940 conn(0x7ff5a40794e0 msgr2=0x7ff5a40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 -- 172.21.15.26:0/18914940 shutdown_connections 2024-05-20T20:53:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.972+0000 7ff5aa934700 1 -- 172.21.15.26:0/18914940 wait complete. 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 Processor -- start 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 -- start start 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 0x7ff5a419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a4102e20 0x7ff5a419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5a41084f0 0x7ff5a41a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41124b0 con 0x7ff5a4102460 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4112330 con 0x7ff5a4102e20 2024-05-20T20:53:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5aa934700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4112630 con 0x7ff5a41084f0 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.973+0000 7ff5a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 0x7ff5a419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 0x7ff5a419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41444/0 (socket says 172.21.15.26:41444) 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 -- 172.21.15.26:0/1620330555 learned_addr learned my addr 172.21.15.26:0/1620330555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff59bfff700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a4102e20 0x7ff5a419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a8ed1700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5a41084f0 0x7ff5a41a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 -- 172.21.15.26:0/1620330555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5a41084f0 msgr2=0x7ff5a41a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5a41084f0 0x7ff5a41a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 -- 172.21.15.26:0/1620330555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a4102e20 msgr2=0x7ff5a419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a4102e20 0x7ff5a419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 -- 172.21.15.26:0/1620330555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a41a5e70 con 0x7ff5a4102460 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 0x7ff5a419a8b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c9b0 tx=0x7ff59400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594007b00 con 0x7ff5a4102460 2024-05-20T20:53:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59400ce50 con 0x7ff5a4102460 2024-05-20T20:53:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a41a6040 con 0x7ff5a4102460 2024-05-20T20:53:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594018660 con 0x7ff5a4102460 2024-05-20T20:53:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.974+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41a65a0 con 0x7ff5a4102460 2024-05-20T20:53:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.975+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a406a230 con 0x7ff5a4102460 2024-05-20T20:53:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.977+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff59400f450 con 0x7ff5a4102460 2024-05-20T20:53:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.977+0000 7ff5a1ffb700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5900b2900 0x7ff5900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.977+0000 7ff59bfff700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5900b2900 0x7ff5900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.978+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff5940e2140 con 0x7ff5a4102460 2024-05-20T20:53:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.978+0000 7ff59bfff700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5900b2900 0x7ff5900b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4079b50 tx=0x7ff58c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:01.981+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff594014030 con 0x7ff5a4102460 2024-05-20T20:53:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.242+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff5a402e6e0 con 0x7ff5a4102460 2024-05-20T20:53:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.310+0000 7ff5a1ffb700 1 -- 172.21.15.26:0/1620330555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ff5940aa0c0 con 0x7ff5a4102460 2024-05-20T20:53:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5900b2900 msgr2=0x7ff5900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5900b2900 0x7ff5900b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4079b50 tx=0x7ff58c0073d0 comp rx=0 tx=0).stop 2024-05-20T20:53:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 msgr2=0x7ff5a419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 0x7ff5a419a8b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c9b0 tx=0x7ff59400ccc0 comp rx=0 tx=0).stop 2024-05-20T20:53:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 shutdown_connections 2024-05-20T20:53:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5900b2900 0x7ff5900b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a4102460 0x7ff5a419a8b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5a41084f0 0x7ff5a41a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 --2- 172.21.15.26:0/1620330555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a4102e20 0x7ff5a419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 >> 172.21.15.26:0/1620330555 conn(0x7ff5a40794e0 msgr2=0x7ff5a41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 shutdown_connections 2024-05-20T20:53:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:02.314+0000 7ff5aa934700 1 -- 172.21.15.26:0/1620330555 wait complete. 2024-05-20T20:53:02.631 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:02 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] up:active 2024-05-20T20:53:02.632 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:02 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-20T20:53:02.632 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:02 smithi026 ceph-mon[31990]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-20T20:53:02.632 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:02 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1620330555' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-20T20:53:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:02 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] up:active 2024-05-20T20:53:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:02 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-20T20:53:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:02 smithi055 ceph-mon[35213]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-20T20:53:02.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:02 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1620330555' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-20T20:53:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:02 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] up:active 2024-05-20T20:53:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:02 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-20T20:53:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:02 smithi144 ceph-mon[35660]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-20T20:53:02.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:02 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1620330555' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-20T20:53:02.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-20T20:53:03.213 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:03.654 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:03 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1620330555' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-20T20:53:03.654 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:03 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-20T20:53:03.654 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:03 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:53:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:03 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1620330555' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-20T20:53:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:03 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-20T20:53:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:03 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:53:03.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:03 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1620330555' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-20T20:53:03.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:03 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-20T20:53:03.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:03 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:53:04.616 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:04 smithi026 ceph-mon[31990]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-20T20:53:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:04 smithi055 ceph-mon[35213]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-20T20:53:04.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:04 smithi144 ceph-mon[35660]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-20T20:53:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 -- 172.21.15.26:0/3334359975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040102e00 msgr2=0x7f104010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 --2- 172.21.15.26:0/3334359975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040102e00 0x7f104010f2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1038009990 tx=0x7f1038009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 -- 172.21.15.26:0/3334359975 shutdown_connections 2024-05-20T20:53:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 --2- 172.21.15.26:0/3334359975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040102e00 0x7f104010f2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 --2- 172.21.15.26:0/3334359975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1040108520 0x7f1040108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 --2- 172.21.15.26:0/3334359975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040102440 0x7f10401028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 -- 172.21.15.26:0/3334359975 >> 172.21.15.26:0/3334359975 conn(0x7f10400793f0 msgr2=0x7f1040079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 -- 172.21.15.26:0/3334359975 shutdown_connections 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.865+0000 7f1045dc0700 1 -- 172.21.15.26:0/3334359975 wait complete. 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 Processor -- start 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 -- start start 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040102440 0x7f104019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1040102e00 0x7f104019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 0x7f1040199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10401124d0 con 0x7f1040108520 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1040112350 con 0x7f1040102440 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f1045dc0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1040112650 con 0x7f1040102e00 2024-05-20T20:53:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f103ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 0x7f1040199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f103ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 0x7f1040199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41454/0 (socket says 172.21.15.26:41454) 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.866+0000 7f103ffff700 1 -- 172.21.15.26:0/4203231035 learned_addr learned my addr 172.21.15.26:0/4203231035 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103f7fe700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040102440 0x7f104019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103effd700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1040102e00 0x7f104019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103ffff700 1 -- 172.21.15.26:0/4203231035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1040102e00 msgr2=0x7f104019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103ffff700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1040102e00 0x7f104019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103ffff700 1 -- 172.21.15.26:0/4203231035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040102440 msgr2=0x7f104019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103ffff700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040102440 0x7f104019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103ffff700 1 -- 172.21.15.26:0/4203231035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1040199ff0 con 0x7f1040108520 2024-05-20T20:53:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103ffff700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 0x7f1040199890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f103800c010 tx=0x7f10380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038004830 con 0x7f1040108520 2024-05-20T20:53:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1038005020 con 0x7f1040108520 2024-05-20T20:53:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104019a280 con 0x7f1040108520 2024-05-20T20:53:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.867+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103800fa70 con 0x7f1040108520 2024-05-20T20:53:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.868+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10401a65a0 con 0x7f1040108520 2024-05-20T20:53:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.869+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1040103b60 con 0x7f1040108520 2024-05-20T20:53:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.870+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f10380054f0 con 0x7f1040108520 2024-05-20T20:53:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.871+0000 7f103cff9700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10280b2880 0x7f10280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.871+0000 7f103f7fe700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10280b2880 0x7f10280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.871+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f10380e3470 con 0x7f1040108520 2024-05-20T20:53:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.871+0000 7f103f7fe700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10280b2880 0x7f10280b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f104019a9d0 tx=0x7f10300079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:04.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:04.874+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1038013070 con 0x7f1040108520 2024-05-20T20:53:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.138+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f104002e560 con 0x7f1040108520 2024-05-20T20:53:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.328+0000 7f103cff9700 1 -- 172.21.15.26:0/4203231035 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f10380b1070 con 0x7f1040108520 2024-05-20T20:53:05.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10280b2880 msgr2=0x7f10280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10280b2880 0x7f10280b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f104019a9d0 tx=0x7f10300079e0 comp rx=0 tx=0).stop 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 msgr2=0x7f1040199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 0x7f1040199890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f103800c010 tx=0x7f10380047b0 comp rx=0 tx=0).stop 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 shutdown_connections 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10280b2880 0x7f10280b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1040108520 0x7f1040199890 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040102440 0x7f104019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 --2- 172.21.15.26:0/4203231035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1040102e00 0x7f104019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.330+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 >> 172.21.15.26:0/4203231035 conn(0x7f10400793f0 msgr2=0x7f104006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:05.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.331+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 shutdown_connections 2024-05-20T20:53:05.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:05.331+0000 7f1045dc0700 1 -- 172.21.15.26:0/4203231035 wait complete. 2024-05-20T20:53:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:05 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4203231035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-20T20:53:05.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:05 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4203231035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-20T20:53:05.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:05 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4203231035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-20T20:53:06.108 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-20T20:53:06.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-20T20:53:06.363 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:06.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:06 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4203231035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-20T20:53:06.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:06 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:53:06.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:06 smithi026 ceph-mon[31990]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-20T20:53:06.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:06 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4203231035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-20T20:53:06.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:06 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:53:06.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:06 smithi055 ceph-mon[35213]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-20T20:53:06.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:06 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4203231035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-20T20:53:06.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:06 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:53:06.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:06 smithi144 ceph-mon[35660]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.000+0000 7fa1ce670700 1 -- 172.21.15.26:0/526932078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c81096a0 msgr2=0x7fa1c8109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.000+0000 7fa1ce670700 1 --2- 172.21.15.26:0/526932078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c81096a0 0x7fa1c8109a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009990 tx=0x7fa1b8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.000+0000 7fa1ce670700 1 -- 172.21.15.26:0/526932078 shutdown_connections 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.000+0000 7fa1ce670700 1 --2- 172.21.15.26:0/526932078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c81096a0 0x7fa1c8109a80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.000+0000 7fa1ce670700 1 --2- 172.21.15.26:0/526932078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103f50 0x7fa1c8110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.000+0000 7fa1ce670700 1 --2- 172.21.15.26:0/526932078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c8103590 0x7fa1c8103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- 172.21.15.26:0/526932078 >> 172.21.15.26:0/526932078 conn(0x7fa1c807a6f0 msgr2=0x7fa1c807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- 172.21.15.26:0/526932078 shutdown_connections 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- 172.21.15.26:0/526932078 wait complete. 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 Processor -- start 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- start start 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103590 0x7fa1c819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 0x7fa1c819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c81096a0 0x7fa1c8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8113620 con 0x7fa1c8103f50 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c81134a0 con 0x7fa1c8103590 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.001+0000 7fa1ce670700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c81137a0 con 0x7fa1c81096a0 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 0x7fa1c819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 0x7fa1c819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41480/0 (socket says 172.21.15.26:41480) 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 -- 172.21.15.26:0/2278217594 learned_addr learned my addr 172.21.15.26:0/2278217594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1c7fff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103590 0x7fa1c819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1ccc0d700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c81096a0 0x7fa1c8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 -- 172.21.15.26:0/2278217594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c81096a0 msgr2=0x7fa1c8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c81096a0 0x7fa1c8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 -- 172.21.15.26:0/2278217594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103590 msgr2=0x7fa1c819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103590 0x7fa1c819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 -- 172.21.15.26:0/2278217594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c819a0c0 con 0x7fa1c8103f50 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1c7fff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103590 0x7fa1c819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1ccc0d700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c81096a0 0x7fa1c8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1bffff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 0x7fa1c819fc90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c9b0 tx=0x7fa1b000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0007b00 con 0x7fa1c8103f50 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1b0007ca0 con 0x7fa1c8103f50 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c819a3b0 con 0x7fa1c8103f50 2024-05-20T20:53:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0018660 con 0x7fa1c8103f50 2024-05-20T20:53:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.002+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c81a6620 con 0x7fa1c8103f50 2024-05-20T20:53:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.003+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c806a230 con 0x7fa1c8103f50 2024-05-20T20:53:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.005+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa1b000f450 con 0x7fa1c8103f50 2024-05-20T20:53:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.005+0000 7fa1c5ffb700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1b40b2880 0x7fa1b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.005+0000 7fa1c7fff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1b40b2880 0x7fa1b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.006+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fa1b00e2450 con 0x7fa1c8103f50 2024-05-20T20:53:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.006+0000 7fa1c7fff700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1b40b2880 0x7fa1b40b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8000c00 tx=0x7fa1b8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.009+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1b0014030 con 0x7fa1c8103f50 2024-05-20T20:53:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.257+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa1c8063980 con 0x7fa1b40b2880 2024-05-20T20:53:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.282+0000 7fa1c5ffb700 1 -- 172.21.15.26:0/2278217594 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa1c8063980 con 0x7fa1b40b2880 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1b40b2880 msgr2=0x7fa1b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1b40b2880 0x7fa1b40b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8000c00 tx=0x7fa1b8005fd0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 msgr2=0x7fa1c819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 0x7fa1c819fc90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c9b0 tx=0x7fa1b000cd70 comp rx=0 tx=0).stop 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 shutdown_connections 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1c81096a0 0x7fa1c8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1c8103590 0x7fa1c819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1b40b2880 0x7fa1b40b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.284+0000 7fa1ce670700 1 --2- 172.21.15.26:0/2278217594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c8103f50 0x7fa1c819fc90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.285+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 >> 172.21.15.26:0/2278217594 conn(0x7fa1c807a6f0 msgr2=0x7fa1c81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.285+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 shutdown_connections 2024-05-20T20:53:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:08.285+0000 7fa1ce670700 1 -- 172.21.15.26:0/2278217594 wait complete. 2024-05-20T20:53:08.611 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:53:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:53:08.262+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:53:08.611 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:53:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:53:08.262+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:53:08.612 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:53:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:53:08.262+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:53:08.612 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:53:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:53:08.262+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:53:08.612 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:53:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:53:08.262+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:53:08.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:08 smithi026 ceph-mon[31990]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-20T20:53:08.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:08 smithi026 ceph-mon[31990]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:08.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-20T20:53:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:08 smithi144 ceph-mon[35660]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-20T20:53:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:08 smithi144 ceph-mon[35660]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:09.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:08 smithi055 ceph-mon[35213]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-20T20:53:09.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:08 smithi055 ceph-mon[35213]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:09.234 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:10.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:10 smithi026 ceph-mon[31990]: pgmap v295: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-20T20:53:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:10 smithi144 ceph-mon[35660]: pgmap v295: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-20T20:53:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 -- 172.21.15.26:0/3316532144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c1084b0 msgr2=0x7f441c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 --2- 172.21.15.26:0/3316532144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c1084b0 0x7f441c108890 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4418009990 tx=0x7f4418009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 -- 172.21.15.26:0/3316532144 shutdown_connections 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 --2- 172.21.15.26:0/3316532144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c1084b0 0x7f441c108890 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 --2- 172.21.15.26:0/3316532144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441c102e30 0x7f441c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 --2- 172.21.15.26:0/3316532144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f441c102470 0x7f441c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 -- 172.21.15.26:0/3316532144 >> 172.21.15.26:0/3316532144 conn(0x7f441c0793f0 msgr2=0x7f441c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 -- 172.21.15.26:0/3316532144 shutdown_connections 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.191+0000 7f44244f5700 1 -- 172.21.15.26:0/3316532144 wait complete. 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 Processor -- start 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 -- start start 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 0x7f441c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441c102e30 0x7f441c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f441c1084b0 0x7f441c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1124c0 con 0x7f441c102470 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f441c112340 con 0x7f441c1084b0 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f44244f5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f441c112640 con 0x7f441c102e30 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 0x7f441c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 0x7f441c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56436/0 (socket says 172.21.15.26:56436) 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 -- 172.21.15.26:0/2512642780 learned_addr learned my addr 172.21.15.26:0/2512642780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4421a90700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441c102e30 0x7f441c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422a92700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f441c1084b0 0x7f441c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 -- 172.21.15.26:0/2512642780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441c102e30 msgr2=0x7f441c1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441c102e30 0x7f441c1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 -- 172.21.15.26:0/2512642780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f441c1084b0 msgr2=0x7f441c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.192+0000 7f4422291700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f441c1084b0 0x7f441c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f4422291700 1 -- 172.21.15.26:0/2512642780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c19c400 con 0x7f441c102470 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f4422291700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 0x7f441c1a1a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4418000c00 tx=0x7f44180057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418027070 con 0x7f441c102470 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c19c690 con 0x7f441c102470 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f441800fb00 con 0x7f441c102470 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418022b70 con 0x7f441c102470 2024-05-20T20:53:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.193+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a8800 con 0x7f441c102470 2024-05-20T20:53:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.194+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441c103b60 con 0x7f441c102470 2024-05-20T20:53:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.195+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4418022d10 con 0x7f441c102470 2024-05-20T20:53:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.196+0000 7f44137fe700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f44080b2910 0x7f44080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.196+0000 7f4421a90700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f44080b2910 0x7f44080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.196+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f44180e33a0 con 0x7f441c102470 2024-05-20T20:53:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.197+0000 7f4421a90700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f44080b2910 0x7f44080b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f441c19d270 tx=0x7f440c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:10 smithi055 ceph-mon[35213]: pgmap v295: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-20T20:53:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.202+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4418031080 con 0x7f441c102470 2024-05-20T20:53:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.447+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f441c100d70 con 0x7f44080b2910 2024-05-20T20:53:11.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.473+0000 7f44137fe700 1 -- 172.21.15.26:0/2512642780 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f441c100d70 con 0x7f44080b2910 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f44080b2910 msgr2=0x7f44080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f44080b2910 0x7f44080b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f441c19d270 tx=0x7f440c00b410 comp rx=0 tx=0).stop 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 msgr2=0x7f441c1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 0x7f441c1a1a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4418000c00 tx=0x7f44180057a0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 shutdown_connections 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f44080b2910 0x7f44080b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f441c102470 0x7f441c1a1a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f441c1084b0 0x7f441c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 --2- 172.21.15.26:0/2512642780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441c102e30 0x7f441c1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 >> 172.21.15.26:0/2512642780 conn(0x7f441c0793f0 msgr2=0x7f441c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:11.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 shutdown_connections 2024-05-20T20:53:11.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:11.476+0000 7f44244f5700 1 -- 172.21.15.26:0/2512642780 wait complete. 2024-05-20T20:53:12.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:11 smithi144 ceph-mon[35660]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-20T20:53:12.141 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:11 smithi026 ceph-mon[31990]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-20T20:53:12.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-20T20:53:12.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:11 smithi055 ceph-mon[35213]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-20T20:53:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:12.860 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:12 smithi026 ceph-mon[31990]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:12.860 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:12 smithi026 ceph-mon[31990]: pgmap v296: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-20T20:53:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:12 smithi144 ceph-mon[35660]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:12 smithi144 ceph-mon[35660]: pgmap v296: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-20T20:53:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:12 smithi055 ceph-mon[35213]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:12 smithi055 ceph-mon[35213]: pgmap v296: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-20T20:53:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 -- 172.21.15.26:0/2231537683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 msgr2=0x7f198010fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 --2- 172.21.15.26:0/2231537683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f198010fae0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1974009990 tx=0x7f1974009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 -- 172.21.15.26:0/2231537683 shutdown_connections 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 --2- 172.21.15.26:0/2231537683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1980110020 0x7f1980112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 --2- 172.21.15.26:0/2231537683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bc70 0x7f198006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 --2- 172.21.15.26:0/2231537683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f198010fae0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 -- 172.21.15.26:0/2231537683 >> 172.21.15.26:0/2231537683 conn(0x7f19800793f0 msgr2=0x7f1980079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 -- 172.21.15.26:0/2231537683 shutdown_connections 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 -- 172.21.15.26:0/2231537683 wait complete. 2024-05-20T20:53:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 Processor -- start 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.166+0000 7f1987c8a700 1 -- start start 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006bc70 0x7f19801a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f19801a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1980110020 0x7f198019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19801157b0 con 0x7f198006c590 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1980115630 con 0x7f198006bc70 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1980115930 con 0x7f1980110020 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f19801a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f19801a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56456/0 (socket says 172.21.15.26:56456) 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 -- 172.21.15.26:0/1487397443 learned_addr learned my addr 172.21.15.26:0/1487397443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 -- 172.21.15.26:0/1487397443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1980110020 msgr2=0x7f198019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1980110020 0x7f198019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 -- 172.21.15.26:0/1487397443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006bc70 msgr2=0x7f19801a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006bc70 0x7f19801a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 -- 172.21.15.26:0/1487397443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198019c410 con 0x7f198006c590 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1985225700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f19801a1fa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19740057c0 tx=0x7f1974009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974027070 con 0x7f198006c590 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f197400fda0 con 0x7f198006c590 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198019c6a0 con 0x7f198006c590 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.167+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974022ca0 con 0x7f198006c590 2024-05-20T20:53:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.168+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a89e0 con 0x7f198006c590 2024-05-20T20:53:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.168+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1980102a50 con 0x7f198006c590 2024-05-20T20:53:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.170+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f197402a070 con 0x7f198006c590 2024-05-20T20:53:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.170+0000 7f197effd700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19680b2990 0x7f19680b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.171+0000 7f1985a26700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19680b2990 0x7f19680b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:14.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.171+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f19740e31d0 con 0x7f198006c590 2024-05-20T20:53:14.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.171+0000 7f1985a26700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19680b2990 0x7f19680b4e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f197000d0e0 tx=0x7f19700085a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:14.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.174+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1974031080 con 0x7f198006c590 2024-05-20T20:53:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.422+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f198019cff0 con 0x7f19680b2990 2024-05-20T20:53:14.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.428+0000 7f197effd700 1 -- 172.21.15.26:0/1487397443 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f198019cff0 con 0x7f19680b2990 2024-05-20T20:53:14.427 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95 2024-05-20T20:53:14.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19680b2990 msgr2=0x7f19680b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:14.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19680b2990 0x7f19680b4e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f197000d0e0 tx=0x7f19700085a0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 msgr2=0x7f19801a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:14.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f19801a1fa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19740057c0 tx=0x7f1974009f90 comp rx=0 tx=0).stop 2024-05-20T20:53:14.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 shutdown_connections 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1980110020 0x7f198019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006bc70 0x7f19801a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f19680b2990 0x7f19680b4e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 --2- 172.21.15.26:0/1487397443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198006c590 0x7f19801a1fa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 >> 172.21.15.26:0/1487397443 conn(0x7f19800793f0 msgr2=0x7f1980111530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 shutdown_connections 2024-05-20T20:53:14.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:14.430+0000 7f1987c8a700 1 -- 172.21.15.26:0/1487397443 wait complete. 2024-05-20T20:53:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:14 smithi144 ceph-mon[35660]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-20T20:53:15.091 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:14 smithi026 ceph-mon[31990]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-20T20:53:15.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-20T20:53:15.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:14 smithi055 ceph-mon[35213]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-20T20:53:15.376 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:16.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:15 smithi144 ceph-mon[35660]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:16.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:15 smithi055 ceph-mon[35213]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:16.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:15 smithi026 ceph-mon[31990]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:16 smithi144 ceph-mon[35660]: pgmap v298: 97 pgs: 97 active+clean; 602 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-20T20:53:17.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:16 smithi055 ceph-mon[35213]: pgmap v298: 97 pgs: 97 active+clean; 602 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-20T20:53:17.113 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:16 smithi026 ceph-mon[31990]: pgmap v298: 97 pgs: 97 active+clean; 602 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-20T20:53:17.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 -- 172.21.15.26:0/3074911215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681037f0 msgr2=0x7faf68103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:17.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 --2- 172.21.15.26:0/3074911215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681037f0 0x7faf68103bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b0d0 tx=0x7faf6400b3e0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 -- 172.21.15.26:0/3074911215 shutdown_connections 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 --2- 172.21.15.26:0/3074911215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681037f0 0x7faf68103bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 --2- 172.21.15.26:0/3074911215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf681088f0 0x7faf68110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 --2- 172.21.15.26:0/3074911215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf681041a0 0x7faf681081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 -- 172.21.15.26:0/3074911215 >> 172.21.15.26:0/3074911215 conn(0x7faf6807a6a0 msgr2=0x7faf6807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 -- 172.21.15.26:0/3074911215 shutdown_connections 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.126+0000 7faf7040c700 1 -- 172.21.15.26:0/3074911215 wait complete. 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 Processor -- start 2024-05-20T20:53:17.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 -- start start 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf681037f0 0x7faf6819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf681041a0 0x7faf6819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 0x7faf68199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf68113660 con 0x7faf681088f0 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faf681134e0 con 0x7faf681037f0 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.127+0000 7faf7040c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf681137e0 con 0x7faf681041a0 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 0x7faf68199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:17.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 0x7faf68199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56460/0 (socket says 172.21.15.26:56460) 2024-05-20T20:53:17.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 -- 172.21.15.26:0/2517232594 learned_addr learned my addr 172.21.15.26:0/2517232594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:17.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e1a8700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf681037f0 0x7faf6819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:17.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6d9a7700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf681041a0 0x7faf6819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:17.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 -- 172.21.15.26:0/2517232594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf681041a0 msgr2=0x7faf6819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:17.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf681041a0 0x7faf6819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 -- 172.21.15.26:0/2517232594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf681037f0 msgr2=0x7faf6819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:17.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf681037f0 0x7faf6819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 -- 172.21.15.26:0/2517232594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6819a1d0 con 0x7faf681088f0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf6e9a9700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 0x7faf68199a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf6000b550 tx=0x7faf6000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60015020 con 0x7faf681088f0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6819a4c0 con 0x7faf681088f0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf60011cc0 con 0x7faf681088f0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60010b80 con 0x7faf681088f0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.128+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf681a6730 con 0x7faf681088f0 2024-05-20T20:53:17.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.130+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6810bb30 con 0x7faf681088f0 2024-05-20T20:53:17.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.131+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7faf60018020 con 0x7faf681088f0 2024-05-20T20:53:17.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.132+0000 7faf5f7fe700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf540b2990 0x7faf540b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:17.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.132+0000 7faf6e1a8700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf540b2990 0x7faf540b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:17.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.132+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7faf600e1d80 con 0x7faf681088f0 2024-05-20T20:53:17.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.132+0000 7faf6e1a8700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf540b2990 0x7faf540b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf64009fd0 tx=0x7faf6400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:17.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.135+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf6001f020 con 0x7faf681088f0 2024-05-20T20:53:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.392+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faf68063980 con 0x7faf540b2990 2024-05-20T20:53:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.418+0000 7faf5f7fe700 1 -- 172.21.15.26:0/2517232594 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf68063980 con 0x7faf540b2990 2024-05-20T20:53:17.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf540b2990 msgr2=0x7faf540b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf540b2990 0x7faf540b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf64009fd0 tx=0x7faf6400b040 comp rx=0 tx=0).stop 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 msgr2=0x7faf68199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 0x7faf68199a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf6000b550 tx=0x7faf6000b860 comp rx=0 tx=0).stop 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 shutdown_connections 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf540b2990 0x7faf540b4e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf681088f0 0x7faf68199a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf681037f0 0x7faf6819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 --2- 172.21.15.26:0/2517232594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf681041a0 0x7faf6819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:17.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 >> 172.21.15.26:0/2517232594 conn(0x7faf6807a6a0 msgr2=0x7faf68101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:17.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 shutdown_connections 2024-05-20T20:53:17.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:17.420+0000 7faf7040c700 1 -- 172.21.15.26:0/2517232594 wait complete. 2024-05-20T20:53:18.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-20T20:53:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:18.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:18 smithi026 ceph-mon[31990]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:18.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:18 smithi026 ceph-mon[31990]: pgmap v299: 97 pgs: 97 active+clean; 602 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-20T20:53:19.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:18 smithi144 ceph-mon[35660]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:19.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:18 smithi144 ceph-mon[35660]: pgmap v299: 97 pgs: 97 active+clean; 602 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-20T20:53:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:18 smithi055 ceph-mon[35213]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:18 smithi055 ceph-mon[35213]: pgmap v299: 97 pgs: 97 active+clean; 602 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 -- 172.21.15.26:0/2114635849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 msgr2=0x7ff09006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 --2- 172.21.15.26:0/2114635849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09006c090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff078009990 tx=0x7ff078009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 -- 172.21.15.26:0/2114635849 shutdown_connections 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 --2- 172.21.15.26:0/2114635849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff09006c5d0 0x7ff0901103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 --2- 172.21.15.26:0/2114635849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 0x7ff0901099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 --2- 172.21.15.26:0/2114635849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09006c090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 -- 172.21.15.26:0/2114635849 >> 172.21.15.26:0/2114635849 conn(0x7ff0900793f0 msgr2=0x7ff090079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 -- 172.21.15.26:0/2114635849 shutdown_connections 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.129+0000 7ff094882700 1 -- 172.21.15.26:0/2114635849 wait complete. 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 Processor -- start 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 -- start start 2024-05-20T20:53:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff09006c5d0 0x7ff09019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 0x7ff090199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff090113600 con 0x7ff09006bc10 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff090113480 con 0x7ff09006c5d0 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff094882700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff090113780 con 0x7ff0901095c0 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57708/0 (socket says 172.21.15.26:57708) 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 -- 172.21.15.26:0/3071627728 learned_addr learned my addr 172.21.15.26:0/3071627728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff087fff700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff09006c5d0 0x7ff09019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 -- 172.21.15.26:0/3071627728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 msgr2=0x7ff090199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08ed9d700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 0x7ff090199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 0x7ff090199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 -- 172.21.15.26:0/3071627728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff09006c5d0 msgr2=0x7ff09019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff09006c5d0 0x7ff09019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08e59c700 1 -- 172.21.15.26:0/3071627728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09019a190 con 0x7ff09006bc10 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.130+0000 7ff08ed9d700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 0x7ff090199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.131+0000 7ff08e59c700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09019f870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b530 tx=0x7ff08000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.131+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080015020 con 0x7ff09006bc10 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.131+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff080011cc0 con 0x7ff09006bc10 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.131+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09019a480 con 0x7ff09006bc10 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.131+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080010d30 con 0x7ff09006bc10 2024-05-20T20:53:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.131+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a6740 con 0x7ff09006bc10 2024-05-20T20:53:20.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.132+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09010baa0 con 0x7ff09006bc10 2024-05-20T20:53:20.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.134+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff080018020 con 0x7ff09006bc10 2024-05-20T20:53:20.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.135+0000 7ff0877fe700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff07c0b2910 0x7ff07c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:20.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.135+0000 7ff087fff700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff07c0b2910 0x7ff07c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:20.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.135+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff0800e2040 con 0x7ff09006bc10 2024-05-20T20:53:20.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.136+0000 7ff087fff700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff07c0b2910 0x7ff07c0b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff078005850 tx=0x7ff07801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:20.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.139+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff08001f020 con 0x7ff09006bc10 2024-05-20T20:53:20.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.390+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff090063980 con 0x7ff07c0b2910 2024-05-20T20:53:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.395+0000 7ff0877fe700 1 -- 172.21.15.26:0/3071627728 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff090063980 con 0x7ff07c0b2910 2024-05-20T20:53:20.394 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_1/bee3620f-f020-4db2-beda-550f403ba15c 2024-05-20T20:53:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff07c0b2910 msgr2=0x7ff07c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff07c0b2910 0x7ff07c0b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff078005850 tx=0x7ff07801a040 comp rx=0 tx=0).stop 2024-05-20T20:53:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 msgr2=0x7ff09019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09019f870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b530 tx=0x7ff08000b8f0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 shutdown_connections 2024-05-20T20:53:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff07c0b2910 0x7ff07c0b4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff09006bc10 0x7ff09019f870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0901095c0 0x7ff090199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 --2- 172.21.15.26:0/3071627728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff09006c5d0 0x7ff09019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 >> 172.21.15.26:0/3071627728 conn(0x7ff0900793f0 msgr2=0x7ff090101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 shutdown_connections 2024-05-20T20:53:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:20.398+0000 7ff094882700 1 -- 172.21.15.26:0/3071627728 wait complete. 2024-05-20T20:53:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:20 smithi026 ceph-mon[31990]: pgmap v300: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-20T20:53:21.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:53:21.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:20 smithi144 ceph-mon[35660]: pgmap v300: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-20T20:53:21.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:20 smithi055 ceph-mon[35213]: pgmap v300: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-20T20:53:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:21.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:21 smithi026 ceph-mon[31990]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:22.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:21 smithi144 ceph-mon[35660]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:22.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:21 smithi055 ceph-mon[35213]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:53:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 -- 172.21.15.26:0/2756506904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6124109630 msgr2=0x7f6124109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 --2- 172.21.15.26:0/2756506904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6124109630 0x7f6124109a10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61140099c0 tx=0x7f6114009cd0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 -- 172.21.15.26:0/2756506904 shutdown_connections 2024-05-20T20:53:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 --2- 172.21.15.26:0/2756506904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6124109630 0x7f6124109a10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 --2- 172.21.15.26:0/2756506904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f612406c6d0 0x7f6124110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 --2- 172.21.15.26:0/2756506904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f612406bd10 0x7f612406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.025+0000 7f612978d700 1 -- 172.21.15.26:0/2756506904 >> 172.21.15.26:0/2756506904 conn(0x7f61240794e0 msgr2=0x7f61240798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.026+0000 7f612978d700 1 -- 172.21.15.26:0/2756506904 shutdown_connections 2024-05-20T20:53:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.026+0000 7f612978d700 1 -- 172.21.15.26:0/2756506904 wait complete. 2024-05-20T20:53:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.026+0000 7f612978d700 1 Processor -- start 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 -- start start 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f612406bd10 0x7f612419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 0x7f612419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6124109630 0x7f61241997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61241135f0 con 0x7f612406c6d0 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113470 con 0x7f612406bd10 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f612978d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113770 con 0x7f6124109630 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 0x7f612419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 0x7f612419fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57712/0 (socket says 172.21.15.26:57712) 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 -- 172.21.15.26:0/2186342337 learned_addr learned my addr 172.21.15.26:0/2186342337 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f6122ffd700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f612406bd10 0x7f612419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61237fe700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6124109630 0x7f61241997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 -- 172.21.15.26:0/2186342337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6124109630 msgr2=0x7f61241997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6124109630 0x7f61241997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 -- 172.21.15.26:0/2186342337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f612406bd10 msgr2=0x7f612419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f612406bd10 0x7f612419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.027+0000 7f61227fc700 1 -- 172.21.15.26:0/2186342337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6124199fb0 con 0x7f612406c6d0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.028+0000 7f61227fc700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 0x7f612419fb90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f611800c9b0 tx=0x7f611800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.028+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118007b00 con 0x7f612406c6d0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.028+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419a2a0 con 0x7f612406c6d0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.028+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6118007ca0 con 0x7f612406c6d0 2024-05-20T20:53:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.028+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118018660 con 0x7f612406c6d0 2024-05-20T20:53:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.028+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a64b0 con 0x7f612406c6d0 2024-05-20T20:53:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.029+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6124104ca0 con 0x7f612406c6d0 2024-05-20T20:53:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.030+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f611800f450 con 0x7f612406c6d0 2024-05-20T20:53:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.031+0000 7f610bfff700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f610c0b2910 0x7f610c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.031+0000 7f6122ffd700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f610c0b2910 0x7f610c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.031+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f61180e2580 con 0x7f612406c6d0 2024-05-20T20:53:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.031+0000 7f6122ffd700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f610c0b2910 0x7f610c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6114000c00 tx=0x7f611401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.034+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6118014030 con 0x7f612406c6d0 2024-05-20T20:53:23.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:22 smithi144 ceph-mon[35660]: pgmap v301: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-20T20:53:23.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:22 smithi055 ceph-mon[35213]: pgmap v301: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-20T20:53:23.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:22 smithi026 ceph-mon[31990]: pgmap v301: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-20T20:53:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.302+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f612419ad00 con 0x7f612406c6d0 2024-05-20T20:53:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.303+0000 7f610bfff700 1 -- 172.21.15.26:0/2186342337 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f61180aa380 con 0x7f612406c6d0 2024-05-20T20:53:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:23.303 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:53:05.138905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T20:53:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f610c0b2910 msgr2=0x7f610c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f610c0b2910 0x7f610c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6114000c00 tx=0x7f611401a040 comp rx=0 tx=0).stop 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 msgr2=0x7f612419fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 0x7f612419fb90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f611800c9b0 tx=0x7f611800cd70 comp rx=0 tx=0).stop 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 shutdown_connections 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6124109630 0x7f61241997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f612406bd10 0x7f612419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f610c0b2910 0x7f610c0b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 --2- 172.21.15.26:0/2186342337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612406c6d0 0x7f612419fb90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 >> 172.21.15.26:0/2186342337 conn(0x7f61240794e0 msgr2=0x7f6124100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 shutdown_connections 2024-05-20T20:53:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:23.305+0000 7f612978d700 1 -- 172.21.15.26:0/2186342337 wait complete. 2024-05-20T20:53:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:53:23.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:23 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2186342337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:53:23.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:53:24.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:23 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2186342337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:53:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:24.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:23 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2186342337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:53:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:24 smithi026 ceph-mon[31990]: pgmap v302: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-20T20:53:25.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:24 smithi055 ceph-mon[35213]: pgmap v302: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-20T20:53:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:24 smithi144 ceph-mon[35660]: pgmap v302: 97 pgs: 97 active+clean; 622 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-20T20:53:25.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 -- 172.21.15.26:0/4021583161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa688076090 msgr2=0x7fa68810fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:25.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 --2- 172.21.15.26:0/4021583161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa688076090 0x7fa68810fa30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa678009990 tx=0x7fa678009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:25.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 -- 172.21.15.26:0/4021583161 shutdown_connections 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 --2- 172.21.15.26:0/4021583161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa68810ff70 0x7fa688112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 --2- 172.21.15.26:0/4021583161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa688075770 0x7fa688075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 --2- 172.21.15.26:0/4021583161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa688076090 0x7fa68810fa30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 -- 172.21.15.26:0/4021583161 >> 172.21.15.26:0/4021583161 conn(0x7fa68807a6f0 msgr2=0x7fa68807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 -- 172.21.15.26:0/4021583161 shutdown_connections 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.915+0000 7fa68ffad700 1 -- 172.21.15.26:0/4021583161 wait complete. 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 Processor -- start 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 -- start start 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa688075770 0x7fa68819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688076090 0x7fa68819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 0x7fa6881a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881158a0 con 0x7fa68810ff70 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa688115720 con 0x7fa688075770 2024-05-20T20:53:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68ffad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa688115a20 con 0x7fa688076090 2024-05-20T20:53:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68e54a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 0x7fa6881a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68e54a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 0x7fa6881a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57738/0 (socket says 172.21.15.26:57738) 2024-05-20T20:53:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.916+0000 7fa68e54a700 1 -- 172.21.15.26:0/1696225708 learned_addr learned my addr 172.21.15.26:0/1696225708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68d548700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688076090 0x7fa68819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68dd49700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa688075770 0x7fa68819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68e54a700 1 -- 172.21.15.26:0/1696225708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688076090 msgr2=0x7fa68819c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68e54a700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688076090 0x7fa68819c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68e54a700 1 -- 172.21.15.26:0/1696225708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa688075770 msgr2=0x7fa68819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68e54a700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa688075770 0x7fa68819be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68e54a700 1 -- 172.21.15.26:0/1696225708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6881a0e60 con 0x7fa68810ff70 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68e54a700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 0x7fa6881a0720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa68000d740 tx=0x7fa68000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680016070 con 0x7fa68810ff70 2024-05-20T20:53:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa68000f690 con 0x7fa68810ff70 2024-05-20T20:53:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680003970 con 0x7fa68810ff70 2024-05-20T20:53:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6881a1150 con 0x7fa68810ff70 2024-05-20T20:53:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.917+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881a89d0 con 0x7fa68810ff70 2024-05-20T20:53:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.918+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68806a230 con 0x7fa68810ff70 2024-05-20T20:53:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.920+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa680004220 con 0x7fa68810ff70 2024-05-20T20:53:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.921+0000 7fa67effd700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6740b2990 0x7fa6740b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.921+0000 7fa68dd49700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6740b2990 0x7fa6740b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.921+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fa6800e21f0 con 0x7fa68810ff70 2024-05-20T20:53:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.921+0000 7fa68dd49700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6740b2990 0x7fa6740b4e50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa684009f20 tx=0x7fa6840094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:25.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:25.924+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6800a9ea0 con 0x7fa68810ff70 2024-05-20T20:53:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.147+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa68802e560 con 0x7fa68810ff70 2024-05-20T20:53:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.149+0000 7fa67effd700 1 -- 172.21.15.26:0/1696225708 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22172 (secure 0 0 0) 0x7fa6800a97a0 con 0x7fa68810ff70 2024-05-20T20:53:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:26.149 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:52:47.191576+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:53:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6740b2990 msgr2=0x7fa6740b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6740b2990 0x7fa6740b4e50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa684009f20 tx=0x7fa6840094f0 comp rx=0 tx=0).stop 2024-05-20T20:53:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 msgr2=0x7fa6881a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 0x7fa6881a0720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa68000d740 tx=0x7fa68000da50 comp rx=0 tx=0).stop 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 shutdown_connections 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa6740b2990 0x7fa6740b4e50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68810ff70 0x7fa6881a0720 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa688075770 0x7fa68819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 --2- 172.21.15.26:0/1696225708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688076090 0x7fa68819c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 >> 172.21.15.26:0/1696225708 conn(0x7fa68807a6f0 msgr2=0x7fa688100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 shutdown_connections 2024-05-20T20:53:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:26.151+0000 7fa68ffad700 1 -- 172.21.15.26:0/1696225708 wait complete. 2024-05-20T20:53:26.879 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-20T20:53:26.880 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:53:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:27.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:26 smithi055 ceph-mon[35213]: pgmap v303: 97 pgs: 97 active+clean; 634 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-20T20:53:27.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:26 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1696225708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:27.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:26 smithi026 ceph-mon[31990]: pgmap v303: 97 pgs: 97 active+clean; 634 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-20T20:53:27.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:26 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1696225708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:26 smithi144 ceph-mon[35660]: pgmap v303: 97 pgs: 97 active+clean; 634 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-20T20:53:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:26 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1696225708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 -- 172.21.15.26:0/3539469428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06c470 msgr2=0x7f893c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 --2- 172.21.15.26:0/3539469428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06c470 0x7f893c1103c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f893400b0a0 tx=0x7f893400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 -- 172.21.15.26:0/3539469428 shutdown_connections 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 --2- 172.21.15.26:0/3539469428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06c470 0x7f893c1103c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 --2- 172.21.15.26:0/3539469428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f893c1095a0 0x7f893c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 --2- 172.21.15.26:0/3539469428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f893c06bab0 0x7f893c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 -- 172.21.15.26:0/3539469428 >> 172.21.15.26:0/3539469428 conn(0x7f893c0793b0 msgr2=0x7f893c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 -- 172.21.15.26:0/3539469428 shutdown_connections 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.761+0000 7f8941eb7700 1 -- 172.21.15.26:0/3539469428 wait complete. 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 Processor -- start 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 -- start start 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 0x7f893c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f893c06c470 0x7f893c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f893c1095a0 0x7f893c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1135e0 con 0x7f893c06bab0 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f893c113460 con 0x7f893c1095a0 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f8941eb7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f893c113760 con 0x7f893c06c470 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 0x7f893c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 0x7f893c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57746/0 (socket says 172.21.15.26:57746) 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 -- 172.21.15.26:0/2072183102 learned_addr learned my addr 172.21.15.26:0/2072183102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893bfff700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f893c1095a0 0x7f893c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893affd700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f893c06c470 0x7f893c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 -- 172.21.15.26:0/2072183102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f893c06c470 msgr2=0x7f893c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f893c06c470 0x7f893c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 -- 172.21.15.26:0/2072183102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f893c1095a0 msgr2=0x7f893c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f893c1095a0 0x7f893c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.762+0000 7f893b7fe700 1 -- 172.21.15.26:0/2072183102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c19a060 con 0x7f893c06bab0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.763+0000 7f893b7fe700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 0x7f893c19f740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c9b0 tx=0x7f892c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.763+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c007b00 con 0x7f893c06bab0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.763+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c19a2f0 con 0x7f893c06bab0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.763+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f892c00ce50 con 0x7f893c06bab0 2024-05-20T20:53:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.763+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c018660 con 0x7f893c06bab0 2024-05-20T20:53:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.763+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1a6610 con 0x7f893c06bab0 2024-05-20T20:53:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.764+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c06a230 con 0x7f893c06bab0 2024-05-20T20:53:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.765+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f892c00f450 con 0x7f893c06bab0 2024-05-20T20:53:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.766+0000 7f8938ff9700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f89280b2910 0x7f89280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.766+0000 7f893affd700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f89280b2910 0x7f89280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.766+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f892c0e23c0 con 0x7f893c06bab0 2024-05-20T20:53:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.767+0000 7f893affd700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f89280b2910 0x7f89280b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f893c079a90 tx=0x7f8924009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.770+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f892c014030 con 0x7f893c06bab0 2024-05-20T20:53:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.984+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f893c063980 con 0x7f89280b2910 2024-05-20T20:53:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.992+0000 7f8938ff9700 1 -- 172.21.15.26:0/2072183102 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258832 (secure 0 0 0) 0x7f893c063980 con 0x7f89280b2910 2024-05-20T20:53:28.996 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f89280b2910 msgr2=0x7f89280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f89280b2910 0x7f89280b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f893c079a90 tx=0x7f8924009190 comp rx=0 tx=0).stop 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 msgr2=0x7f893c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 0x7f893c19f740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c9b0 tx=0x7f892c00ccc0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 shutdown_connections 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f89280b2910 0x7f89280b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c06bab0 0x7f893c19f740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f893c1095a0 0x7f893c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 --2- 172.21.15.26:0/2072183102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f893c06c470 0x7f893c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.995+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 >> 172.21.15.26:0/2072183102 conn(0x7f893c0793b0 msgr2=0x7f893c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.996+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 shutdown_connections 2024-05-20T20:53:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:28.996+0000 7f8941eb7700 1 -- 172.21.15.26:0/2072183102 wait complete. 2024-05-20T20:53:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:53:29.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:28 smithi055 ceph-mon[35213]: pgmap v304: 97 pgs: 97 active+clean; 634 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-20T20:53:29.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:28 smithi026 ceph-mon[31990]: pgmap v304: 97 pgs: 97 active+clean; 634 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-20T20:53:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:28 smithi144 ceph-mon[35660]: pgmap v304: 97 pgs: 97 active+clean; 634 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-20T20:53:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-05-20T20:53:27.690712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649046,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":247,"num_read_kb":209,"num_write":280,"num_write_kb":2253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355348,"kb_used_data":10356,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586476,"statfs":{"total":1151940427776,"available":1151576551424,"internally_reserved":0,"allocated":10604544,"data_stored":6836490,"data_compressed":110067,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32230,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":18,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003168"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243219+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:52:47.243219+0000","last_peered":"2024-05-20T20:52:47.243219+0000","last_clean":"2024-05-20T20:52:47.243219+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:52:47.243219+0000","last_undegraded":"2024-05-20T20:52:47.243219+0000","last_fullsized":"2024-05-20T20:52:47.243219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243205+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:52:47.243205+0000","last_peered":"2024-05-20T20:52:47.243205+0000","last_clean":"2024-05-20T20:52:47.243205+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:52:47.243205+0000","last_undegraded":"2024-05-20T20:52:47.243205+0000","last_fullsized":"2024-05-20T20:52:47.243205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242336+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:52:47.242336+0000","last_peered":"2024-05-20T20:52:47.242336+0000","last_clean":"2024-05-20T20:52:47.242336+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:52:47.242336+0000","last_undegraded":"2024-05-20T20:52:47.242336+0000","last_fullsized":"2024-05-20T20:52:47.242336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.234285+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:52:47.234285+0000","last_peered":"2024-05-20T20:52:47.234285+0000","last_clean":"2024-05-20T20:52:47.234285+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:52:47.234285+0000","last_undegraded":"2024-05-20T20:52:47.234285+0000","last_fullsized":"2024-05-20T20:52:47.234285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.245144+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:52:47.245144+0000","last_peered":"2024-05-20T20:52:47.245144+0000","last_clean":"2024-05-20T20:52:47.245144+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:52:47.245144+0000","last_undegraded":"2024-05-20T20:52:47.245144+0000","last_fullsized":"2024-05-20T20:52:47.245144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243678+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:52:47.243678+0000","last_peered":"2024-05-20T20:52:47.243678+0000","last_clean":"2024-05-20T20:52:47.243678+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:52:47.243678+0000","last_undegraded":"2024-05-20T20:52:47.243678+0000","last_fullsized":"2024-05-20T20:52:47.243678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243243+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:52:47.243243+0000","last_peered":"2024-05-20T20:52:47.243243+0000","last_clean":"2024-05-20T20:52:47.243243+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:52:47.243243+0000","last_undegraded":"2024-05-20T20:52:47.243243+0000","last_fullsized":"2024-05-20T20:52:47.243243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241215+0000","last_change":"2024-05-20T20:52:45.221289+0000","last_active":"2024-05-20T20:52:47.241215+0000","last_peered":"2024-05-20T20:52:47.241215+0000","last_clean":"2024-05-20T20:52:47.241215+0000","last_became_active":"2024-05-20T20:52:45.220972+0000","last_became_peered":"2024-05-20T20:52:45.220972+0000","last_unstale":"2024-05-20T20:52:47.241215+0000","last_undegraded":"2024-05-20T20:52:47.241215+0000","last_fullsized":"2024-05-20T20:52:47.241215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:52.180359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.248231+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:52:47.248231+0000","last_peered":"2024-05-20T20:52:47.248231+0000","last_clean":"2024-05-20T20:52:47.248231+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:52:47.248231+0000","last_undegraded":"2024-05-20T20:52:47.248231+0000","last_fullsized":"2024-05-20T20:52:47.248231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244308+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:52:47.244308+0000","last_peered":"2024-05-20T20:52:47.244308+0000","last_clean":"2024-05-20T20:52:47.244308+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:52:47.244308+0000","last_undegraded":"2024-05-20T20:52:47.244308+0000","last_fullsized":"2024-05-20T20:52:47.244308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243755+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:52:47.243755+0000","last_peered":"2024-05-20T20:52:47.243755+0000","last_clean":"2024-05-20T20:52:47.243755+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:52:47.243755+0000","last_undegraded":"2024-05-20T20:52:47.243755+0000","last_fullsized":"2024-05-20T20:52:47.243755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.234284+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:52:47.234284+0000","last_peered":"2024-05-20T20:52:47.234284+0000","last_clean":"2024-05-20T20:52:47.234284+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:52:47.234284+0000","last_undegraded":"2024-05-20T20:52:47.234284+0000","last_fullsized":"2024-05-20T20:52:47.234284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229150+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:52:47.229150+0000","last_peered":"2024-05-20T20:52:47.229150+0000","last_clean":"2024-05-20T20:52:47.229150+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:52:47.229150+0000","last_undegraded":"2024-05-20T20:52:47.229150+0000","last_fullsized":"2024-05-20T20:52:47.229150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242184+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:52:47.242184+0000","last_peered":"2024-05-20T20:52:47.242184+0000","last_clean":"2024-05-20T20:52:47.242184+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:52:47.242184+0000","last_undegraded":"2024-05-20T20:52:47.242184+0000","last_fullsized":"2024-05-20T20:52:47.242184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242733+0000","last_change":"2024-05-20T20:52:45.217524+0000","last_active":"2024-05-20T20:52:47.242733+0000","last_peered":"2024-05-20T20:52:47.242733+0000","last_clean":"2024-05-20T20:52:47.242733+0000","last_became_active":"2024-05-20T20:52:45.217117+0000","last_became_peered":"2024-05-20T20:52:45.217117+0000","last_unstale":"2024-05-20T20:52:47.242733+0000","last_undegraded":"2024-05-20T20:52:47.242733+0000","last_fullsized":"2024-05-20T20:52:47.242733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:02.482446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247187+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:52:47.247187+0000","last_peered":"2024-05-20T20:52:47.247187+0000","last_clean":"2024-05-20T20:52:47.247187+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:52:47.247187+0000","last_undegraded":"2024-05-20T20:52:47.247187+0000","last_fullsized":"2024-05-20T20:52:47.247187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243434+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:52:47.243434+0000","last_peered":"2024-05-20T20:52:47.243434+0000","last_clean":"2024-05-20T20:52:47.243434+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:52:47.243434+0000","last_undegraded":"2024-05-20T20:52:47.243434+0000","last_fullsized":"2024-05-20T20:52:47.243434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244769+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:52:47.244769+0000","last_peered":"2024-05-20T20:52:47.244769+0000","last_clean":"2024-05-20T20:52:47.244769+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:52:47.244769+0000","last_undegraded":"2024-05-20T20:52:47.244769+0000","last_fullsized":"2024-05-20T20:52:47.244769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242115+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:52:47.242115+0000","last_peered":"2024-05-20T20:52:47.242115+0000","last_clean":"2024-05-20T20:52:47.242115+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:52:47.242115+0000","last_undegraded":"2024-05-20T20:52:47.242115+0000","last_fullsized":"2024-05-20T20:52:47.242115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313691+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:53:00.313691+0000","last_peered":"2024-05-20T20:53:00.313691+0000","last_clean":"2024-05-20T20:53:00.313691+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:53:00.313691+0000","last_undegraded":"2024-05-20T20:53:00.313691+0000","last_fullsized":"2024-05-20T20:53:00.313691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242128+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:52:47.242128+0000","last_peered":"2024-05-20T20:52:47.242128+0000","last_clean":"2024-05-20T20:52:47.242128+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:52:47.242128+0000","last_undegraded":"2024-05-20T20:52:47.242128+0000","last_fullsized":"2024-05-20T20:52:47.242128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244288+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:52:47.244288+0000","last_peered":"2024-05-20T20:52:47.244288+0000","last_clean":"2024-05-20T20:52:47.244288+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:52:47.244288+0000","last_undegraded":"2024-05-20T20:52:47.244288+0000","last_fullsized":"2024-05-20T20:52:47.244288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242991+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:52:47.242991+0000","last_peered":"2024-05-20T20:52:47.242991+0000","last_clean":"2024-05-20T20:52:47.242991+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:52:47.242991+0000","last_undegraded":"2024-05-20T20:52:47.242991+0000","last_fullsized":"2024-05-20T20:52:47.242991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313132+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:53:00.313132+0000","last_peered":"2024-05-20T20:53:00.313132+0000","last_clean":"2024-05-20T20:53:00.313132+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:53:00.313132+0000","last_undegraded":"2024-05-20T20:53:00.313132+0000","last_fullsized":"2024-05-20T20:53:00.313132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247575+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:52:47.247575+0000","last_peered":"2024-05-20T20:52:47.247575+0000","last_clean":"2024-05-20T20:52:47.247575+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:52:47.247575+0000","last_undegraded":"2024-05-20T20:52:47.247575+0000","last_fullsized":"2024-05-20T20:52:47.247575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.248679+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:52:47.248679+0000","last_peered":"2024-05-20T20:52:47.248679+0000","last_clean":"2024-05-20T20:52:47.248679+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:52:47.248679+0000","last_undegraded":"2024-05-20T20:52:47.248679+0000","last_fullsized":"2024-05-20T20:52:47.248679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229861+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:52:47.229861+0000","last_peered":"2024-05-20T20:52:47.229861+0000","last_clean":"2024-05-20T20:52:47.229861+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:52:47.229861+0000","last_undegraded":"2024-05-20T20:52:47.229861+0000","last_fullsized":"2024-05-20T20:52:47.229861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:26.362586+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:53:26.362586+0000","last_peered":"2024-05-20T20:53:26.362586+0000","last_clean":"2024-05-20T20:53:26.362586+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:53:26.362586+0000","last_undegraded":"2024-05-20T20:53:26.362586+0000","last_fullsized":"2024-05-20T20:53:26.362586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244726+0000","last_change":"2024-05-20T20:52:45.224750+0000","last_active":"2024-05-20T20:52:47.244726+0000","last_peered":"2024-05-20T20:52:47.244726+0000","last_clean":"2024-05-20T20:52:47.244726+0000","last_became_active":"2024-05-20T20:52:45.224375+0000","last_became_peered":"2024-05-20T20:52:45.224375+0000","last_unstale":"2024-05-20T20:52:47.244726+0000","last_undegraded":"2024-05-20T20:52:47.244726+0000","last_fullsized":"2024-05-20T20:52:47.244726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:37:40.671173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.314017+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:53:00.314017+0000","last_peered":"2024-05-20T20:53:00.314017+0000","last_clean":"2024-05-20T20:53:00.314017+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:53:00.314017+0000","last_undegraded":"2024-05-20T20:53:00.314017+0000","last_fullsized":"2024-05-20T20:53:00.314017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229761+0000","last_change":"2024-05-20T20:52:45.227224+0000","last_active":"2024-05-20T20:52:47.229761+0000","last_peered":"2024-05-20T20:52:47.229761+0000","last_clean":"2024-05-20T20:52:47.229761+0000","last_became_active":"2024-05-20T20:52:45.226803+0000","last_became_peered":"2024-05-20T20:52:45.226803+0000","last_unstale":"2024-05-20T20:52:47.229761+0000","last_undegraded":"2024-05-20T20:52:47.229761+0000","last_fullsized":"2024-05-20T20:52:47.229761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:10.352003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313177+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:53:00.313177+0000","last_peered":"2024-05-20T20:53:00.313177+0000","last_clean":"2024-05-20T20:53:00.313177+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:53:00.313177+0000","last_undegraded":"2024-05-20T20:53:00.313177+0000","last_fullsized":"2024-05-20T20:53:00.313177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241025+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:52:47.241025+0000","last_peered":"2024-05-20T20:52:47.241025+0000","last_clean":"2024-05-20T20:52:47.241025+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:52:47.241025+0000","last_undegraded":"2024-05-20T20:52:47.241025+0000","last_fullsized":"2024-05-20T20:52:47.241025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229352+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:52:47.229352+0000","last_peered":"2024-05-20T20:52:47.229352+0000","last_clean":"2024-05-20T20:52:47.229352+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:52:47.229352+0000","last_undegraded":"2024-05-20T20:52:47.229352+0000","last_fullsized":"2024-05-20T20:52:47.229352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244344+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:52:47.244344+0000","last_peered":"2024-05-20T20:52:47.244344+0000","last_clean":"2024-05-20T20:52:47.244344+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:52:47.244344+0000","last_undegraded":"2024-05-20T20:52:47.244344+0000","last_fullsized":"2024-05-20T20:52:47.244344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:23.357400+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:53:23.357400+0000","last_peered":"2024-05-20T20:53:23.357400+0000","last_clean":"2024-05-20T20:53:23.357400+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:53:23.357400+0000","last_undegraded":"2024-05-20T20:53:23.357400+0000","last_fullsized":"2024-05-20T20:53:23.357400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.248568+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:52:47.248568+0000","last_peered":"2024-05-20T20:52:47.248568+0000","last_clean":"2024-05-20T20:52:47.248568+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:52:47.248568+0000","last_undegraded":"2024-05-20T20:52:47.248568+0000","last_fullsized":"2024-05-20T20:52:47.248568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.305767+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:52:59.305767+0000","last_peered":"2024-05-20T20:52:59.305767+0000","last_clean":"2024-05-20T20:52:59.305767+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:52:59.305767+0000","last_undegraded":"2024-05-20T20:52:59.305767+0000","last_fullsized":"2024-05-20T20:52:59.305767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243154+0000","last_change":"2024-05-20T20:52:45.223010+0000","last_active":"2024-05-20T20:52:47.243154+0000","last_peered":"2024-05-20T20:52:47.243154+0000","last_clean":"2024-05-20T20:52:47.243154+0000","last_became_active":"2024-05-20T20:52:45.222670+0000","last_became_peered":"2024-05-20T20:52:45.222670+0000","last_unstale":"2024-05-20T20:52:47.243154+0000","last_undegraded":"2024-05-20T20:52:47.243154+0000","last_fullsized":"2024-05-20T20:52:47.243154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:15:38.918727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304633+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:52:59.304633+0000","last_peered":"2024-05-20T20:52:59.304633+0000","last_clean":"2024-05-20T20:52:59.304633+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:52:59.304633+0000","last_undegraded":"2024-05-20T20:52:59.304633+0000","last_fullsized":"2024-05-20T20:52:59.304633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247701+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:52:47.247701+0000","last_peered":"2024-05-20T20:52:47.247701+0000","last_clean":"2024-05-20T20:52:47.247701+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:52:47.247701+0000","last_undegraded":"2024-05-20T20:52:47.247701+0000","last_fullsized":"2024-05-20T20:52:47.247701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'25","reported_seq":100,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:24.362897+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:53:24.362897+0000","last_peered":"2024-05-20T20:53:24.362897+0000","last_clean":"2024-05-20T20:53:24.362897+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:53:24.362897+0000","last_undegraded":"2024-05-20T20:53:24.362897+0000","last_fullsized":"2024-05-20T20:53:24.362897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54443,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":53,"num_read_kb":78,"num_write":26,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.245183+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:52:47.245183+0000","last_peered":"2024-05-20T20:52:47.245183+0000","last_clean":"2024-05-20T20:52:47.245183+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:52:47.245183+0000","last_undegraded":"2024-05-20T20:52:47.245183+0000","last_fullsized":"2024-05-20T20:52:47.245183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.314444+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:53:00.314444+0000","last_peered":"2024-05-20T20:53:00.314444+0000","last_clean":"2024-05-20T20:53:00.314444+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:53:00.314444+0000","last_undegraded":"2024-05-20T20:53:00.314444+0000","last_fullsized":"2024-05-20T20:53:00.314444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.228788+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:52:47.228788+0000","last_peered":"2024-05-20T20:52:47.228788+0000","last_clean":"2024-05-20T20:52:47.228788+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:52:47.228788+0000","last_undegraded":"2024-05-20T20:52:47.228788+0000","last_fullsized":"2024-05-20T20:52:47.228788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304246+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:52:59.304246+0000","last_peered":"2024-05-20T20:52:59.304246+0000","last_clean":"2024-05-20T20:52:59.304246+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:52:59.304246+0000","last_undegraded":"2024-05-20T20:52:59.304246+0000","last_fullsized":"2024-05-20T20:52:59.304246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243607+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:52:47.243607+0000","last_peered":"2024-05-20T20:52:47.243607+0000","last_clean":"2024-05-20T20:52:47.243607+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:52:47.243607+0000","last_undegraded":"2024-05-20T20:52:47.243607+0000","last_fullsized":"2024-05-20T20:52:47.243607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:25.364369+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:53:25.364369+0000","last_peered":"2024-05-20T20:53:25.364369+0000","last_clean":"2024-05-20T20:53:25.364369+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:53:25.364369+0000","last_undegraded":"2024-05-20T20:53:25.364369+0000","last_fullsized":"2024-05-20T20:53:25.364369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243553+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:52:47.243553+0000","last_peered":"2024-05-20T20:52:47.243553+0000","last_clean":"2024-05-20T20:52:47.243553+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:52:47.243553+0000","last_undegraded":"2024-05-20T20:52:47.243553+0000","last_fullsized":"2024-05-20T20:52:47.243553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.314497+0000","last_change":"2024-05-20T20:52:43.010733+0000","last_active":"2024-05-20T20:53:00.314497+0000","last_peered":"2024-05-20T20:53:00.314497+0000","last_clean":"2024-05-20T20:53:00.314497+0000","last_became_active":"2024-05-20T20:52:43.010317+0000","last_became_peered":"2024-05-20T20:52:43.010317+0000","last_unstale":"2024-05-20T20:53:00.314497+0000","last_undegraded":"2024-05-20T20:53:00.314497+0000","last_fullsized":"2024-05-20T20:53:00.314497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:28.507898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229459+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:52:47.229459+0000","last_peered":"2024-05-20T20:52:47.229459+0000","last_clean":"2024-05-20T20:52:47.229459+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:52:47.229459+0000","last_undegraded":"2024-05-20T20:52:47.229459+0000","last_fullsized":"2024-05-20T20:52:47.229459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:01.309962+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:53:01.309962+0000","last_peered":"2024-05-20T20:53:01.309962+0000","last_clean":"2024-05-20T20:53:01.309962+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:53:01.309962+0000","last_undegraded":"2024-05-20T20:53:01.309962+0000","last_fullsized":"2024-05-20T20:53:01.309962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242801+0000","last_change":"2024-05-20T20:52:45.226982+0000","last_active":"2024-05-20T20:52:47.242801+0000","last_peered":"2024-05-20T20:52:47.242801+0000","last_clean":"2024-05-20T20:52:47.242801+0000","last_became_active":"2024-05-20T20:52:45.226428+0000","last_became_peered":"2024-05-20T20:52:45.226428+0000","last_unstale":"2024-05-20T20:52:47.242801+0000","last_undegraded":"2024-05-20T20:52:47.242801+0000","last_fullsized":"2024-05-20T20:52:47.242801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:51:11.124006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244449+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:52:47.244449+0000","last_peered":"2024-05-20T20:52:47.244449+0000","last_clean":"2024-05-20T20:52:47.244449+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:52:47.244449+0000","last_undegraded":"2024-05-20T20:52:47.244449+0000","last_fullsized":"2024-05-20T20:52:47.244449+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243450+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:52:47.243450+0000","last_peered":"2024-05-20T20:52:47.243450+0000","last_clean":"2024-05-20T20:52:47.243450+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:52:47.243450+0000","last_undegraded":"2024-05-20T20:52:47.243450+0000","last_fullsized":"2024-05-20T20:52:47.243450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243464+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:52:47.243464+0000","last_peered":"2024-05-20T20:52:47.243464+0000","last_clean":"2024-05-20T20:52:47.243464+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:52:47.243464+0000","last_undegraded":"2024-05-20T20:52:47.243464+0000","last_fullsized":"2024-05-20T20:52:47.243464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244140+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:52:47.244140+0000","last_peered":"2024-05-20T20:52:47.244140+0000","last_clean":"2024-05-20T20:52:47.244140+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:52:47.244140+0000","last_undegraded":"2024-05-20T20:52:47.244140+0000","last_fullsized":"2024-05-20T20:52:47.244140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229813+0000","last_change":"2024-05-20T20:52:45.220067+0000","last_active":"2024-05-20T20:52:47.229813+0000","last_peered":"2024-05-20T20:52:47.229813+0000","last_clean":"2024-05-20T20:52:47.229813+0000","last_became_active":"2024-05-20T20:52:45.219656+0000","last_became_peered":"2024-05-20T20:52:45.219656+0000","last_unstale":"2024-05-20T20:52:47.229813+0000","last_undegraded":"2024-05-20T20:52:47.229813+0000","last_fullsized":"2024-05-20T20:52:47.229813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:05.930913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:25.363839+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:53:25.363839+0000","last_peered":"2024-05-20T20:53:25.363839+0000","last_clean":"2024-05-20T20:53:25.363839+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:53:25.363839+0000","last_undegraded":"2024-05-20T20:53:25.363839+0000","last_fullsized":"2024-05-20T20:53:25.363839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241625+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:52:47.241625+0000","last_peered":"2024-05-20T20:52:47.241625+0000","last_clean":"2024-05-20T20:52:47.241625+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:52:47.241625+0000","last_undegraded":"2024-05-20T20:52:47.241625+0000","last_fullsized":"2024-05-20T20:52:47.241625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:02.322592+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:53:02.322592+0000","last_peered":"2024-05-20T20:53:02.322592+0000","last_clean":"2024-05-20T20:53:02.322592+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:53:02.322592+0000","last_undegraded":"2024-05-20T20:53:02.322592+0000","last_fullsized":"2024-05-20T20:53:02.322592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242225+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:52:47.242225+0000","last_peered":"2024-05-20T20:52:47.242225+0000","last_clean":"2024-05-20T20:52:47.242225+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:52:47.242225+0000","last_undegraded":"2024-05-20T20:52:47.242225+0000","last_fullsized":"2024-05-20T20:52:47.242225+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.228664+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:52:47.228664+0000","last_peered":"2024-05-20T20:52:47.228664+0000","last_clean":"2024-05-20T20:52:47.228664+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:52:47.228664+0000","last_undegraded":"2024-05-20T20:52:47.228664+0000","last_fullsized":"2024-05-20T20:52:47.228664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313836+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:53:00.313836+0000","last_peered":"2024-05-20T20:53:00.313836+0000","last_clean":"2024-05-20T20:53:00.313836+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:53:00.313836+0000","last_undegraded":"2024-05-20T20:53:00.313836+0000","last_fullsized":"2024-05-20T20:53:00.313836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.301853+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:52:59.301853+0000","last_peered":"2024-05-20T20:52:59.301853+0000","last_clean":"2024-05-20T20:52:59.301853+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:52:59.301853+0000","last_undegraded":"2024-05-20T20:52:59.301853+0000","last_fullsized":"2024-05-20T20:52:59.301853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.228877+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:52:47.228877+0000","last_peered":"2024-05-20T20:52:47.228877+0000","last_clean":"2024-05-20T20:52:47.228877+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:52:47.228877+0000","last_undegraded":"2024-05-20T20:52:47.228877+0000","last_fullsized":"2024-05-20T20:52:47.228877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'8","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:18.238582+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:53:18.238582+0000","last_peered":"2024-05-20T20:53:18.238582+0000","last_clean":"2024-05-20T20:53:18.238582+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:53:18.238582+0000","last_undegraded":"2024-05-20T20:53:18.238582+0000","last_fullsized":"2024-05-20T20:53:18.238582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242254+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:52:47.242254+0000","last_peered":"2024-05-20T20:52:47.242254+0000","last_clean":"2024-05-20T20:52:47.242254+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:52:47.242254+0000","last_undegraded":"2024-05-20T20:52:47.242254+0000","last_fullsized":"2024-05-20T20:52:47.242254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.303566+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:52:59.303566+0000","last_peered":"2024-05-20T20:52:59.303566+0000","last_clean":"2024-05-20T20:52:59.303566+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:52:59.303566+0000","last_undegraded":"2024-05-20T20:52:59.303566+0000","last_fullsized":"2024-05-20T20:52:59.303566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243426+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:52:47.243426+0000","last_peered":"2024-05-20T20:52:47.243426+0000","last_clean":"2024-05-20T20:52:47.243426+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:52:47.243426+0000","last_undegraded":"2024-05-20T20:52:47.243426+0000","last_fullsized":"2024-05-20T20:52:47.243426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.305086+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:52:59.305086+0000","last_peered":"2024-05-20T20:52:59.305086+0000","last_clean":"2024-05-20T20:52:59.305086+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:52:59.305086+0000","last_undegraded":"2024-05-20T20:52:59.305086+0000","last_fullsized":"2024-05-20T20:52:59.305086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244652+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:52:47.244652+0000","last_peered":"2024-05-20T20:52:47.244652+0000","last_clean":"2024-05-20T20:52:47.244652+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:52:47.244652+0000","last_undegraded":"2024-05-20T20:52:47.244652+0000","last_fullsized":"2024-05-20T20:52:47.244652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304565+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:52:59.304565+0000","last_peered":"2024-05-20T20:52:59.304565+0000","last_clean":"2024-05-20T20:52:59.304565+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:52:59.304565+0000","last_undegraded":"2024-05-20T20:52:59.304565+0000","last_fullsized":"2024-05-20T20:52:59.304565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241787+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:52:47.241787+0000","last_peered":"2024-05-20T20:52:47.241787+0000","last_clean":"2024-05-20T20:52:47.241787+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:52:47.241787+0000","last_undegraded":"2024-05-20T20:52:47.241787+0000","last_fullsized":"2024-05-20T20:52:47.241787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242545+0000","last_change":"2024-05-20T20:52:42.203379+0000","last_active":"2024-05-20T20:52:47.242545+0000","last_peered":"2024-05-20T20:52:47.242545+0000","last_clean":"2024-05-20T20:52:47.242545+0000","last_became_active":"2024-05-20T20:52:42.203016+0000","last_became_peered":"2024-05-20T20:52:42.203016+0000","last_unstale":"2024-05-20T20:52:47.242545+0000","last_undegraded":"2024-05-20T20:52:47.242545+0000","last_fullsized":"2024-05-20T20:52:47.242545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:35.875685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241840+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:52:47.241840+0000","last_peered":"2024-05-20T20:52:47.241840+0000","last_clean":"2024-05-20T20:52:47.241840+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:52:47.241840+0000","last_undegraded":"2024-05-20T20:52:47.241840+0000","last_fullsized":"2024-05-20T20:52:47.241840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304045+0000","last_change":"2024-05-20T20:52:42.205004+0000","last_active":"2024-05-20T20:52:59.304045+0000","last_peered":"2024-05-20T20:52:59.304045+0000","last_clean":"2024-05-20T20:52:59.304045+0000","last_became_active":"2024-05-20T20:52:42.204608+0000","last_became_peered":"2024-05-20T20:52:42.204608+0000","last_unstale":"2024-05-20T20:52:59.304045+0000","last_undegraded":"2024-05-20T20:52:59.304045+0000","last_fullsized":"2024-05-20T20:52:59.304045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:55.602215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243305+0000","last_change":"2024-05-20T20:52:45.226470+0000","last_active":"2024-05-20T20:52:47.243305+0000","last_peered":"2024-05-20T20:52:47.243305+0000","last_clean":"2024-05-20T20:52:47.243305+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:52:47.243305+0000","last_undegraded":"2024-05-20T20:52:47.243305+0000","last_fullsized":"2024-05-20T20:52:47.243305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:55:28.220915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313599+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:53:00.313599+0000","last_peered":"2024-05-20T20:53:00.313599+0000","last_clean":"2024-05-20T20:53:00.313599+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:53:00.313599+0000","last_undegraded":"2024-05-20T20:53:00.313599+0000","last_fullsized":"2024-05-20T20:53:00.313599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243673+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:52:47.243673+0000","last_peered":"2024-05-20T20:52:47.243673+0000","last_clean":"2024-05-20T20:52:47.243673+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:52:47.243673+0000","last_undegraded":"2024-05-20T20:52:47.243673+0000","last_fullsized":"2024-05-20T20:52:47.243673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247661+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:52:47.247661+0000","last_peered":"2024-05-20T20:52:47.247661+0000","last_clean":"2024-05-20T20:52:47.247661+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:52:47.247661+0000","last_undegraded":"2024-05-20T20:52:47.247661+0000","last_fullsized":"2024-05-20T20:52:47.247661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:48.485088+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:52:48.485088+0000","last_peered":"2024-05-20T20:52:48.485088+0000","last_clean":"2024-05-20T20:52:48.485088+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:52:48.485088+0000","last_undegraded":"2024-05-20T20:52:48.485088+0000","last_fullsized":"2024-05-20T20:52:48.485088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241644+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:52:47.241644+0000","last_peered":"2024-05-20T20:52:47.241644+0000","last_clean":"2024-05-20T20:52:47.241644+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:52:47.241644+0000","last_undegraded":"2024-05-20T20:52:47.241644+0000","last_fullsized":"2024-05-20T20:52:47.241644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243580+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:52:47.243580+0000","last_peered":"2024-05-20T20:52:47.243580+0000","last_clean":"2024-05-20T20:52:47.243580+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:52:47.243580+0000","last_undegraded":"2024-05-20T20:52:47.243580+0000","last_fullsized":"2024-05-20T20:52:47.243580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242046+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:52:47.242046+0000","last_peered":"2024-05-20T20:52:47.242046+0000","last_clean":"2024-05-20T20:52:47.242046+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:52:47.242046+0000","last_undegraded":"2024-05-20T20:52:47.242046+0000","last_fullsized":"2024-05-20T20:52:47.242046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242280+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:52:47.242280+0000","last_peered":"2024-05-20T20:52:47.242280+0000","last_clean":"2024-05-20T20:52:47.242280+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:52:47.242280+0000","last_undegraded":"2024-05-20T20:52:47.242280+0000","last_fullsized":"2024-05-20T20:52:47.242280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242375+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:52:47.242375+0000","last_peered":"2024-05-20T20:52:47.242375+0000","last_clean":"2024-05-20T20:52:47.242375+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:52:47.242375+0000","last_undegraded":"2024-05-20T20:52:47.242375+0000","last_fullsized":"2024-05-20T20:52:47.242375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243367+0000","last_change":"2024-05-20T20:52:45.215328+0000","last_active":"2024-05-20T20:52:47.243367+0000","last_peered":"2024-05-20T20:52:47.243367+0000","last_clean":"2024-05-20T20:52:47.243367+0000","last_became_active":"2024-05-20T20:52:45.215017+0000","last_became_peered":"2024-05-20T20:52:45.215017+0000","last_unstale":"2024-05-20T20:52:47.243367+0000","last_undegraded":"2024-05-20T20:52:47.243367+0000","last_fullsized":"2024-05-20T20:52:47.243367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:00.453508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229054+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:52:47.229054+0000","last_peered":"2024-05-20T20:52:47.229054+0000","last_clean":"2024-05-20T20:52:47.229054+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:52:47.229054+0000","last_undegraded":"2024-05-20T20:52:47.229054+0000","last_fullsized":"2024-05-20T20:52:47.229054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242452+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:52:47.242452+0000","last_peered":"2024-05-20T20:52:47.242452+0000","last_clean":"2024-05-20T20:52:47.242452+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:52:47.242452+0000","last_undegraded":"2024-05-20T20:52:47.242452+0000","last_fullsized":"2024-05-20T20:52:47.242452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242584+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:52:47.242584+0000","last_peered":"2024-05-20T20:52:47.242584+0000","last_clean":"2024-05-20T20:52:47.242584+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:52:47.242584+0000","last_undegraded":"2024-05-20T20:52:47.242584+0000","last_fullsized":"2024-05-20T20:52:47.242584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242100+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:52:47.242100+0000","last_peered":"2024-05-20T20:52:47.242100+0000","last_clean":"2024-05-20T20:52:47.242100+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:52:47.242100+0000","last_undegraded":"2024-05-20T20:52:47.242100+0000","last_fullsized":"2024-05-20T20:52:47.242100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229550+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:52:47.229550+0000","last_peered":"2024-05-20T20:52:47.229550+0000","last_clean":"2024-05-20T20:52:47.229550+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:52:47.229550+0000","last_undegraded":"2024-05-20T20:52:47.229550+0000","last_fullsized":"2024-05-20T20:52:47.229550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241060+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:52:47.241060+0000","last_peered":"2024-05-20T20:52:47.241060+0000","last_clean":"2024-05-20T20:52:47.241060+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:52:47.241060+0000","last_undegraded":"2024-05-20T20:52:47.241060+0000","last_fullsized":"2024-05-20T20:52:47.241060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247735+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:52:47.247735+0000","last_peered":"2024-05-20T20:52:47.247735+0000","last_clean":"2024-05-20T20:52:47.247735+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:52:47.247735+0000","last_undegraded":"2024-05-20T20:52:47.247735+0000","last_fullsized":"2024-05-20T20:52:47.247735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229042+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:52:47.229042+0000","last_peered":"2024-05-20T20:52:47.229042+0000","last_clean":"2024-05-20T20:52:47.229042+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:52:47.229042+0000","last_undegraded":"2024-05-20T20:52:47.229042+0000","last_fullsized":"2024-05-20T20:52:47.229042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242015+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:52:47.242015+0000","last_peered":"2024-05-20T20:52:47.242015+0000","last_clean":"2024-05-20T20:52:47.242015+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:52:47.242015+0000","last_undegraded":"2024-05-20T20:52:47.242015+0000","last_fullsized":"2024-05-20T20:52:47.242015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58444,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":171,"num_read_kb":145,"num_write":155,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":1054343,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070497,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266726,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":1000114,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626481,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":461889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":463239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":408481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":999407,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":55793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:53:29.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:53:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:29.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:29 smithi026 ceph-mon[31990]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:53:30.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:29 smithi055 ceph-mon[35213]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:53:30.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:29 smithi144 ceph-mon[35660]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:53:31.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:30 smithi026 ceph-mon[31990]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-20T20:53:31.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:30 smithi055 ceph-mon[35213]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-20T20:53:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:30 smithi144 ceph-mon[35660]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 -- 172.21.15.26:0/2137638645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c103820 msgr2=0x7f644c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 --2- 172.21.15.26:0/2137638645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c103820 0x7f644c103c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f64480099c0 tx=0x7f6448009cd0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 -- 172.21.15.26:0/2137638645 shutdown_connections 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 --2- 172.21.15.26:0/2137638645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c103820 0x7f644c103c00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 --2- 172.21.15.26:0/2137638645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f644c108910 0x7f644c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 --2- 172.21.15.26:0/2137638645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644c1041d0 0x7f644c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 -- 172.21.15.26:0/2137638645 >> 172.21.15.26:0/2137638645 conn(0x7f644c07a6b0 msgr2=0x7f644c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 -- 172.21.15.26:0/2137638645 shutdown_connections 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.588+0000 7f645462a700 1 -- 172.21.15.26:0/2137638645 wait complete. 2024-05-20T20:53:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 Processor -- start 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 -- start start 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f644c103820 0x7f644c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 0x7f644c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644c108910 0x7f644c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f644c113600 con 0x7f644c1041d0 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f644c113480 con 0x7f644c103820 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f645462a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f644c113780 con 0x7f644c108910 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 0x7f644c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 0x7f644c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39762/0 (socket says 172.21.15.26:39762) 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 -- 172.21.15.26:0/1627147125 learned_addr learned my addr 172.21.15.26:0/1627147125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f64523c6700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f644c103820 0x7f644c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6452bc7700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644c108910 0x7f644c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 -- 172.21.15.26:0/1627147125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644c108910 msgr2=0x7f644c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644c108910 0x7f644c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 -- 172.21.15.26:0/1627147125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f644c103820 msgr2=0x7f644c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f644c103820 0x7f644c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 -- 172.21.15.26:0/1627147125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c19a080 con 0x7f644c1041d0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.589+0000 7f6451bc5700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 0x7f644c19fca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f643c00b530 tx=0x7f643c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.590+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c015020 con 0x7f644c1041d0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.590+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f643c011cc0 con 0x7f644c1041d0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.590+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c19a370 con 0x7f644c1041d0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.590+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c010dc0 con 0x7f644c1041d0 2024-05-20T20:53:31.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.590+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c1a6630 con 0x7f644c1041d0 2024-05-20T20:53:31.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.591+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c10bb50 con 0x7f644c1041d0 2024-05-20T20:53:31.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.593+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f643c018020 con 0x7f644c1041d0 2024-05-20T20:53:31.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.594+0000 7f64437fe700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f64380b2910 0x7f64380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:31.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.594+0000 7f64523c6700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f64380b2910 0x7f64380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:31.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.594+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f643c0e1fe0 con 0x7f644c1041d0 2024-05-20T20:53:31.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.594+0000 7f64523c6700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f64380b2910 0x7f64380b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6448009990 tx=0x7f644801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:31.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.597+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f643c01f020 con 0x7f644c1041d0 2024-05-20T20:53:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.805+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f644c063980 con 0x7f64380b2910 2024-05-20T20:53:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.813+0000 7f64437fe700 1 -- 172.21.15.26:0/1627147125 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258835 (secure 0 0 0) 0x7f644c063980 con 0x7f64380b2910 2024-05-20T20:53:31.818 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f64380b2910 msgr2=0x7f64380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f64380b2910 0x7f64380b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6448009990 tx=0x7f644801a040 comp rx=0 tx=0).stop 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 msgr2=0x7f644c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 0x7f644c19fca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f643c00b530 tx=0x7f643c00b840 comp rx=0 tx=0).stop 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 shutdown_connections 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644c108910 0x7f644c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f644c103820 0x7f644c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f64380b2910 0x7f64380b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 --2- 172.21.15.26:0/1627147125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1041d0 0x7f644c19fca0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:31.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.816+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 >> 172.21.15.26:0/1627147125 conn(0x7f644c07a6b0 msgr2=0x7f644c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:31.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.817+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 shutdown_connections 2024-05-20T20:53:31.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:31.817+0000 7f645462a700 1 -- 172.21.15.26:0/1627147125 wait complete. 2024-05-20T20:53:31.821 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:53:32.484 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-05-20T20:53:31.691958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650184,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":263,"num_read_kb":221,"num_write":286,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355348,"kb_used_data":10356,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586476,"statfs":{"total":1151940427776,"available":1151576551424,"internally_reserved":0,"allocated":10604544,"data_stored":6839904,"data_compressed":110067,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13380,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":44,"num_write":12,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003292"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243219+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:52:47.243219+0000","last_peered":"2024-05-20T20:52:47.243219+0000","last_clean":"2024-05-20T20:52:47.243219+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:52:47.243219+0000","last_undegraded":"2024-05-20T20:52:47.243219+0000","last_fullsized":"2024-05-20T20:52:47.243219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243205+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:52:47.243205+0000","last_peered":"2024-05-20T20:52:47.243205+0000","last_clean":"2024-05-20T20:52:47.243205+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:52:47.243205+0000","last_undegraded":"2024-05-20T20:52:47.243205+0000","last_fullsized":"2024-05-20T20:52:47.243205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242336+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:52:47.242336+0000","last_peered":"2024-05-20T20:52:47.242336+0000","last_clean":"2024-05-20T20:52:47.242336+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:52:47.242336+0000","last_undegraded":"2024-05-20T20:52:47.242336+0000","last_fullsized":"2024-05-20T20:52:47.242336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.234285+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:52:47.234285+0000","last_peered":"2024-05-20T20:52:47.234285+0000","last_clean":"2024-05-20T20:52:47.234285+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:52:47.234285+0000","last_undegraded":"2024-05-20T20:52:47.234285+0000","last_fullsized":"2024-05-20T20:52:47.234285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.245144+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:52:47.245144+0000","last_peered":"2024-05-20T20:52:47.245144+0000","last_clean":"2024-05-20T20:52:47.245144+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:52:47.245144+0000","last_undegraded":"2024-05-20T20:52:47.245144+0000","last_fullsized":"2024-05-20T20:52:47.245144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243678+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:52:47.243678+0000","last_peered":"2024-05-20T20:52:47.243678+0000","last_clean":"2024-05-20T20:52:47.243678+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:52:47.243678+0000","last_undegraded":"2024-05-20T20:52:47.243678+0000","last_fullsized":"2024-05-20T20:52:47.243678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243243+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:52:47.243243+0000","last_peered":"2024-05-20T20:52:47.243243+0000","last_clean":"2024-05-20T20:52:47.243243+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:52:47.243243+0000","last_undegraded":"2024-05-20T20:52:47.243243+0000","last_fullsized":"2024-05-20T20:52:47.243243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241215+0000","last_change":"2024-05-20T20:52:45.221289+0000","last_active":"2024-05-20T20:52:47.241215+0000","last_peered":"2024-05-20T20:52:47.241215+0000","last_clean":"2024-05-20T20:52:47.241215+0000","last_became_active":"2024-05-20T20:52:45.220972+0000","last_became_peered":"2024-05-20T20:52:45.220972+0000","last_unstale":"2024-05-20T20:52:47.241215+0000","last_undegraded":"2024-05-20T20:52:47.241215+0000","last_fullsized":"2024-05-20T20:52:47.241215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:52.180359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.248231+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:52:47.248231+0000","last_peered":"2024-05-20T20:52:47.248231+0000","last_clean":"2024-05-20T20:52:47.248231+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:52:47.248231+0000","last_undegraded":"2024-05-20T20:52:47.248231+0000","last_fullsized":"2024-05-20T20:52:47.248231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244308+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:52:47.244308+0000","last_peered":"2024-05-20T20:52:47.244308+0000","last_clean":"2024-05-20T20:52:47.244308+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:52:47.244308+0000","last_undegraded":"2024-05-20T20:52:47.244308+0000","last_fullsized":"2024-05-20T20:52:47.244308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243755+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:52:47.243755+0000","last_peered":"2024-05-20T20:52:47.243755+0000","last_clean":"2024-05-20T20:52:47.243755+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:52:47.243755+0000","last_undegraded":"2024-05-20T20:52:47.243755+0000","last_fullsized":"2024-05-20T20:52:47.243755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.234284+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:52:47.234284+0000","last_peered":"2024-05-20T20:52:47.234284+0000","last_clean":"2024-05-20T20:52:47.234284+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:52:47.234284+0000","last_undegraded":"2024-05-20T20:52:47.234284+0000","last_fullsized":"2024-05-20T20:52:47.234284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229150+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:52:47.229150+0000","last_peered":"2024-05-20T20:52:47.229150+0000","last_clean":"2024-05-20T20:52:47.229150+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:52:47.229150+0000","last_undegraded":"2024-05-20T20:52:47.229150+0000","last_fullsized":"2024-05-20T20:52:47.229150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242184+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:52:47.242184+0000","last_peered":"2024-05-20T20:52:47.242184+0000","last_clean":"2024-05-20T20:52:47.242184+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:52:47.242184+0000","last_undegraded":"2024-05-20T20:52:47.242184+0000","last_fullsized":"2024-05-20T20:52:47.242184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242733+0000","last_change":"2024-05-20T20:52:45.217524+0000","last_active":"2024-05-20T20:52:47.242733+0000","last_peered":"2024-05-20T20:52:47.242733+0000","last_clean":"2024-05-20T20:52:47.242733+0000","last_became_active":"2024-05-20T20:52:45.217117+0000","last_became_peered":"2024-05-20T20:52:45.217117+0000","last_unstale":"2024-05-20T20:52:47.242733+0000","last_undegraded":"2024-05-20T20:52:47.242733+0000","last_fullsized":"2024-05-20T20:52:47.242733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:02.482446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247187+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:52:47.247187+0000","last_peered":"2024-05-20T20:52:47.247187+0000","last_clean":"2024-05-20T20:52:47.247187+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:52:47.247187+0000","last_undegraded":"2024-05-20T20:52:47.247187+0000","last_fullsized":"2024-05-20T20:52:47.247187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243434+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:52:47.243434+0000","last_peered":"2024-05-20T20:52:47.243434+0000","last_clean":"2024-05-20T20:52:47.243434+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:52:47.243434+0000","last_undegraded":"2024-05-20T20:52:47.243434+0000","last_fullsized":"2024-05-20T20:52:47.243434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244769+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:52:47.244769+0000","last_peered":"2024-05-20T20:52:47.244769+0000","last_clean":"2024-05-20T20:52:47.244769+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:52:47.244769+0000","last_undegraded":"2024-05-20T20:52:47.244769+0000","last_fullsized":"2024-05-20T20:52:47.244769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242115+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:52:47.242115+0000","last_peered":"2024-05-20T20:52:47.242115+0000","last_clean":"2024-05-20T20:52:47.242115+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:52:47.242115+0000","last_undegraded":"2024-05-20T20:52:47.242115+0000","last_fullsized":"2024-05-20T20:52:47.242115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313691+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:53:00.313691+0000","last_peered":"2024-05-20T20:53:00.313691+0000","last_clean":"2024-05-20T20:53:00.313691+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:53:00.313691+0000","last_undegraded":"2024-05-20T20:53:00.313691+0000","last_fullsized":"2024-05-20T20:53:00.313691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242128+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:52:47.242128+0000","last_peered":"2024-05-20T20:52:47.242128+0000","last_clean":"2024-05-20T20:52:47.242128+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:52:47.242128+0000","last_undegraded":"2024-05-20T20:52:47.242128+0000","last_fullsized":"2024-05-20T20:52:47.242128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244288+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:52:47.244288+0000","last_peered":"2024-05-20T20:52:47.244288+0000","last_clean":"2024-05-20T20:52:47.244288+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:52:47.244288+0000","last_undegraded":"2024-05-20T20:52:47.244288+0000","last_fullsized":"2024-05-20T20:52:47.244288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242991+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:52:47.242991+0000","last_peered":"2024-05-20T20:52:47.242991+0000","last_clean":"2024-05-20T20:52:47.242991+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:52:47.242991+0000","last_undegraded":"2024-05-20T20:52:47.242991+0000","last_fullsized":"2024-05-20T20:52:47.242991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313132+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:53:00.313132+0000","last_peered":"2024-05-20T20:53:00.313132+0000","last_clean":"2024-05-20T20:53:00.313132+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:53:00.313132+0000","last_undegraded":"2024-05-20T20:53:00.313132+0000","last_fullsized":"2024-05-20T20:53:00.313132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247575+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:52:47.247575+0000","last_peered":"2024-05-20T20:52:47.247575+0000","last_clean":"2024-05-20T20:52:47.247575+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:52:47.247575+0000","last_undegraded":"2024-05-20T20:52:47.247575+0000","last_fullsized":"2024-05-20T20:52:47.247575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.248679+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:52:47.248679+0000","last_peered":"2024-05-20T20:52:47.248679+0000","last_clean":"2024-05-20T20:52:47.248679+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:52:47.248679+0000","last_undegraded":"2024-05-20T20:52:47.248679+0000","last_fullsized":"2024-05-20T20:52:47.248679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229861+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:52:47.229861+0000","last_peered":"2024-05-20T20:52:47.229861+0000","last_clean":"2024-05-20T20:52:47.229861+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:52:47.229861+0000","last_undegraded":"2024-05-20T20:52:47.229861+0000","last_fullsized":"2024-05-20T20:52:47.229861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:26.362586+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:53:26.362586+0000","last_peered":"2024-05-20T20:53:26.362586+0000","last_clean":"2024-05-20T20:53:26.362586+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:53:26.362586+0000","last_undegraded":"2024-05-20T20:53:26.362586+0000","last_fullsized":"2024-05-20T20:53:26.362586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244726+0000","last_change":"2024-05-20T20:52:45.224750+0000","last_active":"2024-05-20T20:52:47.244726+0000","last_peered":"2024-05-20T20:52:47.244726+0000","last_clean":"2024-05-20T20:52:47.244726+0000","last_became_active":"2024-05-20T20:52:45.224375+0000","last_became_peered":"2024-05-20T20:52:45.224375+0000","last_unstale":"2024-05-20T20:52:47.244726+0000","last_undegraded":"2024-05-20T20:52:47.244726+0000","last_fullsized":"2024-05-20T20:52:47.244726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:37:40.671173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.314017+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:53:00.314017+0000","last_peered":"2024-05-20T20:53:00.314017+0000","last_clean":"2024-05-20T20:53:00.314017+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:53:00.314017+0000","last_undegraded":"2024-05-20T20:53:00.314017+0000","last_fullsized":"2024-05-20T20:53:00.314017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229761+0000","last_change":"2024-05-20T20:52:45.227224+0000","last_active":"2024-05-20T20:52:47.229761+0000","last_peered":"2024-05-20T20:52:47.229761+0000","last_clean":"2024-05-20T20:52:47.229761+0000","last_became_active":"2024-05-20T20:52:45.226803+0000","last_became_peered":"2024-05-20T20:52:45.226803+0000","last_unstale":"2024-05-20T20:52:47.229761+0000","last_undegraded":"2024-05-20T20:52:47.229761+0000","last_fullsized":"2024-05-20T20:52:47.229761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:10.352003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313177+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:53:00.313177+0000","last_peered":"2024-05-20T20:53:00.313177+0000","last_clean":"2024-05-20T20:53:00.313177+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:53:00.313177+0000","last_undegraded":"2024-05-20T20:53:00.313177+0000","last_fullsized":"2024-05-20T20:53:00.313177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241025+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:52:47.241025+0000","last_peered":"2024-05-20T20:52:47.241025+0000","last_clean":"2024-05-20T20:52:47.241025+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:52:47.241025+0000","last_undegraded":"2024-05-20T20:52:47.241025+0000","last_fullsized":"2024-05-20T20:52:47.241025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229352+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:52:47.229352+0000","last_peered":"2024-05-20T20:52:47.229352+0000","last_clean":"2024-05-20T20:52:47.229352+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:52:47.229352+0000","last_undegraded":"2024-05-20T20:52:47.229352+0000","last_fullsized":"2024-05-20T20:52:47.229352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244344+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:52:47.244344+0000","last_peered":"2024-05-20T20:52:47.244344+0000","last_clean":"2024-05-20T20:52:47.244344+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:52:47.244344+0000","last_undegraded":"2024-05-20T20:52:47.244344+0000","last_fullsized":"2024-05-20T20:52:47.244344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:28.365013+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:53:28.365013+0000","last_peered":"2024-05-20T20:53:28.365013+0000","last_clean":"2024-05-20T20:53:28.365013+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:53:28.365013+0000","last_undegraded":"2024-05-20T20:53:28.365013+0000","last_fullsized":"2024-05-20T20:53:28.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.248568+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:52:47.248568+0000","last_peered":"2024-05-20T20:52:47.248568+0000","last_clean":"2024-05-20T20:52:47.248568+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:52:47.248568+0000","last_undegraded":"2024-05-20T20:52:47.248568+0000","last_fullsized":"2024-05-20T20:52:47.248568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.305767+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:52:59.305767+0000","last_peered":"2024-05-20T20:52:59.305767+0000","last_clean":"2024-05-20T20:52:59.305767+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:52:59.305767+0000","last_undegraded":"2024-05-20T20:52:59.305767+0000","last_fullsized":"2024-05-20T20:52:59.305767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243154+0000","last_change":"2024-05-20T20:52:45.223010+0000","last_active":"2024-05-20T20:52:47.243154+0000","last_peered":"2024-05-20T20:52:47.243154+0000","last_clean":"2024-05-20T20:52:47.243154+0000","last_became_active":"2024-05-20T20:52:45.222670+0000","last_became_peered":"2024-05-20T20:52:45.222670+0000","last_unstale":"2024-05-20T20:52:47.243154+0000","last_undegraded":"2024-05-20T20:52:47.243154+0000","last_fullsized":"2024-05-20T20:52:47.243154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:15:38.918727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304633+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:52:59.304633+0000","last_peered":"2024-05-20T20:52:59.304633+0000","last_clean":"2024-05-20T20:52:59.304633+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:52:59.304633+0000","last_undegraded":"2024-05-20T20:52:59.304633+0000","last_fullsized":"2024-05-20T20:52:59.304633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247701+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:52:47.247701+0000","last_peered":"2024-05-20T20:52:47.247701+0000","last_clean":"2024-05-20T20:52:47.247701+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:52:47.247701+0000","last_undegraded":"2024-05-20T20:52:47.247701+0000","last_fullsized":"2024-05-20T20:52:47.247701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'27","reported_seq":113,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:29.369723+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:53:29.369723+0000","last_peered":"2024-05-20T20:53:29.369723+0000","last_clean":"2024-05-20T20:53:29.369723+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:53:29.369723+0000","last_undegraded":"2024-05-20T20:53:29.369723+0000","last_fullsized":"2024-05-20T20:53:29.369723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55581,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":85,"num_write":28,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.245183+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:52:47.245183+0000","last_peered":"2024-05-20T20:52:47.245183+0000","last_clean":"2024-05-20T20:52:47.245183+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:52:47.245183+0000","last_undegraded":"2024-05-20T20:52:47.245183+0000","last_fullsized":"2024-05-20T20:52:47.245183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.314444+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:53:00.314444+0000","last_peered":"2024-05-20T20:53:00.314444+0000","last_clean":"2024-05-20T20:53:00.314444+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:53:00.314444+0000","last_undegraded":"2024-05-20T20:53:00.314444+0000","last_fullsized":"2024-05-20T20:53:00.314444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.228788+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:52:47.228788+0000","last_peered":"2024-05-20T20:52:47.228788+0000","last_clean":"2024-05-20T20:52:47.228788+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:52:47.228788+0000","last_undegraded":"2024-05-20T20:52:47.228788+0000","last_fullsized":"2024-05-20T20:52:47.228788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304246+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:52:59.304246+0000","last_peered":"2024-05-20T20:52:59.304246+0000","last_clean":"2024-05-20T20:52:59.304246+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:52:59.304246+0000","last_undegraded":"2024-05-20T20:52:59.304246+0000","last_fullsized":"2024-05-20T20:52:59.304246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243607+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:52:47.243607+0000","last_peered":"2024-05-20T20:52:47.243607+0000","last_clean":"2024-05-20T20:52:47.243607+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:52:47.243607+0000","last_undegraded":"2024-05-20T20:52:47.243607+0000","last_fullsized":"2024-05-20T20:52:47.243607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:25.364369+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:53:25.364369+0000","last_peered":"2024-05-20T20:53:25.364369+0000","last_clean":"2024-05-20T20:53:25.364369+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:53:25.364369+0000","last_undegraded":"2024-05-20T20:53:25.364369+0000","last_fullsized":"2024-05-20T20:53:25.364369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243553+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:52:47.243553+0000","last_peered":"2024-05-20T20:52:47.243553+0000","last_clean":"2024-05-20T20:52:47.243553+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:52:47.243553+0000","last_undegraded":"2024-05-20T20:52:47.243553+0000","last_fullsized":"2024-05-20T20:52:47.243553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.314497+0000","last_change":"2024-05-20T20:52:43.010733+0000","last_active":"2024-05-20T20:53:00.314497+0000","last_peered":"2024-05-20T20:53:00.314497+0000","last_clean":"2024-05-20T20:53:00.314497+0000","last_became_active":"2024-05-20T20:52:43.010317+0000","last_became_peered":"2024-05-20T20:52:43.010317+0000","last_unstale":"2024-05-20T20:53:00.314497+0000","last_undegraded":"2024-05-20T20:53:00.314497+0000","last_fullsized":"2024-05-20T20:53:00.314497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:28.507898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229459+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:52:47.229459+0000","last_peered":"2024-05-20T20:52:47.229459+0000","last_clean":"2024-05-20T20:52:47.229459+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:52:47.229459+0000","last_undegraded":"2024-05-20T20:52:47.229459+0000","last_fullsized":"2024-05-20T20:52:47.229459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:01.309962+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:53:01.309962+0000","last_peered":"2024-05-20T20:53:01.309962+0000","last_clean":"2024-05-20T20:53:01.309962+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:53:01.309962+0000","last_undegraded":"2024-05-20T20:53:01.309962+0000","last_fullsized":"2024-05-20T20:53:01.309962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242801+0000","last_change":"2024-05-20T20:52:45.226982+0000","last_active":"2024-05-20T20:52:47.242801+0000","last_peered":"2024-05-20T20:52:47.242801+0000","last_clean":"2024-05-20T20:52:47.242801+0000","last_became_active":"2024-05-20T20:52:45.226428+0000","last_became_peered":"2024-05-20T20:52:45.226428+0000","last_unstale":"2024-05-20T20:52:47.242801+0000","last_undegraded":"2024-05-20T20:52:47.242801+0000","last_fullsized":"2024-05-20T20:52:47.242801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:51:11.124006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244449+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:52:47.244449+0000","last_peered":"2024-05-20T20:52:47.244449+0000","last_clean":"2024-05-20T20:52:47.244449+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:52:47.244449+0000","last_undegraded":"2024-05-20T20:52:47.244449+0000","last_fullsized":"2024-05-20T20:52:47.244449+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243450+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:52:47.243450+0000","last_peered":"2024-05-20T20:52:47.243450+0000","last_clean":"2024-05-20T20:52:47.243450+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:52:47.243450+0000","last_undegraded":"2024-05-20T20:52:47.243450+0000","last_fullsized":"2024-05-20T20:52:47.243450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243464+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:52:47.243464+0000","last_peered":"2024-05-20T20:52:47.243464+0000","last_clean":"2024-05-20T20:52:47.243464+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:52:47.243464+0000","last_undegraded":"2024-05-20T20:52:47.243464+0000","last_fullsized":"2024-05-20T20:52:47.243464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244140+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:52:47.244140+0000","last_peered":"2024-05-20T20:52:47.244140+0000","last_clean":"2024-05-20T20:52:47.244140+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:52:47.244140+0000","last_undegraded":"2024-05-20T20:52:47.244140+0000","last_fullsized":"2024-05-20T20:52:47.244140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229813+0000","last_change":"2024-05-20T20:52:45.220067+0000","last_active":"2024-05-20T20:52:47.229813+0000","last_peered":"2024-05-20T20:52:47.229813+0000","last_clean":"2024-05-20T20:52:47.229813+0000","last_became_active":"2024-05-20T20:52:45.219656+0000","last_became_peered":"2024-05-20T20:52:45.219656+0000","last_unstale":"2024-05-20T20:52:47.229813+0000","last_undegraded":"2024-05-20T20:52:47.229813+0000","last_fullsized":"2024-05-20T20:52:47.229813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:05.930913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:25.363839+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:53:25.363839+0000","last_peered":"2024-05-20T20:53:25.363839+0000","last_clean":"2024-05-20T20:53:25.363839+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:53:25.363839+0000","last_undegraded":"2024-05-20T20:53:25.363839+0000","last_fullsized":"2024-05-20T20:53:25.363839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241625+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:52:47.241625+0000","last_peered":"2024-05-20T20:52:47.241625+0000","last_clean":"2024-05-20T20:52:47.241625+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:52:47.241625+0000","last_undegraded":"2024-05-20T20:52:47.241625+0000","last_fullsized":"2024-05-20T20:52:47.241625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:02.322592+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:53:02.322592+0000","last_peered":"2024-05-20T20:53:02.322592+0000","last_clean":"2024-05-20T20:53:02.322592+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:53:02.322592+0000","last_undegraded":"2024-05-20T20:53:02.322592+0000","last_fullsized":"2024-05-20T20:53:02.322592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242225+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:52:47.242225+0000","last_peered":"2024-05-20T20:52:47.242225+0000","last_clean":"2024-05-20T20:52:47.242225+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:52:47.242225+0000","last_undegraded":"2024-05-20T20:52:47.242225+0000","last_fullsized":"2024-05-20T20:52:47.242225+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.228664+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:52:47.228664+0000","last_peered":"2024-05-20T20:52:47.228664+0000","last_clean":"2024-05-20T20:52:47.228664+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:52:47.228664+0000","last_undegraded":"2024-05-20T20:52:47.228664+0000","last_fullsized":"2024-05-20T20:52:47.228664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313836+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:53:00.313836+0000","last_peered":"2024-05-20T20:53:00.313836+0000","last_clean":"2024-05-20T20:53:00.313836+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:53:00.313836+0000","last_undegraded":"2024-05-20T20:53:00.313836+0000","last_fullsized":"2024-05-20T20:53:00.313836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.301853+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:52:59.301853+0000","last_peered":"2024-05-20T20:52:59.301853+0000","last_clean":"2024-05-20T20:52:59.301853+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:52:59.301853+0000","last_undegraded":"2024-05-20T20:52:59.301853+0000","last_fullsized":"2024-05-20T20:52:59.301853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.228877+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:52:47.228877+0000","last_peered":"2024-05-20T20:52:47.228877+0000","last_clean":"2024-05-20T20:52:47.228877+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:52:47.228877+0000","last_undegraded":"2024-05-20T20:52:47.228877+0000","last_fullsized":"2024-05-20T20:52:47.228877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:24.239546+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:53:24.239546+0000","last_peered":"2024-05-20T20:53:24.239546+0000","last_clean":"2024-05-20T20:53:24.239546+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:53:24.239546+0000","last_undegraded":"2024-05-20T20:53:24.239546+0000","last_fullsized":"2024-05-20T20:53:24.239546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242254+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:52:47.242254+0000","last_peered":"2024-05-20T20:52:47.242254+0000","last_clean":"2024-05-20T20:52:47.242254+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:52:47.242254+0000","last_undegraded":"2024-05-20T20:52:47.242254+0000","last_fullsized":"2024-05-20T20:52:47.242254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.303566+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:52:59.303566+0000","last_peered":"2024-05-20T20:52:59.303566+0000","last_clean":"2024-05-20T20:52:59.303566+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:52:59.303566+0000","last_undegraded":"2024-05-20T20:52:59.303566+0000","last_fullsized":"2024-05-20T20:52:59.303566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243426+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:52:47.243426+0000","last_peered":"2024-05-20T20:52:47.243426+0000","last_clean":"2024-05-20T20:52:47.243426+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:52:47.243426+0000","last_undegraded":"2024-05-20T20:52:47.243426+0000","last_fullsized":"2024-05-20T20:52:47.243426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.305086+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:52:59.305086+0000","last_peered":"2024-05-20T20:52:59.305086+0000","last_clean":"2024-05-20T20:52:59.305086+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:52:59.305086+0000","last_undegraded":"2024-05-20T20:52:59.305086+0000","last_fullsized":"2024-05-20T20:52:59.305086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.244652+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:52:47.244652+0000","last_peered":"2024-05-20T20:52:47.244652+0000","last_clean":"2024-05-20T20:52:47.244652+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:52:47.244652+0000","last_undegraded":"2024-05-20T20:52:47.244652+0000","last_fullsized":"2024-05-20T20:52:47.244652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304565+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:52:59.304565+0000","last_peered":"2024-05-20T20:52:59.304565+0000","last_clean":"2024-05-20T20:52:59.304565+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:52:59.304565+0000","last_undegraded":"2024-05-20T20:52:59.304565+0000","last_fullsized":"2024-05-20T20:52:59.304565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241787+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:52:47.241787+0000","last_peered":"2024-05-20T20:52:47.241787+0000","last_clean":"2024-05-20T20:52:47.241787+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:52:47.241787+0000","last_undegraded":"2024-05-20T20:52:47.241787+0000","last_fullsized":"2024-05-20T20:52:47.241787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242545+0000","last_change":"2024-05-20T20:52:42.203379+0000","last_active":"2024-05-20T20:52:47.242545+0000","last_peered":"2024-05-20T20:52:47.242545+0000","last_clean":"2024-05-20T20:52:47.242545+0000","last_became_active":"2024-05-20T20:52:42.203016+0000","last_became_peered":"2024-05-20T20:52:42.203016+0000","last_unstale":"2024-05-20T20:52:47.242545+0000","last_undegraded":"2024-05-20T20:52:47.242545+0000","last_fullsized":"2024-05-20T20:52:47.242545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:35.875685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241840+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:52:47.241840+0000","last_peered":"2024-05-20T20:52:47.241840+0000","last_clean":"2024-05-20T20:52:47.241840+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:52:47.241840+0000","last_undegraded":"2024-05-20T20:52:47.241840+0000","last_fullsized":"2024-05-20T20:52:47.241840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:59.304045+0000","last_change":"2024-05-20T20:52:42.205004+0000","last_active":"2024-05-20T20:52:59.304045+0000","last_peered":"2024-05-20T20:52:59.304045+0000","last_clean":"2024-05-20T20:52:59.304045+0000","last_became_active":"2024-05-20T20:52:42.204608+0000","last_became_peered":"2024-05-20T20:52:42.204608+0000","last_unstale":"2024-05-20T20:52:59.304045+0000","last_undegraded":"2024-05-20T20:52:59.304045+0000","last_fullsized":"2024-05-20T20:52:59.304045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:55.602215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243305+0000","last_change":"2024-05-20T20:52:45.226470+0000","last_active":"2024-05-20T20:52:47.243305+0000","last_peered":"2024-05-20T20:52:47.243305+0000","last_clean":"2024-05-20T20:52:47.243305+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:52:47.243305+0000","last_undegraded":"2024-05-20T20:52:47.243305+0000","last_fullsized":"2024-05-20T20:52:47.243305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:55:28.220915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:53:00.313599+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:53:00.313599+0000","last_peered":"2024-05-20T20:53:00.313599+0000","last_clean":"2024-05-20T20:53:00.313599+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:53:00.313599+0000","last_undegraded":"2024-05-20T20:53:00.313599+0000","last_fullsized":"2024-05-20T20:53:00.313599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243673+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:52:47.243673+0000","last_peered":"2024-05-20T20:52:47.243673+0000","last_clean":"2024-05-20T20:52:47.243673+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:52:47.243673+0000","last_undegraded":"2024-05-20T20:52:47.243673+0000","last_fullsized":"2024-05-20T20:52:47.243673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247661+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:52:47.247661+0000","last_peered":"2024-05-20T20:52:47.247661+0000","last_clean":"2024-05-20T20:52:47.247661+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:52:47.247661+0000","last_undegraded":"2024-05-20T20:52:47.247661+0000","last_fullsized":"2024-05-20T20:52:47.247661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:48.485088+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:52:48.485088+0000","last_peered":"2024-05-20T20:52:48.485088+0000","last_clean":"2024-05-20T20:52:48.485088+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:52:48.485088+0000","last_undegraded":"2024-05-20T20:52:48.485088+0000","last_fullsized":"2024-05-20T20:52:48.485088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241644+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:52:47.241644+0000","last_peered":"2024-05-20T20:52:47.241644+0000","last_clean":"2024-05-20T20:52:47.241644+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:52:47.241644+0000","last_undegraded":"2024-05-20T20:52:47.241644+0000","last_fullsized":"2024-05-20T20:52:47.241644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243580+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:52:47.243580+0000","last_peered":"2024-05-20T20:52:47.243580+0000","last_clean":"2024-05-20T20:52:47.243580+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:52:47.243580+0000","last_undegraded":"2024-05-20T20:52:47.243580+0000","last_fullsized":"2024-05-20T20:52:47.243580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242046+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:52:47.242046+0000","last_peered":"2024-05-20T20:52:47.242046+0000","last_clean":"2024-05-20T20:52:47.242046+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:52:47.242046+0000","last_undegraded":"2024-05-20T20:52:47.242046+0000","last_fullsized":"2024-05-20T20:52:47.242046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242280+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:52:47.242280+0000","last_peered":"2024-05-20T20:52:47.242280+0000","last_clean":"2024-05-20T20:52:47.242280+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:52:47.242280+0000","last_undegraded":"2024-05-20T20:52:47.242280+0000","last_fullsized":"2024-05-20T20:52:47.242280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242375+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:52:47.242375+0000","last_peered":"2024-05-20T20:52:47.242375+0000","last_clean":"2024-05-20T20:52:47.242375+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:52:47.242375+0000","last_undegraded":"2024-05-20T20:52:47.242375+0000","last_fullsized":"2024-05-20T20:52:47.242375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.243367+0000","last_change":"2024-05-20T20:52:45.215328+0000","last_active":"2024-05-20T20:52:47.243367+0000","last_peered":"2024-05-20T20:52:47.243367+0000","last_clean":"2024-05-20T20:52:47.243367+0000","last_became_active":"2024-05-20T20:52:45.215017+0000","last_became_peered":"2024-05-20T20:52:45.215017+0000","last_unstale":"2024-05-20T20:52:47.243367+0000","last_undegraded":"2024-05-20T20:52:47.243367+0000","last_fullsized":"2024-05-20T20:52:47.243367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:00.453508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229054+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:52:47.229054+0000","last_peered":"2024-05-20T20:52:47.229054+0000","last_clean":"2024-05-20T20:52:47.229054+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:52:47.229054+0000","last_undegraded":"2024-05-20T20:52:47.229054+0000","last_fullsized":"2024-05-20T20:52:47.229054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242452+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:52:47.242452+0000","last_peered":"2024-05-20T20:52:47.242452+0000","last_clean":"2024-05-20T20:52:47.242452+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:52:47.242452+0000","last_undegraded":"2024-05-20T20:52:47.242452+0000","last_fullsized":"2024-05-20T20:52:47.242452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242584+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:52:47.242584+0000","last_peered":"2024-05-20T20:52:47.242584+0000","last_clean":"2024-05-20T20:52:47.242584+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:52:47.242584+0000","last_undegraded":"2024-05-20T20:52:47.242584+0000","last_fullsized":"2024-05-20T20:52:47.242584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242100+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:52:47.242100+0000","last_peered":"2024-05-20T20:52:47.242100+0000","last_clean":"2024-05-20T20:52:47.242100+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:52:47.242100+0000","last_undegraded":"2024-05-20T20:52:47.242100+0000","last_fullsized":"2024-05-20T20:52:47.242100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229550+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:52:47.229550+0000","last_peered":"2024-05-20T20:52:47.229550+0000","last_clean":"2024-05-20T20:52:47.229550+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:52:47.229550+0000","last_undegraded":"2024-05-20T20:52:47.229550+0000","last_fullsized":"2024-05-20T20:52:47.229550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.241060+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:52:47.241060+0000","last_peered":"2024-05-20T20:52:47.241060+0000","last_clean":"2024-05-20T20:52:47.241060+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:52:47.241060+0000","last_undegraded":"2024-05-20T20:52:47.241060+0000","last_fullsized":"2024-05-20T20:52:47.241060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.247735+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:52:47.247735+0000","last_peered":"2024-05-20T20:52:47.247735+0000","last_clean":"2024-05-20T20:52:47.247735+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:52:47.247735+0000","last_undegraded":"2024-05-20T20:52:47.247735+0000","last_fullsized":"2024-05-20T20:52:47.247735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.229042+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:52:47.229042+0000","last_peered":"2024-05-20T20:52:47.229042+0000","last_clean":"2024-05-20T20:52:47.229042+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:52:47.229042+0000","last_undegraded":"2024-05-20T20:52:47.229042+0000","last_fullsized":"2024-05-20T20:52:47.229042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-20T20:52:47.242015+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:52:47.242015+0000","last_peered":"2024-05-20T20:52:47.242015+0000","last_clean":"2024-05-20T20:52:47.242015+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:52:47.242015+0000","last_undegraded":"2024-05-20T20:52:47.242015+0000","last_fullsized":"2024-05-20T20:52:47.242015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59582,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":187,"num_read_kb":157,"num_write":161,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":178746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710743,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":1055481,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266726,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":1000114,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626481,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":463027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":464377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":408481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":999407,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":55581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:53:32.488 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-20T20:53:32.488 INFO:tasks.cephadm:Setting up client nodes... 2024-05-20T20:53:32.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-20T20:53:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:33.114 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:32 smithi026 ceph-mon[31990]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T20:53:33.114 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:32 smithi026 ceph-mon[31990]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:53:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:32 smithi055 ceph-mon[35213]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T20:53:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:32 smithi055 ceph-mon[35213]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:53:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:32 smithi144 ceph-mon[35660]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T20:53:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:32 smithi144 ceph-mon[35660]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.423+0000 7f25a60c2700 1 -- 172.21.15.26:0/1709136334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a01092b0 msgr2=0x7f25a010b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.423+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1709136334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a01092b0 0x7f25a010b720 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f259800b0a0 tx=0x7f259800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 -- 172.21.15.26:0/1709136334 shutdown_connections 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1709136334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a01092b0 0x7f25a010b720 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1709136334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25a0104050 0x7f25a0106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1709136334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a0106980 0x7f25a0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 -- 172.21.15.26:0/1709136334 >> 172.21.15.26:0/1709136334 conn(0x7f25a00fda60 msgr2=0x7f25a00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 -- 172.21.15.26:0/1709136334 shutdown_connections 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 -- 172.21.15.26:0/1709136334 wait complete. 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 Processor -- start 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.424+0000 7f25a60c2700 1 -- start start 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f25a60c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 0x7f25a019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f25a60c2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25a0106980 0x7f25a019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f25a60c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a01092b0 0x7f25a01a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f25a60c2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25a010ef50 con 0x7f25a0104050 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f25a60c2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25a010edd0 con 0x7f25a0106980 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f25a60c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25a010f0d0 con 0x7f25a01092b0 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 0x7f25a019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 0x7f25a019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39788/0 (socket says 172.21.15.26:39788) 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 -- 172.21.15.26:0/1608107514 learned_addr learned my addr 172.21.15.26:0/1608107514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259ffff700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a01092b0 0x7f25a01a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259effd700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25a0106980 0x7f25a019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 -- 172.21.15.26:0/1608107514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a01092b0 msgr2=0x7f25a01a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a01092b0 0x7f25a01a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 -- 172.21.15.26:0/1608107514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25a0106980 msgr2=0x7f25a019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25a0106980 0x7f25a019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 -- 172.21.15.26:0/1608107514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a01a33b0 con 0x7f25a0104050 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259ffff700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a01092b0 0x7f25a01a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.425+0000 7f259f7fe700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 0x7f25a019b750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f259000d9b0 tx=0x7f259000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.426+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590016070 con 0x7f25a0104050 2024-05-20T20:53:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.426+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2590004020 con 0x7f25a0104050 2024-05-20T20:53:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.426+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a01a36a0 con 0x7f25a0104050 2024-05-20T20:53:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.426+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590004bb0 con 0x7f25a0104050 2024-05-20T20:53:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.426+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a01a3c30 con 0x7f25a0104050 2024-05-20T20:53:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.427+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a0051a80 con 0x7f25a0104050 2024-05-20T20:53:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.428+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f259000bb60 con 0x7f25a0104050 2024-05-20T20:53:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.429+0000 7f259cff9700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25880b2a30 0x7f25880b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.429+0000 7f259effd700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25880b2a30 0x7f25880b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.429+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f25900e2080 con 0x7f25a0104050 2024-05-20T20:53:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.430+0000 7f259effd700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25880b2a30 0x7f25880b4ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2594006fd0 tx=0x7f2594008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.433+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f259001f020 con 0x7f25a0104050 2024-05-20T20:53:34.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.736+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f25a002e590 con 0x7f25a0104050 2024-05-20T20:53:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.739+0000 7f259cff9700 1 -- 172.21.15.26:0/1608107514 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f25900b0e20 con 0x7f25a0104050 2024-05-20T20:53:34.739 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-05-20T20:53:34.739 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBOuEtml2AGLBAAmP9lTHbMJ6ZXOHe05cQ05w== 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25880b2a30 msgr2=0x7f25880b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25880b2a30 0x7f25880b4ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2594006fd0 tx=0x7f2594008040 comp rx=0 tx=0).stop 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 msgr2=0x7f25a019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 0x7f25a019b750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f259000d9b0 tx=0x7f259000dd70 comp rx=0 tx=0).stop 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 shutdown_connections 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25880b2a30 0x7f25880b4ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a0104050 0x7f25a019b750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25a01092b0 0x7f25a01a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 --2- 172.21.15.26:0/1608107514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25a0106980 0x7f25a019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 >> 172.21.15.26:0/1608107514 conn(0x7f25a00fda60 msgr2=0x7f25a01036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:34.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 shutdown_connections 2024-05-20T20:53:34.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:34.741+0000 7f25a60c2700 1 -- 172.21.15.26:0/1608107514 wait complete. 2024-05-20T20:53:35.059 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:34 smithi026 ceph-mon[31990]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-20T20:53:35.060 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:34 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1608107514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-20T20:53:35.060 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:34 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1608107514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-20T20:53:35.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:34 smithi055 ceph-mon[35213]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-20T20:53:35.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:34 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1608107514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-20T20:53:35.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:34 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1608107514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-20T20:53:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:34 smithi144 ceph-mon[35660]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-20T20:53:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:34 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1608107514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-20T20:53:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:34 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1608107514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-20T20:53:35.392 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:53:35.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-20T20:53:35.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-20T20:53:35.434 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-20T20:53:35.435 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-20T20:53:35.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-20T20:53:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:36.114 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-20T20:53:36.114 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-20T20:53:36.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:35 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-05-20T20:53:36.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2024-05-20T20:53:36.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-20T20:53:36.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:35 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-20T20:53:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-20T20:53:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-20T20:53:36.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:35 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-20T20:53:37.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-20T20:53:37.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-20T20:53:37.089 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:36 smithi026 ceph-mon[31990]: osdmap e79: 12 total, 12 up, 12 in 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-05-20T20:53:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2024-05-20T20:53:37.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-20T20:53:37.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-20T20:53:37.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:36 smithi055 ceph-mon[35213]: osdmap e79: 12 total, 12 up, 12 in 2024-05-20T20:53:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-20T20:53:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-20T20:53:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-20T20:53:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-20T20:53:37.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:36 smithi144 ceph-mon[35660]: osdmap e79: 12 total, 12 up, 12 in 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 -- 172.21.15.26:0/135193632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 msgr2=0x7f58a0110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 --2- 172.21.15.26:0/135193632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a0110490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f589800b0d0 tx=0x7f589800b3e0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 -- 172.21.15.26:0/135193632 shutdown_connections 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 --2- 172.21.15.26:0/135193632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a0110490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 --2- 172.21.15.26:0/135193632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58a0103820 0x7f58a0103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 --2- 172.21.15.26:0/135193632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a01041d0 0x7f58a01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 -- 172.21.15.26:0/135193632 >> 172.21.15.26:0/135193632 conn(0x7f58a007a6b0 msgr2=0x7f58a007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 -- 172.21.15.26:0/135193632 shutdown_connections 2024-05-20T20:53:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 -- 172.21.15.26:0/135193632 wait complete. 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.307+0000 7f58a558f700 1 Processor -- start 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 -- start start 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a0103820 0x7f58a0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58a01041d0 0x7f58a0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58a01135b0 con 0x7f58a0108910 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0113430 con 0x7f58a01041d0 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f58a558f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0113730 con 0x7f58a0103820 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a00762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39802/0 (socket says 172.21.15.26:39802) 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 -- 172.21.15.26:0/617621125 learned_addr learned my addr 172.21.15.26:0/617621125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589effd700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a0103820 0x7f58a0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589e7fc700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58a01041d0 0x7f58a0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 -- 172.21.15.26:0/617621125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a0103820 msgr2=0x7f58a0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a0103820 0x7f58a0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 -- 172.21.15.26:0/617621125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58a01041d0 msgr2=0x7f58a0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58a01041d0 0x7f58a0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.308+0000 7f589f7fe700 1 -- 172.21.15.26:0/617621125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a01a7f90 con 0x7f58a0108910 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.309+0000 7f589f7fe700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a00762c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5898009850 tx=0x7f5898007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.309+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589801f070 con 0x7f58a0108910 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.309+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a01a8220 con 0x7f58a0108910 2024-05-20T20:53:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.309+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5898004c50 con 0x7f58a0108910 2024-05-20T20:53:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.309+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a01a86d0 con 0x7f58a0108910 2024-05-20T20:53:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.309+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589801b690 con 0x7f58a0108910 2024-05-20T20:53:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.310+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588c005330 con 0x7f58a0108910 2024-05-20T20:53:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.311+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5898022070 con 0x7f58a0108910 2024-05-20T20:53:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.312+0000 7f5887fff700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58880b2910 0x7f58880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.312+0000 7f589effd700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58880b2910 0x7f58880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.312+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f589800f020 con 0x7f58a0108910 2024-05-20T20:53:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.313+0000 7f589effd700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58880b2910 0x7f58880b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5890005950 tx=0x7f589000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.316+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5898029020 con 0x7f58a0108910 2024-05-20T20:53:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.603+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f588c0058f0 con 0x7f58a0108910 2024-05-20T20:53:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.609+0000 7f5887fff700 1 -- 172.21.15.26:0/617621125 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f58980ab420 con 0x7f58a0108910 2024-05-20T20:53:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:37.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58880b2910 msgr2=0x7f58880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58880b2910 0x7f58880b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5890005950 tx=0x7f589000b410 comp rx=0 tx=0).stop 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 msgr2=0x7f58a00762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a00762c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5898009850 tx=0x7f5898007a20 comp rx=0 tx=0).stop 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 shutdown_connections 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58880b2910 0x7f58880b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a0108910 0x7f58a00762c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a0103820 0x7f58a0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 --2- 172.21.15.26:0/617621125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58a01041d0 0x7f58a0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 >> 172.21.15.26:0/617621125 conn(0x7f58a007a6b0 msgr2=0x7f58a01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 shutdown_connections 2024-05-20T20:53:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:37.612+0000 7f58a558f700 1 -- 172.21.15.26:0/617621125 wait complete. 2024-05-20T20:53:38.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:37 smithi055 ceph-mon[35213]: osdmap e80: 12 total, 12 up, 12 in 2024-05-20T20:53:38.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:37 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/617621125' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-20T20:53:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:37 smithi026 ceph-mon[31990]: osdmap e80: 12 total, 12 up, 12 in 2024-05-20T20:53:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:37 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/617621125' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-20T20:53:38.272 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2583575856},{"type":"v1","addr":"172.21.15.26:6801","nonce":2583575856}]},"active_addr":"172.21.15.26:6801/2583575856","active_change":"2024-05-20T20:45:35.597832+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.26:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":432920170}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1062974415}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1998339259}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3316882063}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3170793376}]}]} 2024-05-20T20:53:38.279 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-20T20:53:38.279 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-20T20:53:38.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:53:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:37 smithi144 ceph-mon[35660]: osdmap e80: 12 total, 12 up, 12 in 2024-05-20T20:53:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:37 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/617621125' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-20T20:53:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:38.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:38 smithi026 ceph-mon[31990]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 5 op/s; 13 B/s, 0 objects/s recovering 2024-05-20T20:53:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:38 smithi055 ceph-mon[35213]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 5 op/s; 13 B/s, 0 objects/s recovering 2024-05-20T20:53:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:38 smithi144 ceph-mon[35660]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 5 op/s; 13 B/s, 0 objects/s recovering 2024-05-20T20:53:39.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:39 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:53:40.165 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:39 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:53:40.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 -- 172.21.15.26:0/609117125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102430 msgr2=0x7f00241028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:40.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 --2- 172.21.15.26:0/609117125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102430 0x7f00241028b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0018009990 tx=0x7f0018009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 -- 172.21.15.26:0/609117125 shutdown_connections 2024-05-20T20:53:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 --2- 172.21.15.26:0/609117125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102df0 0x7f002410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 --2- 172.21.15.26:0/609117125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0024108510 0x7f00241088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 --2- 172.21.15.26:0/609117125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102430 0x7f00241028b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 -- 172.21.15.26:0/609117125 >> 172.21.15.26:0/609117125 conn(0x7f00240793b0 msgr2=0x7f00240797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 -- 172.21.15.26:0/609117125 shutdown_connections 2024-05-20T20:53:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.199+0000 7f002a789700 1 -- 172.21.15.26:0/609117125 wait complete. 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.200+0000 7f002a789700 1 Processor -- start 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.200+0000 7f002a789700 1 -- start start 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f002a789700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 0x7f002419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f002a789700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102df0 0x7f002419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f002a789700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0024108510 0x7f0024198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f002a789700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0024112490 con 0x7f0024102df0 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f002a789700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0024112310 con 0x7f0024108510 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f002a789700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0024112610 con 0x7f0024102430 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f00237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102df0 0x7f002419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 0x7f002419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 0x7f002419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:52994/0 (socket says 172.21.15.26:52994) 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 -- 172.21.15.26:0/4153413994 learned_addr learned my addr 172.21.15.26:0/4153413994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0028d26700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0024108510 0x7f0024198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 -- 172.21.15.26:0/4153413994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0024108510 msgr2=0x7f0024198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0024108510 0x7f0024198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 -- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102df0 msgr2=0x7f002419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102df0 0x7f002419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 -- 172.21.15.26:0/4153413994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0024198fd0 con 0x7f0024102430 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.201+0000 7f0023fff700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 0x7f002419e6b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0014007910 tx=0x7f0014007c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.202+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014004020 con 0x7f0024102430 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.202+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0014004830 con 0x7f0024102430 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.202+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014005480 con 0x7f0024102430 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.202+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00241992c0 con 0x7f0024102430 2024-05-20T20:53:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.202+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002406bab0 con 0x7f0024102430 2024-05-20T20:53:40.204 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:39 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:53:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.203+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0024103b50 con 0x7f0024102430 2024-05-20T20:53:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.205+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f001401f020 con 0x7f0024102430 2024-05-20T20:53:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.206+0000 7f00217fa700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f000c0b2910 0x7f000c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.207+0000 7f00237fe700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f000c0b2910 0x7f000c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.207+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f00140e3650 con 0x7f0024102430 2024-05-20T20:53:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.207+0000 7f00237fe700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f000c0b2910 0x7f000c0b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0024079af0 tx=0x7f0018005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.208+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00140ab190 con 0x7f0024102430 2024-05-20T20:53:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.439+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f002406c4e0 con 0x7f0024102430 2024-05-20T20:53:40.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.441+0000 7f00217fa700 1 -- 172.21.15.26:0/4153413994 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22698 (secure 0 0 0) 0x7f00140a6020 con 0x7f0024102430 2024-05-20T20:53:40.441 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:40.441 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:53:36.838831+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:53:40.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f000c0b2910 msgr2=0x7f000c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:40.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f000c0b2910 0x7f000c0b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0024079af0 tx=0x7f0018005fb0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 msgr2=0x7f002419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 0x7f002419e6b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0014007910 tx=0x7f0014007c20 comp rx=0 tx=0).stop 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 shutdown_connections 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0024108510 0x7f0024198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0024102430 0x7f002419e6b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f000c0b2910 0x7f000c0b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 --2- 172.21.15.26:0/4153413994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0024102df0 0x7f002419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 >> 172.21.15.26:0/4153413994 conn(0x7f00240793b0 msgr2=0x7f0024100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 shutdown_connections 2024-05-20T20:53:40.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:40.443+0000 7f002a789700 1 -- 172.21.15.26:0/4153413994 wait complete. 2024-05-20T20:53:41.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:40 smithi055 ceph-mon[35213]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 895 B/s wr, 7 op/s; 13 B/s, 0 objects/s recovering 2024-05-20T20:53:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:40 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4153413994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:41.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:40 smithi026 ceph-mon[31990]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 895 B/s wr, 7 op/s; 13 B/s, 0 objects/s recovering 2024-05-20T20:53:41.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:40 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4153413994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:40 smithi144 ceph-mon[35660]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 895 B/s wr, 7 op/s; 13 B/s, 0 objects/s recovering 2024-05-20T20:53:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:40 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4153413994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:41.309 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-20T20:53:41.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:53:41.796 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:41 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:41 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:42.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:41 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:43 smithi026 ceph-mon[31990]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 895 B/s wr, 5 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T20:53:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:43 smithi055 ceph-mon[35213]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 895 B/s wr, 5 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T20:53:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 -- 172.21.15.26:0/3613666555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 msgr2=0x7f388c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 --2- 172.21.15.26:0/3613666555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c1088e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f388800b0a0 tx=0x7f388800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 -- 172.21.15.26:0/3613666555 shutdown_connections 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 --2- 172.21.15.26:0/3613666555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c1088e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 --2- 172.21.15.26:0/3613666555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f388c103f90 0x7f388c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 --2- 172.21.15.26:0/3613666555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f388c1035f0 0x7f388c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 -- 172.21.15.26:0/3613666555 >> 172.21.15.26:0/3613666555 conn(0x7f388c07a6a0 msgr2=0x7f388c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 -- 172.21.15.26:0/3613666555 shutdown_connections 2024-05-20T20:53:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.451+0000 7f3893a60700 1 -- 172.21.15.26:0/3613666555 wait complete. 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 Processor -- start 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 -- start start 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f388c1035f0 0x7f388c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f388c103f90 0x7f388c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113690 con 0x7f388c108500 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113510 con 0x7f388c1035f0 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.452+0000 7f3893a60700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113810 con 0x7f388c103f90 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57596/0 (socket says 172.21.15.26:57596) 2024-05-20T20:53:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 -- 172.21.15.26:0/3379644215 learned_addr learned my addr 172.21.15.26:0/3379644215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f38917fc700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f388c1035f0 0x7f388c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3890ffb700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f388c103f90 0x7f388c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 -- 172.21.15.26:0/3379644215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f388c103f90 msgr2=0x7f388c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f388c103f90 0x7f388c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 -- 172.21.15.26:0/3379644215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f388c1035f0 msgr2=0x7f388c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f388c1035f0 0x7f388c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 -- 172.21.15.26:0/3379644215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c19a0f0 con 0x7f388c108500 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3891ffd700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c199990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f388400c970 tx=0x7f388400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884004a30 con 0x7f388c108500 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c19a380 con 0x7f388c108500 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38840054b0 con 0x7f388c108500 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884007a90 con 0x7f388c108500 2024-05-20T20:53:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.453+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a66a0 con 0x7f388c108500 2024-05-20T20:53:43.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.454+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c06a230 con 0x7f388c108500 2024-05-20T20:53:43.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.456+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3884007c50 con 0x7f388c108500 2024-05-20T20:53:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.457+0000 7f38827fc700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38780b2910 0x7f38780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.457+0000 7f38917fc700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38780b2910 0x7f38780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.457+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f38840e2200 con 0x7f388c108500 2024-05-20T20:53:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.458+0000 7f38917fc700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38780b2910 0x7f38780b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3888009fd0 tx=0x7f388800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.461+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3884014030 con 0x7f388c108500 2024-05-20T20:53:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:43 smithi144 ceph-mon[35660]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 895 B/s wr, 5 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T20:53:43.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.699+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f388c10bb30 con 0x7f388c108500 2024-05-20T20:53:43.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.701+0000 7f38827fc700 1 -- 172.21.15.26:0/3379644215 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22698 (secure 0 0 0) 0x7f3884005660 con 0x7f388c108500 2024-05-20T20:53:43.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:53:43.702 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:53:36.838831+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38780b2910 msgr2=0x7f38780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38780b2910 0x7f38780b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3888009fd0 tx=0x7f388800b7b0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 msgr2=0x7f388c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c199990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f388400c970 tx=0x7f388400cc80 comp rx=0 tx=0).stop 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 shutdown_connections 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38780b2910 0x7f38780b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388c108500 0x7f388c199990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f388c1035f0 0x7f388c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 --2- 172.21.15.26:0/3379644215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f388c103f90 0x7f388c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 >> 172.21.15.26:0/3379644215 conn(0x7f388c07a6a0 msgr2=0x7f388c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.704+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 shutdown_connections 2024-05-20T20:53:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:43.705+0000 7f3893a60700 1 -- 172.21.15.26:0/3379644215 wait complete. 2024-05-20T20:53:44.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-20T20:53:44.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-20T20:53:44.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-20T20:53:44.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-20T20:53:44.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-20T20:53:44.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-20T20:53:44.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-20T20:53:44.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-20T20:53:44.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-20T20:53:44.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-20T20:53:44.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-20T20:53:44.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-20T20:53:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:44 smithi026 ceph-mon[31990]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T20:53:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:44 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3379644215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:44.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:44.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:44.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:44 smithi144 ceph-mon[35660]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T20:53:44.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:44 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3379644215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:44.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:44.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:44.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:44 smithi055 ceph-mon[35213]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T20:53:44.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:44 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3379644215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:53:45.038 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:45.055 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:45.063 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:45.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:45 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:53:45.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:45 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:53:45.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:45 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:45.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:45 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:53:45.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:45 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:53:45.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:45 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:45.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:45 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:53:45.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:45 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:53:45.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:45 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:53:46.413 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:46.416 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:46.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:46 smithi026 ceph-mon[31990]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 1 objects/s recovering 2024-05-20T20:53:46.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:46 smithi144 ceph-mon[35660]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 1 objects/s recovering 2024-05-20T20:53:46.951 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:46 smithi055 ceph-mon[35213]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 1 objects/s recovering 2024-05-20T20:53:46.962 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:47.894 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:47.894 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:53:49.041 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:48 smithi055 ceph-mon[35213]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 10 B/s, 1 objects/s recovering 2024-05-20T20:53:49.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:48 smithi144 ceph-mon[35660]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 10 B/s, 1 objects/s recovering 2024-05-20T20:53:49.108 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:48 smithi026 ceph-mon[31990]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 10 B/s, 1 objects/s recovering 2024-05-20T20:53:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:50 smithi026 ceph-mon[31990]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T20:53:51.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:50 smithi144 ceph-mon[35660]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T20:53:51.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:50 smithi055 ceph-mon[35213]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T20:53:53.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:52 smithi144 ceph-mon[35660]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T20:53:53.093 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:52 smithi026 ceph-mon[31990]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T20:53:53.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:52 smithi055 ceph-mon[35213]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 -- 172.21.15.26:0/3898521050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908102440 msgr2=0x7f49081028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3898521050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908102440 0x7f49081028c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009990 tx=0x7f48f8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 -- 172.21.15.26:0/3898521050 shutdown_connections 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3898521050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4908102e00 0x7f490810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3898521050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4908108520 0x7f4908108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3898521050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908102440 0x7f49081028c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 -- 172.21.15.26:0/3898521050 >> 172.21.15.26:0/3898521050 conn(0x7f49080793f0 msgr2=0x7f4908079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 -- 172.21.15.26:0/3898521050 shutdown_connections 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.202+0000 7f490dfe3700 1 -- 172.21.15.26:0/3898521050 wait complete. 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 Processor -- start 2024-05-20T20:53:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 -- start start 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4908102440 0x7f4908104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4908102e00 0x7f49081045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908108520 0x7f4908104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49081124d0 con 0x7f4908108520 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112350 con 0x7f4908102440 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f490dfe3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112650 con 0x7f4908102e00 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f4907fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908108520 0x7f4908104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f4907fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908108520 0x7f4908104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40324/0 (socket says 172.21.15.26:40324) 2024-05-20T20:53:53.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f4907fff700 1 -- 172.21.15.26:0/3249821409 learned_addr learned my addr 172.21.15.26:0/3249821409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.203+0000 7f4906ffd700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4908102e00 0x7f49081045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f49077fe700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4908102440 0x7f4908104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4907fff700 1 -- 172.21.15.26:0/3249821409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4908102e00 msgr2=0x7f49081045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4907fff700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4908102e00 0x7f49081045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4907fff700 1 -- 172.21.15.26:0/3249821409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4908102440 msgr2=0x7f4908104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4907fff700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4908102440 0x7f4908104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4907fff700 1 -- 172.21.15.26:0/3249821409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49081a8150 con 0x7f4908108520 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f49077fe700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4908102440 0x7f4908104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:53:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4907fff700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908108520 0x7f4908104b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4900007ee0 tx=0x7f490000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490000dbd0 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f490000dd70 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49081a8440 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000058d0 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.204+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a8930 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.205+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4908063c50 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.207+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f490001a020 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f4904ff9700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48f00b2910 0x7f48f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f49077fe700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48f00b2910 0x7f48f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f49000e22b0 con 0x7f4908108520 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f4904ff9700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] conn(0x7f48f00bc550 0x7f48f00be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 --> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49000020b0 con 0x7f48f00bc550 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f49000e2700 con 0x7f4908108520 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f4906ffd700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] conn(0x7f48f00bc550 0x7f48f00be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.208+0000 7f49077fe700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48f00b2910 0x7f48f00b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005fd0 tx=0x7f48fc005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.209+0000 7f4906ffd700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] conn(0x7f48f00bc550 0x7f48f00be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.209+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== osd.0 v2:172.21.15.26:6802/3292543364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49000020b0 con 0x7f48f00bc550 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.228+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 --> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4908064050 con 0x7f48f00bc550 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.228+0000 7f4904ff9700 1 -- 172.21.15.26:0/3249821409 <== osd.0 v2:172.21.15.26:6802/3292543364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4908064050 con 0x7f48f00bc550 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] conn(0x7f48f00bc550 msgr2=0x7f48f00be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] conn(0x7f48f00bc550 0x7f48f00be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48f00b2910 msgr2=0x7f48f00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48f00b2910 0x7f48f00b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005fd0 tx=0x7f48fc005db0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908108520 msgr2=0x7f4908104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 --2- 172.21.15.26:0/3249821409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908108520 0x7f4908104b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4900007ee0 tx=0x7f490000d3b0 comp rx=0 tx=0).stop 2024-05-20T20:53:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f4907fff700 1 -- 172.21.15.26:0/3249821409 reap_dead start 2024-05-20T20:53:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 shutdown_connections 2024-05-20T20:53:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 >> 172.21.15.26:0/3249821409 conn(0x7f49080793f0 msgr2=0x7f490806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 shutdown_connections 2024-05-20T20:53:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:53.229+0000 7f490dfe3700 1 -- 172.21.15.26:0/3249821409 wait complete. 2024-05-20T20:53:53.915 INFO:teuthology.orchestra.run.smithi026.stdout:34359738452 2024-05-20T20:53:53.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-20T20:53:55.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:54 smithi026 ceph-mon[31990]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:53:55.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:54 smithi144 ceph-mon[35660]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:53:55.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:54 smithi055 ceph-mon[35213]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:53:56.858 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:56 smithi026 ceph-mon[31990]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:53:57.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:56 smithi144 ceph-mon[35660]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:53:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 -- 172.21.15.26:0/3618425785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 msgr2=0x7ff9a8112470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3618425785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a8112470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b0a0 tx=0x7ff9a000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 -- 172.21.15.26:0/3618425785 shutdown_connections 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3618425785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a8112470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3618425785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a806bc50 0x7ff9a806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3618425785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a806c570 0x7ff9a810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 -- 172.21.15.26:0/3618425785 >> 172.21.15.26:0/3618425785 conn(0x7ff9a8078490 msgr2=0x7ff9a80788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 -- 172.21.15.26:0/3618425785 shutdown_connections 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.943+0000 7ff9ae384700 1 -- 172.21.15.26:0/3618425785 wait complete. 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 Processor -- start 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 -- start start 2024-05-20T20:53:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a806bc50 0x7ff9a81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a806c570 0x7ff9a81a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81157f0 con 0x7ff9a8110080 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8115670 con 0x7ff9a806bc50 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ae384700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8115970 con 0x7ff9a806c570 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ac921700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ac921700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40334/0 (socket says 172.21.15.26:40334) 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9ac921700 1 -- 172.21.15.26:0/3543898738 learned_addr learned my addr 172.21.15.26:0/3543898738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9a77fe700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a806c570 0x7ff9a81a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.944+0000 7ff9a7fff700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a806bc50 0x7ff9a81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ac921700 1 -- 172.21.15.26:0/3543898738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a806c570 msgr2=0x7ff9a81a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ac921700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a806c570 0x7ff9a81a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ac921700 1 -- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a806bc50 msgr2=0x7ff9a81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ac921700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a806bc50 0x7ff9a81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ac921700 1 -- 172.21.15.26:0/3543898738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a819c450 con 0x7ff9a8110080 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ac921700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a819bb70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b830 tx=0x7ff9a0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0021070 con 0x7ff9a8110080 2024-05-20T20:53:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9a0007c80 con 0x7ff9a8110080 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a819c6e0 con 0x7ff9a8110080 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00052a0 con 0x7ff9a8110080 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.945+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a81a89d0 con 0x7ff9a8110080 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.946+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9a8063c90 con 0x7ff9a8110080 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.947+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff9a0024070 con 0x7ff9a8110080 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a57fa700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9940b2990 0x7ff9940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a7fff700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9940b2990 0x7ff9940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7ff9a000f020 con 0x7ff9a8110080 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a57fa700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] conn(0x7ff9940bc5d0 0x7ff9940bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 --> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9940bf140 con 0x7ff9940bc5d0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7ff9a00e3fc0 con 0x7ff9a8110080 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.948+0000 7ff9a7fff700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9940b2990 0x7ff9940b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff998005fd0 tx=0x7ff99800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.949+0000 7ff9a77fe700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] conn(0x7ff9940bc5d0 0x7ff9940bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.949+0000 7ff9a77fe700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] conn(0x7ff9940bc5d0 0x7ff9940bea70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.950+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== osd.4 v2:172.21.15.55:6808/1113585138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9940bf140 con 0x7ff9940bc5d0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.967+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 --> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9a8064090 con 0x7ff9940bc5d0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.967+0000 7ff9a57fa700 1 -- 172.21.15.26:0/3543898738 <== osd.4 v2:172.21.15.55:6808/1113585138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9a8064090 con 0x7ff9940bc5d0 2024-05-20T20:53:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.967+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] conn(0x7ff9940bc5d0 msgr2=0x7ff9940bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.967+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] conn(0x7ff9940bc5d0 0x7ff9940bea70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9940b2990 msgr2=0x7ff9940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9940b2990 0x7ff9940b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff998005fd0 tx=0x7ff99800a560 comp rx=0 tx=0).stop 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 msgr2=0x7ff9a819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 --2- 172.21.15.26:0/3543898738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9a8110080 0x7ff9a819bb70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b830 tx=0x7ff9a0004db0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ac921700 1 -- 172.21.15.26:0/3543898738 reap_dead start 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 shutdown_connections 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 >> 172.21.15.26:0/3543898738 conn(0x7ff9a8078490 msgr2=0x7ff9a8111840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 shutdown_connections 2024-05-20T20:53:57.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:54.968+0000 7ff9ae384700 1 -- 172.21.15.26:0/3543898738 wait complete. 2024-05-20T20:53:57.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:56 smithi055 ceph-mon[35213]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 -- 172.21.15.26:0/718910840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c107000 msgr2=0x7f0a3c10eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 --2- 172.21.15.26:0/718910840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c107000 0x7f0a3c10eb40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009990 tx=0x7f0a34009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 -- 172.21.15.26:0/718910840 shutdown_connections 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 --2- 172.21.15.26:0/718910840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c107000 0x7f0a3c10eb40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 --2- 172.21.15.26:0/718910840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a3c102160 0x7f0a3c102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 --2- 172.21.15.26:0/718910840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a3c102b10 0x7f0a3c1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 -- 172.21.15.26:0/718910840 >> 172.21.15.26:0/718910840 conn(0x7f0a3c077780 msgr2=0x7f0a3c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:57.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 -- 172.21.15.26:0/718910840 shutdown_connections 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.487+0000 7f0a4379e700 1 -- 172.21.15.26:0/718910840 wait complete. 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 Processor -- start 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 -- start start 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a3c102160 0x7f0a3c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c102b10 0x7f0a3c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a3c107000 0x7f0a3c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1114c0 con 0x7f0a3c102b10 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c111340 con 0x7f0a3c107000 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.488+0000 7f0a4379e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c111640 con 0x7f0a3c102160 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c102b10 0x7f0a3c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c102b10 0x7f0a3c19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40376/0 (socket says 172.21.15.26:40376) 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 -- 172.21.15.26:0/1333342578 learned_addr learned my addr 172.21.15.26:0/1333342578 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:57.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a41d3b700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a3c107000 0x7f0a3c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a4153a700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a3c102160 0x7f0a3c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 -- 172.21.15.26:0/1333342578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a3c102160 msgr2=0x7f0a3c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a3c102160 0x7f0a3c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 -- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a3c107000 msgr2=0x7f0a3c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a3c107000 0x7f0a3c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 -- 172.21.15.26:0/1333342578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a0d50 con 0x7f0a3c102b10 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a40d39700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c102b10 0x7f0a3c19c280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000c970 tx=0x7f0a3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004a30 con 0x7f0a3c102b10 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a300054b0 con 0x7f0a3c102b10 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a1040 con 0x7f0a3c102b10 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3000eb40 con 0x7f0a3c102b10 2024-05-20T20:53:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.489+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1a88c0 con 0x7f0a3c102b10 2024-05-20T20:53:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.490+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a3c100880 con 0x7f0a3c102b10 2024-05-20T20:53:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.493+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0a3000ed00 con 0x7f0a3c102b10 2024-05-20T20:53:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.494+0000 7f0a3a7fc700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a240b2910 0x7f0a240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a4153a700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a240b2910 0x7f0a240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f0a300e28a0 con 0x7f0a3c102b10 2024-05-20T20:53:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a3a7fc700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] conn(0x7f0a240bc550 0x7f0a240be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 --> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a3001de70 con 0x7f0a240bc550 2024-05-20T20:53:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f0a300e2cf0 con 0x7f0a3c102b10 2024-05-20T20:53:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a4153a700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a240b2910 0x7f0a240b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c007610 tx=0x7f0a2c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a41d3b700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] conn(0x7f0a240bc550 0x7f0a240be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.495+0000 7f0a41d3b700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] conn(0x7f0a240bc550 0x7f0a240be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.496+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== osd.1 v2:172.21.15.55:6800/1106200301 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a3001de70 con 0x7f0a240bc550 2024-05-20T20:53:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.513+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 --> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a3c103340 con 0x7f0a240bc550 2024-05-20T20:53:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.514+0000 7f0a3a7fc700 1 -- 172.21.15.26:0/1333342578 <== osd.1 v2:172.21.15.55:6800/1106200301 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a3c103340 con 0x7f0a240bc550 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.514+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] conn(0x7f0a240bc550 msgr2=0x7f0a240be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.514+0000 7f0a4379e700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] conn(0x7f0a240bc550 0x7f0a240be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a240b2910 msgr2=0x7f0a240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a240b2910 0x7f0a240b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c007610 tx=0x7f0a2c00c040 comp rx=0 tx=0).stop 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c102b10 msgr2=0x7f0a3c19c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 --2- 172.21.15.26:0/1333342578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a3c102b10 0x7f0a3c19c280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000c970 tx=0x7f0a3000cc80 comp rx=0 tx=0).stop 2024-05-20T20:53:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a41d3b700 1 -- 172.21.15.26:0/1333342578 reap_dead start 2024-05-20T20:53:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 shutdown_connections 2024-05-20T20:53:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 >> 172.21.15.26:0/1333342578 conn(0x7f0a3c077780 msgr2=0x7f0a3c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 shutdown_connections 2024-05-20T20:53:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:57.515+0000 7f0a4379e700 1 -- 172.21.15.26:0/1333342578 wait complete. 2024-05-20T20:53:58.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 -- 172.21.15.26:0/267563113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4108510 msgr2=0x7facd41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:58.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 --2- 172.21.15.26:0/267563113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4108510 0x7facd41088f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7facd0009990 tx=0x7facd0009ca0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 -- 172.21.15.26:0/267563113 shutdown_connections 2024-05-20T20:53:58.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 --2- 172.21.15.26:0/267563113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4108510 0x7facd41088f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 --2- 172.21.15.26:0/267563113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd4102df0 0x7facd410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 --2- 172.21.15.26:0/267563113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facd4102430 0x7facd41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 -- 172.21.15.26:0/267563113 >> 172.21.15.26:0/267563113 conn(0x7facd40793b0 msgr2=0x7facd40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 -- 172.21.15.26:0/267563113 shutdown_connections 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.800+0000 7facdc5d6700 1 -- 172.21.15.26:0/267563113 wait complete. 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 Processor -- start 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 -- start start 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facd4102430 0x7facd419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4102df0 0x7facd419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd4108510 0x7facd41a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facd4112470 con 0x7facd4102df0 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7facd41122f0 con 0x7facd4108510 2024-05-20T20:53:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdc5d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facd41125f0 con 0x7facd4102430 2024-05-20T20:53:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4102df0 0x7facd419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4102df0 0x7facd419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40354/0 (socket says 172.21.15.26:40354) 2024-05-20T20:53:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 -- 172.21.15.26:0/3639255584 learned_addr learned my addr 172.21.15.26:0/3639255584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:53:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facdab73700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd4108510 0x7facd41a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facda372700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facd4102430 0x7facd419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 -- 172.21.15.26:0/3639255584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facd4102430 msgr2=0x7facd419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facd4102430 0x7facd419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 -- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd4108510 msgr2=0x7facd41a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd4108510 0x7facd41a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.801+0000 7facd9b71700 1 -- 172.21.15.26:0/3639255584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41a5ed0 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7facda372700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facd4102430 0x7facd419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7facd9b71700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4102df0 0x7facd419ae50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7facc400c9b0 tx=0x7facc400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4007b00 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41a6100 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41a6690 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facc4007ca0 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.802+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4018660 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.803+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facd4063c50 con 0x7facd4102df0 2024-05-20T20:53:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.804+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7facc400f450 con 0x7facd4102df0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.805+0000 7faccb7fe700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7facc00b2910 0x7facc00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.805+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7facc40e2810 con 0x7facd4102df0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.805+0000 7facda372700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7facc00b2910 0x7facc00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.805+0000 7faccb7fe700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] conn(0x7facc00bc550 0x7facc00be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.805+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 --> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facc4010e80 con 0x7facc00bc550 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.805+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7facc40e2c60 con 0x7facd4102df0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.806+0000 7facdab73700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] conn(0x7facc00bc550 0x7facc00be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.806+0000 7facda372700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7facc00b2910 0x7facc00b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7facd000b5c0 tx=0x7facd001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.806+0000 7facdab73700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] conn(0x7facc00bc550 0x7facc00be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.807+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== osd.7 v2:172.21.15.55:6816/1141160889 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facc4010e80 con 0x7facc00bc550 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.824+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 --> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7facd4063e60 con 0x7facc00bc550 2024-05-20T20:53:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7faccb7fe700 1 -- 172.21.15.26:0/3639255584 <== osd.7 v2:172.21.15.55:6816/1141160889 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7facd4063e60 con 0x7facc00bc550 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] conn(0x7facc00bc550 msgr2=0x7facc00be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] conn(0x7facc00bc550 0x7facc00be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7facc00b2910 msgr2=0x7facc00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7facc00b2910 0x7facc00b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7facd000b5c0 tx=0x7facd001a040 comp rx=0 tx=0).stop 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4102df0 msgr2=0x7facd419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 --2- 172.21.15.26:0/3639255584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4102df0 0x7facd419ae50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7facc400c9b0 tx=0x7facc400ccc0 comp rx=0 tx=0).stop 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdab73700 1 -- 172.21.15.26:0/3639255584 reap_dead start 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 shutdown_connections 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 >> 172.21.15.26:0/3639255584 conn(0x7facd40793b0 msgr2=0x7facd41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 shutdown_connections 2024-05-20T20:53:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:53:56.825+0000 7facdc5d6700 1 -- 172.21.15.26:0/3639255584 wait complete. 2024-05-20T20:53:58.992 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:53:58 smithi026 ceph-mon[31990]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:53:59.027 INFO:teuthology.orchestra.run.smithi026.stdout:201863462962 2024-05-20T20:53:59.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-20T20:53:59.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:53:58 smithi144 ceph-mon[35660]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:53:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:53:58 smithi055 ceph-mon[35213]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:53:59.942 INFO:teuthology.orchestra.run.smithi026.stdout:133143986241 2024-05-20T20:53:59.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-20T20:54:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:55834574928 2024-05-20T20:54:00.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-20T20:54:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 -- 172.21.15.26:0/4036205354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 msgr2=0x7f26dc1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 --2- 172.21.15.26:0/4036205354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1103f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b0a0 tx=0x7f26d400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 -- 172.21.15.26:0/4036205354 shutdown_connections 2024-05-20T20:54:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 --2- 172.21.15.26:0/4036205354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1103f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 --2- 172.21.15.26:0/4036205354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26dc101fd0 0x7f26dc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 --2- 172.21.15.26:0/4036205354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26dc102980 0x7f26dc108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 -- 172.21.15.26:0/4036205354 >> 172.21.15.26:0/4036205354 conn(0x7f26dc0776f0 msgr2=0x7f26dc077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 -- 172.21.15.26:0/4036205354 shutdown_connections 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.936+0000 7f26e0803700 1 -- 172.21.15.26:0/4036205354 wait complete. 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.937+0000 7f26e0803700 1 Processor -- start 2024-05-20T20:54:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.937+0000 7f26e0803700 1 -- start start 2024-05-20T20:54:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26e0803700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26dc101fd0 0x7f26dc102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26e0803700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26dc102980 0x7f26dc103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26e0803700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26e0803700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc113550 con 0x7f26dc108840 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26e0803700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1133d0 con 0x7f26dc101fd0 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26e0803700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1136d0 con 0x7f26dc102980 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26dc101fd0 0x7f26dc102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47500/0 (socket says 172.21.15.26:47500) 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 -- 172.21.15.26:0/3527949711 learned_addr learned my addr 172.21.15.26:0/3527949711 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26da59c700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26dc102980 0x7f26dc103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 -- 172.21.15.26:0/3527949711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26dc102980 msgr2=0x7f26dc103490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26dc102980 0x7f26dc103490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 -- 172.21.15.26:0/3527949711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26dc101fd0 msgr2=0x7f26dc102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26dc101fd0 0x7f26dc102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26db59e700 1 -- 172.21.15.26:0/3527949711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc1a7f10 con 0x7f26dc108840 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.938+0000 7f26dad9d700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26dc101fd0 0x7f26dc102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.939+0000 7f26db59e700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1039d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b830 tx=0x7f26d4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.939+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4021070 con 0x7f26dc108840 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.939+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc1a81a0 con 0x7f26dc108840 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.939+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a85f0 con 0x7f26dc108840 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.939+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26d4007b80 con 0x7f26dc108840 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.939+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d40052a0 con 0x7f26dc108840 2024-05-20T20:54:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.940+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26c8001030 con 0x7f26dc108840 2024-05-20T20:54:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.941+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f26d4024070 con 0x7f26dc108840 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.942+0000 7f26c3fff700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26c40b2990 0x7f26c40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.942+0000 7f26dad9d700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26c40b2990 0x7f26c40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.942+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f26d40e3890 con 0x7f26dc108840 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.942+0000 7f26c3fff700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] conn(0x7f26c40bc5d0 0x7f26c40bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.942+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 --> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26c40bf140 con 0x7f26c40bc5d0 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.942+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f26d40e3ce0 con 0x7f26dc108840 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.943+0000 7f26da59c700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] conn(0x7f26c40bc5d0 0x7f26c40bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.943+0000 7f26dad9d700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26c40b2990 0x7f26c40b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26cc007fd0 tx=0x7f26cc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.943+0000 7f26da59c700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] conn(0x7f26c40bc5d0 0x7f26c40bea70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.944+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== osd.6 v2:172.21.15.26:6818/1035586360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26c40bf140 con 0x7f26c40bc5d0 2024-05-20T20:54:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.963+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 --> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26c8002e60 con 0x7f26c40bc5d0 2024-05-20T20:54:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.963+0000 7f26c3fff700 1 -- 172.21.15.26:0/3527949711 <== osd.6 v2:172.21.15.26:6818/1035586360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f26c8002e60 con 0x7f26c40bc5d0 2024-05-20T20:54:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] conn(0x7f26c40bc5d0 msgr2=0x7f26c40bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] conn(0x7f26c40bc5d0 0x7f26c40bea70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26c40b2990 msgr2=0x7f26c40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26c40b2990 0x7f26c40b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26cc007fd0 tx=0x7f26cc0073d0 comp rx=0 tx=0).stop 2024-05-20T20:54:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 msgr2=0x7f26dc1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 --2- 172.21.15.26:0/3527949711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc108840 0x7f26dc1039d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b830 tx=0x7f26d4004d00 comp rx=0 tx=0).stop 2024-05-20T20:54:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26db59e700 1 -- 172.21.15.26:0/3527949711 reap_dead start 2024-05-20T20:54:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 shutdown_connections 2024-05-20T20:54:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 >> 172.21.15.26:0/3527949711 conn(0x7f26dc0776f0 msgr2=0x7f26dc0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 shutdown_connections 2024-05-20T20:54:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:00.964+0000 7f26e0803700 1 -- 172.21.15.26:0/3527949711 wait complete. 2024-05-20T20:54:01.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:00 smithi026 ceph-mon[31990]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:01.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:00 smithi144 ceph-mon[35660]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:01.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:00 smithi055 ceph-mon[35213]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 -- 172.21.15.26:0/109796932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4102e00 msgr2=0x7f71b410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 --2- 172.21.15.26:0/109796932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4102e00 0x7f71b410f2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b0a0 tx=0x7f71ac00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 -- 172.21.15.26:0/109796932 shutdown_connections 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 --2- 172.21.15.26:0/109796932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4102e00 0x7f71b410f2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 --2- 172.21.15.26:0/109796932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71b4108520 0x7f71b4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 --2- 172.21.15.26:0/109796932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b4102440 0x7f71b41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 -- 172.21.15.26:0/109796932 >> 172.21.15.26:0/109796932 conn(0x7f71b40793f0 msgr2=0x7f71b4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 -- 172.21.15.26:0/109796932 shutdown_connections 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.868+0000 7f71bb993700 1 -- 172.21.15.26:0/109796932 wait complete. 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 Processor -- start 2024-05-20T20:54:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 -- start start 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71b4102440 0x7f71b419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b4102e00 0x7f71b419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4108520 0x7f71b41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4112480 con 0x7f71b4108520 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4112300 con 0x7f71b4102e00 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71bb993700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4112600 con 0x7f71b4102440 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4108520 0x7f71b41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4108520 0x7f71b41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47514/0 (socket says 172.21.15.26:47514) 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 -- 172.21.15.26:0/3583453372 learned_addr learned my addr 172.21.15.26:0/3583453372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b8f2e700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b4102e00 0x7f71b419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b972f700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71b4102440 0x7f71b419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 -- 172.21.15.26:0/3583453372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71b4102440 msgr2=0x7f71b419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71b4102440 0x7f71b419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 -- 172.21.15.26:0/3583453372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b4102e00 msgr2=0x7f71b419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b4102e00 0x7f71b419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.869+0000 7f71b9f30700 1 -- 172.21.15.26:0/3583453372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419a080 con 0x7f71b4108520 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.870+0000 7f71b9f30700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4108520 0x7f71b41998a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b830 tx=0x7f71ac004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.870+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac021070 con 0x7f71b4108520 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.870+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71ac007b80 con 0x7f71b4108520 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.870+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b419a310 con 0x7f71b4108520 2024-05-20T20:54:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.870+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0052a0 con 0x7f71b4108520 2024-05-20T20:54:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.870+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a6530 con 0x7f71b4108520 2024-05-20T20:54:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.871+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71b4063cd0 con 0x7f71b4108520 2024-05-20T20:54:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.872+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f71ac024070 con 0x7f71b4108520 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.873+0000 7f71aa7fc700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71a00b2910 0x7f71a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.873+0000 7f71b972f700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71a00b2910 0x7f71a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.873+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f71ac0e3930 con 0x7f71b4108520 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.874+0000 7f71aa7fc700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] conn(0x7f71a00bc550 0x7f71a00be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.874+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 --> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71a00bf0c0 con 0x7f71a00bc550 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.874+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f71ac0e3d80 con 0x7f71b4108520 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.874+0000 7f71b8f2e700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] conn(0x7f71a00bc550 0x7f71a00be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:01.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.874+0000 7f71b972f700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71a00b2910 0x7f71a00b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005fd0 tx=0x7f71b000af20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:01.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.874+0000 7f71b8f2e700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] conn(0x7f71a00bc550 0x7f71a00be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:01.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.875+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== osd.9 v2:172.21.15.26:6826/2188316601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71a00bf0c0 con 0x7f71a00bc550 2024-05-20T20:54:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.893+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 --> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71b4064150 con 0x7f71a00bc550 2024-05-20T20:54:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.893+0000 7f71aa7fc700 1 -- 172.21.15.26:0/3583453372 <== osd.9 v2:172.21.15.26:6826/2188316601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71b4064150 con 0x7f71a00bc550 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] conn(0x7f71a00bc550 msgr2=0x7f71a00be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] conn(0x7f71a00bc550 0x7f71a00be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71a00b2910 msgr2=0x7f71a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71a00b2910 0x7f71a00b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005fd0 tx=0x7f71b000af20 comp rx=0 tx=0).stop 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4108520 msgr2=0x7f71b41998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 --2- 172.21.15.26:0/3583453372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b4108520 0x7f71b41998a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b830 tx=0x7f71ac004d00 comp rx=0 tx=0).stop 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71b9f30700 1 -- 172.21.15.26:0/3583453372 reap_dead start 2024-05-20T20:54:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 shutdown_connections 2024-05-20T20:54:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 >> 172.21.15.26:0/3583453372 conn(0x7f71b40793f0 msgr2=0x7f71b406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 shutdown_connections 2024-05-20T20:54:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:01.894+0000 7f71bb993700 1 -- 172.21.15.26:0/3583453372 wait complete. 2024-05-20T20:54:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 -- 172.21.15.26:0/30059590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102440 msgr2=0x7f90c01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 --2- 172.21.15.26:0/30059590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102440 0x7f90c01028c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009990 tx=0x7f90b4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 -- 172.21.15.26:0/30059590 shutdown_connections 2024-05-20T20:54:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 --2- 172.21.15.26:0/30059590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90c0102e00 0x7f90c010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 --2- 172.21.15.26:0/30059590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c0108520 0x7f90c0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 --2- 172.21.15.26:0/30059590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102440 0x7f90c01028c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 -- 172.21.15.26:0/30059590 >> 172.21.15.26:0/30059590 conn(0x7f90c00793f0 msgr2=0x7f90c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 -- 172.21.15.26:0/30059590 shutdown_connections 2024-05-20T20:54:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.554+0000 7f90c6304700 1 -- 172.21.15.26:0/30059590 wait complete. 2024-05-20T20:54:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 Processor -- start 2024-05-20T20:54:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 -- start start 2024-05-20T20:54:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c0102440 0x7f90c019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102e00 0x7f90c019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90c0108520 0x7f90c01999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0113610 con 0x7f90c0102e00 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0113490 con 0x7f90c0108520 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.555+0000 7f90c6304700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0113790 con 0x7f90c0102440 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102e00 0x7f90c019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102e00 0x7f90c019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47536/0 (socket says 172.21.15.26:47536) 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 -- 172.21.15.26:0/3073897956 learned_addr learned my addr 172.21.15.26:0/3073897956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bffff700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c0102440 0x7f90c019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90c48a1700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90c0108520 0x7f90c01999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 -- 172.21.15.26:0/3073897956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c0102440 msgr2=0x7f90c019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c0102440 0x7f90c019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 -- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90c0108520 msgr2=0x7f90c01999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90c0108520 0x7f90c01999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 -- 172.21.15.26:0/3073897956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c019a110 con 0x7f90c0102e00 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bf7fe700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102e00 0x7f90c019fd30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f90b4000c00 tx=0x7f90b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4027070 con 0x7f90c0102e00 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90b400fb00 con 0x7f90c0102e00 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c019a3a0 con 0x7f90c0102e00 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4022c40 con 0x7f90c0102e00 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.556+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a66c0 con 0x7f90c0102e00 2024-05-20T20:54:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.557+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90c0063cd0 con 0x7f90c0102e00 2024-05-20T20:54:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.559+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f90b402a070 con 0x7f90c0102e00 2024-05-20T20:54:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bd7fa700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90a80b2990 0x7f90a80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bffff700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90a80b2990 0x7f90a80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f90b40e3f70 con 0x7f90c0102e00 2024-05-20T20:54:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bd7fa700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] conn(0x7f90a80bc5d0 0x7f90a80bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 --> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90b400b0b0 con 0x7f90a80bc5d0 2024-05-20T20:54:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f90b40e4320 con 0x7f90c0102e00 2024-05-20T20:54:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90bffff700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90a80b2990 0x7f90a80b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90c019aaf0 tx=0x7f90b000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.560+0000 7f90c48a1700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] conn(0x7f90a80bc5d0 0x7f90a80bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.561+0000 7f90c48a1700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] conn(0x7f90a80bc5d0 0x7f90a80bea70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:02.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.562+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== osd.10 v2:172.21.15.55:6824/46093047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90b400b0b0 con 0x7f90a80bc5d0 2024-05-20T20:54:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.579+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 --> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90c00640d0 con 0x7f90a80bc5d0 2024-05-20T20:54:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.579+0000 7f90bd7fa700 1 -- 172.21.15.26:0/3073897956 <== osd.10 v2:172.21.15.55:6824/46093047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f90c00640d0 con 0x7f90a80bc5d0 2024-05-20T20:54:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] conn(0x7f90a80bc5d0 msgr2=0x7f90a80bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] conn(0x7f90a80bc5d0 0x7f90a80bea70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90a80b2990 msgr2=0x7f90a80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90a80b2990 0x7f90a80b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90c019aaf0 tx=0x7f90b000b3c0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102e00 msgr2=0x7f90c019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 --2- 172.21.15.26:0/3073897956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c0102e00 0x7f90c019fd30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f90b4000c00 tx=0x7f90b40047b0 comp rx=0 tx=0).stop 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c48a1700 1 -- 172.21.15.26:0/3073897956 reap_dead start 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 shutdown_connections 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 >> 172.21.15.26:0/3073897956 conn(0x7f90c00793f0 msgr2=0x7f90c0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 shutdown_connections 2024-05-20T20:54:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:02.580+0000 7f90c6304700 1 -- 172.21.15.26:0/3073897956 wait complete. 2024-05-20T20:54:02.992 INFO:teuthology.orchestra.run.smithi026.stdout:180388626488 2024-05-20T20:54:02.993 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-20T20:54:03.020 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:02 smithi026 ceph-mon[31990]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-20T20:54:03.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:02 smithi144 ceph-mon[35660]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-20T20:54:03.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:02 smithi055 ceph-mon[35213]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-20T20:54:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 -- 172.21.15.26:0/1528476815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102470 msgr2=0x7f37cc1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 --2- 172.21.15.26:0/1528476815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102470 0x7f37cc1028f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009990 tx=0x7f37b4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 -- 172.21.15.26:0/1528476815 shutdown_connections 2024-05-20T20:54:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 --2- 172.21.15.26:0/1528476815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc102e30 0x7f37cc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 --2- 172.21.15.26:0/1528476815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc1084b0 0x7f37cc108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 --2- 172.21.15.26:0/1528476815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102470 0x7f37cc1028f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 -- 172.21.15.26:0/1528476815 >> 172.21.15.26:0/1528476815 conn(0x7f37cc0784b0 msgr2=0x7f37cc0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 -- 172.21.15.26:0/1528476815 shutdown_connections 2024-05-20T20:54:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.803+0000 7f37d19aa700 1 -- 172.21.15.26:0/1528476815 wait complete. 2024-05-20T20:54:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 Processor -- start 2024-05-20T20:54:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 -- start start 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc102470 0x7f37cc104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102e30 0x7f37cc1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc1084b0 0x7f37cc104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc112490 con 0x7f37cc102e30 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc112310 con 0x7f37cc102470 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37d19aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc112610 con 0x7f37cc1084b0 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.804+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102e30 0x7f37cc1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102e30 0x7f37cc1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47550/0 (socket says 172.21.15.26:47550) 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 -- 172.21.15.26:0/2000667415 learned_addr learned my addr 172.21.15.26:0/2000667415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37caffd700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc102470 0x7f37cc104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37cb7fe700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc1084b0 0x7f37cc104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 -- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc1084b0 msgr2=0x7f37cc104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37cc1084b0 0x7f37cc104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 -- 172.21.15.26:0/2000667415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc102470 msgr2=0x7f37cc104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37cc102470 0x7f37cc104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 -- 172.21.15.26:0/2000667415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1a80f0 con 0x7f37cc102e30 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37ca7fc700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102e30 0x7f37cc1045c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000c00 tx=0x7f37b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4027070 con 0x7f37cc102e30 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b400fb80 con 0x7f37cc102e30 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1a8380 con 0x7f37cc102e30 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4022d00 con 0x7f37cc102e30 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.805+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1a87d0 con 0x7f37cc102e30 2024-05-20T20:54:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.806+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37cc063c50 con 0x7f37cc102e30 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.808+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f37b402a070 con 0x7f37cc102e30 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37d09a8700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b80b2910 0x7f37b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37caffd700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b80b2910 0x7f37b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f37b40e38e0 con 0x7f37cc102e30 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37d09a8700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] conn(0x7f37b80bc550 0x7f37b80be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 --> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37b40095d0 con 0x7f37b80bc550 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f37b40e3d30 con 0x7f37cc102e30 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37caffd700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b80b2910 0x7f37b80b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37bc003f10 tx=0x7f37bc009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.809+0000 7f37cb7fe700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] conn(0x7f37b80bc550 0x7f37b80be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.810+0000 7f37cb7fe700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] conn(0x7f37b80bc550 0x7f37b80be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:03.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.811+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== osd.5 v2:172.21.15.144:6808/1448183794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37b40095d0 con 0x7f37b80bc550 2024-05-20T20:54:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.828+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 --> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37cc064050 con 0x7f37b80bc550 2024-05-20T20:54:03.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d09a8700 1 -- 172.21.15.26:0/2000667415 <== osd.5 v2:172.21.15.144:6808/1448183794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37cc064050 con 0x7f37b80bc550 2024-05-20T20:54:03.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] conn(0x7f37b80bc550 msgr2=0x7f37b80be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] conn(0x7f37b80bc550 0x7f37b80be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b80b2910 msgr2=0x7f37b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37b80b2910 0x7f37b80b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37bc003f10 tx=0x7f37bc009180 comp rx=0 tx=0).stop 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102e30 msgr2=0x7f37cc1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 --2- 172.21.15.26:0/2000667415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37cc102e30 0x7f37cc1045c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000c00 tx=0x7f37b40047b0 comp rx=0 tx=0).stop 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37cb7fe700 1 -- 172.21.15.26:0/2000667415 reap_dead start 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 shutdown_connections 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.829+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 >> 172.21.15.26:0/2000667415 conn(0x7f37cc0784b0 msgr2=0x7f37cc06c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.830+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 shutdown_connections 2024-05-20T20:54:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:03.830+0000 7f37d19aa700 1 -- 172.21.15.26:0/2000667415 wait complete. 2024-05-20T20:54:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 -- 172.21.15.26:0/827802282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 msgr2=0x7f04b4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/827802282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b4102e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f04b000b0d0 tx=0x7f04b000b3e0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 -- 172.21.15.26:0/827802282 shutdown_connections 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/827802282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b4102e60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/827802282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b41077d0 0x7f04b410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/827802282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04b4103430 0x7f04b41070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 -- 172.21.15.26:0/827802282 >> 172.21.15.26:0/827802282 conn(0x7f04b40793f0 msgr2=0x7f04b4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 -- 172.21.15.26:0/827802282 shutdown_connections 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.742+0000 7f04bc4d9700 1 -- 172.21.15.26:0/827802282 wait complete. 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 Processor -- start 2024-05-20T20:54:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 -- start start 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b4103430 0x7f04b419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04b41077d0 0x7f04b4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4112510 con 0x7f04b4102a80 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4112390 con 0x7f04b41077d0 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04bc4d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4112690 con 0x7f04b4103430 2024-05-20T20:54:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04ba275700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04ba275700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47574/0 (socket says 172.21.15.26:47574) 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04ba275700 1 -- 172.21.15.26:0/1439488187 learned_addr learned my addr 172.21.15.26:0/1439488187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04b9a74700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b4103430 0x7f04b419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.743+0000 7f04baa76700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04b41077d0 0x7f04b4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ba275700 1 -- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b4103430 msgr2=0x7f04b419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ba275700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b4103430 0x7f04b419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ba275700 1 -- 172.21.15.26:0/1439488187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04b41077d0 msgr2=0x7f04b4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ba275700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04b41077d0 0x7f04b4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ba275700 1 -- 172.21.15.26:0/1439488187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b419a210 con 0x7f04b4102a80 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ba275700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b419f8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f04b0007a20 tx=0x7f04b0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b001f070 con 0x7f04b4102a80 2024-05-20T20:54:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04b0004c50 con 0x7f04b4102a80 2024-05-20T20:54:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b419a4a0 con 0x7f04b4102a80 2024-05-20T20:54:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b001b690 con 0x7f04b4102a80 2024-05-20T20:54:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.744+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b41a67c0 con 0x7f04b4102a80 2024-05-20T20:54:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.745+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04b4063bd0 con 0x7f04b4102a80 2024-05-20T20:54:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.747+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f04b0022070 con 0x7f04b4102a80 2024-05-20T20:54:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.747+0000 7f04ab7fe700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04a00b2910 0x7f04a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04b9a74700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04a00b2910 0x7f04a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f04b000f020 con 0x7f04b4102a80 2024-05-20T20:54:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04ab7fe700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] conn(0x7f04a00bc550 0x7f04a00be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 --> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04a00bf0c0 con 0x7f04a00bc550 2024-05-20T20:54:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f04b00e3bd0 con 0x7f04b4102a80 2024-05-20T20:54:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04b9a74700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04a00b2910 0x7f04a00b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f04b41073b0 tx=0x7f04a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.748+0000 7f04baa76700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] conn(0x7f04a00bc550 0x7f04a00be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.749+0000 7f04baa76700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] conn(0x7f04a00bc550 0x7f04a00be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.750+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== osd.2 v2:172.21.15.144:6800/632346919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04a00bf0c0 con 0x7f04a00bc550 2024-05-20T20:54:04.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.768+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 --> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04b4064080 con 0x7f04a00bc550 2024-05-20T20:54:04.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.768+0000 7f04ab7fe700 1 -- 172.21.15.26:0/1439488187 <== osd.2 v2:172.21.15.144:6800/632346919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f04b4064080 con 0x7f04a00bc550 2024-05-20T20:54:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] conn(0x7f04a00bc550 msgr2=0x7f04a00be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] conn(0x7f04a00bc550 0x7f04a00be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04a00b2910 msgr2=0x7f04a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04a00b2910 0x7f04a00b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f04b41073b0 tx=0x7f04a4009260 comp rx=0 tx=0).stop 2024-05-20T20:54:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 msgr2=0x7f04b419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 --2- 172.21.15.26:0/1439488187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4102a80 0x7f04b419f8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f04b0007a20 tx=0x7f04b0007a50 comp rx=0 tx=0).stop 2024-05-20T20:54:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04baa76700 1 -- 172.21.15.26:0/1439488187 reap_dead start 2024-05-20T20:54:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 shutdown_connections 2024-05-20T20:54:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 >> 172.21.15.26:0/1439488187 conn(0x7f04b40793f0 msgr2=0x7f04b406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 shutdown_connections 2024-05-20T20:54:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:04.769+0000 7f04bc4d9700 1 -- 172.21.15.26:0/1439488187 wait complete. 2024-05-20T20:54:04.991 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:04 smithi026 ceph-mon[31990]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-20T20:54:05.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:04 smithi144 ceph-mon[35660]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-20T20:54:05.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:04 smithi055 ceph-mon[35213]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 -- 172.21.15.26:0/1748375556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188107460 msgr2=0x7f0188107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 --2- 172.21.15.26:0/1748375556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188107460 0x7f0188107840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0178009990 tx=0x7f0178009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 -- 172.21.15.26:0/1748375556 shutdown_connections 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 --2- 172.21.15.26:0/1748375556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188107460 0x7f0188107840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 --2- 172.21.15.26:0/1748375556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01881030e0 0x7f018810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 --2- 172.21.15.26:0/1748375556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0188102720 0x7f0188102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 -- 172.21.15.26:0/1748375556 >> 172.21.15.26:0/1748375556 conn(0x7f0188079300 msgr2=0x7f0188079710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 -- 172.21.15.26:0/1748375556 shutdown_connections 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 -- 172.21.15.26:0/1748375556 wait complete. 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 Processor -- start 2024-05-20T20:54:05.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.491+0000 7f018e3db700 1 -- start start 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188102720 0x7f018819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01881030e0 0x7f018819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0188107460 0x7f0188199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01881125f0 con 0x7f0188102720 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0188112470 con 0x7f0188107460 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0188112770 con 0x7f01881030e0 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188102720 0x7f018819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188102720 0x7f018819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47590/0 (socket says 172.21.15.26:47590) 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 -- 172.21.15.26:0/91234547 learned_addr learned my addr 172.21.15.26:0/91234547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f017ffff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01881030e0 0x7f018819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 -- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01881030e0 msgr2=0x7f018819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:05.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018c978700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0188107460 0x7f0188199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01881030e0 0x7f018819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 -- 172.21.15.26:0/91234547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0188107460 msgr2=0x7f0188199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0188107460 0x7f0188199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 -- 172.21.15.26:0/91234547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018819a0d0 con 0x7f0188102720 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f017ffff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01881030e0 0x7f018819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018c978700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0188107460 0x7f0188199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0187fff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188102720 0x7f018819f7b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01780057c0 tx=0x7f0178009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178027070 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.492+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018819a360 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.493+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f017800fda0 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.493+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178022ca0 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.493+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01881a6680 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.493+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0188063c50 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.495+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f017802a070 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.495+0000 7f0185ffb700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f01740b2910 0x7f01740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f017ffff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f01740b2910 0x7f01740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f01780e3640 con 0x7f0188102720 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f0185ffb700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] conn(0x7f01740bc550 0x7f01740be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 --> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f017802ee80 con 0x7f01740bc550 2024-05-20T20:54:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f01780e3a90 con 0x7f0188102720 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f017ffff700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f01740b2910 0x7f01740b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0170005fd0 tx=0x7f0170005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.496+0000 7f018c978700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] conn(0x7f01740bc550 0x7f01740be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.497+0000 7f018c978700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] conn(0x7f01740bc550 0x7f01740be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.498+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== osd.8 v2:172.21.15.144:6816/2473562796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f017802ee80 con 0x7f01740bc550 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.514+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 --> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0188064050 con 0x7f01740bc550 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f0185ffb700 1 -- 172.21.15.26:0/91234547 <== osd.8 v2:172.21.15.144:6816/2473562796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0188064050 con 0x7f01740bc550 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] conn(0x7f01740bc550 msgr2=0x7f01740be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] conn(0x7f01740bc550 0x7f01740be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f01740b2910 msgr2=0x7f01740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f01740b2910 0x7f01740b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0170005fd0 tx=0x7f0170005e20 comp rx=0 tx=0).stop 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188102720 msgr2=0x7f018819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 --2- 172.21.15.26:0/91234547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188102720 0x7f018819f7b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01780057c0 tx=0x7f0178009f90 comp rx=0 tx=0).stop 2024-05-20T20:54:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018c978700 1 -- 172.21.15.26:0/91234547 reap_dead start 2024-05-20T20:54:05.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 shutdown_connections 2024-05-20T20:54:05.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 >> 172.21.15.26:0/91234547 conn(0x7f0188079300 msgr2=0x7f018806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:05.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 shutdown_connections 2024-05-20T20:54:05.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:05.515+0000 7f018e3db700 1 -- 172.21.15.26:0/91234547 wait complete. 2024-05-20T20:54:06.319 INFO:teuthology.orchestra.run.smithi026.stdout:253403070505 2024-05-20T20:54:06.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-20T20:54:06.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:06 smithi026 ceph-mon[31990]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:06.949 INFO:teuthology.orchestra.run.smithi026.stdout:274877906980 2024-05-20T20:54:06.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-20T20:54:07.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:06 smithi144 ceph-mon[35660]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 -- 172.21.15.26:0/48437512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102430 msgr2=0x7f73a01028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 --2- 172.21.15.26:0/48437512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102430 0x7f73a01028b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7394009990 tx=0x7f7394009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 -- 172.21.15.26:0/48437512 shutdown_connections 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 --2- 172.21.15.26:0/48437512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a0102df0 0x7f73a010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 --2- 172.21.15.26:0/48437512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0108510 0x7f73a01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 --2- 172.21.15.26:0/48437512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102430 0x7f73a01028b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 -- 172.21.15.26:0/48437512 >> 172.21.15.26:0/48437512 conn(0x7f73a00793b0 msgr2=0x7f73a00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 -- 172.21.15.26:0/48437512 shutdown_connections 2024-05-20T20:54:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.143+0000 7f73a5e53700 1 -- 172.21.15.26:0/48437512 wait complete. 2024-05-20T20:54:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.144+0000 7f73a5e53700 1 Processor -- start 2024-05-20T20:54:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 -- start start 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0102430 0x7f73a019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102df0 0x7f73a019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a0108510 0x7f73a01a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01124e0 con 0x7f73a0102df0 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f73a0112360 con 0x7f73a0108510 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f73a5e53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73a0112660 con 0x7f73a0102430 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f739effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102df0 0x7f73a019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f739ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a0108510 0x7f73a01a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f739effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102df0 0x7f73a019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47608/0 (socket says 172.21.15.26:47608) 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f739effd700 1 -- 172.21.15.26:0/2229553185 learned_addr learned my addr 172.21.15.26:0/2229553185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.145+0000 7f739f7fe700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0102430 0x7f73a019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739effd700 1 -- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0102430 msgr2=0x7f73a019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739effd700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0102430 0x7f73a019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739effd700 1 -- 172.21.15.26:0/2229553185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a0108510 msgr2=0x7f73a01a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739effd700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a0108510 0x7f73a01a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739effd700 1 -- 172.21.15.26:0/2229553185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a01a5ea0 con 0x7f73a0102df0 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739ffff700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a0108510 0x7f73a01a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739effd700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102df0 0x7f73a019ae20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7394005910 tx=0x7f73940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394027070 con 0x7f73a0102df0 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a01a60d0 con 0x7f73a0102df0 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a01a6580 con 0x7f73a0102df0 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f739400fb00 con 0x7f73a0102df0 2024-05-20T20:54:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.146+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394022b90 con 0x7f73a0102df0 2024-05-20T20:54:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.147+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f738c001060 con 0x7f73a0102df0 2024-05-20T20:54:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.149+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f739402a070 con 0x7f73a0102df0 2024-05-20T20:54:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739cff9700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73880b2990 0x7f73880b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739f7fe700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73880b2990 0x7f73880b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f73940e3ae0 con 0x7f73a0102df0 2024-05-20T20:54:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739cff9700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] conn(0x7f73880bc5d0 0x7f73880bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 --> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73880bf140 con 0x7f73880bc5d0 2024-05-20T20:54:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f73940e3f30 con 0x7f73a0102df0 2024-05-20T20:54:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739f7fe700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73880b2990 0x7f73880b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73900045b0 tx=0x7f739000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.150+0000 7f739ffff700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] conn(0x7f73880bc5d0 0x7f73880bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.151+0000 7f739ffff700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] conn(0x7f73880bc5d0 0x7f73880bea70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.152+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== osd.11 v2:172.21.15.144:6824/2410307143 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73880bf140 con 0x7f73880bc5d0 2024-05-20T20:54:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.169+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 --> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f738c002e90 con 0x7f73880bc5d0 2024-05-20T20:54:07.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.170+0000 7f739cff9700 1 -- 172.21.15.26:0/2229553185 <== osd.11 v2:172.21.15.144:6824/2410307143 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f738c002e90 con 0x7f73880bc5d0 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.170+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] conn(0x7f73880bc5d0 msgr2=0x7f73880bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.170+0000 7f73a5e53700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] conn(0x7f73880bc5d0 0x7f73880bea70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.170+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73880b2990 msgr2=0x7f73880b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.170+0000 7f73a5e53700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73880b2990 0x7f73880b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73900045b0 tx=0x7f739000b3c0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102df0 msgr2=0x7f73a019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f73a5e53700 1 --2- 172.21.15.26:0/2229553185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a0102df0 0x7f73a019ae20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7394005910 tx=0x7f73940047b0 comp rx=0 tx=0).stop 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f739ffff700 1 -- 172.21.15.26:0/2229553185 reap_dead start 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 shutdown_connections 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 >> 172.21.15.26:0/2229553185 conn(0x7f73a00793b0 msgr2=0x7f73a01004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 shutdown_connections 2024-05-20T20:54:07.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:07.171+0000 7f73a5e53700 1 -- 172.21.15.26:0/2229553185 wait complete. 2024-05-20T20:54:07.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:06 smithi055 ceph-mon[35213]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:07.497 INFO:teuthology.orchestra.run.smithi026.stdout:154618822718 2024-05-20T20:54:07.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-20T20:54:07.840 INFO:teuthology.orchestra.run.smithi026.stdout:77309411405 2024-05-20T20:54:07.841 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-20T20:54:08.792 INFO:teuthology.orchestra.run.smithi026.stdout:300647710751 2024-05-20T20:54:08.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-20T20:54:09.026 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:08 smithi026 ceph-mon[31990]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T20:54:09.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:08 smithi144 ceph-mon[35660]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T20:54:09.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:08 smithi055 ceph-mon[35213]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T20:54:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.258+0000 7f7866a2e700 1 -- 172.21.15.26:0/1232715781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 msgr2=0x7f78601028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.258+0000 7f7866a2e700 1 --2- 172.21.15.26:0/1232715781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f78601028c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7854009990 tx=0x7f7854009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 -- 172.21.15.26:0/1232715781 shutdown_connections 2024-05-20T20:54:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 --2- 172.21.15.26:0/1232715781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7860102e00 0x7f786010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 --2- 172.21.15.26:0/1232715781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860108520 0x7f7860108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 --2- 172.21.15.26:0/1232715781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f78601028c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 -- 172.21.15.26:0/1232715781 >> 172.21.15.26:0/1232715781 conn(0x7f78600793f0 msgr2=0x7f7860079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 -- 172.21.15.26:0/1232715781 shutdown_connections 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 -- 172.21.15.26:0/1232715781 wait complete. 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 Processor -- start 2024-05-20T20:54:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.259+0000 7f7866a2e700 1 -- start start 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7866a2e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f7860104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7866a2e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860102e00 0x7f78601045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7866a2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7860108520 0x7f7860104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7866a2e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7860112480 con 0x7f7860102440 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7866a2e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7860112300 con 0x7f7860102e00 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7866a2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7860112600 con 0x7f7860108520 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f7860104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f7860104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47630/0 (socket says 172.21.15.26:47630) 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 -- 172.21.15.26:0/3362419260 learned_addr learned my addr 172.21.15.26:0/3362419260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785f7fe700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860102e00 0x7f78601045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7864fcb700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7860108520 0x7f7860104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 -- 172.21.15.26:0/3362419260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7860108520 msgr2=0x7f7860104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7860108520 0x7f7860104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 -- 172.21.15.26:0/3362419260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860102e00 msgr2=0x7f78601045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860102e00 0x7f78601045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 -- 172.21.15.26:0/3362419260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78601a7fd0 con 0x7f7860102440 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f7864fcb700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7860108520 0x7f7860104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785ffff700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f7860104080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f785000d950 tx=0x7f785000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.260+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850016070 con 0x7f7860102440 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.261+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78601a82c0 con 0x7f7860102440 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.261+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7850009dd0 con 0x7f7860102440 2024-05-20T20:54:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.261+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000f3c0 con 0x7f7860102440 2024-05-20T20:54:09.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.261+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78601a87d0 con 0x7f7860102440 2024-05-20T20:54:09.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.261+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7860063c50 con 0x7f7860102440 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.263+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7850019070 con 0x7f7860102440 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785d7fa700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78480b2910 0x7f78480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785f7fe700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78480b2910 0x7f78480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f78500e24c0 con 0x7f7860102440 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785d7fa700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] conn(0x7f78480bc550 0x7f78480be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 --> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78480bf0c0 con 0x7f78480bc550 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f78500e2910 con 0x7f7860102440 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f7864fcb700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] conn(0x7f78480bc550 0x7f78480be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f785f7fe700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78480b2910 0x7f78480b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f78540098e0 tx=0x7f7854005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:09.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.264+0000 7f7864fcb700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] conn(0x7f78480bc550 0x7f78480be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:09.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.265+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== osd.3 v2:172.21.15.26:6810/2074985040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78480bf0c0 con 0x7f78480bc550 2024-05-20T20:54:09.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 --> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78600640d0 con 0x7f78480bc550 2024-05-20T20:54:09.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f785d7fa700 1 -- 172.21.15.26:0/3362419260 <== osd.3 v2:172.21.15.26:6810/2074985040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f78600640d0 con 0x7f78480bc550 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] conn(0x7f78480bc550 msgr2=0x7f78480be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] conn(0x7f78480bc550 0x7f78480be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78480b2910 msgr2=0x7f78480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78480b2910 0x7f78480b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f78540098e0 tx=0x7f7854005960 comp rx=0 tx=0).stop 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 msgr2=0x7f7860104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7866a2e700 1 --2- 172.21.15.26:0/3362419260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7860102440 0x7f7860104080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f785000d950 tx=0x7f785000dc60 comp rx=0 tx=0).stop 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.282+0000 7f7864fcb700 1 -- 172.21.15.26:0/3362419260 reap_dead start 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.283+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 shutdown_connections 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.283+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 >> 172.21.15.26:0/3362419260 conn(0x7f78600793f0 msgr2=0x7f786006b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:09.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.283+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 shutdown_connections 2024-05-20T20:54:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:09.283+0000 7f7866a2e700 1 -- 172.21.15.26:0/3362419260 wait complete. 2024-05-20T20:54:09.510 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:09.511 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:107374182472 2024-05-20T20:54:10.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-20T20:54:10.458 INFO:teuthology.orchestra.run.smithi026.stdout:227633266734 2024-05-20T20:54:10.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-20T20:54:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:10.682 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:10 smithi026 ceph-mon[31990]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:11.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:10 smithi144 ceph-mon[35660]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:11.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:10 smithi055 ceph-mon[35213]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:13.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:12 smithi144 ceph-mon[35660]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:13.068 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:12 smithi026 ceph-mon[31990]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:13.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:12 smithi055 ceph-mon[35213]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:14.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:14 smithi026 ceph-mon[31990]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-20T20:54:15.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:14 smithi144 ceph-mon[35660]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-20T20:54:15.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:14 smithi055 ceph-mon[35213]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-20T20:54:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 -- 172.21.15.26:0/1222861192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102470 msgr2=0x7f684c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 --2- 172.21.15.26:0/1222861192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102470 0x7f684c1028f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6834009990 tx=0x7f6834009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 -- 172.21.15.26:0/1222861192 shutdown_connections 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 --2- 172.21.15.26:0/1222861192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c102e30 0x7f684c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 --2- 172.21.15.26:0/1222861192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f684c1084b0 0x7f684c108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 --2- 172.21.15.26:0/1222861192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102470 0x7f684c1028f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 -- 172.21.15.26:0/1222861192 >> 172.21.15.26:0/1222861192 conn(0x7f684c0784b0 msgr2=0x7f684c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 -- 172.21.15.26:0/1222861192 shutdown_connections 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.801+0000 7f6852a27700 1 -- 172.21.15.26:0/1222861192 wait complete. 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 Processor -- start 2024-05-20T20:54:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 -- start start 2024-05-20T20:54:15.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f684c102470 0x7f684c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 0x7f684c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c1084b0 0x7f684c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1124f0 con 0x7f684c102e30 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f684c112370 con 0x7f684c102470 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6852a27700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f684c112670 con 0x7f684c1084b0 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 0x7f684c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 0x7f684c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49354/0 (socket says 172.21.15.26:49354) 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 -- 172.21.15.26:0/3536653360 learned_addr learned my addr 172.21.15.26:0/3536653360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f684bfff700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f684c102470 0x7f684c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6850fc4700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c1084b0 0x7f684c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 -- 172.21.15.26:0/3536653360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c1084b0 msgr2=0x7f684c199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c1084b0 0x7f684c199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 -- 172.21.15.26:0/3536653360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f684c102470 msgr2=0x7f684c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f684c102470 0x7f684c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6843fff700 1 -- 172.21.15.26:0/3536653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c19a1c0 con 0x7f684c102e30 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.802+0000 7f6850fc4700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c1084b0 0x7f684c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.803+0000 7f6843fff700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 0x7f684c19fd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6834005910 tx=0x7f68340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.803+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834004830 con 0x7f684c102e30 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.803+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6834005020 con 0x7f684c102e30 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.803+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c19a450 con 0x7f684c102e30 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.803+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683400f8b0 con 0x7f684c102e30 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.803+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c1a6720 con 0x7f684c102e30 2024-05-20T20:54:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.804+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c103b60 con 0x7f684c102e30 2024-05-20T20:54:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.805+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f68340054d0 con 0x7f684c102e30 2024-05-20T20:54:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.806+0000 7f6849ffb700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68380b2990 0x7f68380b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.806+0000 7f684bfff700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68380b2990 0x7f68380b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.806+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f68340e35e0 con 0x7f684c102e30 2024-05-20T20:54:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.807+0000 7f684bfff700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68380b2990 0x7f68380b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f684c1079d0 tx=0x7f683c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.810+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6834013070 con 0x7f684c102e30 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.996+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/3030962560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48102e00 msgr2=0x7f3c4810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.996+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/3030962560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48102e00 0x7f3c4810f2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b0a0 tx=0x7f3c4000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/3030962560 shutdown_connections 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/3030962560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48102e00 0x7f3c4810f2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/3030962560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48108520 0x7f3c48108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/3030962560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c48102440 0x7f3c481028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/3030962560 >> 172.21.15.26:0/3030962560 conn(0x7f3c480793f0 msgr2=0x7f3c48079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/3030962560 shutdown_connections 2024-05-20T20:54:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/3030962560 wait complete. 2024-05-20T20:54:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 Processor -- start 2024-05-20T20:54:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.997+0000 7f3c4e4c8700 1 -- start start 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4e4c8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48102440 0x7f3c48104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4e4c8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c48102e00 0x7f3c481045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4e4c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 0x7f3c48104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4e4c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112480 con 0x7f3c48108520 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4e4c8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112300 con 0x7f3c48102e00 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4e4c8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112600 con 0x7f3c48102440 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 0x7f3c48104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 0x7f3c48104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49368/0 (socket says 172.21.15.26:49368) 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 -- 172.21.15.26:0/807030577 learned_addr learned my addr 172.21.15.26:0/807030577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c3ffff700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c48102e00 0x7f3c481045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c47fff700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48102440 0x7f3c48104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 -- 172.21.15.26:0/807030577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48102440 msgr2=0x7f3c48104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48102440 0x7f3c48104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 -- 172.21.15.26:0/807030577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c48102e00 msgr2=0x7f3c481045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c48102e00 0x7f3c481045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 -- 172.21.15.26:0/807030577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c481a7fc0 con 0x7f3c48108520 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c47fff700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48102440 0x7f3c48104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c4ca65700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 0x7f3c48104b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b830 tx=0x7f3c40004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.998+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40021070 con 0x7f3c48108520 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.999+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481a8250 con 0x7f3c48108520 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.999+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c40007b80 con 0x7f3c48108520 2024-05-20T20:54:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.999+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400052a0 con 0x7f3c48108520 2024-05-20T20:54:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:15.999+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481a86a0 con 0x7f3c48108520 2024-05-20T20:54:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.000+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4806a230 con 0x7f3c48108520 2024-05-20T20:54:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.001+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3c40024070 con 0x7f3c48108520 2024-05-20T20:54:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.002+0000 7f3c45ffb700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3c340b2910 0x7f3c340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.002+0000 7f3c47fff700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3c340b2910 0x7f3c340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.002+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f3c4000f020 con 0x7f3c48108520 2024-05-20T20:54:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.002+0000 7f3c47fff700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3c340b2910 0x7f3c340b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005fd0 tx=0x7f3c38005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.005+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c4002b020 con 0x7f3c48108520 2024-05-20T20:54:16.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.039+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f684c02e560 con 0x7f684c102e30 2024-05-20T20:54:16.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.040+0000 7f6849ffb700 1 -- 172.21.15.26:0/3536653360 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f68340ab1b0 con 0x7f684c102e30 2024-05-20T20:54:16.039 INFO:teuthology.orchestra.run.smithi026.stdout:201863462966 2024-05-20T20:54:16.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68380b2990 msgr2=0x7f68380b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:16.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68380b2990 0x7f68380b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f684c1079d0 tx=0x7f683c0073d0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 msgr2=0x7f684c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 0x7f684c19fd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6834005910 tx=0x7f68340047b0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 shutdown_connections 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c1084b0 0x7f684c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f684c102470 0x7f684c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68380b2990 0x7f68380b4e50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 --2- 172.21.15.26:0/3536653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c102e30 0x7f684c19fd90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 >> 172.21.15.26:0/3536653360 conn(0x7f684c0784b0 msgr2=0x7f684c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 shutdown_connections 2024-05-20T20:54:16.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.042+0000 7f6852a27700 1 -- 172.21.15.26:0/3536653360 wait complete. 2024-05-20T20:54:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.232+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3c4802e7c0 con 0x7f3c48108520 2024-05-20T20:54:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.232+0000 7f3c45ffb700 1 -- 172.21.15.26:0/807030577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3c400aba20 con 0x7f3c48108520 2024-05-20T20:54:16.231 INFO:teuthology.orchestra.run.smithi026.stdout:133143986245 2024-05-20T20:54:16.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3c340b2910 msgr2=0x7f3c340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:16.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3c340b2910 0x7f3c340b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005fd0 tx=0x7f3c38005db0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 msgr2=0x7f3c48104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:16.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 0x7f3c48104b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b830 tx=0x7f3c40004d00 comp rx=0 tx=0).stop 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 shutdown_connections 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3c340b2910 0x7f3c340b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c48108520 0x7f3c48104b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c48102440 0x7f3c48104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 --2- 172.21.15.26:0/807030577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c48102e00 0x7f3c481045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 >> 172.21.15.26:0/807030577 conn(0x7f3c480793f0 msgr2=0x7f3c4806b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 shutdown_connections 2024-05-20T20:54:16.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:16.234+0000 7f3c4e4c8700 1 -- 172.21.15.26:0/807030577 wait complete. 2024-05-20T20:54:16.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:16 smithi026 ceph-mon[31990]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-20T20:54:16.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:16 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3536653360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-20T20:54:16.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:16 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/807030577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-20T20:54:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:16 smithi144 ceph-mon[35660]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-20T20:54:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:16 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3536653360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-20T20:54:17.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:16 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/807030577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-20T20:54:17.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:16 smithi055 ceph-mon[35213]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-20T20:54:17.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:16 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3536653360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-20T20:54:17.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:16 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/807030577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-20T20:54:18.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462966 for osd.7 2024-05-20T20:54:18.427 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:19.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:18 smithi144 ceph-mon[35660]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:19.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:18 smithi026 ceph-mon[31990]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:18 smithi055 ceph-mon[35213]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:19.865 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986241 got 133143986245 for osd.4 2024-05-20T20:54:19.865 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/2615223781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db406c470 msgr2=0x7f7db41103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/2615223781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db406c470 0x7f7db41103c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b0a0 tx=0x7f7dac00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/2615223781 shutdown_connections 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/2615223781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db406c470 0x7f7db41103c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/2615223781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db41095a0 0x7f7db4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/2615223781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7db406bab0 0x7f7db406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/2615223781 >> 172.21.15.26:0/2615223781 conn(0x7f7db40793b0 msgr2=0x7f7db40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/2615223781 shutdown_connections 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.062+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/2615223781 wait complete. 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 Processor -- start 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 -- start start 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db406bab0 0x7f7db419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7db406c470 0x7f7db419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 0x7f7db4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4113560 con 0x7f7db41095a0 2024-05-20T20:54:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41133e0 con 0x7f7db406c470 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbc5a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41136e0 con 0x7f7db406bab0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbab44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 0x7f7db4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbab44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 0x7f7db4199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47062/0 (socket says 172.21.15.26:47062) 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.063+0000 7f7dbab44700 1 -- 172.21.15.26:0/211666293 learned_addr learned my addr 172.21.15.26:0/211666293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dba343700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db406bab0 0x7f7db419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7db9b42700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7db406c470 0x7f7db419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbab44700 1 -- 172.21.15.26:0/211666293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db406bab0 msgr2=0x7f7db419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbab44700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db406bab0 0x7f7db419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbab44700 1 -- 172.21.15.26:0/211666293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7db406c470 msgr2=0x7f7db419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbab44700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7db406c470 0x7f7db419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbab44700 1 -- 172.21.15.26:0/211666293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db419a0b0 con 0x7f7db41095a0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbab44700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 0x7f7db4199950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b780 tx=0x7f7dac004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac021070 con 0x7f7db41095a0 2024-05-20T20:54:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dac007b80 con 0x7f7db41095a0 2024-05-20T20:54:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db419a340 con 0x7f7db41095a0 2024-05-20T20:54:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0052a0 con 0x7f7db41095a0 2024-05-20T20:54:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.064+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db41a6660 con 0x7f7db41095a0 2024-05-20T20:54:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.065+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db406a230 con 0x7f7db41095a0 2024-05-20T20:54:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.067+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7dac024070 con 0x7f7db41095a0 2024-05-20T20:54:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.068+0000 7f7dab7fe700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7da00b2910 0x7f7da00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.068+0000 7f7dba343700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7da00b2910 0x7f7da00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.068+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f7dac00f020 con 0x7f7db41095a0 2024-05-20T20:54:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.068+0000 7f7dba343700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7da00b2910 0x7f7da00b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7db419aa90 tx=0x7f7db0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:20.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.072+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7dac02b020 con 0x7f7db41095a0 2024-05-20T20:54:20.072 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.292+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7db402e560 con 0x7f7db41095a0 2024-05-20T20:54:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.293+0000 7f7dab7fe700 1 -- 172.21.15.26:0/211666293 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7dac0aba40 con 0x7f7db41095a0 2024-05-20T20:54:20.292 INFO:teuthology.orchestra.run.smithi026.stdout:274877906984 2024-05-20T20:54:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7da00b2910 msgr2=0x7f7da00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7da00b2910 0x7f7da00b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7db419aa90 tx=0x7f7db0009260 comp rx=0 tx=0).stop 2024-05-20T20:54:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 msgr2=0x7f7db4199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 0x7f7db4199950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b780 tx=0x7f7dac004d00 comp rx=0 tx=0).stop 2024-05-20T20:54:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 shutdown_connections 2024-05-20T20:54:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7da00b2910 0x7f7da00b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7db41095a0 0x7f7db4199950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db406bab0 0x7f7db419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 --2- 172.21.15.26:0/211666293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7db406c470 0x7f7db419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 >> 172.21.15.26:0/211666293 conn(0x7f7db40793b0 msgr2=0x7f7db41012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 shutdown_connections 2024-05-20T20:54:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:20.295+0000 7f7dbc5a7700 1 -- 172.21.15.26:0/211666293 wait complete. 2024-05-20T20:54:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:20 smithi026 ceph-mon[31990]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:20 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/211666293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-20T20:54:21.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:20 smithi144 ceph-mon[35660]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:21.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:20 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/211666293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-20T20:54:21.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:20 smithi055 ceph-mon[35213]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:21.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:20 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/211666293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-20T20:54:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 -- 172.21.15.26:0/2913295476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 msgr2=0x7f58e41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 --2- 172.21.15.26:0/2913295476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e41028c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009990 tx=0x7f58d8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 -- 172.21.15.26:0/2913295476 shutdown_connections 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 --2- 172.21.15.26:0/2913295476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e4102e00 0x7f58e410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 --2- 172.21.15.26:0/2913295476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4108520 0x7f58e4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 --2- 172.21.15.26:0/2913295476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e41028c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 -- 172.21.15.26:0/2913295476 >> 172.21.15.26:0/2913295476 conn(0x7f58e40793f0 msgr2=0x7f58e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.863+0000 7f58e95a5700 1 -- 172.21.15.26:0/2913295476 shutdown_connections 2024-05-20T20:54:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.864+0000 7f58e95a5700 1 -- 172.21.15.26:0/2913295476 wait complete. 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.864+0000 7f58e95a5700 1 Processor -- start 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.864+0000 7f58e95a5700 1 -- start start 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e95a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e95a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4102e00 0x7f58e419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e95a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e4108520 0x7f58e41a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e95a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41124a0 con 0x7f58e4102440 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e95a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4112320 con 0x7f58e4102e00 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e95a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4112620 con 0x7f58e4108520 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47086/0 (socket says 172.21.15.26:47086) 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 -- 172.21.15.26:0/1032196924 learned_addr learned my addr 172.21.15.26:0/1032196924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e27fc700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4102e00 0x7f58e419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e37fe700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e4108520 0x7f58e41a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 -- 172.21.15.26:0/1032196924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e4108520 msgr2=0x7f58e41a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e4108520 0x7f58e41a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 -- 172.21.15.26:0/1032196924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4102e00 msgr2=0x7f58e419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4102e00 0x7f58e419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 -- 172.21.15.26:0/1032196924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41a5f90 con 0x7f58e4102440 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e27fc700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4102e00 0x7f58e419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.865+0000 7f58e2ffd700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e419a9d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c920 tx=0x7f58d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.866+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4007b00 con 0x7f58e4102440 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.866+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e41a61c0 con 0x7f58e4102440 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.866+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58d4004cc0 con 0x7f58e4102440 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.866+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40054a0 con 0x7f58e4102440 2024-05-20T20:54:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.866+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a6700 con 0x7f58e4102440 2024-05-20T20:54:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.867+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e406a230 con 0x7f58e4102440 2024-05-20T20:54:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.868+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f58d4004750 con 0x7f58e4102440 2024-05-20T20:54:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.869+0000 7f58cbfff700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58cc0b2910 0x7f58cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.869+0000 7f58e27fc700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58cc0b2910 0x7f58cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.869+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f58d40e23e0 con 0x7f58e4102440 2024-05-20T20:54:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.869+0000 7f58e27fc700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58cc0b2910 0x7f58cc0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58d8006010 tx=0x7f58d801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:21.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:21.872+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58d4014030 con 0x7f58e4102440 2024-05-20T20:54:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.096+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f58e402e720 con 0x7f58e4102440 2024-05-20T20:54:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.096+0000 7f58cbfff700 1 -- 172.21.15.26:0/1032196924 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58d40a9f20 con 0x7f58e4102440 2024-05-20T20:54:22.096 INFO:teuthology.orchestra.run.smithi026.stdout:55834574933 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58cc0b2910 msgr2=0x7f58cc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58cc0b2910 0x7f58cc0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58d8006010 tx=0x7f58d801a040 comp rx=0 tx=0).stop 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 msgr2=0x7f58e419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e419a9d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c920 tx=0x7f58d400cce0 comp rx=0 tx=0).stop 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 shutdown_connections 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f58cc0b2910 0x7f58cc0b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e4102440 0x7f58e419a9d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e4108520 0x7f58e41a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 --2- 172.21.15.26:0/1032196924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58e4102e00 0x7f58e419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 >> 172.21.15.26:0/1032196924 conn(0x7f58e40793f0 msgr2=0x7f58e4100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 shutdown_connections 2024-05-20T20:54:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:22.099+0000 7f58e95a5700 1 -- 172.21.15.26:0/1032196924 wait complete. 2024-05-20T20:54:22.715 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906980 got 274877906984 for osd.10 2024-05-20T20:54:22.715 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:22.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:22 smithi026 ceph-mon[31990]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:22.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:22 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1032196924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-20T20:54:23.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:22 smithi144 ceph-mon[35660]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:23.050 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:22 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1032196924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-20T20:54:23.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:22 smithi055 ceph-mon[35213]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:23.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:22 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1032196924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-20T20:54:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 -- 172.21.15.26:0/2566414371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98108910 msgr2=0x7f1f98110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/2566414371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98108910 0x7f1f98110490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000b0d0 tx=0x7f1f9000b3e0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 -- 172.21.15.26:0/2566414371 shutdown_connections 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/2566414371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98108910 0x7f1f98110490 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/2566414371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f98103820 0x7f1f98103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/2566414371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f981041d0 0x7f1f981081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 -- 172.21.15.26:0/2566414371 >> 172.21.15.26:0/2566414371 conn(0x7f1f9807a6b0 msgr2=0x7f1f9807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 -- 172.21.15.26:0/2566414371 shutdown_connections 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 -- 172.21.15.26:0/2566414371 wait complete. 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 Processor -- start 2024-05-20T20:54:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.352+0000 7f1fa02f2700 1 -- start start 2024-05-20T20:54:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1fa02f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 0x7f1f98075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1fa02f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981041d0 0x7f1f98075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1fa02f2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f98108910 0x7f1f980762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1fa02f2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981135b0 con 0x7f1f98103820 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1fa02f2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98113430 con 0x7f1f98108910 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1fa02f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98113730 con 0x7f1f981041d0 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 0x7f1f98075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 0x7f1f98075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47112/0 (socket says 172.21.15.26:47112) 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 -- 172.21.15.26:0/754494129 learned_addr learned my addr 172.21.15.26:0/754494129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9d88d700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981041d0 0x7f1f98075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 -- 172.21.15.26:0/754494129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981041d0 msgr2=0x7f1f98075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981041d0 0x7f1f98075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e88f700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f98108910 0x7f1f980762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 -- 172.21.15.26:0/754494129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f98108910 msgr2=0x7f1f980762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f98108910 0x7f1f980762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 -- 172.21.15.26:0/754494129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f981a7f70 con 0x7f1f98103820 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e88f700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f98108910 0x7f1f980762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.353+0000 7f1f9e08e700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 0x7f1f98075840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400d980 tx=0x7f1f9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.354+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94017070 con 0x7f1f98103820 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.354+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f94005020 con 0x7f1f98103820 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.354+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f981a8260 con 0x7f1f98103820 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.354+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009ab0 con 0x7f1f98103820 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.354+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981a8750 con 0x7f1f98103820 2024-05-20T20:54:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.355+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9806a230 con 0x7f1f98103820 2024-05-20T20:54:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.356+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1f940051d0 con 0x7f1f98103820 2024-05-20T20:54:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.357+0000 7f1f8f7fe700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f840b2910 0x7f1f840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.357+0000 7f1f9d88d700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f840b2910 0x7f1f840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.357+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f1f940e26d0 con 0x7f1f98103820 2024-05-20T20:54:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.357+0000 7f1f9d88d700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f840b2910 0x7f1f840b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f98103110 tx=0x7f1f8800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.360+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f94020020 con 0x7f1f98103820 2024-05-20T20:54:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.576+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1f9802e560 con 0x7f1f98103820 2024-05-20T20:54:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.576+0000 7f1f8f7fe700 1 -- 172.21.15.26:0/754494129 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f940cae20 con 0x7f1f98103820 2024-05-20T20:54:23.575 INFO:teuthology.orchestra.run.smithi026.stdout:77309411408 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f840b2910 msgr2=0x7f1f840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f840b2910 0x7f1f840b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f98103110 tx=0x7f1f8800a300 comp rx=0 tx=0).stop 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 msgr2=0x7f1f98075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 0x7f1f98075840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400d980 tx=0x7f1f9400dc90 comp rx=0 tx=0).stop 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 shutdown_connections 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f840b2910 0x7f1f840b4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.578+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98103820 0x7f1f98075840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.579+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f98108910 0x7f1f980762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.579+0000 7f1fa02f2700 1 --2- 172.21.15.26:0/754494129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981041d0 0x7f1f98075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.579+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 >> 172.21.15.26:0/754494129 conn(0x7f1f9807a6b0 msgr2=0x7f1f98101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:23.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.579+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 shutdown_connections 2024-05-20T20:54:23.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.579+0000 7f1fa02f2700 1 -- 172.21.15.26:0/754494129 wait complete. 2024-05-20T20:54:23.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:23 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/754494129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 -- 172.21.15.26:0/4234435705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 msgr2=0x7f5e6c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 --2- 172.21.15.26:0/4234435705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c102380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009990 tx=0x7f5e54009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 -- 172.21.15.26:0/4234435705 shutdown_connections 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 --2- 172.21.15.26:0/4234435705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6c1028c0 0x7f5e6c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 --2- 172.21.15.26:0/4234435705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 0x7f5e6c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 --2- 172.21.15.26:0/4234435705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c102380 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 -- 172.21.15.26:0/4234435705 >> 172.21.15.26:0/4234435705 conn(0x7f5e6c077780 msgr2=0x7f5e6c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 -- 172.21.15.26:0/4234435705 shutdown_connections 2024-05-20T20:54:23.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 -- 172.21.15.26:0/4234435705 wait complete. 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 Processor -- start 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.997+0000 7f5e709f8700 1 -- start start 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e709f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e709f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6c1028c0 0x7f5e6c103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e709f8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 0x7f5e6c103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e709f8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c113620 con 0x7f5e6c101f00 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e709f8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1134a0 con 0x7f5e6c108620 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e709f8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1137a0 con 0x7f5e6c1028c0 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47134/0 (socket says 172.21.15.26:47134) 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 -- 172.21.15.26:0/2364363433 learned_addr learned my addr 172.21.15.26:0/2364363433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6ad9d700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 0x7f5e6c103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e69d9b700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6c1028c0 0x7f5e6c103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 -- 172.21.15.26:0/2364363433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6c1028c0 msgr2=0x7f5e6c103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6c1028c0 0x7f5e6c103620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 -- 172.21.15.26:0/2364363433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 msgr2=0x7f5e6c103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 0x7f5e6c103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 -- 172.21.15.26:0/2364363433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c1a7fa0 con 0x7f5e6c101f00 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6ad9d700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 0x7f5e6c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e6a59c700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c1030e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c990 tx=0x7f5e5c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.998+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c007b00 con 0x7f5e6c101f00 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.999+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6c1a8290 con 0x7f5e6c101f00 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.999+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e5c007ca0 con 0x7f5e6c101f00 2024-05-20T20:54:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.999+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c018660 con 0x7f5e6c101f00 2024-05-20T20:54:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:23.999+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6c1a87d0 con 0x7f5e6c101f00 2024-05-20T20:54:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.000+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6c06a230 con 0x7f5e6c101f00 2024-05-20T20:54:24.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.002+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5e5c004030 con 0x7f5e6c101f00 2024-05-20T20:54:24.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.003+0000 7f5e637fe700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5e580b2910 0x7f5e580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.003+0000 7f5e69d9b700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5e580b2910 0x7f5e580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.003+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f5e5c0e2570 con 0x7f5e6c101f00 2024-05-20T20:54:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.003+0000 7f5e69d9b700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5e580b2910 0x7f5e580b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e54006010 tx=0x7f5e54005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.006+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e5c054df0 con 0x7f5e6c101f00 2024-05-20T20:54:24.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:23 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/754494129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-20T20:54:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.229+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5e6c02e560 con 0x7f5e6c101f00 2024-05-20T20:54:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.230+0000 7f5e637fe700 1 -- 172.21.15.26:0/2364363433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5e5c014030 con 0x7f5e6c101f00 2024-05-20T20:54:24.229 INFO:teuthology.orchestra.run.smithi026.stdout:154618822722 2024-05-20T20:54:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5e580b2910 msgr2=0x7f5e580b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5e580b2910 0x7f5e580b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e54006010 tx=0x7f5e54005c40 comp rx=0 tx=0).stop 2024-05-20T20:54:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 msgr2=0x7f5e6c1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c1030e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c990 tx=0x7f5e5c00cd50 comp rx=0 tx=0).stop 2024-05-20T20:54:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 shutdown_connections 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5e580b2910 0x7f5e580b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e6c101f00 0x7f5e6c1030e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e6c108620 0x7f5e6c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 --2- 172.21.15.26:0/2364363433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6c1028c0 0x7f5e6c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 >> 172.21.15.26:0/2364363433 conn(0x7f5e6c077780 msgr2=0x7f5e6c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 shutdown_connections 2024-05-20T20:54:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:24.232+0000 7f5e709f8700 1 -- 172.21.15.26:0/2364363433 wait complete. 2024-05-20T20:54:24.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:23 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/754494129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-20T20:54:24.828 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822718 got 154618822722 for osd.5 2024-05-20T20:54:24.829 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:25.066 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:24 smithi026 ceph-mon[31990]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:25.066 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:24 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2364363433' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-20T20:54:25.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:24 smithi055 ceph-mon[35213]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:25.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:24 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2364363433' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-20T20:54:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:24 smithi144 ceph-mon[35660]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:24 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2364363433' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-20T20:54:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 -- 172.21.15.26:0/1829696772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 msgr2=0x7fd14c109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1829696772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c109980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009990 tx=0x7fd13c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 -- 172.21.15.26:0/1829696772 shutdown_connections 2024-05-20T20:54:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1829696772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c109980 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1829696772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd14c06c470 0x7fd14c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1829696772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c06bab0 0x7fd14c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 -- 172.21.15.26:0/1829696772 >> 172.21.15.26:0/1829696772 conn(0x7fd14c0793b0 msgr2=0x7fd14c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 -- 172.21.15.26:0/1829696772 shutdown_connections 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.019+0000 7fd1515e7700 1 -- 172.21.15.26:0/1829696772 wait complete. 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 Processor -- start 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 -- start start 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c06bab0 0x7fd14c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd14c06c470 0x7fd14c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c113560 con 0x7fd14c1095a0 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1133e0 con 0x7fd14c06c470 2024-05-20T20:54:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd1515e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1136e0 con 0x7fd14c06bab0 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47154/0 (socket says 172.21.15.26:47154) 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.020+0000 7fd14b7fe700 1 -- 172.21.15.26:0/1319872889 learned_addr learned my addr 172.21.15.26:0/1319872889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14a7fc700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd14c06c470 0x7fd14c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14affd700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c06bab0 0x7fd14c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14b7fe700 1 -- 172.21.15.26:0/1319872889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c06bab0 msgr2=0x7fd14c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14b7fe700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c06bab0 0x7fd14c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14b7fe700 1 -- 172.21.15.26:0/1319872889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd14c06c470 msgr2=0x7fd14c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14b7fe700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd14c06c470 0x7fd14c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14b7fe700 1 -- 172.21.15.26:0/1319872889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c19a020 con 0x7fd14c1095a0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd14b7fe700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c1998c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd144007d70 tx=0x7fd14400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144016020 con 0x7fd14c1095a0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd14400ac90 con 0x7fd14c1095a0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.021+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c19a310 con 0x7fd14c1095a0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.022+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440115e0 con 0x7fd14c1095a0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.022+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c1a65d0 con 0x7fd14c1095a0 2024-05-20T20:54:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.023+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14c10ba80 con 0x7fd14c1095a0 2024-05-20T20:54:26.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.024+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd144019020 con 0x7fd14c1095a0 2024-05-20T20:54:26.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.025+0000 7fd133fff700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1340b2990 0x7fd1340b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.025+0000 7fd14affd700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1340b2990 0x7fd1340b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.025+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fd1440e26d0 con 0x7fd14c1095a0 2024-05-20T20:54:26.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.025+0000 7fd14affd700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1340b2990 0x7fd1340b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd14c19aa00 tx=0x7fd13c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:26.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.029+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd144020020 con 0x7fd14c1095a0 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 -- 172.21.15.26:0/2347727560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4102440 msgr2=0x7ffac41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/2347727560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4102440 0x7ffac41028c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009990 tx=0x7ffab8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 -- 172.21.15.26:0/2347727560 shutdown_connections 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/2347727560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102e00 0x7ffac410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/2347727560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4108520 0x7ffac4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/2347727560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4102440 0x7ffac41028c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.236+0000 7ffaca6b5700 1 -- 172.21.15.26:0/2347727560 >> 172.21.15.26:0/2347727560 conn(0x7ffac40793f0 msgr2=0x7ffac4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.237+0000 7ffaca6b5700 1 -- 172.21.15.26:0/2347727560 shutdown_connections 2024-05-20T20:54:26.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.237+0000 7ffaca6b5700 1 -- 172.21.15.26:0/2347727560 wait complete. 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.237+0000 7ffaca6b5700 1 Processor -- start 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.237+0000 7ffaca6b5700 1 -- start start 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffaca6b5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102440 0x7ffac4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffaca6b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4102e00 0x7ffac41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffaca6b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4108520 0x7ffac4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffaca6b5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4112480 con 0x7ffac4108520 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffaca6b5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4112300 con 0x7ffac4102440 2024-05-20T20:54:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffaca6b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4112600 con 0x7ffac4102e00 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac8c52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4108520 0x7ffac4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac8c52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4108520 0x7ffac4104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47170/0 (socket says 172.21.15.26:47170) 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac8c52700 1 -- 172.21.15.26:0/1764124957 learned_addr learned my addr 172.21.15.26:0/1764124957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac37fe700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4102e00 0x7ffac41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac3fff700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102440 0x7ffac4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac37fe700 1 -- 172.21.15.26:0/1764124957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102440 msgr2=0x7ffac4104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac37fe700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102440 0x7ffac4104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac37fe700 1 -- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4108520 msgr2=0x7ffac4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac37fe700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4108520 0x7ffac4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac37fe700 1 -- 172.21.15.26:0/1764124957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac41a7fc0 con 0x7ffac4102e00 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.238+0000 7ffac3fff700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102440 0x7ffac4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.239+0000 7ffac37fe700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4102e00 0x7ffac41045c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffab8000c00 tx=0x7ffab80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.239+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8026070 con 0x7ffac4102e00 2024-05-20T20:54:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.239+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffab8021410 con 0x7ffac4102e00 2024-05-20T20:54:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.239+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8017650 con 0x7ffac4102e00 2024-05-20T20:54:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.239+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac41a8250 con 0x7ffac4102e00 2024-05-20T20:54:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.240+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a8650 con 0x7ffac4102e00 2024-05-20T20:54:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.241+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac406a230 con 0x7ffac4102e00 2024-05-20T20:54:26.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.242+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ffab8029070 con 0x7ffac4102e00 2024-05-20T20:54:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.243+0000 7ffac17fa700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ffaac0b2910 0x7ffaac0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.243+0000 7ffac3fff700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ffaac0b2910 0x7ffaac0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.243+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7ffab80e3940 con 0x7ffac4102e00 2024-05-20T20:54:26.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.244+0000 7ffac3fff700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ffaac0b2910 0x7ffaac0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009fd0 tx=0x7ffab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:26.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.247+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffab8030070 con 0x7ffac4102e00 2024-05-20T20:54:26.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.255+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd14c10b460 con 0x7fd14c1095a0 2024-05-20T20:54:26.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.256+0000 7fd133fff700 1 -- 172.21.15.26:0/1319872889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1440aa380 con 0x7fd14c1095a0 2024-05-20T20:54:26.255 INFO:teuthology.orchestra.run.smithi026.stdout:107374182475 2024-05-20T20:54:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1340b2990 msgr2=0x7fd1340b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1340b2990 0x7fd1340b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd14c19aa00 tx=0x7fd13c005960 comp rx=0 tx=0).stop 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 msgr2=0x7fd14c1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c1998c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd144007d70 tx=0x7fd14400a340 comp rx=0 tx=0).stop 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 shutdown_connections 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1340b2990 0x7fd1340b4e50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1095a0 0x7fd14c1998c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c06bab0 0x7fd14c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 --2- 172.21.15.26:0/1319872889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd14c06c470 0x7fd14c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 >> 172.21.15.26:0/1319872889 conn(0x7fd14c0793b0 msgr2=0x7fd14c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.258+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 shutdown_connections 2024-05-20T20:54:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.259+0000 7fd1515e7700 1 -- 172.21.15.26:0/1319872889 wait complete. 2024-05-20T20:54:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.496+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ffac402e7c0 con 0x7ffac4102e00 2024-05-20T20:54:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.497+0000 7ffac17fa700 1 -- 172.21.15.26:0/1764124957 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffab8013070 con 0x7ffac4102e00 2024-05-20T20:54:26.497 INFO:teuthology.orchestra.run.smithi026.stdout:180388626493 2024-05-20T20:54:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ffaac0b2910 msgr2=0x7ffaac0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ffaac0b2910 0x7ffaac0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009fd0 tx=0x7ffab4009260 comp rx=0 tx=0).stop 2024-05-20T20:54:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4102e00 msgr2=0x7ffac41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4102e00 0x7ffac41045c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffab8000c00 tx=0x7ffab80047b0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 shutdown_connections 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ffaac0b2910 0x7ffaac0b4dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac4108520 0x7ffac4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac4102440 0x7ffac4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 --2- 172.21.15.26:0/1764124957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac4102e00 0x7ffac41045c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 >> 172.21.15.26:0/1764124957 conn(0x7ffac40793f0 msgr2=0x7ffac406b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 shutdown_connections 2024-05-20T20:54:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:26.499+0000 7ffaca6b5700 1 -- 172.21.15.26:0/1764124957 wait complete. 2024-05-20T20:54:26.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411408 for osd.2 2024-05-20T20:54:26.879 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:27.095 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:26 smithi026 ceph-mon[31990]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:27.095 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:26 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1319872889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-20T20:54:27.095 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:26 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1764124957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-20T20:54:27.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:26 smithi055 ceph-mon[35213]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:27.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:26 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1319872889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-20T20:54:27.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:26 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1764124957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-20T20:54:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:26 smithi144 ceph-mon[35660]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:26 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1319872889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-20T20:54:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:26 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1764124957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-20T20:54:28.094 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182475 for osd.3 2024-05-20T20:54:28.094 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:29.061 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:28 smithi026 ceph-mon[31990]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:29.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626488 got 180388626493 for osd.6 2024-05-20T20:54:29.093 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:29.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:28 smithi055 ceph-mon[35213]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:28 smithi144 ceph-mon[35660]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:29.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574933 for osd.1 2024-05-20T20:54:29.540 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/2023781722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 msgr2=0x7f2cf810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/2023781722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf810f2e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b0a0 tx=0x7f2cf000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/2023781722 shutdown_connections 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/2023781722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf810f2e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/2023781722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cf8108510 0x7f2cf81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/2023781722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cf8102430 0x7f2cf81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/2023781722 >> 172.21.15.26:0/2023781722 conn(0x7f2cf80793b0 msgr2=0x7f2cf80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/2023781722 shutdown_connections 2024-05-20T20:54:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.022+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/2023781722 wait complete. 2024-05-20T20:54:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.023+0000 7f2cfcd9a700 1 Processor -- start 2024-05-20T20:54:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.023+0000 7f2cfcd9a700 1 -- start start 2024-05-20T20:54:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.023+0000 7f2cfcd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cf8102430 0x7f2cf819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.023+0000 7f2cfcd9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cfcd9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cf8108510 0x7f2cf8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cfcd9a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf81135f0 con 0x7f2cf8102df0 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cfcd9a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf8113470 con 0x7f2cf8102430 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cfcd9a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf8113770 con 0x7f2cf8108510 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57460/0 (socket says 172.21.15.26:57460) 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 -- 172.21.15.26:0/1225348228 learned_addr learned my addr 172.21.15.26:0/1225348228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf659c700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cf8102430 0x7f2cf819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf6d9d700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cf8108510 0x7f2cf8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 -- 172.21.15.26:0/1225348228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cf8108510 msgr2=0x7f2cf8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cf8108510 0x7f2cf8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 -- 172.21.15.26:0/1225348228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cf8102430 msgr2=0x7f2cf819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cf8102430 0x7f2cf819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 -- 172.21.15.26:0/1225348228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf819a070 con 0x7f2cf8102df0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cf5d9b700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf819fc00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c9b0 tx=0x7f2cec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007b00 con 0x7f2cf8102df0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cec007ca0 con 0x7f2cf8102df0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.024+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf819a360 con 0x7f2cf8102df0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.025+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec018660 con 0x7f2cf8102df0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.025+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf81a6520 con 0x7f2cf8102df0 2024-05-20T20:54:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.026+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf8103b50 con 0x7f2cf8102df0 2024-05-20T20:54:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.027+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2cec00f450 con 0x7f2cf8102df0 2024-05-20T20:54:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.028+0000 7f2ce77fe700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2ce00b2910 0x7f2ce00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.028+0000 7f2cf659c700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2ce00b2910 0x7f2ce00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.028+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f2cec0e23a0 con 0x7f2cf8102df0 2024-05-20T20:54:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.028+0000 7f2cf659c700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2ce00b2910 0x7f2ce00b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005950 tx=0x7f2ce800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:30.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.032+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cec014030 con 0x7f2cf8102df0 2024-05-20T20:54:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.252+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2cf802e560 con 0x7f2cf8102df0 2024-05-20T20:54:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.253+0000 7f2ce77fe700 1 -- 172.21.15.26:0/1225348228 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2cec0a9f40 con 0x7f2cf8102df0 2024-05-20T20:54:30.253 INFO:teuthology.orchestra.run.smithi026.stdout:300647710755 2024-05-20T20:54:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2ce00b2910 msgr2=0x7f2ce00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2ce00b2910 0x7f2ce00b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005950 tx=0x7f2ce800a300 comp rx=0 tx=0).stop 2024-05-20T20:54:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 msgr2=0x7f2cf819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf819fc00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c9b0 tx=0x7f2cec00ccc0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 shutdown_connections 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cf8108510 0x7f2cf8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cf8102430 0x7f2cf819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2ce00b2910 0x7f2ce00b4dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 --2- 172.21.15.26:0/1225348228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf8102df0 0x7f2cf819fc00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 >> 172.21.15.26:0/1225348228 conn(0x7f2cf80793b0 msgr2=0x7f2cf8100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 shutdown_connections 2024-05-20T20:54:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:30.255+0000 7f2cfcd9a700 1 -- 172.21.15.26:0/1225348228 wait complete. 2024-05-20T20:54:30.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:30 smithi026 ceph-mon[31990]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:30.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:30 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1225348228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-20T20:54:30.980 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710751 got 300647710755 for osd.11 2024-05-20T20:54:30.980 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:31.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:30 smithi055 ceph-mon[35213]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:31.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:30 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1225348228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-20T20:54:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:30 smithi144 ceph-mon[35660]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:30 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1225348228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-20T20:54:31.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 -- 172.21.15.26:0/1606674142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 msgr2=0x7f7f4c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 --2- 172.21.15.26:0/1606674142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c10f2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009990 tx=0x7f7f44009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 -- 172.21.15.26:0/1606674142 shutdown_connections 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 --2- 172.21.15.26:0/1606674142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c10f2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 --2- 172.21.15.26:0/1606674142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f4c108520 0x7f7f4c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 --2- 172.21.15.26:0/1606674142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f4c102440 0x7f7f4c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 -- 172.21.15.26:0/1606674142 >> 172.21.15.26:0/1606674142 conn(0x7f7f4c0793f0 msgr2=0x7f7f4c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 -- 172.21.15.26:0/1606674142 shutdown_connections 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.558+0000 7f7f534e7700 1 -- 172.21.15.26:0/1606674142 wait complete. 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 Processor -- start 2024-05-20T20:54:31.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 -- start start 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f4c102440 0x7f7f4c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f4c108520 0x7f7f4c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1124a0 con 0x7f7f4c102e00 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112320 con 0x7f7f4c102440 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f534e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112620 con 0x7f7f4c108520 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57476/0 (socket says 172.21.15.26:57476) 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 -- 172.21.15.26:0/3138595887 learned_addr learned my addr 172.21.15.26:0/3138595887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f51283700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f4c102440 0x7f7f4c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f51a84700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f4c108520 0x7f7f4c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 -- 172.21.15.26:0/3138595887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f4c108520 msgr2=0x7f7f4c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f4c108520 0x7f7f4c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 -- 172.21.15.26:0/3138595887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f4c102440 msgr2=0x7f7f4c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f4c102440 0x7f7f4c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.559+0000 7f7f50a82700 1 -- 172.21.15.26:0/3138595887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19a0b0 con 0x7f7f4c102e00 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.560+0000 7f7f50a82700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c19fc80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00d980 tx=0x7f7f3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.560+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c017070 con 0x7f7f4c102e00 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.560+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19a3a0 con 0x7f7f4c102e00 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.560+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f3c005020 con 0x7f7f4c102e00 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.560+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00f980 con 0x7f7f4c102e00 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.560+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a6610 con 0x7f7f4c102e00 2024-05-20T20:54:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.561+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c06a230 con 0x7f7f4c102e00 2024-05-20T20:54:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.562+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7f3c004a30 con 0x7f7f4c102e00 2024-05-20T20:54:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.563+0000 7f7f427fc700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7f380b2910 0x7f7f380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.563+0000 7f7f51283700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7f380b2910 0x7f7f380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:31.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.563+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f7f3c0e25f0 con 0x7f7f4c102e00 2024-05-20T20:54:31.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.564+0000 7f7f51283700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7f380b2910 0x7f7f380b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009550 tx=0x7f7f480091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.567+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f3c020020 con 0x7f7f4c102e00 2024-05-20T20:54:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.789+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7f4c02e560 con 0x7f7f4c102e00 2024-05-20T20:54:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.790+0000 7f7f427fc700 1 -- 172.21.15.26:0/3138595887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f3c0a5020 con 0x7f7f4c102e00 2024-05-20T20:54:31.790 INFO:teuthology.orchestra.run.smithi026.stdout:34359738459 2024-05-20T20:54:31.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7f380b2910 msgr2=0x7f7f380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:31.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7f380b2910 0x7f7f380b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009550 tx=0x7f7f480091b0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 msgr2=0x7f7f4c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c19fc80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00d980 tx=0x7f7f3c00dc90 comp rx=0 tx=0).stop 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 shutdown_connections 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f4c108520 0x7f7f4c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f4c102440 0x7f7f4c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7f380b2910 0x7f7f380b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 --2- 172.21.15.26:0/3138595887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f4c102e00 0x7f7f4c19fc80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 >> 172.21.15.26:0/3138595887 conn(0x7f7f4c0793f0 msgr2=0x7f7f4c06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 shutdown_connections 2024-05-20T20:54:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:31.792+0000 7f7f534e7700 1 -- 172.21.15.26:0/3138595887 wait complete. 2024-05-20T20:54:32.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:31 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3138595887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-20T20:54:32.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:31 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3138595887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-20T20:54:32.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:31 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3138595887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-20T20:54:32.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/1541799388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 msgr2=0x7f4e74108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/1541799388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74108930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e640099c0 tx=0x7f4e64009cd0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/1541799388 shutdown_connections 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/1541799388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74108930 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/1541799388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e74102e30 0x7f4e7410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/1541799388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e74102470 0x7f4e741028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/1541799388 >> 172.21.15.26:0/1541799388 conn(0x7f4e74079480 msgr2=0x7f4e74079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/1541799388 shutdown_connections 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.717+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/1541799388 wait complete. 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 Processor -- start 2024-05-20T20:54:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 -- start start 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e74102470 0x7f4e741040b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e74102e30 0x7f4e741045f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e74112550 con 0x7f4e74108550 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741123d0 con 0x7f4e74102470 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e7a9c8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741126d0 con 0x7f4e74102e30 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74104b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57486/0 (socket says 172.21.15.26:57486) 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 -- 172.21.15.26:0/278725955 learned_addr learned my addr 172.21.15.26:0/278725955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e73fff700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e74102470 0x7f4e741040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e737fe700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e74102e30 0x7f4e741045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 -- 172.21.15.26:0/278725955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e74102e30 msgr2=0x7f4e741045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e74102e30 0x7f4e741045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 -- 172.21.15.26:0/278725955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e74102470 msgr2=0x7f4e741040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.718+0000 7f4e78f65700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e74102470 0x7f4e741040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e78f65700 1 -- 172.21.15.26:0/278725955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e741a8090 con 0x7f4e74108550 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e78f65700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74104b30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b970 tx=0x7f4e6c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c019070 con 0x7f4e74108550 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e741a8380 con 0x7f4e74108550 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e6c00cdc0 con 0x7f4e74108550 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c014e30 con 0x7f4e74108550 2024-05-20T20:54:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.719+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e741a88c0 con 0x7f4e74108550 2024-05-20T20:54:32.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.720+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e74103b90 con 0x7f4e74108550 2024-05-20T20:54:32.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.721+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4e6c01c070 con 0x7f4e74108550 2024-05-20T20:54:32.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.722+0000 7f4e717fa700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e5c0b2910 0x7f4e5c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:32.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.722+0000 7f4e73fff700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e5c0b2910 0x7f4e5c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:32.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.722+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f4e6c0e1af0 con 0x7f4e74108550 2024-05-20T20:54:32.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.723+0000 7f4e73fff700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e5c0b2910 0x7f4e5c0b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e640058e0 tx=0x7f4e64005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:32.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.726+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e6c0a9740 con 0x7f4e74108550 2024-05-20T20:54:32.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.945+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4e741a8be0 con 0x7f4e74108550 2024-05-20T20:54:32.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.945+0000 7f4e717fa700 1 -- 172.21.15.26:0/278725955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4e6c0a9040 con 0x7f4e74108550 2024-05-20T20:54:32.945 INFO:teuthology.orchestra.run.smithi026.stdout:253403070511 2024-05-20T20:54:32.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e5c0b2910 msgr2=0x7f4e5c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:32.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e5c0b2910 0x7f4e5c0b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e640058e0 tx=0x7f4e64005fb0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 msgr2=0x7f4e74104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:32.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74104b30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b970 tx=0x7f4e6c00bc80 comp rx=0 tx=0).stop 2024-05-20T20:54:32.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 shutdown_connections 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e5c0b2910 0x7f4e5c0b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e74108550 0x7f4e74104b30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e74102470 0x7f4e741040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 --2- 172.21.15.26:0/278725955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e74102e30 0x7f4e741045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 >> 172.21.15.26:0/278725955 conn(0x7f4e74079480 msgr2=0x7f4e741063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 shutdown_connections 2024-05-20T20:54:32.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:32.948+0000 7f4e7a9c8700 1 -- 172.21.15.26:0/278725955 wait complete. 2024-05-20T20:54:33.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:32 smithi055 ceph-mon[35213]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:33.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:32 smithi026 ceph-mon[31990]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:32 smithi144 ceph-mon[35660]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:33.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738459 for osd.0 2024-05-20T20:54:33.439 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:33.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 -- 172.21.15.26:0/366182989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a441095b0 msgr2=0x7f5a44109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 --2- 172.21.15.26:0/366182989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a441095b0 0x7f5a44109990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000b0a0 tx=0x7f5a4000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 -- 172.21.15.26:0/366182989 shutdown_connections 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 --2- 172.21.15.26:0/366182989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a441095b0 0x7f5a44109990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 --2- 172.21.15.26:0/366182989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4406bbe0 0x7f5a441103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 --2- 172.21.15.26:0/366182989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4406b220 0x7f5a4406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 -- 172.21.15.26:0/366182989 >> 172.21.15.26:0/366182989 conn(0x7f5a440793f0 msgr2=0x7f5a44079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 -- 172.21.15.26:0/366182989 shutdown_connections 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.887+0000 7f5a4b695700 1 -- 172.21.15.26:0/366182989 wait complete. 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 Processor -- start 2024-05-20T20:54:33.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 -- start start 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4406b220 0x7f5a44102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 0x7f5a44102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a441095b0 0x7f5a44102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441135f0 con 0x7f5a4406bbe0 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44113470 con 0x7f5a4406b220 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a4b695700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44113770 con 0x7f5a441095b0 2024-05-20T20:54:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 0x7f5a44102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 0x7f5a44102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57518/0 (socket says 172.21.15.26:57518) 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 -- 172.21.15.26:0/2663432328 learned_addr learned my addr 172.21.15.26:0/2663432328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a49431700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4406b220 0x7f5a44102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a49c32700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a441095b0 0x7f5a44102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 -- 172.21.15.26:0/2663432328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a441095b0 msgr2=0x7f5a44102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a441095b0 0x7f5a44102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 -- 172.21.15.26:0/2663432328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4406b220 msgr2=0x7f5a44102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4406b220 0x7f5a44102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.888+0000 7f5a48c30700 1 -- 172.21.15.26:0/2663432328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a441a8140 con 0x7f5a4406bbe0 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.889+0000 7f5a48c30700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 0x7f5a44102a50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d950 tx=0x7f5a3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.889+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34016070 con 0x7f5a4406bbe0 2024-05-20T20:54:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.889+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a3400fd20 con 0x7f5a4406bbe0 2024-05-20T20:54:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.889+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a441a83d0 con 0x7f5a4406bbe0 2024-05-20T20:54:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.889+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400e3b0 con 0x7f5a4406bbe0 2024-05-20T20:54:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.889+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a441a8960 con 0x7f5a4406bbe0 2024-05-20T20:54:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.890+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4410ba90 con 0x7f5a4406bbe0 2024-05-20T20:54:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.891+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5a34019070 con 0x7f5a4406bbe0 2024-05-20T20:54:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.892+0000 7f5a3a7fc700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a300b2990 0x7f5a300b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.893+0000 7f5a49431700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a300b2990 0x7f5a300b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.893+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f5a340e1df0 con 0x7f5a4406bbe0 2024-05-20T20:54:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.893+0000 7f5a49431700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a300b2990 0x7f5a300b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009fd0 tx=0x7f5a40006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:33.896+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a340a9990 con 0x7f5a4406bbe0 2024-05-20T20:54:33.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:33 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/278725955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-20T20:54:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.115+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5a44103a40 con 0x7f5a4406bbe0 2024-05-20T20:54:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.116+0000 7f5a3a7fc700 1 -- 172.21.15.26:0/2663432328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a340a9290 con 0x7f5a4406bbe0 2024-05-20T20:54:34.116 INFO:teuthology.orchestra.run.smithi026.stdout:227633266740 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a300b2990 msgr2=0x7f5a300b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a300b2990 0x7f5a300b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009fd0 tx=0x7f5a40006210 comp rx=0 tx=0).stop 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 msgr2=0x7f5a44102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 0x7f5a44102a50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d950 tx=0x7f5a3400dc60 comp rx=0 tx=0).stop 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 shutdown_connections 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a441095b0 0x7f5a44102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4406b220 0x7f5a44102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a300b2990 0x7f5a300b4e50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 --2- 172.21.15.26:0/2663432328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4406bbe0 0x7f5a44102a50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.118+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 >> 172.21.15.26:0/2663432328 conn(0x7f5a440793f0 msgr2=0x7f5a441012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.119+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 shutdown_connections 2024-05-20T20:54:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:34.119+0000 7f5a4b695700 1 -- 172.21.15.26:0/2663432328 wait complete. 2024-05-20T20:54:34.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:33 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/278725955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-20T20:54:34.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:33 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/278725955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-20T20:54:34.473 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070511 for osd.9 2024-05-20T20:54:34.473 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:35.082 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:34 smithi026 ceph-mon[31990]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:35.083 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:34 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2663432328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-20T20:54:35.115 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266734 got 227633266740 for osd.8 2024-05-20T20:54:35.115 DEBUG:teuthology.parallel:result is None 2024-05-20T20:54:35.115 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-20T20:54:35.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:54:35.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:34 smithi055 ceph-mon[35213]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:35.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:34 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2663432328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-20T20:54:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:34 smithi144 ceph-mon[35660]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:34 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2663432328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-20T20:54:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 -- 172.21.15.26:0/2411825876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 msgr2=0x7f8f8c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 --2- 172.21.15.26:0/2411825876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c110450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f840099c0 tx=0x7f8f84009cd0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 -- 172.21.15.26:0/2411825876 shutdown_connections 2024-05-20T20:54:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 --2- 172.21.15.26:0/2411825876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c110450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 --2- 172.21.15.26:0/2411825876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f8c109680 0x7f8f8c109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 --2- 172.21.15.26:0/2411825876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c1035e0 0x7f8f8c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 -- 172.21.15.26:0/2411825876 >> 172.21.15.26:0/2411825876 conn(0x7f8f8c07a6f0 msgr2=0x7f8f8c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 -- 172.21.15.26:0/2411825876 shutdown_connections 2024-05-20T20:54:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 -- 172.21.15.26:0/2411825876 wait complete. 2024-05-20T20:54:37.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:36 smithi026 ceph-mon[31990]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:37.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:36 smithi026 ceph-mon[31990]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-20T20:54:37.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]: dispatch 2024-05-20T20:54:37.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:36 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.026+0000 7f8f91958700 1 Processor -- start 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 -- start start 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f8c1035e0 0x7f8f8c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c109680 0x7f8f8c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1136e0 con 0x7f8f8c103fa0 2024-05-20T20:54:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c113560 con 0x7f8f8c109680 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f91958700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c113860 con 0x7f8f8c1035e0 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57536/0 (socket says 172.21.15.26:57536) 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8a7fc700 1 -- 172.21.15.26:0/3917141186 learned_addr learned my addr 172.21.15.26:0/3917141186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8affd700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f8c1035e0 0x7f8f8c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8b7fe700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c109680 0x7f8f8c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8a7fc700 1 -- 172.21.15.26:0/3917141186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f8c1035e0 msgr2=0x7f8f8c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.027+0000 7f8f8a7fc700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f8c1035e0 0x7f8f8c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f8a7fc700 1 -- 172.21.15.26:0/3917141186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c109680 msgr2=0x7f8f8c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f8a7fc700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c109680 0x7f8f8c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f8a7fc700 1 -- 172.21.15.26:0/3917141186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c1a8050 con 0x7f8f8c103fa0 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f8b7fe700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c109680 0x7f8f8c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:54:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f8a7fc700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c075d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c9b0 tx=0x7f8f7400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74007b00 con 0x7f8f8c103fa0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f74007ca0 con 0x7f8f8c103fa0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.028+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74018660 con 0x7f8f8c103fa0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.029+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c1a8340 con 0x7f8f8c103fa0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.029+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1a8830 con 0x7f8f8c103fa0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.030+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c10bb10 con 0x7f8f8c103fa0 2024-05-20T20:54:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.031+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f7400f450 con 0x7f8f8c103fa0 2024-05-20T20:54:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.032+0000 7f8f90956700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f780b2900 0x7f8f780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.032+0000 7f8f8affd700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f780b2900 0x7f8f780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.032+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8327+0+0 (secure 0 0 0) 0x7f8f740e2260 con 0x7f8f8c103fa0 2024-05-20T20:54:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.032+0000 7f8f8affd700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f780b2900 0x7f8f780b4dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00a400 tx=0x7f8f7c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.037+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f74014030 con 0x7f8f8c103fa0 2024-05-20T20:54:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:36 smithi055 ceph-mon[35213]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:36 smithi055 ceph-mon[35213]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-20T20:54:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]: dispatch 2024-05-20T20:54:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:36 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-20T20:54:37.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.248+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f8c063980 con 0x7f8f780b2900 2024-05-20T20:54:37.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.256+0000 7f8f90956700 1 -- 172.21.15.26:0/3917141186 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258998 (secure 0 0 0) 0x7f8f8c063980 con 0x7f8f780b2900 2024-05-20T20:54:37.263 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:37.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f780b2900 msgr2=0x7f8f780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:37.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f780b2900 0x7f8f780b4dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00a400 tx=0x7f8f7c00d040 comp rx=0 tx=0).stop 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 msgr2=0x7f8f8c075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c075d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c9b0 tx=0x7f8f7400ccc0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 shutdown_connections 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f8c109680 0x7f8f8c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f8c1035e0 0x7f8f8c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f780b2900 0x7f8f780b4dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 --2- 172.21.15.26:0/3917141186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f8c103fa0 0x7f8f8c075d80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 >> 172.21.15.26:0/3917141186 conn(0x7f8f8c07a6f0 msgr2=0x7f8f8c1021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 shutdown_connections 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:37.259+0000 7f8f91958700 1 -- 172.21.15.26:0/3917141186 wait complete. 2024-05-20T20:54:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:54:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:36 smithi144 ceph-mon[35660]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:36 smithi144 ceph-mon[35660]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-20T20:54:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]: dispatch 2024-05-20T20:54:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:36 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-20T20:54:37.900 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-05-20T20:54:36.856541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656429,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":651,"num_read_kb":421,"num_write":288,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355704,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586120,"statfs":{"total":1151940427776,"available":1151576186880,"internally_reserved":0,"allocated":10911744,"data_stored":7065855,"data_compressed":179439,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.136994"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844798+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:53:36.844798+0000","last_peered":"2024-05-20T20:53:36.844798+0000","last_clean":"2024-05-20T20:53:36.844798+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:53:36.844798+0000","last_undegraded":"2024-05-20T20:53:36.844798+0000","last_fullsized":"2024-05-20T20:53:36.844798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.706594+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:35.706594+0000","last_peered":"2024-05-20T20:54:35.706594+0000","last_clean":"2024-05-20T20:54:35.706594+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:35.706594+0000","last_undegraded":"2024-05-20T20:54:35.706594+0000","last_fullsized":"2024-05-20T20:54:35.706594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.846259+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:53:36.846259+0000","last_peered":"2024-05-20T20:53:36.846259+0000","last_clean":"2024-05-20T20:53:36.846259+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:53:36.846259+0000","last_undegraded":"2024-05-20T20:53:36.846259+0000","last_fullsized":"2024-05-20T20:53:36.846259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.874916+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:53:36.874916+0000","last_peered":"2024-05-20T20:53:36.874916+0000","last_clean":"2024-05-20T20:53:36.874916+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:53:36.874916+0000","last_undegraded":"2024-05-20T20:53:36.874916+0000","last_fullsized":"2024-05-20T20:53:36.874916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.847376+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:53:36.847376+0000","last_peered":"2024-05-20T20:53:36.847376+0000","last_clean":"2024-05-20T20:53:36.847376+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:53:36.847376+0000","last_undegraded":"2024-05-20T20:53:36.847376+0000","last_fullsized":"2024-05-20T20:53:36.847376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875168+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:53:36.875168+0000","last_peered":"2024-05-20T20:53:36.875168+0000","last_clean":"2024-05-20T20:53:36.875168+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:53:36.875168+0000","last_undegraded":"2024-05-20T20:53:36.875168+0000","last_fullsized":"2024-05-20T20:53:36.875168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848298+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:53:36.848298+0000","last_peered":"2024-05-20T20:53:36.848298+0000","last_clean":"2024-05-20T20:53:36.848298+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:53:36.848298+0000","last_undegraded":"2024-05-20T20:53:36.848298+0000","last_fullsized":"2024-05-20T20:53:36.848298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.717314+0000","last_change":"2024-05-20T20:52:45.221289+0000","last_active":"2024-05-20T20:54:35.717314+0000","last_peered":"2024-05-20T20:54:35.717314+0000","last_clean":"2024-05-20T20:54:35.717314+0000","last_became_active":"2024-05-20T20:52:45.220972+0000","last_became_peered":"2024-05-20T20:52:45.220972+0000","last_unstale":"2024-05-20T20:54:35.717314+0000","last_undegraded":"2024-05-20T20:54:35.717314+0000","last_fullsized":"2024-05-20T20:54:35.717314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:52.180359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845129+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:53:36.845129+0000","last_peered":"2024-05-20T20:53:36.845129+0000","last_clean":"2024-05-20T20:53:36.845129+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:53:36.845129+0000","last_undegraded":"2024-05-20T20:53:36.845129+0000","last_fullsized":"2024-05-20T20:53:36.845129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.876127+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:53:36.876127+0000","last_peered":"2024-05-20T20:53:36.876127+0000","last_clean":"2024-05-20T20:53:36.876127+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:53:36.876127+0000","last_undegraded":"2024-05-20T20:53:36.876127+0000","last_fullsized":"2024-05-20T20:53:36.876127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849114+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:53:36.849114+0000","last_peered":"2024-05-20T20:53:36.849114+0000","last_clean":"2024-05-20T20:53:36.849114+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:53:36.849114+0000","last_undegraded":"2024-05-20T20:53:36.849114+0000","last_fullsized":"2024-05-20T20:53:36.849114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.874766+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:53:36.874766+0000","last_peered":"2024-05-20T20:53:36.874766+0000","last_clean":"2024-05-20T20:53:36.874766+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:53:36.874766+0000","last_undegraded":"2024-05-20T20:53:36.874766+0000","last_fullsized":"2024-05-20T20:53:36.874766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875650+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:53:36.875650+0000","last_peered":"2024-05-20T20:53:36.875650+0000","last_clean":"2024-05-20T20:53:36.875650+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:53:36.875650+0000","last_undegraded":"2024-05-20T20:53:36.875650+0000","last_fullsized":"2024-05-20T20:53:36.875650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.723859+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:35.723859+0000","last_peered":"2024-05-20T20:54:35.723859+0000","last_clean":"2024-05-20T20:54:35.723859+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:35.723859+0000","last_undegraded":"2024-05-20T20:54:35.723859+0000","last_fullsized":"2024-05-20T20:54:35.723859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.721167+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:35.721167+0000","last_peered":"2024-05-20T20:54:35.721167+0000","last_clean":"2024-05-20T20:54:35.721167+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:35.721167+0000","last_undegraded":"2024-05-20T20:54:35.721167+0000","last_fullsized":"2024-05-20T20:54:35.721167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844391+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:53:36.844391+0000","last_peered":"2024-05-20T20:53:36.844391+0000","last_clean":"2024-05-20T20:53:36.844391+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:53:36.844391+0000","last_undegraded":"2024-05-20T20:53:36.844391+0000","last_fullsized":"2024-05-20T20:53:36.844391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845448+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:53:36.845448+0000","last_peered":"2024-05-20T20:53:36.845448+0000","last_clean":"2024-05-20T20:53:36.845448+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:53:36.845448+0000","last_undegraded":"2024-05-20T20:53:36.845448+0000","last_fullsized":"2024-05-20T20:53:36.845448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845398+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:53:36.845398+0000","last_peered":"2024-05-20T20:53:36.845398+0000","last_clean":"2024-05-20T20:53:36.845398+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:53:36.845398+0000","last_undegraded":"2024-05-20T20:53:36.845398+0000","last_fullsized":"2024-05-20T20:53:36.845398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849918+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:53:36.849918+0000","last_peered":"2024-05-20T20:53:36.849918+0000","last_clean":"2024-05-20T20:53:36.849918+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:53:36.849918+0000","last_undegraded":"2024-05-20T20:53:36.849918+0000","last_fullsized":"2024-05-20T20:53:36.849918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845568+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:53:36.845568+0000","last_peered":"2024-05-20T20:53:36.845568+0000","last_clean":"2024-05-20T20:53:36.845568+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:53:36.845568+0000","last_undegraded":"2024-05-20T20:53:36.845568+0000","last_fullsized":"2024-05-20T20:53:36.845568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.715169+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:35.715169+0000","last_peered":"2024-05-20T20:54:35.715169+0000","last_clean":"2024-05-20T20:54:35.715169+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:35.715169+0000","last_undegraded":"2024-05-20T20:54:35.715169+0000","last_fullsized":"2024-05-20T20:54:35.715169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.843126+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:53:36.843126+0000","last_peered":"2024-05-20T20:53:36.843126+0000","last_clean":"2024-05-20T20:53:36.843126+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:53:36.843126+0000","last_undegraded":"2024-05-20T20:53:36.843126+0000","last_fullsized":"2024-05-20T20:53:36.843126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.846261+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:53:36.846261+0000","last_peered":"2024-05-20T20:53:36.846261+0000","last_clean":"2024-05-20T20:53:36.846261+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:53:36.846261+0000","last_undegraded":"2024-05-20T20:53:36.846261+0000","last_fullsized":"2024-05-20T20:53:36.846261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848065+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:53:36.848065+0000","last_peered":"2024-05-20T20:53:36.848065+0000","last_clean":"2024-05-20T20:53:36.848065+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:53:36.848065+0000","last_undegraded":"2024-05-20T20:53:36.848065+0000","last_fullsized":"2024-05-20T20:53:36.848065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844604+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:53:36.844604+0000","last_peered":"2024-05-20T20:53:36.844604+0000","last_clean":"2024-05-20T20:53:36.844604+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:53:36.844604+0000","last_undegraded":"2024-05-20T20:53:36.844604+0000","last_fullsized":"2024-05-20T20:53:36.844604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.847682+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:53:36.847682+0000","last_peered":"2024-05-20T20:53:36.847682+0000","last_clean":"2024-05-20T20:53:36.847682+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:53:36.847682+0000","last_undegraded":"2024-05-20T20:53:36.847682+0000","last_fullsized":"2024-05-20T20:53:36.847682+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.843262+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:53:36.843262+0000","last_peered":"2024-05-20T20:53:36.843262+0000","last_clean":"2024-05-20T20:53:36.843262+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:53:36.843262+0000","last_undegraded":"2024-05-20T20:53:36.843262+0000","last_fullsized":"2024-05-20T20:53:36.843262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":122,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:30.457578+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:54:30.457578+0000","last_peered":"2024-05-20T20:54:30.457578+0000","last_clean":"2024-05-20T20:54:30.457578+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:54:30.457578+0000","last_undegraded":"2024-05-20T20:54:30.457578+0000","last_fullsized":"2024-05-20T20:54:30.457578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.851548+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:53:36.851548+0000","last_peered":"2024-05-20T20:53:36.851548+0000","last_clean":"2024-05-20T20:53:36.851548+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:53:36.851548+0000","last_undegraded":"2024-05-20T20:53:36.851548+0000","last_fullsized":"2024-05-20T20:53:36.851548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875044+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:53:36.875044+0000","last_peered":"2024-05-20T20:53:36.875044+0000","last_clean":"2024-05-20T20:53:36.875044+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:53:36.875044+0000","last_undegraded":"2024-05-20T20:53:36.875044+0000","last_fullsized":"2024-05-20T20:53:36.875044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.877027+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:53:36.877027+0000","last_peered":"2024-05-20T20:53:36.877027+0000","last_clean":"2024-05-20T20:53:36.877027+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:53:36.877027+0000","last_undegraded":"2024-05-20T20:53:36.877027+0000","last_fullsized":"2024-05-20T20:53:36.877027+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875147+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:53:36.875147+0000","last_peered":"2024-05-20T20:53:36.875147+0000","last_clean":"2024-05-20T20:53:36.875147+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:53:36.875147+0000","last_undegraded":"2024-05-20T20:53:36.875147+0000","last_fullsized":"2024-05-20T20:53:36.875147+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.713824+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:35.713824+0000","last_peered":"2024-05-20T20:54:35.713824+0000","last_clean":"2024-05-20T20:54:35.713824+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:35.713824+0000","last_undegraded":"2024-05-20T20:54:35.713824+0000","last_fullsized":"2024-05-20T20:54:35.713824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.842781+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:53:36.842781+0000","last_peered":"2024-05-20T20:53:36.842781+0000","last_clean":"2024-05-20T20:53:36.842781+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:53:36.842781+0000","last_undegraded":"2024-05-20T20:53:36.842781+0000","last_fullsized":"2024-05-20T20:53:36.842781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845077+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:53:36.845077+0000","last_peered":"2024-05-20T20:53:36.845077+0000","last_clean":"2024-05-20T20:53:36.845077+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:53:36.845077+0000","last_undegraded":"2024-05-20T20:53:36.845077+0000","last_fullsized":"2024-05-20T20:53:36.845077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":122,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:33.461580+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:54:33.461580+0000","last_peered":"2024-05-20T20:54:33.461580+0000","last_clean":"2024-05-20T20:54:33.461580+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:54:33.461580+0000","last_undegraded":"2024-05-20T20:54:33.461580+0000","last_fullsized":"2024-05-20T20:54:33.461580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845039+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:53:36.845039+0000","last_peered":"2024-05-20T20:53:36.845039+0000","last_clean":"2024-05-20T20:53:36.845039+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:53:36.845039+0000","last_undegraded":"2024-05-20T20:53:36.845039+0000","last_fullsized":"2024-05-20T20:53:36.845039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848882+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:53:36.848882+0000","last_peered":"2024-05-20T20:53:36.848882+0000","last_clean":"2024-05-20T20:53:36.848882+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:53:36.848882+0000","last_undegraded":"2024-05-20T20:53:36.848882+0000","last_fullsized":"2024-05-20T20:53:36.848882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.720148+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:35.720148+0000","last_peered":"2024-05-20T20:54:35.720148+0000","last_clean":"2024-05-20T20:54:35.720148+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:35.720148+0000","last_undegraded":"2024-05-20T20:54:35.720148+0000","last_fullsized":"2024-05-20T20:54:35.720148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.843381+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:53:36.843381+0000","last_peered":"2024-05-20T20:53:36.843381+0000","last_clean":"2024-05-20T20:53:36.843381+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:53:36.843381+0000","last_undegraded":"2024-05-20T20:53:36.843381+0000","last_fullsized":"2024-05-20T20:53:36.843381+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:53:36.844712+0000","last_peered":"2024-05-20T20:53:36.844712+0000","last_clean":"2024-05-20T20:53:36.844712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:53:36.844712+0000","last_undegraded":"2024-05-20T20:53:36.844712+0000","last_fullsized":"2024-05-20T20:53:36.844712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":250,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:34.463243+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:54:34.463243+0000","last_peered":"2024-05-20T20:54:34.463243+0000","last_clean":"2024-05-20T20:54:34.463243+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:54:34.463243+0000","last_undegraded":"2024-05-20T20:54:34.463243+0000","last_fullsized":"2024-05-20T20:54:34.463243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.846993+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:53:36.846993+0000","last_peered":"2024-05-20T20:53:36.846993+0000","last_clean":"2024-05-20T20:53:36.846993+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:53:36.846993+0000","last_undegraded":"2024-05-20T20:53:36.846993+0000","last_fullsized":"2024-05-20T20:53:36.846993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849091+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:53:36.849091+0000","last_peered":"2024-05-20T20:53:36.849091+0000","last_clean":"2024-05-20T20:53:36.849091+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:53:36.849091+0000","last_undegraded":"2024-05-20T20:53:36.849091+0000","last_fullsized":"2024-05-20T20:53:36.849091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875372+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:53:36.875372+0000","last_peered":"2024-05-20T20:53:36.875372+0000","last_clean":"2024-05-20T20:53:36.875372+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:53:36.875372+0000","last_undegraded":"2024-05-20T20:53:36.875372+0000","last_fullsized":"2024-05-20T20:53:36.875372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.843689+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:53:36.843689+0000","last_peered":"2024-05-20T20:53:36.843689+0000","last_clean":"2024-05-20T20:53:36.843689+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:53:36.843689+0000","last_undegraded":"2024-05-20T20:53:36.843689+0000","last_fullsized":"2024-05-20T20:53:36.843689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849072+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:53:36.849072+0000","last_peered":"2024-05-20T20:53:36.849072+0000","last_clean":"2024-05-20T20:53:36.849072+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:53:36.849072+0000","last_undegraded":"2024-05-20T20:53:36.849072+0000","last_fullsized":"2024-05-20T20:53:36.849072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":123,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:30.457870+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:54:30.457870+0000","last_peered":"2024-05-20T20:54:30.457870+0000","last_clean":"2024-05-20T20:54:30.457870+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:54:30.457870+0000","last_undegraded":"2024-05-20T20:54:30.457870+0000","last_fullsized":"2024-05-20T20:54:30.457870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848185+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:53:36.848185+0000","last_peered":"2024-05-20T20:53:36.848185+0000","last_clean":"2024-05-20T20:53:36.848185+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:53:36.848185+0000","last_undegraded":"2024-05-20T20:53:36.848185+0000","last_fullsized":"2024-05-20T20:53:36.848185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.889437+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:53:36.889437+0000","last_peered":"2024-05-20T20:53:36.889437+0000","last_clean":"2024-05-20T20:53:36.889437+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:53:36.889437+0000","last_undegraded":"2024-05-20T20:53:36.889437+0000","last_fullsized":"2024-05-20T20:53:36.889437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.842921+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:53:36.842921+0000","last_peered":"2024-05-20T20:53:36.842921+0000","last_clean":"2024-05-20T20:53:36.842921+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:53:36.842921+0000","last_undegraded":"2024-05-20T20:53:36.842921+0000","last_fullsized":"2024-05-20T20:53:36.842921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848794+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:53:36.848794+0000","last_peered":"2024-05-20T20:53:36.848794+0000","last_clean":"2024-05-20T20:53:36.848794+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:53:36.848794+0000","last_undegraded":"2024-05-20T20:53:36.848794+0000","last_fullsized":"2024-05-20T20:53:36.848794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.876597+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:53:36.876597+0000","last_peered":"2024-05-20T20:53:36.876597+0000","last_clean":"2024-05-20T20:53:36.876597+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:53:36.876597+0000","last_undegraded":"2024-05-20T20:53:36.876597+0000","last_fullsized":"2024-05-20T20:53:36.876597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845158+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:53:36.845158+0000","last_peered":"2024-05-20T20:53:36.845158+0000","last_clean":"2024-05-20T20:53:36.845158+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:53:36.845158+0000","last_undegraded":"2024-05-20T20:53:36.845158+0000","last_fullsized":"2024-05-20T20:53:36.845158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875786+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:53:36.875786+0000","last_peered":"2024-05-20T20:53:36.875786+0000","last_clean":"2024-05-20T20:53:36.875786+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:53:36.875786+0000","last_undegraded":"2024-05-20T20:53:36.875786+0000","last_fullsized":"2024-05-20T20:53:36.875786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848879+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:53:36.848879+0000","last_peered":"2024-05-20T20:53:36.848879+0000","last_clean":"2024-05-20T20:53:36.848879+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:53:36.848879+0000","last_undegraded":"2024-05-20T20:53:36.848879+0000","last_fullsized":"2024-05-20T20:53:36.848879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.874705+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:53:36.874705+0000","last_peered":"2024-05-20T20:53:36.874705+0000","last_clean":"2024-05-20T20:53:36.874705+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:53:36.874705+0000","last_undegraded":"2024-05-20T20:53:36.874705+0000","last_fullsized":"2024-05-20T20:53:36.874705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.877177+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:53:36.877177+0000","last_peered":"2024-05-20T20:53:36.877177+0000","last_clean":"2024-05-20T20:53:36.877177+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:53:36.877177+0000","last_undegraded":"2024-05-20T20:53:36.877177+0000","last_fullsized":"2024-05-20T20:53:36.877177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:30.457303+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:54:30.457303+0000","last_peered":"2024-05-20T20:54:30.457303+0000","last_clean":"2024-05-20T20:54:30.457303+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:54:30.457303+0000","last_undegraded":"2024-05-20T20:54:30.457303+0000","last_fullsized":"2024-05-20T20:54:30.457303+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.728775+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:35.728775+0000","last_peered":"2024-05-20T20:54:35.728775+0000","last_clean":"2024-05-20T20:54:35.728775+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:35.728775+0000","last_undegraded":"2024-05-20T20:54:35.728775+0000","last_fullsized":"2024-05-20T20:54:35.728775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875606+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:53:36.875606+0000","last_peered":"2024-05-20T20:53:36.875606+0000","last_clean":"2024-05-20T20:53:36.875606+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:53:36.875606+0000","last_undegraded":"2024-05-20T20:53:36.875606+0000","last_fullsized":"2024-05-20T20:53:36.875606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848501+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:53:36.848501+0000","last_peered":"2024-05-20T20:53:36.848501+0000","last_clean":"2024-05-20T20:53:36.848501+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:53:36.848501+0000","last_undegraded":"2024-05-20T20:53:36.848501+0000","last_fullsized":"2024-05-20T20:53:36.848501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875011+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:53:36.875011+0000","last_peered":"2024-05-20T20:53:36.875011+0000","last_clean":"2024-05-20T20:53:36.875011+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:53:36.875011+0000","last_undegraded":"2024-05-20T20:53:36.875011+0000","last_fullsized":"2024-05-20T20:53:36.875011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848937+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:53:36.848937+0000","last_peered":"2024-05-20T20:53:36.848937+0000","last_clean":"2024-05-20T20:53:36.848937+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:53:36.848937+0000","last_undegraded":"2024-05-20T20:53:36.848937+0000","last_fullsized":"2024-05-20T20:53:36.848937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.847796+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:53:36.847796+0000","last_peered":"2024-05-20T20:53:36.847796+0000","last_clean":"2024-05-20T20:53:36.847796+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:53:36.847796+0000","last_undegraded":"2024-05-20T20:53:36.847796+0000","last_fullsized":"2024-05-20T20:53:36.847796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875314+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:53:36.875314+0000","last_peered":"2024-05-20T20:53:36.875314+0000","last_clean":"2024-05-20T20:53:36.875314+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:53:36.875314+0000","last_undegraded":"2024-05-20T20:53:36.875314+0000","last_fullsized":"2024-05-20T20:53:36.875314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845272+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:53:36.845272+0000","last_peered":"2024-05-20T20:53:36.845272+0000","last_clean":"2024-05-20T20:53:36.845272+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:53:36.845272+0000","last_undegraded":"2024-05-20T20:53:36.845272+0000","last_fullsized":"2024-05-20T20:53:36.845272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.724677+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:35.724677+0000","last_peered":"2024-05-20T20:54:35.724677+0000","last_clean":"2024-05-20T20:54:35.724677+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:35.724677+0000","last_undegraded":"2024-05-20T20:54:35.724677+0000","last_fullsized":"2024-05-20T20:54:35.724677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875126+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:53:36.875126+0000","last_peered":"2024-05-20T20:53:36.875126+0000","last_clean":"2024-05-20T20:53:36.875126+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:53:36.875126+0000","last_undegraded":"2024-05-20T20:53:36.875126+0000","last_fullsized":"2024-05-20T20:53:36.875126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.724229+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:35.724229+0000","last_peered":"2024-05-20T20:54:35.724229+0000","last_clean":"2024-05-20T20:54:35.724229+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:35.724229+0000","last_undegraded":"2024-05-20T20:54:35.724229+0000","last_fullsized":"2024-05-20T20:54:35.724229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844410+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:53:36.844410+0000","last_peered":"2024-05-20T20:53:36.844410+0000","last_clean":"2024-05-20T20:53:36.844410+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:53:36.844410+0000","last_undegraded":"2024-05-20T20:53:36.844410+0000","last_fullsized":"2024-05-20T20:53:36.844410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.846131+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:53:36.846131+0000","last_peered":"2024-05-20T20:53:36.846131+0000","last_clean":"2024-05-20T20:53:36.846131+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:53:36.846131+0000","last_undegraded":"2024-05-20T20:53:36.846131+0000","last_fullsized":"2024-05-20T20:53:36.846131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.842436+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:53:36.842436+0000","last_peered":"2024-05-20T20:53:36.842436+0000","last_clean":"2024-05-20T20:53:36.842436+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:53:36.842436+0000","last_undegraded":"2024-05-20T20:53:36.842436+0000","last_fullsized":"2024-05-20T20:53:36.842436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845428+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:53:36.845428+0000","last_peered":"2024-05-20T20:53:36.845428+0000","last_clean":"2024-05-20T20:53:36.845428+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:53:36.845428+0000","last_undegraded":"2024-05-20T20:53:36.845428+0000","last_fullsized":"2024-05-20T20:53:36.845428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.854521+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:53:36.854521+0000","last_peered":"2024-05-20T20:53:36.854521+0000","last_clean":"2024-05-20T20:53:36.854521+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:53:36.854521+0000","last_undegraded":"2024-05-20T20:53:36.854521+0000","last_fullsized":"2024-05-20T20:53:36.854521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.719046+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:35.719046+0000","last_peered":"2024-05-20T20:54:35.719046+0000","last_clean":"2024-05-20T20:54:35.719046+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:35.719046+0000","last_undegraded":"2024-05-20T20:54:35.719046+0000","last_fullsized":"2024-05-20T20:54:35.719046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.858684+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:53:36.858684+0000","last_peered":"2024-05-20T20:53:36.858684+0000","last_clean":"2024-05-20T20:53:36.858684+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:53:36.858684+0000","last_undegraded":"2024-05-20T20:53:36.858684+0000","last_fullsized":"2024-05-20T20:53:36.858684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849268+0000","last_change":"2024-05-20T20:52:45.226470+0000","last_active":"2024-05-20T20:53:36.849268+0000","last_peered":"2024-05-20T20:53:36.849268+0000","last_clean":"2024-05-20T20:53:36.849268+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:53:36.849268+0000","last_undegraded":"2024-05-20T20:53:36.849268+0000","last_fullsized":"2024-05-20T20:53:36.849268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:55:28.220915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849332+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:53:36.849332+0000","last_peered":"2024-05-20T20:53:36.849332+0000","last_clean":"2024-05-20T20:53:36.849332+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:53:36.849332+0000","last_undegraded":"2024-05-20T20:53:36.849332+0000","last_fullsized":"2024-05-20T20:53:36.849332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849361+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:53:36.849361+0000","last_peered":"2024-05-20T20:53:36.849361+0000","last_clean":"2024-05-20T20:53:36.849361+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:53:36.849361+0000","last_undegraded":"2024-05-20T20:53:36.849361+0000","last_fullsized":"2024-05-20T20:53:36.849361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.843089+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:53:36.843089+0000","last_peered":"2024-05-20T20:53:36.843089+0000","last_clean":"2024-05-20T20:53:36.843089+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:53:36.843089+0000","last_undegraded":"2024-05-20T20:53:36.843089+0000","last_fullsized":"2024-05-20T20:53:36.843089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848597+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:53:36.848597+0000","last_peered":"2024-05-20T20:53:36.848597+0000","last_clean":"2024-05-20T20:53:36.848597+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:53:36.848597+0000","last_undegraded":"2024-05-20T20:53:36.848597+0000","last_fullsized":"2024-05-20T20:53:36.848597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875336+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:53:36.875336+0000","last_peered":"2024-05-20T20:53:36.875336+0000","last_clean":"2024-05-20T20:53:36.875336+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:53:36.875336+0000","last_undegraded":"2024-05-20T20:53:36.875336+0000","last_fullsized":"2024-05-20T20:53:36.875336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849532+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:53:36.849532+0000","last_peered":"2024-05-20T20:53:36.849532+0000","last_clean":"2024-05-20T20:53:36.849532+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:53:36.849532+0000","last_undegraded":"2024-05-20T20:53:36.849532+0000","last_fullsized":"2024-05-20T20:53:36.849532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845542+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:53:36.845542+0000","last_peered":"2024-05-20T20:53:36.845542+0000","last_clean":"2024-05-20T20:53:36.845542+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:53:36.845542+0000","last_undegraded":"2024-05-20T20:53:36.845542+0000","last_fullsized":"2024-05-20T20:53:36.845542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.848483+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:53:36.848483+0000","last_peered":"2024-05-20T20:53:36.848483+0000","last_clean":"2024-05-20T20:53:36.848483+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:53:36.848483+0000","last_undegraded":"2024-05-20T20:53:36.848483+0000","last_fullsized":"2024-05-20T20:53:36.848483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.849374+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:53:36.849374+0000","last_peered":"2024-05-20T20:53:36.849374+0000","last_clean":"2024-05-20T20:53:36.849374+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:53:36.849374+0000","last_undegraded":"2024-05-20T20:53:36.849374+0000","last_fullsized":"2024-05-20T20:53:36.849374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.719657+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:35.719657+0000","last_peered":"2024-05-20T20:54:35.719657+0000","last_clean":"2024-05-20T20:54:35.719657+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:35.719657+0000","last_undegraded":"2024-05-20T20:54:35.719657+0000","last_fullsized":"2024-05-20T20:54:35.719657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875210+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:53:36.875210+0000","last_peered":"2024-05-20T20:53:36.875210+0000","last_clean":"2024-05-20T20:53:36.875210+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:53:36.875210+0000","last_undegraded":"2024-05-20T20:53:36.875210+0000","last_fullsized":"2024-05-20T20:53:36.875210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845742+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:53:36.845742+0000","last_peered":"2024-05-20T20:53:36.845742+0000","last_clean":"2024-05-20T20:53:36.845742+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:53:36.845742+0000","last_undegraded":"2024-05-20T20:53:36.845742+0000","last_fullsized":"2024-05-20T20:53:36.845742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.850221+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:53:36.850221+0000","last_peered":"2024-05-20T20:53:36.850221+0000","last_clean":"2024-05-20T20:53:36.850221+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:53:36.850221+0000","last_undegraded":"2024-05-20T20:53:36.850221+0000","last_fullsized":"2024-05-20T20:53:36.850221+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.717256+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:35.717256+0000","last_peered":"2024-05-20T20:54:35.717256+0000","last_clean":"2024-05-20T20:54:35.717256+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:35.717256+0000","last_undegraded":"2024-05-20T20:54:35.717256+0000","last_fullsized":"2024-05-20T20:54:35.717256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.842841+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:53:36.842841+0000","last_peered":"2024-05-20T20:53:36.842841+0000","last_clean":"2024-05-20T20:53:36.842841+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:53:36.842841+0000","last_undegraded":"2024-05-20T20:53:36.842841+0000","last_fullsized":"2024-05-20T20:53:36.842841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.713384+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:35.713384+0000","last_peered":"2024-05-20T20:54:35.713384+0000","last_clean":"2024-05-20T20:54:35.713384+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:35.713384+0000","last_undegraded":"2024-05-20T20:54:35.713384+0000","last_fullsized":"2024-05-20T20:54:35.713384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844705+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:53:36.844705+0000","last_peered":"2024-05-20T20:53:36.844705+0000","last_clean":"2024-05-20T20:53:36.844705+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:53:36.844705+0000","last_undegraded":"2024-05-20T20:53:36.844705+0000","last_fullsized":"2024-05-20T20:53:36.844705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875721+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:53:36.875721+0000","last_peered":"2024-05-20T20:53:36.875721+0000","last_clean":"2024-05-20T20:53:36.875721+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:53:36.875721+0000","last_undegraded":"2024-05-20T20:53:36.875721+0000","last_fullsized":"2024-05-20T20:53:36.875721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844979+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:53:36.844979+0000","last_peered":"2024-05-20T20:53:36.844979+0000","last_clean":"2024-05-20T20:53:36.844979+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:53:36.844979+0000","last_undegraded":"2024-05-20T20:53:36.844979+0000","last_fullsized":"2024-05-20T20:53:36.844979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65827,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":575,"num_read_kb":357,"num_write":163,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710757,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":425296,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1078994,"data_compressed":42470,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425532,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266740,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":425452,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":1017382,"data_compressed":42470,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":424921,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":486540,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426192,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":487778,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":884736,"data_stored":425232,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":425861,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":1016675,"data_compressed":42470,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:54:37.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:54:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:38.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]': finished 2024-05-20T20:54:38.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:37 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-20T20:54:38.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:37 smithi055 ceph-mon[35213]: osdmap e81: 12 total, 12 up, 12 in 2024-05-20T20:54:38.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:37 smithi055 ceph-mon[35213]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:54:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]': finished 2024-05-20T20:54:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:37 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-20T20:54:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:37 smithi026 ceph-mon[31990]: osdmap e81: 12 total, 12 up, 12 in 2024-05-20T20:54:38.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:37 smithi026 ceph-mon[31990]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:54:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]': finished 2024-05-20T20:54:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:37 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-20T20:54:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:37 smithi144 ceph-mon[35660]: osdmap e81: 12 total, 12 up, 12 in 2024-05-20T20:54:38.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:37 smithi144 ceph-mon[35660]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:54:39.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:38 smithi026 ceph-mon[31990]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-20T20:54:39.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:38 smithi026 ceph-mon[31990]: osdmap e82: 12 total, 12 up, 12 in 2024-05-20T20:54:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:38 smithi055 ceph-mon[35213]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-20T20:54:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:38 smithi055 ceph-mon[35213]: osdmap e82: 12 total, 12 up, 12 in 2024-05-20T20:54:39.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:38 smithi144 ceph-mon[35660]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-20T20:54:39.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:38 smithi144 ceph-mon[35660]: osdmap e82: 12 total, 12 up, 12 in 2024-05-20T20:54:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 -- 172.21.15.26:0/1043280833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8103850 msgr2=0x7fadd8103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1043280833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8103850 0x7fadd8103c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fadc80099c0 tx=0x7fadc8009cd0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 -- 172.21.15.26:0/1043280833 shutdown_connections 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1043280833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8103850 0x7fadd8103c30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1043280833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 0x7fadd81104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1043280833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8104200 0x7fadd81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 -- 172.21.15.26:0/1043280833 >> 172.21.15.26:0/1043280833 conn(0x7fadd807a620 msgr2=0x7fadd807aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 -- 172.21.15.26:0/1043280833 shutdown_connections 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.703+0000 7fadde0d1700 1 -- 172.21.15.26:0/1043280833 wait complete. 2024-05-20T20:54:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 Processor -- start 2024-05-20T20:54:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 -- start start 2024-05-20T20:54:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8103850 0x7fadd819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 0x7fadd819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 0x7fadd8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81135b0 con 0x7fadd8104200 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8113430 con 0x7fadd8103850 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadde0d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8113730 con 0x7fadd8108920 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 0x7fadd819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 0x7fadd819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33834/0 (socket says 172.21.15.26:33834) 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadd6ffd700 1 -- 172.21.15.26:0/1389870985 learned_addr learned my addr 172.21.15.26:0/1389870985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.704+0000 7fadd77fe700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8103850 0x7fadd819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd7fff700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 0x7fadd8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd6ffd700 1 -- 172.21.15.26:0/1389870985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 msgr2=0x7fadd8199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd6ffd700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 0x7fadd8199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd6ffd700 1 -- 172.21.15.26:0/1389870985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8103850 msgr2=0x7fadd819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd6ffd700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8103850 0x7fadd819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd6ffd700 1 -- 172.21.15.26:0/1389870985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd8199fd0 con 0x7fadd8104200 2024-05-20T20:54:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd7fff700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 0x7fadd8199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd6ffd700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 0x7fadd819fbf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c9b0 tx=0x7fadc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0007b00 con 0x7fadd8104200 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadc0007ca0 con 0x7fadd8104200 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd819a2c0 con 0x7fadd8104200 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0018660 con 0x7fadd8104200 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.705+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd81a6580 con 0x7fadd8104200 2024-05-20T20:54:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.706+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd810bb60 con 0x7fadd8104200 2024-05-20T20:54:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.708+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fadc000f450 con 0x7fadd8104200 2024-05-20T20:54:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.708+0000 7fadd4ff9700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fadc40b2900 0x7fadc40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.708+0000 7fadd77fe700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fadc40b2900 0x7fadc40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.709+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fadc00e2440 con 0x7fadd8104200 2024-05-20T20:54:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.709+0000 7fadd77fe700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fadc40b2900 0x7fadc40b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009990 tx=0x7fadc801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.712+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadc0014030 con 0x7fadd8104200 2024-05-20T20:54:39.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.924+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadd8063980 con 0x7fadc40b2900 2024-05-20T20:54:39.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.932+0000 7fadd4ff9700 1 -- 172.21.15.26:0/1389870985 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259154 (secure 0 0 0) 0x7fadd8063980 con 0x7fadc40b2900 2024-05-20T20:54:39.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fadc40b2900 msgr2=0x7fadc40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fadc40b2900 0x7fadc40b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009990 tx=0x7fadc801a040 comp rx=0 tx=0).stop 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 msgr2=0x7fadd819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 0x7fadd819fbf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c9b0 tx=0x7fadc000ccc0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 shutdown_connections 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd8108920 0x7fadd8199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8103850 0x7fadd819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fadc40b2900 0x7fadc40b4dc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 --2- 172.21.15.26:0/1389870985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd8104200 0x7fadd819fbf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:39.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 >> 172.21.15.26:0/1389870985 conn(0x7fadd807a620 msgr2=0x7fadd8109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:39.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 shutdown_connections 2024-05-20T20:54:39.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:39.935+0000 7fadde0d1700 1 -- 172.21.15.26:0/1389870985 wait complete. 2024-05-20T20:54:39.938 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:54:40.637 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-05-20T20:54:39.723387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656819,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":683,"num_read_kb":438,"num_write":295,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":612,"ondisk_log_size":612,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355756,"kb_used_data":10728,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586068,"statfs":{"total":1151940427776,"available":1151576133632,"internally_reserved":0,"allocated":10985472,"data_stored":7118202,"data_compressed":196861,"data_compressed_allocated":258048,"data_compressed_original":2015232,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002329"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844798+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:53:36.844798+0000","last_peered":"2024-05-20T20:53:36.844798+0000","last_clean":"2024-05-20T20:53:36.844798+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:53:36.844798+0000","last_undegraded":"2024-05-20T20:53:36.844798+0000","last_fullsized":"2024-05-20T20:53:36.844798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.706594+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:35.706594+0000","last_peered":"2024-05-20T20:54:35.706594+0000","last_clean":"2024-05-20T20:54:35.706594+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:35.706594+0000","last_undegraded":"2024-05-20T20:54:35.706594+0000","last_fullsized":"2024-05-20T20:54:35.706594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.846259+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:53:36.846259+0000","last_peered":"2024-05-20T20:53:36.846259+0000","last_clean":"2024-05-20T20:53:36.846259+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:53:36.846259+0000","last_undegraded":"2024-05-20T20:53:36.846259+0000","last_fullsized":"2024-05-20T20:53:36.846259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.716651+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:35.716651+0000","last_peered":"2024-05-20T20:54:35.716651+0000","last_clean":"2024-05-20T20:54:35.716651+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:35.716651+0000","last_undegraded":"2024-05-20T20:54:35.716651+0000","last_fullsized":"2024-05-20T20:54:35.716651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.668688+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:37.668688+0000","last_peered":"2024-05-20T20:54:37.668688+0000","last_clean":"2024-05-20T20:54:37.668688+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:37.668688+0000","last_undegraded":"2024-05-20T20:54:37.668688+0000","last_fullsized":"2024-05-20T20:54:37.668688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876404+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876404+0000","last_peered":"2024-05-20T20:54:37.876404+0000","last_clean":"2024-05-20T20:54:37.876404+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876404+0000","last_undegraded":"2024-05-20T20:54:37.876404+0000","last_fullsized":"2024-05-20T20:54:37.876404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870628+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870628+0000","last_peered":"2024-05-20T20:54:37.870628+0000","last_clean":"2024-05-20T20:54:37.870628+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870628+0000","last_undegraded":"2024-05-20T20:54:37.870628+0000","last_fullsized":"2024-05-20T20:54:37.870628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845129+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:53:36.845129+0000","last_peered":"2024-05-20T20:53:36.845129+0000","last_clean":"2024-05-20T20:53:36.845129+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:53:36.845129+0000","last_undegraded":"2024-05-20T20:53:36.845129+0000","last_fullsized":"2024-05-20T20:53:36.845129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876662+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876662+0000","last_peered":"2024-05-20T20:54:37.876662+0000","last_clean":"2024-05-20T20:54:37.876662+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876662+0000","last_undegraded":"2024-05-20T20:54:37.876662+0000","last_fullsized":"2024-05-20T20:54:37.876662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871092+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871092+0000","last_peered":"2024-05-20T20:54:37.871092+0000","last_clean":"2024-05-20T20:54:37.871092+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871092+0000","last_undegraded":"2024-05-20T20:54:37.871092+0000","last_fullsized":"2024-05-20T20:54:37.871092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.716810+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:35.716810+0000","last_peered":"2024-05-20T20:54:35.716810+0000","last_clean":"2024-05-20T20:54:35.716810+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:35.716810+0000","last_undegraded":"2024-05-20T20:54:35.716810+0000","last_fullsized":"2024-05-20T20:54:35.716810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.858033+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:36.858033+0000","last_peered":"2024-05-20T20:54:36.858033+0000","last_clean":"2024-05-20T20:54:36.858033+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:36.858033+0000","last_undegraded":"2024-05-20T20:54:36.858033+0000","last_fullsized":"2024-05-20T20:54:36.858033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.723859+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:35.723859+0000","last_peered":"2024-05-20T20:54:35.723859+0000","last_clean":"2024-05-20T20:54:35.723859+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:35.723859+0000","last_undegraded":"2024-05-20T20:54:35.723859+0000","last_fullsized":"2024-05-20T20:54:35.723859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.721167+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:35.721167+0000","last_peered":"2024-05-20T20:54:35.721167+0000","last_clean":"2024-05-20T20:54:35.721167+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:35.721167+0000","last_undegraded":"2024-05-20T20:54:35.721167+0000","last_fullsized":"2024-05-20T20:54:35.721167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844391+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:53:36.844391+0000","last_peered":"2024-05-20T20:53:36.844391+0000","last_clean":"2024-05-20T20:53:36.844391+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:53:36.844391+0000","last_undegraded":"2024-05-20T20:53:36.844391+0000","last_fullsized":"2024-05-20T20:53:36.844391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845448+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:53:36.845448+0000","last_peered":"2024-05-20T20:53:36.845448+0000","last_clean":"2024-05-20T20:53:36.845448+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:53:36.845448+0000","last_undegraded":"2024-05-20T20:53:36.845448+0000","last_fullsized":"2024-05-20T20:53:36.845448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872551+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872551+0000","last_peered":"2024-05-20T20:54:37.872551+0000","last_clean":"2024-05-20T20:54:37.872551+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872551+0000","last_undegraded":"2024-05-20T20:54:37.872551+0000","last_fullsized":"2024-05-20T20:54:37.872551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.651041+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:37.651041+0000","last_peered":"2024-05-20T20:54:37.651041+0000","last_clean":"2024-05-20T20:54:37.651041+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:37.651041+0000","last_undegraded":"2024-05-20T20:54:37.651041+0000","last_fullsized":"2024-05-20T20:54:37.651041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872652+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:37.872652+0000","last_peered":"2024-05-20T20:54:37.872652+0000","last_clean":"2024-05-20T20:54:37.872652+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:37.872652+0000","last_undegraded":"2024-05-20T20:54:37.872652+0000","last_fullsized":"2024-05-20T20:54:37.872652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.715169+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:35.715169+0000","last_peered":"2024-05-20T20:54:35.715169+0000","last_clean":"2024-05-20T20:54:35.715169+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:35.715169+0000","last_undegraded":"2024-05-20T20:54:35.715169+0000","last_fullsized":"2024-05-20T20:54:35.715169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.843126+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:53:36.843126+0000","last_peered":"2024-05-20T20:53:36.843126+0000","last_clean":"2024-05-20T20:53:36.843126+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:53:36.843126+0000","last_undegraded":"2024-05-20T20:53:36.843126+0000","last_fullsized":"2024-05-20T20:53:36.843126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.846261+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:53:36.846261+0000","last_peered":"2024-05-20T20:53:36.846261+0000","last_clean":"2024-05-20T20:53:36.846261+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:53:36.846261+0000","last_undegraded":"2024-05-20T20:53:36.846261+0000","last_fullsized":"2024-05-20T20:53:36.846261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.669363+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:37.669363+0000","last_peered":"2024-05-20T20:54:37.669363+0000","last_clean":"2024-05-20T20:54:37.669363+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:37.669363+0000","last_undegraded":"2024-05-20T20:54:37.669363+0000","last_fullsized":"2024-05-20T20:54:37.669363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844604+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:53:36.844604+0000","last_peered":"2024-05-20T20:53:36.844604+0000","last_clean":"2024-05-20T20:53:36.844604+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:53:36.844604+0000","last_undegraded":"2024-05-20T20:53:36.844604+0000","last_fullsized":"2024-05-20T20:53:36.844604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.669339+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:37.669339+0000","last_peered":"2024-05-20T20:54:37.669339+0000","last_clean":"2024-05-20T20:54:37.669339+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:37.669339+0000","last_undegraded":"2024-05-20T20:54:37.669339+0000","last_fullsized":"2024-05-20T20:54:37.669339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.866950+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:36.866950+0000","last_peered":"2024-05-20T20:54:36.866950+0000","last_clean":"2024-05-20T20:54:36.866950+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:36.866950+0000","last_undegraded":"2024-05-20T20:54:36.866950+0000","last_fullsized":"2024-05-20T20:54:36.866950+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.866840+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:54:36.866840+0000","last_peered":"2024-05-20T20:54:36.866840+0000","last_clean":"2024-05-20T20:54:36.866840+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:54:36.866840+0000","last_undegraded":"2024-05-20T20:54:36.866840+0000","last_fullsized":"2024-05-20T20:54:36.866840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.669096+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:37.669096+0000","last_peered":"2024-05-20T20:54:37.669096+0000","last_clean":"2024-05-20T20:54:37.669096+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:37.669096+0000","last_undegraded":"2024-05-20T20:54:37.669096+0000","last_fullsized":"2024-05-20T20:54:37.669096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876234+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876234+0000","last_peered":"2024-05-20T20:54:37.876234+0000","last_clean":"2024-05-20T20:54:37.876234+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876234+0000","last_undegraded":"2024-05-20T20:54:37.876234+0000","last_fullsized":"2024-05-20T20:54:37.876234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.867135+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:36.867135+0000","last_peered":"2024-05-20T20:54:36.867135+0000","last_clean":"2024-05-20T20:54:36.867135+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:36.867135+0000","last_undegraded":"2024-05-20T20:54:36.867135+0000","last_fullsized":"2024-05-20T20:54:36.867135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857740+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:36.857740+0000","last_peered":"2024-05-20T20:54:36.857740+0000","last_clean":"2024-05-20T20:54:36.857740+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:36.857740+0000","last_undegraded":"2024-05-20T20:54:36.857740+0000","last_fullsized":"2024-05-20T20:54:36.857740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.713824+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:35.713824+0000","last_peered":"2024-05-20T20:54:35.713824+0000","last_clean":"2024-05-20T20:54:35.713824+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:35.713824+0000","last_undegraded":"2024-05-20T20:54:35.713824+0000","last_fullsized":"2024-05-20T20:54:35.713824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.867365+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:36.867365+0000","last_peered":"2024-05-20T20:54:36.867365+0000","last_clean":"2024-05-20T20:54:36.867365+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:36.867365+0000","last_undegraded":"2024-05-20T20:54:36.867365+0000","last_fullsized":"2024-05-20T20:54:36.867365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872006+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872006+0000","last_peered":"2024-05-20T20:54:37.872006+0000","last_clean":"2024-05-20T20:54:37.872006+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872006+0000","last_undegraded":"2024-05-20T20:54:37.872006+0000","last_fullsized":"2024-05-20T20:54:37.872006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.468522+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:54:38.468522+0000","last_peered":"2024-05-20T20:54:38.468522+0000","last_clean":"2024-05-20T20:54:38.468522+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:54:38.468522+0000","last_undegraded":"2024-05-20T20:54:38.468522+0000","last_fullsized":"2024-05-20T20:54:38.468522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845039+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:53:36.845039+0000","last_peered":"2024-05-20T20:53:36.845039+0000","last_clean":"2024-05-20T20:53:36.845039+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:53:36.845039+0000","last_undegraded":"2024-05-20T20:53:36.845039+0000","last_fullsized":"2024-05-20T20:53:36.845039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871005+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871005+0000","last_peered":"2024-05-20T20:54:37.871005+0000","last_clean":"2024-05-20T20:54:37.871005+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871005+0000","last_undegraded":"2024-05-20T20:54:37.871005+0000","last_fullsized":"2024-05-20T20:54:37.871005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.720148+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:35.720148+0000","last_peered":"2024-05-20T20:54:35.720148+0000","last_clean":"2024-05-20T20:54:35.720148+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:35.720148+0000","last_undegraded":"2024-05-20T20:54:35.720148+0000","last_fullsized":"2024-05-20T20:54:35.720148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.867065+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:36.867065+0000","last_peered":"2024-05-20T20:54:36.867065+0000","last_clean":"2024-05-20T20:54:36.867065+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:36.867065+0000","last_undegraded":"2024-05-20T20:54:36.867065+0000","last_fullsized":"2024-05-20T20:54:36.867065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:53:36.844712+0000","last_peered":"2024-05-20T20:53:36.844712+0000","last_clean":"2024-05-20T20:53:36.844712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:53:36.844712+0000","last_undegraded":"2024-05-20T20:53:36.844712+0000","last_fullsized":"2024-05-20T20:53:36.844712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'32","reported_seq":268,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:39.470311+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:54:39.470311+0000","last_peered":"2024-05-20T20:54:39.470311+0000","last_clean":"2024-05-20T20:54:39.470311+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:54:39.470311+0000","last_undegraded":"2024-05-20T20:54:39.470311+0000","last_fullsized":"2024-05-20T20:54:39.470311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62075,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":163,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.668914+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:37.668914+0000","last_peered":"2024-05-20T20:54:37.668914+0000","last_clean":"2024-05-20T20:54:37.668914+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:37.668914+0000","last_undegraded":"2024-05-20T20:54:37.668914+0000","last_fullsized":"2024-05-20T20:54:37.668914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.651688+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:37.651688+0000","last_peered":"2024-05-20T20:54:37.651688+0000","last_clean":"2024-05-20T20:54:37.651688+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:37.651688+0000","last_undegraded":"2024-05-20T20:54:37.651688+0000","last_fullsized":"2024-05-20T20:54:37.651688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857965+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:36.857965+0000","last_peered":"2024-05-20T20:54:36.857965+0000","last_clean":"2024-05-20T20:54:36.857965+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:36.857965+0000","last_undegraded":"2024-05-20T20:54:36.857965+0000","last_fullsized":"2024-05-20T20:54:36.857965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.866867+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:36.866867+0000","last_peered":"2024-05-20T20:54:36.866867+0000","last_clean":"2024-05-20T20:54:36.866867+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:36.866867+0000","last_undegraded":"2024-05-20T20:54:36.866867+0000","last_fullsized":"2024-05-20T20:54:36.866867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870971+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.870971+0000","last_peered":"2024-05-20T20:54:37.870971+0000","last_clean":"2024-05-20T20:54:37.870971+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.870971+0000","last_undegraded":"2024-05-20T20:54:37.870971+0000","last_fullsized":"2024-05-20T20:54:37.870971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.466584+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:54:36.466584+0000","last_peered":"2024-05-20T20:54:36.466584+0000","last_clean":"2024-05-20T20:54:36.466584+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:54:36.466584+0000","last_undegraded":"2024-05-20T20:54:36.466584+0000","last_fullsized":"2024-05-20T20:54:36.466584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870534+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870534+0000","last_peered":"2024-05-20T20:54:37.870534+0000","last_clean":"2024-05-20T20:54:37.870534+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870534+0000","last_undegraded":"2024-05-20T20:54:37.870534+0000","last_fullsized":"2024-05-20T20:54:37.870534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.668918+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:37.668918+0000","last_peered":"2024-05-20T20:54:37.668918+0000","last_clean":"2024-05-20T20:54:37.668918+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:37.668918+0000","last_undegraded":"2024-05-20T20:54:37.668918+0000","last_fullsized":"2024-05-20T20:54:37.668918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.867109+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:36.867109+0000","last_peered":"2024-05-20T20:54:36.867109+0000","last_clean":"2024-05-20T20:54:36.867109+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:36.867109+0000","last_undegraded":"2024-05-20T20:54:36.867109+0000","last_fullsized":"2024-05-20T20:54:36.867109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.669351+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:37.669351+0000","last_peered":"2024-05-20T20:54:37.669351+0000","last_clean":"2024-05-20T20:54:37.669351+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:37.669351+0000","last_undegraded":"2024-05-20T20:54:37.669351+0000","last_fullsized":"2024-05-20T20:54:37.669351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.876597+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:53:36.876597+0000","last_peered":"2024-05-20T20:53:36.876597+0000","last_clean":"2024-05-20T20:53:36.876597+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:53:36.876597+0000","last_undegraded":"2024-05-20T20:53:36.876597+0000","last_fullsized":"2024-05-20T20:53:36.876597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872114+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872114+0000","last_peered":"2024-05-20T20:54:37.872114+0000","last_clean":"2024-05-20T20:54:37.872114+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872114+0000","last_undegraded":"2024-05-20T20:54:37.872114+0000","last_fullsized":"2024-05-20T20:54:37.872114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876396+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876396+0000","last_peered":"2024-05-20T20:54:37.876396+0000","last_clean":"2024-05-20T20:54:37.876396+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876396+0000","last_undegraded":"2024-05-20T20:54:37.876396+0000","last_fullsized":"2024-05-20T20:54:37.876396+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871328+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871328+0000","last_peered":"2024-05-20T20:54:37.871328+0000","last_clean":"2024-05-20T20:54:37.871328+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871328+0000","last_undegraded":"2024-05-20T20:54:37.871328+0000","last_fullsized":"2024-05-20T20:54:37.871328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875700+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875700+0000","last_peered":"2024-05-20T20:54:37.875700+0000","last_clean":"2024-05-20T20:54:37.875700+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875700+0000","last_undegraded":"2024-05-20T20:54:37.875700+0000","last_fullsized":"2024-05-20T20:54:37.875700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872140+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872140+0000","last_peered":"2024-05-20T20:54:37.872140+0000","last_clean":"2024-05-20T20:54:37.872140+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872140+0000","last_undegraded":"2024-05-20T20:54:37.872140+0000","last_fullsized":"2024-05-20T20:54:37.872140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":128,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857603+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:54:36.857603+0000","last_peered":"2024-05-20T20:54:36.857603+0000","last_clean":"2024-05-20T20:54:36.857603+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:54:36.857603+0000","last_undegraded":"2024-05-20T20:54:36.857603+0000","last_fullsized":"2024-05-20T20:54:36.857603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.728775+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:35.728775+0000","last_peered":"2024-05-20T20:54:35.728775+0000","last_clean":"2024-05-20T20:54:35.728775+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:35.728775+0000","last_undegraded":"2024-05-20T20:54:35.728775+0000","last_fullsized":"2024-05-20T20:54:35.728775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875606+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:53:36.875606+0000","last_peered":"2024-05-20T20:53:36.875606+0000","last_clean":"2024-05-20T20:53:36.875606+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:53:36.875606+0000","last_undegraded":"2024-05-20T20:53:36.875606+0000","last_fullsized":"2024-05-20T20:53:36.875606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870715+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870715+0000","last_peered":"2024-05-20T20:54:37.870715+0000","last_clean":"2024-05-20T20:54:37.870715+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870715+0000","last_undegraded":"2024-05-20T20:54:37.870715+0000","last_fullsized":"2024-05-20T20:54:37.870715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857595+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:36.857595+0000","last_peered":"2024-05-20T20:54:36.857595+0000","last_clean":"2024-05-20T20:54:36.857595+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:36.857595+0000","last_undegraded":"2024-05-20T20:54:36.857595+0000","last_fullsized":"2024-05-20T20:54:36.857595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870842+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870842+0000","last_peered":"2024-05-20T20:54:37.870842+0000","last_clean":"2024-05-20T20:54:37.870842+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870842+0000","last_undegraded":"2024-05-20T20:54:37.870842+0000","last_fullsized":"2024-05-20T20:54:37.870842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.669320+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:37.669320+0000","last_peered":"2024-05-20T20:54:37.669320+0000","last_clean":"2024-05-20T20:54:37.669320+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:37.669320+0000","last_undegraded":"2024-05-20T20:54:37.669320+0000","last_fullsized":"2024-05-20T20:54:37.669320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857814+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:36.857814+0000","last_peered":"2024-05-20T20:54:36.857814+0000","last_clean":"2024-05-20T20:54:36.857814+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:36.857814+0000","last_undegraded":"2024-05-20T20:54:36.857814+0000","last_fullsized":"2024-05-20T20:54:36.857814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872485+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:54:37.872485+0000","last_peered":"2024-05-20T20:54:37.872485+0000","last_clean":"2024-05-20T20:54:37.872485+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:54:37.872485+0000","last_undegraded":"2024-05-20T20:54:37.872485+0000","last_fullsized":"2024-05-20T20:54:37.872485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.724677+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:35.724677+0000","last_peered":"2024-05-20T20:54:35.724677+0000","last_clean":"2024-05-20T20:54:35.724677+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:35.724677+0000","last_undegraded":"2024-05-20T20:54:35.724677+0000","last_fullsized":"2024-05-20T20:54:35.724677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.875126+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:53:36.875126+0000","last_peered":"2024-05-20T20:53:36.875126+0000","last_clean":"2024-05-20T20:53:36.875126+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:53:36.875126+0000","last_undegraded":"2024-05-20T20:53:36.875126+0000","last_fullsized":"2024-05-20T20:53:36.875126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.724229+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:35.724229+0000","last_peered":"2024-05-20T20:54:35.724229+0000","last_clean":"2024-05-20T20:54:35.724229+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:35.724229+0000","last_undegraded":"2024-05-20T20:54:35.724229+0000","last_fullsized":"2024-05-20T20:54:35.724229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844410+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:53:36.844410+0000","last_peered":"2024-05-20T20:53:36.844410+0000","last_clean":"2024-05-20T20:53:36.844410+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:53:36.844410+0000","last_undegraded":"2024-05-20T20:53:36.844410+0000","last_fullsized":"2024-05-20T20:53:36.844410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.668641+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:37.668641+0000","last_peered":"2024-05-20T20:54:37.668641+0000","last_clean":"2024-05-20T20:54:37.668641+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:37.668641+0000","last_undegraded":"2024-05-20T20:54:37.668641+0000","last_fullsized":"2024-05-20T20:54:37.668641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.842436+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:53:36.842436+0000","last_peered":"2024-05-20T20:53:36.842436+0000","last_clean":"2024-05-20T20:53:36.842436+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:53:36.842436+0000","last_undegraded":"2024-05-20T20:53:36.842436+0000","last_fullsized":"2024-05-20T20:53:36.842436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845428+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:53:36.845428+0000","last_peered":"2024-05-20T20:53:36.845428+0000","last_clean":"2024-05-20T20:53:36.845428+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:53:36.845428+0000","last_undegraded":"2024-05-20T20:53:36.845428+0000","last_fullsized":"2024-05-20T20:53:36.845428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.854521+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:53:36.854521+0000","last_peered":"2024-05-20T20:53:36.854521+0000","last_clean":"2024-05-20T20:53:36.854521+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:53:36.854521+0000","last_undegraded":"2024-05-20T20:53:36.854521+0000","last_fullsized":"2024-05-20T20:53:36.854521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.719046+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:35.719046+0000","last_peered":"2024-05-20T20:54:35.719046+0000","last_clean":"2024-05-20T20:54:35.719046+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:35.719046+0000","last_undegraded":"2024-05-20T20:54:35.719046+0000","last_fullsized":"2024-05-20T20:54:35.719046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.858684+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:53:36.858684+0000","last_peered":"2024-05-20T20:53:36.858684+0000","last_clean":"2024-05-20T20:53:36.858684+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:53:36.858684+0000","last_undegraded":"2024-05-20T20:53:36.858684+0000","last_fullsized":"2024-05-20T20:53:36.858684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876093+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.876093+0000","last_peered":"2024-05-20T20:54:37.876093+0000","last_clean":"2024-05-20T20:54:37.876093+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.876093+0000","last_undegraded":"2024-05-20T20:54:37.876093+0000","last_fullsized":"2024-05-20T20:54:37.876093+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.651839+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:37.651839+0000","last_peered":"2024-05-20T20:54:37.651839+0000","last_clean":"2024-05-20T20:54:37.651839+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:37.651839+0000","last_undegraded":"2024-05-20T20:54:37.651839+0000","last_fullsized":"2024-05-20T20:54:37.651839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871226+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871226+0000","last_peered":"2024-05-20T20:54:37.871226+0000","last_clean":"2024-05-20T20:54:37.871226+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871226+0000","last_undegraded":"2024-05-20T20:54:37.871226+0000","last_fullsized":"2024-05-20T20:54:37.871226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.866887+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:36.866887+0000","last_peered":"2024-05-20T20:54:36.866887+0000","last_clean":"2024-05-20T20:54:36.866887+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:36.866887+0000","last_undegraded":"2024-05-20T20:54:36.866887+0000","last_fullsized":"2024-05-20T20:54:36.866887+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870908+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870908+0000","last_peered":"2024-05-20T20:54:37.870908+0000","last_clean":"2024-05-20T20:54:37.870908+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870908+0000","last_undegraded":"2024-05-20T20:54:37.870908+0000","last_fullsized":"2024-05-20T20:54:37.870908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857936+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:36.857936+0000","last_peered":"2024-05-20T20:54:36.857936+0000","last_clean":"2024-05-20T20:54:36.857936+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:36.857936+0000","last_undegraded":"2024-05-20T20:54:36.857936+0000","last_fullsized":"2024-05-20T20:54:36.857936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871214+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871214+0000","last_peered":"2024-05-20T20:54:37.871214+0000","last_clean":"2024-05-20T20:54:37.871214+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871214+0000","last_undegraded":"2024-05-20T20:54:37.871214+0000","last_fullsized":"2024-05-20T20:54:37.871214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845542+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:53:36.845542+0000","last_peered":"2024-05-20T20:53:36.845542+0000","last_clean":"2024-05-20T20:53:36.845542+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:53:36.845542+0000","last_undegraded":"2024-05-20T20:53:36.845542+0000","last_fullsized":"2024-05-20T20:53:36.845542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.650945+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:37.650945+0000","last_peered":"2024-05-20T20:54:37.650945+0000","last_clean":"2024-05-20T20:54:37.650945+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:37.650945+0000","last_undegraded":"2024-05-20T20:54:37.650945+0000","last_fullsized":"2024-05-20T20:54:37.650945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.650952+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:37.650952+0000","last_peered":"2024-05-20T20:54:37.650952+0000","last_clean":"2024-05-20T20:54:37.650952+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:37.650952+0000","last_undegraded":"2024-05-20T20:54:37.650952+0000","last_fullsized":"2024-05-20T20:54:37.650952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.719657+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:35.719657+0000","last_peered":"2024-05-20T20:54:35.719657+0000","last_clean":"2024-05-20T20:54:35.719657+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:35.719657+0000","last_undegraded":"2024-05-20T20:54:35.719657+0000","last_fullsized":"2024-05-20T20:54:35.719657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.857729+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:36.857729+0000","last_peered":"2024-05-20T20:54:36.857729+0000","last_clean":"2024-05-20T20:54:36.857729+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:36.857729+0000","last_undegraded":"2024-05-20T20:54:36.857729+0000","last_fullsized":"2024-05-20T20:54:36.857729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.845742+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:53:36.845742+0000","last_peered":"2024-05-20T20:53:36.845742+0000","last_clean":"2024-05-20T20:53:36.845742+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:53:36.845742+0000","last_undegraded":"2024-05-20T20:53:36.845742+0000","last_fullsized":"2024-05-20T20:53:36.845742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.651087+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:37.651087+0000","last_peered":"2024-05-20T20:54:37.651087+0000","last_clean":"2024-05-20T20:54:37.651087+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:37.651087+0000","last_undegraded":"2024-05-20T20:54:37.651087+0000","last_fullsized":"2024-05-20T20:54:37.651087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.717256+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:35.717256+0000","last_peered":"2024-05-20T20:54:35.717256+0000","last_clean":"2024-05-20T20:54:35.717256+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:35.717256+0000","last_undegraded":"2024-05-20T20:54:35.717256+0000","last_fullsized":"2024-05-20T20:54:35.717256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.867382+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:36.867382+0000","last_peered":"2024-05-20T20:54:36.867382+0000","last_clean":"2024-05-20T20:54:36.867382+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:36.867382+0000","last_undegraded":"2024-05-20T20:54:36.867382+0000","last_fullsized":"2024-05-20T20:54:36.867382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:54:35.713384+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:35.713384+0000","last_peered":"2024-05-20T20:54:35.713384+0000","last_clean":"2024-05-20T20:54:35.713384+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:35.713384+0000","last_undegraded":"2024-05-20T20:54:35.713384+0000","last_fullsized":"2024-05-20T20:54:35.713384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844705+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:53:36.844705+0000","last_peered":"2024-05-20T20:53:36.844705+0000","last_clean":"2024-05-20T20:53:36.844705+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:53:36.844705+0000","last_undegraded":"2024-05-20T20:53:36.844705+0000","last_fullsized":"2024-05-20T20:53:36.844705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-20T20:54:36.858074+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:36.858074+0000","last_peered":"2024-05-20T20:54:36.858074+0000","last_clean":"2024-05-20T20:54:36.858074+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:36.858074+0000","last_undegraded":"2024-05-20T20:54:36.858074+0000","last_fullsized":"2024-05-20T20:54:36.858074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-20T20:53:36.844979+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:53:36.844979+0000","last_peered":"2024-05-20T20:53:36.844979+0000","last_clean":"2024-05-20T20:53:36.844979+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:53:36.844979+0000","last_undegraded":"2024-05-20T20:53:36.844979+0000","last_fullsized":"2024-05-20T20:53:36.844979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":376,"ondisk_log_size":376,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66217,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":607,"num_read_kb":374,"num_write":170,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710757,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":425296,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":1096512,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266741,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":425452,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":1026108,"data_compressed":45371,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":424921,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":486540,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426192,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":488027,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":884736,"data_stored":425232,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":426002,"data_compressed":5781,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":1025260,"data_compressed":45371,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:54:40.641 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-20T20:54:40.641 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-20T20:54:40.642 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-20T20:54:40.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-20T20:54:40.901 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:41.119 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:40 smithi026 ceph-mon[31990]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-20T20:54:41.119 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:40 smithi026 ceph-mon[31990]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:54:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:40 smithi055 ceph-mon[35213]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-20T20:54:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:40 smithi055 ceph-mon[35213]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:54:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:40 smithi144 ceph-mon[35660]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-20T20:54:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:40 smithi144 ceph-mon[35660]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:54:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 -- 172.21.15.26:0/672591701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 msgr2=0x7f06c01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 --2- 172.21.15.26:0/672591701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c01081c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f06b40099c0 tx=0x7f06b4009cd0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 -- 172.21.15.26:0/672591701 shutdown_connections 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 --2- 172.21.15.26:0/672591701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06c01088f0 0x7f06c0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 --2- 172.21.15.26:0/672591701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c01037f0 0x7f06c0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 --2- 172.21.15.26:0/672591701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c01081c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 -- 172.21.15.26:0/672591701 >> 172.21.15.26:0/672591701 conn(0x7f06c007a6a0 msgr2=0x7f06c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 -- 172.21.15.26:0/672591701 shutdown_connections 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.588+0000 7f06c7b52700 1 -- 172.21.15.26:0/672591701 wait complete. 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 Processor -- start 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 -- start start 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06c01037f0 0x7f06c019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:42.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c01088f0 0x7f06c01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0113580 con 0x7f06c01041a0 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0113400 con 0x7f06c01037f0 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c7b52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0113700 con 0x7f06c01088f0 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c50ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c50ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33860/0 (socket says 172.21.15.26:33860) 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c50ed700 1 -- 172.21.15.26:0/1348945960 learned_addr learned my addr 172.21.15.26:0/1348945960 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c58ee700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06c01037f0 0x7f06c019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c60ef700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c01088f0 0x7f06c01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.589+0000 7f06c50ed700 1 -- 172.21.15.26:0/1348945960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c01088f0 msgr2=0x7f06c01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c50ed700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c01088f0 0x7f06c01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c50ed700 1 -- 172.21.15.26:0/1348945960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06c01037f0 msgr2=0x7f06c019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c50ed700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06c01037f0 0x7f06c019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c50ed700 1 -- 172.21.15.26:0/1348945960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c019a040 con 0x7f06c01041a0 2024-05-20T20:54:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c50ed700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c019fc60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000c00 tx=0x7f06b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4026070 con 0x7f06c01041a0 2024-05-20T20:54:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06b4005030 con 0x7f06c01041a0 2024-05-20T20:54:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c019a2d0 con 0x7f06c01041a0 2024-05-20T20:54:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004b70 con 0x7f06c01041a0 2024-05-20T20:54:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.590+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01a65f0 con 0x7f06c01041a0 2024-05-20T20:54:42.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.591+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c010bac0 con 0x7f06c01041a0 2024-05-20T20:54:42.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.593+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f06b4029070 con 0x7f06c01041a0 2024-05-20T20:54:42.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.594+0000 7f06beffd700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06a80b2880 0x7f06a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:42.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.594+0000 7f06c58ee700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06a80b2880 0x7f06a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:42.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.594+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f06b40e3130 con 0x7f06c01041a0 2024-05-20T20:54:42.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.595+0000 7f06c58ee700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06a80b2880 0x7f06a80b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06b00060b0 tx=0x7f06b000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:42.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.598+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06b4030020 con 0x7f06c01041a0 2024-05-20T20:54:42.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.859+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f06c0104c80 con 0x7f06c01041a0 2024-05-20T20:54:42.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.860+0000 7f06beffd700 1 -- 172.21.15.26:0/1348945960 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f06c0104c80 con 0x7f06c01041a0 2024-05-20T20:54:42.860 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:42.860 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06a80b2880 msgr2=0x7f06a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06a80b2880 0x7f06a80b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06b00060b0 tx=0x7f06b000c040 comp rx=0 tx=0).stop 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 msgr2=0x7f06c019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c019fc60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000c00 tx=0x7f06b400f690 comp rx=0 tx=0).stop 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 shutdown_connections 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c01088f0 0x7f06c01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06c01037f0 0x7f06c019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06a80b2880 0x7f06a80b4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 --2- 172.21.15.26:0/1348945960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c01041a0 0x7f06c019fc60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.862+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 >> 172.21.15.26:0/1348945960 conn(0x7f06c007a6a0 msgr2=0x7f06c0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.863+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 shutdown_connections 2024-05-20T20:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:42.863+0000 7f06c7b52700 1 -- 172.21.15.26:0/1348945960 wait complete. 2024-05-20T20:54:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:42 smithi055 ceph-mon[35213]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-20T20:54:43.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:42 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1348945960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-20T20:54:43.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:42 smithi026 ceph-mon[31990]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-20T20:54:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:42 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1348945960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-20T20:54:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:42 smithi144 ceph-mon[35660]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-20T20:54:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:42 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1348945960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-20T20:54:43.550 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-20T20:54:43.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:54:43.803 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:45.028 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:44 smithi026 ceph-mon[31990]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-20T20:54:45.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:44 smithi055 ceph-mon[35213]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-20T20:54:45.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 -- 172.21.15.26:0/1172531715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 msgr2=0x7f37801028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:45.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 --2- 172.21.15.26:0/1172531715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f37801028b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3768009990 tx=0x7f3768009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 -- 172.21.15.26:0/1172531715 shutdown_connections 2024-05-20T20:54:45.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 --2- 172.21.15.26:0/1172531715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780102df0 0x7f378010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 --2- 172.21.15.26:0/1172531715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780108510 0x7f37801088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 --2- 172.21.15.26:0/1172531715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f37801028b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.280+0000 7f378667c700 1 -- 172.21.15.26:0/1172531715 >> 172.21.15.26:0/1172531715 conn(0x7f37800793b0 msgr2=0x7f37800797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:45.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.281+0000 7f378667c700 1 -- 172.21.15.26:0/1172531715 shutdown_connections 2024-05-20T20:54:45.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.281+0000 7f378667c700 1 -- 172.21.15.26:0/1172531715 wait complete. 2024-05-20T20:54:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 Processor -- start 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 -- start start 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f3780104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780102df0 0x7f37801045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780108510 0x7f3780104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3780112470 con 0x7f3780102430 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37801122f0 con 0x7f3780102df0 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f378667c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37801125f0 con 0x7f3780108510 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f377ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f3780104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f377ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f3780104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33890/0 (socket says 172.21.15.26:33890) 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f377ffff700 1 -- 172.21.15.26:0/1686331365 learned_addr learned my addr 172.21.15.26:0/1686331365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.282+0000 7f377f7fe700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780102df0 0x7f37801045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f3784c19700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780108510 0x7f3780104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377ffff700 1 -- 172.21.15.26:0/1686331365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780108510 msgr2=0x7f3780104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377ffff700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780108510 0x7f3780104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377ffff700 1 -- 172.21.15.26:0/1686331365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780102df0 msgr2=0x7f37801045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377ffff700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780102df0 0x7f37801045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377ffff700 1 -- 172.21.15.26:0/1686331365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37801a80b0 con 0x7f3780102430 2024-05-20T20:54:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377f7fe700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780102df0 0x7f37801045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377ffff700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f3780104070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f377000d950 tx=0x7f377000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770016070 con 0x7f3780102430 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f377000fd20 con 0x7f3780102430 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37801a83a0 con 0x7f3780102430 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377000e3b0 con 0x7f3780102430 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.283+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801a88b0 con 0x7f3780102430 2024-05-20T20:54:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.284+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378006a230 con 0x7f3780102430 2024-05-20T20:54:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.286+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3770019070 con 0x7f3780102430 2024-05-20T20:54:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.286+0000 7f377d7fa700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f376c0b2880 0x7f376c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.287+0000 7f377f7fe700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f376c0b2880 0x7f376c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.287+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f37700e2060 con 0x7f3780102430 2024-05-20T20:54:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.287+0000 7f377f7fe700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f376c0b2880 0x7f376c0b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37680098e0 tx=0x7f3768005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.290+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37700a9d20 con 0x7f3780102430 2024-05-20T20:54:45.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:44 smithi144 ceph-mon[35660]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-20T20:54:45.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.554+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f378002e560 con 0x7f3780102430 2024-05-20T20:54:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.555+0000 7f377d7fa700 1 -- 172.21.15.26:0/1686331365 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f37700a9620 con 0x7f3780102430 2024-05-20T20:54:45.555 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:45.555 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:53:05.138905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T20:54:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f376c0b2880 msgr2=0x7f376c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f376c0b2880 0x7f376c0b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37680098e0 tx=0x7f3768005960 comp rx=0 tx=0).stop 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 msgr2=0x7f3780104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f3780104070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f377000d950 tx=0x7f377000dc60 comp rx=0 tx=0).stop 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 shutdown_connections 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f376c0b2880 0x7f376c0b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3780102430 0x7f3780104070 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780108510 0x7f3780104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 --2- 172.21.15.26:0/1686331365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3780102df0 0x7f37801045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.557+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 >> 172.21.15.26:0/1686331365 conn(0x7f37800793b0 msgr2=0x7f378006bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.558+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 shutdown_connections 2024-05-20T20:54:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:45.558+0000 7f378667c700 1 -- 172.21.15.26:0/1686331365 wait complete. 2024-05-20T20:54:45.559 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:54:46.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:45 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:54:46.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:45 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1686331365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:46.148 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:45 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:54:46.148 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:45 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1686331365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:46.149 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:45 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:54:46.149 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:45 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1686331365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:46.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:54:47.016 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:47.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:46 smithi026 ceph-mon[31990]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 577 B/s wr, 7 op/s 2024-05-20T20:54:47.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:47.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:46 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:47.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:46 smithi055 ceph-mon[35213]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 577 B/s wr, 7 op/s 2024-05-20T20:54:47.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:47.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:46 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:46 smithi144 ceph-mon[35660]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 577 B/s wr, 7 op/s 2024-05-20T20:54:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:46 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:47 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:47 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:47 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:47 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:47 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.744+0000 7f7d629e4700 1 -- 172.21.15.26:0/1287112103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1088f0 msgr2=0x7f7d5c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.744+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1287112103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c110470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400b0d0 tx=0x7f7d5400b3e0 comp rx=0 tx=0).stop 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.744+0000 7f7d629e4700 1 -- 172.21.15.26:0/1287112103 shutdown_connections 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.744+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1287112103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c110470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.744+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1287112103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1037f0 0x7f7d5c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.744+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1287112103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.745+0000 7f7d629e4700 1 -- 172.21.15.26:0/1287112103 >> 172.21.15.26:0/1287112103 conn(0x7f7d5c07a6a0 msgr2=0x7f7d5c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.745+0000 7f7d629e4700 1 -- 172.21.15.26:0/1287112103 shutdown_connections 2024-05-20T20:54:48.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.745+0000 7f7d629e4700 1 -- 172.21.15.26:0/1287112103 wait complete. 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.745+0000 7f7d629e4700 1 Processor -- start 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.745+0000 7f7d629e4700 1 -- start start 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d629e4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d5c1037f0 0x7f7d5c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d629e4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d629e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d629e4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1135f0 con 0x7f7d5c1041a0 2024-05-20T20:54:48.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d629e4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c113470 con 0x7f7d5c1037f0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d629e4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c113770 con 0x7f7d5c1088f0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33906/0 (socket says 172.21.15.26:33906) 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 -- 172.21.15.26:0/1524643131 learned_addr learned my addr 172.21.15.26:0/1524643131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5bfff700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d5c1037f0 0x7f7d5c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d60f81700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 -- 172.21.15.26:0/1524643131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1088f0 msgr2=0x7f7d5c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 -- 172.21.15.26:0/1524643131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d5c1037f0 msgr2=0x7f7d5c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d5c1037f0 0x7f7d5c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 -- 172.21.15.26:0/1524643131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c19a090 con 0x7f7d5c1041a0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d60f81700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d5b7fe700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c19fc60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000d950 tx=0x7f7d5000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.746+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50016070 con 0x7f7d5c1041a0 2024-05-20T20:54:48.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.747+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c19a380 con 0x7f7d5c1041a0 2024-05-20T20:54:48.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.747+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d50010460 con 0x7f7d5c1041a0 2024-05-20T20:54:48.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.747+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c0757f0 con 0x7f7d5c1041a0 2024-05-20T20:54:48.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.747+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000f620 con 0x7f7d5c1041a0 2024-05-20T20:54:48.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.748+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c06a230 con 0x7f7d5c1041a0 2024-05-20T20:54:48.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.749+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7d50019070 con 0x7f7d5c1041a0 2024-05-20T20:54:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.750+0000 7f7d597fa700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7d440b2880 0x7f7d440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.750+0000 7f7d5bfff700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7d440b2880 0x7f7d440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.750+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7d500e21f0 con 0x7f7d5c1041a0 2024-05-20T20:54:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.751+0000 7f7d5bfff700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7d440b2880 0x7f7d440b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c006fd0 tx=0x7f7d4c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:48.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:48.754+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d500a9eb0 con 0x7f7d5c1041a0 2024-05-20T20:54:49.039 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:48 smithi026 ceph-mon[31990]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-20T20:54:49.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.039+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d5c02e560 con 0x7f7d5c1041a0 2024-05-20T20:54:49.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.039+0000 7f7d597fa700 1 -- 172.21.15.26:0/1524643131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f7d500a97b0 con 0x7f7d5c1041a0 2024-05-20T20:54:49.040 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:49.040 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:53:05.138905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T20:54:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7d440b2880 msgr2=0x7f7d440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7d440b2880 0x7f7d440b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c006fd0 tx=0x7f7d4c008040 comp rx=0 tx=0).stop 2024-05-20T20:54:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 msgr2=0x7f7d5c19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c19fc60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000d950 tx=0x7f7d5000dd10 comp rx=0 tx=0).stop 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 shutdown_connections 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d5c1088f0 0x7f7d5c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d5c1037f0 0x7f7d5c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7d440b2880 0x7f7d440b4d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 --2- 172.21.15.26:0/1524643131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d5c1041a0 0x7f7d5c19fc60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 >> 172.21.15.26:0/1524643131 conn(0x7f7d5c07a6a0 msgr2=0x7f7d5c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 shutdown_connections 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:49.042+0000 7f7d629e4700 1 -- 172.21.15.26:0/1524643131 wait complete. 2024-05-20T20:54:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:54:49.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:48 smithi144 ceph-mon[35660]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-20T20:54:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:48 smithi055 ceph-mon[35213]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-20T20:54:49.765 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:54:50.023 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:50 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1524643131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:54:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:54:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:50 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:50 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1524643131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:54:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:54:50.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:50 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:50 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1524643131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:54:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:54:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:50 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:54:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:51 smithi026 ceph-mon[31990]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 259 B/s wr, 6 op/s 2024-05-20T20:54:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:51 smithi144 ceph-mon[35660]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 259 B/s wr, 6 op/s 2024-05-20T20:54:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:51 smithi055 ceph-mon[35213]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 259 B/s wr, 6 op/s 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 -- 172.21.15.26:0/3453223315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2714101f00 msgr2=0x7f2714102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 --2- 172.21.15.26:0/3453223315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2714101f00 0x7f2714102380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2700009990 tx=0x7f2700009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 -- 172.21.15.26:0/3453223315 shutdown_connections 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 --2- 172.21.15.26:0/3453223315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27141028c0 0x7f2714110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 --2- 172.21.15.26:0/3453223315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2714108620 0x7f2714108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 --2- 172.21.15.26:0/3453223315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2714101f00 0x7f2714102380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 -- 172.21.15.26:0/3453223315 >> 172.21.15.26:0/3453223315 conn(0x7f2714077780 msgr2=0x7f2714077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 -- 172.21.15.26:0/3453223315 shutdown_connections 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.537+0000 7f271ab4e700 1 -- 172.21.15.26:0/3453223315 wait complete. 2024-05-20T20:54:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 Processor -- start 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 -- start start 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2714101f00 0x7f271419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 0x7f271419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2714108620 0x7f27141999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2714113660 con 0x7f27141028c0 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f27141134e0 con 0x7f2714101f00 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f271ab4e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27141137e0 con 0x7f2714108620 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 0x7f271419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 0x7f271419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54488/0 (socket says 172.21.15.26:54488) 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 -- 172.21.15.26:0/2636633650 learned_addr learned my addr 172.21.15.26:0/2636633650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f27188ea700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2714101f00 0x7f271419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 -- 172.21.15.26:0/2636633650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2714108620 msgr2=0x7f27141999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2714108620 0x7f27141999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 -- 172.21.15.26:0/2636633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2714101f00 msgr2=0x7f271419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2714101f00 0x7f271419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 -- 172.21.15.26:0/2636633650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271419a140 con 0x7f27141028c0 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f27188ea700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2714101f00 0x7f271419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.538+0000 7f2713fff700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 0x7f271419fd10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f2700009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.539+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700027070 con 0x7f27141028c0 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.539+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f270000fc70 con 0x7f27141028c0 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.539+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271419a3d0 con 0x7f27141028c0 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.539+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700022d40 con 0x7f27141028c0 2024-05-20T20:54:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.539+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141a66a0 con 0x7f27141028c0 2024-05-20T20:54:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.540+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271406a230 con 0x7f27141028c0 2024-05-20T20:54:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.542+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f270002a070 con 0x7f27141028c0 2024-05-20T20:54:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.543+0000 7f2711ffb700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27040b2880 0x7f27040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.543+0000 7f27188ea700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27040b2880 0x7f27040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.543+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f27000e37c0 con 0x7f27141028c0 2024-05-20T20:54:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.543+0000 7f27188ea700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27040b2880 0x7f27040b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2714108360 tx=0x7f2708008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.546+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2700031070 con 0x7f27141028c0 2024-05-20T20:54:51.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.767+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f271402e560 con 0x7f27141028c0 2024-05-20T20:54:51.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.769+0000 7f2711ffb700 1 -- 172.21.15.26:0/2636633650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22791 (secure 0 0 0) 0x7f2700013070 con 0x7f27141028c0 2024-05-20T20:54:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:51.769 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:54:37.856459+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27040b2880 msgr2=0x7f27040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27040b2880 0x7f27040b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2714108360 tx=0x7f2708008040 comp rx=0 tx=0).stop 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 msgr2=0x7f271419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 0x7f271419fd10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f2700009f90 comp rx=0 tx=0).stop 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 shutdown_connections 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2714108620 0x7f27141999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2714101f00 0x7f271419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27040b2880 0x7f27040b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 --2- 172.21.15.26:0/2636633650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27141028c0 0x7f271419fd10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:51.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 >> 172.21.15.26:0/2636633650 conn(0x7f2714077780 msgr2=0x7f2714100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:51.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 shutdown_connections 2024-05-20T20:54:51.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:51.771+0000 7f271ab4e700 1 -- 172.21.15.26:0/2636633650 wait complete. 2024-05-20T20:54:52.120 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2636633650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:54:52.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:52 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2636633650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:54:52.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:54:52.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:52 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2636633650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:54:52.665 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:53.039 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:53 smithi026 ceph-mon[31990]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:53.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:53 smithi144 ceph-mon[35660]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:53 smithi055 ceph-mon[35213]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:54:54.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 -- 172.21.15.26:0/2627214557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c06b220 msgr2=0x7f425c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:54.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 --2- 172.21.15.26:0/2627214557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c06b220 0x7f425c06b6a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4244009990 tx=0x7f4244009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 -- 172.21.15.26:0/2627214557 shutdown_connections 2024-05-20T20:54:54.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 --2- 172.21.15.26:0/2627214557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f425c06bbe0 0x7f425c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 --2- 172.21.15.26:0/2627214557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1095b0 0x7f425c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 --2- 172.21.15.26:0/2627214557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c06b220 0x7f425c06b6a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 -- 172.21.15.26:0/2627214557 >> 172.21.15.26:0/2627214557 conn(0x7f425c0793f0 msgr2=0x7f425c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:54.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 -- 172.21.15.26:0/2627214557 shutdown_connections 2024-05-20T20:54:54.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.316+0000 7f42611f0700 1 -- 172.21.15.26:0/2627214557 wait complete. 2024-05-20T20:54:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.317+0000 7f42611f0700 1 Processor -- start 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.317+0000 7f42611f0700 1 -- start start 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f42611f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c06b220 0x7f425c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f42611f0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f425c06bbe0 0x7f425c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f42611f0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 0x7f425c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f42611f0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f425c113570 con 0x7f425c1095b0 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f42611f0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1133f0 con 0x7f425c06bbe0 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f42611f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1136f0 con 0x7f425c06b220 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 0x7f425c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 0x7f425c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54512/0 (socket says 172.21.15.26:54512) 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 -- 172.21.15.26:0/1134363652 learned_addr learned my addr 172.21.15.26:0/1134363652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425ad9d700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c06b220 0x7f425c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425259c700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f425c06bbe0 0x7f425c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 -- 172.21.15.26:0/1134363652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c06b220 msgr2=0x7f425c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c06b220 0x7f425c102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 -- 172.21.15.26:0/1134363652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f425c06bbe0 msgr2=0x7f425c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f425c06bbe0 0x7f425c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 -- 172.21.15.26:0/1134363652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c1a8050 con 0x7f425c1095b0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.318+0000 7f425b59e700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 0x7f425c102f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f425400d980 tx=0x7f425400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.319+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254016070 con 0x7f425c1095b0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.319+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c1a8340 con 0x7f425c1095b0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.319+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4254004830 con 0x7f425c1095b0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.319+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254005480 con 0x7f425c1095b0 2024-05-20T20:54:54.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.319+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c1a88a0 con 0x7f425c1095b0 2024-05-20T20:54:54.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.320+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c06a230 con 0x7f425c1095b0 2024-05-20T20:54:54.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.321+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4254004220 con 0x7f425c1095b0 2024-05-20T20:54:54.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.322+0000 7f4258d99700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f42480b2880 0x7f42480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:54.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.322+0000 7f425ad9d700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f42480b2880 0x7f42480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:54.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.322+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f42540e24c0 con 0x7f425c1095b0 2024-05-20T20:54:54.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.322+0000 7f425ad9d700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f42480b2880 0x7f42480b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f424c007fd0 tx=0x7f424c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:54.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.325+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f425401f020 con 0x7f425c1095b0 2024-05-20T20:54:54.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.596+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f425c02e560 con 0x7f425c1095b0 2024-05-20T20:54:54.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.597+0000 7f4258d99700 1 -- 172.21.15.26:0/1134363652 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f42540b2030 con 0x7f425c1095b0 2024-05-20T20:54:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:54:54.597 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:53:05.138905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f42480b2880 msgr2=0x7f42480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f42480b2880 0x7f42480b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f424c007fd0 tx=0x7f424c00c040 comp rx=0 tx=0).stop 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 msgr2=0x7f425c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 0x7f425c102f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f425400d980 tx=0x7f425400dc90 comp rx=0 tx=0).stop 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 shutdown_connections 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f42480b2880 0x7f42480b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f425c1095b0 0x7f425c102f90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c06b220 0x7f425c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 --2- 172.21.15.26:0/1134363652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f425c06bbe0 0x7f425c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 >> 172.21.15.26:0/1134363652 conn(0x7f425c0793f0 msgr2=0x7f425c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 shutdown_connections 2024-05-20T20:54:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:54.599+0000 7f42611f0700 1 -- 172.21.15.26:0/1134363652 wait complete. 2024-05-20T20:54:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:54:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:55 smithi026 ceph-mon[31990]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:55 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1134363652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:55.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-20T20:52:47.191542+0000', 'modified': '2024-05-20T20:53:05.138905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24476, 'mds_1': 24455, 'mds_2': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/1712709614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 1712709614}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 1712709614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.55:6835/2325650145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2325650145}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2325650145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/539511953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 539511953}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 539511953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.144:6839/2122637820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 2122637820}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 2122637820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.55:6837/353197532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 353197532}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 353197532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.55:6839/1704683185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6838', 'nonce': 1704683185}, {'type': 'v1', 'addr': '172.21.15.55:6839', 'nonce': 1704683185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24445, 24476, 24455]} 2024-05-20T20:54:55.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-20T20:54:55.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-20T20:54:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:55 smithi144 ceph-mon[35660]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:55 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1134363652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:55 smithi055 ceph-mon[35213]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:55 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1134363652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:54:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 -- 172.21.15.26:0/2266132362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 msgr2=0x7f8068102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 --2- 172.21.15.26:0/2266132362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f8068102c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f80500099c0 tx=0x7f8050009cd0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 -- 172.21.15.26:0/2266132362 shutdown_connections 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 --2- 172.21.15.26:0/2266132362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8068103150 0x7f806810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 --2- 172.21.15.26:0/2266132362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8068108630 0x7f8068108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 --2- 172.21.15.26:0/2266132362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f8068102c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 -- 172.21.15.26:0/2266132362 >> 172.21.15.26:0/2266132362 conn(0x7f80680782b0 msgr2=0x7f80680786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 -- 172.21.15.26:0/2266132362 shutdown_connections 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.193+0000 7f806e428700 1 -- 172.21.15.26:0/2266132362 wait complete. 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 Processor -- start 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 -- start start 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f806819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8068103150 0x7f806819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8068108630 0x7f8068198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8068112550 con 0x7f8068102790 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80681123d0 con 0x7f8068103150 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806e428700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80681126d0 con 0x7f8068108630 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f806819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f806819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54530/0 (socket says 172.21.15.26:54530) 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 -- 172.21.15.26:0/2703221727 learned_addr learned my addr 172.21.15.26:0/2703221727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f805ffff700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8068103150 0x7f806819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f806c9c5700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8068108630 0x7f8068198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 -- 172.21.15.26:0/2703221727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8068108630 msgr2=0x7f8068198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8068108630 0x7f8068198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 -- 172.21.15.26:0/2703221727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8068103150 msgr2=0x7f806819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8068103150 0x7f806819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.194+0000 7f8067fff700 1 -- 172.21.15.26:0/2703221727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8068198fd0 con 0x7f8068102790 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.195+0000 7f8067fff700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f806819e6b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f805800c970 tx=0x7f805800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.195+0000 7f8065ffb700 1 -- 172.21.15.26:0/2703221727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058007aa0 con 0x7f8068102790 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.195+0000 7f8065ffb700 1 -- 172.21.15.26:0/2703221727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8058004cc0 con 0x7f8068102790 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.195+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80681992c0 con 0x7f8068102790 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.195+0000 7f8065ffb700 1 -- 172.21.15.26:0/2703221727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058005450 con 0x7f8068102790 2024-05-20T20:54:57.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.195+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80681a5580 con 0x7f8068102790 2024-05-20T20:54:57.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.197+0000 7f8065ffb700 1 -- 172.21.15.26:0/2703221727 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8058007c40 con 0x7f8068102790 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.198+0000 7f8065ffb700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80540b2880 0x7f80540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f805ffff700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80540b2880 0x7f80540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8065ffb700 1 -- 172.21.15.26:0/2703221727 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f80580e1ff0 con 0x7f8068102790 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 Processor -- start 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 -- start start 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c65810 0x55a895c65bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c661c0 0x55a895c698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f805ffff700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80540b2880 0x7f80540b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8050000c00 tx=0x7f8050005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a895bc4c40 con 0x55a895c69f10 2024-05-20T20:54:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a895bb47e0 con 0x55a895c661c0 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a895bb8820 con 0x55a895c65810 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f806c9c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f806c9c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c6a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54532/0 (socket says 172.21.15.26:54532) 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f806c9c5700 1 -- 172.21.15.26:0/1878614012 learned_addr learned my addr 172.21.15.26:0/1878614012 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f805ffff700 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c661c0 0x55a895c698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.199+0000 7f8067fff700 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c65810 0x55a895c65bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f806c9c5700 1 -- 172.21.15.26:0/1878614012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c65810 msgr2=0x55a895c65bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f806c9c5700 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c65810 0x55a895c65bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f806c9c5700 1 -- 172.21.15.26:0/1878614012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c661c0 msgr2=0x55a895c698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f806c9c5700 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c661c0 0x55a895c698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f806c9c5700 1 -- 172.21.15.26:0/1878614012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a895b04880 con 0x55a895c69f10 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f806c9c5700 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c6a350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8060009e40 tx=0x7f80600077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe9c35258881cec server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f805e7fc700 1 -- 172.21.15.26:0/1878614012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060003a00 con 0x55a895c69f10 2024-05-20T20:54:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f805e7fc700 1 -- 172.21.15.26:0/1878614012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8060003ba0 con 0x55a895c69f10 2024-05-20T20:54:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.200+0000 7f805e7fc700 1 -- 172.21.15.26:0/1878614012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80600048b0 con 0x55a895c69f10 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 -- 172.21.15.26:0/1878614012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 msgr2=0x55a895c6a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c6a350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8060009e40 tx=0x7f80600077e0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 -- 172.21.15.26:0/1878614012 shutdown_connections 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c6a350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c65810 0x55a895c65bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/1878614012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c661c0 0x55a895c698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 -- 172.21.15.26:0/1878614012 >> 172.21.15.26:0/1878614012 conn(0x55a895b93030 msgr2=0x55a895c5fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 -- 172.21.15.26:0/1878614012 shutdown_connections 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 -- 172.21.15.26:0/1878614012 wait complete. 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 -- start start 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.201+0000 7f8078cbeb80 1 Processor -- start 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c71300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c6e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c6eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a895abaf50 con 0x55a895c661c0 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a895b56770 con 0x55a895c65810 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a895c742e0 con 0x55a895c69f10 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 10 client.0 _ll_register_callbacks cb 0x55a895b4f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c6e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c6e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54546/0 (socket says 172.21.15.26:54546) 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 -- 172.21.15.26:0/513011322 learned_addr learned my addr 172.21.15.26:0/513011322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8067fff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c71300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 -- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 msgr2=0x55a895c6eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c6eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 -- 172.21.15.26:0/513011322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 msgr2=0x55a895c71300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c71300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 -- 172.21.15.26:0/513011322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a895b53640 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8067fff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c71300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f805ffff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c6e600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f805000b580 tx=0x7f8050009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8c3730acc8d3b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f80477fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f80477fe700 1 -- 172.21.15.26:0/513011322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050035060 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f80477fe700 1 -- 172.21.15.26:0/513011322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f8050003b90 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.202+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8060003ba0 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f80477fe700 1 -- 172.21.15.26:0/513011322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8050003d90 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 10 client.14958 fetch_fsmap learned FSMap version 25 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a895c7e980 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f80477fe700 1 -- 172.21.15.26:0/513011322 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8060003ba0 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f80477fe700 1 -- 172.21.15.26:0/513011322 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8050039030 con 0x55a895c661c0 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 10 client.14958 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:54:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 10 client.14958 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 20 client.14958 populate_metadata read hostname 'smithi026' 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895c81020 0x55a895c82490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 4 client.14958 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a895c31990 con 0x55a895c81020 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.203+0000 7f8067fff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895c81020 0x55a895c82490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.204+0000 7f8067fff700 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895c81020 0x55a895c82490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1098de988618b442 server_cookie=eabee76e0b9ff2b9 in_seq=0 out_seq=0 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.204+0000 7f80477fe700 10 client.14958 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f80477fe700 1 -- 172.21.15.26:0/513011322 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a895c31990 con 0x55a895c81020 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f80477fe700 10 client.14958 handle_command_reply: tid=0 2024-05-20T20:54:57.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 client.14958 shutdown 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 msgr2=0x55a895c6e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c6e600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f805000b580 tx=0x7f8050009cd0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 shutdown_connections 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895c81020 0x55a895c82490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c6eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f80477fe700 0 client.14958 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c71300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/513011322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c6e600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 >> 172.21.15.26:0/513011322 conn(0x55a895b93030 msgr2=0x55a895c5f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:57.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 shutdown_connections 2024-05-20T20:54:57.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 1 -- 172.21.15.26:0/513011322 wait complete. 2024-05-20T20:54:57.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.205+0000 7f8078cbeb80 20 client.14958 trim_cache size 0 max 16384 2024-05-20T20:54:57.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 Processor -- start 2024-05-20T20:54:57.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 -- start start 2024-05-20T20:54:57.227 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:57 smithi026 ceph-mon[31990]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:57.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c61840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c661c0 0x55a895c6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a895c807d0 con 0x55a895c69f10 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a895c80ea0 con 0x55a895c65810 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a895c81570 con 0x55a895c661c0 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f806c9c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f806c9c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54556/0 (socket says 172.21.15.26:54556) 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.227+0000 7f806c9c5700 1 -- 172.21.15.26:0/3988157829 learned_addr learned my addr 172.21.15.26:0/3988157829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f805ffff700 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c661c0 0x55a895c6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f8067fff700 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c61840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f806c9c5700 1 -- 172.21.15.26:0/3988157829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c661c0 msgr2=0x55a895c6d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f806c9c5700 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c661c0 0x55a895c6d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f806c9c5700 1 -- 172.21.15.26:0/3988157829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 msgr2=0x55a895c61840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f806c9c5700 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c61840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f806c9c5700 1 -- 172.21.15.26:0/3988157829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a895c608f0 con 0x55a895c69f10 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f806c9c5700 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c690f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8060003b90 tx=0x7f8060003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dff8a2f6f96add7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:57.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f805dffb700 1 -- 172.21.15.26:0/3988157829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060022020 con 0x55a895c69f10 2024-05-20T20:54:57.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f805dffb700 1 -- 172.21.15.26:0/3988157829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8060003d60 con 0x55a895c69f10 2024-05-20T20:54:57.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.228+0000 7f805dffb700 1 -- 172.21.15.26:0/3988157829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060004730 con 0x55a895c69f10 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 -- 172.21.15.26:0/3988157829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 msgr2=0x55a895c690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c690f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8060003b90 tx=0x7f8060003bc0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 -- 172.21.15.26:0/3988157829 shutdown_connections 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c69f10 0x55a895c690f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/3988157829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c661c0 0x55a895c6d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 -- 172.21.15.26:0/3988157829 >> 172.21.15.26:0/3988157829 conn(0x55a895b93030 msgr2=0x55a895c64620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 -- 172.21.15.26:0/3988157829 shutdown_connections 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 -- 172.21.15.26:0/3988157829 wait complete. 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.229+0000 7f8078cbeb80 1 -- start start 2024-05-20T20:54:57.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 Processor -- start 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c87b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c88040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a895c82310 con 0x55a895c661c0 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a895c81c40 con 0x55a895c65810 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a895c829e0 con 0x55a895c69f10 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8078cbeb80 10 client.0 _ll_register_callbacks cb 0x55a895c61df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f805ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c87b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f806c9c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c88040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f8067fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f806c9c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c88040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:43184/0 (socket says 172.21.15.26:43184) 2024-05-20T20:54:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.230+0000 7f806c9c5700 1 -- 172.21.15.26:0/74154089 learned_addr learned my addr 172.21.15.26:0/74154089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8067fff700 1 -- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 msgr2=0x55a895c88040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8067fff700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c88040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8067fff700 1 -- 172.21.15.26:0/74154089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 msgr2=0x55a895c87b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8067fff700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c87b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8067fff700 1 -- 172.21.15.26:0/74154089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a895ceb7b0 con 0x55a895c65810 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f805ffff700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f806c9c5700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c88040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8067fff700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c875c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f80580a6890 tx=0x7f8058091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9fcbb5df6f1feba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f80477fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f80477fe700 1 -- 172.21.15.26:0/74154089 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580c2070 con 0x55a895c65810 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f80477fe700 1 -- 172.21.15.26:0/74154089 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f8058015330 con 0x55a895c65810 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8060003d60 con 0x55a895c65810 2024-05-20T20:54:57.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f80477fe700 1 -- 172.21.15.26:0/74154089 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8058015530 con 0x55a895c65810 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8078cbeb80 10 client.24719 fetch_fsmap learned FSMap version 25 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.231+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a895c71ef0 con 0x55a895c65810 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f80477fe700 1 -- 172.21.15.26:0/74154089 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8060003d60 con 0x55a895c65810 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f80477fe700 1 -- 172.21.15.26:0/74154089 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f80580b1f60 con 0x55a895c65810 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f8078cbeb80 10 client.24719 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f8078cbeb80 10 client.24719 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f8078cbeb80 20 client.24719 populate_metadata read hostname 'smithi026' 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895cef450 0x55a895cee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f8078cbeb80 4 client.24719 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "status"}) -- 0x55a895bc6b90 con 0x55a895cef450 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.232+0000 7f805ffff700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895cef450 0x55a895cee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f805ffff700 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895cef450 0x55a895cee160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bd4ad9920518223 server_cookie=7444526991ed8e80 in_seq=0 out_seq=0 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f80477fe700 10 client.24719 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:54:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f80477fe700 1 -- 172.21.15.26:0/74154089 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a895bc6b90 con 0x55a895cef450 2024-05-20T20:54:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f80477fe700 10 client.24719 handle_command_reply: tid=0 2024-05-20T20:54:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 client.24719 shutdown 2024-05-20T20:54:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 msgr2=0x55a895c875c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c875c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f80580a6890 tx=0x7f8058091040 comp rx=0 tx=0).stop 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 shutdown_connections 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a895cef450 0x55a895cee160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a895c69f10 0x55a895c88040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a895c65810 0x55a895c875c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f80477fe700 0 client.24719 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 --2- 172.21.15.26:0/74154089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a895c661c0 0x55a895c87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 >> 172.21.15.26:0/74154089 conn(0x55a895b93030 msgr2=0x55a895c64eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 shutdown_connections 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 1 -- 172.21.15.26:0/74154089 wait complete. 2024-05-20T20:54:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.233+0000 7f8078cbeb80 20 client.24719 trim_cache size 0 max 16384 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24445, 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 116.928718281, 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:54:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T20:54:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 156.157408272 2024-05-20T20:54:57.240 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:54:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80540b2880 msgr2=0x7f80540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80540b2880 0x7f80540b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8050000c00 tx=0x7f8050005fb0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 msgr2=0x7f806819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f806819e6b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f805800c970 tx=0x7f805800cc80 comp rx=0 tx=0).stop 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 shutdown_connections 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80540b2880 0x7f80540b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8068102790 0x7f806819e6b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8068108630 0x7f8068198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 --2- 172.21.15.26:0/2703221727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8068103150 0x7f806819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 >> 172.21.15.26:0/2703221727 conn(0x7f80680782b0 msgr2=0x7f80681015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 shutdown_connections 2024-05-20T20:54:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:57.234+0000 7f806e428700 1 -- 172.21.15.26:0/2703221727 wait complete. 2024-05-20T20:54:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:57 smithi144 ceph-mon[35660]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:57 smithi055 ceph-mon[35213]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:54:57.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-20T20:54:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:54:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:54:59 smithi144 ceph-mon[35660]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:59.348 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:54:59 smithi026 ceph-mon[31990]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:54:59 smithi055 ceph-mon[35213]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:54:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/2899631386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec06bab0 msgr2=0x7fbaec06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/2899631386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec06bab0 0x7fbaec06bf30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009990 tx=0x7fbad4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/2899631386 shutdown_connections 2024-05-20T20:54:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/2899631386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaec06c470 0x7fbaec1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/2899631386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbaec1095a0 0x7fbaec109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/2899631386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec06bab0 0x7fbaec06bf30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/2899631386 >> 172.21.15.26:0/2899631386 conn(0x7fbaec0793b0 msgr2=0x7fbaec0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.746+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/2899631386 shutdown_connections 2024-05-20T20:54:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.747+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/2899631386 wait complete. 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.747+0000 7fbaf0bb4700 1 Processor -- start 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.747+0000 7fbaf0bb4700 1 -- start start 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaf0bb4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbaec06bab0 0x7fbaec19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaf0bb4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaec06c470 0x7fbaec19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaf0bb4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 0x7fbaec199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaf0bb4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1135c0 con 0x7fbaec1095a0 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaf0bb4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec113440 con 0x7fbaec06bab0 2024-05-20T20:54:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaf0bb4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec113740 con 0x7fbaec06c470 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 0x7fbaec199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 0x7fbaec199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46014/0 (socket says 172.21.15.26:46014) 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 -- 172.21.15.26:0/807719074 learned_addr learned my addr 172.21.15.26:0/807719074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaec06c470 0x7fbaec19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaea59c700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbaec06bab0 0x7fbaec19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 -- 172.21.15.26:0/807719074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaec06c470 msgr2=0x7fbaec19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaec06c470 0x7fbaec19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 -- 172.21.15.26:0/807719074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbaec06bab0 msgr2=0x7fbaec19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbaec06bab0 0x7fbaec19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 -- 172.21.15.26:0/807719074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec19a0e0 con 0x7fbaec1095a0 2024-05-20T20:54:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbaead9d700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 0x7fbaec199980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbae400e940 tx=0x7fbae400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.748+0000 7fbae37fe700 1 -- 172.21.15.26:0/807719074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae400cbd0 con 0x7fbaec1095a0 2024-05-20T20:54:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.749+0000 7fbae37fe700 1 -- 172.21.15.26:0/807719074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbae400cd70 con 0x7fbaec1095a0 2024-05-20T20:54:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.749+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec19a3d0 con 0x7fbaec1095a0 2024-05-20T20:54:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.749+0000 7fbae37fe700 1 -- 172.21.15.26:0/807719074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4005570 con 0x7fbaec1095a0 2024-05-20T20:54:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.749+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a6690 con 0x7fbaec1095a0 2024-05-20T20:54:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.752+0000 7fbae37fe700 1 -- 172.21.15.26:0/807719074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbae4004750 con 0x7fbaec1095a0 2024-05-20T20:54:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.753+0000 7fbae37fe700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbad80b2900 0x7fbad80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.753+0000 7fbaea59c700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbad80b2900 0x7fbad80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.753+0000 7fbae37fe700 1 -- 172.21.15.26:0/807719074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbae40e2540 con 0x7fbaec1095a0 2024-05-20T20:54:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbaea59c700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbad80b2900 0x7fbad80b4dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbaec19aac0 tx=0x7fbadc009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 Processor -- start 2024-05-20T20:54:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 -- start start 2024-05-20T20:54:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7236a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a7236e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a722c4c40 con 0x562a7236e320 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a722b47e0 con 0x562a72369c20 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a722b8820 con 0x562a7236a5d0 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a7236e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a7236e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46026/0 (socket says 172.21.15.26:46026) 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbaead9d700 1 -- 172.21.15.26:0/1973646900 learned_addr learned my addr 172.21.15.26:0/1973646900 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbaea59c700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7236a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.754+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaead9d700 1 -- 172.21.15.26:0/1973646900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 msgr2=0x562a7236dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaead9d700 1 -- 172.21.15.26:0/1973646900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 msgr2=0x562a7236a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7236a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaead9d700 1 -- 172.21.15.26:0/1973646900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a72204880 con 0x562a7236e320 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaea59c700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7236a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a7236e760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbae40a6d70 tx=0x7fbae40a9580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6bdf96009abdea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbae0ff9700 1 -- 172.21.15.26:0/1973646900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40c3070 con 0x562a7236e320 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbae0ff9700 1 -- 172.21.15.26:0/1973646900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbae40b22b0 con 0x562a7236e320 2024-05-20T20:54:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.755+0000 7fbae0ff9700 1 -- 172.21.15.26:0/1973646900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40a9f60 con 0x562a7236e320 2024-05-20T20:54:59.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1973646900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 msgr2=0x562a7236e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a7236e760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbae40a6d70 tx=0x7fbae40a9580 comp rx=0 tx=0).stop 2024-05-20T20:54:59.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1973646900 shutdown_connections 2024-05-20T20:54:59.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a7236e760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7236a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1973646900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1973646900 >> 172.21.15.26:0/1973646900 conn(0x562a72293030 msgr2=0x562a7235fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1973646900 shutdown_connections 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1973646900 wait complete. 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 -- start start 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.756+0000 7fbafb44ab80 1 Processor -- start 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a72375790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72372a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a72372fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a721baf50 con 0x562a72369c20 2024-05-20T20:54:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a72256770 con 0x562a7236e320 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a723764e0 con 0x562a7236a5d0 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbafb44ab80 10 client.0 _ll_register_callbacks cb 0x562a7224f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a72375790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a72375790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46032/0 (socket says 172.21.15.26:46032) 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 -- 172.21.15.26:0/1845377899 learned_addr learned my addr 172.21.15.26:0/1845377899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a72372fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72372a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 -- 172.21.15.26:0/1845377899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 msgr2=0x562a72372a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72372a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 -- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 msgr2=0x562a72372fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a72372fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 -- 172.21.15.26:0/1845377899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a72253640 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a72372fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72372a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbaea59c700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a72375790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0169b0 tx=0x7fbadc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be355277c4a00efc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.757+0000 7fbad1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbad1ffb700 1 -- 172.21.15.26:0/1845377899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc026020 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbad1ffb700 1 -- 172.21.15.26:0/1845377899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbadc014a70 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae40b22b0 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbad1ffb700 1 -- 172.21.15.26:0/1845377899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbadc031050 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 10 client.14976 fetch_fsmap learned FSMap version 25 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a72380cf0 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbad1ffb700 1 -- 172.21.15.26:0/1845377899 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbae40b22b0 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbad1ffb700 1 -- 172.21.15.26:0/1845377899 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbadc022a40 con 0x562a72369c20 2024-05-20T20:54:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 10 client.14976 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 10 client.14976 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 20 client.14976 populate_metadata read hostname 'smithi026' 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a72383390 0x562a72384800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 4 client.14976 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.758+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a72331990 con 0x562a72383390 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.759+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a72383390 0x562a72384800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.759+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a72383390 0x562a72384800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df5f00562ea5d21e server_cookie=4f0e7a91c2e93a3f in_seq=0 out_seq=0 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.759+0000 7fbad1ffb700 10 client.14976 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbad1ffb700 1 -- 172.21.15.26:0/1845377899 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562a72331990 con 0x562a72383390 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbad1ffb700 10 client.14976 handle_command_reply: tid=0 2024-05-20T20:54:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 client.14976 shutdown 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 msgr2=0x562a72375790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a72375790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0169b0 tx=0x7fbadc004ad0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 shutdown_connections 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a72383390 0x562a72384800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a72375790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a72372fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1845377899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72372a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 >> 172.21.15.26:0/1845377899 conn(0x562a72293030 msgr2=0x562a7236f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbad1ffb700 0 client.14976 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:54:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 shutdown_connections 2024-05-20T20:54:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1845377899 wait complete. 2024-05-20T20:54:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.760+0000 7fbafb44ab80 20 client.14976 trim_cache size 0 max 16384 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 Processor -- start 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 -- start start 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7235fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72360310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a723713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a72382b40 con 0x562a7236e320 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a72383210 con 0x562a72369c20 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a723838e0 con 0x562a7236a5d0 2024-05-20T20:54:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a723713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a723713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46040/0 (socket says 172.21.15.26:46040) 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 -- 172.21.15.26:0/1409288799 learned_addr learned my addr 172.21.15.26:0/1409288799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 -- 172.21.15.26:0/1409288799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 msgr2=0x562a72360310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72360310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 -- 172.21.15.26:0/1409288799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 msgr2=0x562a7235fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaea59c700 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7235fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7235fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.782+0000 7fbaead9d700 1 -- 172.21.15.26:0/1409288799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a72361640 con 0x562a7236e320 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbaead9d700 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a723713a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbae400cfd0 tx=0x7fbae40a5af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecf7730b1f6f8d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbad3fff700 1 -- 172.21.15.26:0/1409288799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40c8020 con 0x562a7236e320 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbad3fff700 1 -- 172.21.15.26:0/1409288799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbae40a9b00 con 0x562a7236e320 2024-05-20T20:54:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbad3fff700 1 -- 172.21.15.26:0/1409288799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40c3050 con 0x562a7236e320 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1409288799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 msgr2=0x562a723713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a723713a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbae400cfd0 tx=0x7fbae40a5af0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1409288799 shutdown_connections 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a7236e320 0x562a723713a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a72369c20 0x562a7235fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/1409288799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a72360310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.783+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1409288799 >> 172.21.15.26:0/1409288799 conn(0x562a72293030 msgr2=0x562a7235ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1409288799 shutdown_connections 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 -- 172.21.15.26:0/1409288799 wait complete. 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 -- start start 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 Processor -- start 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a7238e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a7236bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a72384680 con 0x562a72369c20 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a72383fb0 con 0x562a7236e320 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a72384d50 con 0x562a7236a5d0 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbafb44ab80 10 client.0 _ll_register_callbacks cb 0x562a7224f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:54:59.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a7238e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a7238e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46056/0 (socket says 172.21.15.26:46056) 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbaea59c700 1 -- 172.21.15.26:0/2507477654 learned_addr learned my addr 172.21.15.26:0/2507477654 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.784+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaea59c700 1 -- 172.21.15.26:0/2507477654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 msgr2=0x562a7236b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaea59c700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaea59c700 1 -- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 msgr2=0x562a7236bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaead9d700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a7236bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaea59c700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a7236bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaea59c700 1 -- 172.21.15.26:0/2507477654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a7238bf60 con 0x562a72369c20 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaead9d700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a7236bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:54:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbaea59c700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a7238e4f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbadc007b70 tx=0x7fbadc005fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a73e284126abb98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbad1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbad1ffb700 1 -- 172.21.15.26:0/2507477654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc03a060 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbad1ffb700 1 -- 172.21.15.26:0/2507477654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbadc014c70 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a722c6b90 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbad1ffb700 1 -- 172.21.15.26:0/2507477654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbadc026070 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbafb44ab80 10 client.14982 fetch_fsmap learned FSMap version 25 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a723eecb0 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.785+0000 7fbad1ffb700 1 -- 172.21.15.26:0/2507477654 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562a722c6b90 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbad1ffb700 1 -- 172.21.15.26:0/2507477654 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbadc03e030 con 0x562a72369c20 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 10 client.14982 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 10 client.14982 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 20 client.14982 populate_metadata read hostname 'smithi026' 2024-05-20T20:54:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a723f38f0 0x562a723f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 4 client.14982 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "status"}) -- 0x562a722c0370 con 0x562a723f38f0 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a723f38f0 0x562a723f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbae9d9b700 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a723f38f0 0x562a723f2600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5f50f435fa97061 server_cookie=e8cdce641c1e95d7 in_seq=0 out_seq=0 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbad1ffb700 10 client.14982 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbad1ffb700 1 -- 172.21.15.26:0/2507477654 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562a722c0370 con 0x562a723f38f0 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbad1ffb700 10 client.14982 handle_command_reply: tid=0 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.786+0000 7fbafb44ab80 1 client.14982 shutdown 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 msgr2=0x562a7238e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a7238e4f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbadc007b70 tx=0x7fbadc005fe0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 shutdown_connections 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a723f38f0 0x562a723f2600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a72369c20 0x562a7238e4f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a7236e320 0x562a7236bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 --2- 172.21.15.26:0/2507477654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7236a5d0 0x562a7236b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 >> 172.21.15.26:0/2507477654 conn(0x562a72293030 msgr2=0x562a7235ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbad1ffb700 0 client.14982 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 shutdown_connections 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 1 -- 172.21.15.26:0/2507477654 wait complete. 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbafb44ab80 20 client.14982 trim_cache size 0 max 16384 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-05-20T20:54:59.793 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24455, 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 120.493099508, 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 161.59365069899999 2024-05-20T20:54:59.794 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbad80b2900 msgr2=0x7fbad80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbad80b2900 0x7fbad80b4dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbaec19aac0 tx=0x7fbadc009040 comp rx=0 tx=0).stop 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 msgr2=0x7fbaec199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.787+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 0x7fbaec199980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbae400e940 tx=0x7fbae400ed00 comp rx=0 tx=0).stop 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 shutdown_connections 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbad80b2900 0x7fbad80b4dc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaec1095a0 0x7fbaec199980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbaec06bab0 0x7fbaec19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 --2- 172.21.15.26:0/807719074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaec06c470 0x7fbaec19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:54:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 >> 172.21.15.26:0/807719074 conn(0x7fbaec0793b0 msgr2=0x7fbaec101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:54:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 shutdown_connections 2024-05-20T20:54:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:54:59.788+0000 7fbaf0bb4700 1 -- 172.21.15.26:0/807719074 wait complete. 2024-05-20T20:55:00.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-20T20:55:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:01.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:01 smithi026 ceph-mon[31990]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:01 smithi144 ceph-mon[35660]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:01.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:01 smithi055 ceph-mon[35213]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:02.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 -- 172.21.15.26:0/2084595575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 msgr2=0x7f1f18110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 --2- 172.21.15.26:0/2084595575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18110490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000b0d0 tx=0x7f1f1000b3e0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 -- 172.21.15.26:0/2084595575 shutdown_connections 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 --2- 172.21.15.26:0/2084595575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18110490 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 --2- 172.21.15.26:0/2084595575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f18103820 0x7f1f18103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 --2- 172.21.15.26:0/2084595575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f181041d0 0x7f1f181081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 -- 172.21.15.26:0/2084595575 >> 172.21.15.26:0/2084595575 conn(0x7f1f1807a6b0 msgr2=0x7f1f1807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 -- 172.21.15.26:0/2084595575 shutdown_connections 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 -- 172.21.15.26:0/2084595575 wait complete. 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.372+0000 7f1f1f128700 1 Processor -- start 2024-05-20T20:55:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 -- start start 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f18103820 0x7f1f1819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f181041d0 0x7f1f1819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181135b0 con 0x7f1f18108910 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18113430 con 0x7f1f18103820 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1f128700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18113730 con 0x7f1f181041d0 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46084/0 (socket says 172.21.15.26:46084) 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 -- 172.21.15.26:0/1776827095 learned_addr learned my addr 172.21.15.26:0/1776827095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f18103820 0x7f1f1819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f17fff700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f181041d0 0x7f1f1819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 -- 172.21.15.26:0/1776827095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f181041d0 msgr2=0x7f1f1819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f181041d0 0x7f1f1819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 -- 172.21.15.26:0/1776827095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f18103820 msgr2=0x7f1f1819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f18103820 0x7f1f1819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 -- 172.21.15.26:0/1776827095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f18199fd0 con 0x7f1f18108910 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.373+0000 7f1f1d6c5700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18199870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009850 tx=0x7f1f10007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.374+0000 7f1f15ffb700 1 -- 172.21.15.26:0/1776827095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1001f070 con 0x7f1f18108910 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.374+0000 7f1f15ffb700 1 -- 172.21.15.26:0/1776827095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f10005500 con 0x7f1f18108910 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.374+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1819a260 con 0x7f1f18108910 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.374+0000 7f1f15ffb700 1 -- 172.21.15.26:0/1776827095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1001b9a0 con 0x7f1f18108910 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.374+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181a6580 con 0x7f1f18108910 2024-05-20T20:55:02.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.376+0000 7f1f15ffb700 1 -- 172.21.15.26:0/1776827095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f10022070 con 0x7f1f18108910 2024-05-20T20:55:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f15ffb700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f080b2880 0x7f1f080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f080b2880 0x7f1f080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f15ffb700 1 -- 172.21.15.26:0/1776827095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1f1000f020 con 0x7f1f18108910 2024-05-20T20:55:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f080b2880 0x7f1f080b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f1819a9b0 tx=0x7f1f0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 Processor -- start 2024-05-20T20:55:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 -- start start 2024-05-20T20:55:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57beb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57beb7f0 0x561f57bef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57bef890 0x561f57befd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f57b44d20 con 0x561f57beae40 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561f57b348c0 con 0x561f57bef890 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.378+0000 7f1f299beb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f57b38900 con 0x561f57beb7f0 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57beb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57beb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46088/0 (socket says 172.21.15.26:46088) 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 -- 172.21.15.26:0/2975225760 learned_addr learned my addr 172.21.15.26:0/2975225760 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f17fff700 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57beb7f0 0x561f57bef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 -- 172.21.15.26:0/2975225760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57beb7f0 msgr2=0x561f57bef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57beb7f0 0x561f57bef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 -- 172.21.15.26:0/2975225760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57bef890 msgr2=0x561f57befd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1d6c5700 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57bef890 0x561f57befd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57bef890 0x561f57befd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 -- 172.21.15.26:0/2975225760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f57addf60 con 0x561f57beae40 2024-05-20T20:55:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57beb220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c010f70 tx=0x7f1f0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5150578b1e06c92f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f02ffd700 1 -- 172.21.15.26:0/2975225760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00be60 con 0x561f57beae40 2024-05-20T20:55:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f02ffd700 1 -- 172.21.15.26:0/2975225760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f0c011db0 con 0x561f57beae40 2024-05-20T20:55:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.379+0000 7f1f02ffd700 1 -- 172.21.15.26:0/2975225760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c01c5f0 con 0x561f57beae40 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 -- 172.21.15.26:0/2975225760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 msgr2=0x561f57beb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57beb220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c010f70 tx=0x7f1f0c013000 comp rx=0 tx=0).stop 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 -- 172.21.15.26:0/2975225760 shutdown_connections 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57beb220 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57bef890 0x561f57befd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 --2- 172.21.15.26:0/2975225760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57beb7f0 0x561f57bef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 -- 172.21.15.26:0/2975225760 >> 172.21.15.26:0/2975225760 conn(0x561f57b121f0 msgr2=0x561f57be1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 -- 172.21.15.26:0/2975225760 shutdown_connections 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 -- 172.21.15.26:0/2975225760 wait complete. 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.380+0000 7f1f299beb80 1 -- start start 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 Processor -- start 2024-05-20T20:55:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beae40 0x561f57bf6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 0x561f57bf40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f57a3af50 con 0x561f57beb7f0 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561f57a9cd50 con 0x561f57beae40 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f57bf7a40 con 0x561f57bef890 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f299beb80 10 client.0 _ll_register_callbacks cb 0x561f57acf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f17fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 0x561f57bf40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f17fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 0x561f57bf40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46102/0 (socket says 172.21.15.26:46102) 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beae40 0x561f57bf6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f17fff700 1 -- 172.21.15.26:0/927088838 learned_addr learned my addr 172.21.15.26:0/927088838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.381+0000 7f1f1d6c5700 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f17fff700 1 -- 172.21.15.26:0/927088838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 msgr2=0x561f57bf45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f17fff700 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f17fff700 1 -- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beae40 msgr2=0x561f57bf6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f17fff700 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beae40 0x561f57bf6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f17fff700 1 -- 172.21.15.26:0/927088838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f57a84880 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f17fff700 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 0x561f57bf40b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011a60 tx=0x7f1f04011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11c24bf98f360b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1ef7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1ef7fff700 1 -- 172.21.15.26:0/927088838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400fcc0 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0c011db0 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1ef7fff700 1 -- 172.21.15.26:0/927088838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1f04004830 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1ef7fff700 1 -- 172.21.15.26:0/927088838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f0400fe60 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f299beb80 10 client.14994 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f57aee710 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.382+0000 7f1ef7fff700 1 -- 172.21.15.26:0/927088838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f0c011db0 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1ef7fff700 1 -- 172.21.15.26:0/927088838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1f04018030 con 0x561f57beb7f0 2024-05-20T20:55:02.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f299beb80 10 client.14994 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f299beb80 10 client.14994 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f299beb80 20 client.14994 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f299beb80 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c04780 0x561f57c05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f299beb80 4 client.14994 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f57b46c70 con 0x561f57c04780 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.383+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c04780 0x561f57c05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c04780 0x561f57c05bf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6e6609762329b8 server_cookie=879f99e7dfe64a13 in_seq=0 out_seq=0 2024-05-20T20:55:02.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1ef7fff700 10 client.14994 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1ef7fff700 1 -- 172.21.15.26:0/927088838 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561f57b46c70 con 0x561f57c04780 2024-05-20T20:55:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1ef7fff700 10 client.14994 handle_command_reply: tid=0 2024-05-20T20:55:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1f299beb80 1 client.14994 shutdown 2024-05-20T20:55:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 msgr2=0x561f57bf40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.384+0000 7f1f299beb80 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 0x561f57bf40b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011a60 tx=0x7f1f04011d70 comp rx=0 tx=0).stop 2024-05-20T20:55:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 shutdown_connections 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c04780 0x561f57c05bf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1ef7fff700 0 client.14994 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beae40 0x561f57bf6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 --2- 172.21.15.26:0/927088838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beb7f0 0x561f57bf40b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 >> 172.21.15.26:0/927088838 conn(0x561f57b121f0 msgr2=0x561f57bf0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 shutdown_connections 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 1 -- 172.21.15.26:0/927088838 wait complete. 2024-05-20T20:55:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.385+0000 7f1f299beb80 20 client.14994 trim_cache size 0 max 16384 2024-05-20T20:55:02.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.406+0000 7f1f299beb80 1 Processor -- start 2024-05-20T20:55:02.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.406+0000 7f1f299beb80 1 -- start start 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57be0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57bf1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f299beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f57c03f30 con 0x561f57beae40 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f299beb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561f57c04600 con 0x561f57beb7f0 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f299beb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f57c04cd0 con 0x561f57bef890 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57be0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57be0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46108/0 (socket says 172.21.15.26:46108) 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 -- 172.21.15.26:0/778210191 learned_addr learned my addr 172.21.15.26:0/778210191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f17fff700 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57bf1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 -- 172.21.15.26:0/778210191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 msgr2=0x561f57bf1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 -- 172.21.15.26:0/778210191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 msgr2=0x561f57bf1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57bf1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 -- 172.21.15.26:0/778210191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f57bf2620 con 0x561f57beae40 2024-05-20T20:55:02.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.407+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57be0920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005d90 tx=0x7f1f0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ba771a20db0b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:02.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f027fc700 1 -- 172.21.15.26:0/778210191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00bdb0 con 0x561f57beae40 2024-05-20T20:55:02.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f027fc700 1 -- 172.21.15.26:0/778210191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f0c0055d0 con 0x561f57beae40 2024-05-20T20:55:02.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f027fc700 1 -- 172.21.15.26:0/778210191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c01e420 con 0x561f57beae40 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 -- 172.21.15.26:0/778210191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 msgr2=0x561f57be0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57be0920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005d90 tx=0x7f1f0c016000 comp rx=0 tx=0).stop 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 -- 172.21.15.26:0/778210191 shutdown_connections 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57be0920 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57bf1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 --2- 172.21.15.26:0/778210191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57bf1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 -- 172.21.15.26:0/778210191 >> 172.21.15.26:0/778210191 conn(0x561f57b121f0 msgr2=0x561f57be0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 -- 172.21.15.26:0/778210191 shutdown_connections 2024-05-20T20:55:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.408+0000 7f1f299beb80 1 -- 172.21.15.26:0/778210191 wait complete. 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 -- start start 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 Processor -- start 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57c08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57c093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57c09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f57c05a70 con 0x561f57beae40 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561f57c053a0 con 0x561f57beb7f0 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f57c06140 con 0x561f57bef890 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f299beb80 10 client.0 _ll_register_callbacks cb 0x561f57acf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57c08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f1cec4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57c08eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46110/0 (socket says 172.21.15.26:46110) 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f1cec4700 1 -- 172.21.15.26:0/3280364682 learned_addr learned my addr 172.21.15.26:0/3280364682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.409+0000 7f1f17fff700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57c093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1d6c5700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57c09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1cec4700 1 -- 172.21.15.26:0/3280364682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 msgr2=0x561f57c09930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57c09930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1cec4700 1 -- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 msgr2=0x561f57c093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57c093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1cec4700 1 -- 172.21.15.26:0/3280364682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f57c0d880 con 0x561f57beae40 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f17fff700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57c093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f1cec4700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57c08eb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0058f0 tx=0x7f1f0c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfcdcdb3c1df160 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1ef7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1ef7fff700 1 -- 172.21.15.26:0/3280364682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c005350 con 0x561f57beae40 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1ef7fff700 1 -- 172.21.15.26:0/3280364682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1f0c004b90 con 0x561f57beae40 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0c0055d0 con 0x561f57beae40 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1ef7fff700 1 -- 172.21.15.26:0/3280364682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f0c004d90 con 0x561f57beae40 2024-05-20T20:55:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f299beb80 10 client.15000 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.410+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f57be6850 con 0x561f57beae40 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1ef7fff700 1 -- 172.21.15.26:0/3280364682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f0c0055d0 con 0x561f57beae40 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1ef7fff700 1 -- 172.21.15.26:0/3280364682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1f0c01eac0 con 0x561f57beae40 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f299beb80 10 client.15000 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f299beb80 10 client.15000 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f299beb80 20 client.15000 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f299beb80 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c70490 0x561f57be9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f299beb80 4 client.15000 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "status"}) -- 0x561f57b40450 con 0x561f57c70490 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.411+0000 7f1f17fff700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c70490 0x561f57be9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f17fff700 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c70490 0x561f57be9a40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c2f220164118b8 server_cookie=3f2aacf498d821ea in_seq=0 out_seq=0 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1ef7fff700 10 client.15000 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1ef7fff700 1 -- 172.21.15.26:0/3280364682 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561f57b40450 con 0x561f57c70490 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1ef7fff700 10 client.15000 handle_command_reply: tid=0 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 client.15000 shutdown 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 msgr2=0x561f57c08eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57c08eb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0058f0 tx=0x7f1f0c011bc0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 shutdown_connections 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561f57c70490 0x561f57be9a40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f57beae40 0x561f57c08eb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f57bef890 0x561f57c09930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 --2- 172.21.15.26:0/3280364682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f57beb7f0 0x561f57c093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 >> 172.21.15.26:0/3280364682 conn(0x561f57b121f0 msgr2=0x561f57bee6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1ef7fff700 0 client.15000 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 shutdown_connections 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 1 -- 172.21.15.26:0/3280364682 wait complete. 2024-05-20T20:55:02.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.412+0000 7f1f299beb80 20 client.15000 trim_cache size 0 max 16384 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24476, 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 135.18487250300001, 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-05-20T20:55:02.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:55:02.418 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T20:55:02.418 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 151.34637001999999 2024-05-20T20:55:02.418 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:55:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f080b2880 msgr2=0x7f1f080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f080b2880 0x7f1f080b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f1819a9b0 tx=0x7f1f0c00b410 comp rx=0 tx=0).stop 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 msgr2=0x7f1f18199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18199870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009850 tx=0x7f1f10007d90 comp rx=0 tx=0).stop 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 shutdown_connections 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f080b2880 0x7f1f080b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f18108910 0x7f1f18199870 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f18103820 0x7f1f1819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 --2- 172.21.15.26:0/1776827095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f181041d0 0x7f1f1819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 >> 172.21.15.26:0/1776827095 conn(0x7f1f1807a6b0 msgr2=0x7f1f181018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 shutdown_connections 2024-05-20T20:55:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:02.413+0000 7f1f1f128700 1 -- 172.21.15.26:0/1776827095 wait complete. 2024-05-20T20:55:03.033 INFO:tasks.cephadm:Setup complete, yielding 2024-05-20T20:55:03.033 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-20T20:55:03.048 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T20:55:03.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-20T20:55:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:03 smithi144 ceph-mon[35660]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:03.423 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:03 smithi026 ceph-mon[31990]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:03 smithi055 ceph-mon[35213]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 -- 172.21.15.26:0/360203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 msgr2=0x7fdbd81039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 --2- 172.21.15.26:0/360203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81039d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80099c0 tx=0x7fdbc8009cd0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 -- 172.21.15.26:0/360203319 shutdown_connections 2024-05-20T20:55:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 --2- 172.21.15.26:0/360203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81039d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 --2- 172.21.15.26:0/360203319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 0x7fdbd810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 --2- 172.21.15.26:0/360203319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 0x7fdbd8104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 -- 172.21.15.26:0/360203319 >> 172.21.15.26:0/360203319 conn(0x7fdbd80feb50 msgr2=0x7fdbd8100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 -- 172.21.15.26:0/360203319 shutdown_connections 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.032+0000 7fdbde476700 1 -- 172.21.15.26:0/360203319 wait complete. 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 Processor -- start 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 -- start start 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 0x7fdbd8199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 0x7fdbd81a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810efd0 con 0x7fdbd81035b0 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810ee50 con 0x7fdbd8105150 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbde476700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810f150 con 0x7fdbd81047b0 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:05.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46124/0 (socket says 172.21.15.26:46124) 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbd7fff700 1 -- 172.21.15.26:0/3957064946 learned_addr learned my addr 172.21.15.26:0/3957064946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbcffff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 0x7fdbd8199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.033+0000 7fdbdca13700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 0x7fdbd81a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd7fff700 1 -- 172.21.15.26:0/3957064946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 msgr2=0x7fdbd8199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd7fff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 0x7fdbd8199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd7fff700 1 -- 172.21.15.26:0/3957064946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 msgr2=0x7fdbd81a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd7fff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 0x7fdbd81a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd7fff700 1 -- 172.21.15.26:0/3957064946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81a1320 con 0x7fdbd81035b0 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbcffff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 0x7fdbd8199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbdca13700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 0x7fdbd81a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd7fff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81996c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800b580 tx=0x7fdbc800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8027070 con 0x7fdbd81035b0 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc800fba0 con 0x7fdbd81035b0 2024-05-20T20:55:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81a15b0 con 0x7fdbd81035b0 2024-05-20T20:55:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80037f0 con 0x7fdbd81035b0 2024-05-20T20:55:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.034+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81a1a90 con 0x7fdbd81035b0 2024-05-20T20:55:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.035+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd806a230 con 0x7fdbd81035b0 2024-05-20T20:55:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.037+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdbc802a070 con 0x7fdbd81035b0 2024-05-20T20:55:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.038+0000 7fdbd5ffb700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdbc40b2880 0x7fdbc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.038+0000 7fdbcffff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdbc40b2880 0x7fdbc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.038+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fdbc80e37c0 con 0x7fdbd81035b0 2024-05-20T20:55:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.038+0000 7fdbcffff700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdbc40b2880 0x7fdbc40b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00099d0 tx=0x7fdbc0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.042+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbc8031070 con 0x7fdbd81035b0 2024-05-20T20:55:05.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:05 smithi026 ceph-mon[31990]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.275+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdbd8063980 con 0x7fdbc40b2880 2024-05-20T20:55:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.276+0000 7fdbd5ffb700 1 -- 172.21.15.26:0/3957064946 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdbd8063980 con 0x7fdbc40b2880 2024-05-20T20:55:05.277 INFO:teuthology.orchestra.run.smithi026.stdout:Backend: cephadm 2024-05-20T20:55:05.277 INFO:teuthology.orchestra.run.smithi026.stdout:Available: Yes 2024-05-20T20:55:05.277 INFO:teuthology.orchestra.run.smithi026.stdout:Paused: No 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdbc40b2880 msgr2=0x7fdbc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdbc40b2880 0x7fdbc40b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00099d0 tx=0x7fdbc0008040 comp rx=0 tx=0).stop 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 msgr2=0x7fdbd81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81996c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800b580 tx=0x7fdbc800f740 comp rx=0 tx=0).stop 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 shutdown_connections 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdbc40b2880 0x7fdbc40b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd81035b0 0x7fdbd81996c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd8105150 0x7fdbd81a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 --2- 172.21.15.26:0/3957064946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbd81047b0 0x7fdbd8199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 >> 172.21.15.26:0/3957064946 conn(0x7fdbd80feb50 msgr2=0x7fdbd8107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 shutdown_connections 2024-05-20T20:55:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:05.279+0000 7fdbde476700 1 -- 172.21.15.26:0/3957064946 wait complete. 2024-05-20T20:55:05.302 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:05 smithi144 ceph-mon[35660]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:05.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:05 smithi055 ceph-mon[35213]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:05.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-20T20:55:06.178 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:07 smithi144 ceph-mon[35660]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:07 smithi144 ceph-mon[35660]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:07.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:07 smithi026 ceph-mon[31990]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:07.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:07 smithi026 ceph-mon[31990]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:07 smithi055 ceph-mon[35213]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:07 smithi055 ceph-mon[35213]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:07.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/1942361285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe841035a0 msgr2=0x7fbe841039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:07.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/1942361285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe841035a0 0x7fbe841039c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe800099c0 tx=0x7fbe80009cd0 comp rx=0 tx=0).stop 2024-05-20T20:55:07.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/1942361285 shutdown_connections 2024-05-20T20:55:07.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/1942361285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe841035a0 0x7fbe841039c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/1942361285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe84105140 0x7fbe8410b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/1942361285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe841047a0 0x7fbe84104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/1942361285 >> 172.21.15.26:0/1942361285 conn(0x7fbe840feb40 msgr2=0x7fbe84100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/1942361285 shutdown_connections 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.859+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/1942361285 wait complete. 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 Processor -- start 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 -- start start 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe841035a0 0x7fbe840772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:07.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe841047a0 0x7fbe84075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 0x7fbe84075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410ef60 con 0x7fbe84105140 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410ede0 con 0x7fbe841047a0 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8c6ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410f0e0 con 0x7fbe841035a0 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 0x7fbe84075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8a449700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe841035a0 0x7fbe840772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 0x7fbe84075e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46156/0 (socket says 172.21.15.26:46156) 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe89c48700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe841047a0 0x7fbe84075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 -- 172.21.15.26:0/2228984581 learned_addr learned my addr 172.21.15.26:0/2228984581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:07.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 -- 172.21.15.26:0/2228984581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe841035a0 msgr2=0x7fbe840772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe841035a0 0x7fbe840772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 -- 172.21.15.26:0/2228984581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe841047a0 msgr2=0x7fbe84075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe841047a0 0x7fbe84075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.860+0000 7fbe8ac4a700 1 -- 172.21.15.26:0/2228984581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe84076720 con 0x7fbe84105140 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.861+0000 7fbe8ac4a700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 0x7fbe84075e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007ee0 tx=0x7fbe7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.861+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c004bb0 con 0x7fbe84105140 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.861+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe7c004d50 con 0x7fbe84105140 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.861+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe841a3850 con 0x7fbe84105140 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.861+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0058b0 con 0x7fbe84105140 2024-05-20T20:55:07.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.861+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe841a3d60 con 0x7fbe84105140 2024-05-20T20:55:07.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.862+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8406a230 con 0x7fbe84105140 2024-05-20T20:55:07.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.863+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbe7c01a020 con 0x7fbe84105140 2024-05-20T20:55:07.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.864+0000 7fbe7b7fe700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe700b2880 0x7fbe700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:07.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.864+0000 7fbe8a449700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe700b2880 0x7fbe700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:07.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.864+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbe7c0e1f10 con 0x7fbe84105140 2024-05-20T20:55:07.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.864+0000 7fbe8a449700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe700b2880 0x7fbe700b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe80003850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:07.867+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe7c010070 con 0x7fbe84105140 2024-05-20T20:55:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.094+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbe84063980 con 0x7fbe700b2880 2024-05-20T20:55:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.107+0000 7fbe7b7fe700 1 -- 172.21.15.26:0/2228984581 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fbe84063980 con 0x7fbe700b2880 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.a smithi026 running (2m) 19s ago 2m 16.0M - 19.0.0-3830-g4ebd1d53 559f9f297837 f0eb87a4777e 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.b smithi055 running (2m) 21s ago 2m 26.3M - 19.0.0-3830-g4ebd1d53 559f9f297837 cf6c4f297a75 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.c smithi144 running (2m) 21s ago 2m 23.1M - 19.0.0-3830-g4ebd1d53 559f9f297837 a4e344c6d0bd 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.d smithi026 running (3m) 19s ago 3m 14.9M - 19.0.0-3830-g4ebd1d53 559f9f297837 e9bd9787d02a 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.e smithi055 running (2m) 21s ago 2m 21.0M - 19.0.0-3830-g4ebd1d53 559f9f297837 19dfeeff8356 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.f smithi144 running (2m) 21s ago 2m 21.9M - 19.0.0-3830-g4ebd1d53 559f9f297837 f4802ebc1add 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.g smithi026 running (2m) 19s ago 2m 15.6M - 19.0.0-3830-g4ebd1d53 559f9f297837 2ef8b14ea257 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.h smithi055 running (2m) 21s ago 2m 21.2M - 19.0.0-3830-g4ebd1d53 559f9f297837 bacb3afa4c4e 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.i smithi144 running (2m) 21s ago 2m 21.6M - 19.0.0-3830-g4ebd1d53 559f9f297837 12ed66c231dd 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.j smithi026 running (3m) 19s ago 3m 17.1M - 19.0.0-3830-g4ebd1d53 559f9f297837 258689f544bb 2024-05-20T20:55:08.108 INFO:teuthology.orchestra.run.smithi026.stdout:mds.k smithi055 running (3m) 21s ago 3m 16.4M - 19.0.0-3830-g4ebd1d53 559f9f297837 dc604ff494df 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mds.l smithi144 running (3m) 21s ago 3m 17.3M - 19.0.0-3830-g4ebd1d53 559f9f297837 287aed43d193 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.x smithi026 *:9283,8765 running (10m) 19s ago 10m 530M - 19.0.0-3830-g4ebd1d53 559f9f297837 42c0ddaaf231 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.y smithi055 *:8443,8765 running (8m) 21s ago 8m 467M - 19.0.0-3830-g4ebd1d53 559f9f297837 b53f9546ce24 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.z smithi144 *:8443,8765 running (8m) 21s ago 8m 469M - 19.0.0-3830-g4ebd1d53 559f9f297837 a2493a2d531c 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mon.a smithi026 running (10m) 19s ago 10m 53.0M 2048M 19.0.0-3830-g4ebd1d53 559f9f297837 1dab537c29f1 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mon.b smithi055 running (8m) 21s ago 8m 40.5M 2048M 19.0.0-3830-g4ebd1d53 559f9f297837 b34a9406fbc1 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:mon.c smithi144 running (8m) 21s ago 8m 41.9M 2048M 19.0.0-3830-g4ebd1d53 559f9f297837 29b7074c07fe 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 smithi026 running (8m) 19s ago 8m 56.9M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 0eea9d19d914 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 smithi055 running (7m) 21s ago 7m 46.5M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 85832347393d 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 smithi144 running (7m) 21s ago 7m 46.6M 4272M 19.0.0-3830-g4ebd1d53 559f9f297837 d72e4239d0a0 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 smithi026 running (6m) 19s ago 6m 56.2M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 ffdfcec885ec 2024-05-20T20:55:08.109 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 smithi055 running (6m) 21s ago 6m 42.9M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 e2c2821221a6 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 smithi144 running (6m) 21s ago 6m 44.0M 4272M 19.0.0-3830-g4ebd1d53 559f9f297837 463642c18c26 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 smithi026 running (5m) 19s ago 5m 42.9M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 75de8d0296e1 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 smithi055 running (5m) 21s ago 5m 44.3M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 1c1989aaaff6 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 smithi144 running (4m) 21s ago 4m 46.7M 4272M 19.0.0-3830-g4ebd1d53 559f9f297837 5e26e79379ed 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 smithi026 running (4m) 19s ago 4m 43.5M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 aa9f945a3244 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 smithi055 running (3m) 21s ago 3m 57.3M 4273M 19.0.0-3830-g4ebd1d53 559f9f297837 3666ef2c8f55 2024-05-20T20:55:08.110 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 smithi144 running (3m) 21s ago 3m 42.7M 4272M 19.0.0-3830-g4ebd1d53 559f9f297837 84ea143d80a2 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe700b2880 msgr2=0x7fbe700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe700b2880 0x7fbe700b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe80003850 comp rx=0 tx=0).stop 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 msgr2=0x7fbe84075e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 0x7fbe84075e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007ee0 tx=0x7fbe7c00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 shutdown_connections 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe700b2880 0x7fbe700b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe84105140 0x7fbe84075e70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe841035a0 0x7fbe840772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 --2- 172.21.15.26:0/2228984581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe841047a0 0x7fbe84075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 >> 172.21.15.26:0/2228984581 conn(0x7fbe840feb40 msgr2=0x7fbe84107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 shutdown_connections 2024-05-20T20:55:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:08.110+0000 7fbe8c6ad700 1 -- 172.21.15.26:0/2228984581 wait complete. 2024-05-20T20:55:08.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-20T20:55:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:09 smithi144 ceph-mon[35660]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:09 smithi144 ceph-mon[35660]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:09.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:09 smithi026 ceph-mon[31990]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:09.375 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:09 smithi026 ceph-mon[31990]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:09 smithi055 ceph-mon[35213]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:09.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:09 smithi055 ceph-mon[35213]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/3372768534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08104040 msgr2=0x7fdb0810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/3372768534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08104040 0x7fdb0810a8f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdb000099c0 tx=0x7fdb00009cd0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/3372768534 shutdown_connections 2024-05-20T20:55:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/3372768534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08104040 0x7fdb0810a8f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/3372768534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb081024a0 0x7fdb081028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/3372768534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb081036a0 0x7fdb08103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/3372768534 >> 172.21.15.26:0/3372768534 conn(0x7fdb080fda40 msgr2=0x7fdb080ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/3372768534 shutdown_connections 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.668+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/3372768534 wait complete. 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 Processor -- start 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 -- start start 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb081024a0 0x7fdb0806bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 0x7fdb0819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb08104040 0x7fdb0819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0810ddd0 con 0x7fdb081036a0 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0810dc50 con 0x7fdb08104040 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0eb2f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0810df50 con 0x7fdb081024a0 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 0x7fdb0819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:10.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 0x7fdb0819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33470/0 (socket says 172.21.15.26:33470) 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 -- 172.21.15.26:0/1048530475 learned_addr learned my addr 172.21.15.26:0/1048530475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0c8cb700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb081024a0 0x7fdb0806bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb0d0cc700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb08104040 0x7fdb0819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 -- 172.21.15.26:0/1048530475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb081024a0 msgr2=0x7fdb0806bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb081024a0 0x7fdb0806bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 -- 172.21.15.26:0/1048530475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb08104040 msgr2=0x7fdb0819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb08104040 0x7fdb0819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.669+0000 7fdb07fff700 1 -- 172.21.15.26:0/1048530475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0819cfa0 con 0x7fdb081036a0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.670+0000 7fdb07fff700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 0x7fdb0819c050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8007e20 tx=0x7fdaf800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.670+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8016020 con 0x7fdb081036a0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.670+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0819d290 con 0x7fdb081036a0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.670+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf800ad40 con 0x7fdb081036a0 2024-05-20T20:55:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.670+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80115e0 con 0x7fdb081036a0 2024-05-20T20:55:10.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.670+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0819d7d0 con 0x7fdb081036a0 2024-05-20T20:55:10.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.671+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0806a230 con 0x7fdb081036a0 2024-05-20T20:55:10.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.672+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdaf8019020 con 0x7fdb081036a0 2024-05-20T20:55:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.673+0000 7fdb05ffb700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf40b2880 0x7fdaf40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.673+0000 7fdb0c8cb700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf40b2880 0x7fdaf40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.673+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fdaf80e1d20 con 0x7fdb081036a0 2024-05-20T20:55:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.674+0000 7fdb0c8cb700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf40b2880 0x7fdaf40b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb08103500 tx=0x7fdaf0007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:10.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.678+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdaf8020070 con 0x7fdb081036a0 2024-05-20T20:55:10.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.902+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fdb081096c0 con 0x7fdaf40b2880 2024-05-20T20:55:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.908+0000 7fdb05ffb700 1 -- 172.21.15.26:0/1048530475 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fdb081096c0 con 0x7fdaf40b2880 2024-05-20T20:55:10.908 INFO:teuthology.orchestra.run.smithi026.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-20T20:55:10.908 INFO:teuthology.orchestra.run.smithi026.stdout:mds.all 12/12 24s ago 3m smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi055=b;smithi055=e;smithi055=h;smithi055=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-05-20T20:55:10.909 INFO:teuthology.orchestra.run.smithi026.stdout:mgr 3/3 24s ago 8m smithi026=x;smithi055=y;smithi144=z;count:3 2024-05-20T20:55:10.909 INFO:teuthology.orchestra.run.smithi026.stdout:mon 3/3 24s ago 8m smithi026:172.21.15.26=a;smithi055:172.21.15.55=b;smithi144:172.21.15.144=c;count:3 2024-05-20T20:55:10.909 INFO:teuthology.orchestra.run.smithi026.stdout:osd 12 24s ago - 2024-05-20T20:55:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf40b2880 msgr2=0x7fdaf40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf40b2880 0x7fdaf40b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb08103500 tx=0x7fdaf0007400 comp rx=0 tx=0).stop 2024-05-20T20:55:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 msgr2=0x7fdb0819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 0x7fdb0819c050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8007e20 tx=0x7fdaf800a340 comp rx=0 tx=0).stop 2024-05-20T20:55:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 shutdown_connections 2024-05-20T20:55:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb08104040 0x7fdb0819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb081024a0 0x7fdb0806bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf40b2880 0x7fdaf40b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 --2- 172.21.15.26:0/1048530475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb081036a0 0x7fdb0819c050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 >> 172.21.15.26:0/1048530475 conn(0x7fdb080fda40 msgr2=0x7fdb08106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 shutdown_connections 2024-05-20T20:55:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:10.910+0000 7fdb0eb2f700 1 -- 172.21.15.26:0/1048530475 wait complete. 2024-05-20T20:55:11.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:11 smithi026 ceph-mon[31990]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:11 smithi055 ceph-mon[35213]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:11 smithi144 ceph-mon[35660]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:11.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-20T20:55:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:12.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:12 smithi026 ceph-mon[31990]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:12.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:12 smithi055 ceph-mon[35213]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:12.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:12 smithi144 ceph-mon[35660]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:13.376 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:13 smithi026 ceph-mon[31990]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:13 smithi055 ceph-mon[35213]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:13 smithi144 ceph-mon[35660]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 -- 172.21.15.26:0/3392515302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 msgr2=0x7f964410a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 --2- 172.21.15.26:0/3392515302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f964410a8f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b0d0 tx=0x7f963c00b3e0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 -- 172.21.15.26:0/3392515302 shutdown_connections 2024-05-20T20:55:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 --2- 172.21.15.26:0/3392515302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f964410a8f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 --2- 172.21.15.26:0/3392515302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96441024a0 0x7f96441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 --2- 172.21.15.26:0/3392515302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96441036a0 0x7f9644103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 -- 172.21.15.26:0/3392515302 >> 172.21.15.26:0/3392515302 conn(0x7f96440fda40 msgr2=0x7f96440ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 -- 172.21.15.26:0/3392515302 shutdown_connections 2024-05-20T20:55:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.606+0000 7f9649edf700 1 -- 172.21.15.26:0/3392515302 wait complete. 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 Processor -- start 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 -- start start 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96441024a0 0x7f9644199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96441036a0 0x7f9644199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f96441a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f964410dee0 con 0x7f9644104040 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f964410dd60 con 0x7f96441024a0 2024-05-20T20:55:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9649edf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f964410e060 con 0x7f96441036a0 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9643fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f96441a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9643fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f96441a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33494/0 (socket says 172.21.15.26:33494) 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9643fff700 1 -- 172.21.15.26:0/997745283 learned_addr learned my addr 172.21.15.26:0/997745283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f96437fe700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96441024a0 0x7f9644199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.607+0000 7f9642ffd700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96441036a0 0x7f9644199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9643fff700 1 -- 172.21.15.26:0/997745283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96441036a0 msgr2=0x7f9644199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9643fff700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96441036a0 0x7f9644199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9643fff700 1 -- 172.21.15.26:0/997745283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96441024a0 msgr2=0x7f9644199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9643fff700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96441024a0 0x7f9644199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9643fff700 1 -- 172.21.15.26:0/997745283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a13c0 con 0x7f9644104040 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9643fff700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f96441a0d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f963c009c30 tx=0x7f963c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c01f030 con 0x7f9644104040 2024-05-20T20:55:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f963c005040 con 0x7f9644104040 2024-05-20T20:55:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c01b690 con 0x7f9644104040 2024-05-20T20:55:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a1650 con 0x7f9644104040 2024-05-20T20:55:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.608+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441a1a70 con 0x7f9644104040 2024-05-20T20:55:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.609+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964406a260 con 0x7f9644104040 2024-05-20T20:55:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.610+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f963c0054f0 con 0x7f9644104040 2024-05-20T20:55:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.611+0000 7f9640ff9700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f962c0b2880 0x7f962c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.611+0000 7f96437fe700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f962c0b2880 0x7f962c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.611+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f963c00f020 con 0x7f9644104040 2024-05-20T20:55:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.612+0000 7f96437fe700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f962c0b2880 0x7f962c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9644103500 tx=0x7f9634009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.615+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f964406a260 con 0x7f9644104040 2024-05-20T20:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.836+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f964406b930 con 0x7f962c0b2880 2024-05-20T20:55:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.837+0000 7f9640ff9700 1 -- 172.21.15.26:0/997745283 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f964406b930 con 0x7f962c0b2880 2024-05-20T20:55:13.836 INFO:teuthology.orchestra.run.smithi026.stdout:HOST ADDR LABELS STATUS 2024-05-20T20:55:13.836 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 172.21.15.26 2024-05-20T20:55:13.836 INFO:teuthology.orchestra.run.smithi026.stdout:smithi055 172.21.15.55 2024-05-20T20:55:13.837 INFO:teuthology.orchestra.run.smithi026.stdout:smithi144 172.21.15.144 2024-05-20T20:55:13.837 INFO:teuthology.orchestra.run.smithi026.stdout:3 hosts in cluster 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f962c0b2880 msgr2=0x7f962c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f962c0b2880 0x7f962c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9644103500 tx=0x7f9634009190 comp rx=0 tx=0).stop 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 msgr2=0x7f96441a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f96441a0d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f963c009c30 tx=0x7f963c009c60 comp rx=0 tx=0).stop 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 shutdown_connections 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f962c0b2880 0x7f962c0b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9644104040 0x7f96441a0d20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96441024a0 0x7f9644199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 --2- 172.21.15.26:0/997745283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96441036a0 0x7f9644199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 >> 172.21.15.26:0/997745283 conn(0x7f96440fda40 msgr2=0x7f964406b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 shutdown_connections 2024-05-20T20:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:13.840+0000 7f9649edf700 1 -- 172.21.15.26:0/997745283 wait complete. 2024-05-20T20:55:14.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-20T20:55:14.740 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:15.180 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:15 smithi026 ceph-mon[31990]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:15.180 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:15 smithi026 ceph-mon[31990]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:15 smithi055 ceph-mon[35213]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:15 smithi055 ceph-mon[35213]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:15 smithi144 ceph-mon[35660]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:15 smithi144 ceph-mon[35660]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 -- 172.21.15.26:0/1894503084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6540101e40 msgr2=0x7f6540104270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 --2- 172.21.15.26:0/1894503084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6540101e40 0x7f6540104270 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f653c009990 tx=0x7f653c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 -- 172.21.15.26:0/1894503084 shutdown_connections 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 --2- 172.21.15.26:0/1894503084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6540101e40 0x7f6540104270 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 --2- 172.21.15.26:0/1894503084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6540107180 0x7f6540107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 --2- 172.21.15.26:0/1894503084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65401047b0 0x7f6540106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 -- 172.21.15.26:0/1894503084 >> 172.21.15.26:0/1894503084 conn(0x7f65400fda60 msgr2=0x7f65400ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 -- 172.21.15.26:0/1894503084 shutdown_connections 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.299+0000 7f65477e3700 1 -- 172.21.15.26:0/1894503084 wait complete. 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 Processor -- start 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 -- start start 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6540101e40 0x7f654019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 0x7f65401a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6540107180 0x7f65401a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6540115530 con 0x7f65401047b0 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f65401153b0 con 0x7f6540107180 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f65477e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65401156b0 con 0x7f6540101e40 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f6544d7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 0x7f65401a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f6544d7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 0x7f65401a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33508/0 (socket says 172.21.15.26:33508) 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f6544d7e700 1 -- 172.21.15.26:0/3546938976 learned_addr learned my addr 172.21.15.26:0/3546938976 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f654557f700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6540101e40 0x7f654019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.300+0000 7f6545d80700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6540107180 0x7f65401a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f6544d7e700 1 -- 172.21.15.26:0/3546938976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6540101e40 msgr2=0x7f654019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f6544d7e700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6540101e40 0x7f654019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f6544d7e700 1 -- 172.21.15.26:0/3546938976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6540107180 msgr2=0x7f65401a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f6544d7e700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6540107180 0x7f65401a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f6544d7e700 1 -- 172.21.15.26:0/3546938976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65401a78c0 con 0x7f65401047b0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f6544d7e700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 0x7f65401a0200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f653000b530 tx=0x7f653000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530015020 con 0x7f65401047b0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6530011cc0 con 0x7f65401047b0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65401a7bb0 con 0x7f65401047b0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530010d30 con 0x7f65401047b0 2024-05-20T20:55:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.301+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65401a80a0 con 0x7f65401047b0 2024-05-20T20:55:16.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.302+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654006a230 con 0x7f65401047b0 2024-05-20T20:55:16.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.304+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6530018020 con 0x7f65401047b0 2024-05-20T20:55:16.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.304+0000 7f65367fc700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f652c0b2880 0x7f652c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.305+0000 7f654557f700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f652c0b2880 0x7f652c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.305+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f65300e1ae0 con 0x7f65401047b0 2024-05-20T20:55:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.305+0000 7f654557f700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f652c0b2880 0x7f652c0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f653c00b5c0 tx=0x7f653c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:16.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.309+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f653001f020 con 0x7f65401047b0 2024-05-20T20:55:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.533+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6540063980 con 0x7f652c0b2880 2024-05-20T20:55:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.535+0000 7f65367fc700 1 -- 172.21.15.26:0/3546938976 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6540063980 con 0x7f652c0b2880 2024-05-20T20:55:16.534 INFO:teuthology.orchestra.run.smithi026.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-20T20:55:16.534 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-20T20:55:16.534 INFO:teuthology.orchestra.run.smithi026.stdout:smithi055 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-05-20T20:55:16.534 INFO:teuthology.orchestra.run.smithi026.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-20T20:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f652c0b2880 msgr2=0x7f652c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f652c0b2880 0x7f652c0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f653c00b5c0 tx=0x7f653c005fb0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 msgr2=0x7f65401a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 0x7f65401a0200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f653000b530 tx=0x7f653000b8f0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 shutdown_connections 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6540107180 0x7f65401a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6540101e40 0x7f654019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f652c0b2880 0x7f652c0b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 --2- 172.21.15.26:0/3546938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65401047b0 0x7f65401a0200 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 >> 172.21.15.26:0/3546938976 conn(0x7f65400fda60 msgr2=0x7f65401052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 shutdown_connections 2024-05-20T20:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:16.537+0000 7f65477e3700 1 -- 172.21.15.26:0/3546938976 wait complete. 2024-05-20T20:55:17.169 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:17 smithi026 ceph-mon[31990]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:17.200 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-20T20:55:17.211 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T20:55:17.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-20T20:55:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:17 smithi055 ceph-mon[35213]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:17 smithi144 ceph-mon[35660]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:18.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:18 smithi026 ceph-mon[31990]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:18.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:18 smithi055 ceph-mon[35213]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:18 smithi144 ceph-mon[35660]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 -- 172.21.15.26:0/1793614335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138104030 msgr2=0x7f2138106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 --2- 172.21.15.26:0/1793614335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138104030 0x7f2138106420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2128009990 tx=0x7f2128009ca0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 -- 172.21.15.26:0/1793614335 shutdown_connections 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 --2- 172.21.15.26:0/1793614335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138104030 0x7f2138106420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 --2- 172.21.15.26:0/1793614335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2138109290 0x7f213810b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 --2- 172.21.15.26:0/1793614335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2138106960 0x7f2138108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 -- 172.21.15.26:0/1793614335 >> 172.21.15.26:0/1793614335 conn(0x7f21380fda40 msgr2=0x7f21380ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 -- 172.21.15.26:0/1793614335 shutdown_connections 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.162+0000 7f213e97e700 1 -- 172.21.15.26:0/1793614335 wait complete. 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 Processor -- start 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 -- start start 2024-05-20T20:55:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2138104030 0x7f2138199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2138106960 0x7f2138199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 0x7f21381a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f213810ef50 con 0x7f2138109290 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f213810edd0 con 0x7f2138106960 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213e97e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f213810f0d0 con 0x7f2138104030 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 0x7f21381a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 0x7f21381a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33526/0 (socket says 172.21.15.26:33526) 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 -- 172.21.15.26:0/2581421390 learned_addr learned my addr 172.21.15.26:0/2581421390 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f212ffff700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2138106960 0x7f2138199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f2137fff700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2138104030 0x7f2138199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 -- 172.21.15.26:0/2581421390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2138104030 msgr2=0x7f2138199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2138104030 0x7f2138199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 -- 172.21.15.26:0/2581421390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2138106960 msgr2=0x7f2138199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2138106960 0x7f2138199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.163+0000 7f213cf1b700 1 -- 172.21.15.26:0/2581421390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381a12f0 con 0x7f2138109290 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.164+0000 7f213cf1b700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 0x7f21381a0c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f213000add0 tx=0x7f213000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.164+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130017020 con 0x7f2138109290 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.164+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21381a15e0 con 0x7f2138109290 2024-05-20T20:55:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.164+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f213000ae00 con 0x7f2138109290 2024-05-20T20:55:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.164+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130005280 con 0x7f2138109290 2024-05-20T20:55:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.164+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21381a1ba0 con 0x7f2138109290 2024-05-20T20:55:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.165+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213806a230 con 0x7f2138109290 2024-05-20T20:55:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.166+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f213000c890 con 0x7f2138109290 2024-05-20T20:55:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.167+0000 7f2135ffb700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21240b2880 0x7f21240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.167+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f21300e1830 con 0x7f2138109290 2024-05-20T20:55:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.167+0000 7f2137fff700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21240b2880 0x7f21240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.167+0000 7f2137fff700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21240b2880 0x7f21240b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f212800b5c0 tx=0x7f2128005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.172+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2130021020 con 0x7f2138109290 2024-05-20T20:55:19.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:19 smithi026 ceph-mon[31990]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:19.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:19 smithi055 ceph-mon[35213]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:19.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.435+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2138064150 con 0x7f2138109290 2024-05-20T20:55:19.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.435+0000 7f2135ffb700 1 -- 172.21.15.26:0/2581421390 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3046 (secure 0 0 0) 0x7f21300a9580 con 0x7f2138109290 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:e25 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:legacy client fscid: 1 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem 'cephfs' (1) 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:fs_name cephfs 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 25 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-05-20T20:52:47.191542+0000 2024-05-20T20:55:19.435 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-05-20T20:53:05.138905+0000 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:tableserver 0 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:root 0 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:session_timeout 300 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:session_autoclose 300 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:max_file_size 1099511627776 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:max_xattr_size 65536 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:required_client_features {} 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure 0 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure_osd_epoch 0 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:max_mds 3 2024-05-20T20:55:19.436 INFO:teuthology.orchestra.run.smithi026.stdout:in 0,1,2 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:up {0=24476,1=24455,2=24445} 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:failed 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:damaged 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:stopped 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:data_pools [3] 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:metadata_pool 2 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:inline_data disabled 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:balancer 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:bal_rank_mask -1 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:standby_count_wanted 0 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:qdb_cluster leader: 24476 members: 24445,24476,24455 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b{0:24476} state up:active seq 6 addr [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c{0:24436} state up:standby-replay seq 1 addr [v2:172.21.15.144:6836/539511953,v1:172.21.15.144:6837/539511953] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.437 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.h{1:24455} state up:active seq 12 addr [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.e{1:24428} state up:standby-replay seq 1 addr [v2:172.21.15.55:6834/2325650145,v1:172.21.15.55:6835/2325650145] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.i{2:24445} state up:active seq 11 addr [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.f{2:24424} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/1712709614,v1:172.21.15.144:6835/1712709614] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:Standby daemons: 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.26:6834/1873553840,v1:172.21.15.26:6835/1873553840] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.26:6836/1148458694,v1:172.21.15.26:6837/1148458694] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.26:6838/841729682,v1:172.21.15.26:6839/841729682] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.26:6840/3119421551,v1:172.21.15.26:6841/3119421551] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.144:6832/3730419260,v1:172.21.15.144:6833/3730419260] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.k{-1:24407} state up:standby seq 1 addr [v2:172.21.15.55:6832/2396855261,v1:172.21.15.55:6833/2396855261] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T20:55:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.437+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21240b2880 msgr2=0x7f21240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.437+0000 7f213e97e700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21240b2880 0x7f21240b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f212800b5c0 tx=0x7f2128005fd0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.437+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 msgr2=0x7f21381a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.437+0000 7f213e97e700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 0x7f21381a0c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f213000add0 tx=0x7f213000c330 comp rx=0 tx=0).stop 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 shutdown_connections 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21240b2880 0x7f21240b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2138109290 0x7f21381a0c50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2138104030 0x7f2138199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 --2- 172.21.15.26:0/2581421390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2138106960 0x7f2138199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 >> 172.21.15.26:0/2581421390 conn(0x7f21380fda40 msgr2=0x7f2138104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 shutdown_connections 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:19.438+0000 7f213e97e700 1 -- 172.21.15.26:0/2581421390 wait complete. 2024-05-20T20:55:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:55:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:19 smithi144 ceph-mon[35660]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:20.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-20T20:55:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:20.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:20 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2581421390' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-20T20:55:20.451 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:20 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2581421390' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-20T20:55:20.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:20 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2581421390' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-20T20:55:21.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:21 smithi055 ceph-mon[35213]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:21 smithi026 ceph-mon[31990]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:21 smithi144 ceph-mon[35660]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 -- 172.21.15.26:0/2406664575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f419010fb80 msgr2=0x7f4190111f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 --2- 172.21.15.26:0/2406664575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f419010fb80 0x7f4190111f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f418800b0a0 tx=0x7f418800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 -- 172.21.15.26:0/2406664575 shutdown_connections 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 --2- 172.21.15.26:0/2406664575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f419010fb80 0x7f4190111f70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 --2- 172.21.15.26:0/2406664575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190077570 0x7f41900759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 --2- 172.21.15.26:0/2406664575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4190075fa0 0x7f4190076420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 -- 172.21.15.26:0/2406664575 >> 172.21.15.26:0/2406664575 conn(0x7f41900feb40 msgr2=0x7f4190100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 -- 172.21.15.26:0/2406664575 shutdown_connections 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.044+0000 7f41973f6700 1 -- 172.21.15.26:0/2406664575 wait complete. 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 Processor -- start 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 -- start start 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190075fa0 0x7f419019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 0x7f41901a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f419010fb80 0x7f41901a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4190115560 con 0x7f4190077570 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41901153e0 con 0x7f419010fb80 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f41973f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41901156e0 con 0x7f4190075fa0 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f4194991700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 0x7f41901a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f4194991700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 0x7f41901a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45794/0 (socket says 172.21.15.26:45794) 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f4194991700 1 -- 172.21.15.26:0/2784404031 learned_addr learned my addr 172.21.15.26:0/2784404031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f4195993700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f419010fb80 0x7f41901a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.045+0000 7f4195192700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190075fa0 0x7f419019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4194991700 1 -- 172.21.15.26:0/2784404031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190075fa0 msgr2=0x7f419019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4194991700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190075fa0 0x7f419019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4194991700 1 -- 172.21.15.26:0/2784404031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f419010fb80 msgr2=0x7f41901a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4194991700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f419010fb80 0x7f41901a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4194991700 1 -- 172.21.15.26:0/2784404031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41901a7800 con 0x7f4190077570 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4195192700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190075fa0 0x7f419019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f4194991700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 0x7f41901a00e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f418000c9c0 tx=0x7f418000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180007ac0 con 0x7f4190077570 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4180007c60 con 0x7f4190077570 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41901a7af0 con 0x7f4190077570 2024-05-20T20:55:22.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180018660 con 0x7f4190077570 2024-05-20T20:55:22.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.046+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901a8030 con 0x7f4190077570 2024-05-20T20:55:22.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.047+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419006a230 con 0x7f4190077570 2024-05-20T20:55:22.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.049+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f418000f450 con 0x7f4190077570 2024-05-20T20:55:22.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.049+0000 7f41867fc700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f417c0b2880 0x7f417c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:22.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.050+0000 7f4195192700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f417c0b2880 0x7f417c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:22.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.050+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f41800e2240 con 0x7f4190077570 2024-05-20T20:55:22.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.050+0000 7f4195192700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f417c0b2880 0x7f417c0b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4190077950 tx=0x7f418c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.054+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4180014030 con 0x7f4190077570 2024-05-20T20:55:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.275+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f419002e560 con 0x7f4190077570 2024-05-20T20:55:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.275+0000 7f41867fc700 1 -- 172.21.15.26:0/2784404031 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5370 (secure 0 0 0) 0x7f419002e560 con 0x7f4190077570 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 82 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-05-20T20:44:33.648165+0000 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-05-20T20:54:37.856459+0000 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:crush_version 26 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:full_ratio 0.95 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:backfillfull_ratio 0.9 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:nearfull_ratio 0.85 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:require_min_compat_client luminous 2024-05-20T20:55:22.275 INFO:teuthology.orchestra.run.smithi026.stdout:min_compat_client luminous 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:require_osd_release squid 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:stretch_mode_enabled false 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:max_osd 12 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6802/3292543364,v1:172.21.15.26:6803/3292543364] [v2:172.21.15.26:6804/3292543364,v1:172.21.15.26:6805/3292543364] exists,up c2c397ed-d333-4e7b-ba99-70756dd5fe29 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6800/1106200301,v1:172.21.15.55:6801/1106200301] [v2:172.21.15.55:6802/1106200301,v1:172.21.15.55:6803/1106200301] exists,up d138e804-8e04-4eaf-a954-9ff798e9c4f0 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/632346919,v1:172.21.15.144:6801/632346919] [v2:172.21.15.144:6802/632346919,v1:172.21.15.144:6803/632346919] exists,up f7569bb5-2603-43fa-8f88-5bbba4b6e930 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6810/2074985040,v1:172.21.15.26:6811/2074985040] [v2:172.21.15.26:6812/2074985040,v1:172.21.15.26:6813/2074985040] exists,up eda9b415-a29a-489c-8d1c-2d893c0170e0 2024-05-20T20:55:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6808/1113585138,v1:172.21.15.55:6809/1113585138] [v2:172.21.15.55:6810/1113585138,v1:172.21.15.55:6811/1113585138] exists,up 7ca8decb-66db-4743-bcb4-ca3efeae40dd 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 up in weight 1 up_from 36 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/1448183794,v1:172.21.15.144:6809/1448183794] [v2:172.21.15.144:6810/1448183794,v1:172.21.15.144:6811/1448183794] exists,up 090b604a-f758-4fa3-b06f-dc15da11739f 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6818/1035586360,v1:172.21.15.26:6819/1035586360] [v2:172.21.15.26:6820/1035586360,v1:172.21.15.26:6821/1035586360] exists,up 54095128-6ffb-408b-8ac6-e8728acec871 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6816/1141160889,v1:172.21.15.55:6817/1141160889] [v2:172.21.15.55:6818/1141160889,v1:172.21.15.55:6819/1141160889] exists,up e9320773-2918-438a-8cf7-f9171c836fd1 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 up in weight 1 up_from 53 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/2473562796,v1:172.21.15.144:6817/2473562796] [v2:172.21.15.144:6818/2473562796,v1:172.21.15.144:6819/2473562796] exists,up c564749e-8c86-4759-9818-af2408d1b735 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 up in weight 1 up_from 59 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6826/2188316601,v1:172.21.15.26:6827/2188316601] [v2:172.21.15.26:6828/2188316601,v1:172.21.15.26:6829/2188316601] exists,up 258768d8-6e31-4618-9ec1-12e184f159a1 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 up in weight 1 up_from 64 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6824/46093047,v1:172.21.15.55:6825/46093047] [v2:172.21.15.55:6826/46093047,v1:172.21.15.55:6827/46093047] exists,up ea416660-9136-4ac1-a30e-f449d892b3cc 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 up in weight 1 up_from 70 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/2410307143,v1:172.21.15.144:6825/2410307143] [v2:172.21.15.144:6826/2410307143,v1:172.21.15.144:6827/2410307143] exists,up d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.a [2,11] 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.b [8,11] 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.10 [2,4] 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.0 [5,9] 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.a [1,9] 2024-05-20T20:55:22.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.e [4,6] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.17 [4,8] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.1a [5,7] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.1b [3,6] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.23 [4,7] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.31 [4,8] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.38 [3,9] 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3367150269 expires 2024-05-21T20:45:35.597719+0000 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/2532144649 expires 2024-05-21T20:45:35.597719+0000 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1750454874 expires 2024-05-21T20:45:35.597719+0000 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3909957816 expires 2024-05-21T20:45:08.150176+0000 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/1825381346 expires 2024-05-21T20:45:08.150176+0000 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1784334339 expires 2024-05-21T20:45:35.597719+0000 2024-05-20T20:55:22.278 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/767936803 expires 2024-05-21T20:45:08.150176+0000 2024-05-20T20:55:22.279 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/2532144649 expires 2024-05-21T20:45:35.597719+0000 2024-05-20T20:55:22.279 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2879790538 expires 2024-05-21T20:45:08.150176+0000 2024-05-20T20:55:22.279 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/1825381346 expires 2024-05-21T20:45:08.150176+0000 2024-05-20T20:55:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f417c0b2880 msgr2=0x7f417c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f417c0b2880 0x7f417c0b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4190077950 tx=0x7f418c008040 comp rx=0 tx=0).stop 2024-05-20T20:55:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 msgr2=0x7f41901a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 0x7f41901a00e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f418000c9c0 tx=0x7f418000ccd0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 shutdown_connections 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f419010fb80 0x7f41901a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4190075fa0 0x7f419019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f417c0b2880 0x7f417c0b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 --2- 172.21.15.26:0/2784404031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4190077570 0x7f41901a00e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 >> 172.21.15.26:0/2784404031 conn(0x7f41900feb40 msgr2=0x7f4190102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 shutdown_connections 2024-05-20T20:55:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:22.278+0000 7f41973f6700 1 -- 172.21.15.26:0/2784404031 wait complete. 2024-05-20T20:55:22.953 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-20T20:55:22.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:55:23.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:23 smithi026 ceph-mon[31990]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:23.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:23 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2784404031' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-20T20:55:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:23.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:23 smithi055 ceph-mon[35213]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:23.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:23 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2784404031' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-20T20:55:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:23 smithi144 ceph-mon[35660]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:23 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2784404031' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-20T20:55:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1827532954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 msgr2=0x7f1d5410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1827532954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5410f330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00b0a0 tx=0x7f1d4c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1827532954 shutdown_connections 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1827532954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5410f330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1827532954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d541084f0 0x7f1d541088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1827532954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d54102460 0x7f1d541028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1827532954 >> 172.21.15.26:0/1827532954 conn(0x7f1d540794e0 msgr2=0x7f1d540798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1827532954 shutdown_connections 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.011+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1827532954 wait complete. 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 Processor -- start 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 -- start start 2024-05-20T20:55:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d54102460 0x7f1d5419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d541084f0 0x7f1d54199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d54112530 con 0x7f1d54102e20 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541123b0 con 0x7f1d541084f0 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5bc09700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541126b0 con 0x7f1d54102460 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d591a4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d591a4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45806/0 (socket says 172.21.15.26:45806) 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d591a4700 1 -- 172.21.15.26:0/1613155857 learned_addr learned my addr 172.21.15.26:0/1613155857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d599a5700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d54102460 0x7f1d5419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.012+0000 7f1d5a1a6700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d541084f0 0x7f1d54199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d591a4700 1 -- 172.21.15.26:0/1613155857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d54102460 msgr2=0x7f1d5419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d591a4700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d54102460 0x7f1d5419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d591a4700 1 -- 172.21.15.26:0/1613155857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d541084f0 msgr2=0x7f1d54199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d591a4700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d541084f0 0x7f1d54199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d591a4700 1 -- 172.21.15.26:0/1613155857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5419a070 con 0x7f1d54102e20 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d591a4700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5419fc90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b530 tx=0x7f1d4400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44015020 con 0x7f1d54102e20 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5419a300 con 0x7f1d54102e20 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d44011cc0 con 0x7f1d54102e20 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44010c80 con 0x7f1d54102e20 2024-05-20T20:55:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.013+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d541a6620 con 0x7f1d54102e20 2024-05-20T20:55:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.014+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5406a230 con 0x7f1d54102e20 2024-05-20T20:55:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.016+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d44018020 con 0x7f1d54102e20 2024-05-20T20:55:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.016+0000 7f1d4affd700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1d400b2880 0x7f1d400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.016+0000 7f1d599a5700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1d400b2880 0x7f1d400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.016+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1d440e1950 con 0x7f1d54102e20 2024-05-20T20:55:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.017+0000 7f1d599a5700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1d400b2880 0x7f1d400b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d5419aa50 tx=0x7f1d5000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:25.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.021+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d4401f020 con 0x7f1d54102e20 2024-05-20T20:55:25.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:25 smithi026 ceph-mon[31990]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.284+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d5402e560 con 0x7f1d54102e20 2024-05-20T20:55:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.284+0000 7f1d4affd700 1 -- 172.21.15.26:0/1613155857 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f1d440a9610 con 0x7f1d54102e20 2024-05-20T20:55:25.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:25.284 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:53:05.138905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1d400b2880 msgr2=0x7f1d400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1d400b2880 0x7f1d400b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d5419aa50 tx=0x7f1d5000a380 comp rx=0 tx=0).stop 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 msgr2=0x7f1d5419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5419fc90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b530 tx=0x7f1d4400b840 comp rx=0 tx=0).stop 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 shutdown_connections 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d541084f0 0x7f1d54199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d54102460 0x7f1d5419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1d400b2880 0x7f1d400b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 --2- 172.21.15.26:0/1613155857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d54102e20 0x7f1d5419fc90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 >> 172.21.15.26:0/1613155857 conn(0x7f1d540794e0 msgr2=0x7f1d5406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 shutdown_connections 2024-05-20T20:55:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:25.287+0000 7f1d5bc09700 1 -- 172.21.15.26:0/1613155857 wait complete. 2024-05-20T20:55:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:55:25.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:25 smithi055 ceph-mon[35213]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:25 smithi144 ceph-mon[35660]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:25.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:55:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:26.381 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:26 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1613155857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:26.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:26 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1613155857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:26 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1613155857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:27.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:27 smithi055 ceph-mon[35213]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:27 smithi026 ceph-mon[31990]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:27 smithi144 ceph-mon[35660]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 -- 172.21.15.26:0/166516286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 msgr2=0x7fef4c112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 --2- 172.21.15.26:0/166516286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c112650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b0a0 tx=0x7fef4400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 -- 172.21.15.26:0/166516286 shutdown_connections 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 --2- 172.21.15.26:0/166516286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c112650 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 --2- 172.21.15.26:0/166516286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 0x7fef4c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 --2- 172.21.15.26:0/166516286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 0x7fef4c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.102+0000 7fef51810700 1 -- 172.21.15.26:0/166516286 >> 172.21.15.26:0/166516286 conn(0x7fef4c101270 msgr2=0x7fef4c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.103+0000 7fef51810700 1 -- 172.21.15.26:0/166516286 shutdown_connections 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.103+0000 7fef51810700 1 -- 172.21.15.26:0/166516286 wait complete. 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.103+0000 7fef51810700 1 Processor -- start 2024-05-20T20:55:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 -- start start 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 0x7fef4c112500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c10f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 0x7fef4c10fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c115880 con 0x7fef4c106140 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c115700 con 0x7fef4c10b780 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef51810700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c115a00 con 0x7fef4c105780 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c10f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c10f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45834/0 (socket says 172.21.15.26:45834) 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 -- 172.21.15.26:0/2530047089 learned_addr learned my addr 172.21.15.26:0/2530047089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4b7fe700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 0x7fef4c10fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4affd700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 0x7fef4c112500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 -- 172.21.15.26:0/2530047089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 msgr2=0x7fef4c112500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 0x7fef4c112500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 -- 172.21.15.26:0/2530047089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 msgr2=0x7fef4c10fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 0x7fef4c10fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4a7fc700 1 -- 172.21.15.26:0/2530047089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c1ac580 con 0x7fef4c106140 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.104+0000 7fef4affd700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 0x7fef4c112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.105+0000 7fef4b7fe700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 0x7fef4c10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.105+0000 7fef4a7fc700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c10f650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d980 tx=0x7fef4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.105+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40017070 con 0x7fef4c106140 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.105+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1ac750 con 0x7fef4c106140 2024-05-20T20:55:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.105+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef40005020 con 0x7fef4c106140 2024-05-20T20:55:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.106+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4000f980 con 0x7fef4c106140 2024-05-20T20:55:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.106+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1acc00 con 0x7fef4c106140 2024-05-20T20:55:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.107+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c106ec0 con 0x7fef4c106140 2024-05-20T20:55:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.108+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef40004a30 con 0x7fef4c106140 2024-05-20T20:55:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.109+0000 7fef5080e700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef340b2880 0x7fef340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.109+0000 7fef4affd700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef340b2880 0x7fef340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.109+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fef400e2130 con 0x7fef4c106140 2024-05-20T20:55:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.109+0000 7fef4affd700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef340b2880 0x7fef340b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef3c005950 tx=0x7fef3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:28.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.113+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef40020020 con 0x7fef4c106140 2024-05-20T20:55:28.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.378+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef4c02e560 con 0x7fef4c106140 2024-05-20T20:55:28.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.379+0000 7fef5080e700 1 -- 172.21.15.26:0/2530047089 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7fef400a9d90 con 0x7fef4c106140 2024-05-20T20:55:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:28.379 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:53:05.138905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.382+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef340b2880 msgr2=0x7fef340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.382+0000 7fef51810700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef340b2880 0x7fef340b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef3c005950 tx=0x7fef3c00b410 comp rx=0 tx=0).stop 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.382+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 msgr2=0x7fef4c10f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.382+0000 7fef51810700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c10f650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d980 tx=0x7fef4000dc90 comp rx=0 tx=0).stop 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 shutdown_connections 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c10b780 0x7fef4c10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef4c105780 0x7fef4c112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef340b2880 0x7fef340b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 --2- 172.21.15.26:0/2530047089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4c106140 0x7fef4c10f650 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 >> 172.21.15.26:0/2530047089 conn(0x7fef4c101270 msgr2=0x7fef4c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 shutdown_connections 2024-05-20T20:55:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:28.383+0000 7fef51810700 1 -- 172.21.15.26:0/2530047089 wait complete. 2024-05-20T20:55:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-05-20T20:55:29.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:55:29.398 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:29.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:29 smithi055 ceph-mon[35213]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:29.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:29 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2530047089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:29 smithi026 ceph-mon[31990]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:29.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:29 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2530047089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:29 smithi144 ceph-mon[35660]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:29 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2530047089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 -- 172.21.15.26:0/2962257105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83701084f0 msgr2=0x7f83701088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 --2- 172.21.15.26:0/2962257105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83701084f0 0x7f83701088d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f836c009990 tx=0x7f836c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 -- 172.21.15.26:0/2962257105 shutdown_connections 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 --2- 172.21.15.26:0/2962257105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83701084f0 0x7f83701088d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 --2- 172.21.15.26:0/2962257105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8370102e20 0x7f837010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 --2- 172.21.15.26:0/2962257105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8370102460 0x7f83701028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.047+0000 7f8377c4a700 1 -- 172.21.15.26:0/2962257105 >> 172.21.15.26:0/2962257105 conn(0x7f83700794e0 msgr2=0x7f83700798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.048+0000 7f8377c4a700 1 -- 172.21.15.26:0/2962257105 shutdown_connections 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.048+0000 7f8377c4a700 1 -- 172.21.15.26:0/2962257105 wait complete. 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.048+0000 7f8377c4a700 1 Processor -- start 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.048+0000 7f8377c4a700 1 -- start start 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 0x7f837019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8370102e20 0x7f837019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83701084f0 0x7f8370199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112550 con 0x7f8370102460 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83701123d0 con 0x7f8370102e20 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83701126d0 con 0x7f83701084f0 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 0x7f837019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 0x7f837019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60944/0 (socket says 172.21.15.26:60944) 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 -- 172.21.15.26:0/3729837073 learned_addr learned my addr 172.21.15.26:0/3729837073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83751e5700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8370102e20 0x7f837019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83761e7700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83701084f0 0x7f8370199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 -- 172.21.15.26:0/3729837073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83701084f0 msgr2=0x7f8370199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83701084f0 0x7f8370199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 -- 172.21.15.26:0/3729837073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8370102e20 msgr2=0x7f837019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8370102e20 0x7f837019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 -- 172.21.15.26:0/3729837073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837019a1a0 con 0x7f8370102460 2024-05-20T20:55:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f83759e6700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 0x7f837019f880 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f836c000c00 tx=0x7f836c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c027070 con 0x7f8370102460 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f836c00fb80 con 0x7f8370102460 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837019a430 con 0x7f8370102460 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.049+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c022d00 con 0x7f8370102460 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.050+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701a6750 con 0x7f8370102460 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.051+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8370103ba0 con 0x7f8370102460 2024-05-20T20:55:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.052+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f836c02a070 con 0x7f8370102460 2024-05-20T20:55:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.053+0000 7f8366ffd700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f835c0b2880 0x7f835c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.053+0000 7f83751e5700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f835c0b2880 0x7f835c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.053+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f836c0e3140 con 0x7f8370102460 2024-05-20T20:55:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.053+0000 7f83751e5700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f835c0b2880 0x7f835c0b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83701077f0 tx=0x7f836000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:31.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.058+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f836c013070 con 0x7f8370102460 2024-05-20T20:55:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.274+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f837019af70 con 0x7f8370102460 2024-05-20T20:55:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.275+0000 7f8366ffd700 1 -- 172.21.15.26:0/3729837073 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22791 (secure 0 0 0) 0x7f836c0aae30 con 0x7f8370102460 2024-05-20T20:55:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:31.276 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:54:37.856459+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.277+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f835c0b2880 msgr2=0x7f835c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.277+0000 7f8377c4a700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f835c0b2880 0x7f835c0b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83701077f0 tx=0x7f836000b3c0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 msgr2=0x7f837019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 0x7f837019f880 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f836c000c00 tx=0x7f836c0047b0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 shutdown_connections 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f835c0b2880 0x7f835c0b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8370102460 0x7f837019f880 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83701084f0 0x7f8370199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 --2- 172.21.15.26:0/3729837073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8370102e20 0x7f837019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 >> 172.21.15.26:0/3729837073 conn(0x7f83700794e0 msgr2=0x7f837006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 shutdown_connections 2024-05-20T20:55:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:31.278+0000 7f8377c4a700 1 -- 172.21.15.26:0/3729837073 wait complete. 2024-05-20T20:55:31.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:31 smithi055 ceph-mon[35213]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:31 smithi026 ceph-mon[31990]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:31 smithi144 ceph-mon[35660]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:32.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-20T20:52:47.191542+0000', 'modified': '2024-05-20T20:53:05.138905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24476, 'mds_1': 24455, 'mds_2': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/1712709614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 1712709614}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 1712709614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.55:6835/2325650145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2325650145}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2325650145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/539511953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 539511953}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 539511953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.144:6839/2122637820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 2122637820}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 2122637820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.55:6837/353197532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 353197532}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 353197532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.55:6839/1704683185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6838', 'nonce': 1704683185}, {'type': 'v1', 'addr': '172.21.15.55:6839', 'nonce': 1704683185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24445, 24476, 24455]} 2024-05-20T20:55:32.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-20T20:55:32.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-20T20:55:32.254 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:32.377 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:32 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3729837073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:32.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:32 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3729837073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:32 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3729837073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:33.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:33 smithi026 ceph-mon[31990]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:33 smithi055 ceph-mon[35213]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:33 smithi144 ceph-mon[35660]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 -- 172.21.15.26:0/351914517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406bcf0 msgr2=0x7f1fa406c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 --2- 172.21.15.26:0/351914517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406bcf0 0x7f1fa406c0d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009990 tx=0x7f1f94009ca0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 -- 172.21.15.26:0/351914517 shutdown_connections 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 --2- 172.21.15.26:0/351914517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406bcf0 0x7f1fa406c0d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 --2- 172.21.15.26:0/351914517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa410ff90 0x7f1fa4112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 --2- 172.21.15.26:0/351914517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa406c610 0x7f1fa410fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 -- 172.21.15.26:0/351914517 >> 172.21.15.26:0/351914517 conn(0x7f1fa40794e0 msgr2=0x7f1fa40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 -- 172.21.15.26:0/351914517 shutdown_connections 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.934+0000 7f1fa9126700 1 -- 172.21.15.26:0/351914517 wait complete. 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 Processor -- start 2024-05-20T20:55:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 -- start start 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa406bcf0 0x7f1fa419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 0x7f1fa419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa410ff90 0x7f1fa41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4115890 con 0x7f1fa406c610 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4115710 con 0x7f1fa410ff90 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa9126700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4115a10 con 0x7f1fa406bcf0 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1f9bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 0x7f1fa419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1f9bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 0x7f1fa419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60960/0 (socket says 172.21.15.26:60960) 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1f9bfff700 1 -- 172.21.15.26:0/2534085058 learned_addr learned my addr 172.21.15.26:0/2534085058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa406bcf0 0x7f1fa419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.935+0000 7f1fa359e700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa410ff90 0x7f1fa41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1f9bfff700 1 -- 172.21.15.26:0/2534085058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa406bcf0 msgr2=0x7f1fa419bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa406bcf0 0x7f1fa419bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1f9bfff700 1 -- 172.21.15.26:0/2534085058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa410ff90 msgr2=0x7f1fa41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa410ff90 0x7f1fa41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1f9bfff700 1 -- 172.21.15.26:0/2534085058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa41a0da0 con 0x7f1fa406c610 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 0x7f1fa419c320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00d980 tx=0x7f1f8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1fa0d99700 1 -- 172.21.15.26:0/2534085058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c017070 con 0x7f1fa406c610 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1fa0d99700 1 -- 172.21.15.26:0/2534085058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f8c005020 con 0x7f1fa406c610 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa41a1090 con 0x7f1fa406c610 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1fa0d99700 1 -- 172.21.15.26:0/2534085058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00f980 con 0x7f1fa406c610 2024-05-20T20:55:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.936+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa41a8910 con 0x7f1fa406c610 2024-05-20T20:55:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.939+0000 7f1fa0d99700 1 -- 172.21.15.26:0/2534085058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f8c004a30 con 0x7f1fa406c610 2024-05-20T20:55:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.939+0000 7f1fa0d99700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f900b2880 0x7f1f900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.939+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f900b2880 0x7f1f900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.939+0000 7f1fa0d99700 1 -- 172.21.15.26:0/2534085058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1f8c0e2040 con 0x7f1fa406c610 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 Processor -- start 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 -- start start 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f900b2880 0x7f1f900b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f94003760 tx=0x7f1f94005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611b99bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9a1c0 0x55d611b9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9df10 0x55d611b9e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d611af8c40 con 0x55d611b99810 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d611ae87e0 con 0x55d611b9a1c0 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d611aec820 con 0x55d611b9df10 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fa2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611b99bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fa2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611b99bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60974/0 (socket says 172.21.15.26:60974) 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/2123066290 learned_addr learned my addr 172.21.15.26:0/2123066290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1fa359e700 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9df10 0x55d611b9e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.940+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9a1c0 0x55d611b9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/2123066290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9df10 msgr2=0x55d611b9e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9df10 0x55d611b9e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/2123066290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9a1c0 msgr2=0x55d611b9d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9a1c0 0x55d611b9d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/2123066290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d611a38880 con 0x55d611b99810 2024-05-20T20:55:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611b99bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009750 tx=0x7f1f9401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b0053d0b937a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1f997fa700 1 -- 172.21.15.26:0/2123066290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f940055e0 con 0x55d611b99810 2024-05-20T20:55:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1f997fa700 1 -- 172.21.15.26:0/2123066290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9400bc60 con 0x55d611b99810 2024-05-20T20:55:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.941+0000 7f1f997fa700 1 -- 172.21.15.26:0/2123066290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400f780 con 0x55d611b99810 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2123066290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 msgr2=0x55d611b99bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611b99bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009750 tx=0x7f1f9401f000 comp rx=0 tx=0).stop 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2123066290 shutdown_connections 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611b99bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9df10 0x55d611b9e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2123066290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9a1c0 0x55d611b9d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2123066290 >> 172.21.15.26:0/2123066290 conn(0x55d611ac7030 msgr2=0x55d611b93e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2123066290 shutdown_connections 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2123066290 wait complete. 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 -- start start 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.942+0000 7f1fb39bcb80 1 Processor -- start 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611ba5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9df10 0x55d611ba2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d6119eef50 con 0x55d611b99810 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d611a8a770 con 0x55d611b9df10 2024-05-20T20:55:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d611ba6010 con 0x55d611b9a1c0 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fb39bcb80 10 client.0 _ll_register_callbacks cb 0x55d611a838d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611ba5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611ba5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60976/0 (socket says 172.21.15.26:60976) 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/3308865638 learned_addr learned my addr 172.21.15.26:0/3308865638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa359e700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9df10 0x55d611ba2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 msgr2=0x55d611ba2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/3308865638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9df10 msgr2=0x55d611ba2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9df10 0x55d611ba2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 -- 172.21.15.26:0/3308865638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d611a87640 con 0x55d611b99810 2024-05-20T20:55:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611ba5300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f940098e0 tx=0x7f1f94003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be48a6572e21ac2b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.943+0000 7f1f8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/3308865638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9403a060 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/3308865638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1f9400fa80 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9400bc60 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/3308865638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f9400fc80 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 10 client.15072 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d611bb06b0 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/3308865638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9400bc60 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/3308865638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1f94022ba0 con 0x55d611b99810 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T20:55:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 20 client.15072 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611bb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 4 client.15072 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.944+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d611b65990 con 0x55d611bb1a00 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.945+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611bb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.945+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611bb3b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aae7cd4b0511752d server_cookie=b85882071421f54e in_seq=0 out_seq=0 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.945+0000 7f1f8a7fc700 10 client.15072 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/3308865638 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d611b65990 con 0x55d611bb1a00 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1f8a7fc700 10 client.15072 handle_command_reply: tid=0 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 client.15072 shutdown 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 msgr2=0x55d611ba5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611ba5300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f940098e0 tx=0x7f1f94003c50 comp rx=0 tx=0).stop 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 shutdown_connections 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611bb3b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b99810 0x55d611ba5300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1f8a7fc700 0 client.15072 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b9df10 0x55d611ba2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/3308865638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 >> 172.21.15.26:0/3308865638 conn(0x55d611ac7030 msgr2=0x55d611b934c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 shutdown_connections 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/3308865638 wait complete. 2024-05-20T20:55:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.946+0000 7f1fb39bcb80 20 client.15072 trim_cache size 0 max 16384 2024-05-20T20:55:33.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.968+0000 7f1fb39bcb80 1 Processor -- start 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.968+0000 7f1fb39bcb80 1 -- start start 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611ba1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611b93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d611bb1ec0 con 0x55d611b9df10 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d611bb2590 con 0x55d611b99810 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d611bb2c60 con 0x55d611b9a1c0 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611b93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611b93c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60990/0 (socket says 172.21.15.26:60990) 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 -- 172.21.15.26:0/2554229156 learned_addr learned my addr 172.21.15.26:0/2554229156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:33.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611ba1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 -- 172.21.15.26:0/2554229156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 msgr2=0x55d611ba4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 -- 172.21.15.26:0/2554229156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 msgr2=0x55d611ba1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611ba1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 -- 172.21.15.26:0/2554229156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d611ba5a40 con 0x55d611b9df10 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611ba1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.969+0000 7f1fa359e700 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611b93c40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009e20 tx=0x7f1f9c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190c56848dd72b25 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1f98ff9700 1 -- 172.21.15.26:0/2554229156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c007c60 con 0x55d611b9df10 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1f98ff9700 1 -- 172.21.15.26:0/2554229156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9c007e00 con 0x55d611b9df10 2024-05-20T20:55:33.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1f98ff9700 1 -- 172.21.15.26:0/2554229156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c01f770 con 0x55d611b9df10 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2554229156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 msgr2=0x55d611b93c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611b93c40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009e20 tx=0x7f1f9c009e50 comp rx=0 tx=0).stop 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2554229156 shutdown_connections 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611b93c40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611ba1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/2554229156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611ba4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2554229156 >> 172.21.15.26:0/2554229156 conn(0x55d611ac7030 msgr2=0x55d611ba3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2554229156 shutdown_connections 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.970+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/2554229156 wait complete. 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 -- start start 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 Processor -- start 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611bbd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611bb9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611bb9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d611bb3a00 con 0x55d611b9df10 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d611bb3330 con 0x55d611b99810 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d611bb40d0 con 0x55d611b9a1c0 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fb39bcb80 10 client.0 _ll_register_callbacks cb 0x55d611a838d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fa359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611bb9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fa359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611bb9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32770/0 (socket says 172.21.15.26:32770) 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fa359e700 1 -- 172.21.15.26:0/766450883 learned_addr learned my addr 172.21.15.26:0/766450883 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1f9bfff700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611bb9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.971+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611bbd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa359e700 1 -- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 msgr2=0x55d611bb9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa359e700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611bb9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa359e700 1 -- 172.21.15.26:0/766450883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 msgr2=0x55d611bbd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa359e700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611bbd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa359e700 1 -- 172.21.15.26:0/766450883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d611bba3d0 con 0x55d611b9df10 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611bbd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fa359e700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611bb9d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c006070 tx=0x7f1f9c017620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d42a11dadb55cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1f8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/766450883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c004ad0 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/766450883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1f9c004c70 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9c007e00 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/766450883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f9c028470 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fb39bcb80 10 client.15078 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d611c1ed00 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/766450883 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9c007e00 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.972+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/766450883 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1f9c032470 con 0x55d611b9df10 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fb39bcb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fb39bcb80 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fb39bcb80 20 client.15078 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611c221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:33.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fb39bcb80 4 client.15078 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "status"}) -- 0x55d611afab90 con 0x55d611bb1a00 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611c221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1fa2d9d700 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611c221b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27f512e275820942 server_cookie=5b8d8852e3ad4580 in_seq=0 out_seq=0 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.973+0000 7f1f8a7fc700 10 client.15078 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/766450883 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d611afab90 con 0x55d611bb1a00 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1f8a7fc700 10 client.15078 handle_command_reply: tid=0 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 client.15078 shutdown 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 msgr2=0x55d611bb9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611bb9d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c006070 tx=0x7f1f9c017620 comp rx=0 tx=0).stop 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 shutdown_connections 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d611bb1a00 0x55d611c221b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d611b9df10 0x55d611bb9d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d611b99810 0x55d611bbd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 --2- 172.21.15.26:0/766450883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d611b9a1c0 0x55d611bb9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1f8a7fc700 0 client.15078 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 >> 172.21.15.26:0/766450883 conn(0x55d611ac7030 msgr2=0x55d611b98eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 shutdown_connections 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 1 -- 172.21.15.26:0/766450883 wait complete. 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fb39bcb80 20 client.15078 trim_cache size 0 max 16384 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24445, 2024-05-20T20:55:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 153.66930153199999, 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 192.897991537 2024-05-20T20:55:33.981 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f900b2880 msgr2=0x7f1f900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f900b2880 0x7f1f900b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f94003760 tx=0x7f1f94005fd0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 msgr2=0x7f1fa419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 0x7f1fa419c320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00d980 tx=0x7f1f8c00dc90 comp rx=0 tx=0).stop 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 shutdown_connections 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa410ff90 0x7f1fa41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa406bcf0 0x7f1fa419bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f900b2880 0x7f1f900b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.974+0000 7f1fa9126700 1 --2- 172.21.15.26:0/2534085058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa406c610 0x7f1fa419c320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.975+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 >> 172.21.15.26:0/2534085058 conn(0x7f1fa40794e0 msgr2=0x7f1fa4111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.975+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 shutdown_connections 2024-05-20T20:55:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:33.975+0000 7f1fa9126700 1 -- 172.21.15.26:0/2534085058 wait complete. 2024-05-20T20:55:34.675 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-20T20:55:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:35.310 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:35 smithi026 ceph-mon[31990]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:35 smithi055 ceph-mon[35213]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:35 smithi144 ceph-mon[35660]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 -- 172.21.15.26:0/2705764781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 msgr2=0x7f8080101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 --2- 172.21.15.26:0/2705764781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080101fb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f80740099c0 tx=0x7f8074009cd0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 -- 172.21.15.26:0/2705764781 shutdown_connections 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 --2- 172.21.15.26:0/2705764781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80801024f0 0x7f80801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 --2- 172.21.15.26:0/2705764781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8080109520 0x7f8080109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 --2- 172.21.15.26:0/2705764781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080101fb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 -- 172.21.15.26:0/2705764781 >> 172.21.15.26:0/2705764781 conn(0x7f8080078ab0 msgr2=0x7f80800ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.637+0000 7f80855cc700 1 -- 172.21.15.26:0/2705764781 shutdown_connections 2024-05-20T20:55:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.638+0000 7f80855cc700 1 -- 172.21.15.26:0/2705764781 wait complete. 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.638+0000 7f80855cc700 1 Processor -- start 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 -- start start 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80801024f0 0x7f8080103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8080109520 0x7f8080103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8080113610 con 0x7f8080101b30 2024-05-20T20:55:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8080113490 con 0x7f80801024f0 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f80855cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8080113790 con 0x7f8080109520 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32802/0 (socket says 172.21.15.26:32802) 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 -- 172.21.15.26:0/566741636 learned_addr learned my addr 172.21.15.26:0/566741636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807e7fc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80801024f0 0x7f8080103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807f7fe700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8080109520 0x7f8080103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 -- 172.21.15.26:0/566741636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8080109520 msgr2=0x7f8080103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8080109520 0x7f8080103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 -- 172.21.15.26:0/566741636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80801024f0 msgr2=0x7f8080103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80801024f0 0x7f8080103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.639+0000 7f807effd700 1 -- 172.21.15.26:0/566741636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80801a8110 con 0x7f8080101b30 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.640+0000 7f807effd700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080102d10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8070007f10 tx=0x7f807000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.640+0000 7f8067fff700 1 -- 172.21.15.26:0/566741636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070004bb0 con 0x7f8080101b30 2024-05-20T20:55:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.640+0000 7f8067fff700 1 -- 172.21.15.26:0/566741636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8070004d50 con 0x7f8080101b30 2024-05-20T20:55:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.640+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a8400 con 0x7f8080101b30 2024-05-20T20:55:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.640+0000 7f8067fff700 1 -- 172.21.15.26:0/566741636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700058f0 con 0x7f8080101b30 2024-05-20T20:55:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.640+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a88c0 con 0x7f8080101b30 2024-05-20T20:55:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.642+0000 7f8067fff700 1 -- 172.21.15.26:0/566741636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f807001a020 con 0x7f8080101b30 2024-05-20T20:55:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.643+0000 7f8067fff700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80680b2880 0x7f80680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.643+0000 7f807e7fc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80680b2880 0x7f80680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.643+0000 7f8067fff700 1 -- 172.21.15.26:0/566741636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f80700e1920 con 0x7f8080101b30 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.643+0000 7f808fe62b80 1 Processor -- start 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.643+0000 7f807e7fc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80680b2880 0x7f80680b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80740058e0 tx=0x7f8074005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 -- start start 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8db86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 0x55e2a8db2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8d10d20 con 0x55e2a8db2850 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8d008c0 con 0x55e2a8db82d0 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f808fe62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8d04900 con 0x55e2a8db3210 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 0x55e2a8db2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 0x55e2a8db2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32806/0 (socket says 172.21.15.26:32806) 2024-05-20T20:55:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 -- 172.21.15.26:0/1916084883 learned_addr learned my addr 172.21.15.26:0/1916084883 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807effd700 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8db86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 -- 172.21.15.26:0/1916084883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 msgr2=0x55e2a8dbf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dbf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 -- 172.21.15.26:0/1916084883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 msgr2=0x55e2a8db86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8db86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 -- 172.21.15.26:0/1916084883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a8ca9f60 con 0x55e2a8db2850 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807effd700 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8db86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.644+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 0x55e2a8db2cd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f80740099c0 tx=0x7f807401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af75e3580b74b56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f80657fa700 1 -- 172.21.15.26:0/1916084883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807400bb10 con 0x55e2a8db2850 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f80657fa700 1 -- 172.21.15.26:0/1916084883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807400bcb0 con 0x55e2a8db2850 2024-05-20T20:55:36.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f80657fa700 1 -- 172.21.15.26:0/1916084883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807400f6c0 con 0x55e2a8db2850 2024-05-20T20:55:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 -- 172.21.15.26:0/1916084883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 msgr2=0x55e2a8db2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 0x55e2a8db2cd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f80740099c0 tx=0x7f807401f000 comp rx=0 tx=0).stop 2024-05-20T20:55:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 -- 172.21.15.26:0/1916084883 shutdown_connections 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dbf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8db86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1916084883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db2850 0x55e2a8db2cd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 -- 172.21.15.26:0/1916084883 >> 172.21.15.26:0/1916084883 conn(0x55e2a8cde1f0 msgr2=0x55e2a8dad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 -- 172.21.15.26:0/1916084883 shutdown_connections 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.645+0000 7f808fe62b80 1 -- 172.21.15.26:0/1916084883 wait complete. 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 -- start start 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 Processor -- start 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dbec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8dbbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8c06f50 con 0x55e2a8db3210 2024-05-20T20:55:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8c68d50 con 0x55e2a8db2850 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dbf630 con 0x55e2a8db82d0 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f808fe62b80 10 client.0 _ll_register_callbacks cb 0x55e2a8c9b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8dbbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8dbbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32822/0 (socket says 172.21.15.26:32822) 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f807e7fc700 1 -- 172.21.15.26:0/1752763971 learned_addr learned my addr 172.21.15.26:0/1752763971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f807effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dbec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.646+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807e7fc700 1 -- 172.21.15.26:0/1752763971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db82d0 msgr2=0x55e2a8dbc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807e7fc700 1 -- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 msgr2=0x55e2a8dbec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dbec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807e7fc700 1 -- 172.21.15.26:0/1752763971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a8c50880 con 0x55e2a8db3210 2024-05-20T20:55:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807effd700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8dbbfe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f80740096a0 tx=0x7f807400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46aaf9c66b3e15f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f8062ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f8062ff5700 1 -- 172.21.15.26:0/1752763971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074037070 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f8062ff5700 1 -- 172.21.15.26:0/1752763971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f807400fdd0 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807400bcb0 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f8062ff5700 1 -- 172.21.15.26:0/1752763971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8074043050 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f808fe62b80 10 client.15090 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.647+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2a8cba710 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f8062ff5700 1 -- 172.21.15.26:0/1752763971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807400bcb0 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f8062ff5700 1 -- 172.21.15.26:0/1752763971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8074030710 con 0x55e2a8db3210 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f808fe62b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f808fe62b80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f808fe62b80 20 client.15090 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8dcbbf0 0x55e2a8dcd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f808fe62b80 4 client.15090 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a8d12c70 con 0x55e2a8dcbbf0 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.648+0000 7f807effd700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8dcbbf0 0x55e2a8dcd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.649+0000 7f807effd700 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8dcbbf0 0x55e2a8dcd060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bedfe86940a9fbfa server_cookie=6a75c4d709c6a098 in_seq=0 out_seq=0 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.649+0000 7f8062ff5700 10 client.15090 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f8062ff5700 1 -- 172.21.15.26:0/1752763971 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e2a8d12c70 con 0x55e2a8dcbbf0 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f8062ff5700 10 client.15090 handle_command_reply: tid=0 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 client.15090 shutdown 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 msgr2=0x55e2a8dbbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8dbbfe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f80740096a0 tx=0x7f807400f710 comp rx=0 tx=0).stop 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 shutdown_connections 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8dcbbf0 0x55e2a8dcd060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1752763971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8dbbfe0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 >> 172.21.15.26:0/1752763971 conn(0x55e2a8cde1f0 msgr2=0x55e2a8dacaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f8062ff5700 0 client.15090 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 shutdown_connections 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 1 -- 172.21.15.26:0/1752763971 wait complete. 2024-05-20T20:55:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.650+0000 7f808fe62b80 20 client.15090 trim_cache size 0 max 16384 2024-05-20T20:55:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 Processor -- start 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 -- start start 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db2850 0x55e2a8db2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8db77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dcb3a0 con 0x55e2a8db3210 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dcba70 con 0x55e2a8db82d0 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f808fe62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dcc140 con 0x55e2a8db2850 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8db77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8db77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32830/0 (socket says 172.21.15.26:32830) 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 -- 172.21.15.26:0/1562600013 learned_addr learned my addr 172.21.15.26:0/1562600013 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 -- 172.21.15.26:0/1562600013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db2850 msgr2=0x55e2a8db2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807effd700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db2850 0x55e2a8db2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db2850 0x55e2a8db2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 -- 172.21.15.26:0/1562600013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 msgr2=0x55e2a8dbcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807e7fc700 1 -- 172.21.15.26:0/1562600013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a8dba550 con 0x55e2a8db3210 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807effd700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db2850 0x55e2a8db2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.672+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f807e7fc700 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8db77d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8074030e50 tx=0x7f8074030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66741112024ad68a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f8064ff9700 1 -- 172.21.15.26:0/1562600013 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074037040 con 0x55e2a8db3210 2024-05-20T20:55:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f8064ff9700 1 -- 172.21.15.26:0/1562600013 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8074030910 con 0x55e2a8db3210 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f8064ff9700 1 -- 172.21.15.26:0/1562600013 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074026ad0 con 0x55e2a8db3210 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 -- 172.21.15.26:0/1562600013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 msgr2=0x55e2a8db77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8db77d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8074030e50 tx=0x7f8074030d00 comp rx=0 tx=0).stop 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 -- 172.21.15.26:0/1562600013 shutdown_connections 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db82d0 0x55e2a8dbcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db2850 0x55e2a8db2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 --2- 172.21.15.26:0/1562600013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db3210 0x55e2a8db77d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 -- 172.21.15.26:0/1562600013 >> 172.21.15.26:0/1562600013 conn(0x55e2a8cde1f0 msgr2=0x55e2a8db70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 -- 172.21.15.26:0/1562600013 shutdown_connections 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.673+0000 7f808fe62b80 1 -- 172.21.15.26:0/1562600013 wait complete. 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 -- start start 2024-05-20T20:55:36.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 Processor -- start 2024-05-20T20:55:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dd6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dd6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 0x55e2a8dac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dccee0 con 0x55e2a8db82d0 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dcc810 con 0x55e2a8db2850 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2a8dcd5b0 con 0x55e2a8db3210 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f808fe62b80 10 client.0 _ll_register_callbacks cb 0x55e2a8c9b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 0x55e2a8dac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 0x55e2a8dac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32842/0 (socket says 172.21.15.26:32842) 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f807f7fe700 1 -- 172.21.15.26:0/3235831581 learned_addr learned my addr 172.21.15.26:0/3235831581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f807e7fc700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dd6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.674+0000 7f807effd700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dd6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807f7fe700 1 -- 172.21.15.26:0/3235831581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 msgr2=0x55e2a8dd6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807f7fe700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dd6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807f7fe700 1 -- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 msgr2=0x55e2a8dd6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807f7fe700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dd6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807f7fe700 1 -- 172.21.15.26:0/3235831581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a8dacc70 con 0x55e2a8db82d0 2024-05-20T20:55:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807effd700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dd6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f807f7fe700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 0x55e2a8dac430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f807800ba60 tx=0x7f807800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ee98d30d2ef0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f8062ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f8062ff5700 1 -- 172.21.15.26:0/3235831581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078003c80 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8074030910 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f8062ff5700 1 -- 172.21.15.26:0/3235831581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f8078003e20 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f8062ff5700 1 -- 172.21.15.26:0/3235831581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8078004830 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f808fe62b80 10 client.15096 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2a8dd4310 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.675+0000 7f8062ff5700 1 -- 172.21.15.26:0/3235831581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8074030910 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f8062ff5700 1 -- 172.21.15.26:0/3235831581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8078026050 con 0x55e2a8db82d0 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f808fe62b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f808fe62b80 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T20:55:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f808fe62b80 20 client.15096 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f808fe62b80 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8e3bd10 0x55e2a8e3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f808fe62b80 4 client.15096 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "status"}) -- 0x55e2a8d0c450 con 0x55e2a8e3bd10 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f807effd700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8e3bd10 0x55e2a8e3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f807effd700 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8e3bd10 0x55e2a8e3aa20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c275eafd0d58bc17 server_cookie=b601961dec49868e in_seq=0 out_seq=0 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.676+0000 7f8062ff5700 10 client.15096 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f8062ff5700 1 -- 172.21.15.26:0/3235831581 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e2a8d0c450 con 0x55e2a8e3bd10 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f8062ff5700 10 client.15096 handle_command_reply: tid=0 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 client.15096 shutdown 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 msgr2=0x55e2a8dac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 0x55e2a8dac430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f807800ba60 tx=0x7f807800bd70 comp rx=0 tx=0).stop 2024-05-20T20:55:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 shutdown_connections 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e2a8e3bd10 0x55e2a8e3aa20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a8db82d0 0x55e2a8dac430 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2a8db2850 0x55e2a8dd6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 --2- 172.21.15.26:0/3235831581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a8db3210 0x55e2a8dd6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 >> 172.21.15.26:0/3235831581 conn(0x55e2a8cde1f0 msgr2=0x55e2a8db5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f8062ff5700 0 client.15096 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 shutdown_connections 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 1 -- 172.21.15.26:0/3235831581 wait complete. 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.677+0000 7f808fe62b80 20 client.15096 trim_cache size 0 max 16384 2024-05-20T20:55:36.683 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24455, 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 157.38327482700001, 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 198.483825771 2024-05-20T20:55:36.684 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:55:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80680b2880 msgr2=0x7f80680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80680b2880 0x7f80680b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80740058e0 tx=0x7f8074005fb0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 msgr2=0x7f8080102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080102d10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8070007f10 tx=0x7f807000d3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 shutdown_connections 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80680b2880 0x7f80680b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101b30 0x7f8080102d10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8080109520 0x7f8080103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 --2- 172.21.15.26:0/566741636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80801024f0 0x7f8080103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 >> 172.21.15.26:0/566741636 conn(0x7f8080078ab0 msgr2=0x7f8080100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 shutdown_connections 2024-05-20T20:55:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:36.678+0000 7f80855cc700 1 -- 172.21.15.26:0/566741636 wait complete. 2024-05-20T20:55:37.409 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:37 smithi026 ceph-mon[31990]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:37.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-20T20:55:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:37 smithi055 ceph-mon[35213]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:37 smithi144 ceph-mon[35660]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:39.396 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:39 smithi026 ceph-mon[31990]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:39.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 -- 172.21.15.26:0/133943323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78076090 msgr2=0x7f3e78114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 --2- 172.21.15.26:0/133943323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78076090 0x7f3e78114200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009990 tx=0x7f3e68009ca0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 -- 172.21.15.26:0/133943323 shutdown_connections 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 --2- 172.21.15.26:0/133943323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e78114740 0x7f3e78116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 --2- 172.21.15.26:0/133943323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78075770 0x7f3e78075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 --2- 172.21.15.26:0/133943323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78076090 0x7f3e78114200 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 -- 172.21.15.26:0/133943323 >> 172.21.15.26:0/133943323 conn(0x7f3e7807a6b0 msgr2=0x7f3e7807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 -- 172.21.15.26:0/133943323 shutdown_connections 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 -- 172.21.15.26:0/133943323 wait complete. 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.396+0000 7f3e800ce700 1 Processor -- start 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 -- start start 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 0x7f3e781a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78076090 0x7f3e781a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e78114740 0x7f3e781a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78119bf0 con 0x7f3e78075770 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78119a70 con 0x7f3e78114740 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e800ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78119d70 con 0x7f3e78076090 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 0x7f3e781a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 0x7f3e781a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32860/0 (socket says 172.21.15.26:32860) 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4194597772 learned_addr learned my addr 172.21.15.26:0/4194597772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e78114740 0x7f3e781a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78076090 0x7f3e781a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4194597772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78076090 msgr2=0x7f3e781a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78076090 0x7f3e781a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4194597772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e78114740 msgr2=0x7f3e781a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e78114740 0x7f3e781a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4194597772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e781a5190 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.397+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 0x7f3e781a0180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400d980 tx=0x7f3e7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.398+0000 7f3e6effd700 1 -- 172.21.15.26:0/4194597772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74017070 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.398+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e781a5480 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.398+0000 7f3e6effd700 1 -- 172.21.15.26:0/4194597772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e74005020 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.398+0000 7f3e6effd700 1 -- 172.21.15.26:0/4194597772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400f980 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.398+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781acd00 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.400+0000 7f3e6effd700 1 -- 172.21.15.26:0/4194597772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3e74004a30 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.401+0000 7f3e6effd700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e640b2900 0x7f3e640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.401+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e640b2900 0x7f3e640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.401+0000 7f3e6effd700 1 -- 172.21.15.26:0/4194597772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f3e740e2070 con 0x7f3e78075770 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.401+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e640b2900 0x7f3e640b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800b5c0 tx=0x7f3e68005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 Processor -- start 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 -- start start 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cdc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cd6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531ce3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564531c34d20 con 0x564531cd6850 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564531c248c0 con 0x564531cdc2d0 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564531c28900 con 0x564531cd7210 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e7d669700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cd6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e7d669700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cd6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32870/0 (socket says 172.21.15.26:32870) 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.402+0000 7f3e7d669700 1 -- 172.21.15.26:0/4090261621 learned_addr learned my addr 172.21.15.26:0/4090261621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531ce3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cdc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7d669700 1 -- 172.21.15.26:0/4090261621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 msgr2=0x564531ce3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531ce3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7d669700 1 -- 172.21.15.26:0/4090261621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 msgr2=0x564531cdc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cdc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7d669700 1 -- 172.21.15.26:0/4090261621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564531bcdf60 con 0x564531cd6850 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cdc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cd6cd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e68015000 tx=0x7f3e6801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd805e6e97713ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e63fff700 1 -- 172.21.15.26:0/4090261621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680056d0 con 0x564531cd6850 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e63fff700 1 -- 172.21.15.26:0/4090261621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e6800bc60 con 0x564531cd6850 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.403+0000 7f3e63fff700 1 -- 172.21.15.26:0/4090261621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800f960 con 0x564531cd6850 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4090261621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 msgr2=0x564531cd6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cd6cd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e68015000 tx=0x7f3e6801f000 comp rx=0 tx=0).stop 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4090261621 shutdown_connections 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531ce3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cdc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4090261621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cd6cd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4090261621 >> 172.21.15.26:0/4090261621 conn(0x564531c021f0 msgr2=0x564531cd1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4090261621 shutdown_connections 2024-05-20T20:55:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4090261621 wait complete. 2024-05-20T20:55:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.404+0000 7f3e8a964b80 1 -- start start 2024-05-20T20:55:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 Processor -- start 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531ce2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531cdff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531ce0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564531b2af50 con 0x564531cd6850 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564531b8cd50 con 0x564531cdc2d0 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564531ce35d0 con 0x564531cd7210 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e8a964b80 10 client.0 _ll_register_callbacks cb 0x564531bbf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531ce2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531ce2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32880/0 (socket says 172.21.15.26:32880) 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4193694779 learned_addr learned my addr 172.21.15.26:0/4193694779 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531cdff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531ce0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4193694779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 msgr2=0x564531cdff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531cdff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 msgr2=0x564531ce0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531ce0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7de6a700 1 -- 172.21.15.26:0/4193694779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564531b74880 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.405+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531ce0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531ce2c30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e740dfe80 tx=0x7f3e74004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f36728e825cbc6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e617fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e617fa700 1 -- 172.21.15.26:0/4193694779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740a9630 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e617fa700 1 -- 172.21.15.26:0/4193694779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f3e740a9aa0 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e6800bc60 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e617fa700 1 -- 172.21.15.26:0/4193694779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e740ad9e0 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 10 client.15108 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564531bde710 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e617fa700 1 -- 172.21.15.26:0/4193694779 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e6800bc60 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e617fa700 1 -- 172.21.15.26:0/4193694779 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3e7408fba0 con 0x564531cd6850 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T20:55:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 20 client.15108 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531cf0030 0x564531cf14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 4 client.15108 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.406+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564531c36c70 con 0x564531cf0030 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.407+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531cf0030 0x564531cf14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.407+0000 7f3e7d669700 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531cf0030 0x564531cf14a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626c3b26d58f79d4 server_cookie=8af68a89d42b03fa in_seq=0 out_seq=0 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.407+0000 7f3e617fa700 10 client.15108 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e617fa700 1 -- 172.21.15.26:0/4193694779 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564531c36c70 con 0x564531cf0030 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e617fa700 10 client.15108 handle_command_reply: tid=0 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 client.15108 shutdown 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 msgr2=0x564531ce2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531ce2c30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e740dfe80 tx=0x7f3e74004e10 comp rx=0 tx=0).stop 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 shutdown_connections 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531cf0030 0x564531cf14a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531ce2c30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531ce0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e617fa700 0 client.15108 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/4193694779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd7210 0x564531cdff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 >> 172.21.15.26:0/4193694779 conn(0x564531c021f0 msgr2=0x564531cd0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 shutdown_connections 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 1 -- 172.21.15.26:0/4193694779 wait complete. 2024-05-20T20:55:39.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.408+0000 7f3e8a964b80 20 client.15108 trim_cache size 0 max 16384 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 Processor -- start 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 -- start start 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cdb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cd7210 0x564531cd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cdc2d0 0x564531cd5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564531cef7e0 con 0x564531cd6850 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564531cefeb0 con 0x564531cd7210 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564531cf0580 con 0x564531cdc2d0 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cdb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cdb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32894/0 (socket says 172.21.15.26:32894) 2024-05-20T20:55:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 -- 172.21.15.26:0/1399090704 learned_addr learned my addr 172.21.15.26:0/1399090704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cdc2d0 0x564531cd5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 -- 172.21.15.26:0/1399090704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cdc2d0 msgr2=0x564531cd5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7d669700 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cd7210 0x564531cd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cdc2d0 0x564531cd5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 -- 172.21.15.26:0/1399090704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cd7210 msgr2=0x564531cd5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cd7210 0x564531cd5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 -- 172.21.15.26:0/1399090704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564531cdbcb0 con 0x564531cd6850 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cdc2d0 0x564531cd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.431+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cdb770 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400d950 tx=0x7f3e74005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb236a1bd82adc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e637fe700 1 -- 172.21.15.26:0/1399090704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740b2490 con 0x564531cd6850 2024-05-20T20:55:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e637fe700 1 -- 172.21.15.26:0/1399090704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e740a9aa0 con 0x564531cd6850 2024-05-20T20:55:39.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e637fe700 1 -- 172.21.15.26:0/1399090704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740b2200 con 0x564531cd6850 2024-05-20T20:55:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 -- 172.21.15.26:0/1399090704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 msgr2=0x564531cdb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cdb770 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400d950 tx=0x7f3e74005440 comp rx=0 tx=0).stop 2024-05-20T20:55:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 -- 172.21.15.26:0/1399090704 shutdown_connections 2024-05-20T20:55:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd6850 0x564531cdb770 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cdc2d0 0x564531cd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/1399090704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cd7210 0x564531cd5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 -- 172.21.15.26:0/1399090704 >> 172.21.15.26:0/1399090704 conn(0x564531c021f0 msgr2=0x564531cdb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 -- 172.21.15.26:0/1399090704 shutdown_connections 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.432+0000 7f3e8a964b80 1 -- 172.21.15.26:0/1399090704 wait complete. 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 -- start start 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 Processor -- start 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd6850 0x564531cfa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 0x564531cfae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cf3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564531cf1320 con 0x564531cd7210 2024-05-20T20:55:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564531cf0c50 con 0x564531cdc2d0 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564531cf19f0 con 0x564531cd6850 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e8a964b80 10 client.0 _ll_register_callbacks cb 0x564531bbf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e7d669700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 0x564531cfae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e7d669700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 0x564531cfae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32908/0 (socket says 172.21.15.26:32908) 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e7d669700 1 -- 172.21.15.26:0/3526949697 learned_addr learned my addr 172.21.15.26:0/3526949697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e7de6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd6850 0x564531cfa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.433+0000 7f3e7e66b700 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cf3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e7d669700 1 -- 172.21.15.26:0/3526949697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd6850 msgr2=0x564531cfa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e7d669700 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd6850 0x564531cfa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e7d669700 1 -- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 msgr2=0x564531cf3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e7d669700 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cf3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e7d669700 1 -- 172.21.15.26:0/3526949697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564531cf4160 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e7d669700 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 0x564531cfae60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005310 tx=0x7f3e68005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbd2c4d1c5bdca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e617fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e617fa700 1 -- 172.21.15.26:0/3526949697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68003de0 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e617fa700 1 -- 172.21.15.26:0/3526949697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f3e6800f540 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e740a9aa0 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e617fa700 1 -- 172.21.15.26:0/3526949697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e68049050 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e8a964b80 10 client.15114 fetch_fsmap learned FSMap version 25 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564531d5c2b0 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.434+0000 7f3e617fa700 1 -- 172.21.15.26:0/3526949697 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e740a9aa0 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e617fa700 1 -- 172.21.15.26:0/3526949697 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3e68040070 con 0x564531cd7210 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e8a964b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e8a964b80 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T20:55:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e8a964b80 20 client.15114 populate_metadata read hostname 'smithi026' 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531d60910 0x564531d5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e8a964b80 4 client.15114 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "status"}) -- 0x564531c30450 con 0x564531d60910 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531d60910 0x564531d5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e617fa700 10 client.15114 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.435+0000 7f3e7de6a700 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531d60910 0x564531d5f620 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563dbfcc07003407 server_cookie=f43953f5ee8cf3c2 in_seq=0 out_seq=0 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e617fa700 1 -- 172.21.15.26:0/3526949697 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564531c30450 con 0x564531d60910 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e617fa700 10 client.15114 handle_command_reply: tid=0 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 client.15114 shutdown 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 msgr2=0x564531cfae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 0x564531cfae60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005310 tx=0x7f3e68005c30 comp rx=0 tx=0).stop 2024-05-20T20:55:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 shutdown_connections 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564531d60910 0x564531d5f620 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564531cdc2d0 0x564531cf3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e617fa700 0 client.15114 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564531cd6850 0x564531cfa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 --2- 172.21.15.26:0/3526949697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564531cd7210 0x564531cfae60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 >> 172.21.15.26:0/3526949697 conn(0x564531c021f0 msgr2=0x564531cd9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 shutdown_connections 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 1 -- 172.21.15.26:0/3526949697 wait complete. 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.436+0000 7f3e8a964b80 20 client.15114 trim_cache size 0 max 16384 2024-05-20T20:55:39.443 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24476, 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 172.208671337, 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 188.37017175400001 2024-05-20T20:55:39.444 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e640b2900 msgr2=0x7f3e640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e640b2900 0x7f3e640b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800b5c0 tx=0x7f3e68005960 comp rx=0 tx=0).stop 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 msgr2=0x7f3e781a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 0x7f3e781a0180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400d980 tx=0x7f3e7400dc90 comp rx=0 tx=0).stop 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 shutdown_connections 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e640b2900 0x7f3e640b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e78075770 0x7f3e781a0180 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e78114740 0x7f3e781a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 --2- 172.21.15.26:0/4194597772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78076090 0x7f3e781a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 >> 172.21.15.26:0/4194597772 conn(0x7f3e7807a6b0 msgr2=0x7f3e781101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 shutdown_connections 2024-05-20T20:55:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:39.437+0000 7f3e800ce700 1 -- 172.21.15.26:0/4194597772 wait complete. 2024-05-20T20:55:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:39 smithi055 ceph-mon[35213]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:39 smithi144 ceph-mon[35660]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:40.171 INFO:teuthology.run_tasks:Running task sequential... 2024-05-20T20:55:40.181 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-20T20:55:40.181 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-20T20:55:40.182 INFO:teuthology.task.print:Enabling mgr modules 2024-05-20T20:55:40.182 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-20T20:55:40.215 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-20T20:55:40.215 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-20T20:55:40.215 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-20T20:55:40.215 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T20:55:40.215 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:40.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:40.216 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-05-20T20:55:40.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:40.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-05-20T20:55:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-05-20T20:55:40.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T20:55:40.303 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-20T20:55:40.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-20T20:55:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:41.329 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:41 smithi026 ceph-mon[31990]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:41 smithi055 ceph-mon[35213]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:41 smithi144 ceph-mon[35660]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:42.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 -- 172.21.15.26:0/2389975501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700102790 msgr2=0x7fe700102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:42.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 --2- 172.21.15.26:0/2389975501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700102790 0x7fe700102c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40099c0 tx=0x7fe6f4009cd0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 -- 172.21.15.26:0/2389975501 shutdown_connections 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 --2- 172.21.15.26:0/2389975501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe700103150 0x7fe70010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 --2- 172.21.15.26:0/2389975501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe700108630 0x7fe700108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 --2- 172.21.15.26:0/2389975501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700102790 0x7fe700102c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 -- 172.21.15.26:0/2389975501 >> 172.21.15.26:0/2389975501 conn(0x7fe7000782b0 msgr2=0x7fe7000786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 -- 172.21.15.26:0/2389975501 shutdown_connections 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.197+0000 7fe704d05700 1 -- 172.21.15.26:0/2389975501 wait complete. 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 Processor -- start 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 -- start start 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe700102790 0x7fe70019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe700103150 0x7fe70019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 0x7fe700199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe700112530 con 0x7fe700108630 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001123b0 con 0x7fe700102790 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe704d05700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001126b0 con 0x7fe700103150 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 0x7fe700199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 0x7fe700199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41140/0 (socket says 172.21.15.26:41140) 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 -- 172.21.15.26:0/294766033 learned_addr learned my addr 172.21.15.26:0/294766033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fdd9b700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe700103150 0x7fe70019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fe59c700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe700102790 0x7fe70019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 -- 172.21.15.26:0/294766033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe700103150 msgr2=0x7fe70019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe700103150 0x7fe70019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 -- 172.21.15.26:0/294766033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe700102790 msgr2=0x7fe70019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe700102790 0x7fe70019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.198+0000 7fe6fed9d700 1 -- 172.21.15.26:0/294766033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70019a0c0 con 0x7fe700108630 2024-05-20T20:55:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.199+0000 7fe6fed9d700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 0x7fe700199960 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007ee0 tx=0x7fe6f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.199+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8004bb0 con 0x7fe700108630 2024-05-20T20:55:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.199+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70019a3b0 con 0x7fe700108630 2024-05-20T20:55:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.199+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f8004d50 con 0x7fe700108630 2024-05-20T20:55:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.199+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80058b0 con 0x7fe700108630 2024-05-20T20:55:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.199+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7001a6670 con 0x7fe700108630 2024-05-20T20:55:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.200+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70006a230 con 0x7fe700108630 2024-05-20T20:55:42.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.201+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe6f801a020 con 0x7fe700108630 2024-05-20T20:55:42.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.202+0000 7fe6ef7fe700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6e80b2880 0x7fe6e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:42.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.203+0000 7fe6fe59c700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6e80b2880 0x7fe6e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:42.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.203+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe6f80e1fb0 con 0x7fe700108630 2024-05-20T20:55:42.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.203+0000 7fe6fe59c700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6e80b2880 0x7fe6e80b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005950 tx=0x7fe6f000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.206+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6f8010070 con 0x7fe700108630 2024-05-20T20:55:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.472+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe7001a6840 con 0x7fe700108630 2024-05-20T20:55:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.472+0000 7fe6ef7fe700 1 -- 172.21.15.26:0/294766033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fe6f80a9c10 con 0x7fe700108630 2024-05-20T20:55:42.472 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T20:55:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6e80b2880 msgr2=0x7fe6e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6e80b2880 0x7fe6e80b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005950 tx=0x7fe6f000a300 comp rx=0 tx=0).stop 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 msgr2=0x7fe700199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 0x7fe700199960 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007ee0 tx=0x7fe6f800d3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 shutdown_connections 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6e80b2880 0x7fe6e80b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe700108630 0x7fe700199960 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe700102790 0x7fe70019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 --2- 172.21.15.26:0/294766033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe700103150 0x7fe70019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 >> 172.21.15.26:0/294766033 conn(0x7fe7000782b0 msgr2=0x7fe700101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 shutdown_connections 2024-05-20T20:55:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:42.474+0000 7fe704d05700 1 -- 172.21.15.26:0/294766033 wait complete. 2024-05-20T20:55:43.233 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T20:55:43.233 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T20:55:43.234 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T20:55:43.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:43.234 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-05-20T20:55:43.254 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T20:55:43.254 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T20:55:43.254 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T20:55:43.254 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-05-20T20:55:43.254 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42005sec preferred_lft 42005sec 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-05-20T20:55:43.255 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-05-20T20:55:43.256 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T20:55:43.256 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-05-20T20:55:43.256 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-05-20T20:55:43.256 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T20:55:43.256 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T20:55:43.256 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-05-20T20:55:43.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-05-20T20:55:43.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-05-20T20:55:43.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-05-20T20:55:43.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T20:55:43.257 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-05-20T20:55:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T20:55:43.418 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:43 smithi026 ceph-mon[31990]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:43.418 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:43 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/294766033' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T20:55:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:43 smithi055 ceph-mon[35213]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:43 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/294766033' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T20:55:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T20:55:43.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:43.461 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T20:55:43.487 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-05-20T20:55:43.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:43.489 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-05-20T20:55:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:43 smithi144 ceph-mon[35660]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T20:55:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:43 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/294766033' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T20:55:43.614 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-05-20T20:55:43.614 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T20:55:43.614 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T20:55:43.614 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T20:55:43.614 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T20:55:43.615 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T20:55:43.615 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-05-20T20:55:43.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T20:55:43.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T20:55:43.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T20:55:43.615 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-05-20T20:55:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T20:55:43.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T20:55:43.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:43.871 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-05-20T20:55:43.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:43.888 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-05-20T20:55:43.944 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T20:55:43.944 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-05-20T20:55:43.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T20:55:43.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-20T20:55:43.944 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-05-20T20:55:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T20:55:44.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T20:55:44.065 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-20T20:55:44.065 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-05-20T20:55:44.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T20:55:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T20:55:44.202 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T20:55:44.202 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-05-20T20:55:44.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-05-20T20:55:44.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T20:55:44.202 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-05-20T20:55:44.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T20:55:44.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T20:55:44.284 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-20T20:55:44.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T20:55:44.284 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:44.300 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T20:55:44.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T20:55:44.300 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:44.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-05-20T20:55:44.424 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-05-20T20:55:44.482 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T20:55:44.483 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1114e7b060ef9fa69eb57b4b4de5c99f68f727b1b738a7c849d73399369443d1/merged 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/a52d55f414cb4cff6bc100a1a688f97c0ade14518bd08ab03a5f91b2182835de/merged 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/fd32b7285b404e3b2138e637553365423d41982e0063336df4268f5702b34699/merged 2024-05-20T20:55:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/76f185b74c4ba94bce32cd20d917f83fff735d44912c54e300facf6a25bd29ed/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/4605621d5135ada2c5bb698b24d10e0c83ac5a44a990a4ead0e00b141f7c6b38/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/5ce9ee27d77885f1ba82d63186b7550dd9780e493f1b69d34ddef459823f5a76/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/f86f76aca28fbebc666c1dd00b4597fe42c36b6d803bde2c34e1b29405f64c92/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1c8f92088a8b7f5d11697420ec84871697647b91fa0713f67941b46ab9b502c1/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/f1a650ea566d83192c39775646a14d647430483e8ead0394c788f8401ea3ae64/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/9cb720dd5fddf480bc10b3df4900a6925187d57f835245621bc6b57bc4b9d7f5/merged 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T20:55:44.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:44.485 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-05-20T20:55:44.540 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T20:55:44.540 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95 --client_fs=cephfs) 2024-05-20T20:55:44.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-05-20T20:55:44.607 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-05-20T20:55:44.665 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-05-20T20:55:44.666 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1114e7b060ef9fa69eb57b4b4de5c99f68f727b1b738a7c849d73399369443d1/merged 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/a52d55f414cb4cff6bc100a1a688f97c0ade14518bd08ab03a5f91b2182835de/merged 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/fd32b7285b404e3b2138e637553365423d41982e0063336df4268f5702b34699/merged 2024-05-20T20:55:44.667 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/76f185b74c4ba94bce32cd20d917f83fff735d44912c54e300facf6a25bd29ed/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/4605621d5135ada2c5bb698b24d10e0c83ac5a44a990a4ead0e00b141f7c6b38/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/5ce9ee27d77885f1ba82d63186b7550dd9780e493f1b69d34ddef459823f5a76/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/f86f76aca28fbebc666c1dd00b4597fe42c36b6d803bde2c34e1b29405f64c92/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1c8f92088a8b7f5d11697420ec84871697647b91fa0713f67941b46ab9b502c1/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/f1a650ea566d83192c39775646a14d647430483e8ead0394c788f8401ea3ae64/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/9cb720dd5fddf480bc10b3df4900a6925187d57f835245621bc6b57bc4b9d7f5/merged 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T20:55:44.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:44.669 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-05-20T20:55:44.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-05-20T20:55:44.694+0000 7fb945d98480 -1 init, newargv = 0x55f6d04db180 newargc=15 2024-05-20T20:55:44.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[88380]: starting ceph client 2024-05-20T20:55:44.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[88380]: starting fuse 2024-05-20T20:55:44.711 INFO:teuthology.orchestra.run.smithi026.stdout:107 2024-05-20T20:55:44.711 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-20T20:55:44.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-05-20T20:55:44.711 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-05-20T20:55:44.711 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-05-20T20:55:44.711 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-05-20T20:55:44.712 DEBUG:teuthology.orchestra.run.smithi026:> 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-05-20T20:55:44.713 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-05-20T20:55:44.833 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.88380.asok 2024-05-20T20:55:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T20:55:44.844 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.88380.asok 2024-05-20T20:55:44.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:44.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.88380.asok status 2024-05-20T20:55:44.984 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:55:44.984 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-05-20T20:55:44.984 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "4ebd1d5361d457738957dc0455bba90102296634", 2024-05-20T20:55:44.984 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-3830-g4ebd1d53 (4ebd1d5361d457738957dc0455bba90102296634) squid (dev)", 2024-05-20T20:55:44.984 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "88380", 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95" 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 15126, 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 15126 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:55:44.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1320296977 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1320296977 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.15126 192.168.0.1:0/1320296977", 2024-05-20T20:55:44.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/1320296977", 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 5, 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 25, 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 82, 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-05-20T20:55:44.987 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:55:44.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:44.993 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-05-20T20:55:44.993 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:45.010 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-05-20T20:55:45.010 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:45.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T20:55:45.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T20:55:45.078 INFO:teuthology.run_tasks:Running task exec... 2024-05-20T20:55:45.089 INFO:teuthology.task.exec:Executing custom commands... 2024-05-20T20:55:45.089 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T20:55:45.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-20T20:55:45.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 -- 172.21.15.26:0/302734519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 msgr2=0x7f948c1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:45.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 --2- 172.21.15.26:0/302734519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c1017f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9488009900 tx=0x7f948802a410 comp rx=0 tx=0).stop 2024-05-20T20:55:45.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 -- 172.21.15.26:0/302734519 shutdown_connections 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 --2- 172.21.15.26:0/302734519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c1026d0 0x7f948c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 --2- 172.21.15.26:0/302734519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c101d30 0x7f948c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 --2- 172.21.15.26:0/302734519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c1017f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 -- 172.21.15.26:0/302734519 >> 172.21.15.26:0/302734519 conn(0x7f948c0fcfd0 msgr2=0x7f948c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 -- 172.21.15.26:0/302734519 shutdown_connections 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.225+0000 7f9494114640 1 -- 172.21.15.26:0/302734519 wait complete. 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 Processor -- start 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 -- start start 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c101d30 0x7f948c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1026d0 0x7f948c1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c110690 con 0x7f948c1013d0 2024-05-20T20:55:45.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f948c110510 con 0x7f948c101d30 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9494114640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f948c110810 con 0x7f948c1026d0 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9491eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9491eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c19ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41162/0 (socket says 172.21.15.26:41162) 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.226+0000 7f9491eb0640 1 -- 172.21.15.26:0/1799227330 learned_addr learned my addr 172.21.15.26:0/1799227330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9491eb0640 1 -- 172.21.15.26:0/1799227330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1026d0 msgr2=0x7f948c1a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9491eb0640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1026d0 0x7f948c1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f94916af640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c101d30 0x7f948c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9491eb0640 1 -- 172.21.15.26:0/1799227330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c101d30 msgr2=0x7f948c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9491eb0640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c101d30 0x7f948c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9491eb0640 1 -- 172.21.15.26:0/1799227330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c1a2990 con 0x7f948c1013d0 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f94916af640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c101d30 0x7f948c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9491eb0640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c19ad70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9488009230 tx=0x7f94880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:45.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948803f030 con 0x7f948c1013d0 2024-05-20T20:55:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c1a2c20 con 0x7f948c1013d0 2024-05-20T20:55:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9488003c90 con 0x7f948c1013d0 2024-05-20T20:55:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488035070 con 0x7f948c1013d0 2024-05-20T20:55:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.227+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c1a3160 con 0x7f948c1013d0 2024-05-20T20:55:45.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.228+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948c10c120 con 0x7f948c1013d0 2024-05-20T20:55:45.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.230+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9488042030 con 0x7f948c1013d0 2024-05-20T20:55:45.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.230+0000 7f9482ffd640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94780b28a0 0x7f94780b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:45.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.230+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9488102eb0 con 0x7f948c1013d0 2024-05-20T20:55:45.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.230+0000 7f94916af640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94780b28a0 0x7f94780b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:45.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.231+0000 7f94916af640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94780b28a0 0x7f94780b4d60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f948c19c280 tx=0x7f947c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:45.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.234+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9488132df0 con 0x7f948c1013d0 2024-05-20T20:55:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:45 smithi055 ceph-mon[35213]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:45.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:45 smithi026 ceph-mon[31990]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:45.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:45.466+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f948c02dba0 con 0x7f948c1013d0 2024-05-20T20:55:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:45 smithi144 ceph-mon[35660]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-20T20:55:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.137+0000 7f9482ffd640 1 -- 172.21.15.26:0/1799227330 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f94880cab70 con 0x7f948c1013d0 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94780b28a0 msgr2=0x7f94780b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94780b28a0 0x7f94780b4d60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f948c19c280 tx=0x7f947c005d30 comp rx=0 tx=0).stop 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 msgr2=0x7f948c19ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c19ad70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9488009230 tx=0x7f94880042f0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 shutdown_connections 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94780b28a0 0x7f94780b4d60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1026d0 0x7f948c1a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c101d30 0x7f948c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 --2- 172.21.15.26:0/1799227330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1013d0 0x7f948c19ad70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 >> 172.21.15.26:0/1799227330 conn(0x7f948c0fcfd0 msgr2=0x7f948c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 shutdown_connections 2024-05-20T20:55:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.140+0000 7f9494114640 1 -- 172.21.15.26:0/1799227330 wait complete. 2024-05-20T20:55:46.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-20T20:55:46.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 -- 172.21.15.26:0/2399619353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84108840 msgr2=0x7f9c8410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 --2- 172.21.15.26:0/2399619353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84108840 0x7f9c8410ac90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02a410 comp rx=0 tx=0).stop 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 -- 172.21.15.26:0/2399619353 shutdown_connections 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 --2- 172.21.15.26:0/2399619353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84108840 0x7f9c8410ac90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 --2- 172.21.15.26:0/2399619353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c84105f10 0x7f9c84108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 --2- 172.21.15.26:0/2399619353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c841035e0 0x7f9c841059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 -- 172.21.15.26:0/2399619353 >> 172.21.15.26:0/2399619353 conn(0x7f9c840fcfd0 msgr2=0x7f9c840ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 -- 172.21.15.26:0/2399619353 shutdown_connections 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.268+0000 7f9c8b483640 1 -- 172.21.15.26:0/2399619353 wait complete. 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 Processor -- start 2024-05-20T20:55:46.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 -- start start 2024-05-20T20:55:46.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c841035e0 0x7f9c84198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 0x7f9c84199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c84108840 0x7f9c841a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410e4d0 con 0x7f9c84105f10 2024-05-20T20:55:46.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410e350 con 0x7f9c841035e0 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8b483640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410e650 con 0x7f9c84108840 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 0x7f9c84199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 0x7f9c84199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41186/0 (socket says 172.21.15.26:41186) 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 -- 172.21.15.26:0/123144509 learned_addr learned my addr 172.21.15.26:0/123144509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c8921f640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c841035e0 0x7f9c84198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c89a20640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c84108840 0x7f9c841a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 -- 172.21.15.26:0/123144509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c84108840 msgr2=0x7f9c841a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c84108840 0x7f9c841a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 -- 172.21.15.26:0/123144509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c841035e0 msgr2=0x7f9c84198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c841035e0 0x7f9c84198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.269+0000 7f9c88a1e640 1 -- 172.21.15.26:0/123144509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c841a06e0 con 0x7f9c84105f10 2024-05-20T20:55:46.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.270+0000 7f9c88a1e640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 0x7f9c84199000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c970 tx=0x7f9c7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:46.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.270+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74007bb0 con 0x7f9c84105f10 2024-05-20T20:55:46.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.270+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c74007d50 con 0x7f9c84105f10 2024-05-20T20:55:46.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.270+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841a09d0 con 0x7f9c84105f10 2024-05-20T20:55:46.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.270+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74004e10 con 0x7f9c84105f10 2024-05-20T20:55:46.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.270+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a0f70 con 0x7f9c84105f10 2024-05-20T20:55:46.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.271+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c840feac0 con 0x7f9c84105f10 2024-05-20T20:55:46.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.272+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c74005000 con 0x7f9c84105f10 2024-05-20T20:55:46.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.273+0000 7f9c7a7fc640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c700b2920 0x7f9c700b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.273+0000 7f9c8921f640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c700b2920 0x7f9c700b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.273+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9c740e2290 con 0x7f9c84105f10 2024-05-20T20:55:46.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.274+0000 7f9c8921f640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c700b2920 0x7f9c700b4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c80005fd0 tx=0x7f9c80009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:46.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.276+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c74014030 con 0x7f9c84105f10 2024-05-20T20:55:46.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:46 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1799227330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-20T20:55:46.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:46 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1799227330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-20T20:55:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.502+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9c84062e90 con 0x7f9c700b2920 2024-05-20T20:55:46.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.517+0000 7f9c7a7fc640 1 -- 172.21.15.26:0/123144509 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f9c84062e90 con 0x7f9c700b2920 2024-05-20T20:55:46.518 INFO:teuthology.orchestra.run.smithi026.stdout:{} 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c700b2920 msgr2=0x7f9c700b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c700b2920 0x7f9c700b4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c80005fd0 tx=0x7f9c80009500 comp rx=0 tx=0).stop 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 msgr2=0x7f9c84199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 0x7f9c84199000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c970 tx=0x7f9c7400ce30 comp rx=0 tx=0).stop 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 shutdown_connections 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c700b2920 0x7f9c700b4de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c84108840 0x7f9c841a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c84105f10 0x7f9c84199000 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 --2- 172.21.15.26:0/123144509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c841035e0 0x7f9c84198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 >> 172.21.15.26:0/123144509 conn(0x7f9c840fcfd0 msgr2=0x7f9c84103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:46.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 shutdown_connections 2024-05-20T20:55:46.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.520+0000 7f9c8b483640 1 -- 172.21.15.26:0/123144509 wait complete. 2024-05-20T20:55:46.533 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-20T20:55:46.547 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-20T20:55:46.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T20:55:46.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:46 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1799227330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 -- 172.21.15.26:0/4257248497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8102ba0 msgr2=0x7f56b8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 --2- 172.21.15.26:0/4257248497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8102ba0 0x7f56b8103020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009ca0 tx=0x7f56a802a8c0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 -- 172.21.15.26:0/4257248497 shutdown_connections 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 --2- 172.21.15.26:0/4257248497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8103560 0x7f56b8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 --2- 172.21.15.26:0/4257248497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8102ba0 0x7f56b8103020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 --2- 172.21.15.26:0/4257248497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 0x7f56b8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 -- 172.21.15.26:0/4257248497 >> 172.21.15.26:0/4257248497 conn(0x7f56b80fcf40 msgr2=0x7f56b80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 -- 172.21.15.26:0/4257248497 shutdown_connections 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.691+0000 7f56bed78640 1 -- 172.21.15.26:0/4257248497 wait complete. 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 Processor -- start 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 -- start start 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 0x7f56b810c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8102ba0 0x7f56b810a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 0x7f56b810aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810e510 con 0x7f56b8103560 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810e390 con 0x7f56b81019b0 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bed78640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810e690 con 0x7f56b8102ba0 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 0x7f56b810aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 0x7f56b810aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41220/0 (socket says 172.21.15.26:41220) 2024-05-20T20:55:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 -- 172.21.15.26:0/4237393224 learned_addr learned my addr 172.21.15.26:0/4237393224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56b7fff640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8102ba0 0x7f56b810a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bcb14640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 0x7f56b810c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 -- 172.21.15.26:0/4237393224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8102ba0 msgr2=0x7f56b810a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8102ba0 0x7f56b810a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 -- 172.21.15.26:0/4237393224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 msgr2=0x7f56b810c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 0x7f56b810c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bd315640 1 -- 172.21.15.26:0/4237393224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b810b740 con 0x7f56b8103560 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.692+0000 7f56bcb14640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 0x7f56b810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.693+0000 7f56bd315640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 0x7f56b810aeb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c970 tx=0x7f56a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.693+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007bb0 con 0x7f56b8103560 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.693+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b81a2cd0 con 0x7f56b8103560 2024-05-20T20:55:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.693+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56a0007d50 con 0x7f56b8103560 2024-05-20T20:55:46.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.693+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004d50 con 0x7f56b8103560 2024-05-20T20:55:46.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.693+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b81a31c0 con 0x7f56b8103560 2024-05-20T20:55:46.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.694+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b8108d50 con 0x7f56b8103560 2024-05-20T20:55:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.695+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f56a0004f10 con 0x7f56b8103560 2024-05-20T20:55:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.696+0000 7f56b5ffb640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56a40b27f0 0x7f56a40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.696+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f56a00e23b0 con 0x7f56b8103560 2024-05-20T20:55:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.696+0000 7f56bcb14640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56a40b27f0 0x7f56a40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.697+0000 7f56bcb14640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56a40b27f0 0x7f56a40b4cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56b8102a00 tx=0x7f56b000a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:46.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.699+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56a0014030 con 0x7f56b8103560 2024-05-20T20:55:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.892+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56b802de30 con 0x7f56b8103560 2024-05-20T20:55:46.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.894+0000 7f56b5ffb640 1 -- 172.21.15.26:0/4237393224 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22791 (secure 0 0 0) 0x7f56a00aa010 con 0x7f56b8103560 2024-05-20T20:55:46.894 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:54:37.856459+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:55:46.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56a40b27f0 msgr2=0x7f56a40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56a40b27f0 0x7f56a40b4cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56b8102a00 tx=0x7f56b000a5e0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 msgr2=0x7f56b810aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 0x7f56b810aeb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c970 tx=0x7f56a000ce30 comp rx=0 tx=0).stop 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 shutdown_connections 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56a40b27f0 0x7f56a40b4cb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56b8103560 0x7f56b810aeb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8102ba0 0x7f56b810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 --2- 172.21.15.26:0/4237393224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b81019b0 0x7f56b810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 >> 172.21.15.26:0/4237393224 conn(0x7f56b80fcf40 msgr2=0x7f56b8105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 shutdown_connections 2024-05-20T20:55:46.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:46.896+0000 7f56bed78640 1 -- 172.21.15.26:0/4237393224 wait complete. 2024-05-20T20:55:46.906 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:55:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:55:46.506+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:55:46.906 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:55:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:55:46.506+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:55:46.907 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:55:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:55:46.506+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:55:46.907 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:55:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:55:46.506+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:55:46.907 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 20:55:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T20:55:46.506+0000 7f5f9270f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T20:55:46.908 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T20:47:57.693552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T20:52:41.091910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T20:52:43.961192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T20:55:46.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T20:55:47.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.016+0000 7f10f02a7640 1 -- 172.21.15.26:0/2012380965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 msgr2=0x7f10e81059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.016+0000 7f10f02a7640 1 --2- 172.21.15.26:0/2012380965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e81059e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009900 tx=0x7f10e402a410 comp rx=0 tx=0).stop 2024-05-20T20:55:47.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- 172.21.15.26:0/2012380965 shutdown_connections 2024-05-20T20:55:47.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 --2- 172.21.15.26:0/2012380965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8108850 0x7f10e810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 --2- 172.21.15.26:0/2012380965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e8105f20 0x7f10e8108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 --2- 172.21.15.26:0/2012380965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e81059e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- 172.21.15.26:0/2012380965 >> 172.21.15.26:0/2012380965 conn(0x7f10e80fcf20 msgr2=0x7f10e80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- 172.21.15.26:0/2012380965 shutdown_connections 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- 172.21.15.26:0/2012380965 wait complete. 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 Processor -- start 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- start start 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8105f20 0x7f10e81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e8108850 0x7f10e81a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10e810e4c0 con 0x7f10e81035f0 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10e810e340 con 0x7f10e8105f20 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.017+0000 7f10f02a7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10e810e640 con 0x7f10e8108850 2024-05-20T20:55:47.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41248/0 (socket says 172.21.15.26:41248) 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 -- 172.21.15.26:0/1601128358 learned_addr learned my addr 172.21.15.26:0/1601128358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee844640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e8108850 0x7f10e81a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ed842640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8105f20 0x7f10e81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 -- 172.21.15.26:0/1601128358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e8108850 msgr2=0x7f10e81a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e8108850 0x7f10e81a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 -- 172.21.15.26:0/1601128358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8105f20 msgr2=0x7f10e81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8105f20 0x7f10e81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 -- 172.21.15.26:0/1601128358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81a08f0 con 0x7f10e81035f0 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ed842640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8105f20 0x7f10e81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:47.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10ee043640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e8198cb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009a20 tx=0x7f10e40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:47.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e403f030 con 0x7f10e81035f0 2024-05-20T20:55:47.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10e4003a80 con 0x7f10e81035f0 2024-05-20T20:55:47.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81a0b80 con 0x7f10e81035f0 2024-05-20T20:55:47.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4035070 con 0x7f10e81035f0 2024-05-20T20:55:47.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.018+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81a10c0 con 0x7f10e81035f0 2024-05-20T20:55:47.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.019+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e80fea10 con 0x7f10e81035f0 2024-05-20T20:55:47.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.021+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f10e4042030 con 0x7f10e81035f0 2024-05-20T20:55:47.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.022+0000 7f10df7fe640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10d40b2870 0x7f10d40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.022+0000 7f10ed842640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10d40b2870 0x7f10d40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.022+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f10e4102cb0 con 0x7f10e81035f0 2024-05-20T20:55:47.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.023+0000 7f10ed842640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10d40b2870 0x7f10d40b4d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10e819a1b0 tx=0x7f10d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.026+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10e40ca910 con 0x7f10e81035f0 2024-05-20T20:55:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:47 smithi026 ceph-mon[31990]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:47 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1799227330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-20T20:55:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:47 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:55:47.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:47 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4237393224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.216+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10e802dda0 con 0x7f10e81035f0 2024-05-20T20:55:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.216+0000 7f10df7fe640 1 -- 172.21.15.26:0/1601128358 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f10e40ca210 con 0x7f10e81035f0 2024-05-20T20:55:47.216 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10d40b2870 msgr2=0x7f10d40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10d40b2870 0x7f10d40b4d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10e819a1b0 tx=0x7f10d8005f90 comp rx=0 tx=0).stop 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 msgr2=0x7f10e8198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e8198cb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009a20 tx=0x7f10e40040c0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 shutdown_connections 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10d40b2870 0x7f10d40b4d30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e8108850 0x7f10e81a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e8105f20 0x7f10e81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 --2- 172.21.15.26:0/1601128358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10e81035f0 0x7f10e8198cb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.218+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 >> 172.21.15.26:0/1601128358 conn(0x7f10e80fcf20 msgr2=0x7f10e81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.219+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 shutdown_connections 2024-05-20T20:55:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.219+0000 7f10f02a7640 1 -- 172.21.15.26:0/1601128358 wait complete. 2024-05-20T20:55:47.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T20:55:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 -- 172.21.15.26:0/3609663194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 msgr2=0x7f205010abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 --2- 172.21.15.26:0/3609663194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f205010abe0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2048009ca0 tx=0x7f204802a8c0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 -- 172.21.15.26:0/3609663194 shutdown_connections 2024-05-20T20:55:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 --2- 172.21.15.26:0/3609663194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f205010abe0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 --2- 172.21.15.26:0/3609663194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2050105e60 0x7f2050108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 --2- 172.21.15.26:0/3609663194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2050103530 0x7f2050105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 -- 172.21.15.26:0/3609663194 >> 172.21.15.26:0/3609663194 conn(0x7f20500fcf40 msgr2=0x7f20500ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 -- 172.21.15.26:0/3609663194 shutdown_connections 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.339+0000 7f20563a4640 1 -- 172.21.15.26:0/3609663194 wait complete. 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 Processor -- start 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 -- start start 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2050103530 0x7f2050198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2050105e60 0x7f2050199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f20501a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f205010e550 con 0x7f2050108790 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f205010e3d0 con 0x7f2050105e60 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f20563a4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f205010e6d0 con 0x7f2050103530 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f20501a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f20501a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41270/0 (socket says 172.21.15.26:41270) 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 -- 172.21.15.26:0/4003464607 learned_addr learned my addr 172.21.15.26:0/4003464607 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2047fff640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2050105e60 0x7f2050199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f204ffff640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2050103530 0x7f2050198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 -- 172.21.15.26:0/4003464607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2050103530 msgr2=0x7f2050198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2050103530 0x7f2050198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 -- 172.21.15.26:0/4003464607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2050105e60 msgr2=0x7f2050199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2050105e60 0x7f2050199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.340+0000 7f2054941640 1 -- 172.21.15.26:0/4003464607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20501a07f0 con 0x7f2050108790 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.341+0000 7f2054941640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f20501a0180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2048031040 tx=0x7f2048003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.341+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048004040 con 0x7f2050108790 2024-05-20T20:55:47.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.341+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2048004670 con 0x7f2050108790 2024-05-20T20:55:47.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.341+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20501a0a80 con 0x7f2050108790 2024-05-20T20:55:47.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.341+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048030dd0 con 0x7f2050108790 2024-05-20T20:55:47.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.341+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20501a0fd0 con 0x7f2050108790 2024-05-20T20:55:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.342+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20500fea30 con 0x7f2050108790 2024-05-20T20:55:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.344+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f20480305c0 con 0x7f2050108790 2024-05-20T20:55:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.345+0000 7f204dffb640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f203c0b2990 0x7f203c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.345+0000 7f204ffff640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f203c0b2990 0x7f203c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.345+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f2048103ca0 con 0x7f2050108790 2024-05-20T20:55:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.345+0000 7f204ffff640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f203c0b2990 0x7f203c0b4e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f204000a810 tx=0x7f2040008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:47.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.348+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20480cb990 con 0x7f2050108790 2024-05-20T20:55:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:47 smithi055 ceph-mon[35213]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:47 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1799227330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-20T20:55:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:47 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:55:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:47 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4237393224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:47.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.538+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2050063650 con 0x7f2050108790 2024-05-20T20:55:47.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.539+0000 7f204dffb640 1 -- 172.21.15.26:0/4003464607 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f20480cb290 con 0x7f2050108790 2024-05-20T20:55:47.539 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f203c0b2990 msgr2=0x7f203c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f203c0b2990 0x7f203c0b4e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f204000a810 tx=0x7f2040008040 comp rx=0 tx=0).stop 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 msgr2=0x7f20501a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f20501a0180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2048031040 tx=0x7f2048003ce0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 shutdown_connections 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f203c0b2990 0x7f203c0b4e50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2050108790 0x7f20501a0180 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2050105e60 0x7f2050199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 --2- 172.21.15.26:0/4003464607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2050103530 0x7f2050198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 >> 172.21.15.26:0/4003464607 conn(0x7f20500fcf40 msgr2=0x7f2050103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 shutdown_connections 2024-05-20T20:55:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.541+0000 7f20563a4640 1 -- 172.21.15.26:0/4003464607 wait complete. 2024-05-20T20:55:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:47 smithi144 ceph-mon[35660]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T20:55:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:47 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1799227330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-20T20:55:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:47 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:55:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:47 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4237393224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:47.552 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T20:55:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 -- 172.21.15.26:0/155956726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac103540 msgr2=0x7ff3ac109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 --2- 172.21.15.26:0/155956726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac103540 0x7ff3ac109dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009ca0 tx=0x7ff3a402a8c0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 -- 172.21.15.26:0/155956726 shutdown_connections 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 --2- 172.21.15.26:0/155956726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac103540 0x7ff3ac109dd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 --2- 172.21.15.26:0/155956726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3ac102b80 0x7ff3ac103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 --2- 172.21.15.26:0/155956726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac101990 0x7ff3ac101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 -- 172.21.15.26:0/155956726 >> 172.21.15.26:0/155956726 conn(0x7ff3ac0fcf00 msgr2=0x7ff3ac0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 -- 172.21.15.26:0/155956726 shutdown_connections 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 -- 172.21.15.26:0/155956726 wait complete. 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.661+0000 7ff3b1112640 1 Processor -- start 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 -- start start 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 0x7ff3ac06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3ac102b80 0x7ff3ac19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac103540 0x7ff3ac19ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d3c0 con 0x7ff3ac101990 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d240 con 0x7ff3ac103540 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3b1112640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d540 con 0x7ff3ac102b80 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 0x7ff3ac06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 0x7ff3ac06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41280/0 (socket says 172.21.15.26:41280) 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 -- 172.21.15.26:0/1305939143 learned_addr learned my addr 172.21.15.26:0/1305939143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3ab59e640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac103540 0x7ff3ac19ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 -- 172.21.15.26:0/1305939143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3ac102b80 msgr2=0x7ff3ac19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3ac102b80 0x7ff3ac19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 -- 172.21.15.26:0/1305939143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac103540 msgr2=0x7ff3ac19ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac103540 0x7ff3ac19ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 -- 172.21.15.26:0/1305939143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac1a0820 con 0x7ff3ac101990 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3ab59e640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac103540 0x7ff3ac19ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3aad9d640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 0x7ff3ac06a770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c970 tx=0x7ff39c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.662+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c004220 con 0x7ff3ac101990 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.663+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff39c007e40 con 0x7ff3ac101990 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.663+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00f450 con 0x7ff3ac101990 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.663+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1a0b10 con 0x7ff3ac101990 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.663+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1a1010 con 0x7ff3ac101990 2024-05-20T20:55:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.664+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ac108dd0 con 0x7ff3ac101990 2024-05-20T20:55:47.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.665+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff39c003610 con 0x7ff3ac101990 2024-05-20T20:55:47.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.666+0000 7ff3a8d99640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff3980b2870 0x7ff3980b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.666+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff39c0e1f60 con 0x7ff3ac101990 2024-05-20T20:55:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.666+0000 7ff3a3fff640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff3980b2870 0x7ff3980b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.666+0000 7ff3a3fff640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff3980b2870 0x7ff3980b4d30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac06a4e0 tx=0x7ff394005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:47.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.670+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff39c014030 con 0x7ff3ac101990 2024-05-20T20:55:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.861+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3ac02dba0 con 0x7ff3ac101990 2024-05-20T20:55:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.861+0000 7ff3a8d99640 1 -- 172.21.15.26:0/1305939143 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7ff3ac02dba0 con 0x7ff3ac101990 2024-05-20T20:55:47.862 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-05-20T20:55:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff3980b2870 msgr2=0x7ff3980b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff3980b2870 0x7ff3980b4d30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac06a4e0 tx=0x7ff394005bf0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 msgr2=0x7ff3ac06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 0x7ff3ac06a770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c970 tx=0x7ff39c00ce30 comp rx=0 tx=0).stop 2024-05-20T20:55:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 shutdown_connections 2024-05-20T20:55:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff3980b2870 0x7ff3980b4d30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3ac103540 0x7ff3ac19ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3ac102b80 0x7ff3ac19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 --2- 172.21.15.26:0/1305939143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac101990 0x7ff3ac06a770 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 >> 172.21.15.26:0/1305939143 conn(0x7ff3ac0fcf00 msgr2=0x7ff3ac105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 shutdown_connections 2024-05-20T20:55:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:47.863+0000 7ff3b1112640 1 -- 172.21.15.26:0/1305939143 wait complete. 2024-05-20T20:55:47.875 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-20T20:55:47.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:55:48.125 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:48.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:48 smithi026 ceph-mon[31990]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:48.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1601128358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4003464607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1305939143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:48 smithi055 ceph-mon[35213]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:48.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1601128358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4003464607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1305939143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:48 smithi144 ceph-mon[35660]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T20:55:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1601128358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4003464607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T20:55:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1305939143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T20:55:49.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:49 smithi055 ceph-mon[35213]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-20T20:55:49.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:49 smithi055 ceph-mon[35213]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-20T20:55:49.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:49 smithi026 ceph-mon[31990]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-20T20:55:49.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:49 smithi026 ceph-mon[31990]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-20T20:55:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:49 smithi144 ceph-mon[35660]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-20T20:55:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:49 smithi144 ceph-mon[35660]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-20T20:55:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 -- 172.21.15.26:0/2758916443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c06bbe0 msgr2=0x7f946c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 --2- 172.21.15.26:0/2758916443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c06bbe0 0x7f946c1103d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f946400b0a0 tx=0x7f946400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 -- 172.21.15.26:0/2758916443 shutdown_connections 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 --2- 172.21.15.26:0/2758916443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c06bbe0 0x7f946c1103d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 --2- 172.21.15.26:0/2758916443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f946c1095b0 0x7f946c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 --2- 172.21.15.26:0/2758916443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c06b220 0x7f946c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 -- 172.21.15.26:0/2758916443 >> 172.21.15.26:0/2758916443 conn(0x7f946c0793f0 msgr2=0x7f946c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 -- 172.21.15.26:0/2758916443 shutdown_connections 2024-05-20T20:55:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.857+0000 7f947395e700 1 -- 172.21.15.26:0/2758916443 wait complete. 2024-05-20T20:55:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 Processor -- start 2024-05-20T20:55:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 -- start start 2024-05-20T20:55:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c06b220 0x7f946c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f946c06bbe0 0x7f946c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 0x7f946c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113590 con 0x7f946c1095b0 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113410 con 0x7f946c06b220 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f947395e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113710 con 0x7f946c06bbe0 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f9471efb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 0x7f946c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f9471efb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 0x7f946c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56318/0 (socket says 172.21.15.26:56318) 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.858+0000 7f9471efb700 1 -- 172.21.15.26:0/1048208657 learned_addr learned my addr 172.21.15.26:0/1048208657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9470ef9700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f946c06bbe0 0x7f946c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f94716fa700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c06b220 0x7f946c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9471efb700 1 -- 172.21.15.26:0/1048208657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f946c06bbe0 msgr2=0x7f946c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9471efb700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f946c06bbe0 0x7f946c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9471efb700 1 -- 172.21.15.26:0/1048208657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c06b220 msgr2=0x7f946c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9471efb700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c06b220 0x7f946c102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9471efb700 1 -- 172.21.15.26:0/1048208657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c1a8010 con 0x7f946c1095b0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f9471efb700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 0x7f946c102f90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f946400b830 tx=0x7f9464004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464021070 con 0x7f946c1095b0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9464007b80 con 0x7f946c1095b0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c1a82a0 con 0x7f946c1095b0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640052a0 con 0x7f946c1095b0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.859+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c1a86f0 con 0x7f946c1095b0 2024-05-20T20:55:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.860+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f946c10ba90 con 0x7f946c1095b0 2024-05-20T20:55:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.862+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f9464024070 con 0x7f946c1095b0 2024-05-20T20:55:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.862+0000 7f94627fc700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94580b2910 0x7f94580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.862+0000 7f94716fa700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94580b2910 0x7f94580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.862+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f946400f020 con 0x7f946c1095b0 2024-05-20T20:55:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.863+0000 7f94716fa700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94580b2910 0x7f94580b4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f946800bbb0 tx=0x7f946800b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:49.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:49.866+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f946402b020 con 0x7f946c1095b0 2024-05-20T20:55:50.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.129+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f946c06a230 con 0x7f946c1095b0 2024-05-20T20:55:50.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.130+0000 7f94627fc700 1 -- 172.21.15.26:0/1048208657 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f94640aba20 con 0x7f946c1095b0 2024-05-20T20:55:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:50.131 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:55:50.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94580b2910 msgr2=0x7f94580b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:50.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94580b2910 0x7f94580b4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f946800bbb0 tx=0x7f946800b590 comp rx=0 tx=0).stop 2024-05-20T20:55:50.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 msgr2=0x7f946c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:50.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 0x7f946c102f90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f946400b830 tx=0x7f9464004d00 comp rx=0 tx=0).stop 2024-05-20T20:55:50.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 shutdown_connections 2024-05-20T20:55:50.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94580b2910 0x7f94580b4dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946c1095b0 0x7f946c102f90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c06b220 0x7f946c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 --2- 172.21.15.26:0/1048208657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f946c06bbe0 0x7f946c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 >> 172.21.15.26:0/1048208657 conn(0x7f946c0793f0 msgr2=0x7f946c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 shutdown_connections 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:50.133+0000 7f947395e700 1 -- 172.21.15.26:0/1048208657 wait complete. 2024-05-20T20:55:50.134 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:55:50.450 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:50 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1048208657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:50.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:50 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1048208657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:50.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:50 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1048208657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:50.907 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-20T20:55:50.908 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-20T20:55:50.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T20:55:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.017+0000 7f77a879d640 1 -- 172.21.15.26:0/3735033956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a01087d0 msgr2=0x7f77a010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.017+0000 7f77a879d640 1 --2- 172.21.15.26:0/3735033956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a01087d0 0x7f77a010abc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7798009ca0 tx=0x7f779802a8c0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- 172.21.15.26:0/3735033956 shutdown_connections 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 --2- 172.21.15.26:0/3735033956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a01087d0 0x7f77a010abc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 --2- 172.21.15.26:0/3735033956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77a0105ea0 0x7f77a0108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 --2- 172.21.15.26:0/3735033956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0103570 0x7f77a0105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- 172.21.15.26:0/3735033956 >> 172.21.15.26:0/3735033956 conn(0x7f77a00fcf00 msgr2=0x7f77a00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- 172.21.15.26:0/3735033956 shutdown_connections 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- 172.21.15.26:0/3735033956 wait complete. 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 Processor -- start 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- start start 2024-05-20T20:55:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 0x7f77a0102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0105ea0 0x7f77a0101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77a01087d0 0x7f77a0101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77a010e4e0 con 0x7f77a0103570 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f77a010e360 con 0x7f77a01087d0 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.018+0000 7f77a879d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a010e660 con 0x7f77a0105ea0 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 0x7f77a0102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 0x7f77a0102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56338/0 (socket says 172.21.15.26:56338) 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 -- 172.21.15.26:0/856179667 learned_addr learned my addr 172.21.15.26:0/856179667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6d3a640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77a01087d0 0x7f77a0101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a5d38640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0105ea0 0x7f77a0101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 -- 172.21.15.26:0/856179667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0105ea0 msgr2=0x7f77a0101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0105ea0 0x7f77a0101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 -- 172.21.15.26:0/856179667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77a01087d0 msgr2=0x7f77a0101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77a01087d0 0x7f77a0101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 -- 172.21.15.26:0/856179667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a01022f0 con 0x7f77a0103570 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a5d38640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0105ea0 0x7f77a0101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a6539640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 0x7f77a0102ee0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f779c00d770 tx=0x7f779c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c017070 con 0x7f77a0103570 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a01a2d00 con 0x7f77a0103570 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f779c0054b0 con 0x7f77a0103570 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c004c60 con 0x7f77a0103570 2024-05-20T20:55:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.019+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a01a31a0 con 0x7f77a0103570 2024-05-20T20:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.020+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a00fe9f0 con 0x7f77a0103570 2024-05-20T20:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.022+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f779c004e00 con 0x7f77a0103570 2024-05-20T20:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.022+0000 7f77977fe640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f778c0b2880 0x7f778c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.023+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f779c0e1e60 con 0x7f77a0103570 2024-05-20T20:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.023+0000 7f77a5d38640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f778c0b2880 0x7f778c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.023+0000 7f77a5d38640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f778c0b2880 0x7f778c0b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77900058c0 tx=0x7f7790005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.027+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f779c0a99c0 con 0x7f77a0103570 2024-05-20T20:55:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.223+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77a0062e90 con 0x7f778c0b2880 2024-05-20T20:55:51.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.231+0000 7f77977fe640 1 -- 172.21.15.26:0/856179667 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259085 (secure 0 0 0) 0x7f77a0062e90 con 0x7f778c0b2880 2024-05-20T20:55:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:51.236 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:55:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f778c0b2880 msgr2=0x7f778c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f778c0b2880 0x7f778c0b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77900058c0 tx=0x7f7790005850 comp rx=0 tx=0).stop 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 msgr2=0x7f77a0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 0x7f77a0102ee0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f779c00d770 tx=0x7f779c00dc30 comp rx=0 tx=0).stop 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 shutdown_connections 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f778c0b2880 0x7f778c0b4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77a01087d0 0x7f77a0101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a0105ea0 0x7f77a0101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 --2- 172.21.15.26:0/856179667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a0103570 0x7f77a0102ee0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 >> 172.21.15.26:0/856179667 conn(0x7f77a00fcf00 msgr2=0x7f77a0103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 shutdown_connections 2024-05-20T20:55:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.235+0000 7f77a879d640 1 -- 172.21.15.26:0/856179667 wait complete. 2024-05-20T20:55:51.246 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-05-20T20:55:49.740900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663394,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1108,"num_read_kb":657,"num_write":308,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":749,"ondisk_log_size":749,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355980,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585844,"statfs":{"total":1151940427776,"available":1151575904256,"internally_reserved":0,"allocated":11227136,"data_stored":7287581,"data_compressed":249159,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6575,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":49,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002718"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870824+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:54:37.870824+0000","last_peered":"2024-05-20T20:54:37.870824+0000","last_clean":"2024-05-20T20:54:37.870824+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:54:37.870824+0000","last_undegraded":"2024-05-20T20:54:37.870824+0000","last_fullsized":"2024-05-20T20:54:37.870824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886671+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:38.886671+0000","last_peered":"2024-05-20T20:54:38.886671+0000","last_clean":"2024-05-20T20:54:38.886671+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:38.886671+0000","last_undegraded":"2024-05-20T20:54:38.886671+0000","last_fullsized":"2024-05-20T20:54:38.886671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886528+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:54:38.886528+0000","last_peered":"2024-05-20T20:54:38.886528+0000","last_clean":"2024-05-20T20:54:38.886528+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:54:38.886528+0000","last_undegraded":"2024-05-20T20:54:38.886528+0000","last_fullsized":"2024-05-20T20:54:38.886528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475208+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:38.475208+0000","last_peered":"2024-05-20T20:54:38.475208+0000","last_clean":"2024-05-20T20:54:38.475208+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:38.475208+0000","last_undegraded":"2024-05-20T20:54:38.475208+0000","last_fullsized":"2024-05-20T20:54:38.475208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887218+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:38.887218+0000","last_peered":"2024-05-20T20:54:38.887218+0000","last_clean":"2024-05-20T20:54:38.887218+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:38.887218+0000","last_undegraded":"2024-05-20T20:54:38.887218+0000","last_fullsized":"2024-05-20T20:54:38.887218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876469+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876469+0000","last_peered":"2024-05-20T20:54:37.876469+0000","last_clean":"2024-05-20T20:54:37.876469+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876469+0000","last_undegraded":"2024-05-20T20:54:37.876469+0000","last_fullsized":"2024-05-20T20:54:37.876469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870690+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870690+0000","last_peered":"2024-05-20T20:54:37.870690+0000","last_clean":"2024-05-20T20:54:37.870690+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870690+0000","last_undegraded":"2024-05-20T20:54:37.870690+0000","last_fullsized":"2024-05-20T20:54:37.870690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766030+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:54:38.766030+0000","last_peered":"2024-05-20T20:54:38.766030+0000","last_clean":"2024-05-20T20:54:38.766030+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:54:38.766030+0000","last_undegraded":"2024-05-20T20:54:38.766030+0000","last_fullsized":"2024-05-20T20:54:38.766030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876712+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876712+0000","last_peered":"2024-05-20T20:54:37.876712+0000","last_clean":"2024-05-20T20:54:37.876712+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876712+0000","last_undegraded":"2024-05-20T20:54:37.876712+0000","last_fullsized":"2024-05-20T20:54:37.876712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871157+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871157+0000","last_peered":"2024-05-20T20:54:37.871157+0000","last_clean":"2024-05-20T20:54:37.871157+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871157+0000","last_undegraded":"2024-05-20T20:54:37.871157+0000","last_fullsized":"2024-05-20T20:54:37.871157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475180+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:38.475180+0000","last_peered":"2024-05-20T20:54:38.475180+0000","last_clean":"2024-05-20T20:54:38.475180+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:38.475180+0000","last_undegraded":"2024-05-20T20:54:38.475180+0000","last_fullsized":"2024-05-20T20:54:38.475180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867869+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:37.867869+0000","last_peered":"2024-05-20T20:54:37.867869+0000","last_clean":"2024-05-20T20:54:37.867869+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:37.867869+0000","last_undegraded":"2024-05-20T20:54:37.867869+0000","last_fullsized":"2024-05-20T20:54:37.867869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886436+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:38.886436+0000","last_peered":"2024-05-20T20:54:38.886436+0000","last_clean":"2024-05-20T20:54:38.886436+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:38.886436+0000","last_undegraded":"2024-05-20T20:54:38.886436+0000","last_fullsized":"2024-05-20T20:54:38.886436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886364+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:38.886364+0000","last_peered":"2024-05-20T20:54:38.886364+0000","last_clean":"2024-05-20T20:54:38.886364+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:38.886364+0000","last_undegraded":"2024-05-20T20:54:38.886364+0000","last_fullsized":"2024-05-20T20:54:38.886364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765447+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:54:38.765447+0000","last_peered":"2024-05-20T20:54:38.765447+0000","last_clean":"2024-05-20T20:54:38.765447+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:54:38.765447+0000","last_undegraded":"2024-05-20T20:54:38.765447+0000","last_fullsized":"2024-05-20T20:54:38.765447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870916+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:54:37.870916+0000","last_peered":"2024-05-20T20:54:37.870916+0000","last_clean":"2024-05-20T20:54:37.870916+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:54:37.870916+0000","last_undegraded":"2024-05-20T20:54:37.870916+0000","last_fullsized":"2024-05-20T20:54:37.870916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872609+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872609+0000","last_peered":"2024-05-20T20:54:37.872609+0000","last_clean":"2024-05-20T20:54:37.872609+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872609+0000","last_undegraded":"2024-05-20T20:54:37.872609+0000","last_fullsized":"2024-05-20T20:54:37.872609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887790+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:38.887790+0000","last_peered":"2024-05-20T20:54:38.887790+0000","last_clean":"2024-05-20T20:54:38.887790+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:38.887790+0000","last_undegraded":"2024-05-20T20:54:38.887790+0000","last_fullsized":"2024-05-20T20:54:38.887790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872721+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:37.872721+0000","last_peered":"2024-05-20T20:54:37.872721+0000","last_clean":"2024-05-20T20:54:37.872721+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:37.872721+0000","last_undegraded":"2024-05-20T20:54:37.872721+0000","last_fullsized":"2024-05-20T20:54:37.872721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474659+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:38.474659+0000","last_peered":"2024-05-20T20:54:38.474659+0000","last_clean":"2024-05-20T20:54:38.474659+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:38.474659+0000","last_undegraded":"2024-05-20T20:54:38.474659+0000","last_fullsized":"2024-05-20T20:54:38.474659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474766+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:54:38.474766+0000","last_peered":"2024-05-20T20:54:38.474766+0000","last_clean":"2024-05-20T20:54:38.474766+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:54:38.474766+0000","last_undegraded":"2024-05-20T20:54:38.474766+0000","last_fullsized":"2024-05-20T20:54:38.474766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886666+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.886666+0000","last_peered":"2024-05-20T20:54:38.886666+0000","last_clean":"2024-05-20T20:54:38.886666+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:54:38.886666+0000","last_undegraded":"2024-05-20T20:54:38.886666+0000","last_fullsized":"2024-05-20T20:54:38.886666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887575+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:38.887575+0000","last_peered":"2024-05-20T20:54:38.887575+0000","last_clean":"2024-05-20T20:54:38.887575+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:38.887575+0000","last_undegraded":"2024-05-20T20:54:38.887575+0000","last_fullsized":"2024-05-20T20:54:38.887575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765538+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:54:38.765538+0000","last_peered":"2024-05-20T20:54:38.765538+0000","last_clean":"2024-05-20T20:54:38.765538+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:54:38.765538+0000","last_undegraded":"2024-05-20T20:54:38.765538+0000","last_fullsized":"2024-05-20T20:54:38.765538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887488+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:38.887488+0000","last_peered":"2024-05-20T20:54:38.887488+0000","last_clean":"2024-05-20T20:54:38.887488+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:38.887488+0000","last_undegraded":"2024-05-20T20:54:38.887488+0000","last_fullsized":"2024-05-20T20:54:38.887488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874582+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:37.874582+0000","last_peered":"2024-05-20T20:54:37.874582+0000","last_clean":"2024-05-20T20:54:37.874582+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:37.874582+0000","last_undegraded":"2024-05-20T20:54:37.874582+0000","last_fullsized":"2024-05-20T20:54:37.874582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":206,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:46.567588+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:55:46.567588+0000","last_peered":"2024-05-20T20:55:46.567588+0000","last_clean":"2024-05-20T20:55:46.567588+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:55:46.567588+0000","last_undegraded":"2024-05-20T20:55:46.567588+0000","last_fullsized":"2024-05-20T20:55:46.567588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887502+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:38.887502+0000","last_peered":"2024-05-20T20:54:38.887502+0000","last_clean":"2024-05-20T20:54:38.887502+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:38.887502+0000","last_undegraded":"2024-05-20T20:54:38.887502+0000","last_fullsized":"2024-05-20T20:54:38.887502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876310+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876310+0000","last_peered":"2024-05-20T20:54:37.876310+0000","last_clean":"2024-05-20T20:54:37.876310+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876310+0000","last_undegraded":"2024-05-20T20:54:37.876310+0000","last_fullsized":"2024-05-20T20:54:37.876310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874230+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:37.874230+0000","last_peered":"2024-05-20T20:54:37.874230+0000","last_clean":"2024-05-20T20:54:37.874230+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:37.874230+0000","last_undegraded":"2024-05-20T20:54:37.874230+0000","last_fullsized":"2024-05-20T20:54:37.874230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867681+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:37.867681+0000","last_peered":"2024-05-20T20:54:37.867681+0000","last_clean":"2024-05-20T20:54:37.867681+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:37.867681+0000","last_undegraded":"2024-05-20T20:54:37.867681+0000","last_fullsized":"2024-05-20T20:54:37.867681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474359+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:38.474359+0000","last_peered":"2024-05-20T20:54:38.474359+0000","last_clean":"2024-05-20T20:54:38.474359+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:38.474359+0000","last_undegraded":"2024-05-20T20:54:38.474359+0000","last_fullsized":"2024-05-20T20:54:38.474359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874492+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:37.874492+0000","last_peered":"2024-05-20T20:54:37.874492+0000","last_clean":"2024-05-20T20:54:37.874492+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:37.874492+0000","last_undegraded":"2024-05-20T20:54:37.874492+0000","last_fullsized":"2024-05-20T20:54:37.874492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872063+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872063+0000","last_peered":"2024-05-20T20:54:37.872063+0000","last_clean":"2024-05-20T20:54:37.872063+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872063+0000","last_undegraded":"2024-05-20T20:54:37.872063+0000","last_fullsized":"2024-05-20T20:54:37.872063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":203,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:48.571135+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:55:48.571135+0000","last_peered":"2024-05-20T20:55:48.571135+0000","last_clean":"2024-05-20T20:55:48.571135+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:55:48.571135+0000","last_undegraded":"2024-05-20T20:55:48.571135+0000","last_fullsized":"2024-05-20T20:55:48.571135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765842+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:54:38.765842+0000","last_peered":"2024-05-20T20:54:38.765842+0000","last_clean":"2024-05-20T20:54:38.765842+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:54:38.765842+0000","last_undegraded":"2024-05-20T20:54:38.765842+0000","last_fullsized":"2024-05-20T20:54:38.765842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871069+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871069+0000","last_peered":"2024-05-20T20:54:37.871069+0000","last_clean":"2024-05-20T20:54:37.871069+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871069+0000","last_undegraded":"2024-05-20T20:54:37.871069+0000","last_fullsized":"2024-05-20T20:54:37.871069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886717+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:38.886717+0000","last_peered":"2024-05-20T20:54:38.886717+0000","last_clean":"2024-05-20T20:54:38.886717+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:38.886717+0000","last_undegraded":"2024-05-20T20:54:38.886717+0000","last_fullsized":"2024-05-20T20:54:38.886717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874541+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:37.874541+0000","last_peered":"2024-05-20T20:54:37.874541+0000","last_clean":"2024-05-20T20:54:37.874541+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:37.874541+0000","last_undegraded":"2024-05-20T20:54:37.874541+0000","last_fullsized":"2024-05-20T20:54:37.874541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.765712+0000","last_peered":"2024-05-20T20:54:38.765712+0000","last_clean":"2024-05-20T20:54:38.765712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:54:38.765712+0000","last_undegraded":"2024-05-20T20:54:38.765712+0000","last_fullsized":"2024-05-20T20:54:38.765712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":415,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:49.572979+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:55:49.572979+0000","last_peered":"2024-05-20T20:55:49.572979+0000","last_clean":"2024-05-20T20:55:49.572979+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:55:49.572979+0000","last_undegraded":"2024-05-20T20:55:49.572979+0000","last_fullsized":"2024-05-20T20:55:49.572979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67273,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":348,"num_read_kb":239,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887265+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:38.887265+0000","last_peered":"2024-05-20T20:54:38.887265+0000","last_clean":"2024-05-20T20:54:38.887265+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:38.887265+0000","last_undegraded":"2024-05-20T20:54:38.887265+0000","last_fullsized":"2024-05-20T20:54:38.887265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887299+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:38.887299+0000","last_peered":"2024-05-20T20:54:38.887299+0000","last_clean":"2024-05-20T20:54:38.887299+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:38.887299+0000","last_undegraded":"2024-05-20T20:54:38.887299+0000","last_fullsized":"2024-05-20T20:54:38.887299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867648+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:37.867648+0000","last_peered":"2024-05-20T20:54:37.867648+0000","last_clean":"2024-05-20T20:54:37.867648+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:37.867648+0000","last_undegraded":"2024-05-20T20:54:37.867648+0000","last_fullsized":"2024-05-20T20:54:37.867648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874116+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:37.874116+0000","last_peered":"2024-05-20T20:54:37.874116+0000","last_clean":"2024-05-20T20:54:37.874116+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:37.874116+0000","last_undegraded":"2024-05-20T20:54:37.874116+0000","last_fullsized":"2024-05-20T20:54:37.874116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871059+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.871059+0000","last_peered":"2024-05-20T20:54:37.871059+0000","last_clean":"2024-05-20T20:54:37.871059+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.871059+0000","last_undegraded":"2024-05-20T20:54:37.871059+0000","last_fullsized":"2024-05-20T20:54:37.871059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":207,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.589891+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:55:45.589891+0000","last_peered":"2024-05-20T20:55:45.589891+0000","last_clean":"2024-05-20T20:55:45.589891+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:55:45.589891+0000","last_undegraded":"2024-05-20T20:55:45.589891+0000","last_fullsized":"2024-05-20T20:55:45.589891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870598+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870598+0000","last_peered":"2024-05-20T20:54:37.870598+0000","last_clean":"2024-05-20T20:54:37.870598+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870598+0000","last_undegraded":"2024-05-20T20:54:37.870598+0000","last_fullsized":"2024-05-20T20:54:37.870598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886999+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:38.886999+0000","last_peered":"2024-05-20T20:54:38.886999+0000","last_clean":"2024-05-20T20:54:38.886999+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:38.886999+0000","last_undegraded":"2024-05-20T20:54:38.886999+0000","last_fullsized":"2024-05-20T20:54:38.886999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874169+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:37.874169+0000","last_peered":"2024-05-20T20:54:37.874169+0000","last_clean":"2024-05-20T20:54:37.874169+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:37.874169+0000","last_undegraded":"2024-05-20T20:54:37.874169+0000","last_fullsized":"2024-05-20T20:54:37.874169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887338+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:38.887338+0000","last_peered":"2024-05-20T20:54:38.887338+0000","last_clean":"2024-05-20T20:54:38.887338+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:38.887338+0000","last_undegraded":"2024-05-20T20:54:38.887338+0000","last_fullsized":"2024-05-20T20:54:38.887338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870840+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:54:37.870840+0000","last_peered":"2024-05-20T20:54:37.870840+0000","last_clean":"2024-05-20T20:54:37.870840+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:54:37.870840+0000","last_undegraded":"2024-05-20T20:54:37.870840+0000","last_fullsized":"2024-05-20T20:54:37.870840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872174+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872174+0000","last_peered":"2024-05-20T20:54:37.872174+0000","last_clean":"2024-05-20T20:54:37.872174+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872174+0000","last_undegraded":"2024-05-20T20:54:37.872174+0000","last_fullsized":"2024-05-20T20:54:37.872174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876474+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876474+0000","last_peered":"2024-05-20T20:54:37.876474+0000","last_clean":"2024-05-20T20:54:37.876474+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876474+0000","last_undegraded":"2024-05-20T20:54:37.876474+0000","last_fullsized":"2024-05-20T20:54:37.876474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871372+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871372+0000","last_peered":"2024-05-20T20:54:37.871372+0000","last_clean":"2024-05-20T20:54:37.871372+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871372+0000","last_undegraded":"2024-05-20T20:54:37.871372+0000","last_fullsized":"2024-05-20T20:54:37.871372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875765+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875765+0000","last_peered":"2024-05-20T20:54:37.875765+0000","last_clean":"2024-05-20T20:54:37.875765+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875765+0000","last_undegraded":"2024-05-20T20:54:37.875765+0000","last_fullsized":"2024-05-20T20:54:37.875765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872208+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872208+0000","last_peered":"2024-05-20T20:54:37.872208+0000","last_clean":"2024-05-20T20:54:37.872208+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872208+0000","last_undegraded":"2024-05-20T20:54:37.872208+0000","last_fullsized":"2024-05-20T20:54:37.872208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":202,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.588360+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:55:45.588360+0000","last_peered":"2024-05-20T20:55:45.588360+0000","last_clean":"2024-05-20T20:55:45.588360+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:55:45.588360+0000","last_undegraded":"2024-05-20T20:55:45.588360+0000","last_fullsized":"2024-05-20T20:55:45.588360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":165,"num_read_kb":165,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474755+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:38.474755+0000","last_peered":"2024-05-20T20:54:38.474755+0000","last_clean":"2024-05-20T20:54:38.474755+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:38.474755+0000","last_undegraded":"2024-05-20T20:54:38.474755+0000","last_fullsized":"2024-05-20T20:54:38.474755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475163+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:54:38.475163+0000","last_peered":"2024-05-20T20:54:38.475163+0000","last_clean":"2024-05-20T20:54:38.475163+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:54:38.475163+0000","last_undegraded":"2024-05-20T20:54:38.475163+0000","last_fullsized":"2024-05-20T20:54:38.475163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870715+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870715+0000","last_peered":"2024-05-20T20:54:37.870715+0000","last_clean":"2024-05-20T20:54:37.870715+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870715+0000","last_undegraded":"2024-05-20T20:54:37.870715+0000","last_fullsized":"2024-05-20T20:54:37.870715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867545+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:37.867545+0000","last_peered":"2024-05-20T20:54:37.867545+0000","last_clean":"2024-05-20T20:54:37.867545+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:37.867545+0000","last_undegraded":"2024-05-20T20:54:37.867545+0000","last_fullsized":"2024-05-20T20:54:37.867545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870913+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870913+0000","last_peered":"2024-05-20T20:54:37.870913+0000","last_clean":"2024-05-20T20:54:37.870913+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870913+0000","last_undegraded":"2024-05-20T20:54:37.870913+0000","last_fullsized":"2024-05-20T20:54:37.870913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887395+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:38.887395+0000","last_peered":"2024-05-20T20:54:38.887395+0000","last_clean":"2024-05-20T20:54:38.887395+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:38.887395+0000","last_undegraded":"2024-05-20T20:54:38.887395+0000","last_fullsized":"2024-05-20T20:54:38.887395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867898+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:37.867898+0000","last_peered":"2024-05-20T20:54:37.867898+0000","last_clean":"2024-05-20T20:54:37.867898+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:37.867898+0000","last_undegraded":"2024-05-20T20:54:37.867898+0000","last_fullsized":"2024-05-20T20:54:37.867898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.261304+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:55:45.261304+0000","last_peered":"2024-05-20T20:55:45.261304+0000","last_clean":"2024-05-20T20:55:45.261304+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:55:45.261304+0000","last_undegraded":"2024-05-20T20:55:45.261304+0000","last_fullsized":"2024-05-20T20:55:45.261304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474726+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:38.474726+0000","last_peered":"2024-05-20T20:54:38.474726+0000","last_clean":"2024-05-20T20:54:38.474726+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:38.474726+0000","last_undegraded":"2024-05-20T20:54:38.474726+0000","last_fullsized":"2024-05-20T20:54:38.474726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475338+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:54:38.475338+0000","last_peered":"2024-05-20T20:54:38.475338+0000","last_clean":"2024-05-20T20:54:38.475338+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:54:38.475338+0000","last_undegraded":"2024-05-20T20:54:38.475338+0000","last_fullsized":"2024-05-20T20:54:38.475338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886677+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:38.886677+0000","last_peered":"2024-05-20T20:54:38.886677+0000","last_clean":"2024-05-20T20:54:38.886677+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:38.886677+0000","last_undegraded":"2024-05-20T20:54:38.886677+0000","last_fullsized":"2024-05-20T20:54:38.886677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765487+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:54:38.765487+0000","last_peered":"2024-05-20T20:54:38.765487+0000","last_clean":"2024-05-20T20:54:38.765487+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:54:38.765487+0000","last_undegraded":"2024-05-20T20:54:38.765487+0000","last_fullsized":"2024-05-20T20:54:38.765487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887085+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:38.887085+0000","last_peered":"2024-05-20T20:54:38.887085+0000","last_clean":"2024-05-20T20:54:38.887085+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:38.887085+0000","last_undegraded":"2024-05-20T20:54:38.887085+0000","last_fullsized":"2024-05-20T20:54:38.887085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474392+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:54:38.474392+0000","last_peered":"2024-05-20T20:54:38.474392+0000","last_clean":"2024-05-20T20:54:38.474392+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:54:38.474392+0000","last_undegraded":"2024-05-20T20:54:38.474392+0000","last_fullsized":"2024-05-20T20:54:38.474392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870487+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:54:37.870487+0000","last_peered":"2024-05-20T20:54:37.870487+0000","last_clean":"2024-05-20T20:54:37.870487+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:54:37.870487+0000","last_undegraded":"2024-05-20T20:54:37.870487+0000","last_fullsized":"2024-05-20T20:54:37.870487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870477+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:54:37.870477+0000","last_peered":"2024-05-20T20:54:37.870477+0000","last_clean":"2024-05-20T20:54:37.870477+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:54:37.870477+0000","last_undegraded":"2024-05-20T20:54:37.870477+0000","last_fullsized":"2024-05-20T20:54:37.870477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474638+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:38.474638+0000","last_peered":"2024-05-20T20:54:38.474638+0000","last_clean":"2024-05-20T20:54:38.474638+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:38.474638+0000","last_undegraded":"2024-05-20T20:54:38.474638+0000","last_fullsized":"2024-05-20T20:54:38.474638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870497+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:54:37.870497+0000","last_peered":"2024-05-20T20:54:37.870497+0000","last_clean":"2024-05-20T20:54:37.870497+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:54:37.870497+0000","last_undegraded":"2024-05-20T20:54:37.870497+0000","last_fullsized":"2024-05-20T20:54:37.870497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.880369+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.880369+0000","last_peered":"2024-05-20T20:54:37.880369+0000","last_clean":"2024-05-20T20:54:37.880369+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.880369+0000","last_undegraded":"2024-05-20T20:54:37.880369+0000","last_fullsized":"2024-05-20T20:54:37.880369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887561+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:38.887561+0000","last_peered":"2024-05-20T20:54:38.887561+0000","last_clean":"2024-05-20T20:54:38.887561+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:38.887561+0000","last_undegraded":"2024-05-20T20:54:38.887561+0000","last_fullsized":"2024-05-20T20:54:38.887561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871302+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871302+0000","last_peered":"2024-05-20T20:54:37.871302+0000","last_clean":"2024-05-20T20:54:37.871302+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871302+0000","last_undegraded":"2024-05-20T20:54:37.871302+0000","last_fullsized":"2024-05-20T20:54:37.871302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874356+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:37.874356+0000","last_peered":"2024-05-20T20:54:37.874356+0000","last_clean":"2024-05-20T20:54:37.874356+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:37.874356+0000","last_undegraded":"2024-05-20T20:54:37.874356+0000","last_fullsized":"2024-05-20T20:54:37.874356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870975+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870975+0000","last_peered":"2024-05-20T20:54:37.870975+0000","last_clean":"2024-05-20T20:54:37.870975+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870975+0000","last_undegraded":"2024-05-20T20:54:37.870975+0000","last_fullsized":"2024-05-20T20:54:37.870975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867985+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:37.867985+0000","last_peered":"2024-05-20T20:54:37.867985+0000","last_clean":"2024-05-20T20:54:37.867985+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:37.867985+0000","last_undegraded":"2024-05-20T20:54:37.867985+0000","last_fullsized":"2024-05-20T20:54:37.867985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871280+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871280+0000","last_peered":"2024-05-20T20:54:37.871280+0000","last_clean":"2024-05-20T20:54:37.871280+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871280+0000","last_undegraded":"2024-05-20T20:54:37.871280+0000","last_fullsized":"2024-05-20T20:54:37.871280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870627+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:54:37.870627+0000","last_peered":"2024-05-20T20:54:37.870627+0000","last_clean":"2024-05-20T20:54:37.870627+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:54:37.870627+0000","last_undegraded":"2024-05-20T20:54:37.870627+0000","last_fullsized":"2024-05-20T20:54:37.870627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887334+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:38.887334+0000","last_peered":"2024-05-20T20:54:38.887334+0000","last_clean":"2024-05-20T20:54:38.887334+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:38.887334+0000","last_undegraded":"2024-05-20T20:54:38.887334+0000","last_fullsized":"2024-05-20T20:54:38.887334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887439+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:38.887439+0000","last_peered":"2024-05-20T20:54:38.887439+0000","last_clean":"2024-05-20T20:54:38.887439+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:38.887439+0000","last_undegraded":"2024-05-20T20:54:38.887439+0000","last_fullsized":"2024-05-20T20:54:38.887439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886573+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:38.886573+0000","last_peered":"2024-05-20T20:54:38.886573+0000","last_clean":"2024-05-20T20:54:38.886573+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:38.886573+0000","last_undegraded":"2024-05-20T20:54:38.886573+0000","last_fullsized":"2024-05-20T20:54:38.886573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867747+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:37.867747+0000","last_peered":"2024-05-20T20:54:37.867747+0000","last_clean":"2024-05-20T20:54:37.867747+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:37.867747+0000","last_undegraded":"2024-05-20T20:54:37.867747+0000","last_fullsized":"2024-05-20T20:54:37.867747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870785+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:54:37.870785+0000","last_peered":"2024-05-20T20:54:37.870785+0000","last_clean":"2024-05-20T20:54:37.870785+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:54:37.870785+0000","last_undegraded":"2024-05-20T20:54:37.870785+0000","last_fullsized":"2024-05-20T20:54:37.870785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887850+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:38.887850+0000","last_peered":"2024-05-20T20:54:38.887850+0000","last_clean":"2024-05-20T20:54:38.887850+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:38.887850+0000","last_undegraded":"2024-05-20T20:54:38.887850+0000","last_fullsized":"2024-05-20T20:54:38.887850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474618+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:38.474618+0000","last_peered":"2024-05-20T20:54:38.474618+0000","last_clean":"2024-05-20T20:54:38.474618+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:38.474618+0000","last_undegraded":"2024-05-20T20:54:38.474618+0000","last_fullsized":"2024-05-20T20:54:38.474618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874544+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:37.874544+0000","last_peered":"2024-05-20T20:54:37.874544+0000","last_clean":"2024-05-20T20:54:37.874544+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:37.874544+0000","last_undegraded":"2024-05-20T20:54:37.874544+0000","last_fullsized":"2024-05-20T20:54:37.874544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474329+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:38.474329+0000","last_peered":"2024-05-20T20:54:38.474329+0000","last_clean":"2024-05-20T20:54:38.474329+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:38.474329+0000","last_undegraded":"2024-05-20T20:54:38.474329+0000","last_fullsized":"2024-05-20T20:54:38.474329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766006+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:54:38.766006+0000","last_peered":"2024-05-20T20:54:38.766006+0000","last_clean":"2024-05-20T20:54:38.766006+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:54:38.766006+0000","last_undegraded":"2024-05-20T20:54:38.766006+0000","last_fullsized":"2024-05-20T20:54:38.766006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867894+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:37.867894+0000","last_peered":"2024-05-20T20:54:37.867894+0000","last_clean":"2024-05-20T20:54:37.867894+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:37.867894+0000","last_undegraded":"2024-05-20T20:54:37.867894+0000","last_fullsized":"2024-05-20T20:54:37.867894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870651+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:54:37.870651+0000","last_peered":"2024-05-20T20:54:37.870651+0000","last_clean":"2024-05-20T20:54:37.870651+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:54:37.870651+0000","last_undegraded":"2024-05-20T20:54:37.870651+0000","last_fullsized":"2024-05-20T20:54:37.870651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72792,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1032,"num_read_kb":593,"num_write":183,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":442424,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1102203,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442580,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1036028,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":442049,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":504410,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":443320,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":511237,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":442360,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":443623,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822739,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1034687,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:55:51.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T20:55:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 -- 172.21.15.26:0/1352931546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0458103570 msgr2=0x7f0458105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 --2- 172.21.15.26:0/1352931546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0458103570 0x7f0458105960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0448009900 tx=0x7f044802a410 comp rx=0 tx=0).stop 2024-05-20T20:55:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 -- 172.21.15.26:0/1352931546 shutdown_connections 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 --2- 172.21.15.26:0/1352931546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04581087d0 0x7f045810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 --2- 172.21.15.26:0/1352931546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0458105ea0 0x7f0458108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 --2- 172.21.15.26:0/1352931546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0458103570 0x7f0458105960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 -- 172.21.15.26:0/1352931546 >> 172.21.15.26:0/1352931546 conn(0x7f04580fcf00 msgr2=0x7f04580ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 -- 172.21.15.26:0/1352931546 shutdown_connections 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.395+0000 7f045cf58640 1 -- 172.21.15.26:0/1352931546 wait complete. 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 Processor -- start 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 -- start start 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0458103570 0x7f0458198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0458105ea0 0x7f04581991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 0x7f04581a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f045810e490 con 0x7f04581087d0 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f045810e310 con 0x7f0458103570 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f045cf58640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f045810e610 con 0x7f0458105ea0 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 0x7f04581a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 0x7f04581a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56364/0 (socket says 172.21.15.26:56364) 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.396+0000 7f0456d9d640 1 -- 172.21.15.26:0/2506330735 learned_addr learned my addr 172.21.15.26:0/2506330735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f044ffff640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0458105ea0 0x7f04581991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f045659c640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0458103570 0x7f0458198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f0456d9d640 1 -- 172.21.15.26:0/2506330735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0458105ea0 msgr2=0x7f04581991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f0456d9d640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0458105ea0 0x7f04581991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f0456d9d640 1 -- 172.21.15.26:0/2506330735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0458103570 msgr2=0x7f0458198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f0456d9d640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0458103570 0x7f0458198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f0456d9d640 1 -- 172.21.15.26:0/2506330735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581a0880 con 0x7f04581087d0 2024-05-20T20:55:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f045659c640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0458103570 0x7f0458198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:55:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f0456d9d640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 0x7f04581a01e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f045000eb10 tx=0x7f045000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450004b20 con 0x7f04581087d0 2024-05-20T20:55:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0450004cc0 con 0x7f04581087d0 2024-05-20T20:55:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450010690 con 0x7f04581087d0 2024-05-20T20:55:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581a0b70 con 0x7f04581087d0 2024-05-20T20:55:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.397+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581a1070 con 0x7f04581087d0 2024-05-20T20:55:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.399+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04580fe9f0 con 0x7f04581087d0 2024-05-20T20:55:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.399+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f0450005020 con 0x7f04581087d0 2024-05-20T20:55:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.400+0000 7f044f7fe640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04440b2880 0x7f04440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.400+0000 7f045659c640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04440b2880 0x7f04440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.400+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f04500e2af0 con 0x7f04581087d0 2024-05-20T20:55:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.401+0000 7f045659c640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04440b2880 0x7f04440b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0448000c00 tx=0x7f0448031620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:51.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.405+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04500aa650 con 0x7f04581087d0 2024-05-20T20:55:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:51 smithi055 ceph-mon[35213]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-20T20:55:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:55:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:51 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:51.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:51 smithi026 ceph-mon[31990]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-20T20:55:51.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:55:51.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:51.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:51 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:51 smithi144 ceph-mon[35660]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-20T20:55:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:55:51.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:51.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:51 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:51.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.613+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0458062e90 con 0x7f04440b2880 2024-05-20T20:55:51.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.620+0000 7f044f7fe640 1 -- 172.21.15.26:0/2506330735 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259085 (secure 0 0 0) 0x7f0458062e90 con 0x7f04440b2880 2024-05-20T20:55:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:51.624 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04440b2880 msgr2=0x7f04440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04440b2880 0x7f04440b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0448000c00 tx=0x7f0448031620 comp rx=0 tx=0).stop 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 msgr2=0x7f04581a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 0x7f04581a01e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f045000eb10 tx=0x7f045000c2d0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 shutdown_connections 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04440b2880 0x7f04440b4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04581087d0 0x7f04581a01e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0458105ea0 0x7f04581991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 --2- 172.21.15.26:0/2506330735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0458103570 0x7f0458198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.623+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 >> 172.21.15.26:0/2506330735 conn(0x7f04580fcf00 msgr2=0x7f0458105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.624+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 shutdown_connections 2024-05-20T20:55:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.624+0000 7f045cf58640 1 -- 172.21.15.26:0/2506330735 wait complete. 2024-05-20T20:55:51.636 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-05-20T20:55:49.740900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663394,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1108,"num_read_kb":657,"num_write":308,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":749,"ondisk_log_size":749,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355980,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585844,"statfs":{"total":1151940427776,"available":1151575904256,"internally_reserved":0,"allocated":11227136,"data_stored":7287581,"data_compressed":249159,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6575,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":49,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002718"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870824+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:54:37.870824+0000","last_peered":"2024-05-20T20:54:37.870824+0000","last_clean":"2024-05-20T20:54:37.870824+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:54:37.870824+0000","last_undegraded":"2024-05-20T20:54:37.870824+0000","last_fullsized":"2024-05-20T20:54:37.870824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886671+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:38.886671+0000","last_peered":"2024-05-20T20:54:38.886671+0000","last_clean":"2024-05-20T20:54:38.886671+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:38.886671+0000","last_undegraded":"2024-05-20T20:54:38.886671+0000","last_fullsized":"2024-05-20T20:54:38.886671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886528+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:54:38.886528+0000","last_peered":"2024-05-20T20:54:38.886528+0000","last_clean":"2024-05-20T20:54:38.886528+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:54:38.886528+0000","last_undegraded":"2024-05-20T20:54:38.886528+0000","last_fullsized":"2024-05-20T20:54:38.886528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475208+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:38.475208+0000","last_peered":"2024-05-20T20:54:38.475208+0000","last_clean":"2024-05-20T20:54:38.475208+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:38.475208+0000","last_undegraded":"2024-05-20T20:54:38.475208+0000","last_fullsized":"2024-05-20T20:54:38.475208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887218+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:38.887218+0000","last_peered":"2024-05-20T20:54:38.887218+0000","last_clean":"2024-05-20T20:54:38.887218+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:38.887218+0000","last_undegraded":"2024-05-20T20:54:38.887218+0000","last_fullsized":"2024-05-20T20:54:38.887218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876469+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876469+0000","last_peered":"2024-05-20T20:54:37.876469+0000","last_clean":"2024-05-20T20:54:37.876469+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876469+0000","last_undegraded":"2024-05-20T20:54:37.876469+0000","last_fullsized":"2024-05-20T20:54:37.876469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870690+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870690+0000","last_peered":"2024-05-20T20:54:37.870690+0000","last_clean":"2024-05-20T20:54:37.870690+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870690+0000","last_undegraded":"2024-05-20T20:54:37.870690+0000","last_fullsized":"2024-05-20T20:54:37.870690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766030+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:54:38.766030+0000","last_peered":"2024-05-20T20:54:38.766030+0000","last_clean":"2024-05-20T20:54:38.766030+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:54:38.766030+0000","last_undegraded":"2024-05-20T20:54:38.766030+0000","last_fullsized":"2024-05-20T20:54:38.766030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876712+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876712+0000","last_peered":"2024-05-20T20:54:37.876712+0000","last_clean":"2024-05-20T20:54:37.876712+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876712+0000","last_undegraded":"2024-05-20T20:54:37.876712+0000","last_fullsized":"2024-05-20T20:54:37.876712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871157+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871157+0000","last_peered":"2024-05-20T20:54:37.871157+0000","last_clean":"2024-05-20T20:54:37.871157+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871157+0000","last_undegraded":"2024-05-20T20:54:37.871157+0000","last_fullsized":"2024-05-20T20:54:37.871157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475180+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:38.475180+0000","last_peered":"2024-05-20T20:54:38.475180+0000","last_clean":"2024-05-20T20:54:38.475180+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:38.475180+0000","last_undegraded":"2024-05-20T20:54:38.475180+0000","last_fullsized":"2024-05-20T20:54:38.475180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867869+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:37.867869+0000","last_peered":"2024-05-20T20:54:37.867869+0000","last_clean":"2024-05-20T20:54:37.867869+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:37.867869+0000","last_undegraded":"2024-05-20T20:54:37.867869+0000","last_fullsized":"2024-05-20T20:54:37.867869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886436+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:38.886436+0000","last_peered":"2024-05-20T20:54:38.886436+0000","last_clean":"2024-05-20T20:54:38.886436+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:38.886436+0000","last_undegraded":"2024-05-20T20:54:38.886436+0000","last_fullsized":"2024-05-20T20:54:38.886436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886364+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:38.886364+0000","last_peered":"2024-05-20T20:54:38.886364+0000","last_clean":"2024-05-20T20:54:38.886364+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:38.886364+0000","last_undegraded":"2024-05-20T20:54:38.886364+0000","last_fullsized":"2024-05-20T20:54:38.886364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765447+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:54:38.765447+0000","last_peered":"2024-05-20T20:54:38.765447+0000","last_clean":"2024-05-20T20:54:38.765447+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:54:38.765447+0000","last_undegraded":"2024-05-20T20:54:38.765447+0000","last_fullsized":"2024-05-20T20:54:38.765447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870916+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:54:37.870916+0000","last_peered":"2024-05-20T20:54:37.870916+0000","last_clean":"2024-05-20T20:54:37.870916+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:54:37.870916+0000","last_undegraded":"2024-05-20T20:54:37.870916+0000","last_fullsized":"2024-05-20T20:54:37.870916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872609+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872609+0000","last_peered":"2024-05-20T20:54:37.872609+0000","last_clean":"2024-05-20T20:54:37.872609+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872609+0000","last_undegraded":"2024-05-20T20:54:37.872609+0000","last_fullsized":"2024-05-20T20:54:37.872609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887790+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:38.887790+0000","last_peered":"2024-05-20T20:54:38.887790+0000","last_clean":"2024-05-20T20:54:38.887790+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:38.887790+0000","last_undegraded":"2024-05-20T20:54:38.887790+0000","last_fullsized":"2024-05-20T20:54:38.887790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872721+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:37.872721+0000","last_peered":"2024-05-20T20:54:37.872721+0000","last_clean":"2024-05-20T20:54:37.872721+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:37.872721+0000","last_undegraded":"2024-05-20T20:54:37.872721+0000","last_fullsized":"2024-05-20T20:54:37.872721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474659+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:38.474659+0000","last_peered":"2024-05-20T20:54:38.474659+0000","last_clean":"2024-05-20T20:54:38.474659+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:38.474659+0000","last_undegraded":"2024-05-20T20:54:38.474659+0000","last_fullsized":"2024-05-20T20:54:38.474659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474766+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:54:38.474766+0000","last_peered":"2024-05-20T20:54:38.474766+0000","last_clean":"2024-05-20T20:54:38.474766+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:54:38.474766+0000","last_undegraded":"2024-05-20T20:54:38.474766+0000","last_fullsized":"2024-05-20T20:54:38.474766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886666+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.886666+0000","last_peered":"2024-05-20T20:54:38.886666+0000","last_clean":"2024-05-20T20:54:38.886666+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:54:38.886666+0000","last_undegraded":"2024-05-20T20:54:38.886666+0000","last_fullsized":"2024-05-20T20:54:38.886666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887575+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:38.887575+0000","last_peered":"2024-05-20T20:54:38.887575+0000","last_clean":"2024-05-20T20:54:38.887575+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:38.887575+0000","last_undegraded":"2024-05-20T20:54:38.887575+0000","last_fullsized":"2024-05-20T20:54:38.887575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765538+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:54:38.765538+0000","last_peered":"2024-05-20T20:54:38.765538+0000","last_clean":"2024-05-20T20:54:38.765538+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:54:38.765538+0000","last_undegraded":"2024-05-20T20:54:38.765538+0000","last_fullsized":"2024-05-20T20:54:38.765538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887488+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:38.887488+0000","last_peered":"2024-05-20T20:54:38.887488+0000","last_clean":"2024-05-20T20:54:38.887488+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:38.887488+0000","last_undegraded":"2024-05-20T20:54:38.887488+0000","last_fullsized":"2024-05-20T20:54:38.887488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874582+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:37.874582+0000","last_peered":"2024-05-20T20:54:37.874582+0000","last_clean":"2024-05-20T20:54:37.874582+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:37.874582+0000","last_undegraded":"2024-05-20T20:54:37.874582+0000","last_fullsized":"2024-05-20T20:54:37.874582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":206,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:46.567588+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:55:46.567588+0000","last_peered":"2024-05-20T20:55:46.567588+0000","last_clean":"2024-05-20T20:55:46.567588+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:55:46.567588+0000","last_undegraded":"2024-05-20T20:55:46.567588+0000","last_fullsized":"2024-05-20T20:55:46.567588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887502+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:38.887502+0000","last_peered":"2024-05-20T20:54:38.887502+0000","last_clean":"2024-05-20T20:54:38.887502+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:38.887502+0000","last_undegraded":"2024-05-20T20:54:38.887502+0000","last_fullsized":"2024-05-20T20:54:38.887502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876310+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876310+0000","last_peered":"2024-05-20T20:54:37.876310+0000","last_clean":"2024-05-20T20:54:37.876310+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876310+0000","last_undegraded":"2024-05-20T20:54:37.876310+0000","last_fullsized":"2024-05-20T20:54:37.876310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874230+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:37.874230+0000","last_peered":"2024-05-20T20:54:37.874230+0000","last_clean":"2024-05-20T20:54:37.874230+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:37.874230+0000","last_undegraded":"2024-05-20T20:54:37.874230+0000","last_fullsized":"2024-05-20T20:54:37.874230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867681+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:37.867681+0000","last_peered":"2024-05-20T20:54:37.867681+0000","last_clean":"2024-05-20T20:54:37.867681+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:37.867681+0000","last_undegraded":"2024-05-20T20:54:37.867681+0000","last_fullsized":"2024-05-20T20:54:37.867681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474359+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:38.474359+0000","last_peered":"2024-05-20T20:54:38.474359+0000","last_clean":"2024-05-20T20:54:38.474359+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:38.474359+0000","last_undegraded":"2024-05-20T20:54:38.474359+0000","last_fullsized":"2024-05-20T20:54:38.474359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874492+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:37.874492+0000","last_peered":"2024-05-20T20:54:37.874492+0000","last_clean":"2024-05-20T20:54:37.874492+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:37.874492+0000","last_undegraded":"2024-05-20T20:54:37.874492+0000","last_fullsized":"2024-05-20T20:54:37.874492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872063+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872063+0000","last_peered":"2024-05-20T20:54:37.872063+0000","last_clean":"2024-05-20T20:54:37.872063+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872063+0000","last_undegraded":"2024-05-20T20:54:37.872063+0000","last_fullsized":"2024-05-20T20:54:37.872063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":203,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:48.571135+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:55:48.571135+0000","last_peered":"2024-05-20T20:55:48.571135+0000","last_clean":"2024-05-20T20:55:48.571135+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:55:48.571135+0000","last_undegraded":"2024-05-20T20:55:48.571135+0000","last_fullsized":"2024-05-20T20:55:48.571135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765842+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:54:38.765842+0000","last_peered":"2024-05-20T20:54:38.765842+0000","last_clean":"2024-05-20T20:54:38.765842+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:54:38.765842+0000","last_undegraded":"2024-05-20T20:54:38.765842+0000","last_fullsized":"2024-05-20T20:54:38.765842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871069+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871069+0000","last_peered":"2024-05-20T20:54:37.871069+0000","last_clean":"2024-05-20T20:54:37.871069+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871069+0000","last_undegraded":"2024-05-20T20:54:37.871069+0000","last_fullsized":"2024-05-20T20:54:37.871069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886717+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:38.886717+0000","last_peered":"2024-05-20T20:54:38.886717+0000","last_clean":"2024-05-20T20:54:38.886717+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:38.886717+0000","last_undegraded":"2024-05-20T20:54:38.886717+0000","last_fullsized":"2024-05-20T20:54:38.886717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874541+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:37.874541+0000","last_peered":"2024-05-20T20:54:37.874541+0000","last_clean":"2024-05-20T20:54:37.874541+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:37.874541+0000","last_undegraded":"2024-05-20T20:54:37.874541+0000","last_fullsized":"2024-05-20T20:54:37.874541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.765712+0000","last_peered":"2024-05-20T20:54:38.765712+0000","last_clean":"2024-05-20T20:54:38.765712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:54:38.765712+0000","last_undegraded":"2024-05-20T20:54:38.765712+0000","last_fullsized":"2024-05-20T20:54:38.765712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":415,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:49.572979+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:55:49.572979+0000","last_peered":"2024-05-20T20:55:49.572979+0000","last_clean":"2024-05-20T20:55:49.572979+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:55:49.572979+0000","last_undegraded":"2024-05-20T20:55:49.572979+0000","last_fullsized":"2024-05-20T20:55:49.572979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67273,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":348,"num_read_kb":239,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887265+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:38.887265+0000","last_peered":"2024-05-20T20:54:38.887265+0000","last_clean":"2024-05-20T20:54:38.887265+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:38.887265+0000","last_undegraded":"2024-05-20T20:54:38.887265+0000","last_fullsized":"2024-05-20T20:54:38.887265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887299+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:38.887299+0000","last_peered":"2024-05-20T20:54:38.887299+0000","last_clean":"2024-05-20T20:54:38.887299+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:38.887299+0000","last_undegraded":"2024-05-20T20:54:38.887299+0000","last_fullsized":"2024-05-20T20:54:38.887299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867648+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:37.867648+0000","last_peered":"2024-05-20T20:54:37.867648+0000","last_clean":"2024-05-20T20:54:37.867648+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:37.867648+0000","last_undegraded":"2024-05-20T20:54:37.867648+0000","last_fullsized":"2024-05-20T20:54:37.867648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874116+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:37.874116+0000","last_peered":"2024-05-20T20:54:37.874116+0000","last_clean":"2024-05-20T20:54:37.874116+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:37.874116+0000","last_undegraded":"2024-05-20T20:54:37.874116+0000","last_fullsized":"2024-05-20T20:54:37.874116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871059+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.871059+0000","last_peered":"2024-05-20T20:54:37.871059+0000","last_clean":"2024-05-20T20:54:37.871059+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.871059+0000","last_undegraded":"2024-05-20T20:54:37.871059+0000","last_fullsized":"2024-05-20T20:54:37.871059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":207,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.589891+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:55:45.589891+0000","last_peered":"2024-05-20T20:55:45.589891+0000","last_clean":"2024-05-20T20:55:45.589891+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:55:45.589891+0000","last_undegraded":"2024-05-20T20:55:45.589891+0000","last_fullsized":"2024-05-20T20:55:45.589891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870598+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870598+0000","last_peered":"2024-05-20T20:54:37.870598+0000","last_clean":"2024-05-20T20:54:37.870598+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870598+0000","last_undegraded":"2024-05-20T20:54:37.870598+0000","last_fullsized":"2024-05-20T20:54:37.870598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886999+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:38.886999+0000","last_peered":"2024-05-20T20:54:38.886999+0000","last_clean":"2024-05-20T20:54:38.886999+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:38.886999+0000","last_undegraded":"2024-05-20T20:54:38.886999+0000","last_fullsized":"2024-05-20T20:54:38.886999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874169+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:37.874169+0000","last_peered":"2024-05-20T20:54:37.874169+0000","last_clean":"2024-05-20T20:54:37.874169+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:37.874169+0000","last_undegraded":"2024-05-20T20:54:37.874169+0000","last_fullsized":"2024-05-20T20:54:37.874169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887338+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:38.887338+0000","last_peered":"2024-05-20T20:54:38.887338+0000","last_clean":"2024-05-20T20:54:38.887338+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:38.887338+0000","last_undegraded":"2024-05-20T20:54:38.887338+0000","last_fullsized":"2024-05-20T20:54:38.887338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870840+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:54:37.870840+0000","last_peered":"2024-05-20T20:54:37.870840+0000","last_clean":"2024-05-20T20:54:37.870840+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:54:37.870840+0000","last_undegraded":"2024-05-20T20:54:37.870840+0000","last_fullsized":"2024-05-20T20:54:37.870840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872174+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872174+0000","last_peered":"2024-05-20T20:54:37.872174+0000","last_clean":"2024-05-20T20:54:37.872174+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872174+0000","last_undegraded":"2024-05-20T20:54:37.872174+0000","last_fullsized":"2024-05-20T20:54:37.872174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876474+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876474+0000","last_peered":"2024-05-20T20:54:37.876474+0000","last_clean":"2024-05-20T20:54:37.876474+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876474+0000","last_undegraded":"2024-05-20T20:54:37.876474+0000","last_fullsized":"2024-05-20T20:54:37.876474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871372+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871372+0000","last_peered":"2024-05-20T20:54:37.871372+0000","last_clean":"2024-05-20T20:54:37.871372+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871372+0000","last_undegraded":"2024-05-20T20:54:37.871372+0000","last_fullsized":"2024-05-20T20:54:37.871372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875765+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875765+0000","last_peered":"2024-05-20T20:54:37.875765+0000","last_clean":"2024-05-20T20:54:37.875765+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875765+0000","last_undegraded":"2024-05-20T20:54:37.875765+0000","last_fullsized":"2024-05-20T20:54:37.875765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872208+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872208+0000","last_peered":"2024-05-20T20:54:37.872208+0000","last_clean":"2024-05-20T20:54:37.872208+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872208+0000","last_undegraded":"2024-05-20T20:54:37.872208+0000","last_fullsized":"2024-05-20T20:54:37.872208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":202,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.588360+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:55:45.588360+0000","last_peered":"2024-05-20T20:55:45.588360+0000","last_clean":"2024-05-20T20:55:45.588360+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:55:45.588360+0000","last_undegraded":"2024-05-20T20:55:45.588360+0000","last_fullsized":"2024-05-20T20:55:45.588360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":165,"num_read_kb":165,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474755+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:38.474755+0000","last_peered":"2024-05-20T20:54:38.474755+0000","last_clean":"2024-05-20T20:54:38.474755+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:38.474755+0000","last_undegraded":"2024-05-20T20:54:38.474755+0000","last_fullsized":"2024-05-20T20:54:38.474755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475163+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:54:38.475163+0000","last_peered":"2024-05-20T20:54:38.475163+0000","last_clean":"2024-05-20T20:54:38.475163+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:54:38.475163+0000","last_undegraded":"2024-05-20T20:54:38.475163+0000","last_fullsized":"2024-05-20T20:54:38.475163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870715+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870715+0000","last_peered":"2024-05-20T20:54:37.870715+0000","last_clean":"2024-05-20T20:54:37.870715+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870715+0000","last_undegraded":"2024-05-20T20:54:37.870715+0000","last_fullsized":"2024-05-20T20:54:37.870715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867545+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:37.867545+0000","last_peered":"2024-05-20T20:54:37.867545+0000","last_clean":"2024-05-20T20:54:37.867545+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:37.867545+0000","last_undegraded":"2024-05-20T20:54:37.867545+0000","last_fullsized":"2024-05-20T20:54:37.867545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870913+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870913+0000","last_peered":"2024-05-20T20:54:37.870913+0000","last_clean":"2024-05-20T20:54:37.870913+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870913+0000","last_undegraded":"2024-05-20T20:54:37.870913+0000","last_fullsized":"2024-05-20T20:54:37.870913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887395+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:38.887395+0000","last_peered":"2024-05-20T20:54:38.887395+0000","last_clean":"2024-05-20T20:54:38.887395+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:38.887395+0000","last_undegraded":"2024-05-20T20:54:38.887395+0000","last_fullsized":"2024-05-20T20:54:38.887395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867898+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:37.867898+0000","last_peered":"2024-05-20T20:54:37.867898+0000","last_clean":"2024-05-20T20:54:37.867898+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:37.867898+0000","last_undegraded":"2024-05-20T20:54:37.867898+0000","last_fullsized":"2024-05-20T20:54:37.867898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.261304+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:55:45.261304+0000","last_peered":"2024-05-20T20:55:45.261304+0000","last_clean":"2024-05-20T20:55:45.261304+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:55:45.261304+0000","last_undegraded":"2024-05-20T20:55:45.261304+0000","last_fullsized":"2024-05-20T20:55:45.261304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474726+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:38.474726+0000","last_peered":"2024-05-20T20:54:38.474726+0000","last_clean":"2024-05-20T20:54:38.474726+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:38.474726+0000","last_undegraded":"2024-05-20T20:54:38.474726+0000","last_fullsized":"2024-05-20T20:54:38.474726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475338+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:54:38.475338+0000","last_peered":"2024-05-20T20:54:38.475338+0000","last_clean":"2024-05-20T20:54:38.475338+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:54:38.475338+0000","last_undegraded":"2024-05-20T20:54:38.475338+0000","last_fullsized":"2024-05-20T20:54:38.475338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886677+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:38.886677+0000","last_peered":"2024-05-20T20:54:38.886677+0000","last_clean":"2024-05-20T20:54:38.886677+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:38.886677+0000","last_undegraded":"2024-05-20T20:54:38.886677+0000","last_fullsized":"2024-05-20T20:54:38.886677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765487+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:54:38.765487+0000","last_peered":"2024-05-20T20:54:38.765487+0000","last_clean":"2024-05-20T20:54:38.765487+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:54:38.765487+0000","last_undegraded":"2024-05-20T20:54:38.765487+0000","last_fullsized":"2024-05-20T20:54:38.765487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887085+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:38.887085+0000","last_peered":"2024-05-20T20:54:38.887085+0000","last_clean":"2024-05-20T20:54:38.887085+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:38.887085+0000","last_undegraded":"2024-05-20T20:54:38.887085+0000","last_fullsized":"2024-05-20T20:54:38.887085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474392+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:54:38.474392+0000","last_peered":"2024-05-20T20:54:38.474392+0000","last_clean":"2024-05-20T20:54:38.474392+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:54:38.474392+0000","last_undegraded":"2024-05-20T20:54:38.474392+0000","last_fullsized":"2024-05-20T20:54:38.474392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870487+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:54:37.870487+0000","last_peered":"2024-05-20T20:54:37.870487+0000","last_clean":"2024-05-20T20:54:37.870487+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:54:37.870487+0000","last_undegraded":"2024-05-20T20:54:37.870487+0000","last_fullsized":"2024-05-20T20:54:37.870487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870477+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:54:37.870477+0000","last_peered":"2024-05-20T20:54:37.870477+0000","last_clean":"2024-05-20T20:54:37.870477+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:54:37.870477+0000","last_undegraded":"2024-05-20T20:54:37.870477+0000","last_fullsized":"2024-05-20T20:54:37.870477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474638+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:38.474638+0000","last_peered":"2024-05-20T20:54:38.474638+0000","last_clean":"2024-05-20T20:54:38.474638+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:38.474638+0000","last_undegraded":"2024-05-20T20:54:38.474638+0000","last_fullsized":"2024-05-20T20:54:38.474638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870497+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:54:37.870497+0000","last_peered":"2024-05-20T20:54:37.870497+0000","last_clean":"2024-05-20T20:54:37.870497+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:54:37.870497+0000","last_undegraded":"2024-05-20T20:54:37.870497+0000","last_fullsized":"2024-05-20T20:54:37.870497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.880369+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.880369+0000","last_peered":"2024-05-20T20:54:37.880369+0000","last_clean":"2024-05-20T20:54:37.880369+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.880369+0000","last_undegraded":"2024-05-20T20:54:37.880369+0000","last_fullsized":"2024-05-20T20:54:37.880369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887561+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:38.887561+0000","last_peered":"2024-05-20T20:54:38.887561+0000","last_clean":"2024-05-20T20:54:38.887561+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:38.887561+0000","last_undegraded":"2024-05-20T20:54:38.887561+0000","last_fullsized":"2024-05-20T20:54:38.887561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871302+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871302+0000","last_peered":"2024-05-20T20:54:37.871302+0000","last_clean":"2024-05-20T20:54:37.871302+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871302+0000","last_undegraded":"2024-05-20T20:54:37.871302+0000","last_fullsized":"2024-05-20T20:54:37.871302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874356+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:37.874356+0000","last_peered":"2024-05-20T20:54:37.874356+0000","last_clean":"2024-05-20T20:54:37.874356+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:37.874356+0000","last_undegraded":"2024-05-20T20:54:37.874356+0000","last_fullsized":"2024-05-20T20:54:37.874356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870975+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870975+0000","last_peered":"2024-05-20T20:54:37.870975+0000","last_clean":"2024-05-20T20:54:37.870975+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870975+0000","last_undegraded":"2024-05-20T20:54:37.870975+0000","last_fullsized":"2024-05-20T20:54:37.870975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867985+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:37.867985+0000","last_peered":"2024-05-20T20:54:37.867985+0000","last_clean":"2024-05-20T20:54:37.867985+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:37.867985+0000","last_undegraded":"2024-05-20T20:54:37.867985+0000","last_fullsized":"2024-05-20T20:54:37.867985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871280+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871280+0000","last_peered":"2024-05-20T20:54:37.871280+0000","last_clean":"2024-05-20T20:54:37.871280+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871280+0000","last_undegraded":"2024-05-20T20:54:37.871280+0000","last_fullsized":"2024-05-20T20:54:37.871280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870627+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:54:37.870627+0000","last_peered":"2024-05-20T20:54:37.870627+0000","last_clean":"2024-05-20T20:54:37.870627+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:54:37.870627+0000","last_undegraded":"2024-05-20T20:54:37.870627+0000","last_fullsized":"2024-05-20T20:54:37.870627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887334+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:38.887334+0000","last_peered":"2024-05-20T20:54:38.887334+0000","last_clean":"2024-05-20T20:54:38.887334+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:38.887334+0000","last_undegraded":"2024-05-20T20:54:38.887334+0000","last_fullsized":"2024-05-20T20:54:38.887334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887439+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:38.887439+0000","last_peered":"2024-05-20T20:54:38.887439+0000","last_clean":"2024-05-20T20:54:38.887439+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:38.887439+0000","last_undegraded":"2024-05-20T20:54:38.887439+0000","last_fullsized":"2024-05-20T20:54:38.887439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886573+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:38.886573+0000","last_peered":"2024-05-20T20:54:38.886573+0000","last_clean":"2024-05-20T20:54:38.886573+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:38.886573+0000","last_undegraded":"2024-05-20T20:54:38.886573+0000","last_fullsized":"2024-05-20T20:54:38.886573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867747+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:37.867747+0000","last_peered":"2024-05-20T20:54:37.867747+0000","last_clean":"2024-05-20T20:54:37.867747+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:37.867747+0000","last_undegraded":"2024-05-20T20:54:37.867747+0000","last_fullsized":"2024-05-20T20:54:37.867747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870785+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:54:37.870785+0000","last_peered":"2024-05-20T20:54:37.870785+0000","last_clean":"2024-05-20T20:54:37.870785+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:54:37.870785+0000","last_undegraded":"2024-05-20T20:54:37.870785+0000","last_fullsized":"2024-05-20T20:54:37.870785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887850+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:38.887850+0000","last_peered":"2024-05-20T20:54:38.887850+0000","last_clean":"2024-05-20T20:54:38.887850+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:38.887850+0000","last_undegraded":"2024-05-20T20:54:38.887850+0000","last_fullsized":"2024-05-20T20:54:38.887850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474618+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:38.474618+0000","last_peered":"2024-05-20T20:54:38.474618+0000","last_clean":"2024-05-20T20:54:38.474618+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:38.474618+0000","last_undegraded":"2024-05-20T20:54:38.474618+0000","last_fullsized":"2024-05-20T20:54:38.474618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874544+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:37.874544+0000","last_peered":"2024-05-20T20:54:37.874544+0000","last_clean":"2024-05-20T20:54:37.874544+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:37.874544+0000","last_undegraded":"2024-05-20T20:54:37.874544+0000","last_fullsized":"2024-05-20T20:54:37.874544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474329+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:38.474329+0000","last_peered":"2024-05-20T20:54:38.474329+0000","last_clean":"2024-05-20T20:54:38.474329+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:38.474329+0000","last_undegraded":"2024-05-20T20:54:38.474329+0000","last_fullsized":"2024-05-20T20:54:38.474329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766006+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:54:38.766006+0000","last_peered":"2024-05-20T20:54:38.766006+0000","last_clean":"2024-05-20T20:54:38.766006+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:54:38.766006+0000","last_undegraded":"2024-05-20T20:54:38.766006+0000","last_fullsized":"2024-05-20T20:54:38.766006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867894+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:37.867894+0000","last_peered":"2024-05-20T20:54:37.867894+0000","last_clean":"2024-05-20T20:54:37.867894+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:37.867894+0000","last_undegraded":"2024-05-20T20:54:37.867894+0000","last_fullsized":"2024-05-20T20:54:37.867894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870651+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:54:37.870651+0000","last_peered":"2024-05-20T20:54:37.870651+0000","last_clean":"2024-05-20T20:54:37.870651+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:54:37.870651+0000","last_undegraded":"2024-05-20T20:54:37.870651+0000","last_fullsized":"2024-05-20T20:54:37.870651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72792,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1032,"num_read_kb":593,"num_write":183,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":442424,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1102203,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442580,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1036028,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":442049,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":504410,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":443320,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":511237,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":442360,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":443623,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822739,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1034687,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:55:51.640 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-20T20:55:51.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T20:55:51.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3690848952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4103540 msgr2=0x7f6cb4109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3690848952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4103540 0x7f6cb4109dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009ca0 tx=0x7f6ca402a8c0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3690848952 shutdown_connections 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3690848952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4103540 0x7f6cb4109dd0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3690848952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3690848952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 0x7f6cb4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3690848952 >> 172.21.15.26:0/3690848952 conn(0x7f6cb40fcf00 msgr2=0x7f6cb40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3690848952 shutdown_connections 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.791+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3690848952 wait complete. 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 Processor -- start 2024-05-20T20:55:51.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 -- start start 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 0x7f6cb4105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4103540 0x7f6cb41044a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410d3c0 con 0x7f6cb4102b80 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410d240 con 0x7f6cb4101990 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb8ea0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410d540 con 0x7f6cb4103540 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56382/0 (socket says 172.21.15.26:56382) 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 -- 172.21.15.26:0/3718520432 learned_addr learned my addr 172.21.15.26:0/3718520432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb2d9d640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4103540 0x7f6cb41044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb259c640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 0x7f6cb4105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 -- 172.21.15.26:0/3718520432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4103540 msgr2=0x7f6cb41044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4103540 0x7f6cb41044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 -- 172.21.15.26:0/3718520432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 msgr2=0x7f6cb4105920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 0x7f6cb4105920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 -- 172.21.15.26:0/3718520432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb4104d30 con 0x7f6cb4102b80 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb2d9d640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4103540 0x7f6cb41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb259c640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 0x7f6cb4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.792+0000 7f6cb1d9b640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103f60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00d770 tx=0x7f6c9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.793+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c017070 con 0x7f6cb4102b80 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.793+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c9c0054b0 con 0x7f6cb4102b80 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.793+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41a2d30 con 0x7f6cb4102b80 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.793+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c004c60 con 0x7f6cb4102b80 2024-05-20T20:55:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.793+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a31d0 con 0x7f6cb4102b80 2024-05-20T20:55:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.794+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb41a3400 con 0x7f6cb4102b80 2024-05-20T20:55:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.795+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f6c9c004e00 con 0x7f6cb4102b80 2024-05-20T20:55:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.796+0000 7f6cab7fe640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ca00b2880 0x7f6ca00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.796+0000 7f6cb259c640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ca00b2880 0x7f6ca00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.796+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f6c9c0e22d0 con 0x7f6cb4102b80 2024-05-20T20:55:51.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.796+0000 7f6cb259c640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ca00b2880 0x7f6ca00b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0058c0 tx=0x7f6cac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:51.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.799+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c9c0a9e30 con 0x7f6cb4102b80 2024-05-20T20:55:51.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.991+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6cb40ff140 con 0x7f6ca00b2880 2024-05-20T20:55:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:51.998+0000 7f6cab7fe640 1 -- 172.21.15.26:0/3718520432 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259085 (secure 0 0 0) 0x7f6cb40ff140 con 0x7f6ca00b2880 2024-05-20T20:55:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ca00b2880 msgr2=0x7f6ca00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ca00b2880 0x7f6ca00b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0058c0 tx=0x7f6cac005850 comp rx=0 tx=0).stop 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 msgr2=0x7f6cb4103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103f60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00d770 tx=0x7f6c9c00dc30 comp rx=0 tx=0).stop 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 shutdown_connections 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ca00b2880 0x7f6ca00b4d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb4103540 0x7f6cb41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb4102b80 0x7f6cb4103f60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 --2- 172.21.15.26:0/3718520432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cb4101990 0x7f6cb4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 >> 172.21.15.26:0/3718520432 conn(0x7f6cb40fcf00 msgr2=0x7f6cb40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 shutdown_connections 2024-05-20T20:55:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:52.002+0000 7f6cb8ea0640 1 -- 172.21.15.26:0/3718520432 wait complete. 2024-05-20T20:55:52.013 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-05-20T20:55:51.741511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663394,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1108,"num_read_kb":657,"num_write":310,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355984,"kb_used_data":10968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585840,"statfs":{"total":1151940427776,"available":1151575900160,"internally_reserved":0,"allocated":11231232,"data_stored":7292286,"data_compressed":249159,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6575,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":39,"num_write":15,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002903"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870824+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:54:37.870824+0000","last_peered":"2024-05-20T20:54:37.870824+0000","last_clean":"2024-05-20T20:54:37.870824+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:54:37.870824+0000","last_undegraded":"2024-05-20T20:54:37.870824+0000","last_fullsized":"2024-05-20T20:54:37.870824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886671+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:38.886671+0000","last_peered":"2024-05-20T20:54:38.886671+0000","last_clean":"2024-05-20T20:54:38.886671+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:38.886671+0000","last_undegraded":"2024-05-20T20:54:38.886671+0000","last_fullsized":"2024-05-20T20:54:38.886671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886528+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:54:38.886528+0000","last_peered":"2024-05-20T20:54:38.886528+0000","last_clean":"2024-05-20T20:54:38.886528+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:54:38.886528+0000","last_undegraded":"2024-05-20T20:54:38.886528+0000","last_fullsized":"2024-05-20T20:54:38.886528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475208+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:38.475208+0000","last_peered":"2024-05-20T20:54:38.475208+0000","last_clean":"2024-05-20T20:54:38.475208+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:38.475208+0000","last_undegraded":"2024-05-20T20:54:38.475208+0000","last_fullsized":"2024-05-20T20:54:38.475208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887218+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:38.887218+0000","last_peered":"2024-05-20T20:54:38.887218+0000","last_clean":"2024-05-20T20:54:38.887218+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:38.887218+0000","last_undegraded":"2024-05-20T20:54:38.887218+0000","last_fullsized":"2024-05-20T20:54:38.887218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876469+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876469+0000","last_peered":"2024-05-20T20:54:37.876469+0000","last_clean":"2024-05-20T20:54:37.876469+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876469+0000","last_undegraded":"2024-05-20T20:54:37.876469+0000","last_fullsized":"2024-05-20T20:54:37.876469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870690+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870690+0000","last_peered":"2024-05-20T20:54:37.870690+0000","last_clean":"2024-05-20T20:54:37.870690+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870690+0000","last_undegraded":"2024-05-20T20:54:37.870690+0000","last_fullsized":"2024-05-20T20:54:37.870690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766030+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:54:38.766030+0000","last_peered":"2024-05-20T20:54:38.766030+0000","last_clean":"2024-05-20T20:54:38.766030+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:54:38.766030+0000","last_undegraded":"2024-05-20T20:54:38.766030+0000","last_fullsized":"2024-05-20T20:54:38.766030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876712+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876712+0000","last_peered":"2024-05-20T20:54:37.876712+0000","last_clean":"2024-05-20T20:54:37.876712+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876712+0000","last_undegraded":"2024-05-20T20:54:37.876712+0000","last_fullsized":"2024-05-20T20:54:37.876712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871157+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871157+0000","last_peered":"2024-05-20T20:54:37.871157+0000","last_clean":"2024-05-20T20:54:37.871157+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871157+0000","last_undegraded":"2024-05-20T20:54:37.871157+0000","last_fullsized":"2024-05-20T20:54:37.871157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475180+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:38.475180+0000","last_peered":"2024-05-20T20:54:38.475180+0000","last_clean":"2024-05-20T20:54:38.475180+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:38.475180+0000","last_undegraded":"2024-05-20T20:54:38.475180+0000","last_fullsized":"2024-05-20T20:54:38.475180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867869+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:37.867869+0000","last_peered":"2024-05-20T20:54:37.867869+0000","last_clean":"2024-05-20T20:54:37.867869+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:37.867869+0000","last_undegraded":"2024-05-20T20:54:37.867869+0000","last_fullsized":"2024-05-20T20:54:37.867869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886436+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:38.886436+0000","last_peered":"2024-05-20T20:54:38.886436+0000","last_clean":"2024-05-20T20:54:38.886436+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:38.886436+0000","last_undegraded":"2024-05-20T20:54:38.886436+0000","last_fullsized":"2024-05-20T20:54:38.886436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886364+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:38.886364+0000","last_peered":"2024-05-20T20:54:38.886364+0000","last_clean":"2024-05-20T20:54:38.886364+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:38.886364+0000","last_undegraded":"2024-05-20T20:54:38.886364+0000","last_fullsized":"2024-05-20T20:54:38.886364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765447+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:54:38.765447+0000","last_peered":"2024-05-20T20:54:38.765447+0000","last_clean":"2024-05-20T20:54:38.765447+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:54:38.765447+0000","last_undegraded":"2024-05-20T20:54:38.765447+0000","last_fullsized":"2024-05-20T20:54:38.765447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870916+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:54:37.870916+0000","last_peered":"2024-05-20T20:54:37.870916+0000","last_clean":"2024-05-20T20:54:37.870916+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:54:37.870916+0000","last_undegraded":"2024-05-20T20:54:37.870916+0000","last_fullsized":"2024-05-20T20:54:37.870916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872609+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872609+0000","last_peered":"2024-05-20T20:54:37.872609+0000","last_clean":"2024-05-20T20:54:37.872609+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872609+0000","last_undegraded":"2024-05-20T20:54:37.872609+0000","last_fullsized":"2024-05-20T20:54:37.872609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887790+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:38.887790+0000","last_peered":"2024-05-20T20:54:38.887790+0000","last_clean":"2024-05-20T20:54:38.887790+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:38.887790+0000","last_undegraded":"2024-05-20T20:54:38.887790+0000","last_fullsized":"2024-05-20T20:54:38.887790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872721+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:37.872721+0000","last_peered":"2024-05-20T20:54:37.872721+0000","last_clean":"2024-05-20T20:54:37.872721+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:37.872721+0000","last_undegraded":"2024-05-20T20:54:37.872721+0000","last_fullsized":"2024-05-20T20:54:37.872721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474659+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:38.474659+0000","last_peered":"2024-05-20T20:54:38.474659+0000","last_clean":"2024-05-20T20:54:38.474659+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:38.474659+0000","last_undegraded":"2024-05-20T20:54:38.474659+0000","last_fullsized":"2024-05-20T20:54:38.474659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474766+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:54:38.474766+0000","last_peered":"2024-05-20T20:54:38.474766+0000","last_clean":"2024-05-20T20:54:38.474766+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:54:38.474766+0000","last_undegraded":"2024-05-20T20:54:38.474766+0000","last_fullsized":"2024-05-20T20:54:38.474766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886666+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.886666+0000","last_peered":"2024-05-20T20:54:38.886666+0000","last_clean":"2024-05-20T20:54:38.886666+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:54:38.886666+0000","last_undegraded":"2024-05-20T20:54:38.886666+0000","last_fullsized":"2024-05-20T20:54:38.886666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887575+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:38.887575+0000","last_peered":"2024-05-20T20:54:38.887575+0000","last_clean":"2024-05-20T20:54:38.887575+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:38.887575+0000","last_undegraded":"2024-05-20T20:54:38.887575+0000","last_fullsized":"2024-05-20T20:54:38.887575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765538+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:54:38.765538+0000","last_peered":"2024-05-20T20:54:38.765538+0000","last_clean":"2024-05-20T20:54:38.765538+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:54:38.765538+0000","last_undegraded":"2024-05-20T20:54:38.765538+0000","last_fullsized":"2024-05-20T20:54:38.765538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887488+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:38.887488+0000","last_peered":"2024-05-20T20:54:38.887488+0000","last_clean":"2024-05-20T20:54:38.887488+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:38.887488+0000","last_undegraded":"2024-05-20T20:54:38.887488+0000","last_fullsized":"2024-05-20T20:54:38.887488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874582+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:37.874582+0000","last_peered":"2024-05-20T20:54:37.874582+0000","last_clean":"2024-05-20T20:54:37.874582+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:37.874582+0000","last_undegraded":"2024-05-20T20:54:37.874582+0000","last_fullsized":"2024-05-20T20:54:37.874582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":206,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:46.567588+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:55:46.567588+0000","last_peered":"2024-05-20T20:55:46.567588+0000","last_clean":"2024-05-20T20:55:46.567588+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:55:46.567588+0000","last_undegraded":"2024-05-20T20:55:46.567588+0000","last_fullsized":"2024-05-20T20:55:46.567588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887502+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:38.887502+0000","last_peered":"2024-05-20T20:54:38.887502+0000","last_clean":"2024-05-20T20:54:38.887502+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:38.887502+0000","last_undegraded":"2024-05-20T20:54:38.887502+0000","last_fullsized":"2024-05-20T20:54:38.887502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876310+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876310+0000","last_peered":"2024-05-20T20:54:37.876310+0000","last_clean":"2024-05-20T20:54:37.876310+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876310+0000","last_undegraded":"2024-05-20T20:54:37.876310+0000","last_fullsized":"2024-05-20T20:54:37.876310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874230+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:37.874230+0000","last_peered":"2024-05-20T20:54:37.874230+0000","last_clean":"2024-05-20T20:54:37.874230+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:37.874230+0000","last_undegraded":"2024-05-20T20:54:37.874230+0000","last_fullsized":"2024-05-20T20:54:37.874230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867681+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:37.867681+0000","last_peered":"2024-05-20T20:54:37.867681+0000","last_clean":"2024-05-20T20:54:37.867681+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:37.867681+0000","last_undegraded":"2024-05-20T20:54:37.867681+0000","last_fullsized":"2024-05-20T20:54:37.867681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474359+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:38.474359+0000","last_peered":"2024-05-20T20:54:38.474359+0000","last_clean":"2024-05-20T20:54:38.474359+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:38.474359+0000","last_undegraded":"2024-05-20T20:54:38.474359+0000","last_fullsized":"2024-05-20T20:54:38.474359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874492+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:37.874492+0000","last_peered":"2024-05-20T20:54:37.874492+0000","last_clean":"2024-05-20T20:54:37.874492+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:37.874492+0000","last_undegraded":"2024-05-20T20:54:37.874492+0000","last_fullsized":"2024-05-20T20:54:37.874492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872063+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872063+0000","last_peered":"2024-05-20T20:54:37.872063+0000","last_clean":"2024-05-20T20:54:37.872063+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872063+0000","last_undegraded":"2024-05-20T20:54:37.872063+0000","last_fullsized":"2024-05-20T20:54:37.872063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":203,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:48.571135+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:55:48.571135+0000","last_peered":"2024-05-20T20:55:48.571135+0000","last_clean":"2024-05-20T20:55:48.571135+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:55:48.571135+0000","last_undegraded":"2024-05-20T20:55:48.571135+0000","last_fullsized":"2024-05-20T20:55:48.571135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765842+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:54:38.765842+0000","last_peered":"2024-05-20T20:54:38.765842+0000","last_clean":"2024-05-20T20:54:38.765842+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:54:38.765842+0000","last_undegraded":"2024-05-20T20:54:38.765842+0000","last_fullsized":"2024-05-20T20:54:38.765842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871069+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871069+0000","last_peered":"2024-05-20T20:54:37.871069+0000","last_clean":"2024-05-20T20:54:37.871069+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871069+0000","last_undegraded":"2024-05-20T20:54:37.871069+0000","last_fullsized":"2024-05-20T20:54:37.871069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886717+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:38.886717+0000","last_peered":"2024-05-20T20:54:38.886717+0000","last_clean":"2024-05-20T20:54:38.886717+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:38.886717+0000","last_undegraded":"2024-05-20T20:54:38.886717+0000","last_fullsized":"2024-05-20T20:54:38.886717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874541+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:37.874541+0000","last_peered":"2024-05-20T20:54:37.874541+0000","last_clean":"2024-05-20T20:54:37.874541+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:37.874541+0000","last_undegraded":"2024-05-20T20:54:37.874541+0000","last_fullsized":"2024-05-20T20:54:37.874541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.765712+0000","last_peered":"2024-05-20T20:54:38.765712+0000","last_clean":"2024-05-20T20:54:38.765712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:54:38.765712+0000","last_undegraded":"2024-05-20T20:54:38.765712+0000","last_fullsized":"2024-05-20T20:54:38.765712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":415,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:49.572979+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:55:49.572979+0000","last_peered":"2024-05-20T20:55:49.572979+0000","last_clean":"2024-05-20T20:55:49.572979+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:55:49.572979+0000","last_undegraded":"2024-05-20T20:55:49.572979+0000","last_fullsized":"2024-05-20T20:55:49.572979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67273,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":348,"num_read_kb":239,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887265+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:38.887265+0000","last_peered":"2024-05-20T20:54:38.887265+0000","last_clean":"2024-05-20T20:54:38.887265+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:38.887265+0000","last_undegraded":"2024-05-20T20:54:38.887265+0000","last_fullsized":"2024-05-20T20:54:38.887265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887299+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:38.887299+0000","last_peered":"2024-05-20T20:54:38.887299+0000","last_clean":"2024-05-20T20:54:38.887299+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:38.887299+0000","last_undegraded":"2024-05-20T20:54:38.887299+0000","last_fullsized":"2024-05-20T20:54:38.887299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867648+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:37.867648+0000","last_peered":"2024-05-20T20:54:37.867648+0000","last_clean":"2024-05-20T20:54:37.867648+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:37.867648+0000","last_undegraded":"2024-05-20T20:54:37.867648+0000","last_fullsized":"2024-05-20T20:54:37.867648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874116+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:37.874116+0000","last_peered":"2024-05-20T20:54:37.874116+0000","last_clean":"2024-05-20T20:54:37.874116+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:37.874116+0000","last_undegraded":"2024-05-20T20:54:37.874116+0000","last_fullsized":"2024-05-20T20:54:37.874116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871059+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.871059+0000","last_peered":"2024-05-20T20:54:37.871059+0000","last_clean":"2024-05-20T20:54:37.871059+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.871059+0000","last_undegraded":"2024-05-20T20:54:37.871059+0000","last_fullsized":"2024-05-20T20:54:37.871059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":207,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.589891+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:55:45.589891+0000","last_peered":"2024-05-20T20:55:45.589891+0000","last_clean":"2024-05-20T20:55:45.589891+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:55:45.589891+0000","last_undegraded":"2024-05-20T20:55:45.589891+0000","last_fullsized":"2024-05-20T20:55:45.589891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870598+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870598+0000","last_peered":"2024-05-20T20:54:37.870598+0000","last_clean":"2024-05-20T20:54:37.870598+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870598+0000","last_undegraded":"2024-05-20T20:54:37.870598+0000","last_fullsized":"2024-05-20T20:54:37.870598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886999+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:38.886999+0000","last_peered":"2024-05-20T20:54:38.886999+0000","last_clean":"2024-05-20T20:54:38.886999+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:38.886999+0000","last_undegraded":"2024-05-20T20:54:38.886999+0000","last_fullsized":"2024-05-20T20:54:38.886999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874169+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:37.874169+0000","last_peered":"2024-05-20T20:54:37.874169+0000","last_clean":"2024-05-20T20:54:37.874169+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:37.874169+0000","last_undegraded":"2024-05-20T20:54:37.874169+0000","last_fullsized":"2024-05-20T20:54:37.874169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887338+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:38.887338+0000","last_peered":"2024-05-20T20:54:38.887338+0000","last_clean":"2024-05-20T20:54:38.887338+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:38.887338+0000","last_undegraded":"2024-05-20T20:54:38.887338+0000","last_fullsized":"2024-05-20T20:54:38.887338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870840+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:54:37.870840+0000","last_peered":"2024-05-20T20:54:37.870840+0000","last_clean":"2024-05-20T20:54:37.870840+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:54:37.870840+0000","last_undegraded":"2024-05-20T20:54:37.870840+0000","last_fullsized":"2024-05-20T20:54:37.870840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872174+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872174+0000","last_peered":"2024-05-20T20:54:37.872174+0000","last_clean":"2024-05-20T20:54:37.872174+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872174+0000","last_undegraded":"2024-05-20T20:54:37.872174+0000","last_fullsized":"2024-05-20T20:54:37.872174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876474+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876474+0000","last_peered":"2024-05-20T20:54:37.876474+0000","last_clean":"2024-05-20T20:54:37.876474+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876474+0000","last_undegraded":"2024-05-20T20:54:37.876474+0000","last_fullsized":"2024-05-20T20:54:37.876474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871372+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871372+0000","last_peered":"2024-05-20T20:54:37.871372+0000","last_clean":"2024-05-20T20:54:37.871372+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871372+0000","last_undegraded":"2024-05-20T20:54:37.871372+0000","last_fullsized":"2024-05-20T20:54:37.871372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875765+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875765+0000","last_peered":"2024-05-20T20:54:37.875765+0000","last_clean":"2024-05-20T20:54:37.875765+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875765+0000","last_undegraded":"2024-05-20T20:54:37.875765+0000","last_fullsized":"2024-05-20T20:54:37.875765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872208+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872208+0000","last_peered":"2024-05-20T20:54:37.872208+0000","last_clean":"2024-05-20T20:54:37.872208+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872208+0000","last_undegraded":"2024-05-20T20:54:37.872208+0000","last_fullsized":"2024-05-20T20:54:37.872208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":202,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.588360+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:55:45.588360+0000","last_peered":"2024-05-20T20:55:45.588360+0000","last_clean":"2024-05-20T20:55:45.588360+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:55:45.588360+0000","last_undegraded":"2024-05-20T20:55:45.588360+0000","last_fullsized":"2024-05-20T20:55:45.588360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":165,"num_read_kb":165,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474755+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:38.474755+0000","last_peered":"2024-05-20T20:54:38.474755+0000","last_clean":"2024-05-20T20:54:38.474755+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:38.474755+0000","last_undegraded":"2024-05-20T20:54:38.474755+0000","last_fullsized":"2024-05-20T20:54:38.474755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475163+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:54:38.475163+0000","last_peered":"2024-05-20T20:54:38.475163+0000","last_clean":"2024-05-20T20:54:38.475163+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:54:38.475163+0000","last_undegraded":"2024-05-20T20:54:38.475163+0000","last_fullsized":"2024-05-20T20:54:38.475163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870715+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870715+0000","last_peered":"2024-05-20T20:54:37.870715+0000","last_clean":"2024-05-20T20:54:37.870715+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870715+0000","last_undegraded":"2024-05-20T20:54:37.870715+0000","last_fullsized":"2024-05-20T20:54:37.870715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867545+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:37.867545+0000","last_peered":"2024-05-20T20:54:37.867545+0000","last_clean":"2024-05-20T20:54:37.867545+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:37.867545+0000","last_undegraded":"2024-05-20T20:54:37.867545+0000","last_fullsized":"2024-05-20T20:54:37.867545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870913+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870913+0000","last_peered":"2024-05-20T20:54:37.870913+0000","last_clean":"2024-05-20T20:54:37.870913+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870913+0000","last_undegraded":"2024-05-20T20:54:37.870913+0000","last_fullsized":"2024-05-20T20:54:37.870913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887395+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:38.887395+0000","last_peered":"2024-05-20T20:54:38.887395+0000","last_clean":"2024-05-20T20:54:38.887395+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:38.887395+0000","last_undegraded":"2024-05-20T20:54:38.887395+0000","last_fullsized":"2024-05-20T20:54:38.887395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867898+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:37.867898+0000","last_peered":"2024-05-20T20:54:37.867898+0000","last_clean":"2024-05-20T20:54:37.867898+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:37.867898+0000","last_undegraded":"2024-05-20T20:54:37.867898+0000","last_fullsized":"2024-05-20T20:54:37.867898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.261304+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:55:45.261304+0000","last_peered":"2024-05-20T20:55:45.261304+0000","last_clean":"2024-05-20T20:55:45.261304+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:55:45.261304+0000","last_undegraded":"2024-05-20T20:55:45.261304+0000","last_fullsized":"2024-05-20T20:55:45.261304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474726+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:38.474726+0000","last_peered":"2024-05-20T20:54:38.474726+0000","last_clean":"2024-05-20T20:54:38.474726+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:38.474726+0000","last_undegraded":"2024-05-20T20:54:38.474726+0000","last_fullsized":"2024-05-20T20:54:38.474726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475338+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:54:38.475338+0000","last_peered":"2024-05-20T20:54:38.475338+0000","last_clean":"2024-05-20T20:54:38.475338+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:54:38.475338+0000","last_undegraded":"2024-05-20T20:54:38.475338+0000","last_fullsized":"2024-05-20T20:54:38.475338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886677+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:38.886677+0000","last_peered":"2024-05-20T20:54:38.886677+0000","last_clean":"2024-05-20T20:54:38.886677+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:38.886677+0000","last_undegraded":"2024-05-20T20:54:38.886677+0000","last_fullsized":"2024-05-20T20:54:38.886677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765487+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:54:38.765487+0000","last_peered":"2024-05-20T20:54:38.765487+0000","last_clean":"2024-05-20T20:54:38.765487+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:54:38.765487+0000","last_undegraded":"2024-05-20T20:54:38.765487+0000","last_fullsized":"2024-05-20T20:54:38.765487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887085+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:38.887085+0000","last_peered":"2024-05-20T20:54:38.887085+0000","last_clean":"2024-05-20T20:54:38.887085+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:38.887085+0000","last_undegraded":"2024-05-20T20:54:38.887085+0000","last_fullsized":"2024-05-20T20:54:38.887085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474392+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:54:38.474392+0000","last_peered":"2024-05-20T20:54:38.474392+0000","last_clean":"2024-05-20T20:54:38.474392+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:54:38.474392+0000","last_undegraded":"2024-05-20T20:54:38.474392+0000","last_fullsized":"2024-05-20T20:54:38.474392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870487+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:54:37.870487+0000","last_peered":"2024-05-20T20:54:37.870487+0000","last_clean":"2024-05-20T20:54:37.870487+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:54:37.870487+0000","last_undegraded":"2024-05-20T20:54:37.870487+0000","last_fullsized":"2024-05-20T20:54:37.870487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.325040+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:55:45.325040+0000","last_peered":"2024-05-20T20:55:45.325040+0000","last_clean":"2024-05-20T20:55:45.325040+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:55:45.325040+0000","last_undegraded":"2024-05-20T20:55:45.325040+0000","last_fullsized":"2024-05-20T20:55:45.325040+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474638+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:38.474638+0000","last_peered":"2024-05-20T20:54:38.474638+0000","last_clean":"2024-05-20T20:54:38.474638+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:38.474638+0000","last_undegraded":"2024-05-20T20:54:38.474638+0000","last_fullsized":"2024-05-20T20:54:38.474638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870497+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:54:37.870497+0000","last_peered":"2024-05-20T20:54:37.870497+0000","last_clean":"2024-05-20T20:54:37.870497+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:54:37.870497+0000","last_undegraded":"2024-05-20T20:54:37.870497+0000","last_fullsized":"2024-05-20T20:54:37.870497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.880369+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.880369+0000","last_peered":"2024-05-20T20:54:37.880369+0000","last_clean":"2024-05-20T20:54:37.880369+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.880369+0000","last_undegraded":"2024-05-20T20:54:37.880369+0000","last_fullsized":"2024-05-20T20:54:37.880369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887561+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:38.887561+0000","last_peered":"2024-05-20T20:54:38.887561+0000","last_clean":"2024-05-20T20:54:38.887561+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:38.887561+0000","last_undegraded":"2024-05-20T20:54:38.887561+0000","last_fullsized":"2024-05-20T20:54:38.887561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871302+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871302+0000","last_peered":"2024-05-20T20:54:37.871302+0000","last_clean":"2024-05-20T20:54:37.871302+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871302+0000","last_undegraded":"2024-05-20T20:54:37.871302+0000","last_fullsized":"2024-05-20T20:54:37.871302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874356+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:37.874356+0000","last_peered":"2024-05-20T20:54:37.874356+0000","last_clean":"2024-05-20T20:54:37.874356+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:37.874356+0000","last_undegraded":"2024-05-20T20:54:37.874356+0000","last_fullsized":"2024-05-20T20:54:37.874356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870975+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870975+0000","last_peered":"2024-05-20T20:54:37.870975+0000","last_clean":"2024-05-20T20:54:37.870975+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870975+0000","last_undegraded":"2024-05-20T20:54:37.870975+0000","last_fullsized":"2024-05-20T20:54:37.870975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867985+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:37.867985+0000","last_peered":"2024-05-20T20:54:37.867985+0000","last_clean":"2024-05-20T20:54:37.867985+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:37.867985+0000","last_undegraded":"2024-05-20T20:54:37.867985+0000","last_fullsized":"2024-05-20T20:54:37.867985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871280+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871280+0000","last_peered":"2024-05-20T20:54:37.871280+0000","last_clean":"2024-05-20T20:54:37.871280+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871280+0000","last_undegraded":"2024-05-20T20:54:37.871280+0000","last_fullsized":"2024-05-20T20:54:37.871280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870627+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:54:37.870627+0000","last_peered":"2024-05-20T20:54:37.870627+0000","last_clean":"2024-05-20T20:54:37.870627+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:54:37.870627+0000","last_undegraded":"2024-05-20T20:54:37.870627+0000","last_fullsized":"2024-05-20T20:54:37.870627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887334+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:38.887334+0000","last_peered":"2024-05-20T20:54:38.887334+0000","last_clean":"2024-05-20T20:54:38.887334+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:38.887334+0000","last_undegraded":"2024-05-20T20:54:38.887334+0000","last_fullsized":"2024-05-20T20:54:38.887334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887439+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:38.887439+0000","last_peered":"2024-05-20T20:54:38.887439+0000","last_clean":"2024-05-20T20:54:38.887439+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:38.887439+0000","last_undegraded":"2024-05-20T20:54:38.887439+0000","last_fullsized":"2024-05-20T20:54:38.887439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886573+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:38.886573+0000","last_peered":"2024-05-20T20:54:38.886573+0000","last_clean":"2024-05-20T20:54:38.886573+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:38.886573+0000","last_undegraded":"2024-05-20T20:54:38.886573+0000","last_fullsized":"2024-05-20T20:54:38.886573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867747+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:37.867747+0000","last_peered":"2024-05-20T20:54:37.867747+0000","last_clean":"2024-05-20T20:54:37.867747+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:37.867747+0000","last_undegraded":"2024-05-20T20:54:37.867747+0000","last_fullsized":"2024-05-20T20:54:37.867747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870785+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:54:37.870785+0000","last_peered":"2024-05-20T20:54:37.870785+0000","last_clean":"2024-05-20T20:54:37.870785+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:54:37.870785+0000","last_undegraded":"2024-05-20T20:54:37.870785+0000","last_fullsized":"2024-05-20T20:54:37.870785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887850+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:38.887850+0000","last_peered":"2024-05-20T20:54:38.887850+0000","last_clean":"2024-05-20T20:54:38.887850+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:38.887850+0000","last_undegraded":"2024-05-20T20:54:38.887850+0000","last_fullsized":"2024-05-20T20:54:38.887850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474618+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:38.474618+0000","last_peered":"2024-05-20T20:54:38.474618+0000","last_clean":"2024-05-20T20:54:38.474618+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:38.474618+0000","last_undegraded":"2024-05-20T20:54:38.474618+0000","last_fullsized":"2024-05-20T20:54:38.474618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874544+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:37.874544+0000","last_peered":"2024-05-20T20:54:37.874544+0000","last_clean":"2024-05-20T20:54:37.874544+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:37.874544+0000","last_undegraded":"2024-05-20T20:54:37.874544+0000","last_fullsized":"2024-05-20T20:54:37.874544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474329+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:38.474329+0000","last_peered":"2024-05-20T20:54:38.474329+0000","last_clean":"2024-05-20T20:54:38.474329+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:38.474329+0000","last_undegraded":"2024-05-20T20:54:38.474329+0000","last_fullsized":"2024-05-20T20:54:38.474329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766006+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:54:38.766006+0000","last_peered":"2024-05-20T20:54:38.766006+0000","last_clean":"2024-05-20T20:54:38.766006+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:54:38.766006+0000","last_undegraded":"2024-05-20T20:54:38.766006+0000","last_fullsized":"2024-05-20T20:54:38.766006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867894+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:37.867894+0000","last_peered":"2024-05-20T20:54:37.867894+0000","last_clean":"2024-05-20T20:54:37.867894+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:37.867894+0000","last_undegraded":"2024-05-20T20:54:37.867894+0000","last_fullsized":"2024-05-20T20:54:37.867894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870651+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:54:37.870651+0000","last_peered":"2024-05-20T20:54:37.870651+0000","last_clean":"2024-05-20T20:54:37.870651+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:54:37.870651+0000","last_undegraded":"2024-05-20T20:54:37.870651+0000","last_fullsized":"2024-05-20T20:54:37.870651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72792,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1032,"num_read_kb":593,"num_write":185,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":442424,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1102203,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442580,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1036028,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":442049,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":509115,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":443320,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":511237,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":442360,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":443623,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822739,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1034687,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:55:52.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:55:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:52.423 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:52 smithi026 ceph-mon[31990]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:52.423 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:55:52.423 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:55:52.423 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:52.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:52 smithi055 ceph-mon[35213]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:52.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:55:52.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:55:52.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:52 smithi144 ceph-mon[35660]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:55:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:55:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:55:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:53 smithi055 ceph-mon[35213]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:53.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:53 smithi055 ceph-mon[35213]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-20T20:55:53.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:53 smithi055 ceph-mon[35213]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:53 smithi026 ceph-mon[31990]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:53 smithi026 ceph-mon[31990]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-20T20:55:53.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:53 smithi026 ceph-mon[31990]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:53 smithi144 ceph-mon[35660]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:53 smithi144 ceph-mon[35660]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-20T20:55:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:53 smithi144 ceph-mon[35660]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:55:54.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 -- 172.21.15.26:0/2088555812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f810ff90 msgr2=0x7f62f8112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 --2- 172.21.15.26:0/2088555812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f810ff90 0x7f62f8112380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f62f000b0a0 tx=0x7f62f000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 -- 172.21.15.26:0/2088555812 shutdown_connections 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 --2- 172.21.15.26:0/2088555812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f810ff90 0x7f62f8112380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 --2- 172.21.15.26:0/2088555812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62f806bcf0 0x7f62f806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 --2- 172.21.15.26:0/2088555812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f806c610 0x7f62f810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.130+0000 7f62ff557700 1 -- 172.21.15.26:0/2088555812 >> 172.21.15.26:0/2088555812 conn(0x7f62f80794e0 msgr2=0x7f62f80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.131+0000 7f62ff557700 1 -- 172.21.15.26:0/2088555812 shutdown_connections 2024-05-20T20:55:54.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.131+0000 7f62ff557700 1 -- 172.21.15.26:0/2088555812 wait complete. 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.131+0000 7f62ff557700 1 Processor -- start 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 -- start start 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 0x7f62f819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f806c610 0x7f62f819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62f810ff90 0x7f62f81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81157f0 con 0x7f62f806bcf0 2024-05-20T20:55:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8115670 con 0x7f62f806c610 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62ff557700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8115970 con 0x7f62f810ff90 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 0x7f62f819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 0x7f62f819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56388/0 (socket says 172.21.15.26:56388) 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 -- 172.21.15.26:0/1439017491 learned_addr learned my addr 172.21.15.26:0/1439017491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fdaf4700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62f810ff90 0x7f62f81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fcaf2700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f806c610 0x7f62f819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 -- 172.21.15.26:0/1439017491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62f810ff90 msgr2=0x7f62f81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62f810ff90 0x7f62f81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 -- 172.21.15.26:0/1439017491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f806c610 msgr2=0x7f62f819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f806c610 0x7f62f819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 -- 172.21.15.26:0/1439017491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f81a0d40 con 0x7f62f806bcf0 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.132+0000 7f62fd2f3700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 0x7f62f819bd80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d980 tx=0x7f62f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.133+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4017070 con 0x7f62f806bcf0 2024-05-20T20:55:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.133+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62f4005020 con 0x7f62f806bcf0 2024-05-20T20:55:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.133+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f81a0fd0 con 0x7f62f806bcf0 2024-05-20T20:55:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.133+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4009ab0 con 0x7f62f806bcf0 2024-05-20T20:55:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.133+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a88b0 con 0x7f62f806bcf0 2024-05-20T20:55:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.134+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f806a230 con 0x7f62f806bcf0 2024-05-20T20:55:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.135+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f62f40051d0 con 0x7f62f806bcf0 2024-05-20T20:55:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.136+0000 7f62ee7fc700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62e40b2910 0x7f62e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.136+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f62f40e2710 con 0x7f62f806bcf0 2024-05-20T20:55:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.136+0000 7f62fcaf2700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62e40b2910 0x7f62e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.136+0000 7f62fcaf2700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62e40b2910 0x7f62e40b4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005950 tx=0x7f62e80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.139+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62f4020020 con 0x7f62f806bcf0 2024-05-20T20:55:54.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.403+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62f802e560 con 0x7f62f806bcf0 2024-05-20T20:55:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.404+0000 7f62ee7fc700 1 -- 172.21.15.26:0/1439017491 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f62f40cae20 con 0x7f62f806bcf0 2024-05-20T20:55:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:54.404 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:55:54.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62e40b2910 msgr2=0x7f62e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:54.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62e40b2910 0x7f62e40b4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005950 tx=0x7f62e80058e0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 msgr2=0x7f62f819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:54.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 0x7f62f819bd80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d980 tx=0x7f62f400dc90 comp rx=0 tx=0).stop 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 shutdown_connections 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62e40b2910 0x7f62e40b4dd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f806bcf0 0x7f62f819bd80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62f810ff90 0x7f62f81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 --2- 172.21.15.26:0/1439017491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f806c610 0x7f62f819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 >> 172.21.15.26:0/1439017491 conn(0x7f62f80794e0 msgr2=0x7f62f8111630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 shutdown_connections 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:54.407+0000 7f62ff557700 1 -- 172.21.15.26:0/1439017491 wait complete. 2024-05-20T20:55:54.408 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:55:55.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:55:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:55 smithi055 ceph-mon[35213]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-20T20:55:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:55 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1439017491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:55.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:55 smithi026 ceph-mon[31990]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-20T20:55:55.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:55 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1439017491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:55 smithi144 ceph-mon[35660]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-20T20:55:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:55 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1439017491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:55:57.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 -- 172.21.15.26:0/172419435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54102460 msgr2=0x7fbf541028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 --2- 172.21.15.26:0/172419435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54102460 0x7fbf541028e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009990 tx=0x7fbf3c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 -- 172.21.15.26:0/172419435 shutdown_connections 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 --2- 172.21.15.26:0/172419435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf54102e20 0x7fbf5410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 --2- 172.21.15.26:0/172419435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf541084f0 0x7fbf541088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 --2- 172.21.15.26:0/172419435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54102460 0x7fbf541028e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 -- 172.21.15.26:0/172419435 >> 172.21.15.26:0/172419435 conn(0x7fbf540794e0 msgr2=0x7fbf540798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 -- 172.21.15.26:0/172419435 shutdown_connections 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.044+0000 7fbf58c87700 1 -- 172.21.15.26:0/172419435 wait complete. 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 Processor -- start 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 -- start start 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf54102460 0x7fbf5419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf54102e20 0x7fbf5419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 0x7fbf541a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf541124d0 con 0x7fbf541084f0 2024-05-20T20:55:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54112350 con 0x7fbf54102e20 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf58c87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54112650 con 0x7fbf54102460 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.045+0000 7fbf52d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 0x7fbf541a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 0x7fbf541a5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56402/0 (socket says 172.21.15.26:56402) 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf5259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf54102460 0x7fbf5419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 -- 172.21.15.26:0/4252565809 learned_addr learned my addr 172.21.15.26:0/4252565809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf51d9b700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf54102e20 0x7fbf5419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 -- 172.21.15.26:0/4252565809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf54102460 msgr2=0x7fbf5419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf54102460 0x7fbf5419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 -- 172.21.15.26:0/4252565809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf54102e20 msgr2=0x7fbf5419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf54102e20 0x7fbf5419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 -- 172.21.15.26:0/4252565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541a5f90 con 0x7fbf541084f0 2024-05-20T20:55:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf52d9d700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 0x7fbf541a5a50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c007ee0 tx=0x7fbf4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c004bb0 con 0x7fbf541084f0 2024-05-20T20:55:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541a61c0 con 0x7fbf541084f0 2024-05-20T20:55:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf4c004d50 con 0x7fbf541084f0 2024-05-20T20:55:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0058b0 con 0x7fbf541084f0 2024-05-20T20:55:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.046+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541a66d0 con 0x7fbf541084f0 2024-05-20T20:55:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.047+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5410a9f0 con 0x7fbf541084f0 2024-05-20T20:55:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.049+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fbf4c01a020 con 0x7fbf541084f0 2024-05-20T20:55:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.049+0000 7fbf4b7fe700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf400b2990 0x7fbf400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:55:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.050+0000 7fbf5259c700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf400b2990 0x7fbf400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:55:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.050+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbf4c0e1a10 con 0x7fbf541084f0 2024-05-20T20:55:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.050+0000 7fbf5259c700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf400b2990 0x7fbf400b4e50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf440045b0 tx=0x7fbf4400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:55:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.054+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf4c010070 con 0x7fbf541084f0 2024-05-20T20:55:57.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.272+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf5410a3d0 con 0x7fbf541084f0 2024-05-20T20:55:57.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.274+0000 7fbf4b7fe700 1 -- 172.21.15.26:0/4252565809 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22791 (secure 0 0 0) 0x7fbf4c0a9570 con 0x7fbf541084f0 2024-05-20T20:55:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:55:57.274 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:54:37.856459+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:55:57.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf400b2990 msgr2=0x7fbf400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:57.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf400b2990 0x7fbf400b4e50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf440045b0 tx=0x7fbf4400b3c0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 msgr2=0x7fbf541a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:55:57.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 0x7fbf541a5a50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c007ee0 tx=0x7fbf4c00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 shutdown_connections 2024-05-20T20:55:57.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbf400b2990 0x7fbf400b4e50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541084f0 0x7fbf541a5a50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf54102460 0x7fbf5419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 --2- 172.21.15.26:0/4252565809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf54102e20 0x7fbf5419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:55:57.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 >> 172.21.15.26:0/4252565809 conn(0x7fbf540794e0 msgr2=0x7fbf54105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:55:57.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 shutdown_connections 2024-05-20T20:55:57.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:55:57.276+0000 7fbf58c87700 1 -- 172.21.15.26:0/4252565809 wait complete. 2024-05-20T20:55:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:57 smithi055 ceph-mon[35213]: pgmap v382: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:55:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:57 smithi026 ceph-mon[31990]: pgmap v382: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:55:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:57 smithi144 ceph-mon[35660]: pgmap v382: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:55:57.891 DEBUG:tasks.fwd_scrub:Yielding 2024-05-20T20:55:57.891 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-20T20:55:57.901 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T20:55:57.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-20T20:55:57.902 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-20T20:55:57.902 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:55:57.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:55:57.905 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-20T20:55:57.920 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-20T20:55:57.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:55:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:55:58.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:58 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4252565809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:58.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:58 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4252565809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:58.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:58 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4252565809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:55:59.303 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:55:59 smithi026 ceph-mon[31990]: pgmap v383: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:55:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:55:59 smithi055 ceph-mon[35213]: pgmap v383: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:55:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:55:59 smithi144 ceph-mon[35660]: pgmap v383: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:01.234 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:01 smithi026 ceph-mon[31990]: pgmap v384: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 -- 172.21.15.26:0/1396793416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 msgr2=0x7fe9741081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/1396793416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe9741081c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe9640099c0 tx=0x7fe964009cd0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 -- 172.21.15.26:0/1396793416 shutdown_connections 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/1396793416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9741088f0 0x7fe974110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/1396793416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9741037f0 0x7fe974103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/1396793416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe9741081c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 -- 172.21.15.26:0/1396793416 >> 172.21.15.26:0/1396793416 conn(0x7fe97407a6a0 msgr2=0x7fe97407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 -- 172.21.15.26:0/1396793416 shutdown_connections 2024-05-20T20:56:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.283+0000 7fe97b8d3700 1 -- 172.21.15.26:0/1396793416 wait complete. 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 Processor -- start 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 -- start start 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9741037f0 0x7fe97419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe97419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9741088f0 0x7fe974199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe974113560 con 0x7fe9741041a0 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741133e0 con 0x7fe9741088f0 2024-05-20T20:56:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97b8d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741136e0 con 0x7fe9741037f0 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe978e6e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe97419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe978e6e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe97419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51948/0 (socket says 172.21.15.26:51948) 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe978e6e700 1 -- 172.21.15.26:0/721840516 learned_addr learned my addr 172.21.15.26:0/721840516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.284+0000 7fe97966f700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9741037f0 0x7fe97419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe979e70700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9741088f0 0x7fe974199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe978e6e700 1 -- 172.21.15.26:0/721840516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9741037f0 msgr2=0x7fe97419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe978e6e700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9741037f0 0x7fe97419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe978e6e700 1 -- 172.21.15.26:0/721840516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9741088f0 msgr2=0x7fe974199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe978e6e700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9741088f0 0x7fe974199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe978e6e700 1 -- 172.21.15.26:0/721840516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97419a060 con 0x7fe9741041a0 2024-05-20T20:56:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe978e6e700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe97419fc80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe964000c00 tx=0x7fe9640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400f8a0 con 0x7fe9741041a0 2024-05-20T20:56:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe964004830 con 0x7fe9741041a0 2024-05-20T20:56:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97419a2f0 con 0x7fe9741041a0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964005410 con 0x7fe9741041a0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.285+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741a6610 con 0x7fe9741041a0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.287+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe974104c80 con 0x7fe9741041a0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.288+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fe964029060 con 0x7fe9741041a0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.289+0000 7fe96a7fc700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9600b2a60 0x7fe9600b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.289+0000 7fe97966f700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9600b2a60 0x7fe9600b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.289+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe9640e2df0 con 0x7fe9741041a0 2024-05-20T20:56:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.289+0000 7fe97966f700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9600b2a60 0x7fe9600b4f20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe97419aa40 tx=0x7fe97000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.293+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe964013070 con 0x7fe9741041a0 2024-05-20T20:56:01.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 -- 172.21.15.26:0/3013735618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 msgr2=0x7fa724108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 --2- 172.21.15.26:0/3013735618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724108900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa714009990 tx=0x7fa714009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 -- 172.21.15.26:0/3013735618 shutdown_connections 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 --2- 172.21.15.26:0/3013735618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724108900 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 --2- 172.21.15.26:0/3013735618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 0x7fa72410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 --2- 172.21.15.26:0/3013735618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 0x7fa7241028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.179+0000 7fa729bb4700 1 -- 172.21.15.26:0/3013735618 >> 172.21.15.26:0/3013735618 conn(0x7fa7240793f0 msgr2=0x7fa724079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.180+0000 7fa729bb4700 1 -- 172.21.15.26:0/3013735618 shutdown_connections 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.180+0000 7fa729bb4700 1 -- 172.21.15.26:0/3013735618 wait complete. 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.180+0000 7fa729bb4700 1 Processor -- start 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.180+0000 7fa729bb4700 1 -- start start 2024-05-20T20:56:01.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa729bb4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 0x7fa724104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa729bb4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 0x7fa7241045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa729bb4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa729bb4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa724112480 con 0x7fa724108520 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa729bb4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa724112300 con 0x7fa724102e00 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa729bb4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa724112600 con 0x7fa724102440 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51900/0 (socket says 172.21.15.26:51900) 2024-05-20T20:56:01.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 -- 172.21.15.26:0/725589708 learned_addr learned my addr 172.21.15.26:0/725589708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa7237fe700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 0x7fa724104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa71affd700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 0x7fa7241045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 -- 172.21.15.26:0/725589708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 msgr2=0x7fa724104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 0x7fa724104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 -- 172.21.15.26:0/725589708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 msgr2=0x7fa7241045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 0x7fa7241045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 -- 172.21.15.26:0/725589708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7241a80a0 con 0x7fa724108520 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa71affd700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 0x7fa7241045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa7237fe700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 0x7fa724104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.181+0000 7fa723fff700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724104b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ef30 tx=0x7fa71c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.182+0000 7fa7217fa700 1 -- 172.21.15.26:0/725589708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0092e0 con 0x7fa724108520 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.182+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7241a8330 con 0x7fa724108520 2024-05-20T20:56:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.182+0000 7fa7217fa700 1 -- 172.21.15.26:0/725589708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa71c004bb0 con 0x7fa724108520 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.182+0000 7fa7217fa700 1 -- 172.21.15.26:0/725589708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c004020 con 0x7fa724108520 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.182+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7241a8870 con 0x7fa724108520 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.185+0000 7fa7217fa700 1 -- 172.21.15.26:0/725589708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fa71c01b030 con 0x7fa724108520 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa7217fa700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa7100b2910 0x7fa7100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa73444ab80 1 Processor -- start 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa73444ab80 1 -- start start 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa7237fe700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa7100b2910 0x7fa7100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa7217fa700 1 -- 172.21.15.26:0/725589708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa71c0e1e00 con 0x7fa724108520 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bc4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8bc89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc8fe0 0x561ab8bc9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.186+0000 7fa73444ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561ab8b23d10 con 0x561ab8bc48e0 2024-05-20T20:56:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa73444ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561ab8b138b0 con 0x561ab8bc5290 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa73444ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561ab8b178f0 con 0x561ab8bc8fe0 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bc4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bc4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51908/0 (socket says 172.21.15.26:51908) 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 -- 172.21.15.26:0/348649055 learned_addr learned my addr 172.21.15.26:0/348649055 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa723fff700 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc8fe0 0x561ab8bc9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa71affd700 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8bc89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa7100b2910 0x7fa7100b4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b2e0 tx=0x7fa714005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 -- 172.21.15.26:0/348649055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc8fe0 msgr2=0x561ab8bc9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc8fe0 0x561ab8bc9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 -- 172.21.15.26:0/348649055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 msgr2=0x561ab8bc89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8bc89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 -- 172.21.15.26:0/348649055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab8a63950 con 0x561ab8bc48e0 2024-05-20T20:56:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa71affd700 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8bc89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa7237fe700 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bc4cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa714009990 tx=0x7fa714027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f7035b362191c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.187+0000 7fa719ffb700 1 -- 172.21.15.26:0/348649055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714005050 con 0x561ab8bc48e0 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa719ffb700 1 -- 172.21.15.26:0/348649055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa714005680 con 0x561ab8bc48e0 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa719ffb700 1 -- 172.21.15.26:0/348649055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140038e0 con 0x561ab8bc48e0 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 -- 172.21.15.26:0/348649055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 msgr2=0x561ab8bc4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bc4cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa714009990 tx=0x7fa714027000 comp rx=0 tx=0).stop 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 -- 172.21.15.26:0/348649055 shutdown_connections 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bc4cc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc8fe0 0x561ab8bc9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 --2- 172.21.15.26:0/348649055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8bc89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 -- 172.21.15.26:0/348649055 >> 172.21.15.26:0/348649055 conn(0x561ab8af2100 msgr2=0x561ab8bbef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 -- 172.21.15.26:0/348649055 shutdown_connections 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.188+0000 7fa73444ab80 1 -- 172.21.15.26:0/348649055 wait complete. 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 -- start start 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 Processor -- start 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bd0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc5290 0x561ab8bcd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8bcdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561ab8a1a020 con 0x561ab8bc48e0 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561ab8ab5840 con 0x561ab8bc8fe0 2024-05-20T20:56:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561ab8bd1140 con 0x561ab8bc5290 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa73444ab80 10 client.0 _ll_register_callbacks cb 0x561ab8aae9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bd0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bd0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51924/0 (socket says 172.21.15.26:51924) 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.189+0000 7fa7237fe700 1 -- 172.21.15.26:0/3287885675 learned_addr learned my addr 172.21.15.26:0/3287885675 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa71affd700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc5290 0x561ab8bcd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa723fff700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8bcdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa7237fe700 1 -- 172.21.15.26:0/3287885675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc5290 msgr2=0x561ab8bcd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa7237fe700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc5290 0x561ab8bcd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa7237fe700 1 -- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 msgr2=0x561ab8bcdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa7237fe700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8bcdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa7237fe700 1 -- 172.21.15.26:0/3287885675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab8ab2710 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa7237fe700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bd0480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa714009ce0 tx=0x7fa714005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107afa9231c576c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa702ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa702ffd700 1 -- 172.21.15.26:0/3287885675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714005050 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa714005680 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa702ffd700 1 -- 172.21.15.26:0/3287885675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa7140051f0 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa702ffd700 1 -- 172.21.15.26:0/3287885675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa714046050 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa73444ab80 10 client.15216 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.190+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561ab8bdb890 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa702ffd700 1 -- 172.21.15.26:0/3287885675 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa714005680 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa702ffd700 1 -- 172.21.15.26:0/3287885675 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa714030480 con 0x561ab8bc48e0 2024-05-20T20:56:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa73444ab80 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa73444ab80 10 client.15216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa73444ab80 20 client.15216 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa73444ab80 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8bddd60 0x561ab8be0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa73444ab80 4 client.15216 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab8b90a60 con 0x561ab8bddd60 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.191+0000 7fa71affd700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8bddd60 0x561ab8be0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.192+0000 7fa71affd700 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8bddd60 0x561ab8be0150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7c9e2696f47d1 server_cookie=d79a311139b19134 in_seq=0 out_seq=0 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.192+0000 7fa702ffd700 10 client.15216 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa702ffd700 1 -- 172.21.15.26:0/3287885675 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561ab8b90a60 con 0x561ab8bddd60 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa702ffd700 10 client.15216 handle_command_reply: tid=0 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 client.15216 shutdown 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 msgr2=0x561ab8bd0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bd0480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa714009ce0 tx=0x7fa714005a00 comp rx=0 tx=0).stop 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 shutdown_connections 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8bddd60 0x561ab8be0150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc48e0 0x561ab8bd0480 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8bcdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa702ffd700 0 client.15216 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 --2- 172.21.15.26:0/3287885675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc5290 0x561ab8bcd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 >> 172.21.15.26:0/3287885675 conn(0x561ab8af2100 msgr2=0x561ab8bbe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 shutdown_connections 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 1 -- 172.21.15.26:0/3287885675 wait complete. 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.193+0000 7fa73444ab80 20 client.15216 trim_cache size 0 max 16384 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 Processor -- start 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 -- start start 2024-05-20T20:56:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8bcc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 0x561ab8bbed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8be3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241b26a0 con 0x561ab8bc5290 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241aff30 con 0x561ab8bc8fe0 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa73444ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241add80 con 0x561ab8bc48e0 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa71affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 0x561ab8bbed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa71affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 0x561ab8bbed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51928/0 (socket says 172.21.15.26:51928) 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa71affd700 1 -- 172.21.15.26:0/473165370 learned_addr learned my addr 172.21.15.26:0/473165370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa7237fe700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8bcc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.215+0000 7fa723fff700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8be3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa71affd700 1 -- 172.21.15.26:0/473165370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 msgr2=0x561ab8bcc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa71affd700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8bcc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa71affd700 1 -- 172.21.15.26:0/473165370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 msgr2=0x561ab8be3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa71affd700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8be3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa71affd700 1 -- 172.21.15.26:0/473165370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab8bc4f10 con 0x561ab8bc5290 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa7237fe700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8bcc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa723fff700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8be3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa71affd700 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 0x561ab8bbed10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x561ab8bced50 tx=0x7fa70c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d80ec80c065ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa7197fa700 1 -- 172.21.15.26:0/473165370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c004050 con 0x561ab8bc5290 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa7197fa700 1 -- 172.21.15.26:0/473165370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa70c026070 con 0x561ab8bc5290 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.216+0000 7fa7197fa700 1 -- 172.21.15.26:0/473165370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c010430 con 0x561ab8bc5290 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 -- 172.21.15.26:0/473165370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 msgr2=0x561ab8bbed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 0x561ab8bbed10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x561ab8bced50 tx=0x7fa70c009260 comp rx=0 tx=0).stop 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 -- 172.21.15.26:0/473165370 shutdown_connections 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc8fe0 0x561ab8be3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8bcc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 --2- 172.21.15.26:0/473165370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc5290 0x561ab8bbed10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 -- 172.21.15.26:0/473165370 >> 172.21.15.26:0/473165370 conn(0x561ab8af2100 msgr2=0x561ab8bc81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 -- 172.21.15.26:0/473165370 shutdown_connections 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 -- 172.21.15.26:0/473165370 wait complete. 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.217+0000 7fa73444ab80 1 -- start start 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 Processor -- start 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8c3c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8c3cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 0x561ab8c40b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241ae3f0 con 0x561ab8bc8fe0 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241af860 con 0x561ab8bc5290 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241aeac0 con 0x561ab8bc48e0 2024-05-20T20:56:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa73444ab80 10 client.0 _ll_register_callbacks cb 0x561ab8aae9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 0x561ab8c40b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 0x561ab8c40b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51940/0 (socket says 172.21.15.26:51940) 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 -- 172.21.15.26:0/2302865911 learned_addr learned my addr 172.21.15.26:0/2302865911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa71affd700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8c3cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa7237fe700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8c3c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 -- 172.21.15.26:0/2302865911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 msgr2=0x561ab8c3c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8c3c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 -- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 msgr2=0x561ab8c3cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8c3cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa723fff700 1 -- 172.21.15.26:0/2302865911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab8c41220 con 0x561ab8bc8fe0 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.218+0000 7fa7237fe700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8c3c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa723fff700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 0x561ab8c40b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0a6560 tx=0x7fa71c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75507a2417c57afc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa702ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa702ffd700 1 -- 172.21.15.26:0/2302865911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0a92f0 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa702ffd700 1 -- 172.21.15.26:0/2302865911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa71c0b1e40 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70c026070 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa702ffd700 1 -- 172.21.15.26:0/2302865911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa71c0ad6a0 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 10 client.15222 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561ab8c49830 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa702ffd700 1 -- 172.21.15.26:0/2302865911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa70c026070 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa702ffd700 1 -- 172.21.15.26:0/2302865911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa71c0a4cf0 con 0x561ab8bc8fe0 2024-05-20T20:56:01.411 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "ce702d78-8efe-4dcd-95c9-fd8a3c68a1a6", 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 10 client.15222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 20 client.15222 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8c49ee0 0x561ab8c4c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 4 client.15222 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:56:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.219+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ab8b25c60 con 0x561ab8c49ee0 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.220+0000 7fa7237fe700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8c49ee0 0x561ab8c4c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.220+0000 7fa702ffd700 10 client.15222 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.220+0000 7fa7237fe700 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8c49ee0 0x561ab8c4c2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c6362e65062d59 server_cookie=7902b2175bee598 in_seq=0 out_seq=0 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa702ffd700 1 -- 172.21.15.26:0/2302865911 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ab8b25c60 con 0x561ab8c49ee0 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa702ffd700 10 client.15222 handle_command_reply: tid=0 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 client.15222 shutdown 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 msgr2=0x561ab8c40b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 0x561ab8c40b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0a6560 tx=0x7fa71c004f90 comp rx=0 tx=0).stop 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 shutdown_connections 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561ab8c49ee0 0x561ab8c4c2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561ab8bc8fe0 0x561ab8c40b40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ab8bc48e0 0x561ab8c3c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa702ffd700 0 client.15222 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 --2- 172.21.15.26:0/2302865911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ab8bc5290 0x561ab8c3cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 >> 172.21.15.26:0/2302865911 conn(0x561ab8af2100 msgr2=0x561ab8bcc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 shutdown_connections 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 1 -- 172.21.15.26:0/2302865911 wait complete. 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.222+0000 7fa73444ab80 20 client.15222 trim_cache size 0 max 16384 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa7100b2910 msgr2=0x7fa7100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa7100b2910 0x7fa7100b4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b2e0 tx=0x7fa714005fb0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 msgr2=0x7fa724104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724104b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ef30 tx=0x7fa71c00c3a0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 shutdown_connections 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa7100b2910 0x7fa7100b4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724108520 0x7fa724104b00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa724102440 0x7fa724104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 --2- 172.21.15.26:0/725589708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa724102e00 0x7fa7241045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 >> 172.21.15.26:0/725589708 conn(0x7fa7240793f0 msgr2=0x7fa72406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 shutdown_connections 2024-05-20T20:56:01.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.223+0000 7fa729bb4700 1 -- 172.21.15.26:0/725589708 wait complete. 2024-05-20T20:56:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:01 smithi055 ceph-mon[35213]: pgmap v384: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-20T20:56:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.511+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe974063980 con 0x7fe9600b2a60 2024-05-20T20:56:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.519+0000 7fe96a7fc700 1 -- 172.21.15.26:0/721840516 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259085 (secure 0 0 0) 0x7fe974063980 con 0x7fe9600b2a60 2024-05-20T20:56:01.523 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9600b2a60 msgr2=0x7fe9600b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9600b2a60 0x7fe9600b4f20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe97419aa40 tx=0x7fe97000a380 comp rx=0 tx=0).stop 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 msgr2=0x7fe97419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe97419fc80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe964000c00 tx=0x7fe9640047b0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 shutdown_connections 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9741088f0 0x7fe974199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9741037f0 0x7fe97419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9600b2a60 0x7fe9600b4f20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 --2- 172.21.15.26:0/721840516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9741041a0 0x7fe97419fc80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 >> 172.21.15.26:0/721840516 conn(0x7fe97407a6a0 msgr2=0x7fe9741018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 shutdown_connections 2024-05-20T20:56:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:01.522+0000 7fe97b8d3700 1 -- 172.21.15.26:0/721840516 wait complete. 2024-05-20T20:56:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:56:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:01 smithi144 ceph-mon[35660]: pgmap v384: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-20T20:56:02.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:56:02.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:02 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:56:02.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:02 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:02.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:02 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:56:02.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:02 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:02.466 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:02 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:56:02.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:02 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:02.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:02 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:56:02.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:02 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:02 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:56:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:02 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:02 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:56:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:02 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:02.667 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-05-20T20:55:59.743626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665914,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1169,"num_read_kb":690,"num_write":311,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355984,"kb_used_data":10968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585840,"statfs":{"total":1151940427776,"available":1151575900160,"internally_reserved":0,"allocated":11231232,"data_stored":7299846,"data_compressed":249159,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7718,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":49,"num_write":12,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003190"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870824+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:54:37.870824+0000","last_peered":"2024-05-20T20:54:37.870824+0000","last_clean":"2024-05-20T20:54:37.870824+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:54:37.870824+0000","last_undegraded":"2024-05-20T20:54:37.870824+0000","last_fullsized":"2024-05-20T20:54:37.870824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886671+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:38.886671+0000","last_peered":"2024-05-20T20:54:38.886671+0000","last_clean":"2024-05-20T20:54:38.886671+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:38.886671+0000","last_undegraded":"2024-05-20T20:54:38.886671+0000","last_fullsized":"2024-05-20T20:54:38.886671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886528+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:54:38.886528+0000","last_peered":"2024-05-20T20:54:38.886528+0000","last_clean":"2024-05-20T20:54:38.886528+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:54:38.886528+0000","last_undegraded":"2024-05-20T20:54:38.886528+0000","last_fullsized":"2024-05-20T20:54:38.886528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475208+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:38.475208+0000","last_peered":"2024-05-20T20:54:38.475208+0000","last_clean":"2024-05-20T20:54:38.475208+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:38.475208+0000","last_undegraded":"2024-05-20T20:54:38.475208+0000","last_fullsized":"2024-05-20T20:54:38.475208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887218+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:38.887218+0000","last_peered":"2024-05-20T20:54:38.887218+0000","last_clean":"2024-05-20T20:54:38.887218+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:38.887218+0000","last_undegraded":"2024-05-20T20:54:38.887218+0000","last_fullsized":"2024-05-20T20:54:38.887218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876469+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876469+0000","last_peered":"2024-05-20T20:54:37.876469+0000","last_clean":"2024-05-20T20:54:37.876469+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876469+0000","last_undegraded":"2024-05-20T20:54:37.876469+0000","last_fullsized":"2024-05-20T20:54:37.876469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870690+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870690+0000","last_peered":"2024-05-20T20:54:37.870690+0000","last_clean":"2024-05-20T20:54:37.870690+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870690+0000","last_undegraded":"2024-05-20T20:54:37.870690+0000","last_fullsized":"2024-05-20T20:54:37.870690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766030+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:54:38.766030+0000","last_peered":"2024-05-20T20:54:38.766030+0000","last_clean":"2024-05-20T20:54:38.766030+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:54:38.766030+0000","last_undegraded":"2024-05-20T20:54:38.766030+0000","last_fullsized":"2024-05-20T20:54:38.766030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876712+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876712+0000","last_peered":"2024-05-20T20:54:37.876712+0000","last_clean":"2024-05-20T20:54:37.876712+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876712+0000","last_undegraded":"2024-05-20T20:54:37.876712+0000","last_fullsized":"2024-05-20T20:54:37.876712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871157+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871157+0000","last_peered":"2024-05-20T20:54:37.871157+0000","last_clean":"2024-05-20T20:54:37.871157+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871157+0000","last_undegraded":"2024-05-20T20:54:37.871157+0000","last_fullsized":"2024-05-20T20:54:37.871157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475180+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:38.475180+0000","last_peered":"2024-05-20T20:54:38.475180+0000","last_clean":"2024-05-20T20:54:38.475180+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:38.475180+0000","last_undegraded":"2024-05-20T20:54:38.475180+0000","last_fullsized":"2024-05-20T20:54:38.475180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867869+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:37.867869+0000","last_peered":"2024-05-20T20:54:37.867869+0000","last_clean":"2024-05-20T20:54:37.867869+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:37.867869+0000","last_undegraded":"2024-05-20T20:54:37.867869+0000","last_fullsized":"2024-05-20T20:54:37.867869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886436+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:38.886436+0000","last_peered":"2024-05-20T20:54:38.886436+0000","last_clean":"2024-05-20T20:54:38.886436+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:38.886436+0000","last_undegraded":"2024-05-20T20:54:38.886436+0000","last_fullsized":"2024-05-20T20:54:38.886436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886364+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:38.886364+0000","last_peered":"2024-05-20T20:54:38.886364+0000","last_clean":"2024-05-20T20:54:38.886364+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:38.886364+0000","last_undegraded":"2024-05-20T20:54:38.886364+0000","last_fullsized":"2024-05-20T20:54:38.886364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765447+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:54:38.765447+0000","last_peered":"2024-05-20T20:54:38.765447+0000","last_clean":"2024-05-20T20:54:38.765447+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:54:38.765447+0000","last_undegraded":"2024-05-20T20:54:38.765447+0000","last_fullsized":"2024-05-20T20:54:38.765447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870916+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:54:37.870916+0000","last_peered":"2024-05-20T20:54:37.870916+0000","last_clean":"2024-05-20T20:54:37.870916+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:54:37.870916+0000","last_undegraded":"2024-05-20T20:54:37.870916+0000","last_fullsized":"2024-05-20T20:54:37.870916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872609+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872609+0000","last_peered":"2024-05-20T20:54:37.872609+0000","last_clean":"2024-05-20T20:54:37.872609+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872609+0000","last_undegraded":"2024-05-20T20:54:37.872609+0000","last_fullsized":"2024-05-20T20:54:37.872609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887790+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:38.887790+0000","last_peered":"2024-05-20T20:54:38.887790+0000","last_clean":"2024-05-20T20:54:38.887790+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:38.887790+0000","last_undegraded":"2024-05-20T20:54:38.887790+0000","last_fullsized":"2024-05-20T20:54:38.887790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872721+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:37.872721+0000","last_peered":"2024-05-20T20:54:37.872721+0000","last_clean":"2024-05-20T20:54:37.872721+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:37.872721+0000","last_undegraded":"2024-05-20T20:54:37.872721+0000","last_fullsized":"2024-05-20T20:54:37.872721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474659+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:38.474659+0000","last_peered":"2024-05-20T20:54:38.474659+0000","last_clean":"2024-05-20T20:54:38.474659+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:38.474659+0000","last_undegraded":"2024-05-20T20:54:38.474659+0000","last_fullsized":"2024-05-20T20:54:38.474659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474766+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:54:38.474766+0000","last_peered":"2024-05-20T20:54:38.474766+0000","last_clean":"2024-05-20T20:54:38.474766+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:54:38.474766+0000","last_undegraded":"2024-05-20T20:54:38.474766+0000","last_fullsized":"2024-05-20T20:54:38.474766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886666+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.886666+0000","last_peered":"2024-05-20T20:54:38.886666+0000","last_clean":"2024-05-20T20:54:38.886666+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:54:38.886666+0000","last_undegraded":"2024-05-20T20:54:38.886666+0000","last_fullsized":"2024-05-20T20:54:38.886666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887575+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:38.887575+0000","last_peered":"2024-05-20T20:54:38.887575+0000","last_clean":"2024-05-20T20:54:38.887575+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:38.887575+0000","last_undegraded":"2024-05-20T20:54:38.887575+0000","last_fullsized":"2024-05-20T20:54:38.887575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765538+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:54:38.765538+0000","last_peered":"2024-05-20T20:54:38.765538+0000","last_clean":"2024-05-20T20:54:38.765538+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:54:38.765538+0000","last_undegraded":"2024-05-20T20:54:38.765538+0000","last_fullsized":"2024-05-20T20:54:38.765538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887488+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:38.887488+0000","last_peered":"2024-05-20T20:54:38.887488+0000","last_clean":"2024-05-20T20:54:38.887488+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:38.887488+0000","last_undegraded":"2024-05-20T20:54:38.887488+0000","last_fullsized":"2024-05-20T20:54:38.887488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874582+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:37.874582+0000","last_peered":"2024-05-20T20:54:37.874582+0000","last_clean":"2024-05-20T20:54:37.874582+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:37.874582+0000","last_undegraded":"2024-05-20T20:54:37.874582+0000","last_fullsized":"2024-05-20T20:54:37.874582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":216,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:55.582740+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:55:55.582740+0000","last_peered":"2024-05-20T20:55:55.582740+0000","last_clean":"2024-05-20T20:55:55.582740+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:55:55.582740+0000","last_undegraded":"2024-05-20T20:55:55.582740+0000","last_fullsized":"2024-05-20T20:55:55.582740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887502+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:38.887502+0000","last_peered":"2024-05-20T20:54:38.887502+0000","last_clean":"2024-05-20T20:54:38.887502+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:38.887502+0000","last_undegraded":"2024-05-20T20:54:38.887502+0000","last_fullsized":"2024-05-20T20:54:38.887502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876310+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876310+0000","last_peered":"2024-05-20T20:54:37.876310+0000","last_clean":"2024-05-20T20:54:37.876310+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876310+0000","last_undegraded":"2024-05-20T20:54:37.876310+0000","last_fullsized":"2024-05-20T20:54:37.876310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874230+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:37.874230+0000","last_peered":"2024-05-20T20:54:37.874230+0000","last_clean":"2024-05-20T20:54:37.874230+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:37.874230+0000","last_undegraded":"2024-05-20T20:54:37.874230+0000","last_fullsized":"2024-05-20T20:54:37.874230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867681+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:37.867681+0000","last_peered":"2024-05-20T20:54:37.867681+0000","last_clean":"2024-05-20T20:54:37.867681+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:37.867681+0000","last_undegraded":"2024-05-20T20:54:37.867681+0000","last_fullsized":"2024-05-20T20:54:37.867681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474359+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:38.474359+0000","last_peered":"2024-05-20T20:54:38.474359+0000","last_clean":"2024-05-20T20:54:38.474359+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:38.474359+0000","last_undegraded":"2024-05-20T20:54:38.474359+0000","last_fullsized":"2024-05-20T20:54:38.474359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874492+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:37.874492+0000","last_peered":"2024-05-20T20:54:37.874492+0000","last_clean":"2024-05-20T20:54:37.874492+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:37.874492+0000","last_undegraded":"2024-05-20T20:54:37.874492+0000","last_fullsized":"2024-05-20T20:54:37.874492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872063+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872063+0000","last_peered":"2024-05-20T20:54:37.872063+0000","last_clean":"2024-05-20T20:54:37.872063+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872063+0000","last_undegraded":"2024-05-20T20:54:37.872063+0000","last_fullsized":"2024-05-20T20:54:37.872063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":213,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:58.586550+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:55:58.586550+0000","last_peered":"2024-05-20T20:55:58.586550+0000","last_clean":"2024-05-20T20:55:58.586550+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:55:58.586550+0000","last_undegraded":"2024-05-20T20:55:58.586550+0000","last_fullsized":"2024-05-20T20:55:58.586550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765842+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:54:38.765842+0000","last_peered":"2024-05-20T20:54:38.765842+0000","last_clean":"2024-05-20T20:54:38.765842+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:54:38.765842+0000","last_undegraded":"2024-05-20T20:54:38.765842+0000","last_fullsized":"2024-05-20T20:54:38.765842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871069+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871069+0000","last_peered":"2024-05-20T20:54:37.871069+0000","last_clean":"2024-05-20T20:54:37.871069+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871069+0000","last_undegraded":"2024-05-20T20:54:37.871069+0000","last_fullsized":"2024-05-20T20:54:37.871069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886717+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:38.886717+0000","last_peered":"2024-05-20T20:54:38.886717+0000","last_clean":"2024-05-20T20:54:38.886717+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:38.886717+0000","last_undegraded":"2024-05-20T20:54:38.886717+0000","last_fullsized":"2024-05-20T20:54:38.886717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874541+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:37.874541+0000","last_peered":"2024-05-20T20:54:37.874541+0000","last_clean":"2024-05-20T20:54:37.874541+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:37.874541+0000","last_undegraded":"2024-05-20T20:54:37.874541+0000","last_fullsized":"2024-05-20T20:54:37.874541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.765712+0000","last_peered":"2024-05-20T20:54:38.765712+0000","last_clean":"2024-05-20T20:54:38.765712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:54:38.765712+0000","last_undegraded":"2024-05-20T20:54:38.765712+0000","last_fullsized":"2024-05-20T20:54:38.765712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":437,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:59.588214+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:55:59.588214+0000","last_peered":"2024-05-20T20:55:59.588214+0000","last_clean":"2024-05-20T20:55:59.588214+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:55:59.588214+0000","last_undegraded":"2024-05-20T20:55:59.588214+0000","last_fullsized":"2024-05-20T20:55:59.588214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69793,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":369,"num_read_kb":252,"num_write":38,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887265+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:38.887265+0000","last_peered":"2024-05-20T20:54:38.887265+0000","last_clean":"2024-05-20T20:54:38.887265+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:38.887265+0000","last_undegraded":"2024-05-20T20:54:38.887265+0000","last_fullsized":"2024-05-20T20:54:38.887265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887299+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:38.887299+0000","last_peered":"2024-05-20T20:54:38.887299+0000","last_clean":"2024-05-20T20:54:38.887299+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:38.887299+0000","last_undegraded":"2024-05-20T20:54:38.887299+0000","last_fullsized":"2024-05-20T20:54:38.887299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867648+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:37.867648+0000","last_peered":"2024-05-20T20:54:37.867648+0000","last_clean":"2024-05-20T20:54:37.867648+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:37.867648+0000","last_undegraded":"2024-05-20T20:54:37.867648+0000","last_fullsized":"2024-05-20T20:54:37.867648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874116+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:37.874116+0000","last_peered":"2024-05-20T20:54:37.874116+0000","last_clean":"2024-05-20T20:54:37.874116+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:37.874116+0000","last_undegraded":"2024-05-20T20:54:37.874116+0000","last_fullsized":"2024-05-20T20:54:37.874116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871059+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.871059+0000","last_peered":"2024-05-20T20:54:37.871059+0000","last_clean":"2024-05-20T20:54:37.871059+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.871059+0000","last_undegraded":"2024-05-20T20:54:37.871059+0000","last_fullsized":"2024-05-20T20:54:37.871059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":217,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:55.606706+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:55:55.606706+0000","last_peered":"2024-05-20T20:55:55.606706+0000","last_clean":"2024-05-20T20:55:55.606706+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:55:55.606706+0000","last_undegraded":"2024-05-20T20:55:55.606706+0000","last_fullsized":"2024-05-20T20:55:55.606706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870598+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870598+0000","last_peered":"2024-05-20T20:54:37.870598+0000","last_clean":"2024-05-20T20:54:37.870598+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870598+0000","last_undegraded":"2024-05-20T20:54:37.870598+0000","last_fullsized":"2024-05-20T20:54:37.870598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886999+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:38.886999+0000","last_peered":"2024-05-20T20:54:38.886999+0000","last_clean":"2024-05-20T20:54:38.886999+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:38.886999+0000","last_undegraded":"2024-05-20T20:54:38.886999+0000","last_fullsized":"2024-05-20T20:54:38.886999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874169+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:37.874169+0000","last_peered":"2024-05-20T20:54:37.874169+0000","last_clean":"2024-05-20T20:54:37.874169+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:37.874169+0000","last_undegraded":"2024-05-20T20:54:37.874169+0000","last_fullsized":"2024-05-20T20:54:37.874169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887338+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:38.887338+0000","last_peered":"2024-05-20T20:54:38.887338+0000","last_clean":"2024-05-20T20:54:38.887338+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:38.887338+0000","last_undegraded":"2024-05-20T20:54:38.887338+0000","last_fullsized":"2024-05-20T20:54:38.887338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870840+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:54:37.870840+0000","last_peered":"2024-05-20T20:54:37.870840+0000","last_clean":"2024-05-20T20:54:37.870840+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:54:37.870840+0000","last_undegraded":"2024-05-20T20:54:37.870840+0000","last_fullsized":"2024-05-20T20:54:37.870840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872174+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872174+0000","last_peered":"2024-05-20T20:54:37.872174+0000","last_clean":"2024-05-20T20:54:37.872174+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872174+0000","last_undegraded":"2024-05-20T20:54:37.872174+0000","last_fullsized":"2024-05-20T20:54:37.872174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876474+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876474+0000","last_peered":"2024-05-20T20:54:37.876474+0000","last_clean":"2024-05-20T20:54:37.876474+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876474+0000","last_undegraded":"2024-05-20T20:54:37.876474+0000","last_fullsized":"2024-05-20T20:54:37.876474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871372+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871372+0000","last_peered":"2024-05-20T20:54:37.871372+0000","last_clean":"2024-05-20T20:54:37.871372+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871372+0000","last_undegraded":"2024-05-20T20:54:37.871372+0000","last_fullsized":"2024-05-20T20:54:37.871372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875765+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875765+0000","last_peered":"2024-05-20T20:54:37.875765+0000","last_clean":"2024-05-20T20:54:37.875765+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875765+0000","last_undegraded":"2024-05-20T20:54:37.875765+0000","last_fullsized":"2024-05-20T20:54:37.875765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872208+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872208+0000","last_peered":"2024-05-20T20:54:37.872208+0000","last_clean":"2024-05-20T20:54:37.872208+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872208+0000","last_undegraded":"2024-05-20T20:54:37.872208+0000","last_fullsized":"2024-05-20T20:54:37.872208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":212,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:55.605702+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:55:55.605702+0000","last_peered":"2024-05-20T20:55:55.605702+0000","last_clean":"2024-05-20T20:55:55.605702+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:55:55.605702+0000","last_undegraded":"2024-05-20T20:55:55.605702+0000","last_fullsized":"2024-05-20T20:55:55.605702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474755+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:38.474755+0000","last_peered":"2024-05-20T20:54:38.474755+0000","last_clean":"2024-05-20T20:54:38.474755+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:38.474755+0000","last_undegraded":"2024-05-20T20:54:38.474755+0000","last_fullsized":"2024-05-20T20:54:38.474755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475163+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:54:38.475163+0000","last_peered":"2024-05-20T20:54:38.475163+0000","last_clean":"2024-05-20T20:54:38.475163+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:54:38.475163+0000","last_undegraded":"2024-05-20T20:54:38.475163+0000","last_fullsized":"2024-05-20T20:54:38.475163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870715+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870715+0000","last_peered":"2024-05-20T20:54:37.870715+0000","last_clean":"2024-05-20T20:54:37.870715+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870715+0000","last_undegraded":"2024-05-20T20:54:37.870715+0000","last_fullsized":"2024-05-20T20:54:37.870715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867545+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:37.867545+0000","last_peered":"2024-05-20T20:54:37.867545+0000","last_clean":"2024-05-20T20:54:37.867545+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:37.867545+0000","last_undegraded":"2024-05-20T20:54:37.867545+0000","last_fullsized":"2024-05-20T20:54:37.867545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870913+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870913+0000","last_peered":"2024-05-20T20:54:37.870913+0000","last_clean":"2024-05-20T20:54:37.870913+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870913+0000","last_undegraded":"2024-05-20T20:54:37.870913+0000","last_fullsized":"2024-05-20T20:54:37.870913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887395+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:38.887395+0000","last_peered":"2024-05-20T20:54:38.887395+0000","last_clean":"2024-05-20T20:54:38.887395+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:38.887395+0000","last_undegraded":"2024-05-20T20:54:38.887395+0000","last_fullsized":"2024-05-20T20:54:38.887395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867898+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:37.867898+0000","last_peered":"2024-05-20T20:54:37.867898+0000","last_clean":"2024-05-20T20:54:37.867898+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:37.867898+0000","last_undegraded":"2024-05-20T20:54:37.867898+0000","last_fullsized":"2024-05-20T20:54:37.867898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.261304+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:55:45.261304+0000","last_peered":"2024-05-20T20:55:45.261304+0000","last_clean":"2024-05-20T20:55:45.261304+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:55:45.261304+0000","last_undegraded":"2024-05-20T20:55:45.261304+0000","last_fullsized":"2024-05-20T20:55:45.261304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474726+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:38.474726+0000","last_peered":"2024-05-20T20:54:38.474726+0000","last_clean":"2024-05-20T20:54:38.474726+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:38.474726+0000","last_undegraded":"2024-05-20T20:54:38.474726+0000","last_fullsized":"2024-05-20T20:54:38.474726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475338+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:54:38.475338+0000","last_peered":"2024-05-20T20:54:38.475338+0000","last_clean":"2024-05-20T20:54:38.475338+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:54:38.475338+0000","last_undegraded":"2024-05-20T20:54:38.475338+0000","last_fullsized":"2024-05-20T20:54:38.475338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886677+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:38.886677+0000","last_peered":"2024-05-20T20:54:38.886677+0000","last_clean":"2024-05-20T20:54:38.886677+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:38.886677+0000","last_undegraded":"2024-05-20T20:54:38.886677+0000","last_fullsized":"2024-05-20T20:54:38.886677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765487+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:54:38.765487+0000","last_peered":"2024-05-20T20:54:38.765487+0000","last_clean":"2024-05-20T20:54:38.765487+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:54:38.765487+0000","last_undegraded":"2024-05-20T20:54:38.765487+0000","last_fullsized":"2024-05-20T20:54:38.765487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887085+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:38.887085+0000","last_peered":"2024-05-20T20:54:38.887085+0000","last_clean":"2024-05-20T20:54:38.887085+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:38.887085+0000","last_undegraded":"2024-05-20T20:54:38.887085+0000","last_fullsized":"2024-05-20T20:54:38.887085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474392+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:54:38.474392+0000","last_peered":"2024-05-20T20:54:38.474392+0000","last_clean":"2024-05-20T20:54:38.474392+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:54:38.474392+0000","last_undegraded":"2024-05-20T20:54:38.474392+0000","last_fullsized":"2024-05-20T20:54:38.474392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870487+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:54:37.870487+0000","last_peered":"2024-05-20T20:54:37.870487+0000","last_clean":"2024-05-20T20:54:37.870487+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:54:37.870487+0000","last_undegraded":"2024-05-20T20:54:37.870487+0000","last_fullsized":"2024-05-20T20:54:37.870487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.325040+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:55:45.325040+0000","last_peered":"2024-05-20T20:55:45.325040+0000","last_clean":"2024-05-20T20:55:45.325040+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:55:45.325040+0000","last_undegraded":"2024-05-20T20:55:45.325040+0000","last_fullsized":"2024-05-20T20:55:45.325040+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474638+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:38.474638+0000","last_peered":"2024-05-20T20:54:38.474638+0000","last_clean":"2024-05-20T20:54:38.474638+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:38.474638+0000","last_undegraded":"2024-05-20T20:54:38.474638+0000","last_fullsized":"2024-05-20T20:54:38.474638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870566+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:54:37.870566+0000","last_peered":"2024-05-20T20:54:37.870566+0000","last_clean":"2024-05-20T20:54:37.870566+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:54:37.870566+0000","last_undegraded":"2024-05-20T20:54:37.870566+0000","last_fullsized":"2024-05-20T20:54:37.870566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.880369+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.880369+0000","last_peered":"2024-05-20T20:54:37.880369+0000","last_clean":"2024-05-20T20:54:37.880369+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.880369+0000","last_undegraded":"2024-05-20T20:54:37.880369+0000","last_fullsized":"2024-05-20T20:54:37.880369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887561+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:38.887561+0000","last_peered":"2024-05-20T20:54:38.887561+0000","last_clean":"2024-05-20T20:54:38.887561+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:38.887561+0000","last_undegraded":"2024-05-20T20:54:38.887561+0000","last_fullsized":"2024-05-20T20:54:38.887561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871302+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871302+0000","last_peered":"2024-05-20T20:54:37.871302+0000","last_clean":"2024-05-20T20:54:37.871302+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871302+0000","last_undegraded":"2024-05-20T20:54:37.871302+0000","last_fullsized":"2024-05-20T20:54:37.871302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874356+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:37.874356+0000","last_peered":"2024-05-20T20:54:37.874356+0000","last_clean":"2024-05-20T20:54:37.874356+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:37.874356+0000","last_undegraded":"2024-05-20T20:54:37.874356+0000","last_fullsized":"2024-05-20T20:54:37.874356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870975+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870975+0000","last_peered":"2024-05-20T20:54:37.870975+0000","last_clean":"2024-05-20T20:54:37.870975+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870975+0000","last_undegraded":"2024-05-20T20:54:37.870975+0000","last_fullsized":"2024-05-20T20:54:37.870975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867985+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:37.867985+0000","last_peered":"2024-05-20T20:54:37.867985+0000","last_clean":"2024-05-20T20:54:37.867985+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:37.867985+0000","last_undegraded":"2024-05-20T20:54:37.867985+0000","last_fullsized":"2024-05-20T20:54:37.867985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871280+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871280+0000","last_peered":"2024-05-20T20:54:37.871280+0000","last_clean":"2024-05-20T20:54:37.871280+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871280+0000","last_undegraded":"2024-05-20T20:54:37.871280+0000","last_fullsized":"2024-05-20T20:54:37.871280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870627+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:54:37.870627+0000","last_peered":"2024-05-20T20:54:37.870627+0000","last_clean":"2024-05-20T20:54:37.870627+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:54:37.870627+0000","last_undegraded":"2024-05-20T20:54:37.870627+0000","last_fullsized":"2024-05-20T20:54:37.870627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887334+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:38.887334+0000","last_peered":"2024-05-20T20:54:38.887334+0000","last_clean":"2024-05-20T20:54:38.887334+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:38.887334+0000","last_undegraded":"2024-05-20T20:54:38.887334+0000","last_fullsized":"2024-05-20T20:54:38.887334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887439+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:38.887439+0000","last_peered":"2024-05-20T20:54:38.887439+0000","last_clean":"2024-05-20T20:54:38.887439+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:38.887439+0000","last_undegraded":"2024-05-20T20:54:38.887439+0000","last_fullsized":"2024-05-20T20:54:38.887439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886573+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:38.886573+0000","last_peered":"2024-05-20T20:54:38.886573+0000","last_clean":"2024-05-20T20:54:38.886573+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:38.886573+0000","last_undegraded":"2024-05-20T20:54:38.886573+0000","last_fullsized":"2024-05-20T20:54:38.886573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867747+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:37.867747+0000","last_peered":"2024-05-20T20:54:37.867747+0000","last_clean":"2024-05-20T20:54:37.867747+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:37.867747+0000","last_undegraded":"2024-05-20T20:54:37.867747+0000","last_fullsized":"2024-05-20T20:54:37.867747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870785+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:54:37.870785+0000","last_peered":"2024-05-20T20:54:37.870785+0000","last_clean":"2024-05-20T20:54:37.870785+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:54:37.870785+0000","last_undegraded":"2024-05-20T20:54:37.870785+0000","last_fullsized":"2024-05-20T20:54:37.870785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887850+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:38.887850+0000","last_peered":"2024-05-20T20:54:38.887850+0000","last_clean":"2024-05-20T20:54:38.887850+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:38.887850+0000","last_undegraded":"2024-05-20T20:54:38.887850+0000","last_fullsized":"2024-05-20T20:54:38.887850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474618+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:38.474618+0000","last_peered":"2024-05-20T20:54:38.474618+0000","last_clean":"2024-05-20T20:54:38.474618+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:38.474618+0000","last_undegraded":"2024-05-20T20:54:38.474618+0000","last_fullsized":"2024-05-20T20:54:38.474618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874544+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:37.874544+0000","last_peered":"2024-05-20T20:54:37.874544+0000","last_clean":"2024-05-20T20:54:37.874544+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:37.874544+0000","last_undegraded":"2024-05-20T20:54:37.874544+0000","last_fullsized":"2024-05-20T20:54:37.874544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474329+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:38.474329+0000","last_peered":"2024-05-20T20:54:38.474329+0000","last_clean":"2024-05-20T20:54:38.474329+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:38.474329+0000","last_undegraded":"2024-05-20T20:54:38.474329+0000","last_fullsized":"2024-05-20T20:54:38.474329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766006+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:54:38.766006+0000","last_peered":"2024-05-20T20:54:38.766006+0000","last_clean":"2024-05-20T20:54:38.766006+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:54:38.766006+0000","last_undegraded":"2024-05-20T20:54:38.766006+0000","last_fullsized":"2024-05-20T20:54:38.766006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867894+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:37.867894+0000","last_peered":"2024-05-20T20:54:37.867894+0000","last_clean":"2024-05-20T20:54:37.867894+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:37.867894+0000","last_undegraded":"2024-05-20T20:54:37.867894+0000","last_fullsized":"2024-05-20T20:54:37.867894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870651+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:54:37.870651+0000","last_peered":"2024-05-20T20:54:37.870651+0000","last_clean":"2024-05-20T20:54:37.870651+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:54:37.870651+0000","last_undegraded":"2024-05-20T20:54:37.870651+0000","last_fullsized":"2024-05-20T20:54:37.870651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75312,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1093,"num_read_kb":626,"num_write":186,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":225936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710773,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":442424,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907004,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1104723,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070529,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442580,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462987,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1036028,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":442049,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":511635,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":443320,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":513757,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":442360,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":443623,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1034687,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:56:02.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:56:02.801 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:03 smithi055 ceph-mon[35213]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:03.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:03 smithi055 ceph-mon[35213]: pgmap v385: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-20T20:56:03.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:03 smithi026 ceph-mon[31990]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:03.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:03 smithi026 ceph-mon[31990]: pgmap v385: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-20T20:56:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:03 smithi144 ceph-mon[35660]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:03 smithi144 ceph-mon[35660]: pgmap v385: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-20T20:56:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 -- 172.21.15.26:0/2736064280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac06b220 msgr2=0x7f47ac06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/2736064280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac06b220 0x7f47ac06b6a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f479c009990 tx=0x7f479c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 -- 172.21.15.26:0/2736064280 shutdown_connections 2024-05-20T20:56:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/2736064280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06bbe0 0x7f47ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/2736064280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47ac1095b0 0x7f47ac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/2736064280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac06b220 0x7f47ac06b6a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 -- 172.21.15.26:0/2736064280 >> 172.21.15.26:0/2736064280 conn(0x7f47ac0793f0 msgr2=0x7f47ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 -- 172.21.15.26:0/2736064280 shutdown_connections 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.486+0000 7f47b3f5e700 1 -- 172.21.15.26:0/2736064280 wait complete. 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 Processor -- start 2024-05-20T20:56:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 -- start start 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06b220 0x7f47ac19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47ac06bbe0 0x7f47ac19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 0x7f47ac199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1135f0 con 0x7f47ac1095b0 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac113470 con 0x7f47ac06bbe0 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b3f5e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac113770 con 0x7f47ac06b220 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.487+0000 7f47b24fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 0x7f47ac199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 0x7f47ac199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51964/0 (socket says 172.21.15.26:51964) 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 -- 172.21.15.26:0/648732644 learned_addr learned my addr 172.21.15.26:0/648732644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06b220 0x7f47ac19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b14f9700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47ac06bbe0 0x7f47ac19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 -- 172.21.15.26:0/648732644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06b220 msgr2=0x7f47ac19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06b220 0x7f47ac19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 -- 172.21.15.26:0/648732644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47ac06bbe0 msgr2=0x7f47ac19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47ac06bbe0 0x7f47ac19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 -- 172.21.15.26:0/648732644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac19a0f0 con 0x7f47ac1095b0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06b220 0x7f47ac19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b24fb700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 0x7f47ac199990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b550 tx=0x7f47a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47a2ffd700 1 -- 172.21.15.26:0/648732644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4015020 con 0x7f47ac1095b0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47a2ffd700 1 -- 172.21.15.26:0/648732644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47a4011cc0 con 0x7f47ac1095b0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ac19a3e0 con 0x7f47ac1095b0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47a2ffd700 1 -- 172.21.15.26:0/648732644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4010c70 con 0x7f47ac1095b0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.488+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ac1a66a0 con 0x7f47ac1095b0 2024-05-20T20:56:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.491+0000 7f47a2ffd700 1 -- 172.21.15.26:0/648732644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f47a4018020 con 0x7f47ac1095b0 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.491+0000 7f47a2ffd700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f47980b2910 0x7f47980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f47980b2910 0x7f47980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47a2ffd700 1 -- 172.21.15.26:0/648732644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f47a40e1ab0 con 0x7f47ac1095b0 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 Processor -- start 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f47980b2910 0x7f47980b4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f47a8007fd0 tx=0x7f47a8007830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 -- start start 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550cca7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550ccab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550ccac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56550cc06c90 con 0x56550cca7860 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56550cbf6830 con 0x56550cca8210 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56550cbfa870 con 0x56550ccabf60 2024-05-20T20:56:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47b1cfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550cca7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47b1cfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550cca7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51974/0 (socket says 172.21.15.26:51974) 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.492+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2421906405 learned_addr learned my addr 172.21.15.26:0/2421906405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550ccab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b24fb700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550ccac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2421906405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 msgr2=0x56550ccac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550ccac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2421906405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 msgr2=0x56550ccab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550ccab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2421906405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550cb468d0 con 0x56550cca7860 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b24fb700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550ccac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550ccab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.493+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550cca7c40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47a8018820 tx=0x7f47a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a327e14a9251e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.494+0000 7f4797fff700 1 -- 172.21.15.26:0/2421906405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8026070 con 0x56550cca7860 2024-05-20T20:56:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.494+0000 7f4797fff700 1 -- 172.21.15.26:0/2421906405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47a80180d0 con 0x56550cca7860 2024-05-20T20:56:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.494+0000 7f4797fff700 1 -- 172.21.15.26:0/2421906405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8016430 con 0x56550cca7860 2024-05-20T20:56:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.494+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2421906405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 msgr2=0x56550cca7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.494+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550cca7c40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47a8018820 tx=0x7f47a8011000 comp rx=0 tx=0).stop 2024-05-20T20:56:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2421906405 shutdown_connections 2024-05-20T20:56:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550cca7c40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550ccac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2421906405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550ccab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2421906405 >> 172.21.15.26:0/2421906405 conn(0x56550cbd5080 msgr2=0x56550cca1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2421906405 shutdown_connections 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2421906405 wait complete. 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 -- start start 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.495+0000 7f47be7f4b80 1 Processor -- start 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca7860 0x56550ccb3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca8210 0x56550ccb0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 0x56550ccb0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56550cafcfa0 con 0x56550ccabf60 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56550cb987c0 con 0x56550cca7860 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56550ccb4120 con 0x56550cca8210 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47be7f4b80 10 client.0 _ll_register_callbacks cb 0x56550cb91920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 0x56550ccb0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 0x56550ccb0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51976/0 (socket says 172.21.15.26:51976) 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 -- 172.21.15.26:0/401584548 learned_addr learned my addr 172.21.15.26:0/401584548 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b14f9700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca8210 0x56550ccb0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca7860 0x56550ccb3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 -- 172.21.15.26:0/401584548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca8210 msgr2=0x56550ccb0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca8210 0x56550ccb0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 -- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca7860 msgr2=0x56550ccb3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca7860 0x56550ccb3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 -- 172.21.15.26:0/401584548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550cb95690 con 0x56550ccabf60 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca7860 0x56550ccb3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47b24fb700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 0x56550ccb0b70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b520 tx=0x7f47a40a8cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02f82f2bdc1ba1a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.496+0000 7f47957fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47957fa700 1 -- 172.21.15.26:0/401584548 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4011830 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a80180d0 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47957fa700 1 -- 172.21.15.26:0/401584548 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f47a40119d0 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47957fa700 1 -- 172.21.15.26:0/401584548 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47a40a9330 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 10 client.15240 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56550ccc2bb0 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47957fa700 1 -- 172.21.15.26:0/401584548 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47a80180d0 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47957fa700 1 -- 172.21.15.26:0/401584548 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f47a40a4a40 con 0x56550ccabf60 2024-05-20T20:56:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 10 client.15240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 20 client.15240 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550ccc51f0 0x56550ccc75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 4 client.15240 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.497+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56550cc739e0 con 0x56550ccc51f0 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.498+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550ccc51f0 0x56550ccc75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.498+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550ccc51f0 0x56550ccc75e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c141d6e28eb93a server_cookie=fd9656c03bdd6b66 in_seq=0 out_seq=0 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.498+0000 7f47957fa700 10 client.15240 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47957fa700 1 -- 172.21.15.26:0/401584548 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56550cc739e0 con 0x56550ccc51f0 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47957fa700 10 client.15240 handle_command_reply: tid=0 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 client.15240 shutdown 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 msgr2=0x56550ccb0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 0x56550ccb0b70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b520 tx=0x7f47a40a8cd0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 shutdown_connections 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550ccc51f0 0x56550ccc75e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550ccabf60 0x56550ccb0b70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca7860 0x56550ccb3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/401584548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca8210 0x56550ccb0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47957fa700 0 client.15240 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 >> 172.21.15.26:0/401584548 conn(0x56550cbd5080 msgr2=0x56550cca1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 shutdown_connections 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 1 -- 172.21.15.26:0/401584548 wait complete. 2024-05-20T20:56:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.499+0000 7f47be7f4b80 20 client.15240 trim_cache size 0 max 16384 2024-05-20T20:56:04.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.520+0000 7f47be7f4b80 1 Processor -- start 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 -- start start 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550ccb2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550cccce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550cccd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1ae460 con 0x56550cca7860 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1abcf0 con 0x56550cca8210 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1a9b40 con 0x56550ccabf60 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550ccb2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550ccb2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51990/0 (socket says 172.21.15.26:51990) 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2724890776 learned_addr learned my addr 172.21.15.26:0/2724890776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550cccce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2724890776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 msgr2=0x56550cccd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550cccd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2724890776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 msgr2=0x56550cccce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550cccce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 -- 172.21.15.26:0/2724890776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550ccaebc0 con 0x56550cca7860 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550cccce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550ccb2570 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f47a80184a0 tx=0x7f47a8021b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d1c824de112de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.521+0000 7f47977fe700 1 -- 172.21.15.26:0/2724890776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8026040 con 0x56550cca7860 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47977fe700 1 -- 172.21.15.26:0/2724890776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47a8004d50 con 0x56550cca7860 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47977fe700 1 -- 172.21.15.26:0/2724890776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8021870 con 0x56550cca7860 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2724890776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 msgr2=0x56550ccb2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550ccb2570 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f47a80184a0 tx=0x7f47a8021b30 comp rx=0 tx=0).stop 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2724890776 shutdown_connections 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca7860 0x56550ccb2570 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550ccabf60 0x56550cccd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2724890776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550cca8210 0x56550cccce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2724890776 >> 172.21.15.26:0/2724890776 conn(0x56550cbd5080 msgr2=0x56550ccb1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2724890776 shutdown_connections 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.522+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2724890776 wait complete. 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 -- start start 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 Processor -- start 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca7860 0x56550ccb9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 0x56550ccb9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550ccabf60 0x56550ccb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1aa1b0 con 0x56550cca8210 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1ab620 con 0x56550ccabf60 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1aa880 con 0x56550cca7860 2024-05-20T20:56:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47be7f4b80 10 client.0 _ll_register_callbacks cb 0x56550cb91920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47b14f9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 0x56550ccb9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47b14f9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 0x56550ccb9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51998/0 (socket says 172.21.15.26:51998) 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47b14f9700 1 -- 172.21.15.26:0/2402996432 learned_addr learned my addr 172.21.15.26:0/2402996432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.523+0000 7f47b24fb700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550ccabf60 0x56550ccb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca7860 0x56550ccb9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b14f9700 1 -- 172.21.15.26:0/2402996432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca7860 msgr2=0x56550ccb9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca7860 0x56550ccb9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b14f9700 1 -- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550ccabf60 msgr2=0x56550ccb78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550ccabf60 0x56550ccb78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b14f9700 1 -- 172.21.15.26:0/2402996432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550ccb8080 con 0x56550cca8210 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47b14f9700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 0x56550ccb9940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c0036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71859b91af3c293f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47957fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47957fa700 1 -- 172.21.15.26:0/2402996432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c028020 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47957fa700 1 -- 172.21.15.26:0/2402996432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f479c003bd0 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a8004d50 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47957fa700 1 -- 172.21.15.26:0/2402996432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f479c033050 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47be7f4b80 10 client.15246 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.524+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56550cd307a0 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47957fa700 1 -- 172.21.15.26:0/2402996432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47a8004d50 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47957fa700 1 -- 172.21.15.26:0/2402996432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f479c00f560 con 0x56550cca8210 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47be7f4b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47be7f4b80 10 client.15246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47be7f4b80 20 client.15246 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550cd30f30 0x56550cd33350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47be7f4b80 4 client.15246 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56550cc08be0 con 0x56550cd30f30 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550cd30f30 0x56550cd33350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47b1cfa700 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550cd30f30 0x56550cd33350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7dfb89902256735 server_cookie=7de33a482a83bbd5 in_seq=0 out_seq=0 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.525+0000 7f47957fa700 10 client.15246 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47957fa700 1 -- 172.21.15.26:0/2402996432 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56550cc08be0 con 0x56550cd30f30 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47957fa700 10 client.15246 handle_command_reply: tid=0 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 client.15246 shutdown 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 msgr2=0x56550ccb9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 0x56550ccb9940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c0036a0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 shutdown_connections 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56550cd30f30 0x56550cd33350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550ccabf60 0x56550ccb78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550cca7860 0x56550ccb9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 --2- 172.21.15.26:0/2402996432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550cca8210 0x56550ccb9940 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47957fa700 0 client.15246 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 >> 172.21.15.26:0/2402996432 conn(0x56550cbd5080 msgr2=0x56550ccb26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 shutdown_connections 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 1 -- 172.21.15.26:0/2402996432 wait complete. 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.526+0000 7f47be7f4b80 20 client.15246 trim_cache size 0 max 16384 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "ce702d78-8efe-4dcd-95c9-fd8a3c68a1a6": { 2024-05-20T20:56:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "ce702d78-8efe-4dcd-95c9-fd8a3c68a1a6", 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f47980b2910 msgr2=0x7f47980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f47980b2910 0x7f47980b4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f47a8007fd0 tx=0x7f47a8007830 comp rx=0 tx=0).stop 2024-05-20T20:56:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 msgr2=0x7f47ac199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 0x7f47ac199990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b550 tx=0x7f47a400b910 comp rx=0 tx=0).stop 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 shutdown_connections 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f47980b2910 0x7f47980b4dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ac1095b0 0x7f47ac199990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac06b220 0x7f47ac19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 --2- 172.21.15.26:0/648732644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47ac06bbe0 0x7f47ac19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 >> 172.21.15.26:0/648732644 conn(0x7f47ac0793f0 msgr2=0x7f47ac101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 shutdown_connections 2024-05-20T20:56:04.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:04.527+0000 7f47b3f5e700 1 -- 172.21.15.26:0/648732644 wait complete. 2024-05-20T20:56:05.211 INFO:tasks.cephfs.filesystem:scrub status for tag:ce702d78-8efe-4dcd-95c9-fd8a3c68a1a6 - {'path': '/', 'tag': 'ce702d78-8efe-4dcd-95c9-fd8a3c68a1a6', 'options': 'recursive,force'} 2024-05-20T20:56:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:05.433 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:05 smithi026 ceph-mon[31990]: pgmap v386: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:05 smithi055 ceph-mon[35213]: pgmap v386: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:05 smithi144 ceph-mon[35660]: pgmap v386: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:06.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/2558145878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd641028c0 msgr2=0x7fdd64110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:06.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/2558145878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd641028c0 0x7fdd64110330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b0a0 tx=0x7fdd5c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:56:06.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/2558145878 shutdown_connections 2024-05-20T20:56:06.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/2558145878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd641028c0 0x7fdd64110330 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:06.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/2558145878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd64108620 0x7fdd64108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:06.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/2558145878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd64101f00 0x7fdd64102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/2558145878 >> 172.21.15.26:0/2558145878 conn(0x7fdd64077780 msgr2=0x7fdd64077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/2558145878 shutdown_connections 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.918+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/2558145878 wait complete. 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 Processor -- start 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 -- start start 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd64101f00 0x7fdd641030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641028c0 0x7fdd64103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 0x7fdd64103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64113680 con 0x7fdd64108620 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64113500 con 0x7fdd641028c0 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd6a7ca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64113800 con 0x7fdd64101f00 2024-05-20T20:56:06.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd68d67700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 0x7fdd64103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd63fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd64101f00 0x7fdd641030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd68d67700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 0x7fdd64103b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52030/0 (socket says 172.21.15.26:52030) 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd68d67700 1 -- 172.21.15.26:0/217476103 learned_addr learned my addr 172.21.15.26:0/217476103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.919+0000 7fdd5bfff700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641028c0 0x7fdd64103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd68d67700 1 -- 172.21.15.26:0/217476103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd64101f00 msgr2=0x7fdd641030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd68d67700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd64101f00 0x7fdd641030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd68d67700 1 -- 172.21.15.26:0/217476103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641028c0 msgr2=0x7fdd64103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd68d67700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641028c0 0x7fdd64103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd68d67700 1 -- 172.21.15.26:0/217476103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd641a8130 con 0x7fdd64108620 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd5bfff700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641028c0 0x7fdd64103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd68d67700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 0x7fdd64103b60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b830 tx=0x7fdd5c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:06.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c021070 con 0x7fdd64108620 2024-05-20T20:56:06.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641a83c0 con 0x7fdd64108620 2024-05-20T20:56:06.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd5c007b80 con 0x7fdd64108620 2024-05-20T20:56:06.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0052a0 con 0x7fdd64108620 2024-05-20T20:56:06.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.920+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a8810 con 0x7fdd64108620 2024-05-20T20:56:06.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.921+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6406a230 con 0x7fdd64108620 2024-05-20T20:56:06.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.922+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fdd5c024070 con 0x7fdd64108620 2024-05-20T20:56:06.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.923+0000 7fdd61ffb700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdd500b2910 0x7fdd500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:06.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.923+0000 7fdd63fff700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdd500b2910 0x7fdd500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:06.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.923+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fdd5c00f020 con 0x7fdd64108620 2024-05-20T20:56:06.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.924+0000 7fdd63fff700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdd500b2910 0x7fdd500b4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd64078ce0 tx=0x7fdd54009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:06.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:06.927+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd5c02b020 con 0x7fdd64108620 2024-05-20T20:56:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.137+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd64063980 con 0x7fdd500b2910 2024-05-20T20:56:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.145+0000 7fdd61ffb700 1 -- 172.21.15.26:0/217476103 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259083 (secure 0 0 0) 0x7fdd64063980 con 0x7fdd500b2910 2024-05-20T20:56:07.148 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdd500b2910 msgr2=0x7fdd500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdd500b2910 0x7fdd500b4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd64078ce0 tx=0x7fdd54009190 comp rx=0 tx=0).stop 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 msgr2=0x7fdd64103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 0x7fdd64103b60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b830 tx=0x7fdd5c004d00 comp rx=0 tx=0).stop 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 shutdown_connections 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdd500b2910 0x7fdd500b4dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd64108620 0x7fdd64103b60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd64101f00 0x7fdd641030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 --2- 172.21.15.26:0/217476103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641028c0 0x7fdd64103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.147+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 >> 172.21.15.26:0/217476103 conn(0x7fdd64077780 msgr2=0x7fdd64100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.148+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 shutdown_connections 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:07.148+0000 7fdd6a7ca700 1 -- 172.21.15.26:0/217476103 wait complete. 2024-05-20T20:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:56:07.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:07 smithi026 ceph-mon[31990]: pgmap v387: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:07 smithi055 ceph-mon[35213]: pgmap v387: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:07 smithi144 ceph-mon[35660]: pgmap v387: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:07.724 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-05-20T20:56:05.745363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665914,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1201,"num_read_kb":707,"num_write":311,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355984,"kb_used_data":10968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585840,"statfs":{"total":1151940427776,"available":1151575900160,"internally_reserved":0,"allocated":11231232,"data_stored":7299846,"data_compressed":249159,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2520,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":45,"num_write":1,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003380"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870824+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:54:37.870824+0000","last_peered":"2024-05-20T20:54:37.870824+0000","last_clean":"2024-05-20T20:54:37.870824+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:54:37.870824+0000","last_undegraded":"2024-05-20T20:54:37.870824+0000","last_fullsized":"2024-05-20T20:54:37.870824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886671+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:38.886671+0000","last_peered":"2024-05-20T20:54:38.886671+0000","last_clean":"2024-05-20T20:54:38.886671+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:38.886671+0000","last_undegraded":"2024-05-20T20:54:38.886671+0000","last_fullsized":"2024-05-20T20:54:38.886671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886528+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:54:38.886528+0000","last_peered":"2024-05-20T20:54:38.886528+0000","last_clean":"2024-05-20T20:54:38.886528+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:54:38.886528+0000","last_undegraded":"2024-05-20T20:54:38.886528+0000","last_fullsized":"2024-05-20T20:54:38.886528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475208+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:38.475208+0000","last_peered":"2024-05-20T20:54:38.475208+0000","last_clean":"2024-05-20T20:54:38.475208+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:38.475208+0000","last_undegraded":"2024-05-20T20:54:38.475208+0000","last_fullsized":"2024-05-20T20:54:38.475208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887218+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:38.887218+0000","last_peered":"2024-05-20T20:54:38.887218+0000","last_clean":"2024-05-20T20:54:38.887218+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:38.887218+0000","last_undegraded":"2024-05-20T20:54:38.887218+0000","last_fullsized":"2024-05-20T20:54:38.887218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876469+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876469+0000","last_peered":"2024-05-20T20:54:37.876469+0000","last_clean":"2024-05-20T20:54:37.876469+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876469+0000","last_undegraded":"2024-05-20T20:54:37.876469+0000","last_fullsized":"2024-05-20T20:54:37.876469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870690+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870690+0000","last_peered":"2024-05-20T20:54:37.870690+0000","last_clean":"2024-05-20T20:54:37.870690+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870690+0000","last_undegraded":"2024-05-20T20:54:37.870690+0000","last_fullsized":"2024-05-20T20:54:37.870690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766030+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:54:38.766030+0000","last_peered":"2024-05-20T20:54:38.766030+0000","last_clean":"2024-05-20T20:54:38.766030+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:54:38.766030+0000","last_undegraded":"2024-05-20T20:54:38.766030+0000","last_fullsized":"2024-05-20T20:54:38.766030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876712+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876712+0000","last_peered":"2024-05-20T20:54:37.876712+0000","last_clean":"2024-05-20T20:54:37.876712+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876712+0000","last_undegraded":"2024-05-20T20:54:37.876712+0000","last_fullsized":"2024-05-20T20:54:37.876712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871157+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871157+0000","last_peered":"2024-05-20T20:54:37.871157+0000","last_clean":"2024-05-20T20:54:37.871157+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871157+0000","last_undegraded":"2024-05-20T20:54:37.871157+0000","last_fullsized":"2024-05-20T20:54:37.871157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475180+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:38.475180+0000","last_peered":"2024-05-20T20:54:38.475180+0000","last_clean":"2024-05-20T20:54:38.475180+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:38.475180+0000","last_undegraded":"2024-05-20T20:54:38.475180+0000","last_fullsized":"2024-05-20T20:54:38.475180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867869+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:37.867869+0000","last_peered":"2024-05-20T20:54:37.867869+0000","last_clean":"2024-05-20T20:54:37.867869+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:37.867869+0000","last_undegraded":"2024-05-20T20:54:37.867869+0000","last_fullsized":"2024-05-20T20:54:37.867869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886436+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:38.886436+0000","last_peered":"2024-05-20T20:54:38.886436+0000","last_clean":"2024-05-20T20:54:38.886436+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:38.886436+0000","last_undegraded":"2024-05-20T20:54:38.886436+0000","last_fullsized":"2024-05-20T20:54:38.886436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886364+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:38.886364+0000","last_peered":"2024-05-20T20:54:38.886364+0000","last_clean":"2024-05-20T20:54:38.886364+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:38.886364+0000","last_undegraded":"2024-05-20T20:54:38.886364+0000","last_fullsized":"2024-05-20T20:54:38.886364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765447+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:54:38.765447+0000","last_peered":"2024-05-20T20:54:38.765447+0000","last_clean":"2024-05-20T20:54:38.765447+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:54:38.765447+0000","last_undegraded":"2024-05-20T20:54:38.765447+0000","last_fullsized":"2024-05-20T20:54:38.765447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870916+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:54:37.870916+0000","last_peered":"2024-05-20T20:54:37.870916+0000","last_clean":"2024-05-20T20:54:37.870916+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:54:37.870916+0000","last_undegraded":"2024-05-20T20:54:37.870916+0000","last_fullsized":"2024-05-20T20:54:37.870916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872609+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872609+0000","last_peered":"2024-05-20T20:54:37.872609+0000","last_clean":"2024-05-20T20:54:37.872609+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872609+0000","last_undegraded":"2024-05-20T20:54:37.872609+0000","last_fullsized":"2024-05-20T20:54:37.872609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887790+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:38.887790+0000","last_peered":"2024-05-20T20:54:38.887790+0000","last_clean":"2024-05-20T20:54:38.887790+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:38.887790+0000","last_undegraded":"2024-05-20T20:54:38.887790+0000","last_fullsized":"2024-05-20T20:54:38.887790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:48.257006+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:48.257006+0000","last_peered":"2024-05-20T20:54:48.257006+0000","last_clean":"2024-05-20T20:54:48.257006+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:48.257006+0000","last_undegraded":"2024-05-20T20:54:48.257006+0000","last_fullsized":"2024-05-20T20:54:48.257006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474659+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:38.474659+0000","last_peered":"2024-05-20T20:54:38.474659+0000","last_clean":"2024-05-20T20:54:38.474659+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:38.474659+0000","last_undegraded":"2024-05-20T20:54:38.474659+0000","last_fullsized":"2024-05-20T20:54:38.474659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474766+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:54:38.474766+0000","last_peered":"2024-05-20T20:54:38.474766+0000","last_clean":"2024-05-20T20:54:38.474766+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:54:38.474766+0000","last_undegraded":"2024-05-20T20:54:38.474766+0000","last_fullsized":"2024-05-20T20:54:38.474766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886666+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.886666+0000","last_peered":"2024-05-20T20:54:38.886666+0000","last_clean":"2024-05-20T20:54:38.886666+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:54:38.886666+0000","last_undegraded":"2024-05-20T20:54:38.886666+0000","last_fullsized":"2024-05-20T20:54:38.886666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887575+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:38.887575+0000","last_peered":"2024-05-20T20:54:38.887575+0000","last_clean":"2024-05-20T20:54:38.887575+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:38.887575+0000","last_undegraded":"2024-05-20T20:54:38.887575+0000","last_fullsized":"2024-05-20T20:54:38.887575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765538+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:54:38.765538+0000","last_peered":"2024-05-20T20:54:38.765538+0000","last_clean":"2024-05-20T20:54:38.765538+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:54:38.765538+0000","last_undegraded":"2024-05-20T20:54:38.765538+0000","last_fullsized":"2024-05-20T20:54:38.765538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887488+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:38.887488+0000","last_peered":"2024-05-20T20:54:38.887488+0000","last_clean":"2024-05-20T20:54:38.887488+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:38.887488+0000","last_undegraded":"2024-05-20T20:54:38.887488+0000","last_fullsized":"2024-05-20T20:54:38.887488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874582+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:37.874582+0000","last_peered":"2024-05-20T20:54:37.874582+0000","last_clean":"2024-05-20T20:54:37.874582+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:37.874582+0000","last_undegraded":"2024-05-20T20:54:37.874582+0000","last_fullsized":"2024-05-20T20:54:37.874582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":221,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:00.590496+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:56:00.590496+0000","last_peered":"2024-05-20T20:56:00.590496+0000","last_clean":"2024-05-20T20:56:00.590496+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:56:00.590496+0000","last_undegraded":"2024-05-20T20:56:00.590496+0000","last_fullsized":"2024-05-20T20:56:00.590496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887502+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:38.887502+0000","last_peered":"2024-05-20T20:54:38.887502+0000","last_clean":"2024-05-20T20:54:38.887502+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:38.887502+0000","last_undegraded":"2024-05-20T20:54:38.887502+0000","last_fullsized":"2024-05-20T20:54:38.887502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876310+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876310+0000","last_peered":"2024-05-20T20:54:37.876310+0000","last_clean":"2024-05-20T20:54:37.876310+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876310+0000","last_undegraded":"2024-05-20T20:54:37.876310+0000","last_fullsized":"2024-05-20T20:54:37.876310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874230+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:37.874230+0000","last_peered":"2024-05-20T20:54:37.874230+0000","last_clean":"2024-05-20T20:54:37.874230+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:37.874230+0000","last_undegraded":"2024-05-20T20:54:37.874230+0000","last_fullsized":"2024-05-20T20:54:37.874230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867681+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:37.867681+0000","last_peered":"2024-05-20T20:54:37.867681+0000","last_clean":"2024-05-20T20:54:37.867681+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:37.867681+0000","last_undegraded":"2024-05-20T20:54:37.867681+0000","last_fullsized":"2024-05-20T20:54:37.867681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474359+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:38.474359+0000","last_peered":"2024-05-20T20:54:38.474359+0000","last_clean":"2024-05-20T20:54:38.474359+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:38.474359+0000","last_undegraded":"2024-05-20T20:54:38.474359+0000","last_fullsized":"2024-05-20T20:54:38.474359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874492+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:37.874492+0000","last_peered":"2024-05-20T20:54:37.874492+0000","last_clean":"2024-05-20T20:54:37.874492+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:37.874492+0000","last_undegraded":"2024-05-20T20:54:37.874492+0000","last_fullsized":"2024-05-20T20:54:37.874492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872063+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872063+0000","last_peered":"2024-05-20T20:54:37.872063+0000","last_clean":"2024-05-20T20:54:37.872063+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872063+0000","last_undegraded":"2024-05-20T20:54:37.872063+0000","last_fullsized":"2024-05-20T20:54:37.872063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":218,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:03.594375+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:56:03.594375+0000","last_peered":"2024-05-20T20:56:03.594375+0000","last_clean":"2024-05-20T20:56:03.594375+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:56:03.594375+0000","last_undegraded":"2024-05-20T20:56:03.594375+0000","last_fullsized":"2024-05-20T20:56:03.594375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":183,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765842+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:54:38.765842+0000","last_peered":"2024-05-20T20:54:38.765842+0000","last_clean":"2024-05-20T20:54:38.765842+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:54:38.765842+0000","last_undegraded":"2024-05-20T20:54:38.765842+0000","last_fullsized":"2024-05-20T20:54:38.765842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871069+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871069+0000","last_peered":"2024-05-20T20:54:37.871069+0000","last_clean":"2024-05-20T20:54:37.871069+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871069+0000","last_undegraded":"2024-05-20T20:54:37.871069+0000","last_fullsized":"2024-05-20T20:54:37.871069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886717+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:38.886717+0000","last_peered":"2024-05-20T20:54:38.886717+0000","last_clean":"2024-05-20T20:54:38.886717+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:38.886717+0000","last_undegraded":"2024-05-20T20:54:38.886717+0000","last_fullsized":"2024-05-20T20:54:38.886717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874541+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:37.874541+0000","last_peered":"2024-05-20T20:54:37.874541+0000","last_clean":"2024-05-20T20:54:37.874541+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:37.874541+0000","last_undegraded":"2024-05-20T20:54:37.874541+0000","last_fullsized":"2024-05-20T20:54:37.874541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.765712+0000","last_peered":"2024-05-20T20:54:38.765712+0000","last_clean":"2024-05-20T20:54:38.765712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:54:38.765712+0000","last_undegraded":"2024-05-20T20:54:38.765712+0000","last_fullsized":"2024-05-20T20:54:38.765712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":448,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:04.595901+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:56:04.595901+0000","last_peered":"2024-05-20T20:56:04.595901+0000","last_clean":"2024-05-20T20:56:04.595901+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:56:04.595901+0000","last_undegraded":"2024-05-20T20:56:04.595901+0000","last_fullsized":"2024-05-20T20:56:04.595901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69793,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":380,"num_read_kb":258,"num_write":38,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887265+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:38.887265+0000","last_peered":"2024-05-20T20:54:38.887265+0000","last_clean":"2024-05-20T20:54:38.887265+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:38.887265+0000","last_undegraded":"2024-05-20T20:54:38.887265+0000","last_fullsized":"2024-05-20T20:54:38.887265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887299+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:38.887299+0000","last_peered":"2024-05-20T20:54:38.887299+0000","last_clean":"2024-05-20T20:54:38.887299+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:38.887299+0000","last_undegraded":"2024-05-20T20:54:38.887299+0000","last_fullsized":"2024-05-20T20:54:38.887299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867648+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:37.867648+0000","last_peered":"2024-05-20T20:54:37.867648+0000","last_clean":"2024-05-20T20:54:37.867648+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:37.867648+0000","last_undegraded":"2024-05-20T20:54:37.867648+0000","last_fullsized":"2024-05-20T20:54:37.867648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874116+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:37.874116+0000","last_peered":"2024-05-20T20:54:37.874116+0000","last_clean":"2024-05-20T20:54:37.874116+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:37.874116+0000","last_undegraded":"2024-05-20T20:54:37.874116+0000","last_fullsized":"2024-05-20T20:54:37.874116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871059+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.871059+0000","last_peered":"2024-05-20T20:54:37.871059+0000","last_clean":"2024-05-20T20:54:37.871059+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.871059+0000","last_undegraded":"2024-05-20T20:54:37.871059+0000","last_fullsized":"2024-05-20T20:54:37.871059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":222,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:00.616126+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:56:00.616126+0000","last_peered":"2024-05-20T20:56:00.616126+0000","last_clean":"2024-05-20T20:56:00.616126+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:56:00.616126+0000","last_undegraded":"2024-05-20T20:56:00.616126+0000","last_fullsized":"2024-05-20T20:56:00.616126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870598+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870598+0000","last_peered":"2024-05-20T20:54:37.870598+0000","last_clean":"2024-05-20T20:54:37.870598+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870598+0000","last_undegraded":"2024-05-20T20:54:37.870598+0000","last_fullsized":"2024-05-20T20:54:37.870598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886999+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:38.886999+0000","last_peered":"2024-05-20T20:54:38.886999+0000","last_clean":"2024-05-20T20:54:38.886999+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:38.886999+0000","last_undegraded":"2024-05-20T20:54:38.886999+0000","last_fullsized":"2024-05-20T20:54:38.886999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874169+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:37.874169+0000","last_peered":"2024-05-20T20:54:37.874169+0000","last_clean":"2024-05-20T20:54:37.874169+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:37.874169+0000","last_undegraded":"2024-05-20T20:54:37.874169+0000","last_fullsized":"2024-05-20T20:54:37.874169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887338+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:38.887338+0000","last_peered":"2024-05-20T20:54:38.887338+0000","last_clean":"2024-05-20T20:54:38.887338+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:38.887338+0000","last_undegraded":"2024-05-20T20:54:38.887338+0000","last_fullsized":"2024-05-20T20:54:38.887338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870840+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:54:37.870840+0000","last_peered":"2024-05-20T20:54:37.870840+0000","last_clean":"2024-05-20T20:54:37.870840+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:54:37.870840+0000","last_undegraded":"2024-05-20T20:54:37.870840+0000","last_fullsized":"2024-05-20T20:54:37.870840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872174+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872174+0000","last_peered":"2024-05-20T20:54:37.872174+0000","last_clean":"2024-05-20T20:54:37.872174+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872174+0000","last_undegraded":"2024-05-20T20:54:37.872174+0000","last_fullsized":"2024-05-20T20:54:37.872174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876474+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876474+0000","last_peered":"2024-05-20T20:54:37.876474+0000","last_clean":"2024-05-20T20:54:37.876474+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876474+0000","last_undegraded":"2024-05-20T20:54:37.876474+0000","last_fullsized":"2024-05-20T20:54:37.876474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871372+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871372+0000","last_peered":"2024-05-20T20:54:37.871372+0000","last_clean":"2024-05-20T20:54:37.871372+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871372+0000","last_undegraded":"2024-05-20T20:54:37.871372+0000","last_fullsized":"2024-05-20T20:54:37.871372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875765+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875765+0000","last_peered":"2024-05-20T20:54:37.875765+0000","last_clean":"2024-05-20T20:54:37.875765+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875765+0000","last_undegraded":"2024-05-20T20:54:37.875765+0000","last_fullsized":"2024-05-20T20:54:37.875765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872208+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872208+0000","last_peered":"2024-05-20T20:54:37.872208+0000","last_clean":"2024-05-20T20:54:37.872208+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872208+0000","last_undegraded":"2024-05-20T20:54:37.872208+0000","last_fullsized":"2024-05-20T20:54:37.872208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":217,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:00.615294+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:56:00.615294+0000","last_peered":"2024-05-20T20:56:00.615294+0000","last_clean":"2024-05-20T20:56:00.615294+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:56:00.615294+0000","last_undegraded":"2024-05-20T20:56:00.615294+0000","last_fullsized":"2024-05-20T20:56:00.615294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":180,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474755+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:38.474755+0000","last_peered":"2024-05-20T20:54:38.474755+0000","last_clean":"2024-05-20T20:54:38.474755+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:38.474755+0000","last_undegraded":"2024-05-20T20:54:38.474755+0000","last_fullsized":"2024-05-20T20:54:38.474755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475163+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:54:38.475163+0000","last_peered":"2024-05-20T20:54:38.475163+0000","last_clean":"2024-05-20T20:54:38.475163+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:54:38.475163+0000","last_undegraded":"2024-05-20T20:54:38.475163+0000","last_fullsized":"2024-05-20T20:54:38.475163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870715+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870715+0000","last_peered":"2024-05-20T20:54:37.870715+0000","last_clean":"2024-05-20T20:54:37.870715+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870715+0000","last_undegraded":"2024-05-20T20:54:37.870715+0000","last_fullsized":"2024-05-20T20:54:37.870715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867545+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:37.867545+0000","last_peered":"2024-05-20T20:54:37.867545+0000","last_clean":"2024-05-20T20:54:37.867545+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:37.867545+0000","last_undegraded":"2024-05-20T20:54:37.867545+0000","last_fullsized":"2024-05-20T20:54:37.867545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870913+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870913+0000","last_peered":"2024-05-20T20:54:37.870913+0000","last_clean":"2024-05-20T20:54:37.870913+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870913+0000","last_undegraded":"2024-05-20T20:54:37.870913+0000","last_fullsized":"2024-05-20T20:54:37.870913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887395+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:38.887395+0000","last_peered":"2024-05-20T20:54:38.887395+0000","last_clean":"2024-05-20T20:54:38.887395+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:38.887395+0000","last_undegraded":"2024-05-20T20:54:38.887395+0000","last_fullsized":"2024-05-20T20:54:38.887395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867898+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:37.867898+0000","last_peered":"2024-05-20T20:54:37.867898+0000","last_clean":"2024-05-20T20:54:37.867898+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:37.867898+0000","last_undegraded":"2024-05-20T20:54:37.867898+0000","last_fullsized":"2024-05-20T20:54:37.867898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.261304+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:55:45.261304+0000","last_peered":"2024-05-20T20:55:45.261304+0000","last_clean":"2024-05-20T20:55:45.261304+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:55:45.261304+0000","last_undegraded":"2024-05-20T20:55:45.261304+0000","last_fullsized":"2024-05-20T20:55:45.261304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474726+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:38.474726+0000","last_peered":"2024-05-20T20:54:38.474726+0000","last_clean":"2024-05-20T20:54:38.474726+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:38.474726+0000","last_undegraded":"2024-05-20T20:54:38.474726+0000","last_fullsized":"2024-05-20T20:54:38.474726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475338+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:54:38.475338+0000","last_peered":"2024-05-20T20:54:38.475338+0000","last_clean":"2024-05-20T20:54:38.475338+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:54:38.475338+0000","last_undegraded":"2024-05-20T20:54:38.475338+0000","last_fullsized":"2024-05-20T20:54:38.475338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886677+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:38.886677+0000","last_peered":"2024-05-20T20:54:38.886677+0000","last_clean":"2024-05-20T20:54:38.886677+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:38.886677+0000","last_undegraded":"2024-05-20T20:54:38.886677+0000","last_fullsized":"2024-05-20T20:54:38.886677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765487+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:54:38.765487+0000","last_peered":"2024-05-20T20:54:38.765487+0000","last_clean":"2024-05-20T20:54:38.765487+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:54:38.765487+0000","last_undegraded":"2024-05-20T20:54:38.765487+0000","last_fullsized":"2024-05-20T20:54:38.765487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887085+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:38.887085+0000","last_peered":"2024-05-20T20:54:38.887085+0000","last_clean":"2024-05-20T20:54:38.887085+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:38.887085+0000","last_undegraded":"2024-05-20T20:54:38.887085+0000","last_fullsized":"2024-05-20T20:54:38.887085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474392+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:54:38.474392+0000","last_peered":"2024-05-20T20:54:38.474392+0000","last_clean":"2024-05-20T20:54:38.474392+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:54:38.474392+0000","last_undegraded":"2024-05-20T20:54:38.474392+0000","last_fullsized":"2024-05-20T20:54:38.474392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870487+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:54:37.870487+0000","last_peered":"2024-05-20T20:54:37.870487+0000","last_clean":"2024-05-20T20:54:37.870487+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:54:37.870487+0000","last_undegraded":"2024-05-20T20:54:37.870487+0000","last_fullsized":"2024-05-20T20:54:37.870487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.325040+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:55:45.325040+0000","last_peered":"2024-05-20T20:55:45.325040+0000","last_clean":"2024-05-20T20:55:45.325040+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:55:45.325040+0000","last_undegraded":"2024-05-20T20:55:45.325040+0000","last_fullsized":"2024-05-20T20:55:45.325040+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474638+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:38.474638+0000","last_peered":"2024-05-20T20:54:38.474638+0000","last_clean":"2024-05-20T20:54:38.474638+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:38.474638+0000","last_undegraded":"2024-05-20T20:54:38.474638+0000","last_fullsized":"2024-05-20T20:54:38.474638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870566+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:54:37.870566+0000","last_peered":"2024-05-20T20:54:37.870566+0000","last_clean":"2024-05-20T20:54:37.870566+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:54:37.870566+0000","last_undegraded":"2024-05-20T20:54:37.870566+0000","last_fullsized":"2024-05-20T20:54:37.870566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.880369+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.880369+0000","last_peered":"2024-05-20T20:54:37.880369+0000","last_clean":"2024-05-20T20:54:37.880369+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.880369+0000","last_undegraded":"2024-05-20T20:54:37.880369+0000","last_fullsized":"2024-05-20T20:54:37.880369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887561+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:38.887561+0000","last_peered":"2024-05-20T20:54:38.887561+0000","last_clean":"2024-05-20T20:54:38.887561+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:38.887561+0000","last_undegraded":"2024-05-20T20:54:38.887561+0000","last_fullsized":"2024-05-20T20:54:38.887561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871302+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871302+0000","last_peered":"2024-05-20T20:54:37.871302+0000","last_clean":"2024-05-20T20:54:37.871302+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871302+0000","last_undegraded":"2024-05-20T20:54:37.871302+0000","last_fullsized":"2024-05-20T20:54:37.871302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874356+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:37.874356+0000","last_peered":"2024-05-20T20:54:37.874356+0000","last_clean":"2024-05-20T20:54:37.874356+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:37.874356+0000","last_undegraded":"2024-05-20T20:54:37.874356+0000","last_fullsized":"2024-05-20T20:54:37.874356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870975+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870975+0000","last_peered":"2024-05-20T20:54:37.870975+0000","last_clean":"2024-05-20T20:54:37.870975+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870975+0000","last_undegraded":"2024-05-20T20:54:37.870975+0000","last_fullsized":"2024-05-20T20:54:37.870975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867985+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:37.867985+0000","last_peered":"2024-05-20T20:54:37.867985+0000","last_clean":"2024-05-20T20:54:37.867985+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:37.867985+0000","last_undegraded":"2024-05-20T20:54:37.867985+0000","last_fullsized":"2024-05-20T20:54:37.867985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871280+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871280+0000","last_peered":"2024-05-20T20:54:37.871280+0000","last_clean":"2024-05-20T20:54:37.871280+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871280+0000","last_undegraded":"2024-05-20T20:54:37.871280+0000","last_fullsized":"2024-05-20T20:54:37.871280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870627+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:54:37.870627+0000","last_peered":"2024-05-20T20:54:37.870627+0000","last_clean":"2024-05-20T20:54:37.870627+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:54:37.870627+0000","last_undegraded":"2024-05-20T20:54:37.870627+0000","last_fullsized":"2024-05-20T20:54:37.870627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887334+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:38.887334+0000","last_peered":"2024-05-20T20:54:38.887334+0000","last_clean":"2024-05-20T20:54:38.887334+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:38.887334+0000","last_undegraded":"2024-05-20T20:54:38.887334+0000","last_fullsized":"2024-05-20T20:54:38.887334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887439+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:38.887439+0000","last_peered":"2024-05-20T20:54:38.887439+0000","last_clean":"2024-05-20T20:54:38.887439+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:38.887439+0000","last_undegraded":"2024-05-20T20:54:38.887439+0000","last_fullsized":"2024-05-20T20:54:38.887439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886573+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:38.886573+0000","last_peered":"2024-05-20T20:54:38.886573+0000","last_clean":"2024-05-20T20:54:38.886573+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:38.886573+0000","last_undegraded":"2024-05-20T20:54:38.886573+0000","last_fullsized":"2024-05-20T20:54:38.886573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867747+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:37.867747+0000","last_peered":"2024-05-20T20:54:37.867747+0000","last_clean":"2024-05-20T20:54:37.867747+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:37.867747+0000","last_undegraded":"2024-05-20T20:54:37.867747+0000","last_fullsized":"2024-05-20T20:54:37.867747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870785+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:54:37.870785+0000","last_peered":"2024-05-20T20:54:37.870785+0000","last_clean":"2024-05-20T20:54:37.870785+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:54:37.870785+0000","last_undegraded":"2024-05-20T20:54:37.870785+0000","last_fullsized":"2024-05-20T20:54:37.870785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887850+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:38.887850+0000","last_peered":"2024-05-20T20:54:38.887850+0000","last_clean":"2024-05-20T20:54:38.887850+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:38.887850+0000","last_undegraded":"2024-05-20T20:54:38.887850+0000","last_fullsized":"2024-05-20T20:54:38.887850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474618+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:38.474618+0000","last_peered":"2024-05-20T20:54:38.474618+0000","last_clean":"2024-05-20T20:54:38.474618+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:38.474618+0000","last_undegraded":"2024-05-20T20:54:38.474618+0000","last_fullsized":"2024-05-20T20:54:38.474618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874544+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:37.874544+0000","last_peered":"2024-05-20T20:54:37.874544+0000","last_clean":"2024-05-20T20:54:37.874544+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:37.874544+0000","last_undegraded":"2024-05-20T20:54:37.874544+0000","last_fullsized":"2024-05-20T20:54:37.874544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474329+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:38.474329+0000","last_peered":"2024-05-20T20:54:38.474329+0000","last_clean":"2024-05-20T20:54:38.474329+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:38.474329+0000","last_undegraded":"2024-05-20T20:54:38.474329+0000","last_fullsized":"2024-05-20T20:54:38.474329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766006+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:54:38.766006+0000","last_peered":"2024-05-20T20:54:38.766006+0000","last_clean":"2024-05-20T20:54:38.766006+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:54:38.766006+0000","last_undegraded":"2024-05-20T20:54:38.766006+0000","last_fullsized":"2024-05-20T20:54:38.766006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867894+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:37.867894+0000","last_peered":"2024-05-20T20:54:37.867894+0000","last_clean":"2024-05-20T20:54:37.867894+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:37.867894+0000","last_undegraded":"2024-05-20T20:54:37.867894+0000","last_fullsized":"2024-05-20T20:54:37.867894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870651+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:54:37.870651+0000","last_peered":"2024-05-20T20:54:37.870651+0000","last_clean":"2024-05-20T20:54:37.870651+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:54:37.870651+0000","last_undegraded":"2024-05-20T20:54:37.870651+0000","last_fullsized":"2024-05-20T20:54:37.870651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75312,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1125,"num_read_kb":643,"num_write":186,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":225936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":442424,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907005,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1104723,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070530,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442580,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1036028,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":442049,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":511635,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":443320,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":513757,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":442360,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":443623,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822742,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1034687,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:56:07.728 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-20T20:56:07.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T20:56:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:08.357 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:08 smithi026 ceph-mon[31990]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:08 smithi055 ceph-mon[35213]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:08 smithi144 ceph-mon[35660]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:09.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:09 smithi026 ceph-mon[31990]: pgmap v388: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-20T20:56:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:09 smithi055 ceph-mon[35213]: pgmap v388: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-20T20:56:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:09 smithi144 ceph-mon[35660]: pgmap v388: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-20T20:56:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 -- 172.21.15.26:0/336674655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac103f50 msgr2=0x7f0dac110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 --2- 172.21.15.26:0/336674655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac103f50 0x7f0dac110430 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0da400b0a0 tx=0x7f0da400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 -- 172.21.15.26:0/336674655 shutdown_connections 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 --2- 172.21.15.26:0/336674655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac103f50 0x7f0dac110430 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 --2- 172.21.15.26:0/336674655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dac1096a0 0x7f0dac109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 --2- 172.21.15.26:0/336674655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac103590 0x7f0dac103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 -- 172.21.15.26:0/336674655 >> 172.21.15.26:0/336674655 conn(0x7f0dac07a6f0 msgr2=0x7f0dac07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 -- 172.21.15.26:0/336674655 shutdown_connections 2024-05-20T20:56:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.740+0000 7f0db356d700 1 -- 172.21.15.26:0/336674655 wait complete. 2024-05-20T20:56:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 Processor -- start 2024-05-20T20:56:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 -- start start 2024-05-20T20:56:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac103590 0x7f0dac19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dac103f50 0x7f0dac19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 0x7f0dac199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac113620 con 0x7f0dac1096a0 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1134a0 con 0x7f0dac103f50 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db356d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1137a0 con 0x7f0dac103590 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db1b0a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 0x7f0dac199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db1b0a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 0x7f0dac199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44876/0 (socket says 172.21.15.26:44876) 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.741+0000 7f0db1b0a700 1 -- 172.21.15.26:0/2989949742 learned_addr learned my addr 172.21.15.26:0/2989949742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1309700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac103590 0x7f0dac19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db0b08700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dac103f50 0x7f0dac19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1b0a700 1 -- 172.21.15.26:0/2989949742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac103590 msgr2=0x7f0dac19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1b0a700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac103590 0x7f0dac19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1b0a700 1 -- 172.21.15.26:0/2989949742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dac103f50 msgr2=0x7f0dac19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1b0a700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dac103f50 0x7f0dac19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1b0a700 1 -- 172.21.15.26:0/2989949742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19a060 con 0x7f0dac1096a0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db1b0a700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 0x7f0dac199880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0da40062a0 tx=0x7f0da4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4021070 con 0x7f0dac1096a0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19a2f0 con 0x7f0dac1096a0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da4007c00 con 0x7f0dac1096a0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40052a0 con 0x7f0dac1096a0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.742+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a6510 con 0x7f0dac1096a0 2024-05-20T20:56:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.743+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac104cb0 con 0x7f0dac1096a0 2024-05-20T20:56:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.745+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f0da4024070 con 0x7f0dac1096a0 2024-05-20T20:56:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.745+0000 7f0da27fc700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d980b2910 0x7f0d980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.746+0000 7f0db1309700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d980b2910 0x7f0d980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.746+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0da400f020 con 0x7f0dac1096a0 2024-05-20T20:56:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.746+0000 7f0db1309700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d980b2910 0x7f0d980b4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009a10 tx=0x7f0da8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.749+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0da402b020 con 0x7f0dac1096a0 2024-05-20T20:56:09.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.967+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 --> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0dac063980 con 0x7f0d980b2910 2024-05-20T20:56:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.974+0000 7f0da27fc700 1 -- 172.21.15.26:0/2989949742 <== mgr.14150 v2:172.21.15.26:6800/2583575856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259080 (secure 0 0 0) 0x7f0dac063980 con 0x7f0d980b2910 2024-05-20T20:56:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:09.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d980b2910 msgr2=0x7f0d980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:09.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d980b2910 0x7f0d980b4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009a10 tx=0x7f0da8009190 comp rx=0 tx=0).stop 2024-05-20T20:56:09.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 msgr2=0x7f0dac199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 0x7f0dac199880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0da40062a0 tx=0x7f0da4004dc0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 shutdown_connections 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d980b2910 0x7f0d980b4dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dac1096a0 0x7f0dac199880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac103590 0x7f0dac19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 --2- 172.21.15.26:0/2989949742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dac103f50 0x7f0dac19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.977+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 >> 172.21.15.26:0/2989949742 conn(0x7f0dac07a6f0 msgr2=0x7f0dac102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.978+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 shutdown_connections 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:09.978+0000 7f0db356d700 1 -- 172.21.15.26:0/2989949742 wait complete. 2024-05-20T20:56:09.980 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-05-20T20:56:10.695 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-05-20T20:56:09.746457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665914,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1231,"num_read_kb":722,"num_write":311,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355984,"kb_used_data":10968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585840,"statfs":{"total":1151940427776,"available":1151575900160,"internally_reserved":0,"allocated":11231232,"data_stored":7299846,"data_compressed":249159,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":42,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870824+0000","last_change":"2024-05-20T20:52:45.224150+0000","last_active":"2024-05-20T20:54:37.870824+0000","last_peered":"2024-05-20T20:54:37.870824+0000","last_clean":"2024-05-20T20:54:37.870824+0000","last_became_active":"2024-05-20T20:52:45.223806+0000","last_became_peered":"2024-05-20T20:52:45.223806+0000","last_unstale":"2024-05-20T20:54:37.870824+0000","last_undegraded":"2024-05-20T20:54:37.870824+0000","last_fullsized":"2024-05-20T20:54:37.870824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:19.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886671+0000","last_change":"2024-05-20T20:52:45.226191+0000","last_active":"2024-05-20T20:54:38.886671+0000","last_peered":"2024-05-20T20:54:38.886671+0000","last_clean":"2024-05-20T20:54:38.886671+0000","last_became_active":"2024-05-20T20:52:45.223817+0000","last_became_peered":"2024-05-20T20:52:45.223817+0000","last_unstale":"2024-05-20T20:54:38.886671+0000","last_undegraded":"2024-05-20T20:54:38.886671+0000","last_fullsized":"2024-05-20T20:54:38.886671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:21:04.171407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886528+0000","last_change":"2024-05-20T20:52:45.221470+0000","last_active":"2024-05-20T20:54:38.886528+0000","last_peered":"2024-05-20T20:54:38.886528+0000","last_clean":"2024-05-20T20:54:38.886528+0000","last_became_active":"2024-05-20T20:52:45.221108+0000","last_became_peered":"2024-05-20T20:52:45.221108+0000","last_unstale":"2024-05-20T20:54:38.886528+0000","last_undegraded":"2024-05-20T20:54:38.886528+0000","last_fullsized":"2024-05-20T20:54:38.886528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:21.394617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475208+0000","last_change":"2024-05-20T20:52:45.217267+0000","last_active":"2024-05-20T20:54:38.475208+0000","last_peered":"2024-05-20T20:54:38.475208+0000","last_clean":"2024-05-20T20:54:38.475208+0000","last_became_active":"2024-05-20T20:52:45.216740+0000","last_became_peered":"2024-05-20T20:52:45.216740+0000","last_unstale":"2024-05-20T20:54:38.475208+0000","last_undegraded":"2024-05-20T20:54:38.475208+0000","last_fullsized":"2024-05-20T20:54:38.475208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:11.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887218+0000","last_change":"2024-05-20T20:52:45.221423+0000","last_active":"2024-05-20T20:54:38.887218+0000","last_peered":"2024-05-20T20:54:38.887218+0000","last_clean":"2024-05-20T20:54:38.887218+0000","last_became_active":"2024-05-20T20:52:45.219058+0000","last_became_peered":"2024-05-20T20:52:45.219058+0000","last_unstale":"2024-05-20T20:54:38.887218+0000","last_undegraded":"2024-05-20T20:54:38.887218+0000","last_fullsized":"2024-05-20T20:54:38.887218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:29.541615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876469+0000","last_change":"2024-05-20T20:52:45.218360+0000","last_active":"2024-05-20T20:54:37.876469+0000","last_peered":"2024-05-20T20:54:37.876469+0000","last_clean":"2024-05-20T20:54:37.876469+0000","last_became_active":"2024-05-20T20:52:45.217878+0000","last_became_peered":"2024-05-20T20:52:45.217878+0000","last_unstale":"2024-05-20T20:54:37.876469+0000","last_undegraded":"2024-05-20T20:54:37.876469+0000","last_fullsized":"2024-05-20T20:54:37.876469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:48.647002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870690+0000","last_change":"2024-05-20T20:52:45.229376+0000","last_active":"2024-05-20T20:54:37.870690+0000","last_peered":"2024-05-20T20:54:37.870690+0000","last_clean":"2024-05-20T20:54:37.870690+0000","last_became_active":"2024-05-20T20:52:45.229018+0000","last_became_peered":"2024-05-20T20:52:45.229018+0000","last_unstale":"2024-05-20T20:54:37.870690+0000","last_undegraded":"2024-05-20T20:54:37.870690+0000","last_fullsized":"2024-05-20T20:54:37.870690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:16.430644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.881195+0000","last_change":"2024-05-20T20:54:37.880853+0000","last_active":"2024-05-20T20:54:37.881195+0000","last_peered":"2024-05-20T20:54:37.881195+0000","last_clean":"2024-05-20T20:54:37.881195+0000","last_became_active":"2024-05-20T20:54:37.880536+0000","last_became_peered":"2024-05-20T20:54:37.880536+0000","last_unstale":"2024-05-20T20:54:37.881195+0000","last_undegraded":"2024-05-20T20:54:37.881195+0000","last_fullsized":"2024-05-20T20:54:37.881195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:46.697465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766030+0000","last_change":"2024-05-20T20:52:45.229512+0000","last_active":"2024-05-20T20:54:38.766030+0000","last_peered":"2024-05-20T20:54:38.766030+0000","last_clean":"2024-05-20T20:54:38.766030+0000","last_became_active":"2024-05-20T20:52:45.229173+0000","last_became_peered":"2024-05-20T20:52:45.229173+0000","last_unstale":"2024-05-20T20:54:38.766030+0000","last_undegraded":"2024-05-20T20:54:38.766030+0000","last_fullsized":"2024-05-20T20:54:38.766030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:58:45.024975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876712+0000","last_change":"2024-05-20T20:52:45.225473+0000","last_active":"2024-05-20T20:54:37.876712+0000","last_peered":"2024-05-20T20:54:37.876712+0000","last_clean":"2024-05-20T20:54:37.876712+0000","last_became_active":"2024-05-20T20:52:45.223654+0000","last_became_peered":"2024-05-20T20:52:45.223654+0000","last_unstale":"2024-05-20T20:54:37.876712+0000","last_undegraded":"2024-05-20T20:54:37.876712+0000","last_fullsized":"2024-05-20T20:54:37.876712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:58:19.587566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871157+0000","last_change":"2024-05-20T20:52:45.226589+0000","last_active":"2024-05-20T20:54:37.871157+0000","last_peered":"2024-05-20T20:54:37.871157+0000","last_clean":"2024-05-20T20:54:37.871157+0000","last_became_active":"2024-05-20T20:52:45.225530+0000","last_became_peered":"2024-05-20T20:52:45.225530+0000","last_unstale":"2024-05-20T20:54:37.871157+0000","last_undegraded":"2024-05-20T20:54:37.871157+0000","last_fullsized":"2024-05-20T20:54:37.871157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:00.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475180+0000","last_change":"2024-05-20T20:52:45.225840+0000","last_active":"2024-05-20T20:54:38.475180+0000","last_peered":"2024-05-20T20:54:38.475180+0000","last_clean":"2024-05-20T20:54:38.475180+0000","last_became_active":"2024-05-20T20:52:45.225499+0000","last_became_peered":"2024-05-20T20:52:45.225499+0000","last_unstale":"2024-05-20T20:54:38.475180+0000","last_undegraded":"2024-05-20T20:54:38.475180+0000","last_fullsized":"2024-05-20T20:54:38.475180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:09.308813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867869+0000","last_change":"2024-05-20T20:52:45.223534+0000","last_active":"2024-05-20T20:54:37.867869+0000","last_peered":"2024-05-20T20:54:37.867869+0000","last_clean":"2024-05-20T20:54:37.867869+0000","last_became_active":"2024-05-20T20:52:45.223220+0000","last_became_peered":"2024-05-20T20:52:45.223220+0000","last_unstale":"2024-05-20T20:54:37.867869+0000","last_undegraded":"2024-05-20T20:54:37.867869+0000","last_fullsized":"2024-05-20T20:54:37.867869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:16.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886436+0000","last_change":"2024-05-20T20:52:45.219391+0000","last_active":"2024-05-20T20:54:38.886436+0000","last_peered":"2024-05-20T20:54:38.886436+0000","last_clean":"2024-05-20T20:54:38.886436+0000","last_became_active":"2024-05-20T20:52:45.219003+0000","last_became_peered":"2024-05-20T20:52:45.219003+0000","last_unstale":"2024-05-20T20:54:38.886436+0000","last_undegraded":"2024-05-20T20:54:38.886436+0000","last_fullsized":"2024-05-20T20:54:38.886436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:33.168181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886364+0000","last_change":"2024-05-20T20:53:36.853545+0000","last_active":"2024-05-20T20:54:38.886364+0000","last_peered":"2024-05-20T20:54:38.886364+0000","last_clean":"2024-05-20T20:54:38.886364+0000","last_became_active":"2024-05-20T20:53:36.853019+0000","last_became_peered":"2024-05-20T20:53:36.853019+0000","last_unstale":"2024-05-20T20:54:38.886364+0000","last_undegraded":"2024-05-20T20:54:38.886364+0000","last_fullsized":"2024-05-20T20:54:38.886364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:05:50.100260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765447+0000","last_change":"2024-05-20T20:52:45.228676+0000","last_active":"2024-05-20T20:54:38.765447+0000","last_peered":"2024-05-20T20:54:38.765447+0000","last_clean":"2024-05-20T20:54:38.765447+0000","last_became_active":"2024-05-20T20:52:45.228308+0000","last_became_peered":"2024-05-20T20:52:45.228308+0000","last_unstale":"2024-05-20T20:54:38.765447+0000","last_undegraded":"2024-05-20T20:54:38.765447+0000","last_fullsized":"2024-05-20T20:54:38.765447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:50.851106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870916+0000","last_change":"2024-05-20T20:52:45.223148+0000","last_active":"2024-05-20T20:54:37.870916+0000","last_peered":"2024-05-20T20:54:37.870916+0000","last_clean":"2024-05-20T20:54:37.870916+0000","last_became_active":"2024-05-20T20:52:45.222800+0000","last_became_peered":"2024-05-20T20:52:45.222800+0000","last_unstale":"2024-05-20T20:54:37.870916+0000","last_undegraded":"2024-05-20T20:54:37.870916+0000","last_fullsized":"2024-05-20T20:54:37.870916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:27.185105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872609+0000","last_change":"2024-05-20T20:52:45.223756+0000","last_active":"2024-05-20T20:54:37.872609+0000","last_peered":"2024-05-20T20:54:37.872609+0000","last_clean":"2024-05-20T20:54:37.872609+0000","last_became_active":"2024-05-20T20:52:45.223464+0000","last_became_peered":"2024-05-20T20:52:45.223464+0000","last_unstale":"2024-05-20T20:54:37.872609+0000","last_undegraded":"2024-05-20T20:54:37.872609+0000","last_fullsized":"2024-05-20T20:54:37.872609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:58.230166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887790+0000","last_change":"2024-05-20T20:52:45.219628+0000","last_active":"2024-05-20T20:54:38.887790+0000","last_peered":"2024-05-20T20:54:38.887790+0000","last_clean":"2024-05-20T20:54:38.887790+0000","last_became_active":"2024-05-20T20:52:45.219238+0000","last_became_peered":"2024-05-20T20:52:45.219238+0000","last_unstale":"2024-05-20T20:54:38.887790+0000","last_undegraded":"2024-05-20T20:54:38.887790+0000","last_fullsized":"2024-05-20T20:54:38.887790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:28:44.014419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:48.257006+0000","last_change":"2024-05-20T20:52:42.197742+0000","last_active":"2024-05-20T20:54:48.257006+0000","last_peered":"2024-05-20T20:54:48.257006+0000","last_clean":"2024-05-20T20:54:48.257006+0000","last_became_active":"2024-05-20T20:52:42.197198+0000","last_became_peered":"2024-05-20T20:52:42.197198+0000","last_unstale":"2024-05-20T20:54:48.257006+0000","last_undegraded":"2024-05-20T20:54:48.257006+0000","last_fullsized":"2024-05-20T20:54:48.257006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:39.447952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474659+0000","last_change":"2024-05-20T20:52:45.226126+0000","last_active":"2024-05-20T20:54:38.474659+0000","last_peered":"2024-05-20T20:54:38.474659+0000","last_clean":"2024-05-20T20:54:38.474659+0000","last_became_active":"2024-05-20T20:52:45.225579+0000","last_became_peered":"2024-05-20T20:52:45.225579+0000","last_unstale":"2024-05-20T20:54:38.474659+0000","last_undegraded":"2024-05-20T20:54:38.474659+0000","last_fullsized":"2024-05-20T20:54:38.474659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:50.687391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474766+0000","last_change":"2024-05-20T20:52:43.010022+0000","last_active":"2024-05-20T20:54:38.474766+0000","last_peered":"2024-05-20T20:54:38.474766+0000","last_clean":"2024-05-20T20:54:38.474766+0000","last_became_active":"2024-05-20T20:52:43.009587+0000","last_became_peered":"2024-05-20T20:52:43.009587+0000","last_unstale":"2024-05-20T20:54:38.474766+0000","last_undegraded":"2024-05-20T20:54:38.474766+0000","last_fullsized":"2024-05-20T20:54:38.474766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:54:42.345234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886666+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.886666+0000","last_peered":"2024-05-20T20:54:38.886666+0000","last_clean":"2024-05-20T20:54:38.886666+0000","last_became_active":"2024-05-20T20:52:45.221177+0000","last_became_peered":"2024-05-20T20:52:45.221177+0000","last_unstale":"2024-05-20T20:54:38.886666+0000","last_undegraded":"2024-05-20T20:54:38.886666+0000","last_fullsized":"2024-05-20T20:54:38.886666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:12.817667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887575+0000","last_change":"2024-05-20T20:52:42.202762+0000","last_active":"2024-05-20T20:54:38.887575+0000","last_peered":"2024-05-20T20:54:38.887575+0000","last_clean":"2024-05-20T20:54:38.887575+0000","last_became_active":"2024-05-20T20:52:42.202383+0000","last_became_peered":"2024-05-20T20:52:42.202383+0000","last_unstale":"2024-05-20T20:54:38.887575+0000","last_undegraded":"2024-05-20T20:54:38.887575+0000","last_fullsized":"2024-05-20T20:54:38.887575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:11.709494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765538+0000","last_change":"2024-05-20T20:52:45.222890+0000","last_active":"2024-05-20T20:54:38.765538+0000","last_peered":"2024-05-20T20:54:38.765538+0000","last_clean":"2024-05-20T20:54:38.765538+0000","last_became_active":"2024-05-20T20:52:45.222552+0000","last_became_peered":"2024-05-20T20:52:45.222552+0000","last_unstale":"2024-05-20T20:54:38.765538+0000","last_undegraded":"2024-05-20T20:54:38.765538+0000","last_fullsized":"2024-05-20T20:54:38.765538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:38.906565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887488+0000","last_change":"2024-05-20T20:52:42.202154+0000","last_active":"2024-05-20T20:54:38.887488+0000","last_peered":"2024-05-20T20:54:38.887488+0000","last_clean":"2024-05-20T20:54:38.887488+0000","last_became_active":"2024-05-20T20:52:42.200551+0000","last_became_peered":"2024-05-20T20:52:42.200551+0000","last_unstale":"2024-05-20T20:54:38.887488+0000","last_undegraded":"2024-05-20T20:54:38.887488+0000","last_fullsized":"2024-05-20T20:54:38.887488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:31.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874582+0000","last_change":"2024-05-20T20:52:45.224544+0000","last_active":"2024-05-20T20:54:37.874582+0000","last_peered":"2024-05-20T20:54:37.874582+0000","last_clean":"2024-05-20T20:54:37.874582+0000","last_became_active":"2024-05-20T20:52:45.224163+0000","last_became_peered":"2024-05-20T20:52:45.224163+0000","last_unstale":"2024-05-20T20:54:37.874582+0000","last_undegraded":"2024-05-20T20:54:37.874582+0000","last_fullsized":"2024-05-20T20:54:37.874582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:46.304354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":226,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:05.598047+0000","last_change":"2024-05-20T20:52:42.201260+0000","last_active":"2024-05-20T20:56:05.598047+0000","last_peered":"2024-05-20T20:56:05.598047+0000","last_clean":"2024-05-20T20:56:05.598047+0000","last_became_active":"2024-05-20T20:52:42.200901+0000","last_became_peered":"2024-05-20T20:52:42.200901+0000","last_unstale":"2024-05-20T20:56:05.598047+0000","last_undegraded":"2024-05-20T20:56:05.598047+0000","last_fullsized":"2024-05-20T20:56:05.598047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:23:28.526444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887502+0000","last_change":"2024-05-20T20:53:36.851548+0000","last_active":"2024-05-20T20:54:38.887502+0000","last_peered":"2024-05-20T20:54:38.887502+0000","last_clean":"2024-05-20T20:54:38.887502+0000","last_became_active":"2024-05-20T20:53:36.851137+0000","last_became_peered":"2024-05-20T20:53:36.851137+0000","last_unstale":"2024-05-20T20:54:38.887502+0000","last_undegraded":"2024-05-20T20:54:38.887502+0000","last_fullsized":"2024-05-20T20:54:38.887502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:42:43.782412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876310+0000","last_change":"2024-05-20T20:52:42.197791+0000","last_active":"2024-05-20T20:54:37.876310+0000","last_peered":"2024-05-20T20:54:37.876310+0000","last_clean":"2024-05-20T20:54:37.876310+0000","last_became_active":"2024-05-20T20:52:42.197365+0000","last_became_peered":"2024-05-20T20:52:42.197365+0000","last_unstale":"2024-05-20T20:54:37.876310+0000","last_undegraded":"2024-05-20T20:54:37.876310+0000","last_fullsized":"2024-05-20T20:54:37.876310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:17.194475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874230+0000","last_change":"2024-05-20T20:53:36.877027+0000","last_active":"2024-05-20T20:54:37.874230+0000","last_peered":"2024-05-20T20:54:37.874230+0000","last_clean":"2024-05-20T20:54:37.874230+0000","last_became_active":"2024-05-20T20:53:36.876581+0000","last_became_peered":"2024-05-20T20:53:36.876581+0000","last_unstale":"2024-05-20T20:54:37.874230+0000","last_undegraded":"2024-05-20T20:54:37.874230+0000","last_fullsized":"2024-05-20T20:54:37.874230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:18:48.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867681+0000","last_change":"2024-05-20T20:52:42.190341+0000","last_active":"2024-05-20T20:54:37.867681+0000","last_peered":"2024-05-20T20:54:37.867681+0000","last_clean":"2024-05-20T20:54:37.867681+0000","last_became_active":"2024-05-20T20:52:42.189873+0000","last_became_peered":"2024-05-20T20:52:42.189873+0000","last_unstale":"2024-05-20T20:54:37.867681+0000","last_undegraded":"2024-05-20T20:54:37.867681+0000","last_fullsized":"2024-05-20T20:54:37.867681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:06.621605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474359+0000","last_change":"2024-05-20T20:52:45.208741+0000","last_active":"2024-05-20T20:54:38.474359+0000","last_peered":"2024-05-20T20:54:38.474359+0000","last_clean":"2024-05-20T20:54:38.474359+0000","last_became_active":"2024-05-20T20:52:45.208159+0000","last_became_peered":"2024-05-20T20:52:45.208159+0000","last_unstale":"2024-05-20T20:54:38.474359+0000","last_undegraded":"2024-05-20T20:54:38.474359+0000","last_fullsized":"2024-05-20T20:54:38.474359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:48.585263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874492+0000","last_change":"2024-05-20T20:52:42.197924+0000","last_active":"2024-05-20T20:54:37.874492+0000","last_peered":"2024-05-20T20:54:37.874492+0000","last_clean":"2024-05-20T20:54:37.874492+0000","last_became_active":"2024-05-20T20:52:42.197569+0000","last_became_peered":"2024-05-20T20:52:42.197569+0000","last_unstale":"2024-05-20T20:54:37.874492+0000","last_undegraded":"2024-05-20T20:54:37.874492+0000","last_fullsized":"2024-05-20T20:54:37.874492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:50.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872063+0000","last_change":"2024-05-20T20:52:45.223758+0000","last_active":"2024-05-20T20:54:37.872063+0000","last_peered":"2024-05-20T20:54:37.872063+0000","last_clean":"2024-05-20T20:54:37.872063+0000","last_became_active":"2024-05-20T20:52:45.223429+0000","last_became_peered":"2024-05-20T20:52:45.223429+0000","last_unstale":"2024-05-20T20:54:37.872063+0000","last_undegraded":"2024-05-20T20:54:37.872063+0000","last_fullsized":"2024-05-20T20:54:37.872063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:16.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":223,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:08.602258+0000","last_change":"2024-05-20T20:52:42.199725+0000","last_active":"2024-05-20T20:56:08.602258+0000","last_peered":"2024-05-20T20:56:08.602258+0000","last_clean":"2024-05-20T20:56:08.602258+0000","last_became_active":"2024-05-20T20:52:42.199346+0000","last_became_peered":"2024-05-20T20:52:42.199346+0000","last_unstale":"2024-05-20T20:56:08.602258+0000","last_undegraded":"2024-05-20T20:56:08.602258+0000","last_fullsized":"2024-05-20T20:56:08.602258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:32.565934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765842+0000","last_change":"2024-05-20T20:52:45.224735+0000","last_active":"2024-05-20T20:54:38.765842+0000","last_peered":"2024-05-20T20:54:38.765842+0000","last_clean":"2024-05-20T20:54:38.765842+0000","last_became_active":"2024-05-20T20:52:45.224140+0000","last_became_peered":"2024-05-20T20:52:45.224140+0000","last_unstale":"2024-05-20T20:54:38.765842+0000","last_undegraded":"2024-05-20T20:54:38.765842+0000","last_fullsized":"2024-05-20T20:54:38.765842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:56.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871069+0000","last_change":"2024-05-20T20:52:42.201603+0000","last_active":"2024-05-20T20:54:37.871069+0000","last_peered":"2024-05-20T20:54:37.871069+0000","last_clean":"2024-05-20T20:54:37.871069+0000","last_became_active":"2024-05-20T20:52:42.200558+0000","last_became_peered":"2024-05-20T20:52:42.200558+0000","last_unstale":"2024-05-20T20:54:37.871069+0000","last_undegraded":"2024-05-20T20:54:37.871069+0000","last_fullsized":"2024-05-20T20:54:37.871069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:38.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886717+0000","last_change":"2024-05-20T20:53:36.850581+0000","last_active":"2024-05-20T20:54:38.886717+0000","last_peered":"2024-05-20T20:54:38.886717+0000","last_clean":"2024-05-20T20:54:38.886717+0000","last_became_active":"2024-05-20T20:53:36.850212+0000","last_became_peered":"2024-05-20T20:53:36.850212+0000","last_unstale":"2024-05-20T20:54:38.886717+0000","last_undegraded":"2024-05-20T20:54:38.886717+0000","last_fullsized":"2024-05-20T20:54:38.886717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:28:11.790128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874541+0000","last_change":"2024-05-20T20:52:43.011099+0000","last_active":"2024-05-20T20:54:37.874541+0000","last_peered":"2024-05-20T20:54:37.874541+0000","last_clean":"2024-05-20T20:54:37.874541+0000","last_became_active":"2024-05-20T20:52:43.010523+0000","last_became_peered":"2024-05-20T20:52:43.010523+0000","last_unstale":"2024-05-20T20:54:37.874541+0000","last_undegraded":"2024-05-20T20:54:37.874541+0000","last_fullsized":"2024-05-20T20:54:37.874541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:20:54.530392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765712+0000","last_change":"2024-05-20T20:52:45.221789+0000","last_active":"2024-05-20T20:54:38.765712+0000","last_peered":"2024-05-20T20:54:38.765712+0000","last_clean":"2024-05-20T20:54:38.765712+0000","last_became_active":"2024-05-20T20:52:45.221402+0000","last_became_peered":"2024-05-20T20:52:45.221402+0000","last_unstale":"2024-05-20T20:54:38.765712+0000","last_undegraded":"2024-05-20T20:54:38.765712+0000","last_fullsized":"2024-05-20T20:54:38.765712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:39.615029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":458,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:09.603841+0000","last_change":"2024-05-20T20:52:42.202143+0000","last_active":"2024-05-20T20:56:09.603841+0000","last_peered":"2024-05-20T20:56:09.603841+0000","last_clean":"2024-05-20T20:56:09.603841+0000","last_became_active":"2024-05-20T20:52:42.201737+0000","last_became_peered":"2024-05-20T20:52:42.201737+0000","last_unstale":"2024-05-20T20:56:09.603841+0000","last_undegraded":"2024-05-20T20:56:09.603841+0000","last_fullsized":"2024-05-20T20:56:09.603841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:42.096207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69793,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":390,"num_read_kb":263,"num_write":38,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887265+0000","last_change":"2024-05-20T20:52:45.217570+0000","last_active":"2024-05-20T20:54:38.887265+0000","last_peered":"2024-05-20T20:54:38.887265+0000","last_clean":"2024-05-20T20:54:38.887265+0000","last_became_active":"2024-05-20T20:52:45.216934+0000","last_became_peered":"2024-05-20T20:52:45.216934+0000","last_unstale":"2024-05-20T20:54:38.887265+0000","last_undegraded":"2024-05-20T20:54:38.887265+0000","last_fullsized":"2024-05-20T20:54:38.887265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:01.228611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887299+0000","last_change":"2024-05-20T20:52:42.203262+0000","last_active":"2024-05-20T20:54:38.887299+0000","last_peered":"2024-05-20T20:54:38.887299+0000","last_clean":"2024-05-20T20:54:38.887299+0000","last_became_active":"2024-05-20T20:52:42.202874+0000","last_became_peered":"2024-05-20T20:52:42.202874+0000","last_unstale":"2024-05-20T20:54:38.887299+0000","last_undegraded":"2024-05-20T20:54:38.887299+0000","last_fullsized":"2024-05-20T20:54:38.887299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:24.702321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867648+0000","last_change":"2024-05-20T20:52:45.218442+0000","last_active":"2024-05-20T20:54:37.867648+0000","last_peered":"2024-05-20T20:54:37.867648+0000","last_clean":"2024-05-20T20:54:37.867648+0000","last_became_active":"2024-05-20T20:52:45.218082+0000","last_became_peered":"2024-05-20T20:52:45.218082+0000","last_unstale":"2024-05-20T20:54:37.867648+0000","last_undegraded":"2024-05-20T20:54:37.867648+0000","last_fullsized":"2024-05-20T20:54:37.867648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:20.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874116+0000","last_change":"2024-05-20T20:52:42.198864+0000","last_active":"2024-05-20T20:54:37.874116+0000","last_peered":"2024-05-20T20:54:37.874116+0000","last_clean":"2024-05-20T20:54:37.874116+0000","last_became_active":"2024-05-20T20:52:42.198436+0000","last_became_peered":"2024-05-20T20:52:42.198436+0000","last_unstale":"2024-05-20T20:54:37.874116+0000","last_undegraded":"2024-05-20T20:54:37.874116+0000","last_fullsized":"2024-05-20T20:54:37.874116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:52.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871059+0000","last_change":"2024-05-20T20:52:45.221575+0000","last_active":"2024-05-20T20:54:37.871059+0000","last_peered":"2024-05-20T20:54:37.871059+0000","last_clean":"2024-05-20T20:54:37.871059+0000","last_became_active":"2024-05-20T20:52:45.221172+0000","last_became_peered":"2024-05-20T20:52:45.221172+0000","last_unstale":"2024-05-20T20:54:37.871059+0000","last_undegraded":"2024-05-20T20:54:37.871059+0000","last_fullsized":"2024-05-20T20:54:37.871059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:30:55.734279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:05.625426+0000","last_change":"2024-05-20T20:52:42.199526+0000","last_active":"2024-05-20T20:56:05.625426+0000","last_peered":"2024-05-20T20:56:05.625426+0000","last_clean":"2024-05-20T20:56:05.625426+0000","last_became_active":"2024-05-20T20:52:42.198526+0000","last_became_peered":"2024-05-20T20:52:42.198526+0000","last_unstale":"2024-05-20T20:56:05.625426+0000","last_undegraded":"2024-05-20T20:56:05.625426+0000","last_fullsized":"2024-05-20T20:56:05.625426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:48.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870598+0000","last_change":"2024-05-20T20:52:45.231456+0000","last_active":"2024-05-20T20:54:37.870598+0000","last_peered":"2024-05-20T20:54:37.870598+0000","last_clean":"2024-05-20T20:54:37.870598+0000","last_became_active":"2024-05-20T20:52:45.231079+0000","last_became_peered":"2024-05-20T20:52:45.231079+0000","last_unstale":"2024-05-20T20:54:37.870598+0000","last_undegraded":"2024-05-20T20:54:37.870598+0000","last_fullsized":"2024-05-20T20:54:37.870598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:48:45.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886999+0000","last_change":"2024-05-20T20:53:36.889437+0000","last_active":"2024-05-20T20:54:38.886999+0000","last_peered":"2024-05-20T20:54:38.886999+0000","last_clean":"2024-05-20T20:54:38.886999+0000","last_became_active":"2024-05-20T20:53:36.877547+0000","last_became_peered":"2024-05-20T20:53:36.877547+0000","last_unstale":"2024-05-20T20:54:38.886999+0000","last_undegraded":"2024-05-20T20:54:38.886999+0000","last_fullsized":"2024-05-20T20:54:38.886999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:42.873569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874169+0000","last_change":"2024-05-20T20:52:45.226233+0000","last_active":"2024-05-20T20:54:37.874169+0000","last_peered":"2024-05-20T20:54:37.874169+0000","last_clean":"2024-05-20T20:54:37.874169+0000","last_became_active":"2024-05-20T20:52:45.225908+0000","last_became_peered":"2024-05-20T20:52:45.225908+0000","last_unstale":"2024-05-20T20:54:37.874169+0000","last_undegraded":"2024-05-20T20:54:37.874169+0000","last_fullsized":"2024-05-20T20:54:37.874169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:36.536026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887338+0000","last_change":"2024-05-20T20:52:42.201300+0000","last_active":"2024-05-20T20:54:38.887338+0000","last_peered":"2024-05-20T20:54:38.887338+0000","last_clean":"2024-05-20T20:54:38.887338+0000","last_became_active":"2024-05-20T20:52:42.200941+0000","last_became_peered":"2024-05-20T20:52:42.200941+0000","last_unstale":"2024-05-20T20:54:38.887338+0000","last_undegraded":"2024-05-20T20:54:38.887338+0000","last_fullsized":"2024-05-20T20:54:38.887338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:27.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870840+0000","last_change":"2024-05-20T20:53:36.876597+0000","last_active":"2024-05-20T20:54:37.870840+0000","last_peered":"2024-05-20T20:54:37.870840+0000","last_clean":"2024-05-20T20:54:37.870840+0000","last_became_active":"2024-05-20T20:53:36.876188+0000","last_became_peered":"2024-05-20T20:53:36.876188+0000","last_unstale":"2024-05-20T20:54:37.870840+0000","last_undegraded":"2024-05-20T20:54:37.870840+0000","last_fullsized":"2024-05-20T20:54:37.870840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:37.821933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872174+0000","last_change":"2024-05-20T20:52:43.011552+0000","last_active":"2024-05-20T20:54:37.872174+0000","last_peered":"2024-05-20T20:54:37.872174+0000","last_clean":"2024-05-20T20:54:37.872174+0000","last_became_active":"2024-05-20T20:52:43.010459+0000","last_became_peered":"2024-05-20T20:52:43.010459+0000","last_unstale":"2024-05-20T20:54:37.872174+0000","last_undegraded":"2024-05-20T20:54:37.872174+0000","last_fullsized":"2024-05-20T20:54:37.872174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:33.442358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.876474+0000","last_change":"2024-05-20T20:52:45.217698+0000","last_active":"2024-05-20T20:54:37.876474+0000","last_peered":"2024-05-20T20:54:37.876474+0000","last_clean":"2024-05-20T20:54:37.876474+0000","last_became_active":"2024-05-20T20:52:45.217306+0000","last_became_peered":"2024-05-20T20:52:45.217306+0000","last_unstale":"2024-05-20T20:54:37.876474+0000","last_undegraded":"2024-05-20T20:54:37.876474+0000","last_fullsized":"2024-05-20T20:54:37.876474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:33:59.817100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871372+0000","last_change":"2024-05-20T20:52:42.198011+0000","last_active":"2024-05-20T20:54:37.871372+0000","last_peered":"2024-05-20T20:54:37.871372+0000","last_clean":"2024-05-20T20:54:37.871372+0000","last_became_active":"2024-05-20T20:52:42.196714+0000","last_became_peered":"2024-05-20T20:52:42.196714+0000","last_unstale":"2024-05-20T20:54:37.871372+0000","last_undegraded":"2024-05-20T20:54:37.871372+0000","last_fullsized":"2024-05-20T20:54:37.871372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:00:02.913381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.875765+0000","last_change":"2024-05-20T20:52:45.222351+0000","last_active":"2024-05-20T20:54:37.875765+0000","last_peered":"2024-05-20T20:54:37.875765+0000","last_clean":"2024-05-20T20:54:37.875765+0000","last_became_active":"2024-05-20T20:52:45.222002+0000","last_became_peered":"2024-05-20T20:52:45.222002+0000","last_unstale":"2024-05-20T20:54:37.875765+0000","last_undegraded":"2024-05-20T20:54:37.875765+0000","last_fullsized":"2024-05-20T20:54:37.875765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:25:07.160722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.872208+0000","last_change":"2024-05-20T20:53:36.876638+0000","last_active":"2024-05-20T20:54:37.872208+0000","last_peered":"2024-05-20T20:54:37.872208+0000","last_clean":"2024-05-20T20:54:37.872208+0000","last_became_active":"2024-05-20T20:53:36.876190+0000","last_became_peered":"2024-05-20T20:53:36.876190+0000","last_unstale":"2024-05-20T20:54:37.872208+0000","last_undegraded":"2024-05-20T20:54:37.872208+0000","last_fullsized":"2024-05-20T20:54:37.872208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:27.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":222,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:56:05.624580+0000","last_change":"2024-05-20T20:52:43.009918+0000","last_active":"2024-05-20T20:56:05.624580+0000","last_peered":"2024-05-20T20:56:05.624580+0000","last_clean":"2024-05-20T20:56:05.624580+0000","last_became_active":"2024-05-20T20:52:43.009531+0000","last_became_peered":"2024-05-20T20:52:43.009531+0000","last_unstale":"2024-05-20T20:56:05.624580+0000","last_undegraded":"2024-05-20T20:56:05.624580+0000","last_fullsized":"2024-05-20T20:56:05.624580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:42:50.555937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474755+0000","last_change":"2024-05-20T20:52:45.224662+0000","last_active":"2024-05-20T20:54:38.474755+0000","last_peered":"2024-05-20T20:54:38.474755+0000","last_clean":"2024-05-20T20:54:38.474755+0000","last_became_active":"2024-05-20T20:52:45.224198+0000","last_became_peered":"2024-05-20T20:52:45.224198+0000","last_unstale":"2024-05-20T20:54:38.474755+0000","last_undegraded":"2024-05-20T20:54:38.474755+0000","last_fullsized":"2024-05-20T20:54:38.474755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:36:34.102587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475163+0000","last_change":"2024-05-20T20:52:42.195110+0000","last_active":"2024-05-20T20:54:38.475163+0000","last_peered":"2024-05-20T20:54:38.475163+0000","last_clean":"2024-05-20T20:54:38.475163+0000","last_became_active":"2024-05-20T20:52:42.194677+0000","last_became_peered":"2024-05-20T20:52:42.194677+0000","last_unstale":"2024-05-20T20:54:38.475163+0000","last_undegraded":"2024-05-20T20:54:38.475163+0000","last_fullsized":"2024-05-20T20:54:38.475163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:07:05.857380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":106,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870805+0000","last_change":"2024-05-20T20:51:20.785364+0000","last_active":"2024-05-20T20:54:37.870805+0000","last_peered":"2024-05-20T20:54:37.870805+0000","last_clean":"2024-05-20T20:54:37.870805+0000","last_became_active":"2024-05-20T20:51:20.779754+0000","last_became_peered":"2024-05-20T20:51:20.779754+0000","last_unstale":"2024-05-20T20:54:37.870805+0000","last_undegraded":"2024-05-20T20:54:37.870805+0000","last_fullsized":"2024-05-20T20:54:37.870805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:47:58.310267+0000","last_clean_scrub_stamp":"2024-05-20T20:47:58.310267+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:53:34.812111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867545+0000","last_change":"2024-05-20T20:52:45.212576+0000","last_active":"2024-05-20T20:54:37.867545+0000","last_peered":"2024-05-20T20:54:37.867545+0000","last_clean":"2024-05-20T20:54:37.867545+0000","last_became_active":"2024-05-20T20:52:45.212248+0000","last_became_peered":"2024-05-20T20:52:45.212248+0000","last_unstale":"2024-05-20T20:54:37.867545+0000","last_undegraded":"2024-05-20T20:54:37.867545+0000","last_fullsized":"2024-05-20T20:54:37.867545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:23.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870913+0000","last_change":"2024-05-20T20:52:42.197134+0000","last_active":"2024-05-20T20:54:37.870913+0000","last_peered":"2024-05-20T20:54:37.870913+0000","last_clean":"2024-05-20T20:54:37.870913+0000","last_became_active":"2024-05-20T20:52:42.196650+0000","last_became_peered":"2024-05-20T20:52:42.196650+0000","last_unstale":"2024-05-20T20:54:37.870913+0000","last_undegraded":"2024-05-20T20:54:37.870913+0000","last_fullsized":"2024-05-20T20:54:37.870913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:51:05.453662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887395+0000","last_change":"2024-05-20T20:52:43.010737+0000","last_active":"2024-05-20T20:54:38.887395+0000","last_peered":"2024-05-20T20:54:38.887395+0000","last_clean":"2024-05-20T20:54:38.887395+0000","last_became_active":"2024-05-20T20:52:43.010380+0000","last_became_peered":"2024-05-20T20:52:43.010380+0000","last_unstale":"2024-05-20T20:54:38.887395+0000","last_undegraded":"2024-05-20T20:54:38.887395+0000","last_fullsized":"2024-05-20T20:54:38.887395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.203037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867898+0000","last_change":"2024-05-20T20:52:45.213499+0000","last_active":"2024-05-20T20:54:37.867898+0000","last_peered":"2024-05-20T20:54:37.867898+0000","last_clean":"2024-05-20T20:54:37.867898+0000","last_became_active":"2024-05-20T20:52:45.213173+0000","last_became_peered":"2024-05-20T20:52:45.213173+0000","last_unstale":"2024-05-20T20:54:37.867898+0000","last_undegraded":"2024-05-20T20:54:37.867898+0000","last_fullsized":"2024-05-20T20:54:37.867898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:48.993262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.261304+0000","last_change":"2024-05-20T20:52:42.197707+0000","last_active":"2024-05-20T20:55:45.261304+0000","last_peered":"2024-05-20T20:55:45.261304+0000","last_clean":"2024-05-20T20:55:45.261304+0000","last_became_active":"2024-05-20T20:52:42.197085+0000","last_became_peered":"2024-05-20T20:52:42.197085+0000","last_unstale":"2024-05-20T20:55:45.261304+0000","last_undegraded":"2024-05-20T20:55:45.261304+0000","last_fullsized":"2024-05-20T20:55:45.261304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:20:24.121617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474726+0000","last_change":"2024-05-20T20:52:45.220235+0000","last_active":"2024-05-20T20:54:38.474726+0000","last_peered":"2024-05-20T20:54:38.474726+0000","last_clean":"2024-05-20T20:54:38.474726+0000","last_became_active":"2024-05-20T20:52:45.219847+0000","last_became_peered":"2024-05-20T20:52:45.219847+0000","last_unstale":"2024-05-20T20:54:38.474726+0000","last_undegraded":"2024-05-20T20:54:38.474726+0000","last_fullsized":"2024-05-20T20:54:38.474726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:00.348004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.475338+0000","last_change":"2024-05-20T20:52:42.997196+0000","last_active":"2024-05-20T20:54:38.475338+0000","last_peered":"2024-05-20T20:54:38.475338+0000","last_clean":"2024-05-20T20:54:38.475338+0000","last_became_active":"2024-05-20T20:52:42.996731+0000","last_became_peered":"2024-05-20T20:52:42.996731+0000","last_unstale":"2024-05-20T20:54:38.475338+0000","last_undegraded":"2024-05-20T20:54:38.475338+0000","last_fullsized":"2024-05-20T20:54:38.475338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:09.773385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886677+0000","last_change":"2024-05-20T20:52:45.222161+0000","last_active":"2024-05-20T20:54:38.886677+0000","last_peered":"2024-05-20T20:54:38.886677+0000","last_clean":"2024-05-20T20:54:38.886677+0000","last_became_active":"2024-05-20T20:52:45.221611+0000","last_became_peered":"2024-05-20T20:52:45.221611+0000","last_unstale":"2024-05-20T20:54:38.886677+0000","last_undegraded":"2024-05-20T20:54:38.886677+0000","last_fullsized":"2024-05-20T20:54:38.886677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:32.525823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.765487+0000","last_change":"2024-05-20T20:52:43.010643+0000","last_active":"2024-05-20T20:54:38.765487+0000","last_peered":"2024-05-20T20:54:38.765487+0000","last_clean":"2024-05-20T20:54:38.765487+0000","last_became_active":"2024-05-20T20:52:43.010268+0000","last_became_peered":"2024-05-20T20:52:43.010268+0000","last_unstale":"2024-05-20T20:54:38.765487+0000","last_undegraded":"2024-05-20T20:54:38.765487+0000","last_fullsized":"2024-05-20T20:54:38.765487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:36:32.633778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887085+0000","last_change":"2024-05-20T20:52:45.220988+0000","last_active":"2024-05-20T20:54:38.887085+0000","last_peered":"2024-05-20T20:54:38.887085+0000","last_clean":"2024-05-20T20:54:38.887085+0000","last_became_active":"2024-05-20T20:52:45.218688+0000","last_became_peered":"2024-05-20T20:52:45.218688+0000","last_unstale":"2024-05-20T20:54:38.887085+0000","last_undegraded":"2024-05-20T20:54:38.887085+0000","last_fullsized":"2024-05-20T20:54:38.887085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:56.117311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474392+0000","last_change":"2024-05-20T20:52:42.194629+0000","last_active":"2024-05-20T20:54:38.474392+0000","last_peered":"2024-05-20T20:54:38.474392+0000","last_clean":"2024-05-20T20:54:38.474392+0000","last_became_active":"2024-05-20T20:52:42.194360+0000","last_became_peered":"2024-05-20T20:52:42.194360+0000","last_unstale":"2024-05-20T20:54:38.474392+0000","last_undegraded":"2024-05-20T20:54:38.474392+0000","last_fullsized":"2024-05-20T20:54:38.474392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:50:52.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870487+0000","last_change":"2024-05-20T20:52:45.214278+0000","last_active":"2024-05-20T20:54:37.870487+0000","last_peered":"2024-05-20T20:54:37.870487+0000","last_clean":"2024-05-20T20:54:37.870487+0000","last_became_active":"2024-05-20T20:52:45.213778+0000","last_became_peered":"2024-05-20T20:52:45.213778+0000","last_unstale":"2024-05-20T20:54:37.870487+0000","last_undegraded":"2024-05-20T20:54:37.870487+0000","last_fullsized":"2024-05-20T20:54:37.870487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:41:23.516293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:55:45.325040+0000","last_change":"2024-05-20T20:53:36.853847+0000","last_active":"2024-05-20T20:55:45.325040+0000","last_peered":"2024-05-20T20:55:45.325040+0000","last_clean":"2024-05-20T20:55:45.325040+0000","last_became_active":"2024-05-20T20:53:36.848248+0000","last_became_peered":"2024-05-20T20:53:36.848248+0000","last_unstale":"2024-05-20T20:55:45.325040+0000","last_undegraded":"2024-05-20T20:55:45.325040+0000","last_fullsized":"2024-05-20T20:55:45.325040+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:29:31.621511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474638+0000","last_change":"2024-05-20T20:52:45.212453+0000","last_active":"2024-05-20T20:54:38.474638+0000","last_peered":"2024-05-20T20:54:38.474638+0000","last_clean":"2024-05-20T20:54:38.474638+0000","last_became_active":"2024-05-20T20:52:45.212126+0000","last_became_peered":"2024-05-20T20:52:45.212126+0000","last_unstale":"2024-05-20T20:54:38.474638+0000","last_undegraded":"2024-05-20T20:54:38.474638+0000","last_fullsized":"2024-05-20T20:54:38.474638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:40:32.058780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870566+0000","last_change":"2024-05-20T20:53:36.858200+0000","last_active":"2024-05-20T20:54:37.870566+0000","last_peered":"2024-05-20T20:54:37.870566+0000","last_clean":"2024-05-20T20:54:37.870566+0000","last_became_active":"2024-05-20T20:53:36.851269+0000","last_became_peered":"2024-05-20T20:53:36.851269+0000","last_unstale":"2024-05-20T20:54:37.870566+0000","last_undegraded":"2024-05-20T20:54:37.870566+0000","last_fullsized":"2024-05-20T20:54:37.870566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:08:08.825130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.880369+0000","last_change":"2024-05-20T20:54:37.876093+0000","last_active":"2024-05-20T20:54:37.880369+0000","last_peered":"2024-05-20T20:54:37.880369+0000","last_clean":"2024-05-20T20:54:37.880369+0000","last_became_active":"2024-05-20T20:54:37.875607+0000","last_became_peered":"2024-05-20T20:54:37.875607+0000","last_unstale":"2024-05-20T20:54:37.880369+0000","last_undegraded":"2024-05-20T20:54:37.880369+0000","last_fullsized":"2024-05-20T20:54:37.880369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:32.939181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887561+0000","last_change":"2024-05-20T20:52:43.011931+0000","last_active":"2024-05-20T20:54:38.887561+0000","last_peered":"2024-05-20T20:54:38.887561+0000","last_clean":"2024-05-20T20:54:38.887561+0000","last_became_active":"2024-05-20T20:52:43.010365+0000","last_became_peered":"2024-05-20T20:52:43.010365+0000","last_unstale":"2024-05-20T20:54:38.887561+0000","last_undegraded":"2024-05-20T20:54:38.887561+0000","last_fullsized":"2024-05-20T20:54:38.887561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:22.755984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871302+0000","last_change":"2024-05-20T20:52:45.227873+0000","last_active":"2024-05-20T20:54:37.871302+0000","last_peered":"2024-05-20T20:54:37.871302+0000","last_clean":"2024-05-20T20:54:37.871302+0000","last_became_active":"2024-05-20T20:52:45.224799+0000","last_became_peered":"2024-05-20T20:52:45.224799+0000","last_unstale":"2024-05-20T20:54:37.871302+0000","last_undegraded":"2024-05-20T20:54:37.871302+0000","last_fullsized":"2024-05-20T20:54:37.871302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:18:29.168846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874356+0000","last_change":"2024-05-20T20:52:42.196276+0000","last_active":"2024-05-20T20:54:37.874356+0000","last_peered":"2024-05-20T20:54:37.874356+0000","last_clean":"2024-05-20T20:54:37.874356+0000","last_became_active":"2024-05-20T20:52:42.195769+0000","last_became_peered":"2024-05-20T20:52:42.195769+0000","last_unstale":"2024-05-20T20:54:37.874356+0000","last_undegraded":"2024-05-20T20:54:37.874356+0000","last_fullsized":"2024-05-20T20:54:37.874356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:29:15.115986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870975+0000","last_change":"2024-05-20T20:52:45.231197+0000","last_active":"2024-05-20T20:54:37.870975+0000","last_peered":"2024-05-20T20:54:37.870975+0000","last_clean":"2024-05-20T20:54:37.870975+0000","last_became_active":"2024-05-20T20:52:45.230021+0000","last_became_peered":"2024-05-20T20:52:45.230021+0000","last_unstale":"2024-05-20T20:54:37.870975+0000","last_undegraded":"2024-05-20T20:54:37.870975+0000","last_fullsized":"2024-05-20T20:54:37.870975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:41.566281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867985+0000","last_change":"2024-05-20T20:52:42.201336+0000","last_active":"2024-05-20T20:54:37.867985+0000","last_peered":"2024-05-20T20:54:37.867985+0000","last_clean":"2024-05-20T20:54:37.867985+0000","last_became_active":"2024-05-20T20:52:42.201011+0000","last_became_peered":"2024-05-20T20:52:42.201011+0000","last_unstale":"2024-05-20T20:54:37.867985+0000","last_undegraded":"2024-05-20T20:54:37.867985+0000","last_fullsized":"2024-05-20T20:54:37.867985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:41.168601+0000","last_clean_scrub_stamp":"2024-05-20T20:52:41.168601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:12.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.871280+0000","last_change":"2024-05-20T20:52:45.219424+0000","last_active":"2024-05-20T20:54:37.871280+0000","last_peered":"2024-05-20T20:54:37.871280+0000","last_clean":"2024-05-20T20:54:37.871280+0000","last_became_active":"2024-05-20T20:52:45.219052+0000","last_became_peered":"2024-05-20T20:52:45.219052+0000","last_unstale":"2024-05-20T20:54:37.871280+0000","last_undegraded":"2024-05-20T20:54:37.871280+0000","last_fullsized":"2024-05-20T20:54:37.871280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:22.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870627+0000","last_change":"2024-05-20T20:52:45.219644+0000","last_active":"2024-05-20T20:54:37.870627+0000","last_peered":"2024-05-20T20:54:37.870627+0000","last_clean":"2024-05-20T20:54:37.870627+0000","last_became_active":"2024-05-20T20:52:45.219307+0000","last_became_peered":"2024-05-20T20:52:45.219307+0000","last_unstale":"2024-05-20T20:54:37.870627+0000","last_undegraded":"2024-05-20T20:54:37.870627+0000","last_fullsized":"2024-05-20T20:54:37.870627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:10.037751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887334+0000","last_change":"2024-05-20T20:52:45.225227+0000","last_active":"2024-05-20T20:54:38.887334+0000","last_peered":"2024-05-20T20:54:38.887334+0000","last_clean":"2024-05-20T20:54:38.887334+0000","last_became_active":"2024-05-20T20:52:45.223322+0000","last_became_peered":"2024-05-20T20:52:45.223322+0000","last_unstale":"2024-05-20T20:54:38.887334+0000","last_undegraded":"2024-05-20T20:54:38.887334+0000","last_fullsized":"2024-05-20T20:54:38.887334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:12.893870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887439+0000","last_change":"2024-05-20T20:52:45.220547+0000","last_active":"2024-05-20T20:54:38.887439+0000","last_peered":"2024-05-20T20:54:38.887439+0000","last_clean":"2024-05-20T20:54:38.887439+0000","last_became_active":"2024-05-20T20:52:45.220195+0000","last_became_peered":"2024-05-20T20:52:45.220195+0000","last_unstale":"2024-05-20T20:54:38.887439+0000","last_undegraded":"2024-05-20T20:54:38.887439+0000","last_fullsized":"2024-05-20T20:54:38.887439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:49:28.262465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.886573+0000","last_change":"2024-05-20T20:53:36.876884+0000","last_active":"2024-05-20T20:54:38.886573+0000","last_peered":"2024-05-20T20:54:38.886573+0000","last_clean":"2024-05-20T20:54:38.886573+0000","last_became_active":"2024-05-20T20:53:36.876324+0000","last_became_peered":"2024-05-20T20:53:36.876324+0000","last_unstale":"2024-05-20T20:54:38.886573+0000","last_undegraded":"2024-05-20T20:54:38.886573+0000","last_fullsized":"2024-05-20T20:54:38.886573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:51:24.388004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867747+0000","last_change":"2024-05-20T20:52:45.215454+0000","last_active":"2024-05-20T20:54:37.867747+0000","last_peered":"2024-05-20T20:54:37.867747+0000","last_clean":"2024-05-20T20:54:37.867747+0000","last_became_active":"2024-05-20T20:52:45.215096+0000","last_became_peered":"2024-05-20T20:52:45.215096+0000","last_unstale":"2024-05-20T20:54:37.867747+0000","last_undegraded":"2024-05-20T20:54:37.867747+0000","last_fullsized":"2024-05-20T20:54:37.867747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:44.392073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870785+0000","last_change":"2024-05-20T20:52:45.225747+0000","last_active":"2024-05-20T20:54:37.870785+0000","last_peered":"2024-05-20T20:54:37.870785+0000","last_clean":"2024-05-20T20:54:37.870785+0000","last_became_active":"2024-05-20T20:52:45.225422+0000","last_became_peered":"2024-05-20T20:52:45.225422+0000","last_unstale":"2024-05-20T20:54:37.870785+0000","last_undegraded":"2024-05-20T20:54:37.870785+0000","last_fullsized":"2024-05-20T20:54:37.870785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:32.745216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.887850+0000","last_change":"2024-05-20T20:52:45.223687+0000","last_active":"2024-05-20T20:54:38.887850+0000","last_peered":"2024-05-20T20:54:38.887850+0000","last_clean":"2024-05-20T20:54:38.887850+0000","last_became_active":"2024-05-20T20:52:45.223330+0000","last_became_peered":"2024-05-20T20:52:45.223330+0000","last_unstale":"2024-05-20T20:54:38.887850+0000","last_undegraded":"2024-05-20T20:54:38.887850+0000","last_fullsized":"2024-05-20T20:54:38.887850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:51.897905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474618+0000","last_change":"2024-05-20T20:52:45.215429+0000","last_active":"2024-05-20T20:54:38.474618+0000","last_peered":"2024-05-20T20:54:38.474618+0000","last_clean":"2024-05-20T20:54:38.474618+0000","last_became_active":"2024-05-20T20:52:45.215085+0000","last_became_peered":"2024-05-20T20:52:45.215085+0000","last_unstale":"2024-05-20T20:54:38.474618+0000","last_undegraded":"2024-05-20T20:54:38.474618+0000","last_fullsized":"2024-05-20T20:54:38.474618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:15.850916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.874544+0000","last_change":"2024-05-20T20:52:45.222673+0000","last_active":"2024-05-20T20:54:37.874544+0000","last_peered":"2024-05-20T20:54:37.874544+0000","last_clean":"2024-05-20T20:54:37.874544+0000","last_became_active":"2024-05-20T20:52:45.222354+0000","last_became_peered":"2024-05-20T20:52:45.222354+0000","last_unstale":"2024-05-20T20:54:37.874544+0000","last_undegraded":"2024-05-20T20:54:37.874544+0000","last_fullsized":"2024-05-20T20:54:37.874544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:34.611722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.474329+0000","last_change":"2024-05-20T20:52:45.212649+0000","last_active":"2024-05-20T20:54:38.474329+0000","last_peered":"2024-05-20T20:54:38.474329+0000","last_clean":"2024-05-20T20:54:38.474329+0000","last_became_active":"2024-05-20T20:52:45.211086+0000","last_became_peered":"2024-05-20T20:52:45.211086+0000","last_unstale":"2024-05-20T20:54:38.474329+0000","last_undegraded":"2024-05-20T20:54:38.474329+0000","last_fullsized":"2024-05-20T20:54:38.474329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:26:06.233509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:38.766006+0000","last_change":"2024-05-20T20:52:45.228500+0000","last_active":"2024-05-20T20:54:38.766006+0000","last_peered":"2024-05-20T20:54:38.766006+0000","last_clean":"2024-05-20T20:54:38.766006+0000","last_became_active":"2024-05-20T20:52:45.228100+0000","last_became_peered":"2024-05-20T20:52:45.228100+0000","last_unstale":"2024-05-20T20:54:38.766006+0000","last_undegraded":"2024-05-20T20:54:38.766006+0000","last_fullsized":"2024-05-20T20:54:38.766006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:09.795343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.867894+0000","last_change":"2024-05-20T20:52:45.217914+0000","last_active":"2024-05-20T20:54:37.867894+0000","last_peered":"2024-05-20T20:54:37.867894+0000","last_clean":"2024-05-20T20:54:37.867894+0000","last_became_active":"2024-05-20T20:52:45.217583+0000","last_became_peered":"2024-05-20T20:52:45.217583+0000","last_unstale":"2024-05-20T20:54:37.867894+0000","last_undegraded":"2024-05-20T20:54:37.867894+0000","last_fullsized":"2024-05-20T20:54:37.867894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:32.426491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-20T20:54:37.870651+0000","last_change":"2024-05-20T20:52:45.215693+0000","last_active":"2024-05-20T20:54:37.870651+0000","last_peered":"2024-05-20T20:54:37.870651+0000","last_clean":"2024-05-20T20:54:37.870651+0000","last_became_active":"2024-05-20T20:52:45.215315+0000","last_became_peered":"2024-05-20T20:52:45.215315+0000","last_unstale":"2024-05-20T20:54:37.870651+0000","last_undegraded":"2024-05-20T20:54:37.870651+0000","last_fullsized":"2024-05-20T20:54:37.870651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T20:52:44.192201+0000","last_clean_scrub_stamp":"2024-05-20T20:52:44.192201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:20:12.976396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":253571,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75312,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1155,"num_read_kb":658,"num_write":186,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":225936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":442424,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907006,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1104723,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442660,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266759,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442580,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462989,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1036028,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":442049,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":511635,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":443320,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":513757,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":442360,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":443623,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822743,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1034687,"data_compressed":48280,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35063,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36689,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T20:56:10.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:56:10.958 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2514793923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 msgr2=0x7f9c5c102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2514793923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c102d30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009900 tx=0x7f9c5402a410 comp rx=0 tx=0).stop 2024-05-20T20:56:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2514793923 shutdown_connections 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2514793923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c1044c0 0x7f9c5c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2514793923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c5c103b00 0x7f9c5c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2514793923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c102d30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2514793923 >> 172.21.15.26:0/2514793923 conn(0x7f9c5c0fde80 msgr2=0x7f9c5c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2514793923 shutdown_connections 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.049+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2514793923 wait complete. 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 Processor -- start 2024-05-20T20:56:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 -- start start 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c5c103b00 0x7f9c5c074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c1044c0 0x7f9c5c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c10e3c0 con 0x7f9c5c102910 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c10e240 con 0x7f9c5c103b00 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c62b9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c10e540 con 0x7f9c5c1044c0 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c076800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44902/0 (socket says 172.21.15.26:44902) 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 -- 172.21.15.26:0/2524819323 learned_addr learned my addr 172.21.15.26:0/2524819323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 -- 172.21.15.26:0/2524819323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c1044c0 msgr2=0x7f9c5c075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c1044c0 0x7f9c5c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c6139b640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c5c103b00 0x7f9c5c074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 -- 172.21.15.26:0/2524819323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c5c103b00 msgr2=0x7f9c5c074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c5c103b00 0x7f9c5c074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 -- 172.21.15.26:0/2524819323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c075c10 con 0x7f9c5c102910 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.050+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c076800 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.051+0000 7f9c5affd640 1 -- 172.21.15.26:0/2524819323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5403f030 con 0x7f9c5c102910 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.051+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c1a2ca0 con 0x7f9c5c102910 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.051+0000 7f9c5affd640 1 -- 172.21.15.26:0/2524819323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c54003c90 con 0x7f9c5c102910 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.051+0000 7f9c5affd640 1 -- 172.21.15.26:0/2524819323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54035070 con 0x7f9c5c102910 2024-05-20T20:56:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.051+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1a3190 con 0x7f9c5c102910 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.053+0000 7f9c5affd640 1 -- 172.21.15.26:0/2524819323 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f9c54042030 con 0x7f9c5c102910 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.053+0000 7f9c66bfb740 1 Processor -- start 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 -- start start 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e6290 0x5579850ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850eb400 0x5579850ed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557985056c30 con 0x5579850e58a0 2024-05-20T20:56:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5579850467d0 con 0x5579850e6290 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55798504a810 con 0x5579850eb400 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850e5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44912/0 (socket says 172.21.15.26:44912) 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 -- 172.21.15.26:0/1120761527 learned_addr learned my addr 172.21.15.26:0/1120761527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e6290 0x5579850ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c5affd640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c440b2870 0x7f9c440b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6139b640 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850eb400 0x5579850ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 -- 172.21.15.26:0/1120761527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850eb400 msgr2=0x5579850ed890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850eb400 0x5579850ed890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 -- 172.21.15.26:0/1120761527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e6290 msgr2=0x5579850ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.054+0000 7f9c6239d640 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e6290 0x5579850ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c6239d640 1 -- 172.21.15.26:0/1120761527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557984d5b210 con 0x5579850e58a0 2024-05-20T20:56:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c6239d640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c440b2870 0x7f9c440b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c5affd640 1 -- 172.21.15.26:0/2524819323 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9c54102bc0 con 0x7f9c5c102910 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c6239d640 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850e5cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009ec0 tx=0x7f9c4c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631949b71609a1cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c43fff640 1 -- 172.21.15.26:0/1120761527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c020640 con 0x5579850e58a0 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c43fff640 1 -- 172.21.15.26:0/1120761527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c4c014870 con 0x5579850e58a0 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c43fff640 1 -- 172.21.15.26:0/1120761527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00be30 con 0x5579850e58a0 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.055+0000 7f9c6239d640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c440b2870 0x7f9c440b4d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c01d000 tx=0x7f9c4c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 -- 172.21.15.26:0/1120761527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 msgr2=0x5579850e5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850e5cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009ec0 tx=0x7f9c4c00b420 comp rx=0 tx=0).stop 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 -- 172.21.15.26:0/1120761527 shutdown_connections 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850eb400 0x5579850ed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e6290 0x5579850ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/1120761527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850e5cc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 -- 172.21.15.26:0/1120761527 >> 172.21.15.26:0/1120761527 conn(0x5579850e0f10 msgr2=0x5579850e1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 -- 172.21.15.26:0/1120761527 shutdown_connections 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 -- 172.21.15.26:0/1120761527 wait complete. 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 -- start start 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.056+0000 7f9c66bfb740 1 Processor -- start 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850eb400 0x5579850f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557984d2c790 con 0x5579850e58a0 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557985002890 con 0x5579850eb400 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579850ee3b0 con 0x5579850e6290 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 10 client.0 _ll_register_callbacks cb 0x557984d5aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44928/0 (socket says 172.21.15.26:44928) 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 -- 172.21.15.26:0/3587271253 learned_addr learned my addr 172.21.15.26:0/3587271253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c6139b640 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 -- 172.21.15.26:0/3587271253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 msgr2=0x5579850eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 -- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850eb400 msgr2=0x5579850f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850eb400 0x5579850f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 -- 172.21.15.26:0/3587271253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557984e27620 con 0x5579850e58a0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850efa00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c540c60a0 tx=0x7f9c5402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df07d71b7b6ffe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c417fa640 1 -- 172.21.15.26:0/3587271253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54049070 con 0x5579850e58a0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c417fa640 1 -- 172.21.15.26:0/3587271253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9c540ca480 con 0x5579850e58a0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.057+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4c014870 con 0x5579850e58a0 2024-05-20T20:56:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c417fa640 1 -- 172.21.15.26:0/3587271253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c540c9790 con 0x5579850e58a0 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579850f9ef0 con 0x5579850e58a0 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c417fa640 1 -- 172.21.15.26:0/3587271253 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c4c014870 con 0x5579850e58a0 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c417fa640 1 -- 172.21.15.26:0/3587271253 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9c540d29f0 con 0x5579850e58a0 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 20 client.15270 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x5579850fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 4 client.15270 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557985058b80 con 0x5579850fb670 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.058+0000 7f9c6139b640 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x5579850fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.059+0000 7f9c6139b640 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x5579850fdac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6788e3cefa267948 server_cookie=2d71e75eec533060 in_seq=0 out_seq=0 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.059+0000 7f9c417fa640 10 client.15270 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.059+0000 7f9c417fa640 1 -- 172.21.15.26:0/3587271253 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557985058b80 con 0x5579850fb670 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.059+0000 7f9c417fa640 10 client.15270 handle_command_reply: tid=0 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 client.15270 shutdown 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 msgr2=0x5579850efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850efa00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c540c60a0 tx=0x7f9c5402a410 comp rx=0 tx=0).stop 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 shutdown_connections 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x5579850fdac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850eb400 0x5579850f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c417fa640 0 client.15270 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3587271253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850e58a0 0x5579850efa00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 >> 172.21.15.26:0/3587271253 conn(0x5579850e0f10 msgr2=0x5579850e0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 shutdown_connections 2024-05-20T20:56:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3587271253 wait complete. 2024-05-20T20:56:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.060+0000 7f9c66bfb740 20 client.15270 trim_cache size 0 max 16384 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 Processor -- start 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 -- start start 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850efb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1aea00 con 0x5579850eb400 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1ac2d0 con 0x5579850e58a0 2024-05-20T20:56:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1aa120 con 0x5579850e6290 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c6239d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c6239d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850efb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44944/0 (socket says 172.21.15.26:44944) 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c6239d640 1 -- 172.21.15.26:0/3558873453 learned_addr learned my addr 172.21.15.26:0/3558873453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.075+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6139b640 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6239d640 1 -- 172.21.15.26:0/3558873453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 msgr2=0x5579850f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6239d640 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6239d640 1 -- 172.21.15.26:0/3558873453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 msgr2=0x5579850ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6239d640 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6239d640 1 -- 172.21.15.26:0/3558873453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579850e4a40 con 0x5579850eb400 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c6239d640 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850efb50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009570 tx=0x7f9c4c009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde47c08cf884c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c437fe640 1 -- 172.21.15.26:0/3558873453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c025070 con 0x5579850eb400 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c437fe640 1 -- 172.21.15.26:0/3558873453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c4c02f070 con 0x5579850eb400 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c437fe640 1 -- 172.21.15.26:0/3558873453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c02a7c0 con 0x5579850eb400 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3558873453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 msgr2=0x5579850efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.076+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850efb50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009570 tx=0x7f9c4c009690 comp rx=0 tx=0).stop 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3558873453 shutdown_connections 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850efb50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/3558873453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3558873453 >> 172.21.15.26:0/3558873453 conn(0x5579850e0f10 msgr2=0x5579850e3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3558873453 shutdown_connections 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- 172.21.15.26:0/3558873453 wait complete. 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- start start 2024-05-20T20:56:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 Processor -- start 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1aa790 con 0x5579850eb400 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1abc00 con 0x5579850e58a0 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1aae60 con 0x5579850e6290 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c66bfb740 10 client.0 _ll_register_callbacks cb 0x7f9c54009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c6239d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c6239d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44952/0 (socket says 172.21.15.26:44952) 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.077+0000 7f9c6239d640 1 -- 172.21.15.26:0/2229413535 learned_addr learned my addr 172.21.15.26:0/2229413535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6139b640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6239d640 1 -- 172.21.15.26:0/2229413535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 msgr2=0x5579850f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6239d640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6239d640 1 -- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 msgr2=0x5579850f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6239d640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6239d640 1 -- 172.21.15.26:0/2229413535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579850f60a0 con 0x5579850eb400 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6139b640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c6239d640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850f5ab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004c00 tx=0x7f9c4c013a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a213d81e0156861d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c417fa640 1 -- 172.21.15.26:0/2229413535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c025030 con 0x5579850eb400 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c417fa640 1 -- 172.21.15.26:0/2229413535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9c4c02a740 con 0x5579850eb400 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4c02f070 con 0x5579850eb400 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c417fa640 1 -- 172.21.15.26:0/2229413535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c02a940 con 0x5579850eb400 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c66bfb740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579851662a0 con 0x5579850eb400 2024-05-20T20:56:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.078+0000 7f9c417fa640 1 -- 172.21.15.26:0/2229413535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c4c02f070 con 0x5579850eb400 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c417fa640 1 -- 172.21.15.26:0/2229413535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9c4c020a60 con 0x5579850eb400 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c66bfb740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c66bfb740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c66bfb740 20 client.15276 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x557985168a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c66bfb740 4 client.15276 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557985052360 con 0x5579850fb670 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x557985168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c61b9c640 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x557985168a10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4006de2a05d6063 server_cookie=14f8441e9a9fa37a in_seq=0 out_seq=0 2024-05-20T20:56:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.079+0000 7f9c417fa640 10 client.15276 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c417fa640 1 -- 172.21.15.26:0/2229413535 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557985052360 con 0x5579850fb670 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c417fa640 10 client.15276 handle_command_reply: tid=0 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c66bfb740 1 client.15276 shutdown 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 msgr2=0x5579850f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850f5ab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004c00 tx=0x7f9c4c013a50 comp rx=0 tx=0).stop 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 shutdown_connections 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.091+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5579850fb670 0x557985168a10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5579850eb400 0x5579850f5ab0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579850e6290 0x5579850f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 1 --2- 172.21.15.26:0/2229413535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5579850e58a0 0x5579850f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c417fa640 0 client.15276 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 >> 172.21.15.26:0/2229413535 conn(0x5579850e0f10 msgr2=0x5579850e3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 shutdown_connections 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 1 -- 172.21.15.26:0/2229413535 wait complete. 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c66bfb740 20 client.15276 trim_cache size 0 max 16384 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:56:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:56:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c440b2870 msgr2=0x7f9c440b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c440b2870 0x7f9c440b4d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c01d000 tx=0x7f9c4c01e000 comp rx=0 tx=0).stop 2024-05-20T20:56:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 msgr2=0x7f9c5c076800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c076800 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c540042f0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 shutdown_connections 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c440b2870 0x7f9c440b4d30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c1044c0 0x7f9c5c075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c5c103b00 0x7f9c5c074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 --2- 172.21.15.26:0/2524819323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c102910 0x7f9c5c076800 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 >> 172.21.15.26:0/2524819323 conn(0x7f9c5c0fde80 msgr2=0x7f9c5c107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 shutdown_connections 2024-05-20T20:56:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:11.092+0000 7f9c62b9e640 1 -- 172.21.15.26:0/2524819323 wait complete. 2024-05-20T20:56:11.407 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:11 smithi026 ceph-mon[31990]: pgmap v389: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-20T20:56:11.407 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:11 smithi026 ceph-mon[31990]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:11 smithi055 ceph-mon[35213]: pgmap v389: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-20T20:56:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:11 smithi055 ceph-mon[35213]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:11 smithi144 ceph-mon[35660]: pgmap v389: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-20T20:56:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:11 smithi144 ceph-mon[35660]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T20:56:12.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 -- 172.21.15.26:0/2061742457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c1084b0 msgr2=0x7f594c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 --2- 172.21.15.26:0/2061742457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c1084b0 0x7f594c108890 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f593c009990 tx=0x7f593c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 -- 172.21.15.26:0/2061742457 shutdown_connections 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 --2- 172.21.15.26:0/2061742457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c1084b0 0x7f594c108890 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 --2- 172.21.15.26:0/2061742457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f594c102e30 0x7f594c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 --2- 172.21.15.26:0/2061742457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f594c102470 0x7f594c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 -- 172.21.15.26:0/2061742457 >> 172.21.15.26:0/2061742457 conn(0x7f594c0784b0 msgr2=0x7f594c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 -- 172.21.15.26:0/2061742457 shutdown_connections 2024-05-20T20:56:12.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.631+0000 7f59510ed700 1 -- 172.21.15.26:0/2061742457 wait complete. 2024-05-20T20:56:12.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.632+0000 7f59510ed700 1 Processor -- start 2024-05-20T20:56:12.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.632+0000 7f59510ed700 1 -- start start 2024-05-20T20:56:12.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 0x7f594c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:12.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f594c102e30 0x7f594c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:12.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f594c1084b0 0x7f594c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f594c112490 con 0x7f594c102470 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f594c112310 con 0x7f594c102e30 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f594c112610 con 0x7f594c1084b0 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 0x7f594c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 0x7f594c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44974/0 (socket says 172.21.15.26:44974) 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 -- 172.21.15.26:0/2133425781 learned_addr learned my addr 172.21.15.26:0/2133425781 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594a59c700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f594c102e30 0x7f594c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594b59e700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f594c1084b0 0x7f594c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 -- 172.21.15.26:0/2133425781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f594c1084b0 msgr2=0x7f594c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f594c1084b0 0x7f594c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 -- 172.21.15.26:0/2133425781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f594c102e30 msgr2=0x7f594c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:12.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f594c102e30 0x7f594c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 -- 172.21.15.26:0/2133425781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c19a090 con 0x7f594c102470 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f594ad9d700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 0x7f594c19f770 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f593c0057c0 tx=0x7f593c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c027070 con 0x7f594c102470 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c19a320 con 0x7f594c102470 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f593c00fc00 con 0x7f594c102470 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.633+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c022b30 con 0x7f594c102470 2024-05-20T20:56:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.634+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c1a6640 con 0x7f594c102470 2024-05-20T20:56:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.635+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594c103b60 con 0x7f594c102470 2024-05-20T20:56:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.636+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f593c02a070 con 0x7f594c102470 2024-05-20T20:56:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.637+0000 7f5943fff700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59380b2910 0x7f59380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.637+0000 7f594a59c700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59380b2910 0x7f59380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.637+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f593c0e3520 con 0x7f594c102470 2024-05-20T20:56:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.637+0000 7f594a59c700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59380b2910 0x7f59380b4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f594c06cd90 tx=0x7f593400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.641+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f593c031080 con 0x7f594c102470 2024-05-20T20:56:12.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.917+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f594c1a6870 con 0x7f594c102470 2024-05-20T20:56:12.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.918+0000 7f5943fff700 1 -- 172.21.15.26:0/2133425781 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f593c013070 con 0x7f594c102470 2024-05-20T20:56:12.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:12.918 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59380b2910 msgr2=0x7f59380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59380b2910 0x7f59380b4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f594c06cd90 tx=0x7f593400b3c0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 msgr2=0x7f594c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 0x7f594c19f770 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f593c0057c0 tx=0x7f593c0057f0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 shutdown_connections 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59380b2910 0x7f59380b4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f594c102470 0x7f594c19f770 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f594c1084b0 0x7f594c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 --2- 172.21.15.26:0/2133425781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f594c102e30 0x7f594c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:12.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 >> 172.21.15.26:0/2133425781 conn(0x7f594c0784b0 msgr2=0x7f594c106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:12.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 shutdown_connections 2024-05-20T20:56:12.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:12.920+0000 7f59510ed700 1 -- 172.21.15.26:0/2133425781 wait complete. 2024-05-20T20:56:12.921 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:56:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:13 smithi026 ceph-mon[31990]: pgmap v390: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-20T20:56:13.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:13 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2133425781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:13 smithi144 ceph-mon[35660]: pgmap v390: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-20T20:56:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:13 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2133425781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:13.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:56:13.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:13 smithi055 ceph-mon[35213]: pgmap v390: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-20T20:56:13.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:13 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2133425781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:15.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:15 smithi026 ceph-mon[31990]: pgmap v391: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-20T20:56:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:15 smithi144 ceph-mon[35660]: pgmap v391: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-20T20:56:15.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:15 smithi055 ceph-mon[35213]: pgmap v391: 97 pgs: 97 active+clean; 650 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 -- 172.21.15.26:0/303916132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050102440 msgr2=0x7f90501028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 --2- 172.21.15.26:0/303916132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050102440 0x7f90501028c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9044009990 tx=0x7f9044009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 -- 172.21.15.26:0/303916132 shutdown_connections 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 --2- 172.21.15.26:0/303916132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9050102e00 0x7f905010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 --2- 172.21.15.26:0/303916132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050108520 0x7f9050108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 --2- 172.21.15.26:0/303916132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050102440 0x7f90501028c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 -- 172.21.15.26:0/303916132 >> 172.21.15.26:0/303916132 conn(0x7f90500793f0 msgr2=0x7f9050079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:15.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 -- 172.21.15.26:0/303916132 shutdown_connections 2024-05-20T20:56:15.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 -- 172.21.15.26:0/303916132 wait complete. 2024-05-20T20:56:15.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 Processor -- start 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.719+0000 7f9055e84700 1 -- start start 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f9055e84700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9050102440 0x7f905019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f9055e84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050102e00 0x7f905019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f9055e84700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 0x7f9050199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f9055e84700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90501124a0 con 0x7f9050108520 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f9055e84700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112320 con 0x7f9050102440 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f9055e84700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112620 con 0x7f9050102e00 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 0x7f9050199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 0x7f9050199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45002/0 (socket says 172.21.15.26:45002) 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 -- 172.21.15.26:0/561897128 learned_addr learned my addr 172.21.15.26:0/561897128 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904f7fe700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9050102440 0x7f905019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904effd700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050102e00 0x7f905019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 -- 172.21.15.26:0/561897128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050102e00 msgr2=0x7f905019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050102e00 0x7f905019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 -- 172.21.15.26:0/561897128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9050102440 msgr2=0x7f905019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9050102440 0x7f905019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 -- 172.21.15.26:0/561897128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905019a0c0 con 0x7f9050108520 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904effd700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050102e00 0x7f905019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.720+0000 7f904ffff700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 0x7f9050199960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f904800ef30 tx=0x7f904800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.721+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90480092e0 con 0x7f9050108520 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.721+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9048004bb0 con 0x7f9050108520 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.721+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905019a350 con 0x7f9050108520 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.721+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048004020 con 0x7f9050108520 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.721+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90501a6670 con 0x7f9050108520 2024-05-20T20:56:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.722+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905006a230 con 0x7f9050108520 2024-05-20T20:56:15.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.723+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f904801b030 con 0x7f9050108520 2024-05-20T20:56:15.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.724+0000 7f904cff9700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90380b2910 0x7f90380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:15.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.724+0000 7f904f7fe700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90380b2910 0x7f90380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:15.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.724+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f90480e2930 con 0x7f9050108520 2024-05-20T20:56:15.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.724+0000 7f904f7fe700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90380b2910 0x7f90380b4dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9040005950 tx=0x7f904000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:15.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.727+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9048022020 con 0x7f9050108520 2024-05-20T20:56:15.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.991+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f905002e560 con 0x7f9050108520 2024-05-20T20:56:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.992+0000 7f904cff9700 1 -- 172.21.15.26:0/561897128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f9048010040 con 0x7f9050108520 2024-05-20T20:56:15.992 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:15.992 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.994+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90380b2910 msgr2=0x7f90380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.994+0000 7f9055e84700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90380b2910 0x7f90380b4dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9040005950 tx=0x7f904000a300 comp rx=0 tx=0).stop 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.994+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 msgr2=0x7f9050199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.994+0000 7f9055e84700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 0x7f9050199960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f904800ef30 tx=0x7f904800c3a0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 shutdown_connections 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90380b2910 0x7f90380b4dd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9050108520 0x7f9050199960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9050102440 0x7f905019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 --2- 172.21.15.26:0/561897128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9050102e00 0x7f905019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 >> 172.21.15.26:0/561897128 conn(0x7f90500793f0 msgr2=0x7f905006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 shutdown_connections 2024-05-20T20:56:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:15.995+0000 7f9055e84700 1 -- 172.21.15.26:0/561897128 wait complete. 2024-05-20T20:56:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:56:16.358 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:16 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/561897128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:16.358 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:16 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:56:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:16 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/561897128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:16 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:56:16.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T20:56:16.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:16 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/561897128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:16 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:56:16.930 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:17.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:17 smithi026 ceph-mon[31990]: pgmap v392: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:17 smithi144 ceph-mon[35660]: pgmap v392: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:17 smithi055 ceph-mon[35213]: pgmap v392: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.685+0000 7f6293181700 1 -- 172.21.15.26:0/4293425842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102460 msgr2=0x7f628c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.685+0000 7f6293181700 1 --2- 172.21.15.26:0/4293425842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102460 0x7f628c1028e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f627c009990 tx=0x7f627c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 -- 172.21.15.26:0/4293425842 shutdown_connections 2024-05-20T20:56:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 --2- 172.21.15.26:0/4293425842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c102e20 0x7f628c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 --2- 172.21.15.26:0/4293425842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c1084f0 0x7f628c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 --2- 172.21.15.26:0/4293425842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102460 0x7f628c1028e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 -- 172.21.15.26:0/4293425842 >> 172.21.15.26:0/4293425842 conn(0x7f628c0794e0 msgr2=0x7f628c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 -- 172.21.15.26:0/4293425842 shutdown_connections 2024-05-20T20:56:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 -- 172.21.15.26:0/4293425842 wait complete. 2024-05-20T20:56:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 Processor -- start 2024-05-20T20:56:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.686+0000 7f6293181700 1 -- start start 2024-05-20T20:56:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6293181700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c102460 0x7f628c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6293181700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 0x7f628c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6293181700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c1084f0 0x7f628c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6293181700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112500 con 0x7f628c102e20 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6293181700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112380 con 0x7f628c1084f0 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6293181700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112680 con 0x7f628c102460 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 0x7f628c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 0x7f628c19c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45006/0 (socket says 172.21.15.26:45006) 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 -- 172.21.15.26:0/1806276635 learned_addr learned my addr 172.21.15.26:0/1806276635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f6290f1d700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c102460 0x7f628c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f629171e700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c1084f0 0x7f628c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 -- 172.21.15.26:0/1806276635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c102460 msgr2=0x7f628c19be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c102460 0x7f628c19be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 -- 172.21.15.26:0/1806276635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c1084f0 msgr2=0x7f628c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c1084f0 0x7f628c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 -- 172.21.15.26:0/1806276635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c1a0df0 con 0x7f628c102e20 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.687+0000 7f628bfff700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 0x7f628c19c370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f627c000c00 tx=0x7f627c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.688+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c027070 con 0x7f628c102e20 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.688+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f627c00fb00 con 0x7f628c102e20 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.688+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c1a1080 con 0x7f628c102e20 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.688+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c022a00 con 0x7f628c102e20 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.688+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1a8960 con 0x7f628c102e20 2024-05-20T20:56:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.689+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c06a230 con 0x7f628c102e20 2024-05-20T20:56:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.690+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f627c02a070 con 0x7f628c102e20 2024-05-20T20:56:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.691+0000 7f6289ffb700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62780b2910 0x7f62780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.691+0000 7f6290f1d700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62780b2910 0x7f62780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.691+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f627c0e3120 con 0x7f628c102e20 2024-05-20T20:56:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.692+0000 7f6290f1d700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62780b2910 0x7f62780b4dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f627400abd0 tx=0x7f627400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:18.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.695+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f627c05fdf0 con 0x7f628c102e20 2024-05-20T20:56:18.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.911+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f628c02e560 con 0x7f628c102e20 2024-05-20T20:56:18.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.912+0000 7f6289ffb700 1 -- 172.21.15.26:0/1806276635 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22791 (secure 0 0 0) 0x7f627c013070 con 0x7f628c102e20 2024-05-20T20:56:18.912 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:18.912 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T20:54:37.856459+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T20:56:18.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62780b2910 msgr2=0x7f62780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:18.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62780b2910 0x7f62780b4dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f627400abd0 tx=0x7f627400a380 comp rx=0 tx=0).stop 2024-05-20T20:56:18.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 msgr2=0x7f628c19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:18.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 0x7f628c19c370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f627c000c00 tx=0x7f627c0057a0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 shutdown_connections 2024-05-20T20:56:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c1084f0 0x7f628c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c102460 0x7f628c19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62780b2910 0x7f62780b4dd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 --2- 172.21.15.26:0/1806276635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f628c102e20 0x7f628c19c370 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 >> 172.21.15.26:0/1806276635 conn(0x7f628c0794e0 msgr2=0x7f628c06bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:18.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 shutdown_connections 2024-05-20T20:56:18.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:18.915+0000 7f6293181700 1 -- 172.21.15.26:0/1806276635 wait complete. 2024-05-20T20:56:19.503 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:19 smithi055 ceph-mon[35213]: pgmap v393: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:19.503 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:19 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1806276635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:56:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:19 smithi144 ceph-mon[35660]: pgmap v393: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:19 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1806276635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:56:19.615 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:19 smithi026 ceph-mon[31990]: pgmap v393: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:19.615 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:19 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1806276635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T20:56:19.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-20T20:56:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:21.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:21 smithi026 ceph-mon[31990]: pgmap v394: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:21 smithi144 ceph-mon[35660]: pgmap v394: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:21 smithi055 ceph-mon[35213]: pgmap v394: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:{"run_dir":"/var/run/ceph"} 2024-05-20T20:56:22.172 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-20T20:56:22.172 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 202 2024-05-20T20:56:22.172 DEBUG:tasks.quiescer:Yielding 2024-05-20T20:56:22.172 INFO:teuthology.run_tasks:Running task workunit... 2024-05-20T20:56:22.191 INFO:tasks.workunit:Pulling workunits from ref 4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:56:22.192 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-20T20:56:22.192 INFO:tasks.workunit:timeout=3h 2024-05-20T20:56:22.192 INFO:tasks.workunit:cleanup=True 2024-05-20T20:56:22.192 DEBUG:teuthology.orchestra.run.smithi026:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-20T20:56:22.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:56:22.211 INFO:teuthology.orchestra.run.smithi026.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-20T20:56:22.211 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-20T20:56:22.211 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6bh/107d Inode: 1 Links: 2 2024-05-20T20:56:22.211 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T20:56:22.211 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-20T20:56:22.211 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-20 20:53:11.452901791 +0000 2024-05-20T20:56:22.212 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-20 20:53:11.452901791 +0000 2024-05-20T20:56:22.212 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-20 20:55:45.076365839 +0000 2024-05-20T20:56:22.212 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-05-20T20:56:22.212 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-20T20:56:22.212 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-20T20:56:22.285 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4ebd1d5361d457738957dc0455bba90102296634 2024-05-20T20:56:22.342 INFO:tasks.workunit.client.0.smithi026.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-20T20:56:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:23 smithi144 ceph-mon[35660]: pgmap v395: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-20T20:56:23.596 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:23 smithi026 ceph-mon[31990]: pgmap v395: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-20T20:56:23.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:23 smithi055 ceph-mon[35213]: pgmap v395: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-20T20:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 -- 172.21.15.26:0/69022665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 msgr2=0x7f840010ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 --2- 172.21.15.26:0/69022665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f840010ac00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009ca0 tx=0x7f83f802a8c0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 -- 172.21.15.26:0/69022665 shutdown_connections 2024-05-20T20:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 --2- 172.21.15.26:0/69022665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f840010ac00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 --2- 172.21.15.26:0/69022665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8400105ee0 0x7f84001082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 --2- 172.21.15.26:0/69022665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84001035b0 0x7f84001059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 -- 172.21.15.26:0/69022665 >> 172.21.15.26:0/69022665 conn(0x7f84000fcf20 msgr2=0x7f84000ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 -- 172.21.15.26:0/69022665 shutdown_connections 2024-05-20T20:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.188+0000 7f8404924640 1 -- 172.21.15.26:0/69022665 wait complete. 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 Processor -- start 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 -- start start 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84001035b0 0x7f840019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8400105ee0 0x7f840019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f84001a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f840010e4e0 con 0x7f8400108810 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f840010e360 con 0x7f84001035b0 2024-05-20T20:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f8404924640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f840010e660 con 0x7f8400105ee0 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f84001a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f84001a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41032/0 (socket says 172.21.15.26:41032) 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 -- 172.21.15.26:0/2632035309 learned_addr learned my addr 172.21.15.26:0/2632035309 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fe59c640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84001035b0 0x7f840019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83f7fff640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8400105ee0 0x7f840019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 -- 172.21.15.26:0/2632035309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8400105ee0 msgr2=0x7f840019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8400105ee0 0x7f840019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 -- 172.21.15.26:0/2632035309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84001035b0 msgr2=0x7f840019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84001035b0 0x7f840019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fed9d640 1 -- 172.21.15.26:0/2632035309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84001a2960 con 0x7f8400108810 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.189+0000 7f83fe59c640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84001035b0 0x7f840019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.190+0000 7f83fed9d640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f84001a22c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f83f802aeb0 tx=0x7f83f8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.190+0000 7f83f77fe640 1 -- 172.21.15.26:0/2632035309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8003e10 con 0x7f8400108810 2024-05-20T20:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.190+0000 7f83f77fe640 1 -- 172.21.15.26:0/2632035309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83f80044d0 con 0x7f8400108810 2024-05-20T20:56:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.190+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84001a2bf0 con 0x7f8400108810 2024-05-20T20:56:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.190+0000 7f83f77fe640 1 -- 172.21.15.26:0/2632035309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8030c90 con 0x7f8400108810 2024-05-20T20:56:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.190+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84001a3140 con 0x7f8400108810 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.192+0000 7f83f77fe640 1 -- 172.21.15.26:0/2632035309 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f83f8004680 con 0x7f8400108810 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.192+0000 7f840763e740 1 Processor -- start 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 -- start start 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7501ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7506bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c75096b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567c7472c30 con 0x5567c75016c0 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5567c74627d0 con 0x5567c75020b0 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f840763e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567c7466810 con 0x5567c7507220 2024-05-20T20:56:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7501ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7501ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41044/0 (socket says 172.21.15.26:41044) 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 -- 172.21.15.26:0/1074100212 learned_addr learned my addr 172.21.15.26:0/1074100212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83fe59c640 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7506bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c75096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 -- 172.21.15.26:0/1074100212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 msgr2=0x5567c75096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c75096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 -- 172.21.15.26:0/1074100212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 msgr2=0x5567c7506bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7506bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.193+0000 7f83f7fff640 1 -- 172.21.15.26:0/1074100212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c7177210 con 0x5567c75016c0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83f77fe640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83ec0b2920 0x7f83ec0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83f7fff640 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7501ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009fa0 tx=0x7f83e80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e109b9a406d24e60 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83f4ff9640 1 -- 172.21.15.26:0/1074100212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8010490 con 0x5567c75016c0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83f4ff9640 1 -- 172.21.15.26:0/1074100212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83e8010ac0 con 0x5567c75016c0 2024-05-20T20:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83f4ff9640 1 -- 172.21.15.26:0/1074100212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800f810 con 0x5567c75016c0 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83fe59c640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83ec0b2920 0x7f83ec0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.194+0000 7f83f77fe640 1 -- 172.21.15.26:0/2632035309 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f83f8103a20 con 0x7f8400108810 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 -- 172.21.15.26:0/1074100212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 msgr2=0x5567c7501ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f83fe59c640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83ec0b2920 0x7f83ec0b4de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5567c75024b0 tx=0x7f83f000c510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7501ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009fa0 tx=0x7f83e80097b0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 -- 172.21.15.26:0/1074100212 shutdown_connections 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c75096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7506bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 --2- 172.21.15.26:0/1074100212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7501ae0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 -- 172.21.15.26:0/1074100212 >> 172.21.15.26:0/1074100212 conn(0x5567c74fcd30 msgr2=0x5567c74fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 -- 172.21.15.26:0/1074100212 shutdown_connections 2024-05-20T20:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 -- 172.21.15.26:0/1074100212 wait complete. 2024-05-20T20:56:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 -- start start 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.195+0000 7f840763e740 1 Processor -- start 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75016c0 0x5567c750b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 0x5567c750bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c7507220 0x5567c7512dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567c7148790 con 0x5567c75020b0 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5567c741e890 con 0x5567c7507220 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567c750a1d0 con 0x5567c75016c0 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f840763e740 10 client.0 _ll_register_callbacks cb 0x5567c7176a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 0x5567c750bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 0x5567c750bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41070/0 (socket says 172.21.15.26:41070) 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 -- 172.21.15.26:0/4251989461 learned_addr learned my addr 172.21.15.26:0/4251989461 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83fe59c640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75016c0 0x5567c750b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83fed9d640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c7507220 0x5567c7512dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 -- 172.21.15.26:0/4251989461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75016c0 msgr2=0x5567c750b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75016c0 0x5567c750b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 -- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c7507220 msgr2=0x5567c7512dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c7507220 0x5567c7512dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 -- 172.21.15.26:0/4251989461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c7243620 con 0x5567c75020b0 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83f7fff640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 0x5567c750bd90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f83e80106c0 tx=0x7f83e80106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bbce7ce2f60c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.196+0000 7f83e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.197+0000 7f83e5ffb640 1 -- 172.21.15.26:0/4251989461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8010910 con 0x5567c75020b0 2024-05-20T20:56:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 -- 172.21.15.26:0/54159544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5641084c0 msgr2=0x7ff5641088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 --2- 172.21.15.26:0/54159544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5641084c0 0x7ff5641088a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff558009990 tx=0x7ff558009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 -- 172.21.15.26:0/54159544 shutdown_connections 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 --2- 172.21.15.26:0/54159544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5641084c0 0x7ff5641088a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 --2- 172.21.15.26:0/54159544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff564103600 0x7ff56410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 --2- 172.21.15.26:0/54159544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff564102c40 0x7ff5641030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 -- 172.21.15.26:0/54159544 >> 172.21.15.26:0/54159544 conn(0x7ff564078ab0 msgr2=0x7ff5640ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 -- 172.21.15.26:0/54159544 shutdown_connections 2024-05-20T20:56:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.196+0000 7ff56abf2700 1 -- 172.21.15.26:0/54159544 wait complete. 2024-05-20T20:56:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.197+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e8010ac0 con 0x5567c75020b0 2024-05-20T20:56:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.197+0000 7f83e5ffb640 1 -- 172.21.15.26:0/4251989461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f83e8004050 con 0x5567c75020b0 2024-05-20T20:56:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.197+0000 7f83e5ffb640 1 -- 172.21.15.26:0/4251989461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83e8031050 con 0x5567c75020b0 2024-05-20T20:56:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.197+0000 7f840763e740 10 client.15309 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:24.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.197+0000 7ff56abf2700 1 Processor -- start 2024-05-20T20:56:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.197+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567c7515e00 con 0x5567c75020b0 2024-05-20T20:56:24.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.197+0000 7ff56abf2700 1 -- start start 2024-05-20T20:56:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f83e5ffb640 1 -- 172.21.15.26:0/4251989461 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83e8010ac0 con 0x5567c75020b0 2024-05-20T20:56:24.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56abf2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff564102c40 0x7ff5641a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56abf2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff564103600 0x7ff5641a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56abf2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 0x7ff5641a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56abf2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff564112480 con 0x7ff564103600 2024-05-20T20:56:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56abf2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff564112300 con 0x7ff564102c40 2024-05-20T20:56:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56abf2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff564112600 con 0x7ff5641084c0 2024-05-20T20:56:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f83e5ffb640 1 -- 172.21.15.26:0/4251989461 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f83e80046c0 con 0x5567c75020b0 2024-05-20T20:56:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f840763e740 10 client.15309 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f840763e740 10 client.15309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f840763e740 20 client.15309 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f840763e740 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7517390 0x5567c75197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f840763e740 4 client.15309 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c7474b80 con 0x5567c7517390 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff563fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff564103600 0x7ff5641a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 0x7ff5641a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56898e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff564102c40 0x7ff5641a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 0x7ff5641a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46046/0 (socket says 172.21.15.26:46046) 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 -- 172.21.15.26:0/662950659 learned_addr learned my addr 172.21.15.26:0/662950659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 -- 172.21.15.26:0/662950659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff564102c40 msgr2=0x7ff5641a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff564102c40 0x7ff5641a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 -- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff564103600 msgr2=0x7ff5641a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff564103600 0x7ff5641a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.198+0000 7ff56918f700 1 -- 172.21.15.26:0/662950659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5641a08e0 con 0x7ff5641084c0 2024-05-20T20:56:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.199+0000 7ff56918f700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 0x7ff5641a0160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff55c007ee0 tx=0x7ff55c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.198+0000 7f83fe59c640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7517390 0x5567c75197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.199+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00dd20 con 0x7ff5641084c0 2024-05-20T20:56:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.199+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5641a0bd0 con 0x7ff5641084c0 2024-05-20T20:56:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.199+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff55c004020 con 0x7ff5641084c0 2024-05-20T20:56:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.199+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c015660 con 0x7ff5641084c0 2024-05-20T20:56:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.199+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5641acc30 con 0x7ff5641084c0 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.199+0000 7f83fe59c640 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7517390 0x5567c75197b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b8b8f4040cb6a0 server_cookie=b637dc9b420c1f8d in_seq=0 out_seq=0 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.199+0000 7f83e5ffb640 10 client.15309 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f83e5ffb640 1 -- 172.21.15.26:0/4251989461 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5567c7474b80 con 0x5567c7517390 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f83e5ffb640 10 client.15309 handle_command_reply: tid=0 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 client.15309 shutdown 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 msgr2=0x5567c750bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 0x5567c750bd90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f83e80106c0 tx=0x7f83e80106f0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 shutdown_connections 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7517390 0x5567c75197b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c7507220 0x5567c7512dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75020b0 0x5567c750bd90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 --2- 172.21.15.26:0/4251989461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75016c0 0x5567c750b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 >> 172.21.15.26:0/4251989461 conn(0x5567c74fcd30 msgr2=0x5567c74fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f83e5ffb640 0 client.15309 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.200+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 shutdown_connections 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.201+0000 7f840763e740 1 -- 172.21.15.26:0/4251989461 wait complete. 2024-05-20T20:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.201+0000 7f840763e740 20 client.15309 trim_cache size 0 max 16384 2024-05-20T20:56:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.201+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff564103c50 con 0x7ff5641084c0 2024-05-20T20:56:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.202+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7ff55c003a00 con 0x7ff5641084c0 2024-05-20T20:56:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.203+0000 7ff561ffb700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5540b2910 0x7ff5540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.203+0000 7ff56898e700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5540b2910 0x7ff5540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.203+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff55c0e1e30 con 0x7ff5641084c0 2024-05-20T20:56:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.204+0000 7ff56898e700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5540b2910 0x7ff5540b4dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff55800b650 tx=0x7ff558005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.208+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff55c010070 con 0x7ff5641084c0 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 Processor -- start 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 -- start start 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7503330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7503870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c7512d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84001ae930 con 0x5567c75016c0 2024-05-20T20:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84001ac200 con 0x5567c75020b0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f840763e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84001aa050 con 0x5567c7507220 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7503330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7503330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41078/0 (socket says 172.21.15.26:41078) 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 -- 172.21.15.26:0/772135088 learned_addr learned my addr 172.21.15.26:0/772135088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fed9d640 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c7512d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83f7fff640 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7503870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 -- 172.21.15.26:0/772135088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 msgr2=0x5567c7512d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c7512d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 -- 172.21.15.26:0/772135088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 msgr2=0x5567c7503870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7503870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.217+0000 7f83fe59c640 1 -- 172.21.15.26:0/772135088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c74f86f0 con 0x5567c75016c0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.218+0000 7f83fe59c640 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7503330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f83f001b5c0 tx=0x7f83f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a3adadd160a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.218+0000 7f83e7fff640 1 -- 172.21.15.26:0/772135088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f002c070 con 0x5567c75016c0 2024-05-20T20:56:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.218+0000 7f83e7fff640 1 -- 172.21.15.26:0/772135088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83f0027bc0 con 0x5567c75016c0 2024-05-20T20:56:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.218+0000 7f83e7fff640 1 -- 172.21.15.26:0/772135088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00193e0 con 0x5567c75016c0 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- 172.21.15.26:0/772135088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 msgr2=0x5567c7503330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7503330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f83f001b5c0 tx=0x7f83f0014000 comp rx=0 tx=0).stop 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- 172.21.15.26:0/772135088 shutdown_connections 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c7507220 0x5567c7512d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75020b0 0x5567c7503870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- 172.21.15.26:0/772135088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c75016c0 0x5567c7503330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- 172.21.15.26:0/772135088 >> 172.21.15.26:0/772135088 conn(0x5567c74fcd30 msgr2=0x5567c74f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- 172.21.15.26:0/772135088 shutdown_connections 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- 172.21.15.26:0/772135088 wait complete. 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- start start 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 Processor -- start 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75016c0 0x5567c75101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75020b0 0x5567c75106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 0x5567c75116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84001aa6c0 con 0x5567c7507220 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84001abb30 con 0x5567c75016c0 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84001aad90 con 0x5567c75020b0 2024-05-20T20:56:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f840763e740 10 client.0 _ll_register_callbacks cb 0x7f83f000c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f83fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 0x5567c75116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f83fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 0x5567c75116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41084/0 (socket says 172.21.15.26:41084) 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f83fed9d640 1 -- 172.21.15.26:0/3901522718 learned_addr learned my addr 172.21.15.26:0/3901522718 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f83f7fff640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75020b0 0x5567c75106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.219+0000 7f83fe59c640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75016c0 0x5567c75101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83f7fff640 1 -- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75016c0 msgr2=0x5567c75101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83f7fff640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75016c0 0x5567c75101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83f7fff640 1 -- 172.21.15.26:0/3901522718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 msgr2=0x5567c75116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83f7fff640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 0x5567c75116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83f7fff640 1 -- 172.21.15.26:0/3901522718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c7511ce0 con 0x5567c75020b0 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83fed9d640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 0x5567c75116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83fe59c640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75016c0 0x5567c75101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83f7fff640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75020b0 0x5567c75106e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f83e8004c80 tx=0x7f83e8005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7432046aaeb985b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83e5ffb640 1 -- 172.21.15.26:0/3901522718 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8004890 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f0027bc0 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83e5ffb640 1 -- 172.21.15.26:0/3901522718 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f83e8010760 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f83e5ffb640 1 -- 172.21.15.26:0/3901522718 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83e8009630 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f840763e740 10 client.24940 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.220+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567c7512ff0 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f83e5ffb640 1 -- 172.21.15.26:0/3901522718 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83f0027bc0 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f83e5ffb640 1 -- 172.21.15.26:0/3901522718 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f83e803d020 con 0x5567c75020b0 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f840763e740 10 client.24940 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f840763e740 10 client.24940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f840763e740 20 client.24940 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f840763e740 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7582640 0x5567c7584a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f840763e740 4 client.24940 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:56:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567c746e360 con 0x5567c7582640 2024-05-20T20:56:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.221+0000 7f83fe59c640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7582640 0x5567c7584a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.222+0000 7f83fe59c640 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7582640 0x5567c7584a60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2711a03d5040353 server_cookie=cd490c9338903e8d in_seq=0 out_seq=0 2024-05-20T20:56:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.222+0000 7f83e5ffb640 10 client.24940 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f83e5ffb640 1 -- 172.21.15.26:0/3901522718 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5567c746e360 con 0x5567c7582640 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f83e5ffb640 10 client.24940 handle_command_reply: tid=0 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 client.24940 shutdown 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75020b0 msgr2=0x5567c75106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75020b0 0x5567c75106e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f83e8004c80 tx=0x7f83e8005870 comp rx=0 tx=0).stop 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 shutdown_connections 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567c7582640 0x5567c7584a60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567c7507220 0x5567c75116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f83e5ffb640 0 client.24940 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567c75020b0 0x5567c75106e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 --2- 172.21.15.26:0/3901522718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c75016c0 0x5567c75101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.230+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 >> 172.21.15.26:0/3901522718 conn(0x5567c74fcd30 msgr2=0x5567c750b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 shutdown_connections 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f840763e740 1 -- 172.21.15.26:0/3901522718 wait complete. 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f840763e740 20 client.24940 trim_cache size 0 max 16384 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:56:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:56:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83ec0b2920 msgr2=0x7f83ec0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83ec0b2920 0x7f83ec0b4de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5567c75024b0 tx=0x7f83f000c510 comp rx=0 tx=0).stop 2024-05-20T20:56:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 msgr2=0x7f84001a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f84001a22c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f83f802aeb0 tx=0x7f83f8003c20 comp rx=0 tx=0).stop 2024-05-20T20:56:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 shutdown_connections 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83ec0b2920 0x7f83ec0b4de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8400108810 0x7f84001a22c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8400105ee0 0x7f840019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 --2- 172.21.15.26:0/2632035309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84001035b0 0x7f840019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.231+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 >> 172.21.15.26:0/2632035309 conn(0x7f84000fcf20 msgr2=0x7f8400102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.232+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 shutdown_connections 2024-05-20T20:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:24.232+0000 7f8404924640 1 -- 172.21.15.26:0/2632035309 wait complete. 2024-05-20T20:56:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.475+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5641ace00 con 0x7ff5641084c0 2024-05-20T20:56:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.476+0000 7ff561ffb700 1 -- 172.21.15.26:0/662950659 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7ff55c0a9930 con 0x7ff5641084c0 2024-05-20T20:56:24.476 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:24.476 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5540b2910 msgr2=0x7ff5540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5540b2910 0x7ff5540b4dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff55800b650 tx=0x7ff558005fb0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 msgr2=0x7ff5641a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 0x7ff5641a0160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff55c007ee0 tx=0x7ff55c00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 shutdown_connections 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5641084c0 0x7ff5641a0160 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff564102c40 0x7ff5641a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff5540b2910 0x7ff5540b4dd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 --2- 172.21.15.26:0/662950659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff564103600 0x7ff5641a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 >> 172.21.15.26:0/662950659 conn(0x7ff564078ab0 msgr2=0x7ff564101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 shutdown_connections 2024-05-20T20:56:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:24.479+0000 7ff56abf2700 1 -- 172.21.15.26:0/662950659 wait complete. 2024-05-20T20:56:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:56:25.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-20T20:52:47.191542+0000', 'modified': '2024-05-20T20:55:45.467827+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24476, 'mds_1': 24455, 'mds_2': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/1712709614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 1712709614}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 1712709614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.55:6835/2325650145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2325650145}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2325650145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/539511953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 539511953}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 539511953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.144:6839/2122637820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 2122637820}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 2122637820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.55:6837/353197532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 353197532}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 353197532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.55:6839/1704683185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6838', 'nonce': 1704683185}, {'type': 'v1', 'addr': '172.21.15.55:6839', 'nonce': 1704683185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24445, 24455, 24476]} 2024-05-20T20:56:25.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-20T20:56:25.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-20T20:56:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:25 smithi026 ceph-mon[31990]: pgmap v396: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:25 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/662950659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:25 smithi144 ceph-mon[35660]: pgmap v396: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:25 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/662950659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:25.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:25 smithi055 ceph-mon[35213]: pgmap v396: 97 pgs: 97 active+clean; 651 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-20T20:56:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:25 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/662950659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:27.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 -- 172.21.15.26:0/3439079110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041041a0 msgr2=0x7f3e041081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/3439079110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041041a0 0x7f3e041081c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3df00099c0 tx=0x7f3df0009cd0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 -- 172.21.15.26:0/3439079110 shutdown_connections 2024-05-20T20:56:27.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/3439079110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e041088f0 0x7f3e04110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/3439079110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041037f0 0x7f3e04103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/3439079110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041041a0 0x7f3e041081c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 -- 172.21.15.26:0/3439079110 >> 172.21.15.26:0/3439079110 conn(0x7f3e0407a6a0 msgr2=0x7f3e0407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 -- 172.21.15.26:0/3439079110 shutdown_connections 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 -- 172.21.15.26:0/3439079110 wait complete. 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.167+0000 7f3e0adf3700 1 Processor -- start 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 -- start start 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e041037f0 0x7f3e041a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041041a0 0x7f3e041a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 0x7f3e0419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041135d0 con 0x7f3e041088f0 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04113450 con 0x7f3e041037f0 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e0adf3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04113750 con 0x7f3e041041a0 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 0x7f3e0419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 0x7f3e0419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41110/0 (socket says 172.21.15.26:41110) 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 -- 172.21.15.26:0/1356518788 learned_addr learned my addr 172.21.15.26:0/1356518788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e041037f0 0x7f3e041a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e03fff700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041041a0 0x7f3e041a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 -- 172.21.15.26:0/1356518788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041041a0 msgr2=0x7f3e041a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041041a0 0x7f3e041a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 -- 172.21.15.26:0/1356518788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e041037f0 msgr2=0x7f3e041a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e041037f0 0x7f3e041a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 -- 172.21.15.26:0/1356518788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0419c1f0 con 0x7f3e041088f0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.168+0000 7f3e09390700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 0x7f3e0419ba90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc007d70 tx=0x7f3dfc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.169+0000 7f3e01ffb700 1 -- 172.21.15.26:0/1356518788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc016020 con 0x7f3e041088f0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.169+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0419c4e0 con 0x7f3e041088f0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.169+0000 7f3e01ffb700 1 -- 172.21.15.26:0/1356518788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dfc00ad40 con 0x7f3e041088f0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.169+0000 7f3e01ffb700 1 -- 172.21.15.26:0/1356518788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0115e0 con 0x7f3e041088f0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.169+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a87a0 con 0x7f3e041088f0 2024-05-20T20:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.173+0000 7f3e01ffb700 1 -- 172.21.15.26:0/1356518788 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f3dfc019020 con 0x7f3e041088f0 2024-05-20T20:56:27.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.173+0000 7f3e15689b80 1 Processor -- start 2024-05-20T20:56:27.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.173+0000 7f3e01ffb700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3df40b2910 0x7f3df40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.173+0000 7f3e15689b80 1 -- start start 2024-05-20T20:56:27.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e03fff700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3df40b2910 0x7f3df40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e01ffb700 1 -- 172.21.15.26:0/1356518788 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f3dfc0e2260 con 0x7f3e041088f0 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d2cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d31350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e15689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c8bc40 con 0x55f4a7d30f10 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e15689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c7b7e0 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e15689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c7f820 con 0x55f4a7d2c810 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d31350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d31350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41120/0 (socket says 172.21.15.26:41120) 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 -- 172.21.15.26:0/4072943074 learned_addr learned my addr 172.21.15.26:0/4072943074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d2cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e03fff700 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 -- 172.21.15.26:0/4072943074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 msgr2=0x55f4a7d2cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d2cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 -- 172.21.15.26:0/4072943074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 msgr2=0x55f4a7d308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e03fff700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3df40b2910 0x7f3df40b4dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3e04102e70 tx=0x7f3df00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 -- 172.21.15.26:0/4072943074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a7bcb880 con 0x55f4a7d30f10 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.174+0000 7f3e09390700 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d31350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc018fd0 tx=0x7f3dfc0a9250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9ab27da20f8a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3deeffd700 1 -- 172.21.15.26:0/4072943074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0adb80 con 0x55f4a7d30f10 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3deeffd700 1 -- 172.21.15.26:0/4072943074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dfc0b2630 con 0x55f4a7d30f10 2024-05-20T20:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3deeffd700 1 -- 172.21.15.26:0/4072943074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0912e0 con 0x55f4a7d30f10 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 -- 172.21.15.26:0/4072943074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 msgr2=0x55f4a7d31350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d31350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc018fd0 tx=0x7f3dfc0a9250 comp rx=0 tx=0).stop 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 -- 172.21.15.26:0/4072943074 shutdown_connections 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d31350 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d2cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 --2- 172.21.15.26:0/4072943074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 -- 172.21.15.26:0/4072943074 >> 172.21.15.26:0/4072943074 conn(0x55f4a7c5a030 msgr2=0x55f4a7d26e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 -- 172.21.15.26:0/4072943074 shutdown_connections 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.175+0000 7f3e15689b80 1 -- 172.21.15.26:0/4072943074 wait complete. 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 -- start start 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 Processor -- start 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d38360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7b81f50 con 0x55f4a7d30f10 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c1d770 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d390d0 con 0x55f4a7d2c810 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e15689b80 10 client.0 _ll_register_callbacks cb 0x55f4a7c168d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d35ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41132/0 (socket says 172.21.15.26:41132) 2024-05-20T20:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.176+0000 7f3e09390700 1 -- 172.21.15.26:0/3483463087 learned_addr learned my addr 172.21.15.26:0/3483463087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e03fff700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d38360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e09390700 1 -- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 msgr2=0x55f4a7d38360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e09390700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d38360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e09390700 1 -- 172.21.15.26:0/3483463087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 msgr2=0x55f4a7d35660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e09390700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d35660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e09390700 1 -- 172.21.15.26:0/3483463087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a7c1a640 con 0x55f4a7d30f10 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e09390700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d35ba0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0a9e30 tx=0x7f3dfc0b2460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac143753962a7656 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3de3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3de3fff700 1 -- 172.21.15.26:0/3483463087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0c2030 con 0x55f4a7d30f10 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3de3fff700 1 -- 172.21.15.26:0/3483463087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f3dfc0cc0d0 con 0x55f4a7d30f10 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc0b2630 con 0x55f4a7d30f10 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3de3fff700 1 -- 172.21.15.26:0/3483463087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3dfc0adb80 con 0x55f4a7d30f10 2024-05-20T20:56:27.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e15689b80 10 client.15330 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4a7d437d0 con 0x55f4a7d30f10 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.177+0000 7f3de3fff700 1 -- 172.21.15.26:0/3483463087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dfc0b2630 con 0x55f4a7d30f10 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3de3fff700 1 -- 172.21.15.26:0/3483463087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3dfc06c070 con 0x55f4a7d30f10 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e15689b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e15689b80 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e15689b80 20 client.15330 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e15689b80 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7d46ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e15689b80 4 client.15330 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:27.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a7cf8990 con 0x55f4a7d44b20 2024-05-20T20:56:27.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.178+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7d46ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.179+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7d46ca0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=297ff2c678386ebf server_cookie=40d531bdd615a260 in_seq=0 out_seq=0 2024-05-20T20:56:27.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.179+0000 7f3de3fff700 10 client.15330 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:27.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.179+0000 7f3de3fff700 1 -- 172.21.15.26:0/3483463087 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f4a7cf8990 con 0x55f4a7d44b20 2024-05-20T20:56:27.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.179+0000 7f3de3fff700 10 client.15330 handle_command_reply: tid=0 2024-05-20T20:56:27.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 client.15330 shutdown 2024-05-20T20:56:27.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 msgr2=0x55f4a7d35ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d35ba0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0a9e30 tx=0x7f3dfc0b2460 comp rx=0 tx=0).stop 2024-05-20T20:56:27.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 shutdown_connections 2024-05-20T20:56:27.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7d46ca0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3de3fff700 0 client.15330 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d35ba0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d38360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 --2- 172.21.15.26:0/3483463087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d35660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 >> 172.21.15.26:0/3483463087 conn(0x55f4a7c5a030 msgr2=0x55f4a7d264c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 shutdown_connections 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 1 -- 172.21.15.26:0/3483463087 wait complete. 2024-05-20T20:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.180+0000 7f3e15689b80 20 client.15330 trim_cache size 0 max 16384 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 Processor -- start 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 -- start start 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2c810 0x55f4a7d34e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d26c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d41360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d44fe0 con 0x55f4a7d30f10 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d456b0 con 0x55f4a7d2c810 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e15689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d45d80 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d41360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d41360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41136/0 (socket says 172.21.15.26:41136) 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 -- 172.21.15.26:0/2019330624 learned_addr learned my addr 172.21.15.26:0/2019330624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e03fff700 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d26c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 -- 172.21.15.26:0/2019330624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2d1c0 msgr2=0x55f4a7d26c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d26c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2c810 0x55f4a7d34e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 -- 172.21.15.26:0/2019330624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2c810 msgr2=0x55f4a7d34e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2c810 0x55f4a7d34e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 -- 172.21.15.26:0/2019330624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a7d33bd0 con 0x55f4a7d30f10 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e03fff700 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d26c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.202+0000 7f3e09390700 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d41360 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0aafd0 tx=0x7f3dfc006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa66287f884c25c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3dee7fc700 1 -- 172.21.15.26:0/2019330624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0c2020 con 0x55f4a7d30f10 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3dee7fc700 1 -- 172.21.15.26:0/2019330624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dfc0b2630 con 0x55f4a7d30f10 2024-05-20T20:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3dee7fc700 1 -- 172.21.15.26:0/2019330624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0aab00 con 0x55f4a7d30f10 2024-05-20T20:56:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 -- 172.21.15.26:0/2019330624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 msgr2=0x55f4a7d41360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d41360 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0aafd0 tx=0x7f3dfc006040 comp rx=0 tx=0).stop 2024-05-20T20:56:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 -- 172.21.15.26:0/2019330624 shutdown_connections 2024-05-20T20:56:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d30f10 0x55f4a7d41360 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d2c810 0x55f4a7d34e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 --2- 172.21.15.26:0/2019330624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d26c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 -- 172.21.15.26:0/2019330624 >> 172.21.15.26:0/2019330624 conn(0x55f4a7c5a030 msgr2=0x55f4a7d2beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 -- 172.21.15.26:0/2019330624 shutdown_connections 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.203+0000 7f3e15689b80 1 -- 172.21.15.26:0/2019330624 wait complete. 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 -- start start 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 Processor -- start 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d30f10 0x55f4a7d4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d46b20 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d46450 con 0x55f4a7d30f10 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7d471f0 con 0x55f4a7d2c810 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e15689b80 10 client.0 _ll_register_callbacks cb 0x55f4a7c168d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d4d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41138/0 (socket says 172.21.15.26:41138) 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e03fff700 1 -- 172.21.15.26:0/1023448280 learned_addr learned my addr 172.21.15.26:0/1023448280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.204+0000 7f3e09390700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d30f10 0x55f4a7d4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e03fff700 1 -- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 msgr2=0x55f4a7d4cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e03fff700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d4cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e03fff700 1 -- 172.21.15.26:0/1023448280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d30f10 msgr2=0x55f4a7d4d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e03fff700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d30f10 0x55f4a7d4d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e03fff700 1 -- 172.21.15.26:0/1023448280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a7db10d0 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e09390700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d30f10 0x55f4a7d4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e03fff700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d4d440 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3df00098e0 tx=0x7f3df0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7922b428463a91b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3de3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3de3fff700 1 -- 172.21.15.26:0/1023448280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000bb70 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3de3fff700 1 -- 172.21.15.26:0/1023448280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f3df000bd10 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc0b2630 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3de3fff700 1 -- 172.21.15.26:0/1023448280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3df0030480 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e15689b80 10 client.15336 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.205+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4a7db2350 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3de3fff700 1 -- 172.21.15.26:0/1023448280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dfc0b2630 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3de3fff700 1 -- 172.21.15.26:0/1023448280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3df0024780 con 0x55f4a7d2d1c0 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e15689b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e15689b80 10 client.15336 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e15689b80 20 client.15336 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e15689b80 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7db5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e15689b80 4 client.15336 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:56:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "status"}) -- 0x55f4a7c8db90 con 0x55f4a7d44b20 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.206+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7db5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3de3fff700 10 client.15336 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e08b8f700 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7db5c40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e978cdfdce580565 server_cookie=c30542f690ccd749 in_seq=0 out_seq=0 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3de3fff700 1 -- 172.21.15.26:0/1023448280 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4a7c8db90 con 0x55f4a7d44b20 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3de3fff700 10 client.15336 handle_command_reply: tid=0 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 client.15336 shutdown 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 msgr2=0x55f4a7d4d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d4d440 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3df00098e0 tx=0x7f3df0009cd0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 shutdown_connections 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f4a7d44b20 0x55f4a7db5c40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f4a7d30f10 0x55f4a7d4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4a7d2c810 0x55f4a7d4cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 --2- 172.21.15.26:0/1023448280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4a7d2d1c0 0x55f4a7d4d440 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 >> 172.21.15.26:0/1023448280 conn(0x55f4a7c5a030 msgr2=0x55f4a7d36e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3de3fff700 0 client.15336 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 shutdown_connections 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 1 -- 172.21.15.26:0/1023448280 wait complete. 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.207+0000 7f3e15689b80 20 client.15336 trim_cache size 0 max 16384 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24445, 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 206.90161172200001, 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 246.130301551 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3df40b2910 msgr2=0x7f3df40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3df40b2910 0x7f3df40b4dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3e04102e70 tx=0x7f3df00095f0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 msgr2=0x7f3e0419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 0x7f3e0419ba90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc007d70 tx=0x7f3dfc00a340 comp rx=0 tx=0).stop 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 shutdown_connections 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3df40b2910 0x7f3df40b4dd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e041088f0 0x7f3e0419ba90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e041037f0 0x7f3e041a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 --2- 172.21.15.26:0/1356518788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041041a0 0x7f3e041a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 >> 172.21.15.26:0/1356518788 conn(0x7f3e0407a6a0 msgr2=0x7f3e04101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 shutdown_connections 2024-05-20T20:56:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:27.208+0000 7f3e0adf3700 1 -- 172.21.15.26:0/1356518788 wait complete. 2024-05-20T20:56:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:27 smithi026 ceph-mon[31990]: pgmap v397: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-20T20:56:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:27 smithi144 ceph-mon[35660]: pgmap v397: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-20T20:56:27.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:27 smithi055 ceph-mon[35213]: pgmap v397: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-20T20:56:27.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-20T20:56:28.155 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:29.280 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:29 smithi026 ceph-mon[31990]: pgmap v398: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-20T20:56:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:29 smithi144 ceph-mon[35660]: pgmap v398: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-20T20:56:29.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:29 smithi055 ceph-mon[35213]: pgmap v398: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 -- 172.21.15.26:0/493334472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 msgr2=0x7f1e84108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 --2- 172.21.15.26:0/493334472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e84108890 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009990 tx=0x7f1e74009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 -- 172.21.15.26:0/493334472 shutdown_connections 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 --2- 172.21.15.26:0/493334472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e84108890 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 --2- 172.21.15.26:0/493334472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102e30 0x7f1e8410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 --2- 172.21.15.26:0/493334472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e84102470 0x7f1e841028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 -- 172.21.15.26:0/493334472 >> 172.21.15.26:0/493334472 conn(0x7f1e840793f0 msgr2=0x7f1e84079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 -- 172.21.15.26:0/493334472 shutdown_connections 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 -- 172.21.15.26:0/493334472 wait complete. 2024-05-20T20:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 Processor -- start 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.827+0000 7f1e8a557700 1 -- start start 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e8a557700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102470 0x7f1e8419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e8a557700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e84102e30 0x7f1e8419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e8a557700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e841a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e8a557700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841124f0 con 0x7f1e841084b0 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e8a557700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84112370 con 0x7f1e84102e30 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e8a557700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84112670 con 0x7f1e84102470 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e841a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e841a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47446/0 (socket says 172.21.15.26:47446) 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 -- 172.21.15.26:0/2430617924 learned_addr learned my addr 172.21.15.26:0/2430617924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e837fe700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e84102e30 0x7f1e8419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e83fff700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102470 0x7f1e8419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 -- 172.21.15.26:0/2430617924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102470 msgr2=0x7f1e8419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102470 0x7f1e8419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 -- 172.21.15.26:0/2430617924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e84102e30 msgr2=0x7f1e8419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e84102e30 0x7f1e8419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e88af4700 1 -- 172.21.15.26:0/2430617924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e841a5f70 con 0x7f1e841084b0 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.828+0000 7f1e83fff700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102470 0x7f1e8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.829+0000 7f1e88af4700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e841a5a30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00b970 tx=0x7f1e7c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.829+0000 7f1e817fa700 1 -- 172.21.15.26:0/2430617924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c019070 con 0x7f1e841084b0 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.829+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e841a61a0 con 0x7f1e841084b0 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.829+0000 7f1e817fa700 1 -- 172.21.15.26:0/2430617924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e7c00cdc0 con 0x7f1e841084b0 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.829+0000 7f1e817fa700 1 -- 172.21.15.26:0/2430617924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c014e30 con 0x7f1e841084b0 2024-05-20T20:56:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.829+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841a6660 con 0x7f1e841084b0 2024-05-20T20:56:29.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.832+0000 7f1e817fa700 1 -- 172.21.15.26:0/2430617924 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f1e7c01c070 con 0x7f1e841084b0 2024-05-20T20:56:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.832+0000 7f1e817fa700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1e6c0b2910 0x7f1e6c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 Processor -- start 2024-05-20T20:56:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e83fff700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1e6c0b2910 0x7f1e6c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e817fa700 1 -- 172.21.15.26:0/2430617924 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1e7c0e1690 con 0x7f1e841084b0 2024-05-20T20:56:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 -- start start 2024-05-20T20:56:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 0x5595761c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bc850 0x5595761bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bd210 0x5595761c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55957611ad20 con 0x5595761c22d0 2024-05-20T20:56:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55957610a8c0 con 0x5595761bc850 2024-05-20T20:56:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55957610e900 con 0x5595761bd210 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e83fff700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1e6c0b2910 0x7f1e6c0b4dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003760 tx=0x7f1e7401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e83fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 0x5595761c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e83fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 0x5595761c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47460/0 (socket says 172.21.15.26:47460) 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e83fff700 1 -- 172.21.15.26:0/3074712369 learned_addr learned my addr 172.21.15.26:0/3074712369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e88af4700 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bd210 0x5595761c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.833+0000 7f1e837fe700 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bc850 0x5595761bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e83fff700 1 -- 172.21.15.26:0/3074712369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bd210 msgr2=0x5595761c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e83fff700 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bd210 0x5595761c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e83fff700 1 -- 172.21.15.26:0/3074712369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bc850 msgr2=0x5595761bccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e83fff700 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bc850 0x5595761bccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e83fff700 1 -- 172.21.15.26:0/3074712369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595760b3f60 con 0x5595761c22d0 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e83fff700 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 0x5595761c26b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1e74015000 tx=0x7f1e7401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c509849cd39fc556 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e6a7fc700 1 -- 172.21.15.26:0/3074712369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e740055d0 con 0x5595761c22d0 2024-05-20T20:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e6a7fc700 1 -- 172.21.15.26:0/3074712369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e7400bc60 con 0x5595761c22d0 2024-05-20T20:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.834+0000 7f1e6a7fc700 1 -- 172.21.15.26:0/3074712369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400f9b0 con 0x5595761c22d0 2024-05-20T20:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 -- 172.21.15.26:0/3074712369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 msgr2=0x5595761c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 0x5595761c26b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1e74015000 tx=0x7f1e7401e000 comp rx=0 tx=0).stop 2024-05-20T20:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 -- 172.21.15.26:0/3074712369 shutdown_connections 2024-05-20T20:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761c22d0 0x5595761c26b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bd210 0x5595761c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/3074712369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bc850 0x5595761bccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 -- 172.21.15.26:0/3074712369 >> 172.21.15.26:0/3074712369 conn(0x5595760e81f0 msgr2=0x5595761b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 -- 172.21.15.26:0/3074712369 shutdown_connections 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 -- 172.21.15.26:0/3074712369 wait complete. 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.835+0000 7f1e94dedb80 1 -- start start 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 Processor -- start 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 0x5595761c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bd210 0x5595761c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761c22d0 0x5595761c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559576010f50 con 0x5595761bc850 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559576072d50 con 0x5595761bd210 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5595761c95d0 con 0x5595761c22d0 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 0x5595761c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e94dedb80 10 client.0 _ll_register_callbacks cb 0x5595760a5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 0x5595761c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47462/0 (socket says 172.21.15.26:47462) 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 -- 172.21.15.26:0/360852484 learned_addr learned my addr 172.21.15.26:0/360852484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e88af4700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761c22d0 0x5595761c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e837fe700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bd210 0x5595761c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 -- 172.21.15.26:0/360852484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761c22d0 msgr2=0x5595761c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761c22d0 0x5595761c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 -- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bd210 msgr2=0x5595761c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bd210 0x5595761c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 -- 172.21.15.26:0/360852484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957605a880 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e83fff700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 0x5595761c8ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003ef0 tx=0x7f1e7400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cf1db55e14de3a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.836+0000 7f1e637fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e637fe700 1 -- 172.21.15.26:0/360852484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7403b060 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e637fe700 1 -- 172.21.15.26:0/360852484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1e7400f820 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7400bc60 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e637fe700 1 -- 172.21.15.26:0/360852484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e7400fa40 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 10 client.15348 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595760c4710 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e637fe700 1 -- 172.21.15.26:0/360852484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e7400bc60 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e637fe700 1 -- 172.21.15.26:0/360852484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e7403f030 con 0x5595761bc850 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 10 client.15348 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 10 client.15348 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T20:56:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 20 client.15348 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5595761d4df0 0x5595761d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 4 client.15348 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.837+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957611cc70 con 0x5595761d4df0 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.838+0000 7f1e837fe700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5595761d4df0 0x5595761d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.838+0000 7f1e837fe700 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5595761d4df0 0x5595761d6fb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cfe8567dfdb211 server_cookie=bf9409002168ad59 in_seq=0 out_seq=0 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.838+0000 7f1e637fe700 10 client.15348 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e637fe700 1 -- 172.21.15.26:0/360852484 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55957611cc70 con 0x5595761d4df0 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e637fe700 10 client.15348 handle_command_reply: tid=0 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 client.15348 shutdown 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 msgr2=0x5595761c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 0x5595761c8ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003ef0 tx=0x7f1e7400f460 comp rx=0 tx=0).stop 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 shutdown_connections 2024-05-20T20:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5595761d4df0 0x5595761d6fb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bc850 0x5595761c8ca0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761c22d0 0x5595761c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/360852484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761bd210 0x5595761c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e637fe700 0 client.15348 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 >> 172.21.15.26:0/360852484 conn(0x5595760e81f0 msgr2=0x5595761b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 shutdown_connections 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 1 -- 172.21.15.26:0/360852484 wait complete. 2024-05-20T20:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.839+0000 7f1e94dedb80 20 client.15348 trim_cache size 0 max 16384 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 Processor -- start 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 -- start start 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761b7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5595761d52f0 con 0x5595761bd210 2024-05-20T20:56:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5595761d59c0 con 0x5595761c22d0 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5595761d6090 con 0x5595761bc850 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47470/0 (socket says 172.21.15.26:47470) 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e837fe700 1 -- 172.21.15.26:0/1893317914 learned_addr learned my addr 172.21.15.26:0/1893317914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.861+0000 7f1e83fff700 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e88af4700 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e837fe700 1 -- 172.21.15.26:0/1893317914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 msgr2=0x5595761bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e837fe700 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e837fe700 1 -- 172.21.15.26:0/1893317914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 msgr2=0x5595761b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e837fe700 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e837fe700 1 -- 172.21.15.26:0/1893317914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595761b8c60 con 0x5595761bd210 2024-05-20T20:56:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e88af4700 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e837fe700 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761c0590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5595761c7570 tx=0x7f1e78013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd27490847bd452c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e69ffb700 1 -- 172.21.15.26:0/1893317914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78023070 con 0x5595761bd210 2024-05-20T20:56:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e69ffb700 1 -- 172.21.15.26:0/1893317914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e780044e0 con 0x5595761bd210 2024-05-20T20:56:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.862+0000 7f1e69ffb700 1 -- 172.21.15.26:0/1893317914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78003ae0 con 0x5595761bd210 2024-05-20T20:56:29.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 -- 172.21.15.26:0/1893317914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 msgr2=0x5595761c0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761c0590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5595761c7570 tx=0x7f1e78013740 comp rx=0 tx=0).stop 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 -- 172.21.15.26:0/1893317914 shutdown_connections 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/1893317914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761c0590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 -- 172.21.15.26:0/1893317914 >> 172.21.15.26:0/1893317914 conn(0x5595760e81f0 msgr2=0x5595761c4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 -- 172.21.15.26:0/1893317914 shutdown_connections 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 -- 172.21.15.26:0/1893317914 wait complete. 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 -- start start 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.863+0000 7f1e94dedb80 1 Processor -- start 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761e0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5595761d6e30 con 0x5595761bd210 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5595761d6760 con 0x5595761c22d0 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5595761d7500 con 0x5595761bc850 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e94dedb80 10 client.0 _ll_register_callbacks cb 0x5595760a5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47484/0 (socket says 172.21.15.26:47484) 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e83fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761e0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 -- 172.21.15.26:0/2344017870 learned_addr learned my addr 172.21.15.26:0/2344017870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e88af4700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 -- 172.21.15.26:0/2344017870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 msgr2=0x5595761e0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761e0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 -- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 msgr2=0x5595761da7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761da7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 -- 172.21.15.26:0/2344017870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595761dae90 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e88af4700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761da7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e837fe700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761e0990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e78020000 tx=0x7f1e780133c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789b6fe167307fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.864+0000 7f1e637fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e637fe700 1 -- 172.21.15.26:0/2344017870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78023040 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e637fe700 1 -- 172.21.15.26:0/2344017870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f1e78004050 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e780044e0 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e637fe700 1 -- 172.21.15.26:0/2344017870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e78004250 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 10 client.15354 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559576241be0 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e637fe700 1 -- 172.21.15.26:0/2344017870 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e780044e0 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e637fe700 1 -- 172.21.15.26:0/2344017870 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e78026070 con 0x5595761bd210 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:29.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 10 client.15354 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 20 client.15354 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559576246400 0x559576245110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 4 client.15354 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.865+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "status"}) -- 0x559576116450 con 0x559576246400 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e83fff700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559576246400 0x559576245110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e83fff700 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559576246400 0x559576245110 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56451a3e4c68af1 server_cookie=4d8419eb0174150a in_seq=0 out_seq=0 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e637fe700 10 client.15354 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e637fe700 1 -- 172.21.15.26:0/2344017870 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559576116450 con 0x559576246400 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e637fe700 10 client.15354 handle_command_reply: tid=0 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 client.15354 shutdown 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 msgr2=0x5595761e0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761e0990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e78020000 tx=0x7f1e780133c0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 shutdown_connections 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559576246400 0x559576245110 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5595761c22d0 0x5595761da7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595761bc850 0x5595761e0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 --2- 172.21.15.26:0/2344017870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595761bd210 0x5595761e0990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 >> 172.21.15.26:0/2344017870 conn(0x5595760e81f0 msgr2=0x5595761bc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.866+0000 7f1e637fe700 0 client.15354 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 shutdown_connections 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e94dedb80 1 -- 172.21.15.26:0/2344017870 wait complete. 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e94dedb80 20 client.15354 trim_cache size 0 max 16384 2024-05-20T20:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24455, 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 210.571791373, 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 251.67234202700001 2024-05-20T20:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1e6c0b2910 msgr2=0x7f1e6c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1e6c0b2910 0x7f1e6c0b4dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003760 tx=0x7f1e7401a040 comp rx=0 tx=0).stop 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 msgr2=0x7f1e841a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e841a5a30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00b970 tx=0x7f1e7c00bc80 comp rx=0 tx=0).stop 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 shutdown_connections 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1e6c0b2910 0x7f1e6c0b4dd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e841084b0 0x7f1e841a5a30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84102470 0x7f1e8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 --2- 172.21.15.26:0/2430617924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e84102e30 0x7f1e8419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:29.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 >> 172.21.15.26:0/2430617924 conn(0x7f1e840793f0 msgr2=0x7f1e84100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:29.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 shutdown_connections 2024-05-20T20:56:29.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:29.867+0000 7f1e8a557700 1 -- 172.21.15.26:0/2430617924 wait complete. 2024-05-20T20:56:30.668 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-20T20:56:30.922 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:31.302 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:31 smithi026 ceph-mon[31990]: pgmap v399: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-20T20:56:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:31 smithi144 ceph-mon[35660]: pgmap v399: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-20T20:56:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:31 smithi055 ceph-mon[35213]: pgmap v399: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-20T20:56:32.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 -- 172.21.15.26:0/2423592078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0108620 msgr2=0x7fd5a0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 --2- 172.21.15.26:0/2423592078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0108620 0x7fd5a0108a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd590009990 tx=0x7fd590009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 -- 172.21.15.26:0/2423592078 shutdown_connections 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 --2- 172.21.15.26:0/2423592078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0108620 0x7fd5a0108a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 --2- 172.21.15.26:0/2423592078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5a01028c0 0x7fd5a0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 --2- 172.21.15.26:0/2423592078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5a0101f00 0x7fd5a0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 -- 172.21.15.26:0/2423592078 >> 172.21.15.26:0/2423592078 conn(0x7fd5a0077780 msgr2=0x7fd5a0077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:32.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.529+0000 7fd5a6614700 1 -- 172.21.15.26:0/2423592078 shutdown_connections 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.530+0000 7fd5a6614700 1 -- 172.21.15.26:0/2423592078 wait complete. 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.530+0000 7fd5a6614700 1 Processor -- start 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.530+0000 7fd5a6614700 1 -- start start 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.530+0000 7fd5a6614700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 0x7fd5a019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd5a6614700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5a01028c0 0x7fd5a019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd5a6614700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5a0108620 0x7fd5a01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd5a6614700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0113660 con 0x7fd5a0101f00 2024-05-20T20:56:32.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd5a6614700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01134e0 con 0x7fd5a01028c0 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd5a6614700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01137e0 con 0x7fd5a0108620 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 0x7fd5a019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 0x7fd5a019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47502/0 (socket says 172.21.15.26:47502) 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 -- 172.21.15.26:0/1746924522 learned_addr learned my addr 172.21.15.26:0/1746924522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5a0108620 0x7fd5a01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5a01028c0 0x7fd5a019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 -- 172.21.15.26:0/1746924522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5a0108620 msgr2=0x7fd5a01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5a0108620 0x7fd5a01999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 -- 172.21.15.26:0/1746924522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5a01028c0 msgr2=0x7fd5a019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5a01028c0 0x7fd5a019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 -- 172.21.15.26:0/1746924522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a019a120 con 0x7fd5a0101f00 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59ffff700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 0x7fd5a019f7b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd590000c00 tx=0x7fd5900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59d7fa700 1 -- 172.21.15.26:0/1746924522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590027070 con 0x7fd5a0101f00 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.531+0000 7fd59d7fa700 1 -- 172.21.15.26:0/1746924522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd59000fdb0 con 0x7fd5a0101f00 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.532+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a019a3b0 con 0x7fd5a0101f00 2024-05-20T20:56:32.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.532+0000 7fd59d7fa700 1 -- 172.21.15.26:0/1746924522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900173f0 con 0x7fd5a0101f00 2024-05-20T20:56:32.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.532+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a6680 con 0x7fd5a0101f00 2024-05-20T20:56:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.534+0000 7fd59d7fa700 1 -- 172.21.15.26:0/1746924522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fd59002a070 con 0x7fd5a0101f00 2024-05-20T20:56:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.535+0000 7fd59d7fa700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5880b2990 0x7fd5880b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.535+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5880b2990 0x7fd5880b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.535+0000 7fd59d7fa700 1 -- 172.21.15.26:0/1746924522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd5900e3460 con 0x7fd5a0101f00 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.535+0000 7fd5b0eaab80 1 Processor -- start 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.535+0000 7fd5b0eaab80 1 -- start start 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.535+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5880b2990 0x7fd5880b4e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd594005ea0 tx=0x7fd594005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662db5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662db8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662db9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560662d0fc40 con 0x560662db9320 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560662cff7e0 con 0x560662db55d0 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560662d03820 con 0x560662db4c20 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662db9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662db9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47504/0 (socket says 172.21.15.26:47504) 2024-05-20T20:56:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/567526520 learned_addr learned my addr 172.21.15.26:0/567526520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd59ffff700 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662db5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662db8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/567526520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 msgr2=0x560662db5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662db5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/567526520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 msgr2=0x560662db8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662db8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/567526520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560662c4f880 con 0x560662db9320 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662db9760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd598009f20 tx=0x7fd598009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a492e1c03bfd3780 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5867fc700 1 -- 172.21.15.26:0/567526520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59801a070 con 0x560662db9320 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5867fc700 1 -- 172.21.15.26:0/567526520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd598015410 con 0x560662db9320 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.536+0000 7fd5867fc700 1 -- 172.21.15.26:0/567526520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59800a600 con 0x560662db9320 2024-05-20T20:56:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/567526520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 msgr2=0x560662db9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662db9760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd598009f20 tx=0x7fd598009f50 comp rx=0 tx=0).stop 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/567526520 shutdown_connections 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662db9760 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662db5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/567526520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662db8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/567526520 >> 172.21.15.26:0/567526520 conn(0x560662cde030 msgr2=0x560662daae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/567526520 shutdown_connections 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.537+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/567526520 wait complete. 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 -- start start 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 Processor -- start 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662dc0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 0x560662dbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db9320 0x560662dbdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560662c05f50 con 0x560662db55d0 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560662ca1770 con 0x560662db9320 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560662dc14e0 con 0x560662db4c20 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd5b0eaab80 10 client.0 _ll_register_callbacks cb 0x560662c9a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd59f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 0x560662dbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd59f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 0x560662dbda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47510/0 (socket says 172.21.15.26:47510) 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd59f7fe700 1 -- 172.21.15.26:0/3098330882 learned_addr learned my addr 172.21.15.26:0/3098330882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.538+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662dc0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db9320 0x560662dbdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd59f7fe700 1 -- 172.21.15.26:0/3098330882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 msgr2=0x560662dc0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662dc0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd59f7fe700 1 -- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db9320 msgr2=0x560662dbdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db9320 0x560662dbdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd59f7fe700 1 -- 172.21.15.26:0/3098330882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560662c9e640 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 0x560662dbda70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd5940107b0 tx=0x7fd5940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e219dd9417e082d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd57f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3098330882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59401d8f0 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3098330882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd59401da90 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd598015410 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3098330882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd59401dc90 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd5b0eaab80 10 client.15366 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560662dcbc40 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.539+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3098330882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd598015410 con 0x560662db55d0 2024-05-20T20:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3098330882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd594025400 con 0x560662db55d0 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd5b0eaab80 10 client.15366 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd5b0eaab80 10 client.15366 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd5b0eaab80 20 client.15366 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662dce260 0x560662dcf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd5b0eaab80 4 client.15366 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560662d7c990 con 0x560662dce260 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662dce260 0x560662dcf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd57f7fe700 10 client.15366 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.540+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662dce260 0x560662dcf6d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34619c1817181ff7 server_cookie=f32729e9d3c8a731 in_seq=0 out_seq=0 2024-05-20T20:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.541+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3098330882 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560662d7c990 con 0x560662dce260 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.541+0000 7fd57f7fe700 10 client.15366 handle_command_reply: tid=0 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.541+0000 7fd5b0eaab80 1 client.15366 shutdown 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 msgr2=0x560662dbda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 0x560662dbda70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd5940107b0 tx=0x7fd5940047c0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 shutdown_connections 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662dce260 0x560662dcf6d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db9320 0x560662dbdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662dc0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd57f7fe700 0 client.15366 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3098330882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db55d0 0x560662dbda70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 >> 172.21.15.26:0/3098330882 conn(0x560662cde030 msgr2=0x560662daa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 shutdown_connections 2024-05-20T20:56:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3098330882 wait complete. 2024-05-20T20:56:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.542+0000 7fd5b0eaab80 20 client.15366 trim_cache size 0 max 16384 2024-05-20T20:56:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 Processor -- start 2024-05-20T20:56:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 -- start start 2024-05-20T20:56:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662da7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662dbf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662daac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560662dcda10 con 0x560662db9320 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560662dce0e0 con 0x560662db55d0 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560662dce7b0 con 0x560662db4c20 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5a4bb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662daac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5a4bb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662daac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47522/0 (socket says 172.21.15.26:47522) 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3277951649 learned_addr learned my addr 172.21.15.26:0/3277951649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662dbf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.564+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662da7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3277951649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 msgr2=0x560662da7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662da7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3277951649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 msgr2=0x560662dbf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662dbf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3277951649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560662dc0e50 con 0x560662db9320 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662daac40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ec0 tx=0x7fd598009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a23aaaa4ed8fff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd585ffb700 1 -- 172.21.15.26:0/3277951649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59801a040 con 0x560662db9320 2024-05-20T20:56:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd585ffb700 1 -- 172.21.15.26:0/3277951649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd598015410 con 0x560662db9320 2024-05-20T20:56:32.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.565+0000 7fd585ffb700 1 -- 172.21.15.26:0/3277951649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598020680 con 0x560662db9320 2024-05-20T20:56:32.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3277951649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 msgr2=0x560662daac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662daac40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ec0 tx=0x7fd598009520 comp rx=0 tx=0).stop 2024-05-20T20:56:32.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3277951649 shutdown_connections 2024-05-20T20:56:32.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662daac40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db4c20 0x560662da7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3277951649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db55d0 0x560662dbf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3277951649 >> 172.21.15.26:0/3277951649 conn(0x560662cde030 msgr2=0x560662dbf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3277951649 shutdown_connections 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3277951649 wait complete. 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.566+0000 7fd5b0eaab80 1 -- start start 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 Processor -- start 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db4c20 0x560662dd8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db55d0 0x560662dd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662dd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560662dcf550 con 0x560662db9320 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560662dcee80 con 0x560662db4c20 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560662dcfc20 con 0x560662db55d0 2024-05-20T20:56:32.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5b0eaab80 10 client.0 _ll_register_callbacks cb 0x560662c9a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662dd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662dd3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47530/0 (socket says 172.21.15.26:47530) 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3469409092 learned_addr learned my addr 172.21.15.26:0/3469409092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd59f7fe700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db55d0 0x560662dd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db4c20 0x560662dd8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3469409092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db55d0 msgr2=0x560662dd3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db55d0 0x560662dd3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db4c20 msgr2=0x560662dd8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db4c20 0x560662dd8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd5a4bb1700 1 -- 172.21.15.26:0/3469409092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560662dd3e30 con 0x560662db9320 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.567+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db4c20 0x560662dd8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd5a4bb1700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662dd3870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd598015a10 tx=0x7fd598015a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1a5e57e9e7a00c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd57f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:32.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3469409092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59801a040 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3469409092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd598012040 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd598015410 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3469409092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd59800ccf0 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd5b0eaab80 10 client.15372 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560662e3a7e0 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3469409092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd598015410 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3469409092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd59801d030 con 0x560662db9320 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.568+0000 7fd5b0eaab80 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd5b0eaab80 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T20:56:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd5b0eaab80 20 client.15372 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662e3ec30 0x560662e3d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd5b0eaab80 4 client.15372 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "status"}) -- 0x560662d11b90 con 0x560662e3ec30 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662e3ec30 0x560662e3d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd57f7fe700 10 client.15372 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.569+0000 7fd59ffff700 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662e3ec30 0x560662e3d940 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3d65c78ad5423b server_cookie=ee4e3fc7ed458e64 in_seq=0 out_seq=0 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd57f7fe700 1 -- 172.21.15.26:0/3469409092 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560662d11b90 con 0x560662e3ec30 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd57f7fe700 10 client.15372 handle_command_reply: tid=0 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 client.15372 shutdown 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 msgr2=0x560662dd3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662dd3870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd598015a10 tx=0x7fd598015a40 comp rx=0 tx=0).stop 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 shutdown_connections 2024-05-20T20:56:32.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560662e3ec30 0x560662e3d940 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560662db9320 0x560662dd3870 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560662db4c20 0x560662dd8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 --2- 172.21.15.26:0/3469409092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560662db55d0 0x560662dd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 >> 172.21.15.26:0/3469409092 conn(0x560662cde030 msgr2=0x560662dafeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd57f7fe700 0 client.15372 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 shutdown_connections 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 1 -- 172.21.15.26:0/3469409092 wait complete. 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.570+0000 7fd5b0eaab80 20 client.15372 trim_cache size 0 max 16384 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "9006c62a-16e9-11ef-bc9a-c7b262605968", 2024-05-20T20:56:32.576 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24476, 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 225.34166172600001, 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 241.503159044 2024-05-20T20:56:32.577 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:32.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5880b2990 msgr2=0x7fd5880b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5880b2990 0x7fd5880b4e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd594005ea0 tx=0x7fd594005e30 comp rx=0 tx=0).stop 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 msgr2=0x7fd5a019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 0x7fd5a019f7b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd590000c00 tx=0x7fd5900047b0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 shutdown_connections 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5880b2990 0x7fd5880b4e50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0101f00 0x7fd5a019f7b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5a0108620 0x7fd5a01999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 --2- 172.21.15.26:0/1746924522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5a01028c0 0x7fd5a019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 >> 172.21.15.26:0/1746924522 conn(0x7fd5a0077780 msgr2=0x7fd5a0100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 shutdown_connections 2024-05-20T20:56:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:32.571+0000 7fd5a6614700 1 -- 172.21.15.26:0/1746924522 wait complete. 2024-05-20T20:56:33.270 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-20T20:56:33.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:33 smithi026 ceph-mon[31990]: pgmap v400: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-20T20:56:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:33 smithi144 ceph-mon[35660]: pgmap v400: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-20T20:56:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:33 smithi055 ceph-mon[35213]: pgmap v400: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-20T20:56:35.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:56:35.477 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:35 smithi144 ceph-mon[35660]: pgmap v401: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:35.599 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:35 smithi026 ceph-mon[31990]: pgmap v401: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:35.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:35 smithi055 ceph-mon[35213]: pgmap v401: 97 pgs: 97 active+clean; 657 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:37.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 -- 172.21.15.26:0/3789342927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806bcf0 msgr2=0x7fc91806c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 --2- 172.21.15.26:0/3789342927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806bcf0 0x7fc91806c0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc908009990 tx=0x7fc908009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 -- 172.21.15.26:0/3789342927 shutdown_connections 2024-05-20T20:56:37.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 --2- 172.21.15.26:0/3789342927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806bcf0 0x7fc91806c0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 --2- 172.21.15.26:0/3789342927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91810ff90 0x7fc918112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 --2- 172.21.15.26:0/3789342927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91806c610 0x7fc91810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 -- 172.21.15.26:0/3789342927 >> 172.21.15.26:0/3789342927 conn(0x7fc9180794e0 msgr2=0x7fc9180798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 -- 172.21.15.26:0/3789342927 shutdown_connections 2024-05-20T20:56:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.197+0000 7fc92059e700 1 -- 172.21.15.26:0/3789342927 wait complete. 2024-05-20T20:56:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 Processor -- start 2024-05-20T20:56:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 -- start start 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91806bcf0 0x7fc91819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 0x7fc91819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91810ff90 0x7fc9181a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc918115870 con 0x7fc91806c610 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181156f0 con 0x7fc91810ff90 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc92059e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181159f0 con 0x7fc91806bcf0 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 0x7fc91819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 0x7fc91819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47550/0 (socket says 172.21.15.26:47550) 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 -- 172.21.15.26:0/4287428146 learned_addr learned my addr 172.21.15.26:0/4287428146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91e33a700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91806bcf0 0x7fc91819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91eb3b700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91810ff90 0x7fc9181a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 -- 172.21.15.26:0/4287428146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91806bcf0 msgr2=0x7fc91819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91806bcf0 0x7fc91819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 -- 172.21.15.26:0/4287428146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91810ff90 msgr2=0x7fc9181a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91810ff90 0x7fc9181a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.198+0000 7fc91db39700 1 -- 172.21.15.26:0/4287428146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9181a0d80 con 0x7fc91806c610 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.199+0000 7fc91db39700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 0x7fc91819c300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b530 tx=0x7fc91400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.199+0000 7fc90f7fe700 1 -- 172.21.15.26:0/4287428146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914015020 con 0x7fc91806c610 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.199+0000 7fc90f7fe700 1 -- 172.21.15.26:0/4287428146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc914011cc0 con 0x7fc91806c610 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.199+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9181a1070 con 0x7fc91806c610 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.199+0000 7fc90f7fe700 1 -- 172.21.15.26:0/4287428146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914010c80 con 0x7fc91806c610 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.199+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9181a88f0 con 0x7fc91806c610 2024-05-20T20:56:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.201+0000 7fc90f7fe700 1 -- 172.21.15.26:0/4287428146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fc914018020 con 0x7fc91806c610 2024-05-20T20:56:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.202+0000 7fc90f7fe700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc9040b2990 0x7fc9040b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.202+0000 7fc90f7fe700 1 -- 172.21.15.26:0/4287428146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc9140e1930 con 0x7fc91806c610 2024-05-20T20:56:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.202+0000 7fc91e33a700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc9040b2990 0x7fc9040b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 Processor -- start 2024-05-20T20:56:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc91e33a700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc9040b2990 0x7fc9040b4e50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc908009750 tx=0x7fc90800b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 -- start start 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc32940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc32f60 0x55a8fcc333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8fcb8dc90 con 0x55a8fcc2e860 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a8fcb7d830 con 0x55a8fcc2f210 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8fcb81870 con 0x55a8fcc32f60 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc2ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47556/0 (socket says 172.21.15.26:47556) 2024-05-20T20:56:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc91e33a700 1 -- 172.21.15.26:0/960089109 learned_addr learned my addr 172.21.15.26:0/960089109 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc91eb3b700 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc32f60 0x55a8fcc333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.203+0000 7fc91db39700 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc32940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc91e33a700 1 -- 172.21.15.26:0/960089109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc32f60 msgr2=0x55a8fcc333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc91e33a700 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc32f60 0x55a8fcc333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc91e33a700 1 -- 172.21.15.26:0/960089109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 msgr2=0x55a8fcc32940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc91e33a700 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc32940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc91e33a700 1 -- 172.21.15.26:0/960089109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8fcacd8d0 con 0x55a8fcc2e860 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc91e33a700 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc2ec40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9080036d0 tx=0x7fc90801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca8fb39e18ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc90cff9700 1 -- 172.21.15.26:0/960089109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908034070 con 0x55a8fcc2e860 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc90cff9700 1 -- 172.21.15.26:0/960089109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9080249d0 con 0x55a8fcc2e860 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.204+0000 7fc90cff9700 1 -- 172.21.15.26:0/960089109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90802f9e0 con 0x55a8fcc2e860 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 -- 172.21.15.26:0/960089109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 msgr2=0x55a8fcc2ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc2ec40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9080036d0 tx=0x7fc90801f000 comp rx=0 tx=0).stop 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 -- 172.21.15.26:0/960089109 shutdown_connections 2024-05-20T20:56:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc2ec40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc32f60 0x55a8fcc333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/960089109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc32940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 -- 172.21.15.26:0/960089109 >> 172.21.15.26:0/960089109 conn(0x55a8fcb5c080 msgr2=0x55a8fcc28e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 -- 172.21.15.26:0/960089109 shutdown_connections 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 -- 172.21.15.26:0/960089109 wait complete. 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 -- start start 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 Processor -- start 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.205+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2f210 0x55a8fcc37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc32f60 0x55a8fcc37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8fca83fa0 con 0x55a8fcc2e860 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a8fcb1f7c0 con 0x55a8fcc32f60 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8fcc3b0c0 con 0x55a8fcc2f210 2024-05-20T20:56:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc92ae34b80 10 client.0 _ll_register_callbacks cb 0x55a8fcb18920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc3a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47564/0 (socket says 172.21.15.26:47564) 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 -- 172.21.15.26:0/17728440 learned_addr learned my addr 172.21.15.26:0/17728440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91db39700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2f210 0x55a8fcc37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91eb3b700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc32f60 0x55a8fcc37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 -- 172.21.15.26:0/17728440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2f210 msgr2=0x55a8fcc37740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2f210 0x55a8fcc37740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 -- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc32f60 msgr2=0x55a8fcc37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc32f60 0x55a8fcc37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 -- 172.21.15.26:0/17728440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8fcb1c690 con 0x55a8fcc2e860 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc91e33a700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc3a440 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc908009f20 tx=0x7fc908024770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6403c4fdd6e6589c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc901ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc901ffb700 1 -- 172.21.15.26:0/17728440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908034040 con 0x55a8fcc2e860 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc901ffb700 1 -- 172.21.15.26:0/17728440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc90802f530 con 0x55a8fcc2e860 2024-05-20T20:56:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.206+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9080249d0 con 0x55a8fcc2e860 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc901ffb700 1 -- 172.21.15.26:0/17728440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc90802f730 con 0x55a8fcc2e860 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 10 client.15384 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8fcc45880 con 0x55a8fcc2e860 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc901ffb700 1 -- 172.21.15.26:0/17728440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9080249d0 con 0x55a8fcc2e860 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc901ffb700 1 -- 172.21.15.26:0/17728440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc90803a9a0 con 0x55a8fcc2e860 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 20 client.15384 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc47d50 0x55a8fcc4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 4 client.15384 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8fcbfa9e0 con 0x55a8fcc47d50 2024-05-20T20:56:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.207+0000 7fc91db39700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc47d50 0x55a8fcc4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.208+0000 7fc91db39700 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc47d50 0x55a8fcc4a140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68b21d0d6882fca server_cookie=b3294cb3ec330cff in_seq=0 out_seq=0 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.208+0000 7fc901ffb700 10 client.15384 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.208+0000 7fc901ffb700 1 -- 172.21.15.26:0/17728440 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8fcbfa9e0 con 0x55a8fcc47d50 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.208+0000 7fc901ffb700 10 client.15384 handle_command_reply: tid=0 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.208+0000 7fc92ae34b80 1 client.15384 shutdown 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 msgr2=0x55a8fcc3a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc3a440 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc908009f20 tx=0x7fc908024770 comp rx=0 tx=0).stop 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 shutdown_connections 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc47d50 0x55a8fcc4a140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc3a440 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc32f60 0x55a8fcc37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/17728440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2f210 0x55a8fcc37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 >> 172.21.15.26:0/17728440 conn(0x55a8fcb5c080 msgr2=0x55a8fcc28510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc901ffb700 0 client.15384 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 shutdown_connections 2024-05-20T20:56:37.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 1 -- 172.21.15.26:0/17728440 wait complete. 2024-05-20T20:56:37.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.209+0000 7fc92ae34b80 20 client.15384 trim_cache size 0 max 16384 2024-05-20T20:56:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 Processor -- start 2024-05-20T20:56:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 -- start start 2024-05-20T20:56:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 0x55a8fcc37700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc37c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc28c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181b2760 con 0x55a8fcc2e860 2024-05-20T20:56:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181afff0 con 0x55a8fcc2f210 2024-05-20T20:56:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.232+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181ade40 con 0x55a8fcb5c080 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91eb3b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc28c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 0x55a8fcc37700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 0x55a8fcc37700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:60160/0 (socket says 172.21.15.26:60160) 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 -- 172.21.15.26:0/2242620472 learned_addr learned my addr 172.21.15.26:0/2242620472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91db39700 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc37c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 -- 172.21.15.26:0/2242620472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 msgr2=0x55a8fcc28c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc28c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 -- 172.21.15.26:0/2242620472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 msgr2=0x55a8fcc37c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc37c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91e33a700 1 -- 172.21.15.26:0/2242620472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8fcc31f20 con 0x55a8fcb5c080 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.233+0000 7fc91db39700 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc37c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.234+0000 7fc91e33a700 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 0x55a8fcc37700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c010 tx=0x7fc90802f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8ff4e6d7f06344 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.234+0000 7fc903fff700 1 -- 172.21.15.26:0/2242620472 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908034040 con 0x55a8fcb5c080 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.234+0000 7fc903fff700 1 -- 172.21.15.26:0/2242620472 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc90802fd60 con 0x55a8fcb5c080 2024-05-20T20:56:37.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.234+0000 7fc903fff700 1 -- 172.21.15.26:0/2242620472 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908017a00 con 0x55a8fcb5c080 2024-05-20T20:56:37.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 -- 172.21.15.26:0/2242620472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 msgr2=0x55a8fcc37700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 0x55a8fcc37700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c010 tx=0x7fc90802f6c0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 -- 172.21.15.26:0/2242620472 shutdown_connections 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcc28c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcb5c080 0x55a8fcc37700 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/2242620472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcc2e860 0x55a8fcc37c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 -- 172.21.15.26:0/2242620472 >> 172.21.15.26:0/2242620472 conn(0x55a8fcc3a5a0 msgr2=0x55a8fcc36ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 -- 172.21.15.26:0/2242620472 shutdown_connections 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 -- 172.21.15.26:0/2242620472 wait complete. 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 -- start start 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 Processor -- start 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 0x55a8fccab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2e860 0x55a8fccabd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.235+0000 7fc92ae34b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcca6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181ae4b0 con 0x55a8fcb5c080 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181af920 con 0x55a8fcc2f210 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc92ae34b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181aeb80 con 0x55a8fcc2e860 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc92ae34b80 10 client.0 _ll_register_callbacks cb 0x55a8fcb18920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 0x55a8fccab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 0x55a8fccab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47586/0 (socket says 172.21.15.26:47586) 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 -- 172.21.15.26:0/1660231551 learned_addr learned my addr 172.21.15.26:0/1660231551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91db39700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2e860 0x55a8fccabd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91eb3b700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcca6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 -- 172.21.15.26:0/1660231551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2e860 msgr2=0x55a8fccabd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2e860 0x55a8fccabd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 -- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 msgr2=0x55a8fcca6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcca6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 -- 172.21.15.26:0/1660231551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8fcca6d30 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc91e33a700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 0x55a8fccab850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bb90 tx=0x7fc908024ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860a2f3e7520f89a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc901ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc901ffb700 1 -- 172.21.15.26:0/1660231551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90804c060 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc901ffb700 1 -- 172.21.15.26:0/1660231551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc90803a470 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.236+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc90802fd60 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc901ffb700 1 -- 172.21.15.26:0/1660231551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc908034070 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 10 client.15390 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8fcca6d30 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc901ffb700 1 -- 172.21.15.26:0/1660231551 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc90802fd60 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc901ffb700 1 -- 172.21.15.26:0/1660231551 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc908050030 con 0x55a8fcb5c080 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 20 client.15390 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc32f60 0x55a8fccbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 4 client.15390 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8fcb8fbe0 con 0x55a8fcc32f60 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.237+0000 7fc91db39700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc32f60 0x55a8fccbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc91db39700 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc32f60 0x55a8fccbda70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ad403857f2d643 server_cookie=6f7c04d9b0566df8 in_seq=0 out_seq=0 2024-05-20T20:56:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc901ffb700 10 client.15390 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc901ffb700 1 -- 172.21.15.26:0/1660231551 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a8fcb8fbe0 con 0x55a8fcc32f60 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc901ffb700 10 client.15390 handle_command_reply: tid=0 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 client.15390 shutdown 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 msgr2=0x55a8fccab850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 0x55a8fccab850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bb90 tx=0x7fc908024ac0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 shutdown_connections 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8fcc32f60 0x55a8fccbda70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8fcb5c080 0x55a8fccab850 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8fcc2f210 0x55a8fcca6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 --2- 172.21.15.26:0/1660231551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8fcc2e860 0x55a8fccabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 >> 172.21.15.26:0/1660231551 conn(0x55a8fcc3a5a0 msgr2=0x55a8fcc43c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc901ffb700 0 client.15390 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 shutdown_connections 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.238+0000 7fc92ae34b80 1 -- 172.21.15.26:0/1660231551 wait complete. 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92ae34b80 20 client.15390 trim_cache size 0 max 16384 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:56:37.243 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:56:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc9040b2990 msgr2=0x7fc9040b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc9040b2990 0x7fc9040b4e50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc908009750 tx=0x7fc90800b560 comp rx=0 tx=0).stop 2024-05-20T20:56:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 msgr2=0x7fc91819c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 0x7fc91819c300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b530 tx=0x7fc91400b840 comp rx=0 tx=0).stop 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 shutdown_connections 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91810ff90 0x7fc9181a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91806bcf0 0x7fc91819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc9040b2990 0x7fc9040b4e50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 --2- 172.21.15.26:0/4287428146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91806c610 0x7fc91819c300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.239+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 >> 172.21.15.26:0/4287428146 conn(0x7fc9180794e0 msgr2=0x7fc918111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.240+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 shutdown_connections 2024-05-20T20:56:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:37.240+0000 7fc92059e700 1 -- 172.21.15.26:0/4287428146 wait complete. 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 -- 172.21.15.26:0/1765739929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc102bc0 msgr2=0x7f4bbc103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 --2- 172.21.15.26:0/1765739929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc102bc0 0x7f4bbc103040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009860 tx=0x7f4bb002a340 comp rx=0 tx=0).stop 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 -- 172.21.15.26:0/1765739929 shutdown_connections 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 --2- 172.21.15.26:0/1765739929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc103580 0x7f4bbc109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 --2- 172.21.15.26:0/1765739929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc102bc0 0x7f4bbc103040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 --2- 172.21.15.26:0/1765739929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 -- 172.21.15.26:0/1765739929 >> 172.21.15.26:0/1765739929 conn(0x7f4bbc0fcf60 msgr2=0x7f4bbc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 -- 172.21.15.26:0/1765739929 shutdown_connections 2024-05-20T20:56:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 -- 172.21.15.26:0/1765739929 wait complete. 2024-05-20T20:56:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.325+0000 7f4bc11da640 1 Processor -- start 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 -- start start 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc06b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc102bc0 0x7f4bbc19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bbc103580 0x7f4bbc19ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc10d390 con 0x7f4bbc1019d0 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc10d210 con 0x7f4bbc102bc0 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bc11da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc10d510 con 0x7f4bbc103580 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc06b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47614/0 (socket says 172.21.15.26:47614) 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 -- 172.21.15.26:0/780370028 learned_addr learned my addr 172.21.15.26:0/780370028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bba59c640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc102bc0 0x7f4bbc19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 -- 172.21.15.26:0/780370028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bbc103580 msgr2=0x7f4bbc19ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bbc103580 0x7f4bbc19ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bbc103580 0x7f4bbc19ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 -- 172.21.15.26:0/780370028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc102bc0 msgr2=0x7f4bbc19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc102bc0 0x7f4bbc19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.326+0000 7f4bbad9d640 1 -- 172.21.15.26:0/780370028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc1a07f0 con 0x7f4bbc1019d0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.327+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc06b730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00e9f0 tx=0x7f4bac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.327+0000 7f4ba3fff640 1 -- 172.21.15.26:0/780370028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00ccc0 con 0x7f4bbc1019d0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.327+0000 7f4ba3fff640 1 -- 172.21.15.26:0/780370028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bac0035f0 con 0x7f4bbc1019d0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.327+0000 7f4ba3fff640 1 -- 172.21.15.26:0/780370028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00ce60 con 0x7f4bbc1019d0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.327+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc1a0ae0 con 0x7f4bbc1019d0 2024-05-20T20:56:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.327+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc1a1050 con 0x7f4bbc1019d0 2024-05-20T20:56:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 Processor -- start 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 -- start start 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe634fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe6354bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4ba3fff640 1 -- 172.21.15.26:0/780370028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f4bac004000 con 0x7f4bbc1019d0 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe63576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfe62c0c30 con 0x55cfe634f6c0 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cfe62b07d0 con 0x55cfe6355220 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cfe62b4810 con 0x55cfe63500b0 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe634fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe634fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47622/0 (socket says 172.21.15.26:47622) 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 -- 172.21.15.26:0/1052670480 learned_addr learned my addr 172.21.15.26:0/1052670480 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bba59c640 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe63576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe6354bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 -- 172.21.15.26:0/1052670480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 msgr2=0x55cfe6354bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe6354bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 -- 172.21.15.26:0/1052670480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 msgr2=0x55cfe63576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe63576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbb59e640 1 -- 172.21.15.26:0/1052670480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe5fc5210 con 0x55cfe634f6c0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.330+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe6354bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe634fae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ec0 tx=0x7f4bb400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4b6f691dd19f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4ba17fa640 1 -- 172.21.15.26:0/1052670480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004d20 con 0x55cfe634f6c0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4ba17fa640 1 -- 172.21.15.26:0/1052670480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bb40054b0 con 0x55cfe634f6c0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4ba17fa640 1 -- 172.21.15.26:0/1052670480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40118a0 con 0x55cfe634f6c0 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4ba3fff640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ba40b29a0 0x7f4ba40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ba40b29a0 0x7f4ba40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4ba3fff640 1 -- 172.21.15.26:0/780370028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f4bac0e2700 con 0x7f4bbc1019d0 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/1052670480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 msgr2=0x55cfe634fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bbad9d640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ba40b29a0 0x7f4ba40b4e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55cfe6350b20 tx=0x7f4bac003a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe634fae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ec0 tx=0x7f4bb400f730 comp rx=0 tx=0).stop 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/1052670480 shutdown_connections 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe63576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe6354bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/1052670480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe634fae0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.331+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/1052670480 >> 172.21.15.26:0/1052670480 conn(0x55cfe634ad30 msgr2=0x55cfe634b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/1052670480 shutdown_connections 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/1052670480 wait complete. 2024-05-20T20:56:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 -- start start 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 Processor -- start 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe63597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe63500b0 0x55cfe6359cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe6355220 0x55cfe6360d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfe5f96790 con 0x55cfe634f6c0 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cfe626c890 con 0x55cfe63500b0 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cfe63581d0 con 0x55cfe6355220 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bc3fc8740 10 client.0 _ll_register_callbacks cb 0x55cfe5fc4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe63597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe63597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47632/0 (socket says 172.21.15.26:47632) 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bbb59e640 1 -- 172.21.15.26:0/4115358270 learned_addr learned my addr 172.21.15.26:0/4115358270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bba59c640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe63500b0 0x55cfe6359cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.332+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe6355220 0x55cfe6360d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bbb59e640 1 -- 172.21.15.26:0/4115358270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe6355220 msgr2=0x55cfe6360d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe6355220 0x55cfe6360d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bbb59e640 1 -- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe63500b0 msgr2=0x55cfe6359cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe63500b0 0x55cfe6359cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bbb59e640 1 -- 172.21.15.26:0/4115358270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe6091620 con 0x55cfe634f6c0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe63597b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009370 tx=0x7f4bb40092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef96c5c568d6c22 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4b9eff5640 1 -- 172.21.15.26:0/4115358270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4005860 con 0x55cfe634f6c0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4b9eff5640 1 -- 172.21.15.26:0/4115358270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f4bb4003970 con 0x55cfe634f6c0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb40054b0 con 0x55cfe634f6c0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4b9eff5640 1 -- 172.21.15.26:0/4115358270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4bb4003b70 con 0x55cfe634f6c0 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 10 client.15402 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfe6363d10 con 0x55cfe634f6c0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4b9eff5640 1 -- 172.21.15.26:0/4115358270 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bb40054b0 con 0x55cfe634f6c0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4b9eff5640 1 -- 172.21.15.26:0/4115358270 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4bb40115e0 con 0x55cfe634f6c0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 10 client.15402 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 10 client.15402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 20 client.15402 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63650c0 0x55cfe63674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 4 client.15402 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.333+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe62c2b80 con 0x55cfe63650c0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.334+0000 7f4bba59c640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63650c0 0x55cfe63674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.334+0000 7f4b9eff5640 10 client.15402 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.334+0000 7f4bba59c640 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63650c0 0x55cfe63674b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403010382632a0a0 server_cookie=e0e4ebf526ee4ca4 in_seq=0 out_seq=0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4b9eff5640 1 -- 172.21.15.26:0/4115358270 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cfe62c2b80 con 0x55cfe63650c0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4b9eff5640 10 client.15402 handle_command_reply: tid=0 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 client.15402 shutdown 2024-05-20T20:56:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 msgr2=0x55cfe63597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe63597b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009370 tx=0x7f4bb40092a0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 shutdown_connections 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63650c0 0x55cfe63674b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe6355220 0x55cfe6360d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4b9eff5640 0 client.15402 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe63500b0 0x55cfe6359cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/4115358270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe63597b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 >> 172.21.15.26:0/4115358270 conn(0x55cfe634ad30 msgr2=0x55cfe6349fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 shutdown_connections 2024-05-20T20:56:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/4115358270 wait complete. 2024-05-20T20:56:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.335+0000 7f4bc3fc8740 20 client.15402 trim_cache size 0 max 16384 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 Processor -- start 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 -- start start 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe634f6c0 0x55cfe6345e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 0x55cfe6360cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe636cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1ac8c0 con 0x55cfe63500b0 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1aa190 con 0x55cfe6355220 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1a7fe0 con 0x55cfe634f6c0 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 0x55cfe6360cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 0x55cfe6360cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47646/0 (socket says 172.21.15.26:47646) 2024-05-20T20:56:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 -- 172.21.15.26:0/2662205415 learned_addr learned my addr 172.21.15.26:0/2662205415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe634f6c0 0x55cfe6345e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe636cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 -- 172.21.15.26:0/2662205415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe634f6c0 msgr2=0x55cfe6345e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe634f6c0 0x55cfe6345e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 -- 172.21.15.26:0/2662205415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 msgr2=0x55cfe636cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe636cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.351+0000 7f4bba59c640 1 -- 172.21.15.26:0/2662205415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe634f300 con 0x55cfe63500b0 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bba59c640 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 0x55cfe6360cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005e00 tx=0x7f4bb00097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac804d09849bb22 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4ba0ff9640 1 -- 172.21.15.26:0/2662205415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb004d070 con 0x55cfe63500b0 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4ba0ff9640 1 -- 172.21.15.26:0/2662205415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bb0030650 con 0x55cfe63500b0 2024-05-20T20:56:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4ba0ff9640 1 -- 172.21.15.26:0/2662205415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0037040 con 0x55cfe63500b0 2024-05-20T20:56:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/2662205415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 msgr2=0x55cfe6360cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 0x55cfe6360cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005e00 tx=0x7f4bb00097f0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/2662205415 shutdown_connections 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe636cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe63500b0 0x55cfe6360cf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/2662205415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe634f6c0 0x55cfe6345e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.352+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/2662205415 >> 172.21.15.26:0/2662205415 conn(0x55cfe634ad30 msgr2=0x55cfe6354900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/2662205415 shutdown_connections 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/2662205415 wait complete. 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 -- start start 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 Processor -- start 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe635e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe635e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe635f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1a8650 con 0x55cfe634f6c0 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1a9ac0 con 0x55cfe6355220 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1a8d20 con 0x55cfe63500b0 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bc3fc8740 10 client.0 _ll_register_callbacks cb 0x7f4bb4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe635e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe635e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47662/0 (socket says 172.21.15.26:47662) 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bbb59e640 1 -- 172.21.15.26:0/3949708841 learned_addr learned my addr 172.21.15.26:0/3949708841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bba59c640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe635e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.353+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe635f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bbb59e640 1 -- 172.21.15.26:0/3949708841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 msgr2=0x55cfe635e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe635e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bbb59e640 1 -- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 msgr2=0x55cfe635f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe635f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bbb59e640 1 -- 172.21.15.26:0/3949708841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe6350d80 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bbb59e640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe635e230 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400a010 tx=0x7f4bb4004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4351ae3641c34942 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4b9eff5640 1 -- 172.21.15.26:0/3949708841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4011650 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4b9eff5640 1 -- 172.21.15.26:0/3949708841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f4bb4005260 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb0030650 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4b9eff5640 1 -- 172.21.15.26:0/3949708841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4bb4005460 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bc3fc8740 10 client.15408 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfe6360e60 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.354+0000 7f4b9eff5640 1 -- 172.21.15.26:0/3949708841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bb0030650 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4b9eff5640 1 -- 172.21.15.26:0/3949708841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4bb4035830 con 0x55cfe634f6c0 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bc3fc8740 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bc3fc8740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bc3fc8740 20 client.15408 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63d0090 0x55cfe63d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bc3fc8740 4 client.15408 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:56:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfe62bc360 con 0x55cfe63d0090 2024-05-20T20:56:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bba59c640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63d0090 0x55cfe63d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4bba59c640 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63d0090 0x55cfe63d24b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e37230612dce470 server_cookie=ba983d4857609d0c in_seq=0 out_seq=0 2024-05-20T20:56:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.355+0000 7f4b9eff5640 10 client.15408 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4b9eff5640 1 -- 172.21.15.26:0/3949708841 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cfe62bc360 con 0x55cfe63d0090 2024-05-20T20:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4b9eff5640 10 client.15408 handle_command_reply: tid=0 2024-05-20T20:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 client.15408 shutdown 2024-05-20T20:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 msgr2=0x55cfe635e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe635e230 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400a010 tx=0x7f4bb4004e80 comp rx=0 tx=0).stop 2024-05-20T20:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 shutdown_connections 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cfe63d0090 0x55cfe63d24b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe6355220 0x55cfe635f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe63500b0 0x55cfe635e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4b9eff5640 0 client.15408 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 --2- 172.21.15.26:0/3949708841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe634f6c0 0x55cfe635e230 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 >> 172.21.15.26:0/3949708841 conn(0x55cfe634ad30 msgr2=0x55cfe63548c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 shutdown_connections 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 1 -- 172.21.15.26:0/3949708841 wait complete. 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc3fc8740 20 client.15408 trim_cache size 0 max 16384 2024-05-20T20:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ba40b29a0 msgr2=0x7f4ba40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc11da640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ba40b29a0 0x7f4ba40b4e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55cfe6350b20 tx=0x7f4bac003a10 comp rx=0 tx=0).stop 2024-05-20T20:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 msgr2=0x7f4bbc06b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.363+0000 7f4bc11da640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc06b730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00e9f0 tx=0x7f4bac00eeb0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 shutdown_connections 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ba40b29a0 0x7f4ba40b4e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bbc103580 0x7f4bbc19ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc102bc0 0x7f4bbc19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 --2- 172.21.15.26:0/780370028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1019d0 0x7f4bbc06b730 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 >> 172.21.15.26:0/780370028 conn(0x7f4bbc0fcf60 msgr2=0x7f4bbc105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 shutdown_connections 2024-05-20T20:56:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:37.364+0000 7f4bc11da640 1 -- 172.21.15.26:0/780370028 wait complete. 2024-05-20T20:56:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:37 smithi144 ceph-mon[35660]: pgmap v402: 97 pgs: 97 active+clean; 667 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T20:56:37.646 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:37 smithi026 ceph-mon[31990]: pgmap v402: 97 pgs: 97 active+clean; 667 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T20:56:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:37 smithi055 ceph-mon[35213]: pgmap v402: 97 pgs: 97 active+clean; 667 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T20:56:37.941 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:56:37.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:56:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:39.311 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:39 smithi026 ceph-mon[31990]: pgmap v403: 97 pgs: 97 active+clean; 667 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-20T20:56:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:39 smithi144 ceph-mon[35660]: pgmap v403: 97 pgs: 97 active+clean; 667 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-20T20:56:39.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:39 smithi055 ceph-mon[35213]: pgmap v403: 97 pgs: 97 active+clean; 667 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-20T20:56:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 -- 172.21.15.26:0/741315837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bc70 msgr2=0x7f434c06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 --2- 172.21.15.26:0/741315837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bc70 0x7f434c06c0f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4340009990 tx=0x7f4340009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 -- 172.21.15.26:0/741315837 shutdown_connections 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 --2- 172.21.15.26:0/741315837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06c630 0x7f434c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 --2- 172.21.15.26:0/741315837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c1095b0 0x7f434c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 --2- 172.21.15.26:0/741315837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bc70 0x7f434c06c0f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.845+0000 7f43510f0700 1 -- 172.21.15.26:0/741315837 >> 172.21.15.26:0/741315837 conn(0x7f434c0793f0 msgr2=0x7f434c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.846+0000 7f43510f0700 1 -- 172.21.15.26:0/741315837 shutdown_connections 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.846+0000 7f43510f0700 1 -- 172.21.15.26:0/741315837 wait complete. 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.846+0000 7f43510f0700 1 Processor -- start 2024-05-20T20:56:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 -- start start 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06bc70 0x7f434c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06c630 0x7f434c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 0x7f434c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1135f0 con 0x7f434c1095b0 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f434c113470 con 0x7f434c06c630 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f43510f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f434c113770 con 0x7f434c06bc70 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 0x7f434c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 0x7f434c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40874/0 (socket says 172.21.15.26:40874) 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 -- 172.21.15.26:0/3286908196 learned_addr learned my addr 172.21.15.26:0/3286908196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434a59c700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06c630 0x7f434c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434ad9d700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06bc70 0x7f434c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 -- 172.21.15.26:0/3286908196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06bc70 msgr2=0x7f434c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06bc70 0x7f434c19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 -- 172.21.15.26:0/3286908196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06c630 msgr2=0x7f434c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06c630 0x7f434c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434b59e700 1 -- 172.21.15.26:0/3286908196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c19a150 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.847+0000 7f434ad9d700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06bc70 0x7f434c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.848+0000 7f434b59e700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 0x7f434c1999f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f434400b550 tx=0x7f434400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.848+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344015020 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.848+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434c19a440 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.848+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4344011cc0 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.848+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344010c80 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.848+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434c1a6700 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.849+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f434c06a230 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.850+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f4344018020 con 0x7f434c1095b0 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.851+0000 7f4333fff700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2910 0x7f43340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.851+0000 7f434ad9d700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2910 0x7f43340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.851+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f43440e1fc0 con 0x7f434c1095b0 2024-05-20T20:56:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.852+0000 7f434ad9d700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2910 0x7f43340b4dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f434c19ab30 tx=0x7f433c009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:39.855+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f434401f020 con 0x7f434c1095b0 2024-05-20T20:56:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.118+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f434c02e560 con 0x7f434c1095b0 2024-05-20T20:56:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.118+0000 7f4333fff700 1 -- 172.21.15.26:0/3286908196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f43440a9c30 con 0x7f434c1095b0 2024-05-20T20:56:40.119 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:40.119 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2910 msgr2=0x7f43340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2910 0x7f43340b4dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f434c19ab30 tx=0x7f433c009040 comp rx=0 tx=0).stop 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 msgr2=0x7f434c1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 0x7f434c1999f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f434400b550 tx=0x7f434400b860 comp rx=0 tx=0).stop 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 shutdown_connections 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2910 0x7f43340b4dd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c1095b0 0x7f434c1999f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c06bc70 0x7f434c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 --2- 172.21.15.26:0/3286908196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06c630 0x7f434c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 >> 172.21.15.26:0/3286908196 conn(0x7f434c0793f0 msgr2=0x7f434c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 shutdown_connections 2024-05-20T20:56:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:40.121+0000 7f43510f0700 1 -- 172.21.15.26:0/3286908196 wait complete. 2024-05-20T20:56:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:56:40.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:40 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3286908196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:40.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:40 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3286908196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:40.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:40 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3286908196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:40.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:56:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:41.440 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:41 smithi026 ceph-mon[31990]: pgmap v404: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-20T20:56:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:41 smithi144 ceph-mon[35660]: pgmap v404: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-20T20:56:41.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:41 smithi055 ceph-mon[35213]: pgmap v404: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-20T20:56:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 -- 172.21.15.26:0/2622938226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45541095b0 msgr2=0x7f4554109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 --2- 172.21.15.26:0/2622938226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45541095b0 0x7f4554109990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4544009990 tx=0x7f4544009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 -- 172.21.15.26:0/2622938226 shutdown_connections 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 --2- 172.21.15.26:0/2622938226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45541095b0 0x7f4554109990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 --2- 172.21.15.26:0/2622938226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455406c650 0x7f45541103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 --2- 172.21.15.26:0/2622938226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455406bc90 0x7f455406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 -- 172.21.15.26:0/2622938226 >> 172.21.15.26:0/2622938226 conn(0x7f4554079410 msgr2=0x7f4554079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 -- 172.21.15.26:0/2622938226 shutdown_connections 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 -- 172.21.15.26:0/2622938226 wait complete. 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.836+0000 7f45594c0700 1 Processor -- start 2024-05-20T20:56:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 -- start start 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 0x7f455419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455406c650 0x7f455419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45541095b0 0x7f4554199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45541135d0 con 0x7f455406bc90 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4554113450 con 0x7f455406c650 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45594c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4554113750 con 0x7f45541095b0 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 0x7f455419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 0x7f455419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40902/0 (socket says 172.21.15.26:40902) 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 -- 172.21.15.26:0/1666284906 learned_addr learned my addr 172.21.15.26:0/1666284906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 -- 172.21.15.26:0/1666284906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45541095b0 msgr2=0x7f4554199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45541095b0 0x7f4554199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f45527fc700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455406c650 0x7f455419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 -- 172.21.15.26:0/1666284906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455406c650 msgr2=0x7f455419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455406c650 0x7f455419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 -- 172.21.15.26:0/1666284906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419a0d0 con 0x7f455406bc90 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.837+0000 7f4552ffd700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 0x7f455419f7b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4544005910 tx=0x7f45440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.838+0000 7f453bfff700 1 -- 172.21.15.26:0/1666284906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544027070 con 0x7f455406bc90 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.838+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419a360 con 0x7f455406bc90 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.838+0000 7f453bfff700 1 -- 172.21.15.26:0/1666284906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f454400fc00 con 0x7f455406bc90 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.838+0000 7f453bfff700 1 -- 172.21.15.26:0/1666284906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544022d40 con 0x7f455406bc90 2024-05-20T20:56:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.838+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a6680 con 0x7f455406bc90 2024-05-20T20:56:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.840+0000 7f453bfff700 1 -- 172.21.15.26:0/1666284906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f454402a070 con 0x7f455406bc90 2024-05-20T20:56:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.841+0000 7f453bfff700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f453c0b2910 0x7f453c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.841+0000 7f45527fc700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f453c0b2910 0x7f453c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.841+0000 7f453bfff700 1 -- 172.21.15.26:0/1666284906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f45440e31d0 con 0x7f455406bc90 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.841+0000 7f45527fc700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f453c0b2910 0x7f453c0b4dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4554079a60 tx=0x7f4548005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.841+0000 7f4563d56b80 1 Processor -- start 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 -- start start 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c90730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55afc3be9d70 con 0x55afc3c902d0 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afc3bd9910 con 0x55afc3c95610 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4563d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afc3bdd950 con 0x55afc3c95fc0 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c90730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c90730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40912/0 (socket says 172.21.15.26:40912) 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 -- 172.21.15.26:0/2897655623 learned_addr learned my addr 172.21.15.26:0/2897655623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45527fc700 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4552ffd700 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 -- 172.21.15.26:0/2897655623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 msgr2=0x55afc3c8fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c8fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 -- 172.21.15.26:0/2897655623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 msgr2=0x55afc3c959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 -- 172.21.15.26:0/2897655623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc3b82fb0 con 0x55afc3c902d0 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f4552ffd700 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.842+0000 7f45537fe700 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c90730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f454c009d10 tx=0x7f454c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65881939fe41a928 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f45397fa700 1 -- 172.21.15.26:0/2897655623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c003c00 con 0x55afc3c902d0 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f45397fa700 1 -- 172.21.15.26:0/2897655623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f454c003da0 con 0x55afc3c902d0 2024-05-20T20:56:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f45397fa700 1 -- 172.21.15.26:0/2897655623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0136e0 con 0x55afc3c902d0 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 -- 172.21.15.26:0/2897655623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 msgr2=0x55afc3c90730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c90730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f454c009d10 tx=0x7f454c0077e0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 -- 172.21.15.26:0/2897655623 shutdown_connections 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c90730 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2897655623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c8fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 -- 172.21.15.26:0/2897655623 >> 172.21.15.26:0/2897655623 conn(0x55afc3bb7240 msgr2=0x55afc3c863e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 -- 172.21.15.26:0/2897655623 shutdown_connections 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.843+0000 7f4563d56b80 1 -- 172.21.15.26:0/2897655623 wait complete. 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 -- start start 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 Processor -- start 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55afc3adffa0 con 0x55afc3c902d0 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afc3b41da0 con 0x55afc3c95610 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afc3c9caa0 con 0x55afc3c95fc0 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4563d56b80 10 client.0 _ll_register_callbacks cb 0x55afc3b74950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c9bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40920/0 (socket says 172.21.15.26:40920) 2024-05-20T20:56:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.844+0000 7f4552ffd700 1 -- 172.21.15.26:0/1091302763 learned_addr learned my addr 172.21.15.26:0/1091302763 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f45537fe700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f45527fc700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4552ffd700 1 -- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 msgr2=0x55afc3c995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4552ffd700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4552ffd700 1 -- 172.21.15.26:0/1091302763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 msgr2=0x55afc3c990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4552ffd700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4552ffd700 1 -- 172.21.15.26:0/1091302763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc3b298d0 con 0x55afc3c902d0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4552ffd700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c9bdb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f45440aa360 tx=0x7f4544009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab574c31b33c17c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4536ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4536ff5700 1 -- 172.21.15.26:0/1091302763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440aad60 con 0x55afc3c902d0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4536ff5700 1 -- 172.21.15.26:0/1091302763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f45440920f0 con 0x55afc3c902d0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c003da0 con 0x55afc3c902d0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4536ff5700 1 -- 172.21.15.26:0/1091302763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f454406d070 con 0x55afc3c902d0 2024-05-20T20:56:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4563d56b80 10 client.15426 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afc3b93760 con 0x55afc3c902d0 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.845+0000 7f4536ff5700 1 -- 172.21.15.26:0/1091302763 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f454c003da0 con 0x55afc3c902d0 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4536ff5700 1 -- 172.21.15.26:0/1091302763 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f45440b2fa0 con 0x55afc3c902d0 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4563d56b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4563d56b80 10 client.15426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4563d56b80 20 client.15426 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4563d56b80 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3ca9600 0x55afc3caba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4563d56b80 4 client.15426 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc3bebcc0 con 0x55afc3ca9600 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.846+0000 7f45527fc700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3ca9600 0x55afc3caba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f4536ff5700 10 client.15426 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f45527fc700 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3ca9600 0x55afc3caba80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c59ec32b5ddc512 server_cookie=5d9b0c49e61cd201 in_seq=0 out_seq=0 2024-05-20T20:56:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f4536ff5700 1 -- 172.21.15.26:0/1091302763 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55afc3bebcc0 con 0x55afc3ca9600 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f4536ff5700 10 client.15426 handle_command_reply: tid=0 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f4563d56b80 1 client.15426 shutdown 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 msgr2=0x55afc3c9bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.847+0000 7f4563d56b80 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c9bdb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f45440aa360 tx=0x7f4544009710 comp rx=0 tx=0).stop 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 shutdown_connections 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3ca9600 0x55afc3caba80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c902d0 0x55afc3c9bdb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4536ff5700 0 client.15426 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 --2- 172.21.15.26:0/1091302763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95610 0x55afc3c990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 >> 172.21.15.26:0/1091302763 conn(0x55afc3bb7240 msgr2=0x55afc3c85af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 shutdown_connections 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 1 -- 172.21.15.26:0/1091302763 wait complete. 2024-05-20T20:56:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.848+0000 7f4563d56b80 20 client.15426 trim_cache size 0 max 16384 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 Processor -- start 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 -- start start 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c902d0 0x55afc3ca5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3c96c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ae250 con 0x55afc3c95610 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45541abae0 con 0x55afc3c902d0 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.863+0000 7f4563d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a9930 con 0x55afc3c95fc0 2024-05-20T20:56:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3c96c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3c96c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40934/0 (socket says 172.21.15.26:40934) 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 -- 172.21.15.26:0/2487307408 learned_addr learned my addr 172.21.15.26:0/2487307408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f4552ffd700 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c902d0 0x55afc3ca5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45537fe700 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 -- 172.21.15.26:0/2487307408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 msgr2=0x55afc3c937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 -- 172.21.15.26:0/2487307408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c902d0 msgr2=0x55afc3ca5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c902d0 0x55afc3ca5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 -- 172.21.15.26:0/2487307408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc3c882b0 con 0x55afc3c95610 2024-05-20T20:56:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f45527fc700 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3c96c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f454800b390 tx=0x7f4548013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dc491d4d1ab438 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f4538ff9700 1 -- 172.21.15.26:0/2487307408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454801fc40 con 0x55afc3c95610 2024-05-20T20:56:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f4538ff9700 1 -- 172.21.15.26:0/2487307408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f454802d070 con 0x55afc3c95610 2024-05-20T20:56:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.864+0000 7f4538ff9700 1 -- 172.21.15.26:0/2487307408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548027c60 con 0x55afc3c95610 2024-05-20T20:56:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 -- 172.21.15.26:0/2487307408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 msgr2=0x55afc3c96c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3c96c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f454800b390 tx=0x7f4548013000 comp rx=0 tx=0).stop 2024-05-20T20:56:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 -- 172.21.15.26:0/2487307408 shutdown_connections 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c95fc0 0x55afc3c937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c902d0 0x55afc3ca5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 --2- 172.21.15.26:0/2487307408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3c96c80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 -- 172.21.15.26:0/2487307408 >> 172.21.15.26:0/2487307408 conn(0x55afc3bb7240 msgr2=0x55afc3c8a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 -- 172.21.15.26:0/2487307408 shutdown_connections 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.865+0000 7f4563d56b80 1 -- 172.21.15.26:0/2487307408 wait complete. 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 -- start start 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 Processor -- start 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c902d0 0x55afc3ca2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3cb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95fc0 0x55afc3cb0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a9fa0 con 0x55afc3c95610 2024-05-20T20:56:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ab410 con 0x55afc3c95fc0 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45541aa670 con 0x55afc3c902d0 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4563d56b80 10 client.0 _ll_register_callbacks cb 0x55afc3b74950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3cb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3cb0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40940/0 (socket says 172.21.15.26:40940) 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 -- 172.21.15.26:0/58649676 learned_addr learned my addr 172.21.15.26:0/58649676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45537fe700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95fc0 0x55afc3cb0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f4552ffd700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c902d0 0x55afc3ca2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 -- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c902d0 msgr2=0x55afc3ca2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c902d0 0x55afc3ca2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 -- 172.21.15.26:0/58649676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95fc0 msgr2=0x55afc3cb0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95fc0 0x55afc3cb0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45527fc700 1 -- 172.21.15.26:0/58649676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc3c90020 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.866+0000 7f45537fe700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95fc0 0x55afc3cb0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f45527fc700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3cb0a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4548006010 tx=0x7f454800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dbda06c01fa281 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4536ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4536ff5700 1 -- 172.21.15.26:0/58649676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548035070 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4536ff5700 1 -- 172.21.15.26:0/58649676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f45480275b0 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454802d070 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4536ff5700 1 -- 172.21.15.26:0/58649676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4548041050 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4563d56b80 10 client.15432 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afc3d13e80 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.867+0000 7f4536ff5700 1 -- 172.21.15.26:0/58649676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f454802d070 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4536ff5700 1 -- 172.21.15.26:0/58649676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f45480047f0 con 0x55afc3c95610 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4563d56b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4563d56b80 10 client.15432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4563d56b80 20 client.15432 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4563d56b80 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3d14530 0x55afc3d16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4563d56b80 4 client.15432 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afc3be54a0 con 0x55afc3d14530 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4552ffd700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3d14530 0x55afc3d16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4552ffd700 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3d14530 0x55afc3d16950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b783d571825f009 server_cookie=55bf17b592601984 in_seq=0 out_seq=0 2024-05-20T20:56:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.868+0000 7f4536ff5700 10 client.15432 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4536ff5700 1 -- 172.21.15.26:0/58649676 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afc3be54a0 con 0x55afc3d14530 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4536ff5700 10 client.15432 handle_command_reply: tid=0 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 client.15432 shutdown 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 msgr2=0x55afc3cb0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3cb0a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4548006010 tx=0x7f454800da10 comp rx=0 tx=0).stop 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 shutdown_connections 2024-05-20T20:56:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55afc3d14530 0x55afc3d16950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4536ff5700 0 client.15432 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afc3c95fc0 0x55afc3cb0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afc3c902d0 0x55afc3ca2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 --2- 172.21.15.26:0/58649676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afc3c95610 0x55afc3cb0a60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 >> 172.21.15.26:0/58649676 conn(0x55afc3bb7240 msgr2=0x55afc3c8a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 shutdown_connections 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 1 -- 172.21.15.26:0/58649676 wait complete. 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.869+0000 7f4563d56b80 20 client.15432 trim_cache size 0 max 16384 2024-05-20T20:56:42.874 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:56:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f453c0b2910 msgr2=0x7f453c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f453c0b2910 0x7f453c0b4dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4554079a60 tx=0x7f4548005d50 comp rx=0 tx=0).stop 2024-05-20T20:56:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 msgr2=0x7f455419f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 0x7f455419f7b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4544005910 tx=0x7f45440047b0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 shutdown_connections 2024-05-20T20:56:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f453c0b2910 0x7f453c0b4dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455406bc90 0x7f455419f7b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45541095b0 0x7f4554199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 --2- 172.21.15.26:0/1666284906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455406c650 0x7f455419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 >> 172.21.15.26:0/1666284906 conn(0x7f4554079410 msgr2=0x7f45541011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 shutdown_connections 2024-05-20T20:56:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:42.870+0000 7f45594c0700 1 -- 172.21.15.26:0/1666284906 wait complete. 2024-05-20T20:56:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:43 smithi144 ceph-mon[35660]: pgmap v405: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-20T20:56:43.555 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:43 smithi026 ceph-mon[31990]: pgmap v405: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-20T20:56:43.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:56:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:43 smithi055 ceph-mon[35213]: pgmap v405: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-20T20:56:43.849 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:45.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:45 smithi026 ceph-mon[31990]: pgmap v406: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T20:56:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:45 smithi144 ceph-mon[35660]: pgmap v406: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T20:56:45.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:45 smithi055 ceph-mon[35213]: pgmap v406: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 -- 172.21.15.26:0/1152509354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3141084f0 msgr2=0x7fc3141088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 --2- 172.21.15.26:0/1152509354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3141084f0 0x7fc3141088d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc304009990 tx=0x7fc304009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 -- 172.21.15.26:0/1152509354 shutdown_connections 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 --2- 172.21.15.26:0/1152509354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3141084f0 0x7fc3141088d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 --2- 172.21.15.26:0/1152509354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc314102e20 0x7fc31410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 --2- 172.21.15.26:0/1152509354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314102460 0x7fc3141028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 -- 172.21.15.26:0/1152509354 >> 172.21.15.26:0/1152509354 conn(0x7fc3140794e0 msgr2=0x7fc3140798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 -- 172.21.15.26:0/1152509354 shutdown_connections 2024-05-20T20:56:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 -- 172.21.15.26:0/1152509354 wait complete. 2024-05-20T20:56:45.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 Processor -- start 2024-05-20T20:56:45.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.703+0000 7fc31adf3700 1 -- start start 2024-05-20T20:56:45.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc31adf3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 0x7fc3140ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc31adf3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314102e20 0x7fc3140ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc31adf3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3141084f0 0x7fc3140fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc31adf3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141124b0 con 0x7fc314102460 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc31adf3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc314112330 con 0x7fc3141084f0 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc31adf3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc314112630 con 0x7fc314102e20 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 0x7fc3140ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 0x7fc3140ff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40956/0 (socket says 172.21.15.26:40956) 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 -- 172.21.15.26:0/4064957032 learned_addr learned my addr 172.21.15.26:0/4064957032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc313fff700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314102e20 0x7fc3140ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc319390700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3141084f0 0x7fc3140fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 -- 172.21.15.26:0/4064957032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314102e20 msgr2=0x7fc3140ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314102e20 0x7fc3140ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 -- 172.21.15.26:0/4064957032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3141084f0 msgr2=0x7fc3140fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3141084f0 0x7fc3140fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 -- 172.21.15.26:0/4064957032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3141a8060 con 0x7fc314102460 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.704+0000 7fc318b8f700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 0x7fc3140ff490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc304000c00 tx=0x7fc3040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.705+0000 7fc311ffb700 1 -- 172.21.15.26:0/4064957032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304027070 con 0x7fc314102460 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.705+0000 7fc311ffb700 1 -- 172.21.15.26:0/4064957032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc30400fb00 con 0x7fc314102460 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.705+0000 7fc311ffb700 1 -- 172.21.15.26:0/4064957032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304022c40 con 0x7fc314102460 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.705+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3141a82f0 con 0x7fc314102460 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.705+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3141a8740 con 0x7fc314102460 2024-05-20T20:56:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.707+0000 7fc311ffb700 1 -- 172.21.15.26:0/4064957032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fc30402a070 con 0x7fc314102460 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.708+0000 7fc311ffb700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3000b2910 0x7fc3000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.708+0000 7fc313fff700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3000b2910 0x7fc3000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.708+0000 7fc311ffb700 1 -- 172.21.15.26:0/4064957032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc3040e3320 con 0x7fc314102460 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 Processor -- start 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 -- start start 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc313fff700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3000b2910 0x7fc3000b4dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc31406cd50 tx=0x7fc2fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 0x5576bada4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4a20 0x5576bada8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576bada8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5576bacfac90 con 0x5576bada4070 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5576bacea830 con 0x5576bada8770 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc325689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576bacee870 con 0x5576bada4a20 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 0x5576bada4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 0x5576bada4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40970/0 (socket says 172.21.15.26:40970) 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 -- 172.21.15.26:0/3740590191 learned_addr learned my addr 172.21.15.26:0/3740590191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc319390700 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576bada8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc313fff700 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4a20 0x5576bada8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 -- 172.21.15.26:0/3740590191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4a20 msgr2=0x5576bada8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4a20 0x5576bada8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 -- 172.21.15.26:0/3740590191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 msgr2=0x5576bada8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.709+0000 7fc318b8f700 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576bada8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.710+0000 7fc318b8f700 1 -- 172.21.15.26:0/3740590191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bac3a8d0 con 0x5576bada4070 2024-05-20T20:56:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.710+0000 7fc319390700 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576bada8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.710+0000 7fc318b8f700 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 0x5576bada4450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b3050 tx=0x7fc3040aaf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b618d04055bffc1f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.710+0000 7fc30affd700 1 -- 172.21.15.26:0/3740590191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040aeb60 con 0x5576bada4070 2024-05-20T20:56:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.710+0000 7fc30affd700 1 -- 172.21.15.26:0/3740590191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3040b1e50 con 0x5576bada4070 2024-05-20T20:56:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.710+0000 7fc30affd700 1 -- 172.21.15.26:0/3740590191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040b30f0 con 0x5576bada4070 2024-05-20T20:56:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 -- 172.21.15.26:0/3740590191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 msgr2=0x5576bada4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 0x5576bada4450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b3050 tx=0x7fc3040aaf30 comp rx=0 tx=0).stop 2024-05-20T20:56:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 -- 172.21.15.26:0/3740590191 shutdown_connections 2024-05-20T20:56:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4070 0x5576bada4450 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576bada8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 --2- 172.21.15.26:0/3740590191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4a20 0x5576bada8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 -- 172.21.15.26:0/3740590191 >> 172.21.15.26:0/3740590191 conn(0x5576bacc9080 msgr2=0x5576bad95e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 -- 172.21.15.26:0/3740590191 shutdown_connections 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 -- 172.21.15.26:0/3740590191 wait complete. 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 -- start start 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.711+0000 7fc325689b80 1 Processor -- start 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badafca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5576babf0fa0 con 0x5576bada4a20 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5576bac8c7c0 con 0x5576bada8770 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576badb06f0 con 0x5576bada4070 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 10 client.0 _ll_register_callbacks cb 0x5576bac85920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badacfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40976/0 (socket says 172.21.15.26:40976) 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 -- 172.21.15.26:0/913310418 learned_addr learned my addr 172.21.15.26:0/913310418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc318b8f700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badafca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 -- 172.21.15.26:0/913310418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 msgr2=0x5576badafca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc319390700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badafca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 -- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 msgr2=0x5576badad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badad4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 -- 172.21.15.26:0/913310418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bac89690 con 0x5576bada4a20 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc313fff700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badacfa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0116f0 tx=0x7fc2fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9378ac95e135082c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc2f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc2f3fff700 1 -- 172.21.15.26:0/913310418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc01f840 con 0x5576bada4a20 2024-05-20T20:56:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.712+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3040b1e50 con 0x5576bada4a20 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc2f3fff700 1 -- 172.21.15.26:0/913310418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc2fc01f640 con 0x5576bada4a20 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc2f3fff700 1 -- 172.21.15.26:0/913310418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc01f840 con 0x5576bada4a20 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 10 client.15444 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576badbae90 con 0x5576bada4a20 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc2f3fff700 1 -- 172.21.15.26:0/913310418 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3040b1e50 con 0x5576bada4a20 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc2f3fff700 1 -- 172.21.15.26:0/913310418 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2fc0328b0 con 0x5576bada4a20 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 10 client.15444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 20 client.15444 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576badbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 4 client.15444 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.713+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576bad679e0 con 0x5576badbd360 2024-05-20T20:56:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.714+0000 7fc318b8f700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576badbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.714+0000 7fc318b8f700 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576badbf770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2185f9a3bf797972 server_cookie=d96284da4079a622 in_seq=0 out_seq=0 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.714+0000 7fc2f3fff700 10 client.15444 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc2f3fff700 1 -- 172.21.15.26:0/913310418 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5576bad679e0 con 0x5576badbd360 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc2f3fff700 10 client.15444 handle_command_reply: tid=0 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 client.15444 shutdown 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 msgr2=0x5576badacfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badacfa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0116f0 tx=0x7fc2fc0047c0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 shutdown_connections 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576badbf770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badafca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc2f3fff700 0 client.15444 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 --2- 172.21.15.26:0/913310418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badacfa0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 >> 172.21.15.26:0/913310418 conn(0x5576bacc9080 msgr2=0x5576bad95510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 shutdown_connections 2024-05-20T20:56:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 1 -- 172.21.15.26:0/913310418 wait complete. 2024-05-20T20:56:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.715+0000 7fc325689b80 20 client.15444 trim_cache size 0 max 16384 2024-05-20T20:56:45.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.730+0000 7fc325689b80 1 Processor -- start 2024-05-20T20:56:45.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.730+0000 7fc325689b80 1 -- start start 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc325689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badaf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc325689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badc5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc325689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badc5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc325689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141ae470 con 0x5576bada4a20 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc325689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141abd00 con 0x5576bada8770 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc325689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141a9bb0 con 0x5576bada4070 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badc5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badc5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40982/0 (socket says 172.21.15.26:40982) 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 -- 172.21.15.26:0/3986851880 learned_addr learned my addr 172.21.15.26:0/3986851880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc319390700 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badc5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc318b8f700 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badaf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 -- 172.21.15.26:0/3986851880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 msgr2=0x5576badaf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badaf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 -- 172.21.15.26:0/3986851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 msgr2=0x5576badc5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badc5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 -- 172.21.15.26:0/3986851880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bada46a0 con 0x5576bada4a20 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.731+0000 7fc313fff700 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badc5040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00b780 tx=0x7fc2fc005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f24da647c694235 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc30a7fc700 1 -- 172.21.15.26:0/3986851880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc01f840 con 0x5576bada4a20 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc30a7fc700 1 -- 172.21.15.26:0/3986851880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2fc023070 con 0x5576bada4a20 2024-05-20T20:56:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc30a7fc700 1 -- 172.21.15.26:0/3986851880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc01ce60 con 0x5576bada4a20 2024-05-20T20:56:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 -- 172.21.15.26:0/3986851880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 msgr2=0x5576badc5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badc5040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00b780 tx=0x7fc2fc005860 comp rx=0 tx=0).stop 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 -- 172.21.15.26:0/3986851880 shutdown_connections 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badc5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badaf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 --2- 172.21.15.26:0/3986851880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badc5040 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 -- 172.21.15.26:0/3986851880 >> 172.21.15.26:0/3986851880 conn(0x5576bacc9080 msgr2=0x5576bada7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 -- 172.21.15.26:0/3986851880 shutdown_connections 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.732+0000 7fc325689b80 1 -- 172.21.15.26:0/3986851880 wait complete. 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 -- start start 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 Processor -- start 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badb3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badb37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141aa1c0 con 0x5576bada4a20 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141ab630 con 0x5576bada8770 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141aa890 con 0x5576bada4070 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.733+0000 7fc325689b80 10 client.0 _ll_register_callbacks cb 0x5576bac85920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badb3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badb3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40990/0 (socket says 172.21.15.26:40990) 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 -- 172.21.15.26:0/42123774 learned_addr learned my addr 172.21.15.26:0/42123774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc318b8f700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 -- 172.21.15.26:0/42123774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 msgr2=0x5576badb5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badb5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 -- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 msgr2=0x5576badb37d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T20:56:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badb37d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 -- 172.21.15.26:0/42123774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576badb3fc0 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc318b8f700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc313fff700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badb3290 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0058f0 tx=0x7fc2fc010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b978a45b58fd2063 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc2f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc2f3fff700 1 -- 172.21.15.26:0/42123774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc010e90 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2fc023070 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc2f3fff700 1 -- 172.21.15.26:0/42123774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc2fc027bd0 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc2f3fff700 1 -- 172.21.15.26:0/42123774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc04d050 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.734+0000 7fc325689b80 10 client.15450 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576bae27f50 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc2f3fff700 1 -- 172.21.15.26:0/42123774 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2fc023070 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc2f3fff700 1 -- 172.21.15.26:0/42123774 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2fc0328b0 con 0x5576bada4a20 2024-05-20T20:56:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 10 client.15450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 20 client.15450 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576bae2a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 4 client.15450 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576bacfcbe0 con 0x5576badbd360 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.735+0000 7fc318b8f700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576bae2a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc318b8f700 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576bae2a5e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac37387689ddb398 server_cookie=a5cf1f7deaff4bb1 in_seq=0 out_seq=0 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc2f3fff700 10 client.15450 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc2f3fff700 1 -- 172.21.15.26:0/42123774 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576bacfcbe0 con 0x5576badbd360 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc2f3fff700 10 client.15450 handle_command_reply: tid=0 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 client.15450 shutdown 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 msgr2=0x5576badb3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badb3290 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0058f0 tx=0x7fc2fc010a40 comp rx=0 tx=0).stop 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 shutdown_connections 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5576badbd360 0x5576bae2a5e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576bada8770 0x5576badb37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576bada4070 0x5576badb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 --2- 172.21.15.26:0/42123774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576bada4a20 0x5576badb3290 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 >> 172.21.15.26:0/42123774 conn(0x5576bacc9080 msgr2=0x5576bada70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.736+0000 7fc2f3fff700 0 client.15450 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 shutdown_connections 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc325689b80 1 -- 172.21.15.26:0/42123774 wait complete. 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc325689b80 20 client.15450 trim_cache size 0 max 16384 2024-05-20T20:56:45.741 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:56:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3000b2910 msgr2=0x7fc3000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3000b2910 0x7fc3000b4dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc31406cd50 tx=0x7fc2fc00b410 comp rx=0 tx=0).stop 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 msgr2=0x7fc3140ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 0x7fc3140ff490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc304000c00 tx=0x7fc3040047b0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 shutdown_connections 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3000b2910 0x7fc3000b4dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc314102460 0x7fc3140ff490 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3141084f0 0x7fc3140fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.737+0000 7fc31adf3700 1 --2- 172.21.15.26:0/4064957032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314102e20 0x7fc3140ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.738+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 >> 172.21.15.26:0/4064957032 conn(0x7fc3140794e0 msgr2=0x7fc314101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.738+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 shutdown_connections 2024-05-20T20:56:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:45.738+0000 7fc31adf3700 1 -- 172.21.15.26:0/4064957032 wait complete. 2024-05-20T20:56:46.552 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:56:46.810 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:47 smithi144 ceph-mon[35660]: pgmap v407: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:47.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:47 smithi055 ceph-mon[35213]: pgmap v407: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:47.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:47 smithi026 ceph-mon[31990]: pgmap v407: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-20T20:56:48.271 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-20T20:56:48.271 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-05-20T20:56:48.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:56:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 -- 172.21.15.26:0/2237909371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c109630 msgr2=0x7f455c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 --2- 172.21.15.26:0/2237909371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c109630 0x7f455c109a10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f454c0099c0 tx=0x7f454c009cd0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 -- 172.21.15.26:0/2237909371 shutdown_connections 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 --2- 172.21.15.26:0/2237909371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c109630 0x7f455c109a10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 --2- 172.21.15.26:0/2237909371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c102470 0x7f455c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 --2- 172.21.15.26:0/2237909371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455c101ab0 0x7f455c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.604+0000 7f4562f1f700 1 -- 172.21.15.26:0/2237909371 >> 172.21.15.26:0/2237909371 conn(0x7f455c078a40 msgr2=0x7f455c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 -- 172.21.15.26:0/2237909371 shutdown_connections 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 -- 172.21.15.26:0/2237909371 wait complete. 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 Processor -- start 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 -- start start 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455c101ab0 0x7f455c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 0x7f455c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c109630 0x7f455c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113600 con 0x7f455c102470 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113480 con 0x7f455c101ab0 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.605+0000 7f4562f1f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113780 con 0x7f455c109630 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 0x7f455c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 0x7f455c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41004/0 (socket says 172.21.15.26:41004) 2024-05-20T20:56:48.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 -- 172.21.15.26:0/13058000 learned_addr learned my addr 172.21.15.26:0/13058000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f45614bc700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c109630 0x7f455c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f4560cbb700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455c101ab0 0x7f455c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 -- 172.21.15.26:0/13058000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c109630 msgr2=0x7f455c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c109630 0x7f455c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 -- 172.21.15.26:0/13058000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455c101ab0 msgr2=0x7f455c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455c101ab0 0x7f455c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 -- 172.21.15.26:0/13058000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c19a0a0 con 0x7f455c102470 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f45614bc700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c109630 0x7f455c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f455bfff700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 0x7f455c19fcc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f454400c920 tx=0x7f454400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f4559ffb700 1 -- 172.21.15.26:0/13058000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544007b00 con 0x7f455c102470 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f4559ffb700 1 -- 172.21.15.26:0/13058000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4544004cc0 con 0x7f455c102470 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c19a390 con 0x7f455c102470 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f4559ffb700 1 -- 172.21.15.26:0/13058000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440054f0 con 0x7f455c102470 2024-05-20T20:56:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.606+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a6650 con 0x7f455c102470 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.609+0000 7f4559ffb700 1 -- 172.21.15.26:0/13058000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f4544007ca0 con 0x7f455c102470 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.610+0000 7f4559ffb700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45480b2910 0x7f45480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.610+0000 7f4560cbb700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45480b2910 0x7f45480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.610+0000 7f4559ffb700 1 -- 172.21.15.26:0/13058000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f45440e20a0 con 0x7f455c102470 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.610+0000 7f4560cbb700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45480b2910 0x7f45480b4dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f454c009990 tx=0x7f454c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 Processor -- start 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 -- start start 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37811c6c0 0x55b37811caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378117130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378123460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b378070d70 con 0x55b378117670 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b378060910 con 0x55b378116cb0 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b378064950 con 0x55b37811c6c0 2024-05-20T20:56:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378123460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378123460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41020/0 (socket says 172.21.15.26:41020) 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 -- 172.21.15.26:0/1039939823 learned_addr learned my addr 172.21.15.26:0/1039939823 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f455bfff700 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378117130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f4560cbb700 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37811c6c0 0x55b37811caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 -- 172.21.15.26:0/1039939823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37811c6c0 msgr2=0x55b37811caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37811c6c0 0x55b37811caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 -- 172.21.15.26:0/1039939823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 msgr2=0x55b378117130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378117130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f45614bc700 1 -- 172.21.15.26:0/1039939823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b378009fb0 con 0x55b378117670 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.611+0000 7f4560cbb700 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37811c6c0 0x55b37811caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f45614bc700 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378123460 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4554009db0 tx=0x7f45540077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5635b917e94027f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f4552ffd700 1 -- 172.21.15.26:0/1039939823 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554004bb0 con 0x55b378117670 2024-05-20T20:56:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f4552ffd700 1 -- 172.21.15.26:0/1039939823 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4554004d50 con 0x55b378117670 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f4552ffd700 1 -- 172.21.15.26:0/1039939823 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554005280 con 0x55b378117670 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 -- 172.21.15.26:0/1039939823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 msgr2=0x55b378123460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378123460 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4554009db0 tx=0x7f45540077e0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 -- 172.21.15.26:0/1039939823 shutdown_connections 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378123460 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37811c6c0 0x55b37811caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/1039939823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.612+0000 7f456d7b5b80 1 -- 172.21.15.26:0/1039939823 >> 172.21.15.26:0/1039939823 conn(0x55b37803e240 msgr2=0x55b37810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 -- 172.21.15.26:0/1039939823 shutdown_connections 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 -- 172.21.15.26:0/1039939823 wait complete. 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 -- start start 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 Processor -- start 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 0x55b378123020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378120320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37811c6c0 0x55b378120860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b377f66fa0 con 0x55b378116cb0 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b377fc8da0 con 0x55b37811c6c0 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.613+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b378123b50 con 0x55b378117670 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f456d7b5b80 10 client.0 _ll_register_callbacks cb 0x55b377ffb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 0x55b378123020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 0x55b378123020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41026/0 (socket says 172.21.15.26:41026) 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 -- 172.21.15.26:0/811763451 learned_addr learned my addr 172.21.15.26:0/811763451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f45614bc700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37811c6c0 0x55b378120860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f455bfff700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378120320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 -- 172.21.15.26:0/811763451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 msgr2=0x55b378120320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378120320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 -- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37811c6c0 msgr2=0x55b378120860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37811c6c0 0x55b378120860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 -- 172.21.15.26:0/811763451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b377fb08d0 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f45614bc700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37811c6c0 0x55b378120860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f4560cbb700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 0x55b378123020 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f454c015000 tx=0x7f454c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374de399d7fc548b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f453bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f453bfff700 1 -- 172.21.15.26:0/811763451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c033060 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f453bfff700 1 -- 172.21.15.26:0/811763451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f454c00f460 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4554004d50 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f453bfff700 1 -- 172.21.15.26:0/811763451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f454c00be50 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f456d7b5b80 10 client.15462 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.614+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b37801a760 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f453bfff700 1 -- 172.21.15.26:0/811763451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4554004d50 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f453bfff700 1 -- 172.21.15.26:0/811763451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f454c037030 con 0x55b378116cb0 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f456d7b5b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f456d7b5b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f456d7b5b80 20 client.15462 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b378130570 0x55b3781329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f456d7b5b80 4 client.15462 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b378072cc0 con 0x55b378130570 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.615+0000 7f455bfff700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b378130570 0x55b3781329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f453bfff700 10 client.15462 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f455bfff700 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b378130570 0x55b3781329a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea6a57b42dd8d7c server_cookie=2e8cb72f766ece1 in_seq=0 out_seq=0 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f453bfff700 1 -- 172.21.15.26:0/811763451 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b378072cc0 con 0x55b378130570 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f453bfff700 10 client.15462 handle_command_reply: tid=0 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 client.15462 shutdown 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 msgr2=0x55b378123020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 0x55b378123020 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f454c015000 tx=0x7f454c009310 comp rx=0 tx=0).stop 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 shutdown_connections 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b378130570 0x55b3781329a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378116cb0 0x55b378123020 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37811c6c0 0x55b378120860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/811763451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378120320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 >> 172.21.15.26:0/811763451 conn(0x55b37803e240 msgr2=0x55b37810caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.616+0000 7f453bfff700 0 client.15462 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.617+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 shutdown_connections 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.617+0000 7f456d7b5b80 1 -- 172.21.15.26:0/811763451 wait complete. 2024-05-20T20:56:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.617+0000 7f456d7b5b80 20 client.15462 trim_cache size 0 max 16384 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 Processor -- start 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 -- start start 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37803e240 0x55b37811bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378116cb0 0x55b37810d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378121b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ae3c0 con 0x55b378117670 2024-05-20T20:56:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1abc50 con 0x55b37803e240 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.632+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1a9aa0 con 0x55b378116cb0 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378121b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378121b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41042/0 (socket says 172.21.15.26:41042) 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 -- 172.21.15.26:0/3066169486 learned_addr learned my addr 172.21.15.26:0/3066169486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f4560cbb700 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37803e240 0x55b37811bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f455bfff700 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378116cb0 0x55b37810d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 -- 172.21.15.26:0/3066169486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378116cb0 msgr2=0x55b37810d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378116cb0 0x55b37810d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 -- 172.21.15.26:0/3066169486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37803e240 msgr2=0x55b37811bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37803e240 0x55b37811bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 -- 172.21.15.26:0/3066169486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37810ec90 con 0x55b378117670 2024-05-20T20:56:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f455bfff700 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378116cb0 0x55b37810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45614bc700 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378121b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4554009cd0 tx=0x7f4554004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3797ed0050327ff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45527fc700 1 -- 172.21.15.26:0/3066169486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554004e30 con 0x55b378117670 2024-05-20T20:56:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45527fc700 1 -- 172.21.15.26:0/3066169486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4554015900 con 0x55b378117670 2024-05-20T20:56:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.633+0000 7f45527fc700 1 -- 172.21.15.26:0/3066169486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540052a0 con 0x55b378117670 2024-05-20T20:56:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 -- 172.21.15.26:0/3066169486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 msgr2=0x55b378121b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378121b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4554009cd0 tx=0x7f4554004c40 comp rx=0 tx=0).stop 2024-05-20T20:56:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 -- 172.21.15.26:0/3066169486 shutdown_connections 2024-05-20T20:56:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b378117670 0x55b378121b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b37803e240 0x55b37811bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/3066169486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378116cb0 0x55b37810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 -- 172.21.15.26:0/3066169486 >> 172.21.15.26:0/3066169486 conn(0x55b37811f7c0 msgr2=0x55b378112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 -- 172.21.15.26:0/3066169486 shutdown_connections 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 -- 172.21.15.26:0/3066169486 wait complete. 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 -- start start 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.634+0000 7f456d7b5b80 1 Processor -- start 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 0x55b378129380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378137990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378137ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aa110 con 0x55b37803e240 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab580 con 0x55b378116cb0 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aa7e0 con 0x55b378117670 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f456d7b5b80 10 client.0 _ll_register_callbacks cb 0x55b377ffb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 0x55b378129380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 0x55b378129380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41058/0 (socket says 172.21.15.26:41058) 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 -- 172.21.15.26:0/609722729 learned_addr learned my addr 172.21.15.26:0/609722729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f455bfff700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f45614bc700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378137ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 -- 172.21.15.26:0/609722729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 msgr2=0x55b378137ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378137ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 -- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 msgr2=0x55b378137990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378137990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.635+0000 7f4560cbb700 1 -- 172.21.15.26:0/609722729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3781385d0 con 0x55b37803e240 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f4560cbb700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 0x55b378129380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f454c030010 tx=0x7f454c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9de6dc0ac789ab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f453bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f453bfff700 1 -- 172.21.15.26:0/609722729 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c00f660 con 0x55b37803e240 2024-05-20T20:56:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f453bfff700 1 -- 172.21.15.26:0/609722729 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f454c00f800 con 0x55b37803e240 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4554015900 con 0x55b37803e240 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f453bfff700 1 -- 172.21.15.26:0/609722729 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f454c048050 con 0x55b37803e240 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 10 client.15468 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b37819afa0 con 0x55b37803e240 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f453bfff700 1 -- 172.21.15.26:0/609722729 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4554015900 con 0x55b37803e240 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f453bfff700 1 -- 172.21.15.26:0/609722729 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f454c040020 con 0x55b37803e240 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 20 client.15468 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b37811c6c0 0x55b37819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 4 client.15468 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.636+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b37806c4a0 con 0x55b37811c6c0 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f455bfff700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b37811c6c0 0x55b37819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f455bfff700 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b37811c6c0 0x55b37819d4c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be64f54b32b75ad0 server_cookie=21c9543dca4e783b in_seq=0 out_seq=0 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f453bfff700 10 client.15468 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f453bfff700 1 -- 172.21.15.26:0/609722729 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b37806c4a0 con 0x55b37811c6c0 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f453bfff700 10 client.15468 handle_command_reply: tid=0 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f456d7b5b80 1 client.15468 shutdown 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 msgr2=0x55b378129380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.637+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 0x55b378129380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f454c030010 tx=0x7f454c00bf90 comp rx=0 tx=0).stop 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 shutdown_connections 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b37811c6c0 0x55b37819d4c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b37803e240 0x55b378129380 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b378117670 0x55b378137ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 --2- 172.21.15.26:0/609722729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b378116cb0 0x55b378137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f453bfff700 0 client.15468 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 >> 172.21.15.26:0/609722729 conn(0x55b37811f7c0 msgr2=0x55b378112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 shutdown_connections 2024-05-20T20:56:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 1 -- 172.21.15.26:0/609722729 wait complete. 2024-05-20T20:56:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f456d7b5b80 20 client.15468 trim_cache size 0 max 16384 2024-05-20T20:56:48.643 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:56:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45480b2910 msgr2=0x7f45480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45480b2910 0x7f45480b4dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f454c009990 tx=0x7f454c01a040 comp rx=0 tx=0).stop 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 msgr2=0x7f455c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 0x7f455c19fcc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f454400c920 tx=0x7f454400cce0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 shutdown_connections 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f455c109630 0x7f455c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f455c101ab0 0x7f455c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45480b2910 0x7f45480b4dd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 --2- 172.21.15.26:0/13058000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f455c102470 0x7f455c19fcc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.638+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 >> 172.21.15.26:0/13058000 conn(0x7f455c078a40 msgr2=0x7f455c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.639+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 shutdown_connections 2024-05-20T20:56:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:48.639+0000 7f4562f1f700 1 -- 172.21.15.26:0/13058000 wait complete. 2024-05-20T20:56:49.454 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:49 smithi144 ceph-mon[35660]: pgmap v408: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-20T20:56:49.578 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:49 smithi026 ceph-mon[31990]: pgmap v408: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-20T20:56:49.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:49 smithi055 ceph-mon[35213]: pgmap v408: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-20T20:56:50.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:56:50.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:56:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 -- 172.21.15.26:0/1936513781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8058075430 msgr2=0x7f8058075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 --2- 172.21.15.26:0/1936513781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8058075430 0x7f8058075890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f804c009900 tx=0x7f804c02a480 comp rx=0 tx=0).stop 2024-05-20T20:56:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 -- 172.21.15.26:0/1936513781 shutdown_connections 2024-05-20T20:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 --2- 172.21.15.26:0/1936513781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805810a990 0x7f805810ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 --2- 172.21.15.26:0/1936513781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8058075430 0x7f8058075890 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 --2- 172.21.15.26:0/1936513781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8058076a10 0x7f8058074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 -- 172.21.15.26:0/1936513781 >> 172.21.15.26:0/1936513781 conn(0x7f80580fe070 msgr2=0x7f8058100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 -- 172.21.15.26:0/1936513781 shutdown_connections 2024-05-20T20:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.464+0000 7f805e714640 1 -- 172.21.15.26:0/1936513781 wait complete. 2024-05-20T20:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.465+0000 7f805e714640 1 Processor -- start 2024-05-20T20:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.465+0000 7f805e714640 1 -- start start 2024-05-20T20:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805e714640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8058075430 0x7f805819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805e714640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8058076a10 0x7f805819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805e714640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 0x7f80581a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805e714640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80581106a0 con 0x7f805810a990 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805e714640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8058110520 con 0x7f8058075430 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805e714640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8058110820 con 0x7f8058076a10 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 0x7f80581a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 0x7f80581a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34716/0 (socket says 172.21.15.26:34716) 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 -- 172.21.15.26:0/2348374695 learned_addr learned my addr 172.21.15.26:0/2348374695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f80577fe640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8058076a10 0x7f805819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f8057fff640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8058075430 0x7f805819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 -- 172.21.15.26:0/2348374695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8058076a10 msgr2=0x7f805819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8058076a10 0x7f805819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 -- 172.21.15.26:0/2348374695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8058075430 msgr2=0x7f805819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8058075430 0x7f805819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.466+0000 7f805ccb1640 1 -- 172.21.15.26:0/2348374695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80581a2940 con 0x7f805810a990 2024-05-20T20:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.467+0000 7f805ccb1640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 0x7f80581a22a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f805000eaa0 tx=0x7f805000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.467+0000 7f80557fa640 1 -- 172.21.15.26:0/2348374695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805000cdb0 con 0x7f805810a990 2024-05-20T20:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.467+0000 7f80557fa640 1 -- 172.21.15.26:0/2348374695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8050005020 con 0x7f805810a990 2024-05-20T20:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.467+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80581a2c30 con 0x7f805810a990 2024-05-20T20:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.467+0000 7f80557fa640 1 -- 172.21.15.26:0/2348374695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80500116d0 con 0x7f805810a990 2024-05-20T20:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.467+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80581a3180 con 0x7f805810a990 2024-05-20T20:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.469+0000 7f80557fa640 1 -- 172.21.15.26:0/2348374695 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f80500051d0 con 0x7f805810a990 2024-05-20T20:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 Processor -- start 2024-05-20T20:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 -- start start 2024-05-20T20:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 0x556c42bd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42bdcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bdd220 0x556c42bdf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556c42b48c30 con 0x556c42bd76c0 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556c42b387d0 con 0x556c42bd80b0 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f806142b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c42b3c810 con 0x556c42bdd220 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 0x556c42bd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 0x556c42bd7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34730/0 (socket says 172.21.15.26:34730) 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 -- 172.21.15.26:0/1417927396 learned_addr learned my addr 172.21.15.26:0/1417927396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f8057fff640 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42bdcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bdd220 0x556c42bdf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 -- 172.21.15.26:0/1417927396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bdd220 msgr2=0x556c42bdf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bdd220 0x556c42bdf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 -- 172.21.15.26:0/1417927396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 msgr2=0x556c42bdcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42bdcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.470+0000 7f80577fe640 1 -- 172.21.15.26:0/1417927396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4284d210 con 0x556c42bd76c0 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f80577fe640 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 0x556c42bd7ae0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f804c02a9a0 tx=0x7f804c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4a959a6c1fbc12 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f80557fa640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80400b29a0 0x7f80400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f8057fff640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80400b29a0 0x7f80400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f80557fa640 1 -- 172.21.15.26:0/2348374695 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f80500e2600 con 0x7f805810a990 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f803e7fc640 1 -- 172.21.15.26:0/1417927396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c035070 con 0x556c42bd76c0 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f803e7fc640 1 -- 172.21.15.26:0/1417927396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f804c0044c0 con 0x556c42bd76c0 2024-05-20T20:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.471+0000 7f803e7fc640 1 -- 172.21.15.26:0/1417927396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c003de0 con 0x556c42bd76c0 2024-05-20T20:56:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 -- 172.21.15.26:0/1417927396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 msgr2=0x556c42bd7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 0x556c42bd7ae0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f804c02a9a0 tx=0x7f804c005c00 comp rx=0 tx=0).stop 2024-05-20T20:56:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f8057fff640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80400b29a0 0x7f80400b4e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8048005fd0 tx=0x7f804800b650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 -- 172.21.15.26:0/1417927396 shutdown_connections 2024-05-20T20:56:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bdd220 0x556c42bdf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42bdcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 --2- 172.21.15.26:0/1417927396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd76c0 0x556c42bd7ae0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 -- 172.21.15.26:0/1417927396 >> 172.21.15.26:0/1417927396 conn(0x556c42bd2d30 msgr2=0x556c42bd3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 -- 172.21.15.26:0/1417927396 shutdown_connections 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 -- 172.21.15.26:0/1417927396 wait complete. 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.472+0000 7f806142b740 1 -- start start 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 Processor -- start 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 0x556c42be1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bdd220 0x556c42be8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556c4281e790 con 0x556c42bd80b0 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556c42af4890 con 0x556c42bdd220 2024-05-20T20:56:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c42be0170 con 0x556c42bd76c0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f806142b740 10 client.0 _ll_register_callbacks cb 0x556c4284ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 0x556c42be1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 0x556c42be1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34746/0 (socket says 172.21.15.26:34746) 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 -- 172.21.15.26:0/3418978489 learned_addr learned my addr 172.21.15.26:0/3418978489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f805ccb1640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bdd220 0x556c42be8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f8057fff640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 -- 172.21.15.26:0/3418978489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 msgr2=0x556c42be1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 -- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bdd220 msgr2=0x556c42be8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bdd220 0x556c42be8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.473+0000 7f80577fe640 1 -- 172.21.15.26:0/3418978489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c42919620 con 0x556c42bd80b0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f80577fe640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 0x556c42be1dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f804c005ab0 tx=0x7f804c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0a3a3342bf96f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f80377fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f80377fe640 1 -- 172.21.15.26:0/3418978489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c035030 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f80377fe640 1 -- 172.21.15.26:0/3418978489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f804c003b60 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c42b4ab80 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f80377fe640 1 -- 172.21.15.26:0/3418978489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f804c05a050 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f806142b740 10 client.15480 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c42bebde0 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.474+0000 7f80377fe640 1 -- 172.21.15.26:0/3418978489 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556c42b4ab80 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f80377fe640 1 -- 172.21.15.26:0/3418978489 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f804c03b790 con 0x556c42bd80b0 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f806142b740 10 client.15480 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f806142b740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f806142b740 20 client.15480 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f806142b740 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42bed130 0x556c42bef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f806142b740 4 client.15480 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c42b44360 con 0x556c42bed130 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.475+0000 7f8057fff640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42bed130 0x556c42bef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.476+0000 7f80377fe640 10 client.15480 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.476+0000 7f8057fff640 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42bed130 0x556c42bef520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb46b755d2a683 server_cookie=2a77619e5d77022d in_seq=0 out_seq=0 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.476+0000 7f80377fe640 1 -- 172.21.15.26:0/3418978489 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556c42b44360 con 0x556c42bed130 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.476+0000 7f80377fe640 10 client.15480 handle_command_reply: tid=0 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.476+0000 7f806142b740 1 client.15480 shutdown 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 msgr2=0x556c42be1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 0x556c42be1dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f804c005ab0 tx=0x7f804c003820 comp rx=0 tx=0).stop 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 shutdown_connections 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42bed130 0x556c42bef520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bdd220 0x556c42be8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bd80b0 0x556c42be1dc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f80377fe640 0 client.15480 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 --2- 172.21.15.26:0/3418978489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 >> 172.21.15.26:0/3418978489 conn(0x556c42bd2d30 msgr2=0x556c42bd1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 shutdown_connections 2024-05-20T20:56:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 1 -- 172.21.15.26:0/3418978489 wait complete. 2024-05-20T20:56:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.477+0000 7f806142b740 20 client.15480 trim_cache size 0 max 16384 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.493+0000 7f806142b740 1 Processor -- start 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.493+0000 7f806142b740 1 -- start start 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.493+0000 7f806142b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42bf4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f806142b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f806142b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42bf52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f806142b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80581ae8c0 con 0x556c42bdd220 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f806142b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80581ac190 con 0x556c42bd80b0 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f806142b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80581a9fe0 con 0x556c42bd76c0 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42bf52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42bf52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34754/0 (socket says 172.21.15.26:34754) 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 -- 172.21.15.26:0/3203867160 learned_addr learned my addr 172.21.15.26:0/3203867160 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f8057fff640 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42bf4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f80577fe640 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 -- 172.21.15.26:0/3203867160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 msgr2=0x556c42bf4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42bf4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 -- 172.21.15.26:0/3203867160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 msgr2=0x556c42be8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 -- 172.21.15.26:0/3203867160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c42bd7aa0 con 0x556c42bdd220 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f805ccb1640 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42bf52c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f80500dff80 tx=0x7f80500b22b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77df91b08e2f31b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.494+0000 7f803dffb640 1 -- 172.21.15.26:0/3203867160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805001f040 con 0x556c42bdd220 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f803dffb640 1 -- 172.21.15.26:0/3203867160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80500aa160 con 0x556c42bdd220 2024-05-20T20:56:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f803dffb640 1 -- 172.21.15.26:0/3203867160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80500a79b0 con 0x556c42bdd220 2024-05-20T20:56:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 -- 172.21.15.26:0/3203867160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 msgr2=0x556c42bf52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42bf52c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f80500dff80 tx=0x7f80500b22b0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 -- 172.21.15.26:0/3203867160 shutdown_connections 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42bf52c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 --2- 172.21.15.26:0/3203867160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42bf4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 -- 172.21.15.26:0/3203867160 >> 172.21.15.26:0/3203867160 conn(0x556c42bd2d30 msgr2=0x556c42bd93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 -- 172.21.15.26:0/3203867160 shutdown_connections 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.495+0000 7f806142b740 1 -- 172.21.15.26:0/3203867160 wait complete. 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 -- start start 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 Processor -- start 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42be7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80581aa650 con 0x556c42bdd220 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80581abac0 con 0x556c42bd80b0 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80581aad20 con 0x556c42bd76c0 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f806142b740 10 client.0 _ll_register_callbacks cb 0x7f804c03e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f805ccb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42be7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f805ccb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42be7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34760/0 (socket says 172.21.15.26:34760) 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f805ccb1640 1 -- 172.21.15.26:0/1883909652 learned_addr learned my addr 172.21.15.26:0/1883909652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f80577fe640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.496+0000 7f8057fff640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f805ccb1640 1 -- 172.21.15.26:0/1883909652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 msgr2=0x556c42be6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f805ccb1640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f805ccb1640 1 -- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 msgr2=0x556c42be6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f805ccb1640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f805ccb1640 1 -- 172.21.15.26:0/1883909652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c42be8050 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f805ccb1640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42be7a60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8050009270 tx=0x7f80500a59c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12d95116a777c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f80377fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f80377fe640 1 -- 172.21.15.26:0/1883909652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80500c4050 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f80377fe640 1 -- 172.21.15.26:0/1883909652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f80500a5fc0 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80500aa160 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f80377fe640 1 -- 172.21.15.26:0/1883909652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f805001f040 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f806142b740 10 client.15486 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c42c57cb0 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.497+0000 7f80377fe640 1 -- 172.21.15.26:0/1883909652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80500aa160 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f80377fe640 1 -- 172.21.15.26:0/1883909652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f805008a070 con 0x556c42bdd220 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f806142b740 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f806142b740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f806142b740 20 client.15486 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f806142b740 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42c585d0 0x556c42c5a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f806142b740 4 client.15486 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:56:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c42abe5f0 con 0x556c42c585d0 2024-05-20T20:56:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f8057fff640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42c585d0 0x556c42c5a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f8057fff640 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42c585d0 0x556c42c5a9f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4138c77ce87fa8fc server_cookie=ade3e36be142c0c7 in_seq=0 out_seq=0 2024-05-20T20:56:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.498+0000 7f80377fe640 10 client.15486 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.502+0000 7f80377fe640 1 -- 172.21.15.26:0/1883909652 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556c42abe5f0 con 0x556c42c585d0 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.502+0000 7f80377fe640 10 client.15486 handle_command_reply: tid=0 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.502+0000 7f806142b740 1 client.15486 shutdown 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.502+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 msgr2=0x556c42be7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.502+0000 7f806142b740 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42be7a60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8050009270 tx=0x7f80500a59c0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 shutdown_connections 2024-05-20T20:56:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c42c585d0 0x556c42c5a9f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c42bdd220 0x556c42be7a60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c42bd80b0 0x556c42be6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f80377fe640 0 client.15486 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 --2- 172.21.15.26:0/1883909652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c42bd76c0 0x556c42be6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 >> 172.21.15.26:0/1883909652 conn(0x556c42bd2d30 msgr2=0x556c42bd5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 shutdown_connections 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 1 -- 172.21.15.26:0/1883909652 wait complete. 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f806142b740 20 client.15486 trim_cache size 0 max 16384 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:56:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:56:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80400b29a0 msgr2=0x7f80400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80400b29a0 0x7f80400b4e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8048005fd0 tx=0x7f804800b650 comp rx=0 tx=0).stop 2024-05-20T20:56:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 msgr2=0x7f80581a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 0x7f80581a22a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f805000eaa0 tx=0x7f805000ef60 comp rx=0 tx=0).stop 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 shutdown_connections 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80400b29a0 0x7f80400b4e60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f805810a990 0x7f80581a22a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8058076a10 0x7f805819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 --2- 172.21.15.26:0/2348374695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8058075430 0x7f805819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.503+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 >> 172.21.15.26:0/2348374695 conn(0x7f80580fe070 msgr2=0x7f8058106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.504+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 shutdown_connections 2024-05-20T20:56:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:56:50.504+0000 7f805e714640 1 -- 172.21.15.26:0/2348374695 wait complete. 2024-05-20T20:56:50.772 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 -- 172.21.15.26:0/616889199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e81095b0 msgr2=0x7fc3e8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 --2- 172.21.15.26:0/616889199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e81095b0 0x7fc3e8109990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009990 tx=0x7fc3d8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 -- 172.21.15.26:0/616889199 shutdown_connections 2024-05-20T20:56:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 --2- 172.21.15.26:0/616889199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e81095b0 0x7fc3e8109990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 --2- 172.21.15.26:0/616889199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e806bbe0 0x7fc3e81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 --2- 172.21.15.26:0/616889199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3e806b220 0x7fc3e806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 -- 172.21.15.26:0/616889199 >> 172.21.15.26:0/616889199 conn(0x7fc3e80793f0 msgr2=0x7fc3e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 -- 172.21.15.26:0/616889199 shutdown_connections 2024-05-20T20:56:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 -- 172.21.15.26:0/616889199 wait complete. 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.417+0000 7fc3ed270700 1 Processor -- start 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 -- start start 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 0x7fc3e819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e806bbe0 0x7fc3e819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3e81095b0 0x7fc3e81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81135e0 con 0x7fc3e806b220 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8113460 con 0x7fc3e81095b0 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3ed270700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8113760 con 0x7fc3e806bbe0 2024-05-20T20:56:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 0x7fc3e819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 0x7fc3e819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34772/0 (socket says 172.21.15.26:34772) 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 -- 172.21.15.26:0/3130645324 learned_addr learned my addr 172.21.15.26:0/3130645324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e67fc700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e806bbe0 0x7fc3e819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e77fe700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3e81095b0 0x7fc3e81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 -- 172.21.15.26:0/3130645324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e806bbe0 msgr2=0x7fc3e819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e806bbe0 0x7fc3e819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 -- 172.21.15.26:0/3130645324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3e81095b0 msgr2=0x7fc3e81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3e81095b0 0x7fc3e81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.418+0000 7fc3e6ffd700 1 -- 172.21.15.26:0/3130645324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819a120 con 0x7fc3e806b220 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.419+0000 7fc3e6ffd700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 0x7fc3e819f800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005910 tx=0x7fc3d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.419+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8027070 con 0x7fc3e806b220 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.419+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819a3b0 con 0x7fc3e806b220 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.419+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d800fb00 con 0x7fc3e806b220 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.419+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8022b90 con 0x7fc3e806b220 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.419+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e8102440 con 0x7fc3e806b220 2024-05-20T20:56:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.420+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e810ba90 con 0x7fc3e806b220 2024-05-20T20:56:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.422+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fc3d802a070 con 0x7fc3e806b220 2024-05-20T20:56:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.423+0000 7fc3cffff700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3d00b2910 0x7fc3d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.423+0000 7fc3e67fc700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3d00b2910 0x7fc3d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.423+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc3d80e3a30 con 0x7fc3e806b220 2024-05-20T20:56:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.423+0000 7fc3e67fc700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3d00b2910 0x7fc3d00b4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8079b10 tx=0x7fc3dc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:51.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.426+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3d8031070 con 0x7fc3e806b220 2024-05-20T20:56:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:51 smithi144 ceph-mon[35660]: pgmap v409: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:56:51.595 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:51 smithi026 ceph-mon[31990]: pgmap v409: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.697+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e819aef0 con 0x7fc3e806b220 2024-05-20T20:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.697+0000 7fc3cffff700 1 -- 172.21.15.26:0/3130645324 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fc3d8013070 con 0x7fc3e806b220 2024-05-20T20:56:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:56:51.698 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.699+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3d00b2910 msgr2=0x7fc3d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.699+0000 7fc3ed270700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3d00b2910 0x7fc3d00b4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8079b10 tx=0x7fc3dc00a380 comp rx=0 tx=0).stop 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.699+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 msgr2=0x7fc3e819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.699+0000 7fc3ed270700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 0x7fc3e819f800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005910 tx=0x7fc3d80047b0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 shutdown_connections 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3d00b2910 0x7fc3d00b4dd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e806b220 0x7fc3e819f800 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3e81095b0 0x7fc3e81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 --2- 172.21.15.26:0/3130645324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e806bbe0 0x7fc3e819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 >> 172.21.15.26:0/3130645324 conn(0x7fc3e80793f0 msgr2=0x7fc3e8101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 shutdown_connections 2024-05-20T20:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:51.700+0000 7fc3ed270700 1 -- 172.21.15.26:0/3130645324 wait complete. 2024-05-20T20:56:51.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:51 smithi055 ceph-mon[35213]: pgmap v409: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:56:52.503 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-05-20T20:56:52.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-05-20T20:56:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:52 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3130645324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:52 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:56:52.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3130645324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:52.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:52 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:56:52.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:52 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3130645324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:56:52.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:52 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:56:53.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: pgmap v410: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T20:56:53.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:56:53.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:56:53.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/2203876866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e841095b0 msgr2=0x7f7e84109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/2203876866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e841095b0 0x7f7e84109990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009990 tx=0x7f7e80009ca0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/2203876866 shutdown_connections 2024-05-20T20:56:53.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/2203876866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e841095b0 0x7f7e84109990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/2203876866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8406bbe0 0x7f7e841103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/2203876866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e8406b220 0x7f7e8406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/2203876866 >> 172.21.15.26:0/2203876866 conn(0x7f7e840793f0 msgr2=0x7f7e84079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:53.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/2203876866 shutdown_connections 2024-05-20T20:56:53.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.539+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/2203876866 wait complete. 2024-05-20T20:56:53.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.540+0000 7f7e8c2c4700 1 Processor -- start 2024-05-20T20:56:53.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.540+0000 7f7e8c2c4700 1 -- start start 2024-05-20T20:56:53.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8c2c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8406b220 0x7f7e8419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8c2c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 0x7f7e8419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8c2c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e841095b0 0x7f7e84199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8c2c4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841135c0 con 0x7f7e8406bbe0 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8c2c4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84113440 con 0x7f7e841095b0 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8c2c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84113740 con 0x7f7e8406b220 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 0x7f7e8419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 0x7f7e8419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34794/0 (socket says 172.21.15.26:34794) 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 -- 172.21.15.26:0/263867365 learned_addr learned my addr 172.21.15.26:0/263867365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8a060700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8406b220 0x7f7e8419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8a861700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e841095b0 0x7f7e84199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 -- 172.21.15.26:0/263867365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8406b220 msgr2=0x7f7e8419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8406b220 0x7f7e8419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 -- 172.21.15.26:0/263867365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e841095b0 msgr2=0x7f7e84199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e841095b0 0x7f7e84199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 -- 172.21.15.26:0/263867365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8419a0e0 con 0x7f7e8406bbe0 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8a861700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e841095b0 0x7f7e84199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.541+0000 7f7e8985f700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 0x7f7e8419fd00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400eb30 tx=0x7f7e7400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.542+0000 7f7e7b7fe700 1 -- 172.21.15.26:0/263867365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400cc90 con 0x7f7e8406bbe0 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.542+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8419a3d0 con 0x7f7e8406bbe0 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.542+0000 7f7e7b7fe700 1 -- 172.21.15.26:0/263867365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e7400ce30 con 0x7f7e8406bbe0 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.542+0000 7f7e7b7fe700 1 -- 172.21.15.26:0/263867365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74003e30 con 0x7f7e8406bbe0 2024-05-20T20:56:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.542+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a6690 con 0x7f7e8406bbe0 2024-05-20T20:56:53.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.544+0000 7f7e7b7fe700 1 -- 172.21.15.26:0/263867365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f7e74018430 con 0x7f7e8406bbe0 2024-05-20T20:56:53.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.545+0000 7f7e7b7fe700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e700b2910 0x7f7e700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.545+0000 7f7e7b7fe700 1 -- 172.21.15.26:0/263867365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7e740e26c0 con 0x7f7e8406bbe0 2024-05-20T20:56:53.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.545+0000 7f7e8a060700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e700b2910 0x7f7e700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.545+0000 7f7e96b5ab80 1 Processor -- start 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 -- start start 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8a060700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e700b2910 0x7f7e700b4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000b2e0 tx=0x7f7e8001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ed9540fd10 con 0x55ed954b1290 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ed953ff8b0 con 0x55ed954b08e0 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed954038f0 con 0x55ed954b4110 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8985f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8a060700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8a060700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39082/0 (socket says 172.21.15.26:39082) 2024-05-20T20:56:53.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8a060700 1 -- 172.21.15.26:0/3521800667 learned_addr learned my addr 172.21.15.26:0/3521800667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8a861700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.546+0000 7f7e8985f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34810/0 (socket says 172.21.15.26:34810) 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e8985f700 1 -- 172.21.15.26:0/3521800667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 msgr2=0x55ed954b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e8985f700 1 -- 172.21.15.26:0/3521800667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 msgr2=0x55ed954b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e8985f700 1 -- 172.21.15.26:0/3521800667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9534f950 con 0x55ed954b1290 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954b3af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e740e0010 tx=0x7f7e740a9a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756de92b301869c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e78d70700 1 -- 172.21.15.26:0/3521800667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74009500 con 0x55ed954b1290 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e78d70700 1 -- 172.21.15.26:0/3521800667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e740aae50 con 0x55ed954b1290 2024-05-20T20:56:53.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.547+0000 7f7e78d70700 1 -- 172.21.15.26:0/3521800667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e740a9fe0 con 0x55ed954b1290 2024-05-20T20:56:53.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3521800667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 msgr2=0x55ed954b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954b3af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e740e0010 tx=0x7f7e740a9a20 comp rx=0 tx=0).stop 2024-05-20T20:56:53.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3521800667 shutdown_connections 2024-05-20T20:56:53.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3521800667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954b3af0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3521800667 >> 172.21.15.26:0/3521800667 conn(0x55ed953de100 msgr2=0x55ed954aaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3521800667 shutdown_connections 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.548+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3521800667 wait complete. 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.549+0000 7f7e96b5ab80 1 -- start start 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.549+0000 7f7e96b5ab80 1 Processor -- start 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b08e0 0x55ed954c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954c2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b4110 0x55ed954c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ed95306020 con 0x55ed954b1290 2024-05-20T20:56:53.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e8985f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e8985f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954c2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34822/0 (socket says 172.21.15.26:34822) 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e8a060700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b08e0 0x55ed954c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e8985f700 1 -- 172.21.15.26:0/2542636277 learned_addr learned my addr 172.21.15.26:0/2542636277 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ed953a1840 con 0x55ed954b4110 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e8a861700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b4110 0x55ed954c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed954c5ae0 con 0x55ed954b08e0 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.550+0000 7f7e96b5ab80 10 client.0 _ll_register_callbacks cb 0x55ed9539a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8985f700 1 -- 172.21.15.26:0/2542636277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b08e0 msgr2=0x55ed954c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8985f700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b08e0 0x55ed954c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8985f700 1 -- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b4110 msgr2=0x55ed954c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8985f700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b4110 0x55ed954c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8985f700 1 -- 172.21.15.26:0/2542636277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9539e710 con 0x55ed954b1290 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8a060700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b08e0 0x55ed954c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e8985f700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954c2030 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000c00 tx=0x7f7e740b2850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f39acceaa1a52ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e6dffb700 1 -- 172.21.15.26:0/2542636277 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74009500 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e6dffb700 1 -- 172.21.15.26:0/2542636277 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7e740a8f90 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e740aae50 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e6dffb700 1 -- 172.21.15.26:0/2542636277 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e740b22d0 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e96b5ab80 10 client.15504 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.551+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed954d0230 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e6dffb700 1 -- 172.21.15.26:0/2542636277 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e740aae50 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e6dffb700 1 -- 172.21.15.26:0/2542636277 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7e740cc0d0 con 0x55ed954b1290 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e96b5ab80 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e96b5ab80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e96b5ab80 20 client.15504 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:53.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed954d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e96b5ab80 4 client.15504 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:53.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.552+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed9547ca60 con 0x55ed954d2700 2024-05-20T20:56:53.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.553+0000 7f7e8a060700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed954d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: pgmap v410: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T20:56:53.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:56:53.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:56:53.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.553+0000 7f7e8a060700 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed954d4af0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a5a8ec115660c2 server_cookie=1b6f7b247a3da3d8 in_seq=0 out_seq=0 2024-05-20T20:56:53.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.553+0000 7f7e6dffb700 10 client.15504 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e6dffb700 1 -- 172.21.15.26:0/2542636277 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed9547ca60 con 0x55ed954d2700 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e6dffb700 10 client.15504 handle_command_reply: tid=0 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 client.15504 shutdown 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 msgr2=0x55ed954c2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954c2030 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000c00 tx=0x7f7e740b2850 comp rx=0 tx=0).stop 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 shutdown_connections 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed954d4af0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e6dffb700 0 client.15504 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b4110 0x55ed954c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b08e0 0x55ed954c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/2542636277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b1290 0x55ed954c2030 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.554+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 >> 172.21.15.26:0/2542636277 conn(0x55ed953de100 msgr2=0x55ed954aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:53.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.555+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 shutdown_connections 2024-05-20T20:56:53.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.555+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/2542636277 wait complete. 2024-05-20T20:56:53.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.555+0000 7f7e96b5ab80 20 client.15504 trim_cache size 0 max 16384 2024-05-20T20:56:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.582+0000 7f7e96b5ab80 1 Processor -- start 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 -- start start 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b1290 0x55ed954aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 0x55ed954bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841ae5b0 con 0x55ed954b4110 2024-05-20T20:56:53.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841abe40 con 0x55ed954b08e0 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841a9cb0 con 0x55ed954b1290 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8a861700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 0x55ed954bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8a861700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 0x55ed954bf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34832/0 (socket says 172.21.15.26:34832) 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8a861700 1 -- 172.21.15.26:0/38928505 learned_addr learned my addr 172.21.15.26:0/38928505 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8985f700 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b1290 0x55ed954aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8a060700 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8985f700 1 -- 172.21.15.26:0/38928505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 msgr2=0x55ed954c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8985f700 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8985f700 1 -- 172.21.15.26:0/38928505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 msgr2=0x55ed954bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8985f700 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 0x55ed954bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8985f700 1 -- 172.21.15.26:0/38928505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed954c05a0 con 0x55ed954b1290 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.583+0000 7f7e8a861700 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 0x55ed954bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.584+0000 7f7e8985f700 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b1290 0x55ed954aad10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e740cc490 tx=0x7f7e740a9a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709e5a8b161d08f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.584+0000 7f7e6ffff700 1 -- 172.21.15.26:0/38928505 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e740a59e0 con 0x55ed954b1290 2024-05-20T20:56:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.584+0000 7f7e6ffff700 1 -- 172.21.15.26:0/38928505 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e740aae50 con 0x55ed954b1290 2024-05-20T20:56:53.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.584+0000 7f7e6ffff700 1 -- 172.21.15.26:0/38928505 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e740aa220 con 0x55ed954b1290 2024-05-20T20:56:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/38928505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b1290 msgr2=0x55ed954aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b1290 0x55ed954aad10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e740cc490 tx=0x7f7e740a9a20 comp rx=0 tx=0).stop 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/38928505 shutdown_connections 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b4110 0x55ed954bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b08e0 0x55ed954c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/38928505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b1290 0x55ed954aad10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/38928505 >> 172.21.15.26:0/38928505 conn(0x55ed953de100 msgr2=0x55ed954c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/38928505 shutdown_connections 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/38928505 wait complete. 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 -- start start 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.585+0000 7f7e96b5ab80 1 Processor -- start 2024-05-20T20:56:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 0x55ed954d7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b1290 0x55ed954d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841aa300 con 0x55ed954b08e0 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841ab770 con 0x55ed954b1290 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841aa9d0 con 0x55ed954b4110 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e96b5ab80 10 client.0 _ll_register_callbacks cb 0x55ed9539a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 0x55ed954d7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 0x55ed954d7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34834/0 (socket says 172.21.15.26:34834) 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 -- 172.21.15.26:0/3881556607 learned_addr learned my addr 172.21.15.26:0/3881556607 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b1290 0x55ed954d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a861700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 -- 172.21.15.26:0/3881556607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 msgr2=0x55ed954d61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954d61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 -- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b1290 msgr2=0x55ed954d8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b1290 0x55ed954d8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 -- 172.21.15.26:0/3881556607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed954d6a00 con 0x55ed954b08e0 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b1290 0x55ed954d8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a861700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e8a060700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 0x55ed954d7be0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009720 tx=0x7f7e80009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f41964038d8e245 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.586+0000 7f7e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e6dffb700 1 -- 172.21.15.26:0/3881556607 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003cf0 con 0x55ed954b08e0 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e6dffb700 1 -- 172.21.15.26:0/3881556607 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7e800343f0 con 0x55ed954b08e0 2024-05-20T20:56:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e740aae50 con 0x55ed954b08e0 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e6dffb700 1 -- 172.21.15.26:0/3881556607 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e80043050 con 0x55ed954b08e0 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e96b5ab80 10 client.15510 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed9553dc10 con 0x55ed954b08e0 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.587+0000 7f7e6dffb700 1 -- 172.21.15.26:0/3881556607 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e740aae50 con 0x55ed954b08e0 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e6dffb700 1 -- 172.21.15.26:0/3881556607 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7e80021070 con 0x55ed954b08e0 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e96b5ab80 10 client.15510 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e96b5ab80 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e96b5ab80 20 client.15510 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed95540340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e96b5ab80 4 client.15510 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:56:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed95411c60 con 0x55ed954d2700 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed95540340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e8985f700 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed95540340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3a808364afd070 server_cookie=a46a392261f39faa in_seq=0 out_seq=0 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.588+0000 7f7e6dffb700 10 client.15510 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.589+0000 7f7e6dffb700 1 -- 172.21.15.26:0/3881556607 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed95411c60 con 0x55ed954d2700 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.589+0000 7f7e6dffb700 10 client.15510 handle_command_reply: tid=0 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.589+0000 7f7e96b5ab80 1 client.15510 shutdown 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 msgr2=0x55ed954d7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 0x55ed954d7be0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009720 tx=0x7f7e80009310 comp rx=0 tx=0).stop 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 shutdown_connections 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ed954d2700 0x55ed95540340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed954b08e0 0x55ed954d7be0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed954b4110 0x55ed954d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 --2- 172.21.15.26:0/3881556607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed954b1290 0x55ed954d8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e6dffb700 0 client.15510 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 >> 172.21.15.26:0/3881556607 conn(0x55ed953de100 msgr2=0x55ed954aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 shutdown_connections 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 1 -- 172.21.15.26:0/3881556607 wait complete. 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.590+0000 7f7e96b5ab80 20 client.15510 trim_cache size 0 max 16384 2024-05-20T20:56:53.597 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:56:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "3d50dbd8-58cb-4541-8b1d-536c908fa37b", 2024-05-20T20:56:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:56:53.598 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e700b2910 msgr2=0x7f7e700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e700b2910 0x7f7e700b4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000b2e0 tx=0x7f7e8001a040 comp rx=0 tx=0).stop 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 msgr2=0x7f7e8419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 0x7f7e8419fd00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400eb30 tx=0x7f7e7400ee40 comp rx=0 tx=0).stop 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 shutdown_connections 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e841095b0 0x7f7e84199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8406b220 0x7f7e8419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e700b2910 0x7f7e700b4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 --2- 172.21.15.26:0/263867365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e8406bbe0 0x7f7e8419fd00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 >> 172.21.15.26:0/263867365 conn(0x7f7e840793f0 msgr2=0x7f7e84101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 shutdown_connections 2024-05-20T20:56:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:53.591+0000 7f7e8c2c4700 1 -- 172.21.15.26:0/263867365 wait complete. 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: pgmap v410: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:56:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:56:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:54 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:56:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:54 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:54 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:56:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:54 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:54.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:54 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:56:54.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:54 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:54.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:54 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:56:54.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:54 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:54.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:54 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:56:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:54 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:54 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:56:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:54 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:56:55.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:56:55.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:55 smithi026 ceph-mon[31990]: pgmap v411: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-20T20:56:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:55 smithi144 ceph-mon[35660]: pgmap v411: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-20T20:56:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:56:55.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:55 smithi055 ceph-mon[35213]: pgmap v411: 97 pgs: 97 active+clean; 668 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 -- 172.21.15.26:0/1488801469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102e20 msgr2=0x7fb63c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 --2- 172.21.15.26:0/1488801469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102e20 0x7fb63c10f310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb63400b0a0 tx=0x7fb63400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 -- 172.21.15.26:0/1488801469 shutdown_connections 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 --2- 172.21.15.26:0/1488801469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102e20 0x7fb63c10f310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 --2- 172.21.15.26:0/1488801469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 0x7fb63c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 --2- 172.21.15.26:0/1488801469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102460 0x7fb63c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 -- 172.21.15.26:0/1488801469 >> 172.21.15.26:0/1488801469 conn(0x7fb63c079460 msgr2=0x7fb63c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 -- 172.21.15.26:0/1488801469 shutdown_connections 2024-05-20T20:56:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 -- 172.21.15.26:0/1488801469 wait complete. 2024-05-20T20:56:56.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.137+0000 7fb641442700 1 Processor -- start 2024-05-20T20:56:56.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 -- start start 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 0x7fb63c111be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102e20 0x7fb63c112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 0x7fb63c112660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c06c670 con 0x7fb63c102460 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c06c4f0 con 0x7fb63c1084d0 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb641442700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c06c7f0 con 0x7fb63c102e20 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 0x7fb63c111be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 0x7fb63c111be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34854/0 (socket says 172.21.15.26:34854) 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 -- 172.21.15.26:0/337097241 learned_addr learned my addr 172.21.15.26:0/337097241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 0x7fb63c112660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102e20 0x7fb63c112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 -- 172.21.15.26:0/337097241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102e20 msgr2=0x7fb63c112120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102e20 0x7fb63c112120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 -- 172.21.15.26:0/337097241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 msgr2=0x7fb63c112660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 0x7fb63c112660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63affd700 1 -- 172.21.15.26:0/337097241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c1a7ff0 con 0x7fb63c102460 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102e20 0x7fb63c112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.138+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 0x7fb63c112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.139+0000 7fb63affd700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 0x7fb63c111be0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d980 tx=0x7fb62c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.139+0000 7fb623fff700 1 -- 172.21.15.26:0/337097241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c016070 con 0x7fb63c102460 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.139+0000 7fb623fff700 1 -- 172.21.15.26:0/337097241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb62c004830 con 0x7fb63c102460 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.139+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c1a8280 con 0x7fb63c102460 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.139+0000 7fb623fff700 1 -- 172.21.15.26:0/337097241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c005440 con 0x7fb63c102460 2024-05-20T20:56:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.139+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c1a87c0 con 0x7fb63c102460 2024-05-20T20:56:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.141+0000 7fb623fff700 1 -- 172.21.15.26:0/337097241 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fb62c0055e0 con 0x7fb63c102460 2024-05-20T20:56:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.142+0000 7fb623fff700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6240b29b0 0x7fb6240b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.142+0000 7fb623fff700 1 -- 172.21.15.26:0/337097241 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb62c0e2250 con 0x7fb63c102460 2024-05-20T20:56:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.142+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6240b29b0 0x7fb6240b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 Processor -- start 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 -- start start 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6240b29b0 0x7fb6240b4e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb63c079960 tx=0x7fb630008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92dd980 0x5570d92dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92de340 0x5570d92ea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5570d923ce30 con 0x5570d92e3350 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5570d922c9d0 con 0x5570d92dd980 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570d9230a10 con 0x5570d92de340 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb63affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb63affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34860/0 (socket says 172.21.15.26:34860) 2024-05-20T20:56:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb63affd700 1 -- 172.21.15.26:0/1754689381 learned_addr learned my addr 172.21.15.26:0/1754689381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92dd980 0x5570d92dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.143+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92de340 0x5570d92ea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb63affd700 1 -- 172.21.15.26:0/1754689381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92de340 msgr2=0x5570d92ea180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb63affd700 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92de340 0x5570d92ea180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb63affd700 1 -- 172.21.15.26:0/1754689381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92dd980 msgr2=0x5570d92dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb63affd700 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92dd980 0x5570d92dde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb63affd700 1 -- 172.21.15.26:0/1754689381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d917ca70 con 0x5570d92e3350 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb63affd700 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e3730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb62c004ff0 tx=0x7fb62c0a9e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c18a7481cd906 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb6217fa700 1 -- 172.21.15.26:0/1754689381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0ada00 con 0x5570d92e3350 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb6217fa700 1 -- 172.21.15.26:0/1754689381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb62c0b24b0 con 0x5570d92e3350 2024-05-20T20:56:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.144+0000 7fb6217fa700 1 -- 172.21.15.26:0/1754689381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0a98b0 con 0x5570d92e3350 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1754689381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 msgr2=0x5570d92e3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e3730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb62c004ff0 tx=0x7fb62c0a9e20 comp rx=0 tx=0).stop 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1754689381 shutdown_connections 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e3730 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92de340 0x5570d92ea180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1754689381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92dd980 0x5570d92dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1754689381 >> 172.21.15.26:0/1754689381 conn(0x5570d920b220 msgr2=0x5570d92d8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1754689381 shutdown_connections 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1754689381 wait complete. 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 -- start start 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.145+0000 7fb64bcd8b80 1 Processor -- start 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92de340 0x5570d92e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5570d9133140 con 0x5570d92e3350 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5570d91ce960 con 0x5570d92de340 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570d92ea720 con 0x5570d92dd980 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb64bcd8b80 10 client.0 _ll_register_callbacks cb 0x5570d91c7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34872/0 (socket says 172.21.15.26:34872) 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 -- 172.21.15.26:0/1130983420 learned_addr learned my addr 172.21.15.26:0/1130983420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63affd700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92de340 0x5570d92e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 -- 172.21.15.26:0/1130983420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 msgr2=0x5570d92e9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 -- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92de340 msgr2=0x5570d92e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92de340 0x5570d92e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.146+0000 7fb63b7fe700 1 -- 172.21.15.26:0/1130983420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d91cb830 con 0x5570d92e3350 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e76f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb634007740 tx=0x7fb634007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa169b54bc2d91f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb61eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb61eff5700 1 -- 172.21.15.26:0/1130983420 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634009cf0 con 0x5570d92e3350 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62c0b24b0 con 0x5570d92e3350 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb61eff5700 1 -- 172.21.15.26:0/1130983420 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb634028480 con 0x5570d92e3350 2024-05-20T20:56:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb61eff5700 1 -- 172.21.15.26:0/1130983420 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb634007da0 con 0x5570d92e3350 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb64bcd8b80 10 client.15522 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570d92f4ed0 con 0x5570d92e3350 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb61eff5700 1 -- 172.21.15.26:0/1130983420 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb62c0b24b0 con 0x5570d92e3350 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb61eff5700 1 -- 172.21.15.26:0/1130983420 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb634031050 con 0x5570d92e3350 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb64bcd8b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb64bcd8b80 10 client.15522 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.147+0000 7fb64bcd8b80 20 client.15522 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.148+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d92f7460 0x5570d92f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.148+0000 7fb64bcd8b80 4 client.15522 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.148+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d92a9b80 con 0x5570d92f7460 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.148+0000 7fb63affd700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d92f7460 0x5570d92f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.148+0000 7fb63affd700 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d92f7460 0x5570d92f9880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4917a40c39c8e10a server_cookie=52d591f3bb6d6819 in_seq=0 out_seq=0 2024-05-20T20:56:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.148+0000 7fb61eff5700 10 client.15522 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb61eff5700 1 -- 172.21.15.26:0/1130983420 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5570d92a9b80 con 0x5570d92f7460 2024-05-20T20:56:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb61eff5700 10 client.15522 handle_command_reply: tid=0 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 client.15522 shutdown 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 msgr2=0x5570d92e76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e76f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb634007740 tx=0x7fb634007770 comp rx=0 tx=0).stop 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 shutdown_connections 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d92f7460 0x5570d92f9880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92e3350 0x5570d92e76f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/1130983420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92de340 0x5570d92e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 >> 172.21.15.26:0/1130983420 conn(0x5570d920b220 msgr2=0x5570d92d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb61eff5700 0 client.15522 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 shutdown_connections 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.149+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/1130983420 wait complete. 2024-05-20T20:56:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.150+0000 7fb64bcd8b80 20 client.15522 trim_cache size 0 max 16384 2024-05-20T20:56:56.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 Processor -- start 2024-05-20T20:56:56.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 -- start start 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92e1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1b26f0 con 0x5570d92de340 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1aff80 con 0x5570d92e3350 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.170+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1addd0 con 0x5570d92dd980 2024-05-20T20:56:56.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92e1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34888/0 (socket says 172.21.15.26:34888) 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2002976591 learned_addr learned my addr 172.21.15.26:0/2002976591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63affd700 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2002976591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 msgr2=0x5570d92e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2002976591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 msgr2=0x5570d92e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2002976591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d92d2240 con 0x5570d92de340 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63affd700 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:56.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.171+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92e1540 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb63c113670 tx=0x7fb630010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a33493bd176020 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb620ff9700 1 -- 172.21.15.26:0/2002976591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63000de90 con 0x5570d92de340 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb620ff9700 1 -- 172.21.15.26:0/2002976591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb630022430 con 0x5570d92de340 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb620ff9700 1 -- 172.21.15.26:0/2002976591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6300123e0 con 0x5570d92de340 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2002976591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 msgr2=0x5570d92e1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92e1540 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb63c113670 tx=0x7fb630010000 comp rx=0 tx=0).stop 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2002976591 shutdown_connections 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2002976591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92e1540 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2002976591 >> 172.21.15.26:0/2002976591 conn(0x5570d920b220 msgr2=0x5570d92e0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2002976591 shutdown_connections 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.172+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2002976591 wait complete. 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 -- start start 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 Processor -- start 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92ec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1ae440 con 0x5570d92de340 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1af8b0 con 0x5570d92e3350 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1aeb10 con 0x5570d92dd980 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb63a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb63a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92ecec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34894/0 (socket says 172.21.15.26:34894) 2024-05-20T20:56:56.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb64bcd8b80 10 client.0 _ll_register_callbacks cb 0x5570d91c7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2319294686 learned_addr learned my addr 172.21.15.26:0/2319294686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb63affd700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.173+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2319294686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 msgr2=0x5570d92ec980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92ec980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 msgr2=0x5570d92ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92ed400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63a7fc700 1 -- 172.21.15.26:0/2319294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d9362310 con 0x5570d92de340 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63b7fe700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63affd700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb63a7fc700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92ecec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb630022d70 tx=0x7fb630009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33b74c4bbf354fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb61eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:56.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb61eff5700 1 -- 172.21.15.26:0/2319294686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630022660 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb61eff5700 1 -- 172.21.15.26:0/2319294686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb630022800 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb630022430 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb61eff5700 1 -- 172.21.15.26:0/2319294686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb630014a50 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb64bcd8b80 10 client.15528 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570d93635b0 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.174+0000 7fb61eff5700 1 -- 172.21.15.26:0/2319294686 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb630022430 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb61eff5700 1 -- 172.21.15.26:0/2319294686 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb63003b020 con 0x5570d92de340 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb64bcd8b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb64bcd8b80 10 client.15528 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb64bcd8b80 20 client.15528 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d9363ca0 0x5570d9366090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb64bcd8b80 4 client.15528 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-05-20T20:56:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x5570d923ed80 con 0x5570d9363ca0 2024-05-20T20:56:56.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb63affd700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d9363ca0 0x5570d9366090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:56.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb61eff5700 10 client.15528 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:56.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.175+0000 7fb63affd700 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d9363ca0 0x5570d9366090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405df01bad06c312 server_cookie=a8b0c30b36ea3fc in_seq=0 out_seq=0 2024-05-20T20:56:56.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb61eff5700 1 -- 172.21.15.26:0/2319294686 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5570d923ed80 con 0x5570d9363ca0 2024-05-20T20:56:56.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb61eff5700 10 client.15528 handle_command_reply: tid=0 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 client.15528 shutdown 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 msgr2=0x5570d92ecec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92ecec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb630022d70 tx=0x7fb630009f00 comp rx=0 tx=0).stop 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 shutdown_connections 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5570d9363ca0 0x5570d9366090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5570d92e3350 0x5570d92ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570d92dd980 0x5570d92ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb61eff5700 0 client.15528 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 --2- 172.21.15.26:0/2319294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570d92de340 0x5570d92ecec0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 >> 172.21.15.26:0/2319294686 conn(0x5570d920b220 msgr2=0x5570d92e16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 shutdown_connections 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 1 -- 172.21.15.26:0/2319294686 wait complete. 2024-05-20T20:56:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.185+0000 7fb64bcd8b80 20 client.15528 trim_cache size 0 max 16384 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 2, 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "621f3813": { 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 2, 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:56:56.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 138.8, 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:56.188 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6240b29b0 msgr2=0x7fb6240b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6240b29b0 0x7fb6240b4e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb63c079960 tx=0x7fb630008040 comp rx=0 tx=0).stop 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 msgr2=0x7fb63c111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 0x7fb63c111be0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d980 tx=0x7fb62c00dc90 comp rx=0 tx=0).stop 2024-05-20T20:56:56.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 shutdown_connections 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6240b29b0 0x7fb6240b4e70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63c102460 0x7fb63c111be0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1084d0 0x7fb63c112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 --2- 172.21.15.26:0/337097241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63c102e20 0x7fb63c112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 >> 172.21.15.26:0/337097241 conn(0x7fb63c079460 msgr2=0x7fb63c1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 shutdown_connections 2024-05-20T20:56:56.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:56.186+0000 7fb641442700 1 -- 172.21.15.26:0/337097241 wait complete. 2024-05-20T20:56:56.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:56 smithi026 ceph-mon[31990]: pgmap v412: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:56.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:56 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:56:56.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:56 smithi144 ceph-mon[35660]: pgmap v412: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:56.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:56 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:56:56.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:56 smithi055 ceph-mon[35213]: pgmap v412: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:56:56.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:56 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:56:57.687 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '621f3813', quiesce took 9.4 seconds. Will release after: 9.450000000000001 2024-05-20T20:56:58.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 -- 172.21.15.26:0/2481633208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb32806c5d0 msgr2=0x7fb3281103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 --2- 172.21.15.26:0/2481633208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb32806c5d0 0x7fb3281103f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb32000b0a0 tx=0x7fb32000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 -- 172.21.15.26:0/2481633208 shutdown_connections 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 --2- 172.21.15.26:0/2481633208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb32806c5d0 0x7fb3281103f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 --2- 172.21.15.26:0/2481633208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3281095d0 0x7fb3281099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 --2- 172.21.15.26:0/2481633208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb32806bc10 0x7fb32806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 -- 172.21.15.26:0/2481633208 >> 172.21.15.26:0/2481633208 conn(0x7fb3280793f0 msgr2=0x7fb328079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 -- 172.21.15.26:0/2481633208 shutdown_connections 2024-05-20T20:56:58.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.658+0000 7fb32f942700 1 -- 172.21.15.26:0/2481633208 wait complete. 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 Processor -- start 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 -- start start 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb32806bc10 0x7fb32819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb32806c5d0 0x7fb32819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 0x7fb3281998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281135c0 con 0x7fb3281095d0 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb328113440 con 0x7fb32806bc10 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32f942700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb328113740 con 0x7fb32806c5d0 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32dedf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 0x7fb3281998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32dedf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 0x7fb3281998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34918/0 (socket says 172.21.15.26:34918) 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32dedf700 1 -- 172.21.15.26:0/835532995 learned_addr learned my addr 172.21.15.26:0/835532995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32d6de700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb32806bc10 0x7fb32819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.659+0000 7fb32cedd700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb32806c5d0 0x7fb32819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32dedf700 1 -- 172.21.15.26:0/835532995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb32806c5d0 msgr2=0x7fb32819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32dedf700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb32806c5d0 0x7fb32819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32dedf700 1 -- 172.21.15.26:0/835532995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb32806bc10 msgr2=0x7fb32819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32dedf700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb32806bc10 0x7fb32819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32dedf700 1 -- 172.21.15.26:0/835532995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32819a020 con 0x7fb3281095d0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32dedf700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 0x7fb3281998c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb32000b830 tx=0x7fb320004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb31e7fc700 1 -- 172.21.15.26:0/835532995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320021070 con 0x7fb3281095d0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb31e7fc700 1 -- 172.21.15.26:0/835532995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb320007b80 con 0x7fb3281095d0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32819a2b0 con 0x7fb3281095d0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb31e7fc700 1 -- 172.21.15.26:0/835532995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3200052a0 con 0x7fb3281095d0 2024-05-20T20:56:58.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.660+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3281a65d0 con 0x7fb3281095d0 2024-05-20T20:56:58.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.663+0000 7fb31e7fc700 1 -- 172.21.15.26:0/835532995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fb320024070 con 0x7fb3281095d0 2024-05-20T20:56:58.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.663+0000 7fb31e7fc700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb3140b2910 0x7fb3140b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.664+0000 7fb32d6de700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb3140b2910 0x7fb3140b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.664+0000 7fb31e7fc700 1 -- 172.21.15.26:0/835532995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb32000f020 con 0x7fb3281095d0 2024-05-20T20:56:58.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.664+0000 7fb32d6de700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb3140b2910 0x7fb3140b4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb32819aa00 tx=0x7fb32400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:58.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.664+0000 7fb33a1d8b80 1 Processor -- start 2024-05-20T20:56:58.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.664+0000 7fb33a1d8b80 1 -- start start 2024-05-20T20:56:58.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c458050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c458620 0x55654c45bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c45c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55654c3b2c90 con 0x55654c457c70 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55654c3a2830 con 0x55654c45c370 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55654c3a6870 con 0x55654c458620 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c458050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c458050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34928/0 (socket says 172.21.15.26:34928) 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 -- 172.21.15.26:0/4257890472 learned_addr learned my addr 172.21.15.26:0/4257890472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32dedf700 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c45c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c458620 0x55654c45bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 -- 172.21.15.26:0/4257890472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c458620 msgr2=0x55654c45bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c458620 0x55654c45bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 -- 172.21.15.26:0/4257890472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 msgr2=0x55654c45c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c45c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 -- 172.21.15.26:0/4257890472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654c2f28d0 con 0x55654c457c70 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c458050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb324010e00 tx=0x7fb324013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8ea0cc678476c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb3137fe700 1 -- 172.21.15.26:0/4257890472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32401e420 con 0x55654c457c70 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb3137fe700 1 -- 172.21.15.26:0/4257890472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb32401ea50 con 0x55654c457c70 2024-05-20T20:56:58.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.665+0000 7fb3137fe700 1 -- 172.21.15.26:0/4257890472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32401ca10 con 0x55654c457c70 2024-05-20T20:56:58.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4257890472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 msgr2=0x55654c458050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c458050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb324010e00 tx=0x7fb324013000 comp rx=0 tx=0).stop 2024-05-20T20:56:58.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4257890472 shutdown_connections 2024-05-20T20:56:58.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c458050 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c45c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4257890472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c458620 0x55654c45bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4257890472 >> 172.21.15.26:0/4257890472 conn(0x55654c381080 msgr2=0x55654c44de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4257890472 shutdown_connections 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.666+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4257890472 wait complete. 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 -- start start 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 Processor -- start 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c463850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c460b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c461090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55654c2a8fa0 con 0x55654c457c70 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55654c3447c0 con 0x55654c458620 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55654c464530 con 0x55654c45c370 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb33a1d8b80 10 client.0 _ll_register_callbacks cb 0x55654c33d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:58.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb32d6de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c463850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb32d6de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c463850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34932/0 (socket says 172.21.15.26:34932) 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb32d6de700 1 -- 172.21.15.26:0/4251005524 learned_addr learned my addr 172.21.15.26:0/4251005524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.667+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c460b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32dedf700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c461090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32d6de700 1 -- 172.21.15.26:0/4251005524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 msgr2=0x55654c461090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c461090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32d6de700 1 -- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 msgr2=0x55654c460b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c460b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32d6de700 1 -- 172.21.15.26:0/4251005524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654c341690 con 0x55654c457c70 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c463850 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb32400b740 tx=0x7fb32401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82fd4bb37b23ad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb310ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb310ff9700 1 -- 172.21.15.26:0/4251005524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324036060 con 0x55654c457c70 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb310ff9700 1 -- 172.21.15.26:0/4251005524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb32401e630 con 0x55654c457c70 2024-05-20T20:56:58.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32401ea50 con 0x55654c457c70 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb310ff9700 1 -- 172.21.15.26:0/4251005524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb32401e830 con 0x55654c457c70 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb33a1d8b80 10 client.15540 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.668+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55654c46ecf0 con 0x55654c457c70 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb310ff9700 1 -- 172.21.15.26:0/4251005524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb32401ea50 con 0x55654c457c70 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb310ff9700 1 -- 172.21.15.26:0/4251005524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb32403a030 con 0x55654c457c70 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb33a1d8b80 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb33a1d8b80 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb33a1d8b80 20 client.15540 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:58.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c471270 0x55654c473660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb33a1d8b80 4 client.15540 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55654c41f9e0 con 0x55654c471270 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c471270 0x55654c473660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c471270 0x55654c473660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecf77f8925d7a3c server_cookie=fb0cabae8d87bf97 in_seq=0 out_seq=0 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.669+0000 7fb310ff9700 10 client.15540 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb310ff9700 1 -- 172.21.15.26:0/4251005524 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55654c41f9e0 con 0x55654c471270 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb310ff9700 10 client.15540 handle_command_reply: tid=0 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 client.15540 shutdown 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 msgr2=0x55654c463850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c463850 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb32400b740 tx=0x7fb32401c500 comp rx=0 tx=0).stop 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 shutdown_connections 2024-05-20T20:56:58.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c471270 0x55654c473660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c463850 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c461090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb310ff9700 0 client.15540 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4251005524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c460b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.670+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 >> 172.21.15.26:0/4251005524 conn(0x55654c381080 msgr2=0x55654c44d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.671+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 shutdown_connections 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.671+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4251005524 wait complete. 2024-05-20T20:56:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.671+0000 7fb33a1d8b80 20 client.15540 trim_cache size 0 max 16384 2024-05-20T20:56:58.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.691+0000 7fb33a1d8b80 1 Processor -- start 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 -- start start 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c453050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c44dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c44e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281ae280 con 0x55654c457c70 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281abb10 con 0x55654c458620 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281a9960 con 0x55654c45c370 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c453050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c453050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34944/0 (socket says 172.21.15.26:34944) 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 -- 172.21.15.26:0/3323493354 learned_addr learned my addr 172.21.15.26:0/3323493354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32cedd700 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c44dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32dedf700 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c44e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 -- 172.21.15.26:0/3323493354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 msgr2=0x55654c44e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c44e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 -- 172.21.15.26:0/3323493354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 msgr2=0x55654c44dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c44dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 -- 172.21.15.26:0/3323493354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654c44f520 con 0x55654c457c70 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32dedf700 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c44e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.692+0000 7fb32d6de700 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c453050 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb324000c00 tx=0x7fb324011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75a9c2dde207f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb312ffd700 1 -- 172.21.15.26:0/3323493354 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324036040 con 0x55654c457c70 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb312ffd700 1 -- 172.21.15.26:0/3323493354 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb32401e630 con 0x55654c457c70 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb312ffd700 1 -- 172.21.15.26:0/3323493354 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324004cf0 con 0x55654c457c70 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/3323493354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 msgr2=0x55654c453050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c453050 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb324000c00 tx=0x7fb324011ab0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/3323493354 shutdown_connections 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c457c70 0x55654c453050 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c45c370 0x55654c44e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/3323493354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c458620 0x55654c44dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/3323493354 >> 172.21.15.26:0/3323493354 conn(0x55654c381080 msgr2=0x55654c45b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/3323493354 shutdown_connections 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.693+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/3323493354 wait complete. 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 -- start start 2024-05-20T20:56:58.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 Processor -- start 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c457c70 0x55654c4cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 0x55654c4d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c4d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281a9fd0 con 0x55654c458620 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281ab440 con 0x55654c45c370 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281aa6a0 con 0x55654c457c70 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb33a1d8b80 10 client.0 _ll_register_callbacks cb 0x55654c33d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb32cedd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 0x55654c4d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb32cedd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 0x55654c4d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34954/0 (socket says 172.21.15.26:34954) 2024-05-20T20:56:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb32cedd700 1 -- 172.21.15.26:0/4260064485 learned_addr learned my addr 172.21.15.26:0/4260064485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.694+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c457c70 0x55654c4cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32dedf700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c4d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32cedd700 1 -- 172.21.15.26:0/4260064485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c457c70 msgr2=0x55654c4cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c457c70 0x55654c4cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32cedd700 1 -- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 msgr2=0x55654c4d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c4d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32cedd700 1 -- 172.21.15.26:0/4260064485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654c4d5170 con 0x55654c458620 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb32cedd700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 0x55654c4d0160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb31800a590 tx=0x7fb31801e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f15a2a32e7c264e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb310ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb310ff9700 1 -- 172.21.15.26:0/4260064485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318004a30 con 0x55654c458620 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32401e630 con 0x55654c458620 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb310ff9700 1 -- 172.21.15.26:0/4260064485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb318004bd0 con 0x55654c458620 2024-05-20T20:56:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb310ff9700 1 -- 172.21.15.26:0/4260064485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb318004dd0 con 0x55654c458620 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb33a1d8b80 10 client.15546 fetch_fsmap learned FSMap version 26 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.695+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55654c4dc530 con 0x55654c458620 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb310ff9700 1 -- 172.21.15.26:0/4260064485 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb32401e630 con 0x55654c458620 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb310ff9700 1 -- 172.21.15.26:0/4260064485 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb318012b70 con 0x55654c458620 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb33a1d8b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb33a1d8b80 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb33a1d8b80 20 client.15546 populate_metadata read hostname 'smithi026' 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c4dcc20 0x55654c4df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb33a1d8b80 4 client.15546 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55654c3b4be0 con 0x55654c4dcc20 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c4dcc20 0x55654c4df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb32d6de700 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c4dcc20 0x55654c4df040 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61c6f7091434b3 server_cookie=8c783b782131f451 in_seq=0 out_seq=0 2024-05-20T20:56:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.696+0000 7fb310ff9700 10 client.15546 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb310ff9700 1 -- 172.21.15.26:0/4260064485 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55654c3b4be0 con 0x55654c4dcc20 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb310ff9700 10 client.15546 handle_command_reply: tid=0 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 client.15546 shutdown 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 msgr2=0x55654c4d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 0x55654c4d0160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb31800a590 tx=0x7fb31801e900 comp rx=0 tx=0).stop 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 shutdown_connections 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55654c4dcc20 0x55654c4df040 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55654c45c370 0x55654c4d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb310ff9700 0 client.15546 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55654c457c70 0x55654c4cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 --2- 172.21.15.26:0/4260064485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55654c458620 0x55654c4d0160 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 >> 172.21.15.26:0/4260064485 conn(0x55654c381080 msgr2=0x55654c4603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 shutdown_connections 2024-05-20T20:56:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 1 -- 172.21.15.26:0/4260064485 wait complete. 2024-05-20T20:56:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.697+0000 7fb33a1d8b80 20 client.15546 trim_cache size 0 max 16384 2024-05-20T20:56:58.703 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:56:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:56:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:56:58.703 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb3140b2910 msgr2=0x7fb3140b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb3140b2910 0x7fb3140b4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb32819aa00 tx=0x7fb32400b410 comp rx=0 tx=0).stop 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 msgr2=0x7fb3281998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 0x7fb3281998c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb32000b830 tx=0x7fb320004d00 comp rx=0 tx=0).stop 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 shutdown_connections 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb3140b2910 0x7fb3140b4dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3281095d0 0x7fb3281998c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb32806bc10 0x7fb32819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 --2- 172.21.15.26:0/835532995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb32806c5d0 0x7fb32819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:56:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 >> 172.21.15.26:0/835532995 conn(0x7fb3280793f0 msgr2=0x7fb328101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:56:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 shutdown_connections 2024-05-20T20:56:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:56:58.698+0000 7fb32f942700 1 -- 172.21.15.26:0/835532995 wait complete. 2024-05-20T20:56:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:56:58 smithi026 ceph-mon[31990]: pgmap v413: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:56:59.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:56:58 smithi055 ceph-mon[35213]: pgmap v413: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:56:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:56:58 smithi144 ceph-mon[35660]: pgmap v413: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:56:59.574 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:56:59.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:56:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:00.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:00 smithi026 ceph-mon[31990]: pgmap v414: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:01.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:00 smithi055 ceph-mon[35213]: pgmap v414: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:00 smithi144 ceph-mon[35660]: pgmap v414: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:01.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3806497237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f081026a0 msgr2=0x7f4f08108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:01.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3806497237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f081026a0 0x7f4f08108200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0099c0 tx=0x7f4efc009cd0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3806497237 shutdown_connections 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3806497237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f08108930 0x7f4f081104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3806497237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f08101cf0 0x7f4f081020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3806497237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f081026a0 0x7f4f08108200 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3806497237 >> 172.21.15.26:0/3806497237 conn(0x7f4f08078a40 msgr2=0x7f4f080ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3806497237 shutdown_connections 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.600+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3806497237 wait complete. 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 Processor -- start 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 -- start start 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 0x7f4f08102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f081026a0 0x7f4f081031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f08108930 0x7f4f081036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081135a0 con 0x7f4f08101cf0 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113420 con 0x7f4f08108930 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f0dbed700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113720 con 0x7f4f081026a0 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 0x7f4f08102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 0x7f4f08102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49196/0 (socket says 172.21.15.26:49196) 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f077fe700 1 -- 172.21.15.26:0/3101341143 learned_addr learned my addr 172.21.15.26:0/3101341143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.601+0000 7f4f06ffd700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f081026a0 0x7f4f081031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f07fff700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f08108930 0x7f4f081036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f077fe700 1 -- 172.21.15.26:0/3101341143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f081026a0 msgr2=0x7f4f081031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f077fe700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f081026a0 0x7f4f081031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f077fe700 1 -- 172.21.15.26:0/3101341143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f08108930 msgr2=0x7f4f081036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f077fe700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f08108930 0x7f4f081036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f077fe700 1 -- 172.21.15.26:0/3101341143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a7ff0 con 0x7f4f08101cf0 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f077fe700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 0x7f4f08102c70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80060b0 tx=0x7f4ef800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef801a070 con 0x7f4f08101cf0 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ef800ddd0 con 0x7f4f08101cf0 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8014690 con 0x7f4f08101cf0 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a82e0 con 0x7f4f08101cf0 2024-05-20T20:57:01.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.602+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a8840 con 0x7f4f08101cf0 2024-05-20T20:57:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.603+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08104150 con 0x7f4f08101cf0 2024-05-20T20:57:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.605+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f4ef801d070 con 0x7f4f08101cf0 2024-05-20T20:57:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.605+0000 7f4f04ff9700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ef00b2990 0x7f4ef00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.606+0000 7f4f06ffd700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ef00b2990 0x7f4ef00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.606+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f4ef80e1bd0 con 0x7f4f08101cf0 2024-05-20T20:57:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.606+0000 7f4f06ffd700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ef00b2990 0x7f4ef00b4e50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4efc000c00 tx=0x7f4efc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:01.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.610+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ef8010070 con 0x7f4f08101cf0 2024-05-20T20:57:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.879+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f0802e560 con 0x7f4f08101cf0 2024-05-20T20:57:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.879+0000 7f4f04ff9700 1 -- 172.21.15.26:0/3101341143 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f4ef80a9730 con 0x7f4f08101cf0 2024-05-20T20:57:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:57:01.880 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:57:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ef00b2990 msgr2=0x7f4ef00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ef00b2990 0x7f4ef00b4e50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4efc000c00 tx=0x7f4efc005fb0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 msgr2=0x7f4f08102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 0x7f4f08102c70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80060b0 tx=0x7f4ef800d3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 shutdown_connections 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ef00b2990 0x7f4ef00b4e50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f08101cf0 0x7f4f08102c70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f08108930 0x7f4f081036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 --2- 172.21.15.26:0/3101341143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f081026a0 0x7f4f081031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 >> 172.21.15.26:0/3101341143 conn(0x7f4f08078a40 msgr2=0x7f4f080ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 shutdown_connections 2024-05-20T20:57:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:01.882+0000 7f4f0dbed700 1 -- 172.21.15.26:0/3101341143 wait complete. 2024-05-20T20:57:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:57:02.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:57:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:02 smithi026 ceph-mon[31990]: pgmap v415: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:02.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:02 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3101341143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:03.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:02 smithi055 ceph-mon[35213]: pgmap v415: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:03.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:02 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3101341143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:02 smithi144 ceph-mon[35660]: pgmap v415: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:02 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3101341143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 -- 172.21.15.26:0/3675190268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 msgr2=0x7f59c410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/3675190268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c410ac90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009ca0 tx=0x7f59bc02a8c0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 -- 172.21.15.26:0/3675190268 shutdown_connections 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/3675190268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c410ac90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/3675190268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 0x7f59c4108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/3675190268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c41035e0 0x7f59c41059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 -- 172.21.15.26:0/3675190268 >> 172.21.15.26:0/3675190268 conn(0x7f59c40fcfd0 msgr2=0x7f59c40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 -- 172.21.15.26:0/3675190268 shutdown_connections 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.596+0000 7f59ca8a0640 1 -- 172.21.15.26:0/3675190268 wait complete. 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 Processor -- start 2024-05-20T20:57:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 -- start start 2024-05-20T20:57:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c41035e0 0x7f59c4102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 0x7f59c4101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c4101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410e520 con 0x7f59c4108840 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410e3a0 con 0x7f59c41035e0 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59ca8a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410e6a0 con 0x7f59c4105f10 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59c8e3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c4101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59c8e3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c4101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49212/0 (socket says 172.21.15.26:49212) 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.597+0000 7f59c8e3d640 1 -- 172.21.15.26:0/2772301027 learned_addr learned my addr 172.21.15.26:0/2772301027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c37fe640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 0x7f59c4101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c3fff640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c41035e0 0x7f59c4102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c8e3d640 1 -- 172.21.15.26:0/2772301027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 msgr2=0x7f59c4101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 0x7f59c4101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c8e3d640 1 -- 172.21.15.26:0/2772301027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c41035e0 msgr2=0x7f59c4102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c41035e0 0x7f59c4102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c8e3d640 1 -- 172.21.15.26:0/2772301027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c4102360 con 0x7f59c4108840 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c37fe640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 0x7f59c4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c4101ad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f59bc031040 tx=0x7f59bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c17fa640 1 -- 172.21.15.26:0/2772301027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc032ed0 con 0x7f59c4108840 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c17fa640 1 -- 172.21.15.26:0/2772301027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc0044d0 con 0x7f59c4108840 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c41a2ba0 con 0x7f59c4108840 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59c17fa640 1 -- 172.21.15.26:0/2772301027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc030ca0 con 0x7f59c4108840 2024-05-20T20:57:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.598+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c41a30f0 con 0x7f59c4108840 2024-05-20T20:57:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59c17fa640 1 -- 172.21.15.26:0/2772301027 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f59bc004040 con 0x7f59c4108840 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 Processor -- start 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 -- start start 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59c17fa640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59ac0b2920 0x7f59ac0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd193bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 0x55d5fd1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d5fd0ffc30 con 0x55d5fd194220 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d5fd0ef7d0 con 0x55d5fd18e6c0 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.601+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d5fd0f3810 con 0x55d5fd18f0b0 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c37fe640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59ac0b2920 0x7f59ac0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 0x55d5fd1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 0x55d5fd1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49216/0 (socket says 172.21.15.26:49216) 2024-05-20T20:57:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 -- 172.21.15.26:0/3738925757 learned_addr learned my addr 172.21.15.26:0/3738925757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c3fff640 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c17fa640 1 -- 172.21.15.26:0/2772301027 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f59bc103860 con 0x7f59c4108840 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c37fe640 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd193bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 -- 172.21.15.26:0/3738925757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 msgr2=0x55d5fd193bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd193bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 -- 172.21.15.26:0/3738925757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 msgr2=0x55d5fd18eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 -- 172.21.15.26:0/3738925757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5fce04210 con 0x55d5fd194220 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c3fff640 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c37fe640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59ac0b2920 0x7f59ac0b4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005fd0 tx=0x7f59b80128d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 0x55d5fd1966b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005fa0 tx=0x7f59bc0cac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b414c2beadc3efb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59aa7fc640 1 -- 172.21.15.26:0/3738925757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0de640 con 0x55d5fd194220 2024-05-20T20:57:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59aa7fc640 1 -- 172.21.15.26:0/3738925757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc036070 con 0x55d5fd194220 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.602+0000 7f59aa7fc640 1 -- 172.21.15.26:0/3738925757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0b23a0 con 0x55d5fd194220 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 -- 172.21.15.26:0/3738925757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 msgr2=0x55d5fd1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 0x55d5fd1966b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005fa0 tx=0x7f59bc0cac60 comp rx=0 tx=0).stop 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 -- 172.21.15.26:0/3738925757 shutdown_connections 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd194220 0x55d5fd1966b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd193bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/3738925757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 -- 172.21.15.26:0/3738925757 >> 172.21.15.26:0/3738925757 conn(0x55d5fd189d30 msgr2=0x55d5fd18a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 -- 172.21.15.26:0/3738925757 shutdown_connections 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.603+0000 7f59cd5c0740 1 -- 172.21.15.26:0/3738925757 wait complete. 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 -- start start 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 Processor -- start 2024-05-20T20:57:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd198380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd194220 0x55d5fd19f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d5fcdd5790 con 0x55d5fd18e6c0 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d5fd0ab890 con 0x55d5fd194220 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d5fd0ffc30 con 0x55d5fd18f0b0 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59cd5c0740 10 client.0 _ll_register_callbacks cb 0x55d5fce03a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49218/0 (socket says 172.21.15.26:49218) 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59c3fff640 1 -- 172.21.15.26:0/4181609545 learned_addr learned my addr 172.21.15.26:0/4181609545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd194220 0x55d5fd19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.604+0000 7f59c37fe640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c3fff640 1 -- 172.21.15.26:0/4181609545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 msgr2=0x55d5fd1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c3fff640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c3fff640 1 -- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd194220 msgr2=0x55d5fd19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c3fff640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd194220 0x55d5fd19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c3fff640 1 -- 172.21.15.26:0/4181609545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5fced0620 con 0x55d5fd18e6c0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd194220 0x55d5fd19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59c3fff640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd198380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f59b4012700 tx=0x7f59b4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed0b5b57552314a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59a37fe640 1 -- 172.21.15.26:0/4181609545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4004260 con 0x55d5fd18e6c0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59a37fe640 1 -- 172.21.15.26:0/4181609545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f59b4004400 con 0x55d5fd18e6c0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59bc036070 con 0x55d5fd18e6c0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59a37fe640 1 -- 172.21.15.26:0/4181609545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59b4011940 con 0x55d5fd18e6c0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59cd5c0740 10 client.15564 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5fd1a29b0 con 0x55d5fd18e6c0 2024-05-20T20:57:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.605+0000 7f59a37fe640 1 -- 172.21.15.26:0/4181609545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc036070 con 0x55d5fd18e6c0 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59a37fe640 1 -- 172.21.15.26:0/4181609545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f59b400f440 con 0x55d5fd18e6c0 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59cd5c0740 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59cd5c0740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59cd5c0740 20 client.15564 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd1a4140 0x55d5fd1a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59cd5c0740 4 client.15564 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5fd101b80 con 0x55d5fd1a4140 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59c37fe640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd1a4140 0x55d5fd1a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59c37fe640 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd1a4140 0x55d5fd1a65b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910025467d203a28 server_cookie=8b75ed6e28984664 in_seq=0 out_seq=0 2024-05-20T20:57:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.606+0000 7f59a37fe640 10 client.15564 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59a37fe640 1 -- 172.21.15.26:0/4181609545 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d5fd101b80 con 0x55d5fd1a4140 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59a37fe640 10 client.15564 handle_command_reply: tid=0 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 client.15564 shutdown 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 msgr2=0x55d5fd198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd198380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f59b4012700 tx=0x7f59b4012bc0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 shutdown_connections 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd1a4140 0x55d5fd1a65b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd194220 0x55d5fd19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd18f0b0 0x55d5fd1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59a37fe640 0 client.15564 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/4181609545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd198380 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.607+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 >> 172.21.15.26:0/4181609545 conn(0x55d5fd189d30 msgr2=0x55d5fd188fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.608+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 shutdown_connections 2024-05-20T20:57:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.608+0000 7f59cd5c0740 1 -- 172.21.15.26:0/4181609545 wait complete. 2024-05-20T20:57:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.608+0000 7f59cd5c0740 20 client.15564 trim_cache size 0 max 16384 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 Processor -- start 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 -- start start 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18f0b0 0x55d5fd18d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd197650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41ae940 con 0x55d5fd18e6c0 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41ac210 con 0x55d5fd18f0b0 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41aa060 con 0x55d5fd194220 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49222/0 (socket says 172.21.15.26:49222) 2024-05-20T20:57:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c3fff640 1 -- 172.21.15.26:0/449836543 learned_addr learned my addr 172.21.15.26:0/449836543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c37fe640 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18f0b0 0x55d5fd18d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c3fff640 1 -- 172.21.15.26:0/449836543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 msgr2=0x55d5fd197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c3fff640 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.630+0000 7f59c3fff640 1 -- 172.21.15.26:0/449836543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18f0b0 msgr2=0x55d5fd18d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.631+0000 7f59c3fff640 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18f0b0 0x55d5fd18d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.631+0000 7f59c3fff640 1 -- 172.21.15.26:0/449836543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5fd1a6c30 con 0x55d5fd18e6c0 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.631+0000 7f59c3fff640 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18cc10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f59b400a000 tx=0x7f59b4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be11a0a1f64f2ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.631+0000 7f59a9ffb640 1 -- 172.21.15.26:0/449836543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b402b070 con 0x55d5fd18e6c0 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.631+0000 7f59a9ffb640 1 -- 172.21.15.26:0/449836543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59b400faf0 con 0x55d5fd18e6c0 2024-05-20T20:57:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.631+0000 7f59a9ffb640 1 -- 172.21.15.26:0/449836543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40035f0 con 0x55d5fd18e6c0 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 -- 172.21.15.26:0/449836543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 msgr2=0x55d5fd18cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18cc10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f59b400a000 tx=0x7f59b4009640 comp rx=0 tx=0).stop 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 -- 172.21.15.26:0/449836543 shutdown_connections 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18f0b0 0x55d5fd18d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/449836543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18e6c0 0x55d5fd18cc10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 -- 172.21.15.26:0/449836543 >> 172.21.15.26:0/449836543 conn(0x55d5fd189d30 msgr2=0x55d5fd193930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 -- 172.21.15.26:0/449836543 shutdown_connections 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 -- 172.21.15.26:0/449836543 wait complete. 2024-05-20T20:57:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.632+0000 7f59cd5c0740 1 -- start start 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 Processor -- start 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd19c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 0x55d5fd19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd19d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41aa6d0 con 0x55d5fd18f0b0 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41abb40 con 0x55d5fd18e6c0 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41aada0 con 0x55d5fd194220 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59cd5c0740 10 client.0 _ll_register_callbacks cb 0x7f59b4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 0x55d5fd19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 0x55d5fd19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49230/0 (socket says 172.21.15.26:49230) 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 -- 172.21.15.26:0/495184634 learned_addr learned my addr 172.21.15.26:0/495184634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c3fff640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c8e3d640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 -- 172.21.15.26:0/495184634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 msgr2=0x55d5fd19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 -- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 msgr2=0x55d5fd19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.633+0000 7f59c37fe640 1 -- 172.21.15.26:0/495184634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5fd19df70 con 0x55d5fd18f0b0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59c37fe640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 0x55d5fd19c9c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55d5fd18fb20 tx=0x7f59b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f59f1cd56bdd8c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59a37fe640 1 -- 172.21.15.26:0/495184634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8028590 con 0x55d5fd18f0b0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59a37fe640 1 -- 172.21.15.26:0/495184634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f59b8028730 con 0x55d5fd18f0b0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b400faf0 con 0x55d5fd18f0b0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59a37fe640 1 -- 172.21.15.26:0/495184634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59b8038050 con 0x55d5fd18f0b0 2024-05-20T20:57:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59cd5c0740 10 client.15570 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5fd19f320 con 0x55d5fd18f0b0 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.634+0000 7f59a37fe640 1 -- 172.21.15.26:0/495184634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59b400faf0 con 0x55d5fd18f0b0 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59a37fe640 1 -- 172.21.15.26:0/495184634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f59b8028070 con 0x55d5fd18f0b0 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59cd5c0740 10 client.15570 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59cd5c0740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59cd5c0740 20 client.15570 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd20f470 0x55d5fd211890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59cd5c0740 4 client.15570 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5fd0fb360 con 0x55d5fd20f470 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59c3fff640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd20f470 0x55d5fd211890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59a37fe640 10 client.15570 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.635+0000 7f59c3fff640 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd20f470 0x55d5fd211890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db7340bb6994abb server_cookie=e8019830615a8fa3 in_seq=0 out_seq=0 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.639+0000 7f59a37fe640 1 -- 172.21.15.26:0/495184634 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d5fd0fb360 con 0x55d5fd20f470 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.639+0000 7f59a37fe640 10 client.15570 handle_command_reply: tid=0 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.639+0000 7f59cd5c0740 1 client.15570 shutdown 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 msgr2=0x55d5fd19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 0x55d5fd19c9c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55d5fd18fb20 tx=0x7f59b8004730 comp rx=0 tx=0).stop 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 shutdown_connections 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5fd20f470 0x55d5fd211890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5fd194220 0x55d5fd19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59a37fe640 0 client.15570 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5fd18f0b0 0x55d5fd19c9c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 --2- 172.21.15.26:0/495184634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5fd18e6c0 0x55d5fd19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 >> 172.21.15.26:0/495184634 conn(0x55d5fd189d30 msgr2=0x55d5fd18cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 shutdown_connections 2024-05-20T20:57:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 1 -- 172.21.15.26:0/495184634 wait complete. 2024-05-20T20:57:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59cd5c0740 20 client.15570 trim_cache size 0 max 16384 2024-05-20T20:57:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:57:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:57:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59ac0b2920 msgr2=0x7f59ac0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59ac0b2920 0x7f59ac0b4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005fd0 tx=0x7f59b80128d0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 msgr2=0x7f59c4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.640+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c4101ad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f59bc031040 tx=0x7f59bc003ba0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 shutdown_connections 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f59ac0b2920 0x7f59ac0b4de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c4108840 0x7f59c4101ad0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c4105f10 0x7f59c4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 --2- 172.21.15.26:0/2772301027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c41035e0 0x7f59c4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 >> 172.21.15.26:0/2772301027 conn(0x7f59c40fcfd0 msgr2=0x7f59c4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 shutdown_connections 2024-05-20T20:57:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:03.641+0000 7f59ca8a0640 1 -- 172.21.15.26:0/2772301027 wait complete. 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/3110089833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01088f0 msgr2=0x7fd1c0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/3110089833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01088f0 0x7fd1c0110470 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800b0d0 tx=0x7fd1b800b3e0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/3110089833 shutdown_connections 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/3110089833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01088f0 0x7fd1c0110470 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/3110089833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c01037f0 0x7fd1c0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/3110089833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c01041a0 0x7fd1c01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/3110089833 >> 172.21.15.26:0/3110089833 conn(0x7fd1c007a6a0 msgr2=0x7fd1c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/3110089833 shutdown_connections 2024-05-20T20:57:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.634+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/3110089833 wait complete. 2024-05-20T20:57:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.635+0000 7fd1c5ed5700 1 Processor -- start 2024-05-20T20:57:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.635+0000 7fd1c5ed5700 1 -- start start 2024-05-20T20:57:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1c5ed5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c01037f0 0x7fd1c019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1c5ed5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 0x7fd1c019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1c5ed5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c01088f0 0x7fd1c0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1c5ed5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0113570 con 0x7fd1c01041a0 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1c5ed5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01133f0 con 0x7fd1c01037f0 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1c5ed5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01136f0 con 0x7fd1c01088f0 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 0x7fd1c019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 0x7fd1c019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49242/0 (socket says 172.21.15.26:49242) 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 -- 172.21.15.26:0/2260540328 learned_addr learned my addr 172.21.15.26:0/2260540328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c01037f0 0x7fd1c019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1bffff700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c01088f0 0x7fd1c0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 -- 172.21.15.26:0/2260540328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c01088f0 msgr2=0x7fd1c0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c01088f0 0x7fd1c0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 -- 172.21.15.26:0/2260540328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c01037f0 msgr2=0x7fd1c019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c01037f0 0x7fd1c019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.636+0000 7fd1beffd700 1 -- 172.21.15.26:0/2260540328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c019a010 con 0x7fd1c01041a0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.637+0000 7fd1beffd700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 0x7fd1c019fbf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c9b0 tx=0x7fd1b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.637+0000 7fd1bcff9700 1 -- 172.21.15.26:0/2260540328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007b00 con 0x7fd1c01041a0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.637+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c019a300 con 0x7fd1c01041a0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.637+0000 7fd1bcff9700 1 -- 172.21.15.26:0/2260540328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1b400ce50 con 0x7fd1c01041a0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.637+0000 7fd1bcff9700 1 -- 172.21.15.26:0/2260540328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4018660 con 0x7fd1c01041a0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.637+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c01a6510 con 0x7fd1c01041a0 2024-05-20T20:57:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.639+0000 7fd1bcff9700 1 -- 172.21.15.26:0/2260540328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fd1b400f450 con 0x7fd1c01041a0 2024-05-20T20:57:04.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.640+0000 7fd1bcff9700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1a80b2910 0x7fd1a80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.640+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1a80b2910 0x7fd1a80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.640+0000 7fd1bcff9700 1 -- 172.21.15.26:0/2260540328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd1b40e22a0 con 0x7fd1c01041a0 2024-05-20T20:57:04.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.641+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1a80b2910 0x7fd1a80b4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005950 tx=0x7fd1b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:04.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 Processor -- start 2024-05-20T20:57:04.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 -- start start 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446ab2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446ab6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ab73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e446a11c90 con 0x55e446ab3210 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e446a01830 con 0x55e446ab2860 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e446a05870 con 0x55e446ab6f60 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446ab6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446ab6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49254/0 (socket says 172.21.15.26:49254) 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1beffd700 1 -- 172.21.15.26:0/1445042965 learned_addr learned my addr 172.21.15.26:0/1445042965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446ab2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1bffff700 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ab73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.643+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/1445042965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 msgr2=0x55e446ab73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ab73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/1445042965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 msgr2=0x55e446ab6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446ab6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/1445042965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4469518d0 con 0x55e446ab2860 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1beffd700 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446ab6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446ab2c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005d90 tx=0x7fd1b0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4498e0b28f2d980f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1a5ffb700 1 -- 172.21.15.26:0/1445042965 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0011b80 con 0x55e446ab2860 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1a5ffb700 1 -- 172.21.15.26:0/1445042965 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1b0011d20 con 0x55e446ab2860 2024-05-20T20:57:04.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.644+0000 7fd1a5ffb700 1 -- 172.21.15.26:0/1445042965 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00056d0 con 0x55e446ab2860 2024-05-20T20:57:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1445042965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 msgr2=0x55e446ab2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446ab2c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005d90 tx=0x7fd1b0013000 comp rx=0 tx=0).stop 2024-05-20T20:57:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1445042965 shutdown_connections 2024-05-20T20:57:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ab73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446ab2c40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1445042965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446ab6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1445042965 >> 172.21.15.26:0/1445042965 conn(0x55e4469e0080 msgr2=0x55e446aace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1445042965 shutdown_connections 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1445042965 wait complete. 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.645+0000 7fd1d076bb80 1 -- start start 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 Processor -- start 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab2860 0x55e446abe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446abb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 0x55e446abbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e446907fa0 con 0x55e446ab6f60 2024-05-20T20:57:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e4469a37c0 con 0x55e446ab3210 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e446abf120 con 0x55e446ab2860 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1d076bb80 10 client.0 _ll_register_callbacks cb 0x55e44699c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 0x55e446abbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 0x55e446abbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49264/0 (socket says 172.21.15.26:49264) 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 -- 172.21.15.26:0/1761819839 learned_addr learned my addr 172.21.15.26:0/1761819839 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab2860 0x55e446abe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1beffd700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446abb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 -- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab2860 msgr2=0x55e446abe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab2860 0x55e446abe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 -- 172.21.15.26:0/1761819839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 msgr2=0x55e446abb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446abb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.646+0000 7fd1bffff700 1 -- 172.21.15.26:0/1761819839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4469a0690 con 0x55e446ab6f60 2024-05-20T20:57:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd1bffff700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 0x55e446abbcd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009610 tx=0x7fd1b80074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc7989e32ce5d4a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd19effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd19effd700 1 -- 172.21.15.26:0/1761819839 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8003c10 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b0011d20 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd19effd700 1 -- 172.21.15.26:0/1761819839 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd1b8003db0 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd19effd700 1 -- 172.21.15.26:0/1761819839 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1b8007680 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd1d076bb80 10 client.15582 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e446ac9970 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.647+0000 7fd19effd700 1 -- 172.21.15.26:0/1761819839 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1b0011d20 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd19effd700 1 -- 172.21.15.26:0/1761819839 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1b802f050 con 0x55e446ab6f60 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1d076bb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1d076bb80 10 client.15582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1d076bb80 20 client.15582 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446acbf00 0x55e446ace2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1d076bb80 4 client.15582 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e446a7e9e0 con 0x55e446acbf00 2024-05-20T20:57:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.648+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446acbf00 0x55e446ace2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.649+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446acbf00 0x55e446ace2f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=429d416b39eff4db server_cookie=97e1414eab8ab52c in_seq=0 out_seq=0 2024-05-20T20:57:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.649+0000 7fd19effd700 10 client.15582 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.649+0000 7fd19effd700 1 -- 172.21.15.26:0/1761819839 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e446a7e9e0 con 0x55e446acbf00 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.649+0000 7fd19effd700 10 client.15582 handle_command_reply: tid=0 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.649+0000 7fd1d076bb80 1 client.15582 shutdown 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.649+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 msgr2=0x55e446abbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 0x55e446abbcd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009610 tx=0x7fd1b80074e0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 shutdown_connections 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446acbf00 0x55e446ace2f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab6f60 0x55e446abbcd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab2860 0x55e446abe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1761819839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446abb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 >> 172.21.15.26:0/1761819839 conn(0x55e4469e0080 msgr2=0x55e446aac510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd19effd700 0 client.15582 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 shutdown_connections 2024-05-20T20:57:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1761819839 wait complete. 2024-05-20T20:57:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.650+0000 7fd1d076bb80 20 client.15582 trim_cache size 0 max 16384 2024-05-20T20:57:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 Processor -- start 2024-05-20T20:57:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 -- start start 2024-05-20T20:57:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446abdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446aacc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ad3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01b03e0 con 0x55e446ab3210 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01adc70 con 0x55e446ab2860 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01abac0 con 0x55e446ab6f60 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446aacc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446aacc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49274/0 (socket says 172.21.15.26:49274) 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.665+0000 7fd1beffd700 1 -- 172.21.15.26:0/1162096486 learned_addr learned my addr 172.21.15.26:0/1162096486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446abdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1bffff700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ad3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1beffd700 1 -- 172.21.15.26:0/1162096486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 msgr2=0x55e446ad3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1beffd700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ad3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1beffd700 1 -- 172.21.15.26:0/1162096486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 msgr2=0x55e446abdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1beffd700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446abdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1beffd700 1 -- 172.21.15.26:0/1162096486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e446abea90 con 0x55e446ab3210 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446abdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1bffff700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ad3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1beffd700 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446aacc90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40a9640 tx=0x7fd1b40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e98737a5b2a788a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1a57fa700 1 -- 172.21.15.26:0/1162096486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b406a070 con 0x55e446ab3210 2024-05-20T20:57:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1a57fa700 1 -- 172.21.15.26:0/1162096486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1b40adab0 con 0x55e446ab3210 2024-05-20T20:57:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.666+0000 7fd1a57fa700 1 -- 172.21.15.26:0/1162096486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40c3030 con 0x55e446ab3210 2024-05-20T20:57:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1162096486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 msgr2=0x55e446aacc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446aacc90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40a9640 tx=0x7fd1b40cc040 comp rx=0 tx=0).stop 2024-05-20T20:57:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1162096486 shutdown_connections 2024-05-20T20:57:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446ad3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab2860 0x55e446abdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/1162096486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab3210 0x55e446aacc90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1162096486 >> 172.21.15.26:0/1162096486 conn(0x55e4469e0080 msgr2=0x55e446ab1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1162096486 shutdown_connections 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 -- 172.21.15.26:0/1162096486 wait complete. 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 -- start start 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.667+0000 7fd1d076bb80 1 Processor -- start 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 0x55e446ac1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446ac1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446b34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01ac130 con 0x55e446ab2860 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01ad5a0 con 0x55e446ab3210 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01ac800 con 0x55e446ab6f60 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 0x55e446ac1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1d076bb80 10 client.0 _ll_register_callbacks cb 0x55e44699c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 0x55e446ac1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49282/0 (socket says 172.21.15.26:49282) 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/460711010 learned_addr learned my addr 172.21.15.26:0/460711010 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1beffd700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446ac1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 msgr2=0x55e446b34a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446b34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/460711010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 msgr2=0x55e446ac1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446ac1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 -- 172.21.15.26:0/460711010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e446b34f80 con 0x55e446ab2860 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1beffd700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446ac1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd1bf7fe700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 0x55e446ac1960 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0010f40 tx=0x7fd1b0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b5e6ad2fa24049 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd19effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.668+0000 7fd19effd700 1 -- 172.21.15.26:0/460711010 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0004920 con 0x55e446ab2860 2024-05-20T20:57:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd19effd700 1 -- 172.21.15.26:0/460711010 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd1b0004ac0 con 0x55e446ab2860 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b40adab0 con 0x55e446ab2860 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd19effd700 1 -- 172.21.15.26:0/460711010 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1b0004d00 con 0x55e446ab2860 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 10 client.15588 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e446b361f0 con 0x55e446ab2860 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd19effd700 1 -- 172.21.15.26:0/460711010 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1b40adab0 con 0x55e446ab2860 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd19effd700 1 -- 172.21.15.26:0/460711010 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1b0046050 con 0x55e446ab2860 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 10 client.15588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 20 client.15588 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446b36740 0x55e446b38b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 4 client.15588 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e446a13be0 con 0x55e446b36740 2024-05-20T20:57:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.669+0000 7fd1beffd700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446b36740 0x55e446b38b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1beffd700 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446b36740 0x55e446b38b80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e332e006b55bd090 server_cookie=aa902f0b95c337f8 in_seq=0 out_seq=0 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd19effd700 10 client.15588 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd19effd700 1 -- 172.21.15.26:0/460711010 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e446a13be0 con 0x55e446b36740 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd19effd700 10 client.15588 handle_command_reply: tid=0 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 client.15588 shutdown 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 msgr2=0x55e446ac1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 0x55e446ac1960 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0010f40 tx=0x7fd1b0011ab0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 shutdown_connections 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55e446b36740 0x55e446b38b80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e446ab2860 0x55e446ac1960 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e446ab6f60 0x55e446b34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 --2- 172.21.15.26:0/460711010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e446ab3210 0x55e446ac1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd19effd700 0 client.15588 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 >> 172.21.15.26:0/460711010 conn(0x55e4469e0080 msgr2=0x55e446ab58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.670+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 shutdown_connections 2024-05-20T20:57:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1d076bb80 1 -- 172.21.15.26:0/460711010 wait complete. 2024-05-20T20:57:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1d076bb80 20 client.15588 trim_cache size 0 max 16384 2024-05-20T20:57:04.676 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1a80b2910 msgr2=0x7fd1a80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1a80b2910 0x7fd1a80b4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005950 tx=0x7fd1b000b410 comp rx=0 tx=0).stop 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 msgr2=0x7fd1c019fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 0x7fd1c019fbf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c9b0 tx=0x7fd1b400ccc0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 shutdown_connections 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c01088f0 0x7fd1c0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c01037f0 0x7fd1c019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1a80b2910 0x7fd1a80b4dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 --2- 172.21.15.26:0/2260540328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1c01041a0 0x7fd1c019fbf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 >> 172.21.15.26:0/2260540328 conn(0x7fd1c007a6a0 msgr2=0x7fd1c01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 shutdown_connections 2024-05-20T20:57:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:04.671+0000 7fd1c5ed5700 1 -- 172.21.15.26:0/2260540328 wait complete. 2024-05-20T20:57:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:04 smithi026 ceph-mon[31990]: pgmap v416: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:05.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:04 smithi055 ceph-mon[35213]: pgmap v416: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:04 smithi144 ceph-mon[35660]: pgmap v416: 97 pgs: 97 active+clean; 669 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:05.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:57:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:06.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:06 smithi026 ceph-mon[31990]: pgmap v417: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:57:07.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:57:07.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:06 smithi055 ceph-mon[35213]: pgmap v417: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:57:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:06 smithi144 ceph-mon[35660]: pgmap v417: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.514+0000 7f841756a700 1 -- 172.21.15.26:0/394988100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006c400 msgr2=0x7f841010faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.514+0000 7f841756a700 1 --2- 172.21.15.26:0/394988100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006c400 0x7f841010faa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8400009990 tx=0x7f8400009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 -- 172.21.15.26:0/394988100 shutdown_connections 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 --2- 172.21.15.26:0/394988100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841010ffe0 0x7f84101123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 --2- 172.21.15.26:0/394988100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841006bae0 0x7f841006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 --2- 172.21.15.26:0/394988100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006c400 0x7f841010faa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 -- 172.21.15.26:0/394988100 >> 172.21.15.26:0/394988100 conn(0x7f84100793b0 msgr2=0x7f84100797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 -- 172.21.15.26:0/394988100 shutdown_connections 2024-05-20T20:57:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 -- 172.21.15.26:0/394988100 wait complete. 2024-05-20T20:57:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.515+0000 7f841756a700 1 Processor -- start 2024-05-20T20:57:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 -- start start 2024-05-20T20:57:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 0x7f84101a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841006c400 0x7f84101a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841010ffe0 0x7f841019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84101157a0 con 0x7f841006bae0 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8410115620 con 0x7f841006c400 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f841756a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8410115920 con 0x7f841010ffe0 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 0x7f84101a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 0x7f84101a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49294/0 (socket says 172.21.15.26:49294) 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 -- 172.21.15.26:0/3834713016 learned_addr learned my addr 172.21.15.26:0/3834713016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8414b05700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841006c400 0x7f84101a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415b07700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841010ffe0 0x7f841019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 -- 172.21.15.26:0/3834713016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841010ffe0 msgr2=0x7f841019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841010ffe0 0x7f841019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 -- 172.21.15.26:0/3834713016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841006c400 msgr2=0x7f84101a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841006c400 0x7f84101a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.516+0000 7f8415306700 1 -- 172.21.15.26:0/3834713016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841019c400 con 0x7f841006bae0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.517+0000 7f8415306700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 0x7f84101a1a50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f840c00d950 tx=0x7f840c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.517+0000 7f84067fc700 1 -- 172.21.15.26:0/3834713016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c016070 con 0x7f841006bae0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.517+0000 7f84067fc700 1 -- 172.21.15.26:0/3834713016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f840c00fd20 con 0x7f841006bae0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.517+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841019c6f0 con 0x7f841006bae0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.517+0000 7f84067fc700 1 -- 172.21.15.26:0/3834713016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c00e3b0 con 0x7f841006bae0 2024-05-20T20:57:07.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.517+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a89d0 con 0x7f841006bae0 2024-05-20T20:57:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.519+0000 7f84067fc700 1 -- 172.21.15.26:0/3834713016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f840c019070 con 0x7f841006bae0 2024-05-20T20:57:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.520+0000 7f84067fc700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83fc0b2880 0x7f83fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.520+0000 7f8414b05700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83fc0b2880 0x7f83fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.520+0000 7f84067fc700 1 -- 172.21.15.26:0/3834713016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f840c0e1d70 con 0x7f841006bae0 2024-05-20T20:57:07.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.521+0000 7f8414b05700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83fc0b2880 0x7f83fc0b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f840000b5c0 tx=0x7f8400005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:07.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.522+0000 7f8421e00b80 1 Processor -- start 2024-05-20T20:57:07.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 -- start start 2024-05-20T20:57:07.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336ed9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ed3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336ee0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561336e31d70 con 0x561336ed38a0 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561336e21910 con 0x561336ed9320 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8421e00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561336e25950 con 0x561336ed4260 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8414b05700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ed3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8414b05700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ed3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49306/0 (socket says 172.21.15.26:49306) 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8414b05700 1 -- 172.21.15.26:0/3578596316 learned_addr learned my addr 172.21.15.26:0/3578596316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8415306700 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336ed9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.523+0000 7f8415b07700 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336ee0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415306700 1 -- 172.21.15.26:0/3578596316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 msgr2=0x561336ee0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415306700 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336ee0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415306700 1 -- 172.21.15.26:0/3578596316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 msgr2=0x561336ed3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415306700 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ed3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415306700 1 -- 172.21.15.26:0/3578596316 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561336dcafb0 con 0x561336ed9320 2024-05-20T20:57:07.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415b07700 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336ee0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:07.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f8415306700 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336ed9700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f840c0df7b0 tx=0x7f840c0a91d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=233ef549e7b82b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:07.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f83fb7fe700 1 -- 172.21.15.26:0/3578596316 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c020040 con 0x561336ed9320 2024-05-20T20:57:07.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f83fb7fe700 1 -- 172.21.15.26:0/3578596316 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f840c0a92d0 con 0x561336ed9320 2024-05-20T20:57:07.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.524+0000 7f83fb7fe700 1 -- 172.21.15.26:0/3578596316 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c0b1f00 con 0x561336ed9320 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 -- 172.21.15.26:0/3578596316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 msgr2=0x561336ed9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336ed9700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f840c0df7b0 tx=0x7f840c0a91d0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 -- 172.21.15.26:0/3578596316 shutdown_connections 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336ee0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336ed9700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 --2- 172.21.15.26:0/3578596316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ed3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 -- 172.21.15.26:0/3578596316 >> 172.21.15.26:0/3578596316 conn(0x561336dff240 msgr2=0x561336ece3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 -- 172.21.15.26:0/3578596316 shutdown_connections 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 -- 172.21.15.26:0/3578596316 wait complete. 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.525+0000 7f8421e00b80 1 -- start start 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 Processor -- start 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336edfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336edd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336edd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561336d27fa0 con 0x561336ed38a0 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561336d89da0 con 0x561336ed9320 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561336ee0620 con 0x561336ed4260 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8415306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336edfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8415306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336edfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49320/0 (socket says 172.21.15.26:49320) 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8415306700 1 -- 172.21.15.26:0/622501983 learned_addr learned my addr 172.21.15.26:0/622501983 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8421e00b80 10 client.0 _ll_register_callbacks cb 0x561336dbc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8414b05700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336edd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.526+0000 7f8415b07700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336edd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8415306700 1 -- 172.21.15.26:0/622501983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 msgr2=0x561336edd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8415306700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336edd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8415306700 1 -- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 msgr2=0x561336edd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8415306700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336edd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8415306700 1 -- 172.21.15.26:0/622501983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561336d718d0 con 0x561336ed38a0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8415306700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336edfd00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f840c0a5300 tx=0x7f840c010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2011389914f14b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f83f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f83f8ff9700 1 -- 172.21.15.26:0/622501983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c020050 con 0x561336ed38a0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f83f8ff9700 1 -- 172.21.15.26:0/622501983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f840c0b1ca0 con 0x561336ed38a0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840c0a92d0 con 0x561336ed38a0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f83f8ff9700 1 -- 172.21.15.26:0/622501983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f840c0b2130 con 0x561336ed38a0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8421e00b80 10 client.15600 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.527+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561336ddb760 con 0x561336ed38a0 2024-05-20T20:57:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f83f8ff9700 1 -- 172.21.15.26:0/622501983 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f840c0a92d0 con 0x561336ed38a0 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f83f8ff9700 1 -- 172.21.15.26:0/622501983 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f840c0ad2c0 con 0x561336ed38a0 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8421e00b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8421e00b80 10 client.15600 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8421e00b80 20 client.15600 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8421e00b80 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336eef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8421e00b80 4 client.15600 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561336e33cc0 con 0x561336eed000 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.528+0000 7f8414b05700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336eef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.529+0000 7f83f8ff9700 10 client.15600 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.529+0000 7f8414b05700 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336eef3f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4829d60244dbcd server_cookie=db04266ceee053a7 in_seq=0 out_seq=0 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f83f8ff9700 1 -- 172.21.15.26:0/622501983 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561336e33cc0 con 0x561336eed000 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f83f8ff9700 10 client.15600 handle_command_reply: tid=0 2024-05-20T20:57:07.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 client.15600 shutdown 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 msgr2=0x561336edfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336edfd00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f840c0a5300 tx=0x7f840c010970 comp rx=0 tx=0).stop 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 shutdown_connections 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336eef3f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336edfd00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f83f8ff9700 0 client.15600 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed9320 0x561336edd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 --2- 172.21.15.26:0/622501983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed4260 0x561336edd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 >> 172.21.15.26:0/622501983 conn(0x561336dff240 msgr2=0x561336ecdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.530+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 shutdown_connections 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.531+0000 7f8421e00b80 1 -- 172.21.15.26:0/622501983 wait complete. 2024-05-20T20:57:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.531+0000 7f8421e00b80 20 client.15600 trim_cache size 0 max 16384 2024-05-20T20:57:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 Processor -- start 2024-05-20T20:57:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 -- start start 2024-05-20T20:57:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed38a0 0x561336edbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336ece270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 0x561336eda010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84101b2840 con 0x561336ed9320 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84101b00d0 con 0x561336ed4260 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8421e00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84101adf20 con 0x561336ed38a0 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8415b07700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 0x561336eda010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8415b07700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 0x561336eda010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49326/0 (socket says 172.21.15.26:49326) 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8415b07700 1 -- 172.21.15.26:0/1976869205 learned_addr learned my addr 172.21.15.26:0/1976869205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8414b05700 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336ece270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.549+0000 7f8415306700 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed38a0 0x561336edbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8415b07700 1 -- 172.21.15.26:0/1976869205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed38a0 msgr2=0x561336edbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8415b07700 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed38a0 0x561336edbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8415b07700 1 -- 172.21.15.26:0/1976869205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 msgr2=0x561336ece270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8415b07700 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336ece270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8415b07700 1 -- 172.21.15.26:0/1976869205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561336ee04e0 con 0x561336ed9320 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8414b05700 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336ece270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f8415b07700 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 0x561336eda010 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f84080146b0 tx=0x7f8408007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380e57b30b06af8a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f83faffd700 1 -- 172.21.15.26:0/1976869205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840801a070 con 0x561336ed9320 2024-05-20T20:57:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f83faffd700 1 -- 172.21.15.26:0/1976869205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8408004bb0 con 0x561336ed9320 2024-05-20T20:57:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.550+0000 7f83faffd700 1 -- 172.21.15.26:0/1976869205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080052a0 con 0x561336ed9320 2024-05-20T20:57:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.551+0000 7f8421e00b80 1 -- 172.21.15.26:0/1976869205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 msgr2=0x561336eda010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.551+0000 7f8421e00b80 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 0x561336eda010 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f84080146b0 tx=0x7f8408007950 comp rx=0 tx=0).stop 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.551+0000 7f8421e00b80 1 -- 172.21.15.26:0/1976869205 shutdown_connections 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed9320 0x561336eda010 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed38a0 0x561336edbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 --2- 172.21.15.26:0/1976869205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336ece270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 -- 172.21.15.26:0/1976869205 >> 172.21.15.26:0/1976869205 conn(0x561336dff240 msgr2=0x561336ed3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 -- 172.21.15.26:0/1976869205 shutdown_connections 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 -- 172.21.15.26:0/1976869205 wait complete. 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.552+0000 7f8421e00b80 1 -- start start 2024-05-20T20:57:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 Processor -- start 2024-05-20T20:57:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ef28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336eedc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed9320 0x561336eee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84101ae590 con 0x561336ed38a0 2024-05-20T20:57:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84101afa00 con 0x561336ed4260 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84101aec60 con 0x561336ed9320 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8421e00b80 10 client.0 _ll_register_callbacks cb 0x561336dbc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8415306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ef28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8414b05700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336eedc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8415306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ef28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49336/0 (socket says 172.21.15.26:49336) 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8415306700 1 -- 172.21.15.26:0/531253868 learned_addr learned my addr 172.21.15.26:0/531253868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.553+0000 7f8415b07700 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed9320 0x561336eee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8415306700 1 -- 172.21.15.26:0/531253868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed9320 msgr2=0x561336eee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8415306700 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed9320 0x561336eee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8415306700 1 -- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 msgr2=0x561336eedc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8415306700 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336eedc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8415306700 1 -- 172.21.15.26:0/531253868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561336eee720 con 0x561336ed38a0 2024-05-20T20:57:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8415306700 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ef28a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f840c0a5350 tx=0x7f840c005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dc94d959f0bcbd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f83f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f83f8ff9700 1 -- 172.21.15.26:0/531253868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c020040 con 0x561336ed38a0 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f83f8ff9700 1 -- 172.21.15.26:0/531253868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f840c0b1ca0 con 0x561336ed38a0 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8408004bb0 con 0x561336ed38a0 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f83f8ff9700 1 -- 172.21.15.26:0/531253868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f840c0ca350 con 0x561336ed38a0 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8421e00b80 10 client.15606 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.554+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561336f57600 con 0x561336ed38a0 2024-05-20T20:57:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f83f8ff9700 1 -- 172.21.15.26:0/531253868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8408004bb0 con 0x561336ed38a0 2024-05-20T20:57:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f83f8ff9700 1 -- 172.21.15.26:0/531253868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f840c0c3bc0 con 0x561336ed38a0 2024-05-20T20:57:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8421e00b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8421e00b80 10 client.15606 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:57:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8421e00b80 20 client.15606 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8421e00b80 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336f59b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8421e00b80 4 client.15606 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561336e2d4a0 con 0x561336eed000 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.555+0000 7f8414b05700 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336f59b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f83f8ff9700 10 client.15606 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8414b05700 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336f59b70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e906d68fdbe256b server_cookie=81849e644d65870 in_seq=0 out_seq=0 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f83f8ff9700 1 -- 172.21.15.26:0/531253868 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561336e2d4a0 con 0x561336eed000 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f83f8ff9700 10 client.15606 handle_command_reply: tid=0 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 client.15606 shutdown 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 msgr2=0x561336ef28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ef28a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f840c0a5350 tx=0x7f840c005b40 comp rx=0 tx=0).stop 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 shutdown_connections 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561336eed000 0x561336f59b70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f83f8ff9700 0 client.15606 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561336ed38a0 0x561336ef28a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561336ed9320 0x561336eee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 --2- 172.21.15.26:0/531253868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561336ed4260 0x561336eedc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 >> 172.21.15.26:0/531253868 conn(0x561336dff240 msgr2=0x561336ed6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 shutdown_connections 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.556+0000 7f8421e00b80 1 -- 172.21.15.26:0/531253868 wait complete. 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f8421e00b80 20 client.15606 trim_cache size 0 max 16384 2024-05-20T20:57:07.559 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83fc0b2880 msgr2=0x7f83fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83fc0b2880 0x7f83fc0b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f840000b5c0 tx=0x7f8400005960 comp rx=0 tx=0).stop 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 msgr2=0x7f84101a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 0x7f84101a1a50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f840c00d950 tx=0x7f840c00dc60 comp rx=0 tx=0).stop 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 shutdown_connections 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83fc0b2880 0x7f83fc0b4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f841006bae0 0x7f84101a1a50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841010ffe0 0x7f841019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 --2- 172.21.15.26:0/3834713016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841006c400 0x7f84101a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 >> 172.21.15.26:0/3834713016 conn(0x7f84100793b0 msgr2=0x7f84101114b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 shutdown_connections 2024-05-20T20:57:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:07.557+0000 7f841756a700 1 -- 172.21.15.26:0/3834713016 wait complete. 2024-05-20T20:57:07.778 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:08.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:07 smithi055 ceph-mon[35213]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-20T20:57:08.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:07 smithi026 ceph-mon[31990]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-20T20:57:08.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:07 smithi144 ceph-mon[35660]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-20T20:57:08.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:57:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:09.166 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:08 smithi026 ceph-mon[31990]: pgmap v418: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-20T20:57:09.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:08 smithi055 ceph-mon[35213]: pgmap v418: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-20T20:57:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:08 smithi144 ceph-mon[35660]: pgmap v418: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 -- 172.21.15.26:0/1636081032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 msgr2=0x7fa074110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 --2- 172.21.15.26:0/1636081032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074110420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b0a0 tx=0x7fa06c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 -- 172.21.15.26:0/1636081032 shutdown_connections 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 --2- 172.21.15.26:0/1636081032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074110420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 --2- 172.21.15.26:0/1636081032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0741095e0 0x7fa0741099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 --2- 172.21.15.26:0/1636081032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa07406bcb0 0x7fa07406c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.733+0000 7fa079ead700 1 -- 172.21.15.26:0/1636081032 >> 172.21.15.26:0/1636081032 conn(0x7fa0740784b0 msgr2=0x7fa0740788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.734+0000 7fa079ead700 1 -- 172.21.15.26:0/1636081032 shutdown_connections 2024-05-20T20:57:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.734+0000 7fa079ead700 1 -- 172.21.15.26:0/1636081032 wait complete. 2024-05-20T20:57:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.734+0000 7fa079ead700 1 Processor -- start 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.734+0000 7fa079ead700 1 -- start start 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa079ead700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa07406bcb0 0x7fa074102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa079ead700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa079ead700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0741095e0 0x7fa074102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa079ead700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa074113610 con 0x7fa07406c670 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa079ead700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa074113490 con 0x7fa0741095e0 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa079ead700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa074113790 con 0x7fa07406bcb0 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53352/0 (socket says 172.21.15.26:53352) 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 -- 172.21.15.26:0/21225577 learned_addr learned my addr 172.21.15.26:0/21225577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa0737fe700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa07406bcb0 0x7fa074102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa073fff700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0741095e0 0x7fa074102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 -- 172.21.15.26:0/21225577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa07406bcb0 msgr2=0x7fa074102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa07406bcb0 0x7fa074102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 -- 172.21.15.26:0/21225577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0741095e0 msgr2=0x7fa074102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0741095e0 0x7fa074102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 -- 172.21.15.26:0/21225577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0741a80d0 con 0x7fa07406c670 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa072ffd700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074102a80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa06400c970 tx=0x7fa06400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.735+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064007aa0 con 0x7fa07406c670 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.736+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0741a83c0 con 0x7fa07406c670 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.736+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa064004cc0 con 0x7fa07406c670 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.736+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064005500 con 0x7fa07406c670 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.736+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741a8900 con 0x7fa07406c670 2024-05-20T20:57:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.737+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07410bae0 con 0x7fa07406c670 2024-05-20T20:57:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.738+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa064007c40 con 0x7fa07406c670 2024-05-20T20:57:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.739+0000 7fa070ff9700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0600b2900 0x7fa0600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.739+0000 7fa0737fe700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0600b2900 0x7fa0600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.739+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa0640e2060 con 0x7fa07406c670 2024-05-20T20:57:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.739+0000 7fa0737fe700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0600b2900 0x7fa0600b4dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa05c006fd0 tx=0x7fa05c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:10.744+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0640a9db0 con 0x7fa07406c670 2024-05-20T20:57:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:10 smithi026 ceph-mon[31990]: pgmap v419: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:11.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.021+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa074104ca0 con 0x7fa07406c670 2024-05-20T20:57:11.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.021+0000 7fa070ff9700 1 -- 172.21.15.26:0/21225577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fa064005700 con 0x7fa07406c670 2024-05-20T20:57:11.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:57:11.022 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0600b2900 msgr2=0x7fa0600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0600b2900 0x7fa0600b4dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa05c006fd0 tx=0x7fa05c008040 comp rx=0 tx=0).stop 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 msgr2=0x7fa074102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074102a80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa06400c970 tx=0x7fa06400cd30 comp rx=0 tx=0).stop 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 shutdown_connections 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0741095e0 0x7fa074102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa07406bcb0 0x7fa074102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0600b2900 0x7fa0600b4dc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 --2- 172.21.15.26:0/21225577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa07406c670 0x7fa074102a80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 >> 172.21.15.26:0/21225577 conn(0x7fa0740784b0 msgr2=0x7fa074101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 shutdown_connections 2024-05-20T20:57:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:11.024+0000 7fa079ead700 1 -- 172.21.15.26:0/21225577 wait complete. 2024-05-20T20:57:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:57:11.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:10 smithi055 ceph-mon[35213]: pgmap v419: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:10 smithi144 ceph-mon[35660]: pgmap v419: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:11.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 621f3813 --release --await' 2024-05-20T20:57:11.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 621f3813 --release --await 2024-05-20T20:57:12.131 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:11 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/21225577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:12.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:11 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/21225577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:12.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:11 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/21225577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:12.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 -- 172.21.15.26:0/4032641682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824102470 msgr2=0x7f98241028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 --2- 172.21.15.26:0/4032641682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824102470 0x7f98241028f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f98140099c0 tx=0x7f9814009cd0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 -- 172.21.15.26:0/4032641682 shutdown_connections 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 --2- 172.21.15.26:0/4032641682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9824102e30 0x7f982410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 --2- 172.21.15.26:0/4032641682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824108550 0x7f9824108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 --2- 172.21.15.26:0/4032641682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824102470 0x7f98241028f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 -- 172.21.15.26:0/4032641682 >> 172.21.15.26:0/4032641682 conn(0x7f9824079480 msgr2=0x7f9824079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 -- 172.21.15.26:0/4032641682 shutdown_connections 2024-05-20T20:57:12.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.533+0000 7f982b9a6700 1 -- 172.21.15.26:0/4032641682 wait complete. 2024-05-20T20:57:12.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 Processor -- start 2024-05-20T20:57:12.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 -- start start 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 0x7f982419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9824102e30 0x7f982419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824108550 0x7f98241999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9824113630 con 0x7f9824108550 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98241134b0 con 0x7f9824102470 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f982b9a6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98241137b0 con 0x7f9824102e30 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f9829742700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 0x7f982419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f9829f43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824108550 0x7f98241999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f9829742700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 0x7f982419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:56552/0 (socket says 172.21.15.26:56552) 2024-05-20T20:57:12.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f9829742700 1 -- 172.21.15.26:0/1190909744 learned_addr learned my addr 172.21.15.26:0/1190909744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.534+0000 7f9828f41700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9824102e30 0x7f982419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829742700 1 -- 172.21.15.26:0/1190909744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9824102e30 msgr2=0x7f982419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829742700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9824102e30 0x7f982419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829742700 1 -- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824108550 msgr2=0x7f98241999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829742700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824108550 0x7f98241999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829742700 1 -- 172.21.15.26:0/1190909744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982419a110 con 0x7f9824102470 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829f43700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824108550 0x7f98241999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f9829742700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 0x7f982419f7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f982000e9a0 tx=0x7f982000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f981a7fc700 1 -- 172.21.15.26:0/1190909744 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982000ca10 con 0x7f9824102470 2024-05-20T20:57:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.535+0000 7f981a7fc700 1 -- 172.21.15.26:0/1190909744 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f982000cbb0 con 0x7f9824102470 2024-05-20T20:57:12.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.536+0000 7f981a7fc700 1 -- 172.21.15.26:0/1190909744 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820018680 con 0x7f9824102470 2024-05-20T20:57:12.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.536+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982419a400 con 0x7f9824102470 2024-05-20T20:57:12.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.536+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98241a66c0 con 0x7f9824102470 2024-05-20T20:57:12.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.539+0000 7f981a7fc700 1 -- 172.21.15.26:0/1190909744 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9820010430 con 0x7f9824102470 2024-05-20T20:57:12.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.540+0000 7f983623cb80 1 Processor -- start 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 -- start start 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdee6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdeea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdeeaf70 0x5604fdeeb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5604fde34c90 con 0x5604fdee6870 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5604fde24830 con 0x5604fdeeaf70 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f983623cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604fde28870 con 0x5604fdee7220 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f981a7fc700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98100b2750 0x7f98100b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f9829742700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdeea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f9829f43700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98100b2750 0x7f98100b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f9829742700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdeea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38468/0 (socket says 172.21.15.26:38468) 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.541+0000 7f9829742700 1 -- 172.21.15.26:0/4292249978 learned_addr learned my addr 172.21.15.26:0/4292249978 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829f43700 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdee6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829742700 1 -- 172.21.15.26:0/4292249978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdeeaf70 msgr2=0x5604fdeeb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829742700 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdeeaf70 0x5604fdeeb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829742700 1 -- 172.21.15.26:0/4292249978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 msgr2=0x5604fdee6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829742700 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdee6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829742700 1 -- 172.21.15.26:0/4292249978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fdd748d0 con 0x5604fdee7220 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829f43700 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdee6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f981a7fc700 1 -- 172.21.15.26:0/1190909744 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f98200e2670 con 0x7f9824102470 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829742700 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdeea950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98200882e0 tx=0x7f982006f110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c543efa8cb94d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f9829f43700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98100b2750 0x7f98100b4c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f981c009fd0 tx=0x7f981c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f980f7fe700 1 -- 172.21.15.26:0/4292249978 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200a99a0 con 0x5604fdee7220 2024-05-20T20:57:12.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f980f7fe700 1 -- 172.21.15.26:0/4292249978 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9820010e50 con 0x5604fdee7220 2024-05-20T20:57:12.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.542+0000 7f980f7fe700 1 -- 172.21.15.26:0/4292249978 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200a5360 con 0x5604fdee7220 2024-05-20T20:57:12.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.543+0000 7f983623cb80 1 -- 172.21.15.26:0/4292249978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 msgr2=0x5604fdeea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.543+0000 7f983623cb80 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdeea950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98200882e0 tx=0x7f982006f110 comp rx=0 tx=0).stop 2024-05-20T20:57:12.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 -- 172.21.15.26:0/4292249978 shutdown_connections 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdee6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdeeaf70 0x5604fdeeb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 --2- 172.21.15.26:0/4292249978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdeea950 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 -- 172.21.15.26:0/4292249978 >> 172.21.15.26:0/4292249978 conn(0x5604fde03080 msgr2=0x5604fdecfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 -- 172.21.15.26:0/4292249978 shutdown_connections 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 -- 172.21.15.26:0/4292249978 wait complete. 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.544+0000 7f983623cb80 1 -- start start 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 Processor -- start 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 0x5604fdef2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdeef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdeefb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5604fdd2afa0 con 0x5604fdeeaf70 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5604fddc67c0 con 0x5604fdee7220 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604fdef74e0 con 0x5604fdee6870 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.545+0000 7f983623cb80 10 client.0 _ll_register_callbacks cb 0x5604fddbf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 0x5604fdef2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 0x5604fdef2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38478/0 (socket says 172.21.15.26:38478) 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 -- 172.21.15.26:0/1075997574 learned_addr learned my addr 172.21.15.26:0/1075997574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829f43700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdeefb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 -- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 msgr2=0x5604fdeef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9828f41700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdeef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdeef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 -- 172.21.15.26:0/1075997574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 msgr2=0x5604fdeefb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdeefb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.546+0000 7f9829742700 1 -- 172.21.15.26:0/1075997574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fddc3690 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f9829742700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 0x5604fdef2310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9820009230 tx=0x7f98200b2420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa07147fe58d3f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f980cff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f980cff9700 1 -- 172.21.15.26:0/1075997574 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200a9340 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f980cff9700 1 -- 172.21.15.26:0/1075997574 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f98200b25a0 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9820010e50 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f980cff9700 1 -- 172.21.15.26:0/1075997574 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98200a9520 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f983623cb80 10 client.25174 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.547+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604fdf01b70 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f980cff9700 1 -- 172.21.15.26:0/1075997574 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9820010e50 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f980cff9700 1 -- 172.21.15.26:0/1075997574 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9820072020 con 0x5604fdee6870 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f983623cb80 10 client.25174 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:12.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f983623cb80 10 client.25174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f983623cb80 20 client.25174 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f983623cb80 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf04040 0x5604fdf06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f983623cb80 4 client.25174 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.548+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604fdea19e0 con 0x5604fdf04040 2024-05-20T20:57:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.549+0000 7f9828f41700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf04040 0x5604fdf06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.549+0000 7f9828f41700 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf04040 0x5604fdf06430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68642e86f0c63b59 server_cookie=3699b5be706fbfdf in_seq=0 out_seq=0 2024-05-20T20:57:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.549+0000 7f980cff9700 10 client.25174 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f980cff9700 1 -- 172.21.15.26:0/1075997574 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5604fdea19e0 con 0x5604fdf04040 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f980cff9700 10 client.25174 handle_command_reply: tid=0 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 client.25174 shutdown 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 msgr2=0x5604fdef2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 0x5604fdef2310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9820009230 tx=0x7f98200b2420 comp rx=0 tx=0).stop 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 shutdown_connections 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf04040 0x5604fdf06430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdeefb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee6870 0x5604fdef2310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 --2- 172.21.15.26:0/1075997574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdeef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 >> 172.21.15.26:0/1075997574 conn(0x5604fde03080 msgr2=0x5604fdeec360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.550+0000 7f980cff9700 0 client.25174 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.551+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 shutdown_connections 2024-05-20T20:57:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.551+0000 7f983623cb80 1 -- 172.21.15.26:0/1075997574 wait complete. 2024-05-20T20:57:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.551+0000 7f983623cb80 20 client.25174 trim_cache size 0 max 16384 2024-05-20T20:57:12.562 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:12.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 Processor -- start 2024-05-20T20:57:12.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 -- start start 2024-05-20T20:57:12.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdef1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdee88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdeeaf70 0x5604fdee8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9824074890 con 0x5604fdee6870 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9824072120 con 0x5604fdee7220 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f983623cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f982406ff70 con 0x5604fdeeaf70 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f9829742700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdef1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f9829742700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdef1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53390/0 (socket says 172.21.15.26:53390) 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.570+0000 7f9829742700 1 -- 172.21.15.26:0/4073006924 learned_addr learned my addr 172.21.15.26:0/4073006924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9828f41700 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdee88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829f43700 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdeeaf70 0x5604fdee8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829742700 1 -- 172.21.15.26:0/4073006924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdeeaf70 msgr2=0x5604fdee8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829742700 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdeeaf70 0x5604fdee8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829742700 1 -- 172.21.15.26:0/4073006924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 msgr2=0x5604fdee88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829742700 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdee88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829742700 1 -- 172.21.15.26:0/4073006924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fdeed5b0 con 0x5604fdee6870 2024-05-20T20:57:12.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f9829742700 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdef1550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9820000c00 tx=0x7f98200ae0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e1a77249ed7df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:12.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f980effd700 1 -- 172.21.15.26:0/4073006924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200a99a0 con 0x5604fdee6870 2024-05-20T20:57:12.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f980effd700 1 -- 172.21.15.26:0/4073006924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9820010e50 con 0x5604fdee6870 2024-05-20T20:57:12.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.571+0000 7f980effd700 1 -- 172.21.15.26:0/4073006924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200aa090 con 0x5604fdee6870 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 -- 172.21.15.26:0/4073006924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 msgr2=0x5604fdef1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdef1550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9820000c00 tx=0x7f98200ae0e0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 -- 172.21.15.26:0/4073006924 shutdown_connections 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdee6870 0x5604fdef1550 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdeeaf70 0x5604fdee8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 --2- 172.21.15.26:0/4073006924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee7220 0x5604fdee88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 -- 172.21.15.26:0/4073006924 >> 172.21.15.26:0/4073006924 conn(0x5604fde03080 msgr2=0x5604fdef0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:12.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 -- 172.21.15.26:0/4073006924 shutdown_connections 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.572+0000 7f983623cb80 1 -- 172.21.15.26:0/4073006924 wait complete. 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 -- start start 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 Processor -- start 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee6870 0x5604fdf06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdf06b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdf66840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98240705e0 con 0x5604fdeeaf70 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9824071a50 con 0x5604fdee6870 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9824070cb0 con 0x5604fdee7220 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.573+0000 7f983623cb80 10 client.0 _ll_register_callbacks cb 0x5604fddbf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdf66840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdf66840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53402/0 (socket says 172.21.15.26:53402) 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 -- 172.21.15.26:0/2290889688 learned_addr learned my addr 172.21.15.26:0/2290889688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9828f41700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdf06b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829742700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee6870 0x5604fdf06630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 -- 172.21.15.26:0/2290889688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 msgr2=0x5604fdf06b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdf06b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 -- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee6870 msgr2=0x5604fdf06630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee6870 0x5604fdf06630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 -- 172.21.15.26:0/2290889688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fdf66e80 con 0x5604fdeeaf70 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829742700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee6870 0x5604fdf06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9828f41700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdf06b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f9829f43700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdf66840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f981c0042e0 tx=0x7f981c009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5804995bcc91a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.574+0000 7f980cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f980cff9700 1 -- 172.21.15.26:0/2290889688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c012780 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f980cff9700 1 -- 172.21.15.26:0/2290889688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f981c022040 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9820010e50 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f980cff9700 1 -- 172.21.15.26:0/2290889688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f981c012920 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 10 client.15630 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604fdf6e210 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f980cff9700 1 -- 172.21.15.26:0/2290889688 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9820010e50 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f980cff9700 1 -- 172.21.15.26:0/2290889688 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f981c018040 con 0x5604fdeeaf70 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 20 client.15630 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf6e970 0x5604fdf70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 4 client.15630 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.575+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604fde36be0 con 0x5604fdf6e970 2024-05-20T20:57:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.576+0000 7f9829742700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf6e970 0x5604fdf70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.576+0000 7f980cff9700 10 client.15630 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.576+0000 7f9829742700 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf6e970 0x5604fdf70d90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2f28f6739bcd84 server_cookie=f38ddb18fe22ec30 in_seq=0 out_seq=0 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.576+0000 7f980cff9700 1 -- 172.21.15.26:0/2290889688 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604fde36be0 con 0x5604fdf6e970 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.576+0000 7f980cff9700 10 client.15630 handle_command_reply: tid=0 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 client.15630 shutdown 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 msgr2=0x5604fdf66840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdf66840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f981c0042e0 tx=0x7f981c009100 comp rx=0 tx=0).stop 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 shutdown_connections 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604fdf6e970 0x5604fdf70d90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604fdeeaf70 0x5604fdf66840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604fdee6870 0x5604fdf06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 --2- 172.21.15.26:0/2290889688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604fdee7220 0x5604fdf06b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 >> 172.21.15.26:0/2290889688 conn(0x5604fde03080 msgr2=0x5604fded4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f980cff9700 0 client.15630 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 shutdown_connections 2024-05-20T20:57:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 1 -- 172.21.15.26:0/2290889688 wait complete. 2024-05-20T20:57:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.577+0000 7f983623cb80 20 client.15630 trim_cache size 0 max 16384 2024-05-20T20:57:12.582 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98100b2750 msgr2=0x7f98100b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98100b2750 0x7f98100b4c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f981c009fd0 tx=0x7f981c00b040 comp rx=0 tx=0).stop 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 msgr2=0x7f982419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 0x7f982419f7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f982000e9a0 tx=0x7f982000ecb0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 shutdown_connections 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98100b2750 0x7f98100b4c10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9824108550 0x7f98241999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9824102470 0x7f982419f7f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 --2- 172.21.15.26:0/1190909744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9824102e30 0x7f982419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 >> 172.21.15.26:0/1190909744 conn(0x7f9824079480 msgr2=0x7f9824100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 shutdown_connections 2024-05-20T20:57:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:12.578+0000 7f982b9a6700 1 -- 172.21.15.26:0/1190909744 wait complete. 2024-05-20T20:57:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:12 smithi026 ceph-mon[31990]: pgmap v420: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-20T20:57:13.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:12 smithi055 ceph-mon[35213]: pgmap v420: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-20T20:57:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:12 smithi144 ceph-mon[35660]: pgmap v420: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-20T20:57:14.369 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:57:14.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:57:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:15.152 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:14 smithi026 ceph-mon[31990]: pgmap v421: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:15.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:14 smithi055 ceph-mon[35213]: pgmap v421: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:14 smithi144 ceph-mon[35660]: pgmap v421: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 -- 172.21.15.26:0/3867098221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102470 msgr2=0x7fbef41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 --2- 172.21.15.26:0/3867098221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102470 0x7fbef41028f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbee40099c0 tx=0x7fbee4009cd0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 -- 172.21.15.26:0/3867098221 shutdown_connections 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 --2- 172.21.15.26:0/3867098221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4102e30 0x7fbef410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 --2- 172.21.15.26:0/3867098221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef4108550 0x7fbef4108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 --2- 172.21.15.26:0/3867098221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102470 0x7fbef41028f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 -- 172.21.15.26:0/3867098221 >> 172.21.15.26:0/3867098221 conn(0x7fbef4079480 msgr2=0x7fbef4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 -- 172.21.15.26:0/3867098221 shutdown_connections 2024-05-20T20:57:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.392+0000 7fbefc019700 1 -- 172.21.15.26:0/3867098221 wait complete. 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 Processor -- start 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 -- start start 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef4102470 0x7fbef419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 0x7fbef419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4108550 0x7fbef41a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41124d0 con 0x7fbef4102e30 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4112350 con 0x7fbef4108550 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefc019700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4112650 con 0x7fbef4102470 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbef95b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 0x7fbef419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbef95b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 0x7fbef419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53422/0 (socket says 172.21.15.26:53422) 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbef95b4700 1 -- 172.21.15.26:0/2467157327 learned_addr learned my addr 172.21.15.26:0/2467157327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.393+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4108550 0x7fbef41a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef9db5700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef4102470 0x7fbef419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef95b4700 1 -- 172.21.15.26:0/2467157327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef4102470 msgr2=0x7fbef419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef95b4700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef4102470 0x7fbef419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef95b4700 1 -- 172.21.15.26:0/2467157327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4108550 msgr2=0x7fbef41a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef95b4700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4108550 0x7fbef41a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef95b4700 1 -- 172.21.15.26:0/2467157327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef41a5ed0 con 0x7fbef4102e30 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4108550 0x7fbef41a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbef95b4700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 0x7fbef419ae50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006010 tx=0x7fbee40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbeeaffd700 1 -- 172.21.15.26:0/2467157327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4026070 con 0x7fbef4102e30 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef41a60a0 con 0x7fbef4102e30 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbeeaffd700 1 -- 172.21.15.26:0/2467157327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbee400f460 con 0x7fbef4102e30 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbeeaffd700 1 -- 172.21.15.26:0/2467157327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4009e60 con 0x7fbef4102e30 2024-05-20T20:57:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.394+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef41a6580 con 0x7fbef4102e30 2024-05-20T20:57:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.398+0000 7fbeeaffd700 1 -- 172.21.15.26:0/2467157327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbee4029070 con 0x7fbef4102e30 2024-05-20T20:57:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.398+0000 7fbf068afb80 1 Processor -- start 2024-05-20T20:57:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.398+0000 7fbf068afb80 1 -- start start 2024-05-20T20:57:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7bdbf0 0x556b8c7be070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7be5b0 0x556b8c7ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbf068afb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556b8c71be70 con 0x556b8c7c2720 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbf068afb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556b8c70ba10 con 0x556b8c7bdbf0 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbf068afb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556b8c70fa50 con 0x556b8c7be5b0 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbeeaffd700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbee00b2880 0x7fbee00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53438/0 (socket says 172.21.15.26:53438) 2024-05-20T20:57:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 -- 172.21.15.26:0/4045128878 learned_addr learned my addr 172.21.15.26:0/4045128878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbef95b4700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7be5b0 0x556b8c7ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbee00b2880 0x7fbee00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbef9db5700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7bdbf0 0x556b8c7be070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbeeaffd700 1 -- 172.21.15.26:0/2467157327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbee40e2a30 con 0x7fbef4102e30 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 -- 172.21.15.26:0/4045128878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7be5b0 msgr2=0x556b8c7ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7be5b0 0x556b8c7ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 -- 172.21.15.26:0/4045128878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7bdbf0 msgr2=0x556b8c7be070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7bdbf0 0x556b8c7be070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbefa5b6700 1 -- 172.21.15.26:0/4045128878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c6b50b0 con 0x556b8c7c2720 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbef95b4700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7be5b0 0x556b8c7ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.399+0000 7fbef9db5700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7bdbf0 0x556b8c7be070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.400+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c2b00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbeec004560 tx=0x7fbeec004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8b60a2bcc079cc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.400+0000 7fbedffff700 1 -- 172.21.15.26:0/4045128878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec007700 con 0x556b8c7c2720 2024-05-20T20:57:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.400+0000 7fbedffff700 1 -- 172.21.15.26:0/4045128878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeec009380 con 0x556b8c7c2720 2024-05-20T20:57:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.400+0000 7fbedffff700 1 -- 172.21.15.26:0/4045128878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec01a730 con 0x556b8c7c2720 2024-05-20T20:57:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.400+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbee00b2880 0x7fbee00b4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbeec022000 tx=0x7fbeec01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 -- 172.21.15.26:0/4045128878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 msgr2=0x556b8c7c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c2b00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbeec004560 tx=0x7fbeec004870 comp rx=0 tx=0).stop 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 -- 172.21.15.26:0/4045128878 shutdown_connections 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c2b00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7be5b0 0x556b8c7ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 --2- 172.21.15.26:0/4045128878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7bdbf0 0x556b8c7be070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 -- 172.21.15.26:0/4045128878 >> 172.21.15.26:0/4045128878 conn(0x556b8c6e9340 msgr2=0x556b8c7b88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 -- 172.21.15.26:0/4045128878 shutdown_connections 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 -- 172.21.15.26:0/4045128878 wait complete. 2024-05-20T20:57:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 -- start start 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.401+0000 7fbf068afb80 1 Processor -- start 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556b8c6120a0 con 0x556b8c7c2720 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556b8c673ea0 con 0x556b8c7be5b0 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556b8c7cac80 con 0x556b8c7bdbf0 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbf068afb80 10 client.0 _ll_register_callbacks cb 0x556b8c6a6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbefa5b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbef95b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbefa5b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53442/0 (socket says 172.21.15.26:53442) 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbefa5b6700 1 -- 172.21.15.26:0/380488481 learned_addr learned my addr 172.21.15.26:0/380488481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.402+0000 7fbef9db5700 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbefa5b6700 1 -- 172.21.15.26:0/380488481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 msgr2=0x556b8c7ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbefa5b6700 1 -- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 msgr2=0x556b8c7c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbefa5b6700 1 -- 172.21.15.26:0/380488481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8c65b9d0 con 0x556b8c7c2720 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c7b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbeec007ef0 tx=0x7fbeec01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c820a4d58dcc43fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbedd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbedd7fa700 1 -- 172.21.15.26:0/380488481 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec02f070 con 0x556b8c7c2720 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbedd7fa700 1 -- 172.21.15.26:0/380488481 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbeec007910 con 0x556b8c7c2720 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.403+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeec009380 con 0x556b8c7c2720 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbedd7fa700 1 -- 172.21.15.26:0/380488481 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeec03b050 con 0x556b8c7c2720 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 10 client.15642 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b8c6c5860 con 0x556b8c7c2720 2024-05-20T20:57:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbedd7fa700 1 -- 172.21.15.26:0/380488481 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeec009380 con 0x556b8c7c2720 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbedd7fa700 1 -- 172.21.15.26:0/380488481 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbeec02a070 con 0x556b8c7c2720 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 10 client.15642 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 20 client.15642 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c7d76c0 0x556b8c7d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 4 client.15642 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8c71ddc0 con 0x556b8c7d76c0 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.404+0000 7fbef9db5700 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c7d76c0 0x556b8c7d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.405+0000 7fbef9db5700 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c7d76c0 0x556b8c7d9ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8812a61ee2b4c1 server_cookie=8d4c9ce10c6ed885 in_seq=0 out_seq=0 2024-05-20T20:57:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.405+0000 7fbedd7fa700 10 client.15642 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbedd7fa700 1 -- 172.21.15.26:0/380488481 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b8c71ddc0 con 0x556b8c7d76c0 2024-05-20T20:57:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbedd7fa700 10 client.15642 handle_command_reply: tid=0 2024-05-20T20:57:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 client.15642 shutdown 2024-05-20T20:57:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 msgr2=0x556b8c7c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c7b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbeec007ef0 tx=0x7fbeec01ac80 comp rx=0 tx=0).stop 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 shutdown_connections 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c7d76c0 0x556b8c7d9ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c7b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbedd7fa700 0 client.15642 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 --2- 172.21.15.26:0/380488481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 >> 172.21.15.26:0/380488481 conn(0x556b8c6e9340 msgr2=0x556b8c7bcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 shutdown_connections 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 1 -- 172.21.15.26:0/380488481 wait complete. 2024-05-20T20:57:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.406+0000 7fbf068afb80 20 client.15642 trim_cache size 0 max 16384 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 Processor -- start 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 -- start start 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7bd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41ae2a0 con 0x556b8c7c2720 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41abb30 con 0x556b8c7be5b0 2024-05-20T20:57:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbf068afb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41a99a0 con 0x556b8c7bdbf0 2024-05-20T20:57:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbefa5b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbefa5b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53458/0 (socket says 172.21.15.26:53458) 2024-05-20T20:57:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbefa5b6700 1 -- 172.21.15.26:0/1488032686 learned_addr learned my addr 172.21.15.26:0/1488032686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbef95b4700 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.427+0000 7fbef9db5700 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbefa5b6700 1 -- 172.21.15.26:0/1488032686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 msgr2=0x556b8c7bd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7bd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbefa5b6700 1 -- 172.21.15.26:0/1488032686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 msgr2=0x556b8c7b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbefa5b6700 1 -- 172.21.15.26:0/1488032686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c7c5b80 con 0x556b8c7c2720 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbef95b4700 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c1e80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbeec007d90 tx=0x7fbeec004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7fc8122052b254 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbedf7fe700 1 -- 172.21.15.26:0/1488032686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec02f040 con 0x556b8c7c2720 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbedf7fe700 1 -- 172.21.15.26:0/1488032686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeec0049f0 con 0x556b8c7c2720 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.428+0000 7fbedf7fe700 1 -- 172.21.15.26:0/1488032686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec022c10 con 0x556b8c7c2720 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 -- 172.21.15.26:0/1488032686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 msgr2=0x556b8c7c1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c1e80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbeec007d90 tx=0x7fbeec004870 comp rx=0 tx=0).stop 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 -- 172.21.15.26:0/1488032686 shutdown_connections 2024-05-20T20:57:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7c2720 0x556b8c7c1e80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7bd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 --2- 172.21.15.26:0/1488032686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7be5b0 0x556b8c7b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 -- 172.21.15.26:0/1488032686 >> 172.21.15.26:0/1488032686 conn(0x556b8c6e9340 msgr2=0x556b8c7c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 -- 172.21.15.26:0/1488032686 shutdown_connections 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 -- 172.21.15.26:0/1488032686 wait complete. 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 -- start start 2024-05-20T20:57:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.429+0000 7fbf068afb80 1 Processor -- start 2024-05-20T20:57:15.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 0x556b8c7ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7c2720 0x556b8c849820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41a9ff0 con 0x556b8c7be5b0 2024-05-20T20:57:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41ab460 con 0x556b8c7c2720 2024-05-20T20:57:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41aa6c0 con 0x556b8c7bdbf0 2024-05-20T20:57:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbf068afb80 10 client.0 _ll_register_callbacks cb 0x556b8c6a6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 0x556b8c7ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 0x556b8c7ce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53472/0 (socket says 172.21.15.26:53472) 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 -- 172.21.15.26:0/3978285301 learned_addr learned my addr 172.21.15.26:0/3978285301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef9db5700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbefa5b6700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7c2720 0x556b8c849820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 -- 172.21.15.26:0/3978285301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 msgr2=0x556b8c7ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 -- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7c2720 msgr2=0x556b8c849820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7c2720 0x556b8c849820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.430+0000 7fbef95b4700 1 -- 172.21.15.26:0/3978285301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8c849d60 con 0x556b8c7be5b0 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbef95b4700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 0x556b8c7ce610 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbee40aa010 tx=0x7fbee4073970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b988fe42bbcfa84 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbedd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbedd7fa700 1 -- 172.21.15.26:0/3978285301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40a9b90 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbedd7fa700 1 -- 172.21.15.26:0/3978285301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbee40aa2e0 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b8c7175a0 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbedd7fa700 1 -- 172.21.15.26:0/3978285301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbee40aa780 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 10 client.15648 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b8c84afe0 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbedd7fa700 1 -- 172.21.15.26:0/3978285301 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556b8c7175a0 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbedd7fa700 1 -- 172.21.15.26:0/3978285301 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbee40b29c0 con 0x556b8c7be5b0 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 10 client.15648 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 20 client.15648 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.431+0000 7fbf068afb80 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c84b740 0x556b8c84db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.432+0000 7fbf068afb80 4 client.15648 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "621f3813", "release": true, "await": true}] 2024-05-20T20:57:15.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.432+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "621f3813", "release": true, "await": true}) -- 0x556b8c58e350 con 0x556b8c84b740 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.432+0000 7fbef9db5700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c84b740 0x556b8c84db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.432+0000 7fbedd7fa700 10 client.15648 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.432+0000 7fbef9db5700 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c84b740 0x556b8c84db60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fc26460c7a31a5 server_cookie=81763baf6cba1dec in_seq=0 out_seq=0 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbedd7fa700 1 -- 172.21.15.26:0/3978285301 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556b8c58e350 con 0x556b8c84b740 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbedd7fa700 10 client.15648 handle_command_reply: tid=0 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 client.15648 shutdown 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 msgr2=0x556b8c7ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 0x556b8c7ce610 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbee40aa010 tx=0x7fbee4073970 comp rx=0 tx=0).stop 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 shutdown_connections 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556b8c84b740 0x556b8c84db60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b8c7c2720 0x556b8c849820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b8c7bdbf0 0x556b8c7ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 --2- 172.21.15.26:0/3978285301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b8c7be5b0 0x556b8c7ce610 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbedd7fa700 0 client.15648 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 >> 172.21.15.26:0/3978285301 conn(0x556b8c6e9340 msgr2=0x556b8c7c6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 shutdown_connections 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.434+0000 7fbf068afb80 1 -- 172.21.15.26:0/3978285301 wait complete. 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbf068afb80 20 client.15648 trim_cache size 0 max 16384 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:15.440 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 4, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "621f3813": { 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 4, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 138.8, 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:57:15.441 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:15.442 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbee00b2880 msgr2=0x7fbee00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbee00b2880 0x7fbee00b4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbeec022000 tx=0x7fbeec01e000 comp rx=0 tx=0).stop 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 msgr2=0x7fbef419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 0x7fbef419ae50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006010 tx=0x7fbee40055f0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 shutdown_connections 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbef4108550 0x7fbef41a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef4102470 0x7fbef419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbee00b2880 0x7fbee00b4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 --2- 172.21.15.26:0/2467157327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef4102e30 0x7fbef419ae50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 >> 172.21.15.26:0/2467157327 conn(0x7fbef4079480 msgr2=0x7fbef4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 shutdown_connections 2024-05-20T20:57:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:15.435+0000 7fbefc019700 1 -- 172.21.15.26:0/2467157327 wait complete. 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 -- 172.21.15.26:0/3860909852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 msgr2=0x7fa238103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3860909852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa238103020 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a410 comp rx=0 tx=0).stop 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 -- 172.21.15.26:0/3860909852 shutdown_connections 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3860909852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa238103560 0x7fa238109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3860909852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa238103020 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3860909852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2381019b0 0x7fa238101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 -- 172.21.15.26:0/3860909852 >> 172.21.15.26:0/3860909852 conn(0x7fa2380fcf20 msgr2=0x7fa2380ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 -- 172.21.15.26:0/3860909852 shutdown_connections 2024-05-20T20:57:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.735+0000 7fa23ce72640 1 -- 172.21.15.26:0/3860909852 wait complete. 2024-05-20T20:57:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 Processor -- start 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 -- start start 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2381019b0 0x7fa23806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa23819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa238103560 0x7fa23819ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa23810d390 con 0x7fa238102ba0 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa23810d210 con 0x7fa238103560 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23ce72640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa23810d510 con 0x7fa2381019b0 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa23819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa23819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53486/0 (socket says 172.21.15.26:53486) 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa22ffff640 1 -- 172.21.15.26:0/3477150090 learned_addr learned my addr 172.21.15.26:0/3477150090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa23659c640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2381019b0 0x7fa23806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.736+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa238103560 0x7fa23819ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22ffff640 1 -- 172.21.15.26:0/3477150090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2381019b0 msgr2=0x7fa23806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22ffff640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2381019b0 0x7fa23806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22ffff640 1 -- 172.21.15.26:0/3477150090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa238103560 msgr2=0x7fa23819ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22ffff640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa238103560 0x7fa23819ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22ffff640 1 -- 172.21.15.26:0/3477150090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2381a07e0 con 0x7fa238102ba0 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22ffff640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa23819d8e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa220009a20 tx=0x7fa2200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22f7fe640 1 -- 172.21.15.26:0/3477150090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22003f030 con 0x7fa238102ba0 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2381a0a70 con 0x7fa238102ba0 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a0fb0 con 0x7fa238102ba0 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22f7fe640 1 -- 172.21.15.26:0/3477150090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa220003a80 con 0x7fa238102ba0 2024-05-20T20:57:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.737+0000 7fa22f7fe640 1 -- 172.21.15.26:0/3477150090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220049060 con 0x7fa238102ba0 2024-05-20T20:57:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.739+0000 7fa22f7fe640 1 -- 172.21.15.26:0/3477150090 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa220042030 con 0x7fa238102ba0 2024-05-20T20:57:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.740+0000 7fa22f7fe640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa2240b28a0 0x7fa2240b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.740+0000 7fa23659c640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa2240b28a0 0x7fa2240b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.740+0000 7fa22f7fe640 1 -- 172.21.15.26:0/3477150090 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa220102430 con 0x7fa238102ba0 2024-05-20T20:57:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.740+0000 7fa23fb8d740 1 Processor -- start 2024-05-20T20:57:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 -- start start 2024-05-20T20:57:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa2240b28a0 0x7fa2240b4d60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa238102a00 tx=0x7fa22800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3373bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e3374220 0x55d0e33766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d0e32dfc30 con 0x55d0e336e6c0 2024-05-20T20:57:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d0e32cf7d0 con 0x55d0e336f0b0 2024-05-20T20:57:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0e32d3810 con 0x55d0e3374220 2024-05-20T20:57:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53502/0 (socket says 172.21.15.26:53502) 2024-05-20T20:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 -- 172.21.15.26:0/1530309863 learned_addr learned my addr 172.21.15.26:0/1530309863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa22ffff640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3373bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 -- 172.21.15.26:0/1530309863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e3374220 msgr2=0x55d0e33766b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa236d9d640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e3374220 0x55d0e33766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e3374220 0x55d0e33766b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 -- 172.21.15.26:0/1530309863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 msgr2=0x55d0e3373bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3373bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa23659c640 1 -- 172.21.15.26:0/1530309863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e2fe4210 con 0x55d0e336e6c0 2024-05-20T20:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa22ffff640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3373bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.741+0000 7fa236d9d640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e3374220 0x55d0e33766b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.742+0000 7fa23659c640 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336eae0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa2280058a0 tx=0x7fa228013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a593560641a7e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.742+0000 7fa22cff9640 1 -- 172.21.15.26:0/1530309863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228004220 con 0x55d0e336e6c0 2024-05-20T20:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.742+0000 7fa22cff9640 1 -- 172.21.15.26:0/1530309863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2280043c0 con 0x55d0e336e6c0 2024-05-20T20:57:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.742+0000 7fa22cff9640 1 -- 172.21.15.26:0/1530309863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22801e7e0 con 0x55d0e336e6c0 2024-05-20T20:57:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 -- 172.21.15.26:0/1530309863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 msgr2=0x55d0e336eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336eae0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa2280058a0 tx=0x7fa228013000 comp rx=0 tx=0).stop 2024-05-20T20:57:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 -- 172.21.15.26:0/1530309863 shutdown_connections 2024-05-20T20:57:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e3374220 0x55d0e33766b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3373bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/1530309863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336eae0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 -- 172.21.15.26:0/1530309863 >> 172.21.15.26:0/1530309863 conn(0x55d0e3369d30 msgr2=0x55d0e336a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 -- 172.21.15.26:0/1530309863 shutdown_connections 2024-05-20T20:57:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 -- 172.21.15.26:0/1530309863 wait complete. 2024-05-20T20:57:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 -- start start 2024-05-20T20:57:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.743+0000 7fa23fb8d740 1 Processor -- start 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e3378780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3378cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2fb5790 con 0x55d0e3374220 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d0e328b890 con 0x55d0e336f0b0 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0e33771d0 con 0x55d0e336e6c0 2024-05-20T20:57:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23fb8d740 10 client.0 _ll_register_callbacks cb 0x55d0e2fe3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53506/0 (socket says 172.21.15.26:53506) 2024-05-20T20:57:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 -- 172.21.15.26:0/3771566104 learned_addr learned my addr 172.21.15.26:0/3771566104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa22ffff640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3378cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa23659c640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e3378780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 -- 172.21.15.26:0/3771566104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 msgr2=0x55d0e3378780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e3378780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 -- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 msgr2=0x55d0e3378cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3378cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 -- 172.21.15.26:0/3771566104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e30b0620 con 0x55d0e3374220 2024-05-20T20:57:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337fd00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa230012d20 tx=0x7fa2300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9a6416fbd9119d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.744+0000 7fa21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa21dffb640 1 -- 172.21.15.26:0/3771566104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23001f060 con 0x55d0e3374220 2024-05-20T20:57:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa21dffb640 1 -- 172.21.15.26:0/3771566104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa230003b70 con 0x55d0e3374220 2024-05-20T20:57:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2280043c0 con 0x55d0e3374220 2024-05-20T20:57:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa21dffb640 1 -- 172.21.15.26:0/3771566104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa230010e90 con 0x55d0e3374220 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 10 client.15660 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0e3382d20 con 0x55d0e3374220 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa21dffb640 1 -- 172.21.15.26:0/3771566104 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2280043c0 con 0x55d0e3374220 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa21dffb640 1 -- 172.21.15.26:0/3771566104 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa230023030 con 0x55d0e3374220 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 20 client.15660 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33844d0 0x55d0e3386900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 4 client.15660 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.745+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e32e1b80 con 0x55d0e33844d0 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.746+0000 7fa23659c640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33844d0 0x55d0e3386900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.746+0000 7fa23659c640 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33844d0 0x55d0e3386900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e2dee655a46931a server_cookie=2bc07aefa48cc7e8 in_seq=0 out_seq=0 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.746+0000 7fa21dffb640 10 client.15660 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa21dffb640 1 -- 172.21.15.26:0/3771566104 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d0e32e1b80 con 0x55d0e33844d0 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa21dffb640 10 client.15660 handle_command_reply: tid=0 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 client.15660 shutdown 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 msgr2=0x55d0e337fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337fd00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa230012d20 tx=0x7fa2300107a0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 shutdown_connections 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33844d0 0x55d0e3386900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337fd00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e3378cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa21dffb640 0 client.15660 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3771566104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e3378780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 >> 172.21.15.26:0/3771566104 conn(0x55d0e3369d30 msgr2=0x55d0e3368fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 shutdown_connections 2024-05-20T20:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3771566104 wait complete. 2024-05-20T20:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.747+0000 7fa23fb8d740 20 client.15660 trim_cache size 0 max 16384 2024-05-20T20:57:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 Processor -- start 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 -- start start 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336f0b0 0x55d0e33775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e3374220 0x55d0e3377b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa218010ef0 con 0x55d0e336e6c0 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa21800e7c0 con 0x55d0e3374220 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa21800c610 con 0x55d0e336f0b0 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53516/0 (socket says 172.21.15.26:53516) 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 -- 172.21.15.26:0/2465190152 learned_addr learned my addr 172.21.15.26:0/2465190152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa22ffff640 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336f0b0 0x55d0e33775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa236d9d640 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e3374220 0x55d0e3377b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 -- 172.21.15.26:0/2465190152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336f0b0 msgr2=0x55d0e33775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336f0b0 0x55d0e33775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 -- 172.21.15.26:0/2465190152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e3374220 msgr2=0x55d0e3377b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e3374220 0x55d0e3377b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.766+0000 7fa23659c640 1 -- 172.21.15.26:0/2465190152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e336f940 con 0x55d0e336e6c0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.767+0000 7fa23659c640 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336d320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa228004610 tx=0x7fa2280058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc8362165948203 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.767+0000 7fa21ffff640 1 -- 172.21.15.26:0/2465190152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228010ca0 con 0x55d0e336e6c0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.767+0000 7fa21ffff640 1 -- 172.21.15.26:0/2465190152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa228004270 con 0x55d0e336e6c0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.767+0000 7fa21ffff640 1 -- 172.21.15.26:0/2465190152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280356c0 con 0x55d0e336e6c0 2024-05-20T20:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- 172.21.15.26:0/2465190152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 msgr2=0x55d0e336d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336d320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa228004610 tx=0x7fa2280058d0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- 172.21.15.26:0/2465190152 shutdown_connections 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e3374220 0x55d0e3377b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336f0b0 0x55d0e33775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/2465190152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e336e6c0 0x55d0e336d320 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- 172.21.15.26:0/2465190152 >> 172.21.15.26:0/2465190152 conn(0x55d0e3369d30 msgr2=0x55d0e336cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- 172.21.15.26:0/2465190152 shutdown_connections 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- 172.21.15.26:0/2465190152 wait complete. 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- start start 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 Processor -- start 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e33ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e33ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa21800cc80 con 0x55d0e3374220 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa21800e0f0 con 0x55d0e336f0b0 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa21800d350 con 0x55d0e336e6c0 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa23fb8d740 10 client.0 _ll_register_callbacks cb 0x7fa230009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.768+0000 7fa236d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53522/0 (socket says 172.21.15.26:53522) 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 -- 172.21.15.26:0/3356027944 learned_addr learned my addr 172.21.15.26:0/3356027944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa23659c640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e33ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa22ffff640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e33ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 -- 172.21.15.26:0/3356027944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 msgr2=0x55d0e33ebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e33ebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 -- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 msgr2=0x55d0e33ec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e33ec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 -- 172.21.15.26:0/3356027944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e337e450 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa236d9d640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337de10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa230009b00 tx=0x7fa230004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7275a8774d020d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa21dffb640 1 -- 172.21.15.26:0/3356027944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23000ec20 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa21dffb640 1 -- 172.21.15.26:0/3356027944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa23000edc0 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa228004270 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa21dffb640 1 -- 172.21.15.26:0/3356027944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa230010e90 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa23fb8d740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.769+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0e337f720 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa21dffb640 1 -- 172.21.15.26:0/3356027944 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa228004270 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa21dffb640 1 -- 172.21.15.26:0/3356027944 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa230035020 con 0x55d0e3374220 2024-05-20T20:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23fb8d740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23fb8d740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23fb8d740 20 client.15666 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33f3b90 0x55d0e33f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23fb8d740 4 client.15666 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0e32db360 con 0x55d0e33f3b90 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23659c640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33f3b90 0x55d0e33f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa23659c640 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33f3b90 0x55d0e33f5fb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e9348e8d5371f server_cookie=9bbc7050f1f455b4 in_seq=0 out_seq=0 2024-05-20T20:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.770+0000 7fa21dffb640 10 client.15666 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa21dffb640 1 -- 172.21.15.26:0/3356027944 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d0e32db360 con 0x55d0e33f3b90 2024-05-20T20:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa21dffb640 10 client.15666 handle_command_reply: tid=0 2024-05-20T20:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 client.15666 shutdown 2024-05-20T20:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 msgr2=0x55d0e337de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337de10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa230009b00 tx=0x7fa230004f20 comp rx=0 tx=0).stop 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 shutdown_connections 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0e33f3b90 0x55d0e33f5fb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa21dffb640 0 client.15666 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0e3374220 0x55d0e337de10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0e336f0b0 0x55d0e33ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 --2- 172.21.15.26:0/3356027944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e336e6c0 0x55d0e33ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.776+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 >> 172.21.15.26:0/3356027944 conn(0x55d0e3369d30 msgr2=0x55d0e336d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 shutdown_connections 2024-05-20T20:57:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23fb8d740 1 -- 172.21.15.26:0/3356027944 wait complete. 2024-05-20T20:57:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23fb8d740 20 client.15666 trim_cache size 0 max 16384 2024-05-20T20:57:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:57:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:57:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa2240b28a0 msgr2=0x7fa2240b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa2240b28a0 0x7fa2240b4d60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa238102a00 tx=0x7fa22800b390 comp rx=0 tx=0).stop 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 msgr2=0x7fa23819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa23819d8e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa220009a20 tx=0x7fa2200040c0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 shutdown_connections 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa2240b28a0 0x7fa2240b4d60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa238103560 0x7fa23819ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa238102ba0 0x7fa23819d8e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 --2- 172.21.15.26:0/3477150090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2381019b0 0x7fa23806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 >> 172.21.15.26:0/3477150090 conn(0x7fa2380fcf20 msgr2=0x7fa238105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 shutdown_connections 2024-05-20T20:57:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:16.777+0000 7fa23ce72640 1 -- 172.21.15.26:0/3477150090 wait complete. 2024-05-20T20:57:17.136 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:16 smithi026 ceph-mon[31990]: pgmap v422: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:17.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:16 smithi055 ceph-mon[35213]: pgmap v422: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:17.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:16 smithi144 ceph-mon[35660]: pgmap v422: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T20:57:17.325 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '621f3813', total seconds elapsed: 29.1 2024-05-20T20:57:17.325 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-05-20T20:57:17.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:57:17.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 -- 172.21.15.26:0/1609567900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 msgr2=0x7fe108103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/1609567900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe108103a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009920 tx=0x7fe0f8009c30 comp rx=0 tx=0).stop 2024-05-20T20:57:17.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 -- 172.21.15.26:0/1609567900 shutdown_connections 2024-05-20T20:57:17.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/1609567900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe108103fa0 0x7fe108110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/1609567900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108109680 0x7fe108109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/1609567900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe108103a60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 -- 172.21.15.26:0/1609567900 >> 172.21.15.26:0/1609567900 conn(0x7fe10807a6f0 msgr2=0x7fe10807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 -- 172.21.15.26:0/1609567900 shutdown_connections 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.880+0000 7fe10f2b2700 1 -- 172.21.15.26:0/1609567900 wait complete. 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.881+0000 7fe10f2b2700 1 Processor -- start 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.881+0000 7fe10f2b2700 1 -- start start 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.881+0000 7fe10f2b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe10819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.881+0000 7fe10f2b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108103fa0 0x7fe10819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.881+0000 7fe10f2b2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe108109680 0x7fe1081976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10f2b2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe108111480 con 0x7fe108109680 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10f2b2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe108111300 con 0x7fe1081035e0 2024-05-20T20:57:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe10819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10c84d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108103fa0 0x7fe10819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe10819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:56740/0 (socket says 172.21.15.26:56740) 2024-05-20T20:57:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 -- 172.21.15.26:0/3022512487 learned_addr learned my addr 172.21.15.26:0/3022512487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d84f700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe108109680 0x7fe1081976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe108111600 con 0x7fe108103fa0 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 -- 172.21.15.26:0/3022512487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108103fa0 msgr2=0x7fe10819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108103fa0 0x7fe10819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 -- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe108109680 msgr2=0x7fe1081976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe108109680 0x7fe1081976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.882+0000 7fe10d04e700 1 -- 172.21.15.26:0/3022512487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe108197f70 con 0x7fe1081035e0 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.883+0000 7fe10d04e700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe10819e530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e9a0 tx=0x7fe10400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.883+0000 7fe0fe7fc700 1 -- 172.21.15.26:0/3022512487 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10400ca10 con 0x7fe1081035e0 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.883+0000 7fe0fe7fc700 1 -- 172.21.15.26:0/3022512487 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe10400cbb0 con 0x7fe1081035e0 2024-05-20T20:57:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.883+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe108198260 con 0x7fe1081035e0 2024-05-20T20:57:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.884+0000 7fe0fe7fc700 1 -- 172.21.15.26:0/3022512487 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104018680 con 0x7fe1081035e0 2024-05-20T20:57:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.884+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10819c230 con 0x7fe1081035e0 2024-05-20T20:57:17.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.888+0000 7fe0fe7fc700 1 -- 172.21.15.26:0/3022512487 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe104010430 con 0x7fe1081035e0 2024-05-20T20:57:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe0fe7fc700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0f40b2900 0x7fe0f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe119b48b80 1 Processor -- start 2024-05-20T20:57:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe119b48b80 1 -- start start 2024-05-20T20:57:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe0fe7fc700 1 -- 172.21.15.26:0/3022512487 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe1040e2410 con 0x7fe1081035e0 2024-05-20T20:57:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe10c84d700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0f40b2900 0x7fe0f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c7809c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c78096920 0x557c78096da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.889+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c780972e0 0x557c780a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe119b48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557c77ff4df0 con 0x557c7809c3a0 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c7809c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe119b48b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557c77fe4990 con 0x557c78096920 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c7809c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53538/0 (socket says 172.21.15.26:53538) 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d04e700 1 -- 172.21.15.26:0/2572851108 learned_addr learned my addr 172.21.15.26:0/2572851108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10c84d700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0f40b2900 0x7fe0f40b4dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009680 tx=0x7fe0f8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe119b48b80 1 -- 172.21.15.26:0/2572851108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c77fe89d0 con 0x557c780972e0 2024-05-20T20:57:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d84f700 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c780972e0 0x557c780a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c78096920 0x557c78096da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d84f700 1 -- 172.21.15.26:0/2572851108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c78096920 msgr2=0x557c78096da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d84f700 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c78096920 0x557c78096da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d84f700 1 -- 172.21.15.26:0/2572851108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 msgr2=0x557c7809c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d84f700 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c7809c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.890+0000 7fe10d84f700 1 -- 172.21.15.26:0/2572851108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c77f8e030 con 0x557c780972e0 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.891+0000 7fe10d84f700 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c780972e0 0x557c780a3100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe100009cd0 tx=0x7fe1000099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276e7059e96b33c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.891+0000 7fe0f37fe700 1 -- 172.21.15.26:0/2572851108 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100003a00 con 0x557c780972e0 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.891+0000 7fe0f37fe700 1 -- 172.21.15.26:0/2572851108 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe100003ba0 con 0x557c780972e0 2024-05-20T20:57:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.891+0000 7fe0f37fe700 1 -- 172.21.15.26:0/2572851108 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1000048f0 con 0x557c780972e0 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 -- 172.21.15.26:0/2572851108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c780972e0 msgr2=0x557c780a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c780972e0 0x557c780a3100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe100009cd0 tx=0x7fe1000099e0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 -- 172.21.15.26:0/2572851108 shutdown_connections 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c7809c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c780972e0 0x557c780a3100 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2572851108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c78096920 0x557c78096da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 -- 172.21.15.26:0/2572851108 >> 172.21.15.26:0/2572851108 conn(0x557c77fc22c0 msgr2=0x557c78091460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 -- 172.21.15.26:0/2572851108 shutdown_connections 2024-05-20T20:57:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 -- 172.21.15.26:0/2572851108 wait complete. 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.892+0000 7fe119b48b80 1 -- start start 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 Processor -- start 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe10c84d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe10c84d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53550/0 (socket says 172.21.15.26:53550) 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe10c84d700 1 -- 172.21.15.26:0/2332400030 learned_addr learned my addr 172.21.15.26:0/2332400030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557c77eeb020 con 0x557c780972e0 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557c77f4ce20 con 0x557c7809c3a0 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c780a3760 con 0x557c78096920 2024-05-20T20:57:17.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe119b48b80 10 client.0 _ll_register_callbacks cb 0x557c77f7f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe10d84f700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.893+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10c84d700 1 -- 172.21.15.26:0/2332400030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 msgr2=0x557c780a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10c84d700 1 -- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 msgr2=0x557c780a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10c84d700 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c77f34950 con 0x557c780972e0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a00e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009920 tx=0x7fe0f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b94de83a70f43e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe0f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2332400030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f802f440 con 0x557c780972e0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.894+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2332400030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe0f802f5e0 con 0x557c780972e0 2024-05-20T20:57:17.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe100003ba0 con 0x557c780972e0 2024-05-20T20:57:17.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2332400030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0f802f7e0 con 0x557c780972e0 2024-05-20T20:57:17.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe119b48b80 10 client.15678 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:17.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c77f9e7e0 con 0x557c780972e0 2024-05-20T20:57:17.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2332400030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe100003ba0 con 0x557c780972e0 2024-05-20T20:57:17.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2332400030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0f80237a0 con 0x557c780972e0 2024-05-20T20:57:17.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe119b48b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:17.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.895+0000 7fe119b48b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:17.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.896+0000 7fe119b48b80 20 client.15678 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.896+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c780b0340 0x557c780b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.896+0000 7fe119b48b80 4 client.15678 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.896+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c77ff6d40 con 0x557c780b0340 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.896+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c780b0340 0x557c780b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.896+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c780b0340 0x557c780b2750 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ed1cf79a5e4f3d server_cookie=50484b943393edba in_seq=0 out_seq=0 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.897+0000 7fe0f0ff9700 10 client.15678 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.897+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2332400030 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557c77ff6d40 con 0x557c780b0340 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.897+0000 7fe0f0ff9700 10 client.15678 handle_command_reply: tid=0 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.897+0000 7fe119b48b80 1 client.15678 shutdown 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 msgr2=0x557c780a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a00e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009920 tx=0x7fe0f8000bc0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 shutdown_connections 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c780b0340 0x557c780b2750 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2332400030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a00e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 >> 172.21.15.26:0/2332400030 conn(0x557c77fc22c0 msgr2=0x557c78090b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe0f0ff9700 0 client.15678 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 shutdown_connections 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 1 -- 172.21.15.26:0/2332400030 wait complete. 2024-05-20T20:57:17.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.898+0000 7fe119b48b80 20 client.15678 trim_cache size 0 max 16384 2024-05-20T20:57:17.922 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 Processor -- start 2024-05-20T20:57:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 -- start start 2024-05-20T20:57:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c780972e0 0x557c78096020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c78098dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081bf230 con 0x557c7809c3a0 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081bcac0 con 0x557c780972e0 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.925+0000 7fe119b48b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081ba910 con 0x557c78096920 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d84f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c78098dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d84f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c78098dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53558/0 (socket says 172.21.15.26:53558) 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d84f700 1 -- 172.21.15.26:0/1861047796 learned_addr learned my addr 172.21.15.26:0/1861047796 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10c84d700 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c780972e0 0x557c78096020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 -- 172.21.15.26:0/1861047796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c780972e0 msgr2=0x557c78096020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c780972e0 0x557c78096020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 -- 172.21.15.26:0/1861047796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 msgr2=0x557c78098dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c78098dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 -- 172.21.15.26:0/1861047796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c780a35c0 con 0x557c78096920 2024-05-20T20:57:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.926+0000 7fe10d04e700 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780912f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe1040a59f0 tx=0x7fe1040a5960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9fb72524e32c3d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.927+0000 7fe0f2ffd700 1 -- 172.21.15.26:0/1861047796 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040a9970 con 0x557c78096920 2024-05-20T20:57:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.927+0000 7fe0f2ffd700 1 -- 172.21.15.26:0/1861047796 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe10408f070 con 0x557c78096920 2024-05-20T20:57:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.927+0000 7fe0f2ffd700 1 -- 172.21.15.26:0/1861047796 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040a9bd0 con 0x557c78096920 2024-05-20T20:57:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.927+0000 7fe119b48b80 1 -- 172.21.15.26:0/1861047796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 msgr2=0x557c780912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.927+0000 7fe119b48b80 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780912f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe1040a59f0 tx=0x7fe1040a5960 comp rx=0 tx=0).stop 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 -- 172.21.15.26:0/1861047796 shutdown_connections 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c7809c3a0 0x557c78098dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780912f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 --2- 172.21.15.26:0/1861047796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c780972e0 0x557c78096020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 -- 172.21.15.26:0/1861047796 >> 172.21.15.26:0/1861047796 conn(0x557c77fc22c0 msgr2=0x557c78095910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 -- 172.21.15.26:0/1861047796 shutdown_connections 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 -- 172.21.15.26:0/1861047796 wait complete. 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.928+0000 7fe119b48b80 1 -- start start 2024-05-20T20:57:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 Processor -- start 2024-05-20T20:57:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081baf80 con 0x557c780972e0 2024-05-20T20:57:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081bc3f0 con 0x557c7809c3a0 2024-05-20T20:57:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081bb650 con 0x557c78096920 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe119b48b80 10 client.0 _ll_register_callbacks cb 0x557c77f7f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.929+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38648/0 (socket says 172.21.15.26:38648) 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 -- 172.21.15.26:0/2636464000 learned_addr learned my addr 172.21.15.26:0/2636464000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d84f700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 -- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 msgr2=0x557c780a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 -- 172.21.15.26:0/2636464000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 msgr2=0x557c780a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 -- 172.21.15.26:0/2636464000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c780a41e0 con 0x557c78096920 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe10d04e700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a8bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1040098c0 tx=0x7fe10400c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df18600ff63769f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe0f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:57:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.930+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2636464000 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040add20 con 0x557c78096920 2024-05-20T20:57:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.931+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10408f070 con 0x557c78096920 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.931+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2636464000 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe1040a5f60 con 0x557c78096920 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.931+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2636464000 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe1040aa070 con 0x557c78096920 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.931+0000 7fe119b48b80 10 client.25228 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.931+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c7811ba30 con 0x557c78096920 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.931+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2636464000 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe10408f070 con 0x557c78096920 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2636464000 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe10406f120 con 0x557c78096920 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe119b48b80 10 client.25228 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe119b48b80 10 client.25228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe119b48b80 20 client.25228 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c7811bf90 0x557c7811e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe119b48b80 4 client.25228 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:57:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.932+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557c77ff0520 con 0x557c7811bf90 2024-05-20T20:57:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.933+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c7811bf90 0x557c7811e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.934+0000 7fe10c84d700 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c7811bf90 0x557c7811e3b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780ac16834162516 server_cookie=64f446992ebe6306 in_seq=0 out_seq=0 2024-05-20T20:57:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.934+0000 7fe0f0ff9700 10 client.25228 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.935+0000 7fe0f0ff9700 1 -- 172.21.15.26:0/2636464000 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557c77ff0520 con 0x557c7811bf90 2024-05-20T20:57:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.935+0000 7fe0f0ff9700 10 client.25228 handle_command_reply: tid=0 2024-05-20T20:57:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.935+0000 7fe119b48b80 1 client.25228 shutdown 2024-05-20T20:57:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 msgr2=0x557c780a8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a8bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1040098c0 tx=0x7fe10400c400 comp rx=0 tx=0).stop 2024-05-20T20:57:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 shutdown_connections 2024-05-20T20:57:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557c7811bf90 0x557c7811e3b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557c7809c3a0 0x557c780a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c78096920 0x557c780a8bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 --2- 172.21.15.26:0/2636464000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c780972e0 0x557c780a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 >> 172.21.15.26:0/2636464000 conn(0x557c77fc22c0 msgr2=0x557c780961a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.936+0000 7fe0f0ff9700 0 client.25228 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.937+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 shutdown_connections 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.937+0000 7fe119b48b80 1 -- 172.21.15.26:0/2636464000 wait complete. 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.937+0000 7fe119b48b80 20 client.25228 trim_cache size 0 max 16384 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "c3d73cb9-2d83-4c65-bbd8-565d17909311", 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:57:17.938 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0f40b2900 msgr2=0x7fe0f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0f40b2900 0x7fe0f40b4dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009680 tx=0x7fe0f8019040 comp rx=0 tx=0).stop 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 msgr2=0x7fe10819e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe10819e530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e9a0 tx=0x7fe10400ecb0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 shutdown_connections 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe0f40b2900 0x7fe0f40b4dc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe108109680 0x7fe1081976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1081035e0 0x7fe10819e530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 --2- 172.21.15.26:0/3022512487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108103fa0 0x7fe10819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.938+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 >> 172.21.15.26:0/3022512487 conn(0x7fe10807a6f0 msgr2=0x7fe108102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.939+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 shutdown_connections 2024-05-20T20:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:17.939+0000 7fe10f2b2700 1 -- 172.21.15.26:0/3022512487 wait complete. 2024-05-20T20:57:18.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:57:19.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:18 smithi026 ceph-mon[31990]: pgmap v423: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-20T20:57:19.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:18 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:57:19.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:18 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:19.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:18 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:57:19.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:18 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:19.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:18 smithi055 ceph-mon[35213]: pgmap v423: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-20T20:57:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:18 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:57:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:18 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:18 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:57:19.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:18 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:18 smithi144 ceph-mon[35660]: pgmap v423: 97 pgs: 97 active+clean; 670 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-20T20:57:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:18 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:57:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:18 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:18 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:57:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:18 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 -- 172.21.15.26:0/2236889784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4102810 msgr2=0x7fcae4106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/2236889784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4102810 0x7fcae4106810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcad40099c0 tx=0x7fcad4009cd0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 -- 172.21.15.26:0/2236889784 shutdown_connections 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/2236889784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 0x7fcae410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/2236889784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae4101e60 0x7fcae4102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/2236889784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4102810 0x7fcae4106810 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.103+0000 7fcaebfb9700 1 -- 172.21.15.26:0/2236889784 >> 172.21.15.26:0/2236889784 conn(0x7fcae4078ab0 msgr2=0x7fcae40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.104+0000 7fcaebfb9700 1 -- 172.21.15.26:0/2236889784 shutdown_connections 2024-05-20T20:57:21.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.104+0000 7fcaebfb9700 1 -- 172.21.15.26:0/2236889784 wait complete. 2024-05-20T20:57:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 Processor -- start 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 -- start start 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4101e60 0x7fcae419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae4102810 0x7fcae419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 0x7fcae41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4111400 con 0x7fcae4101e60 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4111280 con 0x7fcae4106ef0 2024-05-20T20:57:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaebfb9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4111580 con 0x7fcae4102810 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcae9d55700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4101e60 0x7fcae419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcae9d55700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4101e60 0x7fcae419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38566/0 (socket says 172.21.15.26:38566) 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcae9d55700 1 -- 172.21.15.26:0/340252905 learned_addr learned my addr 172.21.15.26:0/340252905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcaea556700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 0x7fcae41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.105+0000 7fcae9554700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae4102810 0x7fcae419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaea556700 1 -- 172.21.15.26:0/340252905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae4102810 msgr2=0x7fcae419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaea556700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae4102810 0x7fcae419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaea556700 1 -- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4101e60 msgr2=0x7fcae419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaea556700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4101e60 0x7fcae419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaea556700 1 -- 172.21.15.26:0/340252905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae419a120 con 0x7fcae4106ef0 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaea556700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 0x7fcae41999c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009fb0 tx=0x7fcadc0076c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc019070 con 0x7fcae4106ef0 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae419a410 con 0x7fcae4106ef0 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcadc003c00 con 0x7fcae4106ef0 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc014650 con 0x7fcae4106ef0 2024-05-20T20:57:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.106+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae41a66d0 con 0x7fcae4106ef0 2024-05-20T20:57:21.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.108+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae406a230 con 0x7fcae4106ef0 2024-05-20T20:57:21.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.110+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcadc01c070 con 0x7fcae4106ef0 2024-05-20T20:57:21.110 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:20 smithi026 ceph-mon[31990]: pgmap v424: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:57:21.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.111+0000 7fcadaffd700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcad00b2980 0x7fcad00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:21.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.111+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fcadc0e1c60 con 0x7fcae4106ef0 2024-05-20T20:57:21.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.111+0000 7fcae9d55700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcad00b2980 0x7fcad00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:21.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.112+0000 7fcae9d55700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcad00b2980 0x7fcad00b4e40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcae419ab00 tx=0x7fcae000b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.114+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcadc011070 con 0x7fcae4106ef0 2024-05-20T20:57:21.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:20 smithi055 ceph-mon[35213]: pgmap v424: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:57:21.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:20 smithi144 ceph-mon[35660]: pgmap v424: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-20T20:57:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.505+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcae419aef0 con 0x7fcae4106ef0 2024-05-20T20:57:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.506+0000 7fcadaffd700 1 -- 172.21.15.26:0/340252905 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fcadc0a98c0 con 0x7fcae4106ef0 2024-05-20T20:57:21.505 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:57:21.505 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcad00b2980 msgr2=0x7fcad00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcad00b2980 0x7fcad00b4e40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcae419ab00 tx=0x7fcae000b500 comp rx=0 tx=0).stop 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 msgr2=0x7fcae41999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 0x7fcae41999c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009fb0 tx=0x7fcadc0076c0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 shutdown_connections 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcad00b2980 0x7fcad00b4e40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae4101e60 0x7fcae419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae4106ef0 0x7fcae41999c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 --2- 172.21.15.26:0/340252905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae4102810 0x7fcae419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.508+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 >> 172.21.15.26:0/340252905 conn(0x7fcae4078ab0 msgr2=0x7fcae4105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.509+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 shutdown_connections 2024-05-20T20:57:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:21.509+0000 7fcaebfb9700 1 -- 172.21.15.26:0/340252905 wait complete. 2024-05-20T20:57:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:57:22.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:21 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:57:22.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:21 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/340252905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:22.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:21 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:57:22.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:21 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/340252905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:22.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:21 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:57:22.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:21 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/340252905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:22.304 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-05-20T20:57:22.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-05-20T20:57:22.865 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:22 smithi026 ceph-mon[31990]: pgmap v425: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.904+0000 7fb780cd5700 1 -- 172.21.15.26:0/2900295273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c102e00 msgr2=0x7fb77c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.904+0000 7fb780cd5700 1 --2- 172.21.15.26:0/2900295273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c102e00 0x7fb77c10f2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb77400b0a0 tx=0x7fb77400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 -- 172.21.15.26:0/2900295273 shutdown_connections 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 --2- 172.21.15.26:0/2900295273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c102e00 0x7fb77c10f2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 --2- 172.21.15.26:0/2900295273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb77c108520 0x7fb77c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 --2- 172.21.15.26:0/2900295273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102440 0x7fb77c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 -- 172.21.15.26:0/2900295273 >> 172.21.15.26:0/2900295273 conn(0x7fb77c0793f0 msgr2=0x7fb77c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 -- 172.21.15.26:0/2900295273 shutdown_connections 2024-05-20T20:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.905+0000 7fb780cd5700 1 -- 172.21.15.26:0/2900295273 wait complete. 2024-05-20T20:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.906+0000 7fb780cd5700 1 Processor -- start 2024-05-20T20:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.906+0000 7fb780cd5700 1 -- start start 2024-05-20T20:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb780cd5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb77c102440 0x7fb77c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb780cd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102e00 0x7fb77c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb780cd5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 0x7fb77c1a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb780cd5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1124d0 con 0x7fb77c108520 2024-05-20T20:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb780cd5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c112350 con 0x7fb77c102440 2024-05-20T20:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb780cd5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c112650 con 0x7fb77c102e00 2024-05-20T20:57:22.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 0x7fb77c1a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb779d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102e00 0x7fb77c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 0x7fb77c1a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38584/0 (socket says 172.21.15.26:38584) 2024-05-20T20:57:22.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 -- 172.21.15.26:0/4041171280 learned_addr learned my addr 172.21.15.26:0/4041171280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 -- 172.21.15.26:0/4041171280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102e00 msgr2=0x7fb77c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102e00 0x7fb77c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 -- 172.21.15.26:0/4041171280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb77c102440 msgr2=0x7fb77c19a750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb77c102440 0x7fb77c19a750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb77ad9d700 1 -- 172.21.15.26:0/4041171280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c1a5d10 con 0x7fb77c108520 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.907+0000 7fb779d9b700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102e00 0x7fb77c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.908+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 0x7fb77c1a57d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb774009fd0 tx=0x7fb774004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.908+0000 7fb76b7fe700 1 -- 172.21.15.26:0/4041171280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774022070 con 0x7fb77c108520 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.908+0000 7fb76b7fe700 1 -- 172.21.15.26:0/4041171280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb77401c6c0 con 0x7fb77c108520 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.908+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c198540 con 0x7fb77c108520 2024-05-20T20:57:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.908+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c198960 con 0x7fb77c108520 2024-05-20T20:57:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.908+0000 7fb76b7fe700 1 -- 172.21.15.26:0/4041171280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774004280 con 0x7fb77c108520 2024-05-20T20:57:22.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.912+0000 7fb76b7fe700 1 -- 172.21.15.26:0/4041171280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb774004420 con 0x7fb77c108520 2024-05-20T20:57:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.913+0000 7fb76b7fe700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb7640b2980 0x7fb7640b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.913+0000 7fb76b7fe700 1 -- 172.21.15.26:0/4041171280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb7740e7060 con 0x7fb77c108520 2024-05-20T20:57:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.913+0000 7fb77a59c700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb7640b2980 0x7fb7640b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.913+0000 7fb78b56bb80 1 Processor -- start 2024-05-20T20:57:22.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.914+0000 7fb78b56bb80 1 -- start start 2024-05-20T20:57:22.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.914+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 0x5581047fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.914+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x5581047f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.914+0000 7fb77a59c700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb7640b2980 0x7fb7640b4e40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009680 tx=0x7fb76c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.914+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047f8260 0x558104804080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558104755d70 con 0x5581047fd320 2024-05-20T20:57:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558104745910 con 0x5581047f8260 2024-05-20T20:57:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558104749950 con 0x5581047f78a0 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 0x5581047fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 0x5581047fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38592/0 (socket says 172.21.15.26:38592) 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 -- 172.21.15.26:0/4191998421 learned_addr learned my addr 172.21.15.26:0/4191998421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047f8260 0x558104804080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 -- 172.21.15.26:0/4191998421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 msgr2=0x5581047f7d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x5581047f7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 -- 172.21.15.26:0/4191998421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047f8260 msgr2=0x558104804080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047f8260 0x558104804080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 -- 172.21.15.26:0/4191998421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581046eefb0 con 0x5581047fd320 2024-05-20T20:57:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.915+0000 7fb77a59c700 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 0x5581047fd700 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00eb70 tx=0x7fb76c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84847362158f802 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.916+0000 7fb768ff9700 1 -- 172.21.15.26:0/4191998421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c021070 con 0x5581047fd320 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.916+0000 7fb768ff9700 1 -- 172.21.15.26:0/4191998421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb76c010d50 con 0x5581047fd320 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.916+0000 7fb768ff9700 1 -- 172.21.15.26:0/4191998421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c005280 con 0x5581047fd320 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 -- 172.21.15.26:0/4191998421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 msgr2=0x5581047fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 0x5581047fd700 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00eb70 tx=0x7fb76c011000 comp rx=0 tx=0).stop 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 -- 172.21.15.26:0/4191998421 shutdown_connections 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047fd320 0x5581047fd700 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047f8260 0x558104804080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/4191998421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x5581047f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 -- 172.21.15.26:0/4191998421 >> 172.21.15.26:0/4191998421 conn(0x558104723240 msgr2=0x5581047f23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 -- 172.21.15.26:0/4191998421 shutdown_connections 2024-05-20T20:57:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.917+0000 7fb78b56bb80 1 -- 172.21.15.26:0/4191998421 wait complete. 2024-05-20T20:57:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 -- start start 2024-05-20T20:57:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 Processor -- start 2024-05-20T20:57:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104803ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 0x558104800fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581048014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb779d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 0x558104800fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb779d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 0x558104800fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38594/0 (socket says 172.21.15.26:38594) 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb779d9b700 1 -- 172.21.15.26:0/2639508661 learned_addr learned my addr 172.21.15.26:0/2639508661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55810464bfa0 con 0x5581047f8260 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5581046adda0 con 0x5581047fd320 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.918+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558104804780 con 0x5581047f78a0 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb78b56bb80 10 client.0 _ll_register_callbacks cb 0x5581046e0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb77a59c700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104803ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581048014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb779d9b700 1 -- 172.21.15.26:0/2639508661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 msgr2=0x558104803ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb779d9b700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104803ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb779d9b700 1 -- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 msgr2=0x5581048014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb779d9b700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581048014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb779d9b700 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581046958d0 con 0x5581047f8260 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581048014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb779d9b700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 0x558104800fa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb77000e3e0 tx=0x7fb77000e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec59029d5e151e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.919+0000 7fb761ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.920+0000 7fb761ffb700 1 -- 172.21.15.26:0/2639508661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770016070 con 0x5581047f8260 2024-05-20T20:57:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.920+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c010d50 con 0x5581047f8260 2024-05-20T20:57:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.920+0000 7fb761ffb700 1 -- 172.21.15.26:0/2639508661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb770011670 con 0x5581047f8260 2024-05-20T20:57:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.920+0000 7fb761ffb700 1 -- 172.21.15.26:0/2639508661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb770022050 con 0x5581047f8260 2024-05-20T20:57:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.920+0000 7fb78b56bb80 10 client.15702 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.920+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581046ff760 con 0x5581047f8260 2024-05-20T20:57:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb761ffb700 1 -- 172.21.15.26:0/2639508661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb76c010d50 con 0x5581047f8260 2024-05-20T20:57:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb761ffb700 1 -- 172.21.15.26:0/2639508661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb77000f040 con 0x5581047f8260 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb78b56bb80 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb78b56bb80 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb78b56bb80 20 client.15702 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048112c0 0x5581048136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb78b56bb80 4 client.15702 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.921+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558104757cc0 con 0x5581048112c0 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.922+0000 7fb77a59c700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048112c0 0x5581048136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.922+0000 7fb761ffb700 10 client.15702 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.922+0000 7fb77a59c700 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048112c0 0x5581048136b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25619179bba5e343 server_cookie=c7f0e64e91801bc5 in_seq=0 out_seq=0 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb761ffb700 1 -- 172.21.15.26:0/2639508661 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558104757cc0 con 0x5581048112c0 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb761ffb700 10 client.15702 handle_command_reply: tid=0 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 client.15702 shutdown 2024-05-20T20:57:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 msgr2=0x558104800fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 0x558104800fa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb77000e3e0 tx=0x7fb77000e6f0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 shutdown_connections 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048112c0 0x5581048136b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581048014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104803ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/2639508661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f8260 0x558104800fa0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 >> 172.21.15.26:0/2639508661 conn(0x558104723240 msgr2=0x5581047f1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.923+0000 7fb761ffb700 0 client.15702 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.924+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 shutdown_connections 2024-05-20T20:57:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.924+0000 7fb78b56bb80 1 -- 172.21.15.26:0/2639508661 wait complete. 2024-05-20T20:57:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.924+0000 7fb78b56bb80 20 client.15702 trim_cache size 0 max 16384 2024-05-20T20:57:22.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.947+0000 7fb78b56bb80 1 Processor -- start 2024-05-20T20:57:22.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.947+0000 7fb78b56bb80 1 -- start start 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 0x5581048015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581047fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55810480d870 0x5581047fb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 0x5581048015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 0x5581048015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38600/0 (socket says 172.21.15.26:38600) 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c00fec0 con 0x5581047f78a0 2024-05-20T20:57:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb77a59c700 1 -- 172.21.15.26:0/3687429620 learned_addr learned my addr 172.21.15.26:0/3687429620 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55810480d870 0x5581047fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb779d9b700 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581047fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c00d750 con 0x5581047fd320 2024-05-20T20:57:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.948+0000 7fb78b56bb80 1 -- 172.21.15.26:0/3687429620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c00b5a0 con 0x55810480d870 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb779d9b700 1 -- 172.21.15.26:0/3687429620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55810480d870 msgr2=0x5581047fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb779d9b700 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55810480d870 0x5581047fb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb779d9b700 1 -- 172.21.15.26:0/3687429620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 msgr2=0x5581048015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb779d9b700 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 0x5581048015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb779d9b700 1 -- 172.21.15.26:0/3687429620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581048038e0 con 0x5581047fd320 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb77a59c700 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 0x5581048015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb779d9b700 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581047fc960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x558104802030 tx=0x7fb770010e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95afe0ac4502d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.949+0000 7fb763fff700 1 -- 172.21.15.26:0/3687429620 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77002b070 con 0x5581047fd320 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.950+0000 7fb763fff700 1 -- 172.21.15.26:0/3687429620 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb770011470 con 0x5581047fd320 2024-05-20T20:57:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.950+0000 7fb763fff700 1 -- 172.21.15.26:0/3687429620 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770016070 con 0x5581047fd320 2024-05-20T20:57:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.950+0000 7fb78b56bb80 1 -- 172.21.15.26:0/3687429620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 msgr2=0x5581047fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.950+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581047fc960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x558104802030 tx=0x7fb770010e10 comp rx=0 tx=0).stop 2024-05-20T20:57:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 -- 172.21.15.26:0/3687429620 shutdown_connections 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581047f78a0 0x5581048015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55810480d870 0x5581047fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/3687429620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x5581047fc960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 -- 172.21.15.26:0/3687429620 >> 172.21.15.26:0/3687429620 conn(0x558104723240 msgr2=0x5581047fad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 -- 172.21.15.26:0/3687429620 shutdown_connections 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 -- 172.21.15.26:0/3687429620 wait complete. 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.951+0000 7fb78b56bb80 1 -- start start 2024-05-20T20:57:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 Processor -- start 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104809c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x55810480a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 0x558104806a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c00bc10 con 0x55810480d870 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c00d080 con 0x5581047fd320 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb78b56bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c00c2e0 con 0x5581047f78a0 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 0x558104806a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 0x558104806a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38614/0 (socket says 172.21.15.26:38614) 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb77ad9d700 1 -- 172.21.15.26:0/148579357 learned_addr learned my addr 172.21.15.26:0/148579357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb77a59c700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104809c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.952+0000 7fb779d9b700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x55810480a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb78b56bb80 10 client.0 _ll_register_callbacks cb 0x5581046e0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77ad9d700 1 -- 172.21.15.26:0/148579357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 msgr2=0x558104809c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104809c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77ad9d700 1 -- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 msgr2=0x55810480a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x55810480a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77ad9d700 1 -- 172.21.15.26:0/148579357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558104807270 con 0x55810480d870 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77a59c700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104809c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb77ad9d700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 0x558104806a30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb77400b800 tx=0x7fb77400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dd4d27d074801d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb761ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb761ffb700 1 -- 172.21.15.26:0/148579357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740aed50 con 0x55810480d870 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb761ffb700 1 -- 172.21.15.26:0/148579357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb7740960b0 con 0x55810480d870 2024-05-20T20:57:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.953+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb770011470 con 0x55810480d870 2024-05-20T20:57:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb761ffb700 1 -- 172.21.15.26:0/148579357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7740962b0 con 0x55810480d870 2024-05-20T20:57:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 10 client.15708 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558104885150 con 0x55810480d870 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb761ffb700 1 -- 172.21.15.26:0/148579357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb770011470 con 0x55810480d870 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb761ffb700 1 -- 172.21.15.26:0/148579357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb774011070 con 0x55810480d870 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 20 client.15708 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048857e0 0x558104887c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 4 client.15708 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.954+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5581047514a0 con 0x5581048857e0 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.955+0000 7fb77a59c700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048857e0 0x558104887c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.955+0000 7fb761ffb700 10 client.15708 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb77a59c700 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048857e0 0x558104887c00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9eaa34eef680171 server_cookie=e2b76be86540a3c0 in_seq=0 out_seq=0 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb761ffb700 1 -- 172.21.15.26:0/148579357 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5581047514a0 con 0x5581048857e0 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb761ffb700 10 client.15708 handle_command_reply: tid=0 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 client.15708 shutdown 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 msgr2=0x558104806a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 0x558104806a30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb77400b800 tx=0x7fb77400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 shutdown_connections 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5581048857e0 0x558104887c00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb761ffb700 0 client.15708 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55810480d870 0x558104806a30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581047f78a0 0x558104809c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 --2- 172.21.15.26:0/148579357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581047fd320 0x55810480a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.956+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 >> 172.21.15.26:0/148579357 conn(0x558104723240 msgr2=0x5581047f6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 shutdown_connections 2024-05-20T20:57:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb78b56bb80 1 -- 172.21.15.26:0/148579357 wait complete. 2024-05-20T20:57:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb78b56bb80 20 client.15708 trim_cache size 0 max 16384 2024-05-20T20:57:22.959 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:57:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:57:22.959 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb7640b2980 msgr2=0x7fb7640b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb780cd5700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb7640b2980 0x7fb7640b4e40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009680 tx=0x7fb76c009260 comp rx=0 tx=0).stop 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 msgr2=0x7fb77c1a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb780cd5700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 0x7fb77c1a57d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb774009fd0 tx=0x7fb774004f80 comp rx=0 tx=0).stop 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.957+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 shutdown_connections 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb7640b2980 0x7fb7640b4e40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c108520 0x7fb77c1a57d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb77c102440 0x7fb77c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 --2- 172.21.15.26:0/4041171280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c102e00 0x7fb77c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 >> 172.21.15.26:0/4041171280 conn(0x7fb77c0793f0 msgr2=0x7fb77c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 shutdown_connections 2024-05-20T20:57:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:22.958+0000 7fb780cd5700 1 -- 172.21.15.26:0/4041171280 wait complete. 2024-05-20T20:57:23.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:22 smithi055 ceph-mon[35213]: pgmap v425: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:22 smithi144 ceph-mon[35660]: pgmap v425: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:24.427 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:57:24.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:57:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:25.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:24 smithi026 ceph-mon[31990]: pgmap v426: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:25.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:24 smithi055 ceph-mon[35213]: pgmap v426: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:24 smithi144 ceph-mon[35660]: pgmap v426: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.603+0000 7f0a23789700 1 -- 172.21.15.26:0/1519701663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 msgr2=0x7f0a1c1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.603+0000 7f0a23789700 1 --2- 172.21.15.26:0/1519701663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1081d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0099c0 tx=0x7f0a0c009cd0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.603+0000 7f0a23789700 1 -- 172.21.15.26:0/1519701663 shutdown_connections 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.603+0000 7f0a23789700 1 --2- 172.21.15.26:0/1519701663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c108900 0x7f0a1c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 --2- 172.21.15.26:0/1519701663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a1c06b480 0x7f0a1c06b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 --2- 172.21.15.26:0/1519701663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1081d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 -- 172.21.15.26:0/1519701663 >> 172.21.15.26:0/1519701663 conn(0x7f0a1c0793e0 msgr2=0x7f0a1c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 -- 172.21.15.26:0/1519701663 shutdown_connections 2024-05-20T20:57:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 -- 172.21.15.26:0/1519701663 wait complete. 2024-05-20T20:57:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 Processor -- start 2024-05-20T20:57:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.604+0000 7f0a23789700 1 -- start start 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a23789700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 0x7f0a1c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a23789700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a23789700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a1c108900 0x7f0a1c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a23789700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1135d0 con 0x7f0a1c06be30 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a23789700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c113450 con 0x7f0a1c06b480 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a23789700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c113750 con 0x7f0a1c108900 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 0x7f0a1c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 0x7f0a1c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:45662/0 (socket says 172.21.15.26:45662) 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 -- 172.21.15.26:0/3922915260 learned_addr learned my addr 172.21.15.26:0/3922915260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a20d24700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21d26700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a1c108900 0x7f0a1c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 -- 172.21.15.26:0/3922915260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a1c108900 msgr2=0x7f0a1c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a1c108900 0x7f0a1c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 -- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 msgr2=0x7f0a1c1a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.605+0000 7f0a21525700 1 -- 172.21.15.26:0/3922915260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c19c1c0 con 0x7f0a1c06b480 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a20d24700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a21525700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 0x7f0a1c1a18a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d950 tx=0x7f0a1800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a127fc700 1 -- 172.21.15.26:0/3922915260 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18016070 con 0x7f0a1c06b480 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c19c4b0 con 0x7f0a1c06b480 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1a8770 con 0x7f0a1c06b480 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a127fc700 1 -- 172.21.15.26:0/3922915260 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a18009dd0 con 0x7f0a1c06b480 2024-05-20T20:57:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.606+0000 7f0a127fc700 1 -- 172.21.15.26:0/3922915260 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800f3c0 con 0x7f0a1c06b480 2024-05-20T20:57:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.609+0000 7f0a127fc700 1 -- 172.21.15.26:0/3922915260 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0a18019070 con 0x7f0a1c06b480 2024-05-20T20:57:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.610+0000 7f0a127fc700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a080bb240 0x7f0a080bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.610+0000 7f0a127fc700 1 -- 172.21.15.26:0/3922915260 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0a180e1e80 con 0x7f0a1c06b480 2024-05-20T20:57:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.610+0000 7f0a20d24700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a080bb240 0x7f0a080bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.610+0000 7f0a2e01fb80 1 Processor -- start 2024-05-20T20:57:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.610+0000 7f0a2e01fb80 1 -- start start 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a20d24700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a080bb240 0x7f0a080bd700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c103e00 tx=0x7f0a0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 0x5617ca6b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a2e01fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5617ca611f10 con 0x5617ca6b4650 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a2e01fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5617ca601ab0 con 0x5617ca6b87c0 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a2e01fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5617ca605af0 con 0x5617ca6b3c90 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38652/0 (socket says 172.21.15.26:38652) 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 -- 172.21.15.26:0/108988439 learned_addr learned my addr 172.21.15.26:0/108988439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 -- 172.21.15.26:0/108988439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 msgr2=0x5617ca6b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 -- 172.21.15.26:0/108988439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 msgr2=0x5617ca6b8ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 0x5617ca6b8ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.611+0000 7f0a21d26700 1 -- 172.21.15.26:0/108988439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ca5ab150 con 0x5617ca6b4650 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.612+0000 7f0a21d26700 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c06c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011680 tx=0x7f0a14009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fada0a5f1659b18e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.612+0000 7f0a077fe700 1 -- 172.21.15.26:0/108988439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14019030 con 0x5617ca6b4650 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.612+0000 7f0a077fe700 1 -- 172.21.15.26:0/108988439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a140093a0 con 0x5617ca6b4650 2024-05-20T20:57:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.612+0000 7f0a077fe700 1 -- 172.21.15.26:0/108988439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14007500 con 0x5617ca6b4650 2024-05-20T20:57:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/108988439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 msgr2=0x5617ca6c06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c06c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011680 tx=0x7f0a14009920 comp rx=0 tx=0).stop 2024-05-20T20:57:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/108988439 shutdown_connections 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c06c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 0x5617ca6b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/108988439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/108988439 >> 172.21.15.26:0/108988439 conn(0x5617ca5df3e0 msgr2=0x5617ca6ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/108988439 shutdown_connections 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/108988439 wait complete. 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.613+0000 7f0a2e01fb80 1 -- start start 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 Processor -- start 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6c0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b87c0 0x5617ca6bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a20d24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a20d24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6bd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38656/0 (socket says 172.21.15.26:38656) 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a20d24700 1 -- 172.21.15.26:0/2637587419 learned_addr learned my addr 172.21.15.26:0/2637587419 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a21525700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6c0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5617ca508140 con 0x5617ca6b4650 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5617ca569f40 con 0x5617ca6b3c90 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.614+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5617ca6c0d20 con 0x5617ca6b87c0 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a21d26700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b87c0 0x5617ca6bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a20d24700 1 -- 172.21.15.26:0/2637587419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b87c0 msgr2=0x5617ca6bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a20d24700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b87c0 0x5617ca6bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a20d24700 1 -- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 msgr2=0x5617ca6c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a20d24700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6c0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a20d24700 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ca551a70 con 0x5617ca6b4650 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a2e01fb80 10 client.0 _ll_register_callbacks cb 0x5617ca59caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617ca613e60 con 0x5617ca6b4650 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a20d24700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6bd6f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005f50 tx=0x7f0a0c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8d0228bee0165d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a04ff9700 10 client.15720 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.615+0000 7f0a21525700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6c0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a04ff9700 1 -- 172.21.15.26:0/2637587419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00b730 con 0x5617ca6b4650 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a04ff9700 1 -- 172.21.15.26:0/2637587419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0a0c00b8d0 con 0x5617ca6b4650 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a04ff9700 1 -- 172.21.15.26:0/2637587419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c00bad0 con 0x5617ca6b4650 2024-05-20T20:57:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 10 client.15720 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5617ca5bb900 con 0x5617ca6b4650 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a04ff9700 1 -- 172.21.15.26:0/2637587419 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0a0c00bc70 con 0x5617ca6b4650 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 10 client.15720 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 20 client.15720 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca6cd7e0 0x5617ca6cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 4 client.15720 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.616+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617ca60d640 con 0x5617ca6cd7e0 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.617+0000 7f0a21525700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca6cd7e0 0x5617ca6cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.617+0000 7f0a21525700 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca6cd7e0 0x5617ca6cfbd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfa9e4911715fc79 server_cookie=cc1373a171ce7301 in_seq=0 out_seq=0 2024-05-20T20:57:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.617+0000 7f0a04ff9700 10 client.15720 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a04ff9700 1 -- 172.21.15.26:0/2637587419 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5617ca60d640 con 0x5617ca6cd7e0 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a04ff9700 10 client.15720 handle_command_reply: tid=0 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 client.15720 shutdown 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 msgr2=0x5617ca6bd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6bd6f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005f50 tx=0x7f0a0c009310 comp rx=0 tx=0).stop 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 shutdown_connections 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca6cd7e0 0x5617ca6cfbd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a04ff9700 0 client.15720 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b87c0 0x5617ca6bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6c0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/2637587419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6bd6f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.618+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 >> 172.21.15.26:0/2637587419 conn(0x5617ca5df3e0 msgr2=0x5617ca6b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:25.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.619+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 shutdown_connections 2024-05-20T20:57:25.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.619+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/2637587419 wait complete. 2024-05-20T20:57:25.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.619+0000 7f0a2e01fb80 20 client.15720 trim_cache size 0 max 16384 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 Processor -- start 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 -- start start 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b4650 0x5617ca6c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 0x5617ca6b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1ae2a0 con 0x5617ca6b87c0 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1abb30 con 0x5617ca6b3c90 2024-05-20T20:57:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.640+0000 7f0a2e01fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c10e540 con 0x5617ca6b4650 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 0x5617ca6b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 0x5617ca6b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38668/0 (socket says 172.21.15.26:38668) 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 -- 172.21.15.26:0/834438818 learned_addr learned my addr 172.21.15.26:0/834438818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a20d24700 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b4650 0x5617ca6c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21525700 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 -- 172.21.15.26:0/834438818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b4650 msgr2=0x5617ca6c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b4650 0x5617ca6c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 -- 172.21.15.26:0/834438818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 msgr2=0x5617ca6a8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6a8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 -- 172.21.15.26:0/834438818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ca6b4cd0 con 0x5617ca6b87c0 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21d26700 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 0x5617ca6b9db0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009b30 tx=0x7f0a14009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687d42274d10c7b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.641+0000 7f0a21525700 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.642+0000 7f0a06ffd700 1 -- 172.21.15.26:0/834438818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14022070 con 0x5617ca6b87c0 2024-05-20T20:57:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.642+0000 7f0a06ffd700 1 -- 172.21.15.26:0/834438818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a14003cc0 con 0x5617ca6b87c0 2024-05-20T20:57:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.642+0000 7f0a06ffd700 1 -- 172.21.15.26:0/834438818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14007a30 con 0x5617ca6b87c0 2024-05-20T20:57:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.642+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/834438818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 msgr2=0x5617ca6b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.642+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 0x5617ca6b9db0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009b30 tx=0x7f0a14009e70 comp rx=0 tx=0).stop 2024-05-20T20:57:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/834438818 shutdown_connections 2024-05-20T20:57:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b87c0 0x5617ca6b9db0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b3c90 0x5617ca6a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/834438818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b4650 0x5617ca6c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/834438818 >> 172.21.15.26:0/834438818 conn(0x5617ca5df3e0 msgr2=0x5617ca6b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/834438818 shutdown_connections 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/834438818 wait complete. 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.643+0000 7f0a2e01fb80 1 -- start start 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.644+0000 7f0a2e01fb80 1 Processor -- start 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.644+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.644+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.644+0000 7f0a2e01fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 0x5617ca6c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38670/0 (socket says 172.21.15.26:38670) 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 -- 172.21.15.26:0/1321291172 learned_addr learned my addr 172.21.15.26:0/1321291172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a9ff0 con 0x5617ca6b4650 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1ab460 con 0x5617ca6b87c0 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1aa6c0 con 0x5617ca6b3c90 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a2e01fb80 10 client.0 _ll_register_callbacks cb 0x5617ca59caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a21525700 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 -- 172.21.15.26:0/1321291172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 msgr2=0x5617ca6c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 -- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 msgr2=0x5617ca6c1470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 0x5617ca6c1470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ca6c1bc0 con 0x5617ca6b4650 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a14003cc0 con 0x5617ca6b4650 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a20d24700 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c0f30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005450 tx=0x7f0a0c028050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113e7a57f4ae919 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.645+0000 7f0a04ff9700 10 client.15726 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a04ff9700 1 -- 172.21.15.26:0/1321291172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c024bb0 con 0x5617ca6b4650 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a04ff9700 1 -- 172.21.15.26:0/1321291172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0a0c024d50 con 0x5617ca6b4650 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a04ff9700 1 -- 172.21.15.26:0/1321291172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c039420 con 0x5617ca6b4650 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 10 client.15726 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5617ca7390d0 con 0x5617ca6b4650 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a04ff9700 1 -- 172.21.15.26:0/1321291172 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0a0c0395c0 con 0x5617ca6b4650 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 10 client.15726 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 20 client.15726 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca7395e0 0x5617ca73b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 4 client.15726 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.646+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x5617ca4843f0 con 0x5617ca7395e0 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.647+0000 7f0a21525700 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca7395e0 0x5617ca73b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.647+0000 7f0a21525700 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca7395e0 0x5617ca73b9d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd2dbfe91871cbd5 server_cookie=6d6a3564fa07d6ac in_seq=0 out_seq=0 2024-05-20T20:57:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.647+0000 7f0a04ff9700 10 client.15726 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a04ff9700 1 -- 172.21.15.26:0/1321291172 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5617ca4843f0 con 0x5617ca7395e0 2024-05-20T20:57:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a04ff9700 10 client.15726 handle_command_reply: tid=0 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 client.15726 shutdown 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 msgr2=0x5617ca6c0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c0f30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005450 tx=0x7f0a0c028050 comp rx=0 tx=0).stop 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 shutdown_connections 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617ca7395e0 0x5617ca73b9d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617ca6b87c0 0x5617ca6c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617ca6b3c90 0x5617ca6c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 --2- 172.21.15.26:0/1321291172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617ca6b4650 0x5617ca6c0f30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 >> 172.21.15.26:0/1321291172 conn(0x5617ca5df3e0 msgr2=0x5617ca6b9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.655+0000 7f0a04ff9700 0 client.15726 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.656+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 shutdown_connections 2024-05-20T20:57:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.656+0000 7f0a2e01fb80 1 -- 172.21.15.26:0/1321291172 wait complete. 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.656+0000 7f0a2e01fb80 20 client.15726 trim_cache size 0 max 16384 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 6, 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout: "96e2b476": { 2024-05-20T20:57:25.656 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 6, 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 138.8, 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:57:25.657 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:57:25.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:25.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:25.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:25.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:25.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:25.658 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.656+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a080bb240 msgr2=0x7f0a080bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.656+0000 7f0a23789700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a080bb240 0x7f0a080bd700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c103e00 tx=0x7f0a0c005fb0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 msgr2=0x7f0a1c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 0x7f0a1c1a18a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d950 tx=0x7f0a1800dc60 comp rx=0 tx=0).stop 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 shutdown_connections 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a1c108900 0x7f0a1c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a1c06b480 0x7f0a1c1a18a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0a080bb240 0x7f0a080bd700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 --2- 172.21.15.26:0/3922915260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c06be30 0x7f0a1c1a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 >> 172.21.15.26:0/3922915260 conn(0x7f0a1c0793e0 msgr2=0x7f0a1c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 shutdown_connections 2024-05-20T20:57:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:25.657+0000 7f0a23789700 1 -- 172.21.15.26:0/3922915260 wait complete. 2024-05-20T20:57:26.528 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '96e2b476', quiesce took 9.2 seconds. Will release after: 9.650000000000002 2024-05-20T20:57:27.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:26 smithi055 ceph-mon[35213]: pgmap v427: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:27.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:26 smithi026 ceph-mon[31990]: pgmap v427: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:26 smithi144 ceph-mon[35660]: pgmap v427: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:28.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 -- 172.21.15.26:0/888380225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c06bbe0 msgr2=0x7fa89c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 --2- 172.21.15.26:0/888380225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c06bbe0 0x7fa89c1103d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa89400b0a0 tx=0x7fa89400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 -- 172.21.15.26:0/888380225 shutdown_connections 2024-05-20T20:57:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 --2- 172.21.15.26:0/888380225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c06bbe0 0x7fa89c1103d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 --2- 172.21.15.26:0/888380225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa89c1095b0 0x7fa89c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 --2- 172.21.15.26:0/888380225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.187+0000 7fa8a1910700 1 -- 172.21.15.26:0/888380225 >> 172.21.15.26:0/888380225 conn(0x7fa89c0793f0 msgr2=0x7fa89c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.190+0000 7fa8a1910700 1 -- 172.21.15.26:0/888380225 shutdown_connections 2024-05-20T20:57:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.190+0000 7fa8a1910700 1 -- 172.21.15.26:0/888380225 wait complete. 2024-05-20T20:57:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.191+0000 7fa8a1910700 1 Processor -- start 2024-05-20T20:57:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.191+0000 7fa8a1910700 1 -- start start 2024-05-20T20:57:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.191+0000 7fa8a1910700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.191+0000 7fa8a1910700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa89c06bbe0 0x7fa89c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.191+0000 7fa8a1910700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1095b0 0x7fa89c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa8a1910700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c113590 con 0x7fa89c1095b0 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa8a1910700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c113410 con 0x7fa89c06b220 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa8a1910700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c113710 con 0x7fa89c06bbe0 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:45714/0 (socket says 172.21.15.26:45714) 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 -- 172.21.15.26:0/1356373153 learned_addr learned my addr 172.21.15.26:0/1356373153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89a7fc700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa89c06bbe0 0x7fa89c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:28.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89b7fe700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1095b0 0x7fa89c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:28.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 -- 172.21.15.26:0/1356373153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa89c06bbe0 msgr2=0x7fa89c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:28.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa89c06bbe0 0x7fa89c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 -- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1095b0 msgr2=0x7fa89c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1095b0 0x7fa89c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 -- 172.21.15.26:0/1356373153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c1ac450 con 0x7fa89c06b220 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89b7fe700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1095b0 0x7fa89c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.192+0000 7fa89affd700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c102510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00e9c0 tx=0x7fa88c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.193+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c00cbd0 con 0x7fa89c06b220 2024-05-20T20:57:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.193+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c1ac6e0 con 0x7fa89c06b220 2024-05-20T20:57:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.193+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa88c00cd70 con 0x7fa89c06b220 2024-05-20T20:57:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.194+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c005540 con 0x7fa89c06b220 2024-05-20T20:57:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.194+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1acc20 con 0x7fa89c06b220 2024-05-20T20:57:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.195+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c10ba90 con 0x7fa89c06b220 2024-05-20T20:57:28.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.197+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa88c004750 con 0x7fa89c06b220 2024-05-20T20:57:28.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.198+0000 7fa8a090e700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8840b2980 0x7fa8840b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:28.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.198+0000 7fa89a7fc700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8840b2980 0x7fa8840b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:28.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.199+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa88c0e24f0 con 0x7fa89c06b220 2024-05-20T20:57:28.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.199+0000 7fa89a7fc700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8840b2980 0x7fa8840b4e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa89c103cd0 tx=0x7fa89000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:28.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.203+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa88c0aa150 con 0x7fa89c06b220 2024-05-20T20:57:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.473+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa89c103a40 con 0x7fa89c06b220 2024-05-20T20:57:28.474 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:57:28.474 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:57:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.474+0000 7fa8a090e700 1 -- 172.21.15.26:0/1356373153 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fa88c0a9a50 con 0x7fa89c06b220 2024-05-20T20:57:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8840b2980 msgr2=0x7fa8840b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8840b2980 0x7fa8840b4e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa89c103cd0 tx=0x7fa89000a380 comp rx=0 tx=0).stop 2024-05-20T20:57:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 msgr2=0x7fa89c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c102510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00e9c0 tx=0x7fa88c00ecd0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 shutdown_connections 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8840b2980 0x7fa8840b4e40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1095b0 0x7fa89c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa89c06b220 0x7fa89c102510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 --2- 172.21.15.26:0/1356373153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa89c06bbe0 0x7fa89c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 >> 172.21.15.26:0/1356373153 conn(0x7fa89c0793f0 msgr2=0x7fa89c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 shutdown_connections 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:28.477+0000 7fa8a1910700 1 -- 172.21.15.26:0/1356373153 wait complete. 2024-05-20T20:57:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:57:29.066 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:28 smithi026 ceph-mon[31990]: pgmap v428: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:29.067 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:28 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1356373153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:29.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:57:29.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:28 smithi055 ceph-mon[35213]: pgmap v428: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:29.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:28 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1356373153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:28 smithi144 ceph-mon[35660]: pgmap v428: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:28 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1356373153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 -- 172.21.15.26:0/1829464122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028102ba0 msgr2=0x7f9028103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 --2- 172.21.15.26:0/1829464122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028102ba0 0x7f9028103020 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9014009900 tx=0x7f901402e800 comp rx=0 tx=0).stop 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 -- 172.21.15.26:0/1829464122 shutdown_connections 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 --2- 172.21.15.26:0/1829464122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9028103560 0x7f9028109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 --2- 172.21.15.26:0/1829464122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028102ba0 0x7f9028103020 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 --2- 172.21.15.26:0/1829464122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 0x7f9028101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 -- 172.21.15.26:0/1829464122 >> 172.21.15.26:0/1829464122 conn(0x7f90280fcf20 msgr2=0x7f90280ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 -- 172.21.15.26:0/1829464122 shutdown_connections 2024-05-20T20:57:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.926+0000 7f902efc1640 1 -- 172.21.15.26:0/1829464122 wait complete. 2024-05-20T20:57:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 Processor -- start 2024-05-20T20:57:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 -- start start 2024-05-20T20:57:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 0x7f9028196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9028102ba0 0x7f9028196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 0x7f902819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f902810c2e0 con 0x7f9028103560 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f902810c160 con 0x7f90281019b0 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902efc1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f902810c460 con 0x7f9028102ba0 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902d55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 0x7f902819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902d55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 0x7f902819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60014/0 (socket says 172.21.15.26:60014) 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902d55e640 1 -- 172.21.15.26:0/174477917 learned_addr learned my addr 172.21.15.26:0/174477917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f902cd5d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 0x7f9028196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.927+0000 7f9027fff640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9028102ba0 0x7f9028196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902cd5d640 1 -- 172.21.15.26:0/174477917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9028102ba0 msgr2=0x7f9028196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902cd5d640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9028102ba0 0x7f9028196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902cd5d640 1 -- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 msgr2=0x7f902819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902cd5d640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 0x7f902819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902cd5d640 1 -- 172.21.15.26:0/174477917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902819e5b0 con 0x7f90281019b0 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902d55e640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 0x7f902819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902cd5d640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 0x7f9028196990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f901c00e9c0 tx=0x7f901c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f9025ffb640 1 -- 172.21.15.26:0/174477917 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c00ccc0 con 0x7f90281019b0 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f9025ffb640 1 -- 172.21.15.26:0/174477917 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f901c0044b0 con 0x7f90281019b0 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902819e8a0 con 0x7f90281019b0 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.928+0000 7f9025ffb640 1 -- 172.21.15.26:0/174477917 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c003e10 con 0x7f90281019b0 2024-05-20T20:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.929+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902819ee10 con 0x7f90281019b0 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.932+0000 7f9025ffb640 1 -- 172.21.15.26:0/174477917 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f901c003fd0 con 0x7f90281019b0 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 Processor -- start 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 -- start start 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94acbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ace6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560a94a37c30 con 0x560a94ac66c0 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560a94a277d0 con 0x560a94ac70b0 2024-05-20T20:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9031db5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560a94a2b810 con 0x560a94acc220 2024-05-20T20:57:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f9027fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ace6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f902d55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f902d55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60020/0 (socket says 172.21.15.26:60020) 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.933+0000 7f902d55e640 1 -- 172.21.15.26:0/356067677 learned_addr learned my addr 172.21.15.26:0/356067677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f9025ffb640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90180b2920 0x7f90180b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f902d55e640 1 -- 172.21.15.26:0/356067677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 msgr2=0x560a94ace6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f902d55e640 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ace6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f902d55e640 1 -- 172.21.15.26:0/356067677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 msgr2=0x560a94acbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f902d55e640 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94acbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f902d55e640 1 -- 172.21.15.26:0/356067677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9473c210 con 0x560a94ac66c0 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f9025ffb640 1 -- 172.21.15.26:0/174477917 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f901c0e2720 con 0x7f90281019b0 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.934+0000 7f902d55e640 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac6ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f902000a6c0 tx=0x7f902000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5bbe4438e26e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.935+0000 7f9012ffd640 1 -- 172.21.15.26:0/356067677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020007bb0 con 0x560a94ac66c0 2024-05-20T20:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.935+0000 7f902d55e640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90180b2920 0x7f90180b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.935+0000 7f9012ffd640 1 -- 172.21.15.26:0/356067677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f902000ae20 con 0x560a94ac66c0 2024-05-20T20:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.935+0000 7f9012ffd640 1 -- 172.21.15.26:0/356067677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200145c0 con 0x560a94ac66c0 2024-05-20T20:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 -- 172.21.15.26:0/356067677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 msgr2=0x560a94ac6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f902d55e640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90180b2920 0x7f90180b4de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90200147f0 tx=0x7f9020014760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac6ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f902000a6c0 tx=0x7f902000b760 comp rx=0 tx=0).stop 2024-05-20T20:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 -- 172.21.15.26:0/356067677 shutdown_connections 2024-05-20T20:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ace6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94acbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 --2- 172.21.15.26:0/356067677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac6ae0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 -- 172.21.15.26:0/356067677 >> 172.21.15.26:0/356067677 conn(0x560a94ac1d30 msgr2=0x560a94ac2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 -- 172.21.15.26:0/356067677 shutdown_connections 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.936+0000 7f9031db5740 1 -- 172.21.15.26:0/356067677 wait complete. 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.937+0000 7f9031db5740 1 -- start start 2024-05-20T20:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.937+0000 7f9031db5740 1 Processor -- start 2024-05-20T20:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.937+0000 7f9031db5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac66c0 0x560a94ad0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f9031db5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94ac70b0 0x560a94ad0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f9031db5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 0x560a94ad7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f9031db5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560a9470d790 con 0x560a94acc220 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f9031db5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560a949e3890 con 0x560a94ac66c0 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f902d55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 0x560a94ad7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f902d55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 0x560a94ad7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60022/0 (socket says 172.21.15.26:60022) 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f902d55e640 1 -- 172.21.15.26:0/2248880749 learned_addr learned my addr 172.21.15.26:0/2248880749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f902cd5d640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac66c0 0x560a94ad0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f9027fff640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94ac70b0 0x560a94ad0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.938+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560a94acf1d0 con 0x560a94ac70b0 2024-05-20T20:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9031db5740 10 client.0 _ll_register_callbacks cb 0x560a9473ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9027fff640 1 -- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac66c0 msgr2=0x560a94ad0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9027fff640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac66c0 0x560a94ad0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9027fff640 1 -- 172.21.15.26:0/2248880749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 msgr2=0x560a94ad7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9027fff640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 0x560a94ad7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9027fff640 1 -- 172.21.15.26:0/2248880749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a94808620 con 0x560a94ac70b0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f902d55e640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 0x560a94ad7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9027fff640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94ac70b0 0x560a94ad0d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9014009850 tx=0x7f9014004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c057f4ed9bc5aaf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9007fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9007fff640 1 -- 172.21.15.26:0/2248880749 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901403a070 con 0x560a94ac70b0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.939+0000 7f9007fff640 1 -- 172.21.15.26:0/2248880749 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9014003a00 con 0x560a94ac70b0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902000ae20 con 0x560a94ac70b0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9007fff640 1 -- 172.21.15.26:0/2248880749 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9014003c00 con 0x560a94ac70b0 2024-05-20T20:57:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 10 client.25276 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a94adadc0 con 0x560a94ac70b0 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9007fff640 1 -- 172.21.15.26:0/2248880749 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f902000ae20 con 0x560a94ac70b0 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9007fff640 1 -- 172.21.15.26:0/2248880749 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9014035070 con 0x560a94ac70b0 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 10 client.25276 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 10 client.25276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 20 client.25276 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94ade910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 4 client.25276 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.940+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a94a39b80 con 0x560a94adc500 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.941+0000 7f902cd5d640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94ade910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.941+0000 7f902cd5d640 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94ade910 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161ab6078edeccfe server_cookie=6d3686f078d1cb4a in_seq=0 out_seq=0 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.941+0000 7f9007fff640 10 client.25276 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.942+0000 7f9007fff640 1 -- 172.21.15.26:0/2248880749 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560a94a39b80 con 0x560a94adc500 2024-05-20T20:57:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.942+0000 7f9007fff640 10 client.25276 handle_command_reply: tid=0 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.942+0000 7f9031db5740 1 client.25276 shutdown 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.942+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94ac70b0 msgr2=0x560a94ad0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.942+0000 7f9031db5740 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94ac70b0 0x560a94ad0d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9014009850 tx=0x7f9014004390 comp rx=0 tx=0).stop 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 shutdown_connections 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94ade910 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9007fff640 0 client.25276 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94acc220 0x560a94ad7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94ac70b0 0x560a94ad0d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 --2- 172.21.15.26:0/2248880749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac66c0 0x560a94ad0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 >> 172.21.15.26:0/2248880749 conn(0x560a94ac1d30 msgr2=0x560a94ac0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 shutdown_connections 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 1 -- 172.21.15.26:0/2248880749 wait complete. 2024-05-20T20:57:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.944+0000 7f9031db5740 20 client.25276 trim_cache size 0 max 16384 2024-05-20T20:57:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 Processor -- start 2024-05-20T20:57:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 -- start start 2024-05-20T20:57:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ac88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90281aa580 con 0x560a94ac66c0 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f90281a7e50 con 0x560a94ac70b0 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.965+0000 7f9031db5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90281a5ca0 con 0x560a94acc220 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f902cd5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f902cd5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60034/0 (socket says 172.21.15.26:60034) 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f902cd5d640 1 -- 172.21.15.26:0/1320045542 learned_addr learned my addr 172.21.15.26:0/1320045542 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 -- 172.21.15.26:0/1320045542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 msgr2=0x560a94ac88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ac88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 -- 172.21.15.26:0/1320045542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 msgr2=0x560a94ac8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 -- 172.21.15.26:0/1320045542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a94ac1b60 con 0x560a94ac70b0 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f902cd5d640 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.966+0000 7f9027fff640 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad7d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9014009900 tx=0x7f901402e800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4561540cdc25fee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.967+0000 7f90127fc640 1 -- 172.21.15.26:0/1320045542 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901405c070 con 0x560a94ac70b0 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.967+0000 7f90127fc640 1 -- 172.21.15.26:0/1320045542 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9014041e40 con 0x560a94ac70b0 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.967+0000 7f90127fc640 1 -- 172.21.15.26:0/1320045542 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901403a070 con 0x560a94ac70b0 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.967+0000 7f9031db5740 1 -- 172.21.15.26:0/1320045542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 msgr2=0x560a94ad7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.967+0000 7f9031db5740 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad7d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9014009900 tx=0x7f901402e800 comp rx=0 tx=0).stop 2024-05-20T20:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 -- 172.21.15.26:0/1320045542 shutdown_connections 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ac88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad7d50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 --2- 172.21.15.26:0/1320045542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ac8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 -- 172.21.15.26:0/1320045542 >> 172.21.15.26:0/1320045542 conn(0x560a94ac1d30 msgr2=0x560a94abce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 -- 172.21.15.26:0/1320045542 shutdown_connections 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 -- 172.21.15.26:0/1320045542 wait complete. 2024-05-20T20:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 -- start start 2024-05-20T20:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.968+0000 7f9031db5740 1 Processor -- start 2024-05-20T20:57:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ad5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ad6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90281a6310 con 0x560a94ac66c0 2024-05-20T20:57:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f90281a7780 con 0x560a94ac70b0 2024-05-20T20:57:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90281a69e0 con 0x560a94acc220 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9031db5740 10 client.0 _ll_register_callbacks cb 0x7f9014009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9027fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9027fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51898/0 (socket says 172.21.15.26:51898) 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f9027fff640 1 -- 172.21.15.26:0/1162637630 learned_addr learned my addr 172.21.15.26:0/1162637630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f902cd5d640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ad5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.969+0000 7f902d55e640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ad6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9027fff640 1 -- 172.21.15.26:0/1162637630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 msgr2=0x560a94ad6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9027fff640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ad6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9027fff640 1 -- 172.21.15.26:0/1162637630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 msgr2=0x560a94ad5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9027fff640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ad5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9027fff640 1 -- 172.21.15.26:0/1162637630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a94ac6220 con 0x560a94ac70b0 2024-05-20T20:57:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f902cd5d640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ad5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9027fff640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f901405a040 tx=0x7f90140415d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32605d7bdfdfd64 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9007fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9007fff640 1 -- 172.21.15.26:0/1162637630 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901403a070 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a94a33360 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.970+0000 7f9007fff640 1 -- 172.21.15.26:0/1162637630 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f90140364f0 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9007fff640 1 -- 172.21.15.26:0/1162637630 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9014067050 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 10 client.25316 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a94ad7f60 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9007fff640 1 -- 172.21.15.26:0/1162637630 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9014009520 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9007fff640 1 -- 172.21.15.26:0/1162637630 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9014056030 con 0x560a94ac70b0 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 10 client.25316 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 10 client.25316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 20 client.25316 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94b49730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 4 client.25316 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a94a33360 con 0x560a94adc500 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.971+0000 7f902cd5d640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94b49730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.972+0000 7f9007fff640 10 client.25316 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.972+0000 7f902cd5d640 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94b49730 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9050d3d113fdc33 server_cookie=8ada9f8582d5c710 in_seq=0 out_seq=0 2024-05-20T20:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.976+0000 7f9007fff640 1 -- 172.21.15.26:0/1162637630 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560a94a33360 con 0x560a94adc500 2024-05-20T20:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.976+0000 7f9007fff640 10 client.25316 handle_command_reply: tid=0 2024-05-20T20:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.976+0000 7f9031db5740 1 client.25316 shutdown 2024-05-20T20:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.976+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 msgr2=0x560a94ad5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f901405a040 tx=0x7f90140415d0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 shutdown_connections 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560a94adc500 0x560a94b49730 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9007fff640 0 client.25316 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a94acc220 0x560a94ad6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a94ac70b0 0x560a94ad5870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 --2- 172.21.15.26:0/1162637630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94ac66c0 0x560a94ad5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 >> 172.21.15.26:0/1162637630 conn(0x560a94ac1d30 msgr2=0x560a94ad07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.977+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 shutdown_connections 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.978+0000 7f9031db5740 1 -- 172.21.15.26:0/1162637630 wait complete. 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.978+0000 7f9031db5740 20 client.25316 trim_cache size 0 max 16384 2024-05-20T20:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:57:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:57:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:57:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90180b2920 msgr2=0x7f90180b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90180b2920 0x7f90180b4de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90200147f0 tx=0x7f9020014760 comp rx=0 tx=0).stop 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 msgr2=0x7f9028196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 0x7f9028196990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f901c00e9c0 tx=0x7f901c00ee80 comp rx=0 tx=0).stop 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 shutdown_connections 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f90180b2920 0x7f90180b4de0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9028103560 0x7f902819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9028102ba0 0x7f9028196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 --2- 172.21.15.26:0/174477917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90281019b0 0x7f9028196990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 >> 172.21.15.26:0/174477917 conn(0x7f90280fcf20 msgr2=0x7f9028105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 shutdown_connections 2024-05-20T20:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:29.979+0000 7f902efc1640 1 -- 172.21.15.26:0/174477917 wait complete. 2024-05-20T20:57:31.084 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:30 smithi026 ceph-mon[31990]: pgmap v429: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:31.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:30 smithi055 ceph-mon[35213]: pgmap v429: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:30 smithi144 ceph-mon[35660]: pgmap v429: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 -- 172.21.15.26:0/1954038277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac410aa80 msgr2=0x7fdac410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 --2- 172.21.15.26:0/1954038277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac410aa80 0x7fdac410af00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009990 tx=0x7fdab8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 -- 172.21.15.26:0/1954038277 shutdown_connections 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 --2- 172.21.15.26:0/1954038277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdac411cd20 0x7fdac411f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 --2- 172.21.15.26:0/1954038277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 --2- 172.21.15.26:0/1954038277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac410aa80 0x7fdac410af00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.477+0000 7fdac9b37700 1 -- 172.21.15.26:0/1954038277 >> 172.21.15.26:0/1954038277 conn(0x7fdac406e6b0 msgr2=0x7fdac406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.478+0000 7fdac9b37700 1 -- 172.21.15.26:0/1954038277 shutdown_connections 2024-05-20T20:57:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.478+0000 7fdac9b37700 1 -- 172.21.15.26:0/1954038277 wait complete. 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.478+0000 7fdac9b37700 1 Processor -- start 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 -- start start 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdac410aa80 0x7fdac41ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac41aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac411cd20 0x7fdac41a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4122490 con 0x7fdac411cd20 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4122310 con 0x7fdac410aa80 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac9b37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4122610 con 0x7fdac4111d20 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac41aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdac410aa80 0x7fdac41ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac41aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46818/0 (socket says 172.21.15.26:46818) 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac2ffd700 1 -- 172.21.15.26:0/4092146961 learned_addr learned my addr 172.21.15.26:0/4092146961 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.479+0000 7fdac3fff700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac411cd20 0x7fdac41a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac2ffd700 1 -- 172.21.15.26:0/4092146961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdac410aa80 msgr2=0x7fdac41ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdac410aa80 0x7fdac41ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac2ffd700 1 -- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac411cd20 msgr2=0x7fdac41a87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac411cd20 0x7fdac41a87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac2ffd700 1 -- 172.21.15.26:0/4092146961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac41a8fb0 con 0x7fdac4111d20 2024-05-20T20:57:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac3fff700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac411cd20 0x7fdac41a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac41aeb90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098e0 tx=0x7fdab80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.480+0000 7fdac0ff9700 1 -- 172.21.15.26:0/4092146961 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8027070 con 0x7fdac4111d20 2024-05-20T20:57:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.481+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac41a9240 con 0x7fdac4111d20 2024-05-20T20:57:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.481+0000 7fdac0ff9700 1 -- 172.21.15.26:0/4092146961 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdab8022470 con 0x7fdac4111d20 2024-05-20T20:57:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.481+0000 7fdac0ff9700 1 -- 172.21.15.26:0/4092146961 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8017650 con 0x7fdac4111d20 2024-05-20T20:57:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.481+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41b54b0 con 0x7fdac4111d20 2024-05-20T20:57:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.483+0000 7fdac0ff9700 1 -- 172.21.15.26:0/4092146961 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdab802a070 con 0x7fdac4111d20 2024-05-20T20:57:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.484+0000 7fdac0ff9700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaac0b2900 0x7fdaac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.484+0000 7fdac0ff9700 1 -- 172.21.15.26:0/4092146961 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fdab80e3100 con 0x7fdac4111d20 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdac37fe700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaac0b2900 0x7fdaac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 Processor -- start 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 -- start start 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5629a6109c90 con 0x5629a61ab210 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5629a60f9830 con 0x5629a61aa860 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5629a60fd870 con 0x5629a61aef60 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdac3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdac3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46828/0 (socket says 172.21.15.26:46828) 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdac3fff700 1 -- 172.21.15.26:0/1059228060 learned_addr learned my addr 172.21.15.26:0/1059228060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.485+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac37fe700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaac0b2900 0x7fdaac0b4dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009550 tx=0x7fdabc0091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac3fff700 1 -- 172.21.15.26:0/1059228060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 msgr2=0x5629a61aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac3fff700 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac3fff700 1 -- 172.21.15.26:0/1059228060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 msgr2=0x5629a61ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac3fff700 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac3fff700 1 -- 172.21.15.26:0/1059228060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a60498d0 con 0x5629a61aef60 2024-05-20T20:57:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdac3fff700 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61af3a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011470 tx=0x7fdab4011780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d37e320ab61fa93 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdaa9ffb700 1 -- 172.21.15.26:0/1059228060 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4004020 con 0x5629a61aef60 2024-05-20T20:57:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdaa9ffb700 1 -- 172.21.15.26:0/1059228060 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdab4003680 con 0x5629a61aef60 2024-05-20T20:57:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.486+0000 7fdaa9ffb700 1 -- 172.21.15.26:0/1059228060 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab400e690 con 0x5629a61aef60 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 -- 172.21.15.26:0/1059228060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 msgr2=0x5629a61af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61af3a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011470 tx=0x7fdab4011780 comp rx=0 tx=0).stop 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 -- 172.21.15.26:0/1059228060 shutdown_connections 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61af3a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/1059228060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.487+0000 7fdad43cdb80 1 -- 172.21.15.26:0/1059228060 >> 172.21.15.26:0/1059228060 conn(0x5629a60d8080 msgr2=0x5629a61a4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.488+0000 7fdad43cdb80 1 -- 172.21.15.26:0/1059228060 shutdown_connections 2024-05-20T20:57:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.488+0000 7fdad43cdb80 1 -- 172.21.15.26:0/1059228060 wait complete. 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.488+0000 7fdad43cdb80 1 -- start start 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.488+0000 7fdad43cdb80 1 Processor -- start 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5629a5ffffa0 con 0x5629a61ab210 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5629a609b7c0 con 0x5629a61aa860 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5629a61b7120 con 0x5629a61aef60 2024-05-20T20:57:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdad43cdb80 10 client.0 _ll_register_callbacks cb 0x5629a6094920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60102/0 (socket says 172.21.15.26:60102) 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 -- 172.21.15.26:0/3766411638 learned_addr learned my addr 172.21.15.26:0/3766411638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac3fff700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac37fe700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 -- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 msgr2=0x5629a61b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 -- 172.21.15.26:0/3766411638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 msgr2=0x5629a61b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.489+0000 7fdac2ffd700 1 -- 172.21.15.26:0/3766411638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a6098690 con 0x5629a61ab210 2024-05-20T20:57:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61b3630 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdab80e0bf0 tx=0x7fdab80cd140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6622ba434b5bd160 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdaa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/3766411638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80aae50 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/3766411638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fdab80aa9b0 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab4003680 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/3766411638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdab80aa750 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdad43cdb80 10 client.15762 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.490+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629a61c1830 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/3766411638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdab4003680 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/3766411638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdab80b3120 con 0x5629a61ab210 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdad43cdb80 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdad43cdb80 10 client.15762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdad43cdb80 20 client.15762 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a61c3df0 0x5629a61c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdad43cdb80 4 client.15762 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629a61769e0 con 0x5629a61c3df0 2024-05-20T20:57:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.491+0000 7fdac37fe700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a61c3df0 0x5629a61c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.492+0000 7fdac37fe700 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a61c3df0 0x5629a61c61e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ef379dd09befa86 server_cookie=a8cbb1e106ea485f in_seq=0 out_seq=0 2024-05-20T20:57:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.492+0000 7fdaa2ffd700 10 client.15762 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.492+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/3766411638 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629a61769e0 con 0x5629a61c3df0 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.492+0000 7fdaa2ffd700 10 client.15762 handle_command_reply: tid=0 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 client.15762 shutdown 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 msgr2=0x5629a61b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61b3630 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdab80e0bf0 tx=0x7fdab80cd140 comp rx=0 tx=0).stop 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 shutdown_connections 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a61c3df0 0x5629a61c61e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aef60 0x5629a61b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aa860 0x5629a61b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/3766411638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61ab210 0x5629a61b3630 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 >> 172.21.15.26:0/3766411638 conn(0x5629a60d8080 msgr2=0x5629a61b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdaa2ffd700 0 client.15762 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 shutdown_connections 2024-05-20T20:57:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 1 -- 172.21.15.26:0/3766411638 wait complete. 2024-05-20T20:57:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.493+0000 7fdad43cdb80 20 client.15762 trim_cache size 0 max 16384 2024-05-20T20:57:31.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.508+0000 7fdad43cdb80 1 Processor -- start 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 -- start start 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 0x5629a61cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61ab210 0x5629a61cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aef60 0x5629a61a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41bf230 con 0x5629a61aa860 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41bcac0 con 0x5629a61aef60 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41ba910 con 0x5629a61ab210 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 0x5629a61cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 0x5629a61cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60104/0 (socket says 172.21.15.26:60104) 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdac37fe700 1 -- 172.21.15.26:0/392665213 learned_addr learned my addr 172.21.15.26:0/392665213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdac3fff700 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aef60 0x5629a61a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.509+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61ab210 0x5629a61cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdac37fe700 1 -- 172.21.15.26:0/392665213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61ab210 msgr2=0x5629a61cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdac37fe700 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61ab210 0x5629a61cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdac37fe700 1 -- 172.21.15.26:0/392665213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aef60 msgr2=0x5629a61a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdac37fe700 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aef60 0x5629a61a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdac37fe700 1 -- 172.21.15.26:0/392665213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a61b1ba0 con 0x5629a61aa860 2024-05-20T20:57:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdac37fe700 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 0x5629a61cbad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009790 tx=0x7fdabc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e606e1f283a025af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdaa97fa700 1 -- 172.21.15.26:0/392665213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc007960 con 0x5629a61aa860 2024-05-20T20:57:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdaa97fa700 1 -- 172.21.15.26:0/392665213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdabc024440 con 0x5629a61aa860 2024-05-20T20:57:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.510+0000 7fdaa97fa700 1 -- 172.21.15.26:0/392665213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc016650 con 0x5629a61aa860 2024-05-20T20:57:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 -- 172.21.15.26:0/392665213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 msgr2=0x5629a61cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 0x5629a61cbad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009790 tx=0x7fdabc013000 comp rx=0 tx=0).stop 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 -- 172.21.15.26:0/392665213 shutdown_connections 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aa860 0x5629a61cbad0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61aef60 0x5629a61a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/392665213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61ab210 0x5629a61cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.511+0000 7fdad43cdb80 1 -- 172.21.15.26:0/392665213 >> 172.21.15.26:0/392665213 conn(0x5629a60d8080 msgr2=0x5629a61b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.512+0000 7fdad43cdb80 1 -- 172.21.15.26:0/392665213 shutdown_connections 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.512+0000 7fdad43cdb80 1 -- 172.21.15.26:0/392665213 wait complete. 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.512+0000 7fdad43cdb80 1 -- start start 2024-05-20T20:57:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.512+0000 7fdad43cdb80 1 Processor -- start 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aa860 0x5629a61ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61ab210 0x5629a61aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdad43cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aef60 0x5629a61acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41baf80 con 0x5629a61aef60 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdad43cdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41bc3f0 con 0x5629a61ab210 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aa860 0x5629a61ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aa860 0x5629a61ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46850/0 (socket says 172.21.15.26:46850) 2024-05-20T20:57:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdac37fe700 1 -- 172.21.15.26:0/2159551958 learned_addr learned my addr 172.21.15.26:0/2159551958 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdac2ffd700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61ab210 0x5629a61aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdac3fff700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aef60 0x5629a61acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.513+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41bb650 con 0x5629a61aa860 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdad43cdb80 10 client.0 _ll_register_callbacks cb 0x5629a6094920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdac3fff700 1 -- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aa860 msgr2=0x5629a61ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdac3fff700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aa860 0x5629a61ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdac3fff700 1 -- 172.21.15.26:0/2159551958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61ab210 msgr2=0x5629a61aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdac3fff700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61ab210 0x5629a61aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdac3fff700 1 -- 172.21.15.26:0/2159551958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a622d9a0 con 0x5629a61aef60 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdac3fff700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aef60 0x5629a61acf80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009230 tx=0x7fdab4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0473a8ce78fd5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdaa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/2159551958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4011e00 con 0x5629a61aef60 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/2159551958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fdab40048e0 con 0x5629a61aef60 2024-05-20T20:57:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.514+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc024440 con 0x5629a61aef60 2024-05-20T20:57:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/2159551958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdab4004ae0 con 0x5629a61aef60 2024-05-20T20:57:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdad43cdb80 10 client.15768 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629a622eb60 con 0x5629a61aef60 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/2159551958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdabc024440 con 0x5629a61aef60 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/2159551958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdab4033020 con 0x5629a61aef60 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdad43cdb80 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdad43cdb80 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdad43cdb80 20 client.15768 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.515+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a622f080 0x5629a6231490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.516+0000 7fdad43cdb80 4 client.15768 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.516+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629a610bbe0 con 0x5629a622f080 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.516+0000 7fdac37fe700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a622f080 0x5629a6231490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.516+0000 7fdac37fe700 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a622f080 0x5629a6231490 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ffb8ca917b2def6 server_cookie=fa6da3aa7e7561cc in_seq=0 out_seq=0 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.516+0000 7fdaa2ffd700 10 client.15768 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdaa2ffd700 1 -- 172.21.15.26:0/2159551958 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629a610bbe0 con 0x5629a622f080 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdaa2ffd700 10 client.15768 handle_command_reply: tid=0 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 client.15768 shutdown 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aef60 msgr2=0x5629a61acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aef60 0x5629a61acf80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009230 tx=0x7fdab4005750 comp rx=0 tx=0).stop 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 shutdown_connections 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5629a622f080 0x5629a6231490 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5629a61aef60 0x5629a61acf80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629a61aa860 0x5629a61ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 --2- 172.21.15.26:0/2159551958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629a61ab210 0x5629a61aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 >> 172.21.15.26:0/2159551958 conn(0x5629a60d8080 msgr2=0x5629a61a9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdaa2ffd700 0 client.15768 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 shutdown_connections 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 1 -- 172.21.15.26:0/2159551958 wait complete. 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.517+0000 7fdad43cdb80 20 client.15768 trim_cache size 0 max 16384 2024-05-20T20:57:31.519 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaac0b2900 msgr2=0x7fdaac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaac0b2900 0x7fdaac0b4dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009550 tx=0x7fdabc0091f0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 msgr2=0x7fdac41aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac41aeb90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098e0 tx=0x7fdab80057c0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 shutdown_connections 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaac0b2900 0x7fdaac0b4dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac411cd20 0x7fdac41a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdac410aa80 0x7fdac41ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 --2- 172.21.15.26:0/4092146961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac4111d20 0x7fdac41aeb90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:31.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.518+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 >> 172.21.15.26:0/4092146961 conn(0x7fdac406e6b0 msgr2=0x7fdac411ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:31.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.519+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 shutdown_connections 2024-05-20T20:57:31.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:31.519+0000 7fdac9b37700 1 -- 172.21.15.26:0/4092146961 wait complete. 2024-05-20T20:57:32.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:57:32.518 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:32.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:32 smithi026 ceph-mon[31990]: pgmap v430: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T20:57:33.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:32 smithi055 ceph-mon[35213]: pgmap v430: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T20:57:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:32 smithi144 ceph-mon[35660]: pgmap v430: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T20:57:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.346+0000 7f7c2c886700 1 -- 172.21.15.26:0/1145831985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 msgr2=0x7f7c28109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.346+0000 7f7c2c886700 1 --2- 172.21.15.26:0/1145831985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28109990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009990 tx=0x7f7c18009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 -- 172.21.15.26:0/1145831985 shutdown_connections 2024-05-20T20:57:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 --2- 172.21.15.26:0/1145831985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28109990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 --2- 172.21.15.26:0/1145831985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c2806bbe0 0x7f7c281103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 --2- 172.21.15.26:0/1145831985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c2806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 -- 172.21.15.26:0/1145831985 >> 172.21.15.26:0/1145831985 conn(0x7f7c280793f0 msgr2=0x7f7c28079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 -- 172.21.15.26:0/1145831985 shutdown_connections 2024-05-20T20:57:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.347+0000 7f7c2c886700 1 -- 172.21.15.26:0/1145831985 wait complete. 2024-05-20T20:57:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.348+0000 7f7c2c886700 1 Processor -- start 2024-05-20T20:57:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.348+0000 7f7c2c886700 1 -- start start 2024-05-20T20:57:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2c886700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c28102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2c886700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c2806bbe0 0x7f7c28102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2c886700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2c886700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28113570 con 0x7f7c281095b0 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2c886700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281133f0 con 0x7f7c2806b220 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2c886700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281136f0 con 0x7f7c2806bbe0 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c28102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c28102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51958/0 (socket says 172.21.15.26:51958) 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c2659c700 1 -- 172.21.15.26:0/4022708562 learned_addr learned my addr 172.21.15.26:0/4022708562 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c2806bbe0 0x7f7c28102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.349+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2659c700 1 -- 172.21.15.26:0/4022708562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c2806bbe0 msgr2=0x7f7c28102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2659c700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c2806bbe0 0x7f7c28102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2659c700 1 -- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 msgr2=0x7f7c28102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2659c700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2659c700 1 -- 172.21.15.26:0/4022708562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c281ac4b0 con 0x7f7c2806b220 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2659c700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c28102510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c18000c00 tx=0x7f7c180047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c177fe700 1 -- 172.21.15.26:0/4022708562 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18027070 con 0x7f7c2806b220 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c177fe700 1 -- 172.21.15.26:0/4022708562 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c1800fb80 con 0x7f7c2806b220 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c281ac6e0 con 0x7f7c2806b220 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c177fe700 1 -- 172.21.15.26:0/4022708562 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18022d00 con 0x7f7c2806b220 2024-05-20T20:57:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.350+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c281acb20 con 0x7f7c2806b220 2024-05-20T20:57:34.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.355+0000 7f7c177fe700 1 -- 172.21.15.26:0/4022708562 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7c1802a070 con 0x7f7c2806b220 2024-05-20T20:57:34.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.356+0000 7f7c3711cb80 1 Processor -- start 2024-05-20T20:57:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.356+0000 7f7c3711cb80 1 -- start start 2024-05-20T20:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.356+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc430c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.356+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc434940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc4353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc430c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c3711cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8dc38fc90 con 0x55a8dc430860 2024-05-20T20:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c3711cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a8dc37f830 con 0x55a8dc434f60 2024-05-20T20:57:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc430c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60160/0 (socket says 172.21.15.26:60160) 2024-05-20T20:57:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c3711cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8dc383870 con 0x55a8dc431210 2024-05-20T20:57:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 -- 172.21.15.26:0/2307484074 learned_addr learned my addr 172.21.15.26:0/2307484074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc4353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 -- 172.21.15.26:0/2307484074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 msgr2=0x55a8dc434940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c2659c700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc434940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc434940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 -- 172.21.15.26:0/2307484074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 msgr2=0x55a8dc4353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc4353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 -- 172.21.15.26:0/2307484074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8dc2cf8d0 con 0x55a8dc430860 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc4353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c2659c700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc434940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.357+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc430c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009cd0 tx=0x7f7c20007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03acc8307a9ef6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.358+0000 7f7c14ff9700 1 -- 172.21.15.26:0/2307484074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20019070 con 0x55a8dc430860 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.358+0000 7f7c14ff9700 1 -- 172.21.15.26:0/2307484074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c20007e40 con 0x55a8dc430860 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.358+0000 7f7c14ff9700 1 -- 172.21.15.26:0/2307484074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20013830 con 0x55a8dc430860 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.358+0000 7f7c177fe700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c100b2900 0x7f7c100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.358+0000 7f7c2659c700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c100b2900 0x7f7c100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 -- 172.21.15.26:0/2307484074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 msgr2=0x55a8dc430c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc430c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009cd0 tx=0x7f7c20007500 comp rx=0 tx=0).stop 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 -- 172.21.15.26:0/2307484074 shutdown_connections 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc430c40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc4353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/2307484074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc434940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 -- 172.21.15.26:0/2307484074 >> 172.21.15.26:0/2307484074 conn(0x55a8dc35e080 msgr2=0x55a8dc42ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c2659c700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c100b2900 0x7f7c100b4dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c180a78f0 tx=0x7f7c180e8040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c177fe700 1 -- 172.21.15.26:0/4022708562 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7c180e3090 con 0x7f7c2806b220 2024-05-20T20:57:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 -- 172.21.15.26:0/2307484074 shutdown_connections 2024-05-20T20:57:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.359+0000 7f7c3711cb80 1 -- 172.21.15.26:0/2307484074 wait complete. 2024-05-20T20:57:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 -- start start 2024-05-20T20:57:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 Processor -- start 2024-05-20T20:57:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc4396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc439c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8dc285fa0 con 0x55a8dc430860 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a8dc3217c0 con 0x55a8dc431210 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8dc43d120 con 0x55a8dc434f60 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c3711cb80 10 client.0 _ll_register_callbacks cb 0x55a8dc31a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.360+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60162/0 (socket says 172.21.15.26:60162) 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 -- 172.21.15.26:0/3615218395 learned_addr learned my addr 172.21.15.26:0/3615218395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc4396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc439c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 -- 172.21.15.26:0/3615218395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 msgr2=0x55a8dc439c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc439c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 -- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 msgr2=0x55a8dc4396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc4396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 -- 172.21.15.26:0/3615218395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8dc31e690 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43c3c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7c20017040 tx=0x7f7c200090b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede8c1b7e06ced5c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c0dffb700 1 -- 172.21.15.26:0/3615218395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20027060 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.361+0000 7f7c0dffb700 1 -- 172.21.15.26:0/3615218395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7c200138c0 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.362+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8dc3fc9e0 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.362+0000 7f7c0dffb700 1 -- 172.21.15.26:0/3615218395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7c20019070 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.362+0000 7f7c3711cb80 10 client.15777 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.362+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8dc458830 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.362+0000 7f7c0dffb700 1 -- 172.21.15.26:0/3615218395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a8dc3fc9e0 con 0x55a8dc430860 2024-05-20T20:57:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c0dffb700 1 -- 172.21.15.26:0/3615218395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7c2002b030 con 0x55a8dc430860 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c3711cb80 10 client.15777 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c3711cb80 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c3711cb80 20 client.15777 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc45ae10 0x55a8dc45d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c3711cb80 4 client.15777 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8dc391be0 con 0x55a8dc45ae10 2024-05-20T20:57:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.363+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc45ae10 0x55a8dc45d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.364+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc45ae10 0x55a8dc45d200 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33d425408a637e9b server_cookie=7820d77407c066ff in_seq=0 out_seq=0 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.364+0000 7f7c0dffb700 10 client.15777 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c0dffb700 1 -- 172.21.15.26:0/3615218395 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8dc391be0 con 0x55a8dc45ae10 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c0dffb700 10 client.15777 handle_command_reply: tid=0 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 client.15777 shutdown 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 msgr2=0x55a8dc43c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43c3c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7c20017040 tx=0x7f7c200090b0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 shutdown_connections 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc45ae10 0x55a8dc45d200 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43c3c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc439c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c0dffb700 0 client.15777 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/3615218395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc4396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 >> 172.21.15.26:0/3615218395 conn(0x55a8dc35e080 msgr2=0x55a8dc42a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 shutdown_connections 2024-05-20T20:57:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 1 -- 172.21.15.26:0/3615218395 wait complete. 2024-05-20T20:57:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.365+0000 7f7c3711cb80 20 client.15777 trim_cache size 0 max 16384 2024-05-20T20:57:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.395+0000 7f7c3711cb80 1 Processor -- start 2024-05-20T20:57:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.395+0000 7f7c3711cb80 1 -- start start 2024-05-20T20:57:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.395+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60174/0 (socket says 172.21.15.26:60174) 2024-05-20T20:57:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc42ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 -- 172.21.15.26:0/4016113644 learned_addr learned my addr 172.21.15.26:0/4016113644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:34.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc42b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2806e3f0 con 0x55a8dc430860 2024-05-20T20:57:34.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281a51a0 con 0x55a8dc434f60 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281a2ff0 con 0x55a8dc431210 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 -- 172.21.15.26:0/4016113644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 msgr2=0x55a8dc42ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc42ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 -- 172.21.15.26:0/4016113644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 msgr2=0x55a8dc42b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc42b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 -- 172.21.15.26:0/4016113644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8dc437c50 con 0x55a8dc430860 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.396+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43b600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009330 tx=0x7f7c20029f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5b62f68c11193e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.397+0000 7f7c0ffff700 1 -- 172.21.15.26:0/4016113644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20027070 con 0x55a8dc430860 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.397+0000 7f7c0ffff700 1 -- 172.21.15.26:0/4016113644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c2001f830 con 0x55a8dc430860 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.397+0000 7f7c0ffff700 1 -- 172.21.15.26:0/4016113644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c200139b0 con 0x55a8dc430860 2024-05-20T20:57:34.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 msgr2=0x55a8dc43b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43b600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009330 tx=0x7f7c20029f50 comp rx=0 tx=0).stop 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 shutdown_connections 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43b600 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc434f60 0x55a8dc42b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/4016113644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc431210 0x55a8dc42ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 >> 172.21.15.26:0/4016113644 conn(0x55a8dc35e080 msgr2=0x55a8dc43aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 shutdown_connections 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 -- 172.21.15.26:0/4016113644 wait complete. 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.398+0000 7f7c3711cb80 1 -- start start 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 Processor -- start 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc43fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c26d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60186/0 (socket says 172.21.15.26:60186) 2024-05-20T20:57:34.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c26d9d700 1 -- 172.21.15.26:0/1363287971 learned_addr learned my addr 172.21.15.26:0/1363287971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:34.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc4c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281a3660 con 0x55a8dc430860 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281a4ad0 con 0x55a8dc431210 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281a3d30 con 0x55a8dc434f60 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.399+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc43fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c3711cb80 10 client.0 _ll_register_callbacks cb 0x55a8dc31a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c26d9d700 1 -- 172.21.15.26:0/1363287971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 msgr2=0x55a8dc4c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc4c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c26d9d700 1 -- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 msgr2=0x55a8dc43fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc43fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c26d9d700 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8dc4c3d00 con 0x55a8dc430860 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c2001f830 con 0x55a8dc430860 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c26d9d700 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43f990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c20013480 tx=0x7f7c200060b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f63c0e99e8eea16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c0dffb700 10 client.15783 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c0dffb700 1 -- 172.21.15.26:0/1363287971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20045050 con 0x55a8dc430860 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c0dffb700 1 -- 172.21.15.26:0/1363287971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7c20036640 con 0x55a8dc430860 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c0dffb700 1 -- 172.21.15.26:0/1363287971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7c20027040 con 0x55a8dc430860 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c3711cb80 10 client.15783 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.400+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a8dc4c4df0 con 0x55a8dc430860 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c0dffb700 1 -- 172.21.15.26:0/1363287971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7c20034020 con 0x55a8dc430860 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c3711cb80 10 client.15783 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c3711cb80 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c3711cb80 20 client.15783 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc4c5360 0x55a8dc4c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c3711cb80 4 client.15783 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8dc38b3c0 con 0x55a8dc4c5360 2024-05-20T20:57:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.401+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc4c5360 0x55a8dc4c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c25d9b700 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc4c5360 0x55a8dc4c7780 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba5910d4f15ea8e8 server_cookie=b514c43278695508 in_seq=0 out_seq=0 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c0dffb700 10 client.15783 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c0dffb700 1 -- 172.21.15.26:0/1363287971 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8dc38b3c0 con 0x55a8dc4c5360 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c0dffb700 10 client.15783 handle_command_reply: tid=0 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 client.15783 shutdown 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 msgr2=0x55a8dc43f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43f990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c20013480 tx=0x7f7c200060b0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 shutdown_connections 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55a8dc4c5360 0x55a8dc4c7780 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c0dffb700 0 client.15783 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8dc430860 0x55a8dc43f990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8dc434f60 0x55a8dc4c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 --2- 172.21.15.26:0/1363287971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8dc431210 0x55a8dc43fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 >> 172.21.15.26:0/1363287971 conn(0x55a8dc35e080 msgr2=0x55a8dc43b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 shutdown_connections 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 1 -- 172.21.15.26:0/1363287971 wait complete. 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.402+0000 7f7c3711cb80 20 client.15783 trim_cache size 0 max 16384 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c100b2900 msgr2=0x7f7c100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c100b2900 0x7f7c100b4dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c180a78f0 tx=0x7f7c180e8040 comp rx=0 tx=0).stop 2024-05-20T20:57:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 msgr2=0x7f7c28102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c28102510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c18000c00 tx=0x7f7c180047b0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 shutdown_connections 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c100b2900 0x7f7c100b4dc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c281095b0 0x7f7c28102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2806b220 0x7f7c28102510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 --2- 172.21.15.26:0/4022708562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c2806bbe0 0x7f7c28102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.403+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 >> 172.21.15.26:0/4022708562 conn(0x7f7c280793f0 msgr2=0x7f7c28101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.404+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 shutdown_connections 2024-05-20T20:57:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:34.404+0000 7f7c2c886700 1 -- 172.21.15.26:0/4022708562 wait complete. 2024-05-20T20:57:35.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:34 smithi055 ceph-mon[35213]: pgmap v431: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-20T20:57:35.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:34 smithi026 ceph-mon[31990]: pgmap v431: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-20T20:57:35.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:57:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:34 smithi144 ceph-mon[35660]: pgmap v431: 97 pgs: 97 active+clean; 671 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-20T20:57:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:36.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:57:37.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:36 smithi055 ceph-mon[35213]: pgmap v432: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:37.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:36 smithi026 ceph-mon[31990]: pgmap v432: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:36 smithi144 ceph-mon[35660]: pgmap v432: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:39.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:38 smithi055 ceph-mon[35213]: pgmap v433: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:39.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:38 smithi026 ceph-mon[31990]: pgmap v433: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:38 smithi144 ceph-mon[35660]: pgmap v433: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:40.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:40 smithi026 ceph-mon[31990]: pgmap v434: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:41.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:40 smithi055 ceph-mon[35213]: pgmap v434: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 -- 172.21.15.26:0/1486933471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 msgr2=0x7f56d0103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/1486933471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0103a40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009990 tx=0x7f56c4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 -- 172.21.15.26:0/1486933471 shutdown_connections 2024-05-20T20:57:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/1486933471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d0103f80 0x7f56d0110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/1486933471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56d01095d0 0x7f56d01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/1486933471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0103a40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.285+0000 7f56d4dcb700 1 -- 172.21.15.26:0/1486933471 >> 172.21.15.26:0/1486933471 conn(0x7f56d007a610 msgr2=0x7f56d007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:41.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.286+0000 7f56d4dcb700 1 -- 172.21.15.26:0/1486933471 shutdown_connections 2024-05-20T20:57:41.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.286+0000 7f56d4dcb700 1 -- 172.21.15.26:0/1486933471 wait complete. 2024-05-20T20:57:41.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.287+0000 7f56d4dcb700 1 Processor -- start 2024-05-20T20:57:41.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.287+0000 7f56d4dcb700 1 -- start start 2024-05-20T20:57:41.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.287+0000 7f56d4dcb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.287+0000 7f56d4dcb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56d0103f80 0x7f56d0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56d4dcb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d01095d0 0x7f56d00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37420/0 (socket says 172.21.15.26:37420) 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56ce59c700 1 -- 172.21.15.26:0/4040449424 learned_addr learned my addr 172.21.15.26:0/4040449424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56d0103f80 0x7f56d0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0113630 con 0x7f56d01035c0 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01134b0 con 0x7f56d0103f80 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01137b0 con 0x7f56d01095d0 2024-05-20T20:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.288+0000 7f56ced9d700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d01095d0 0x7f56d00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56ce59c700 1 -- 172.21.15.26:0/4040449424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d01095d0 msgr2=0x7f56d00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56ce59c700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d01095d0 0x7f56d00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56ce59c700 1 -- 172.21.15.26:0/4040449424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56d0103f80 msgr2=0x7f56d0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56ce59c700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56d0103f80 0x7f56d0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56ce59c700 1 -- 172.21.15.26:0/4040449424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d01ac470 con 0x7f56d01035c0 2024-05-20T20:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56ce59c700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0075840 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e9a0 tx=0x7f56c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:41.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56bf7fe700 1 -- 172.21.15.26:0/4040449424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c000ca10 con 0x7f56d01035c0 2024-05-20T20:57:41.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01ac700 con 0x7f56d01035c0 2024-05-20T20:57:41.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.289+0000 7f56bf7fe700 1 -- 172.21.15.26:0/4040449424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56c000cbb0 con 0x7f56d01035c0 2024-05-20T20:57:41.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.290+0000 7f56bf7fe700 1 -- 172.21.15.26:0/4040449424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0018680 con 0x7f56d01035c0 2024-05-20T20:57:41.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.290+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01acc10 con 0x7f56d01035c0 2024-05-20T20:57:41.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.293+0000 7f56bf7fe700 1 -- 172.21.15.26:0/4040449424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f56c0018820 con 0x7f56d01035c0 2024-05-20T20:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.294+0000 7f56bf7fe700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b80b2900 0x7f56b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.295+0000 7f56bf7fe700 1 -- 172.21.15.26:0/4040449424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f56c00e2370 con 0x7f56d01035c0 2024-05-20T20:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.295+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b80b2900 0x7f56b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.295+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b80b2900 0x7f56b80b4dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f56c400b2e0 tx=0x7f56c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 Processor -- start 2024-05-20T20:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 -- start start 2024-05-20T20:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11edb3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edadd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edae260 0x55c11edba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c11ed0bd70 con 0x55c11edad8a0 2024-05-20T20:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c11ecfb910 con 0x55c11edb3320 2024-05-20T20:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56df661b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c11ecff950 con 0x55c11edae260 2024-05-20T20:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11edb3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11edb3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41906/0 (socket says 172.21.15.26:41906) 2024-05-20T20:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.297+0000 7f56ce59c700 1 -- 172.21.15.26:0/494528719 learned_addr learned my addr 172.21.15.26:0/494528719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ced9d700 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edae260 0x55c11edba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ce59c700 1 -- 172.21.15.26:0/494528719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edae260 msgr2=0x55c11edba080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:41.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ce59c700 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edae260 0x55c11edba080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ce59c700 1 -- 172.21.15.26:0/494528719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 msgr2=0x55c11edadd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T20:57:41.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ce59c700 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edadd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ce59c700 1 -- 172.21.15.26:0/494528719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11eca4fb0 con 0x55c11edb3320 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56ce59c700 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11edb3700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56c00a5950 tx=0x7f56c0009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f405916c630a5b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56bcff9700 1 -- 172.21.15.26:0/494528719 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00a98d0 con 0x55c11edb3320 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56bcff9700 1 -- 172.21.15.26:0/494528719 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56c00adc80 con 0x55c11edb3320 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.298+0000 7f56bcff9700 1 -- 172.21.15.26:0/494528719 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00b22a0 con 0x55c11edb3320 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 -- 172.21.15.26:0/494528719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 msgr2=0x55c11edb3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11edb3700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56c00a5950 tx=0x7f56c0009850 comp rx=0 tx=0).stop 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 -- 172.21.15.26:0/494528719 shutdown_connections 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edae260 0x55c11edba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11edb3700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 --2- 172.21.15.26:0/494528719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edadd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.299+0000 7f56df661b80 1 -- 172.21.15.26:0/494528719 >> 172.21.15.26:0/494528719 conn(0x55c11ecd9240 msgr2=0x55c11eda83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:41.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:40 smithi144 ceph-mon[35660]: pgmap v434: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.300+0000 7f56df661b80 1 -- 172.21.15.26:0/494528719 shutdown_connections 2024-05-20T20:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.300+0000 7f56df661b80 1 -- 172.21.15.26:0/494528719 wait complete. 2024-05-20T20:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.300+0000 7f56df661b80 1 -- start start 2024-05-20T20:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.300+0000 7f56df661b80 1 Processor -- start 2024-05-20T20:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56df661b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edb9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56df661b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edae260 0x55c11edb70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56df661b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edb7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edb9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edb9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37448/0 (socket says 172.21.15.26:37448) 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edae260 0x55c11edb70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edae260 0x55c11edb70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41920/0 (socket says 172.21.15.26:41920) 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56df661b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c11ec01fa0 con 0x55c11edad8a0 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c11ec63da0 con 0x55c11edae260 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56ce59c700 1 -- 172.21.15.26:0/642283867 learned_addr learned my addr 172.21.15.26:0/642283867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c11edba680 con 0x55c11edb3320 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.301+0000 7f56ced9d700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edb7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56df661b80 10 client.0 _ll_register_callbacks cb 0x55c11ec96950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ced9d700 1 -- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edae260 msgr2=0x55c11edb70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ced9d700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edae260 0x55c11edb70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ced9d700 1 -- 172.21.15.26:0/642283867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 msgr2=0x55c11edb9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ced9d700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edb9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ced9d700 1 -- 172.21.15.26:0/642283867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11ec4b8d0 con 0x55c11edb3320 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ce59c700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edb9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56ced9d700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edb7630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56c8004070 tx=0x7f56c80040a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2952651f1e3cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56b5ffb700 1 -- 172.21.15.26:0/642283867 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c801a070 con 0x55c11edb3320 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56b5ffb700 1 -- 172.21.15.26:0/642283867 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f56c8007700 con 0x55c11edb3320 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.302+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c00adc80 con 0x55c11edb3320 2024-05-20T20:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56b5ffb700 1 -- 172.21.15.26:0/642283867 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f56c8026050 con 0x55c11edb3320 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 10 client.25318 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c11ecb5760 con 0x55c11edb3320 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56b5ffb700 1 -- 172.21.15.26:0/642283867 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56c00adc80 con 0x55c11edb3320 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56b5ffb700 1 -- 172.21.15.26:0/642283867 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f56c8016930 con 0x55c11edb3320 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 10 client.25318 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 20 client.25318 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11edc7040 0x55c11edc9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 4 client.25318 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.303+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c11ed0dcc0 con 0x55c11edc7040 2024-05-20T20:57:41.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.304+0000 7f56ce59c700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11edc7040 0x55c11edc9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.304+0000 7f56b5ffb700 10 client.25318 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.304+0000 7f56ce59c700 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11edc7040 0x55c11edc9450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384efe3932eda62e server_cookie=82430611f37d06b7 in_seq=0 out_seq=0 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56b5ffb700 1 -- 172.21.15.26:0/642283867 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c11ed0dcc0 con 0x55c11edc7040 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56b5ffb700 10 client.25318 handle_command_reply: tid=0 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 client.25318 shutdown 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 msgr2=0x55c11edb7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edb7630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56c8004070 tx=0x7f56c80040a0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 shutdown_connections 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11edc7040 0x55c11edc9450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edad8a0 0x55c11edb9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.305+0000 7f56df661b80 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edb7630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.306+0000 7f56b5ffb700 0 client.25318 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:41.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.306+0000 7f56df661b80 1 --2- 172.21.15.26:0/642283867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edae260 0x55c11edb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.306+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 >> 172.21.15.26:0/642283867 conn(0x55c11ecd9240 msgr2=0x55c11eda7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:41.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.306+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 shutdown_connections 2024-05-20T20:57:41.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.306+0000 7f56df661b80 1 -- 172.21.15.26:0/642283867 wait complete. 2024-05-20T20:57:41.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.307+0000 7f56df661b80 20 client.25318 trim_cache size 0 max 16384 2024-05-20T20:57:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.336+0000 7f56df661b80 1 Processor -- start 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.336+0000 7f56df661b80 1 -- start start 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56df661b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edad8a0 0x55c11edc36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56df661b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11eda8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56df661b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11eda8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56df661b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b6c40 con 0x55c11edae260 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56df661b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b44d0 con 0x55c11edb3320 2024-05-20T20:57:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56df661b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b2320 con 0x55c11edad8a0 2024-05-20T20:57:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11eda8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11eda8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37458/0 (socket says 172.21.15.26:37458) 2024-05-20T20:57:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56cdd9b700 1 -- 172.21.15.26:0/1522895872 learned_addr learned my addr 172.21.15.26:0/1522895872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.337+0000 7f56ced9d700 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11eda8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56cdd9b700 1 -- 172.21.15.26:0/1522895872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edad8a0 msgr2=0x55c11edc36e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edad8a0 0x55c11edc36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56cdd9b700 1 -- 172.21.15.26:0/1522895872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 msgr2=0x55c11eda8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11eda8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56cdd9b700 1 -- 172.21.15.26:0/1522895872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11edba490 con 0x55c11edae260 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56ced9d700 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11eda8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11eda8eb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000c00 tx=0x7f56c4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43016b9a187a33d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56b7fff700 1 -- 172.21.15.26:0/1522895872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40056b0 con 0x55c11edae260 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56b7fff700 1 -- 172.21.15.26:0/1522895872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56c400bb40 con 0x55c11edae260 2024-05-20T20:57:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.338+0000 7f56b7fff700 1 -- 172.21.15.26:0/1522895872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c400fbc0 con 0x55c11edae260 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 -- 172.21.15.26:0/1522895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 msgr2=0x55c11eda8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11eda8eb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000c00 tx=0x7f56c4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 -- 172.21.15.26:0/1522895872 shutdown_connections 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edb3320 0x55c11eda8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edad8a0 0x55c11edc36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 --2- 172.21.15.26:0/1522895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11eda8eb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.339+0000 7f56df661b80 1 -- 172.21.15.26:0/1522895872 >> 172.21.15.26:0/1522895872 conn(0x55c11ecd9240 msgr2=0x55c11edb0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.340+0000 7f56df661b80 1 -- 172.21.15.26:0/1522895872 shutdown_connections 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.340+0000 7f56df661b80 1 -- 172.21.15.26:0/1522895872 wait complete. 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.340+0000 7f56df661b80 1 -- start start 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.340+0000 7f56df661b80 1 Processor -- start 2024-05-20T20:57:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edad8a0 0x55c11edbfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11edbfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b2990 con 0x55c11edae260 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b3e00 con 0x55c11edad8a0 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b3060 con 0x55c11edb3320 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56df661b80 10 client.0 _ll_register_callbacks cb 0x55c11ec96950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11edbfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11edbfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37464/0 (socket says 172.21.15.26:37464) 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 -- 172.21.15.26:0/677261402 learned_addr learned my addr 172.21.15.26:0/677261402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56ce59c700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edad8a0 0x55c11edbfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56ced9d700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 -- 172.21.15.26:0/677261402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 msgr2=0x55c11edce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 -- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edad8a0 msgr2=0x55c11edbfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edad8a0 0x55c11edbfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56cdd9b700 1 -- 172.21.15.26:0/677261402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11edce8b0 con 0x55c11edae260 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.341+0000 7f56ced9d700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56cdd9b700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11edbfff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f56c40096c0 tx=0x7f56c400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603a1ad78c89b3be server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56b5ffb700 1 -- 172.21.15.26:0/677261402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c403e060 con 0x55c11edae260 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c400bb40 con 0x55c11edae260 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56b5ffb700 1 -- 172.21.15.26:0/677261402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f56c400f520 con 0x55c11edae260 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56b5ffb700 1 -- 172.21.15.26:0/677261402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f56c4035040 con 0x55c11edae260 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56df661b80 10 client.15798 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c11ee315a0 con 0x55c11edae260 2024-05-20T20:57:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.342+0000 7f56b5ffb700 1 -- 172.21.15.26:0/677261402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56c400bb40 con 0x55c11edae260 2024-05-20T20:57:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56b5ffb700 1 -- 172.21.15.26:0/677261402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f56c4038030 con 0x55c11edae260 2024-05-20T20:57:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56df661b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56df661b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56df661b80 20 client.15798 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56df661b80 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11ee31bc0 0x55c11ee33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56df661b80 4 client.15798 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c11ed074a0 con 0x55c11ee31bc0 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.343+0000 7f56ce59c700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11ee31bc0 0x55c11ee33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56ce59c700 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11ee31bc0 0x55c11ee33fe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e7ba3e314c4b64 server_cookie=e5674166265625d in_seq=0 out_seq=0 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56b5ffb700 10 client.15798 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56b5ffb700 1 -- 172.21.15.26:0/677261402 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c11ed074a0 con 0x55c11ee31bc0 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56b5ffb700 10 client.15798 handle_command_reply: tid=0 2024-05-20T20:57:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 client.15798 shutdown 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 msgr2=0x55c11edbfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11edbfff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f56c40096c0 tx=0x7f56c400b730 comp rx=0 tx=0).stop 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 shutdown_connections 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c11ee31bc0 0x55c11ee33fe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c11edb3320 0x55c11edce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c11edad8a0 0x55c11edbfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 --2- 172.21.15.26:0/677261402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c11edae260 0x55c11edbfff0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 >> 172.21.15.26:0/677261402 conn(0x55c11ecd9240 msgr2=0x55c11edb1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56b5ffb700 0 client.15798 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 shutdown_connections 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 1 -- 172.21.15.26:0/677261402 wait complete. 2024-05-20T20:57:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.344+0000 7f56df661b80 20 client.15798 trim_cache size 0 max 16384 2024-05-20T20:57:41.348 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b80b2900 msgr2=0x7f56b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b80b2900 0x7f56b80b4dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f56c400b2e0 tx=0x7f56c4005fb0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 msgr2=0x7f56d0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0075840 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e9a0 tx=0x7f56c000ecb0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 shutdown_connections 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b80b2900 0x7f56b80b4dc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56d01035c0 0x7f56d0075840 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d01095d0 0x7f56d00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 --2- 172.21.15.26:0/4040449424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56d0103f80 0x7f56d0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.345+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 >> 172.21.15.26:0/4040449424 conn(0x7f56d007a610 msgr2=0x7f56d0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.346+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 shutdown_connections 2024-05-20T20:57:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:41.346+0000 7f56d4dcb700 1 -- 172.21.15.26:0/4040449424 wait complete. 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 -- 172.21.15.26:0/745394950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 msgr2=0x7f25c01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 --2- 172.21.15.26:0/745394950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c01030c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009990 tx=0x7f25ac009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 -- 172.21.15.26:0/745394950 shutdown_connections 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 --2- 172.21.15.26:0/745394950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 --2- 172.21.15.26:0/745394950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25c01084c0 0x7f25c01088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 --2- 172.21.15.26:0/745394950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c01030c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 -- 172.21.15.26:0/745394950 >> 172.21.15.26:0/745394950 conn(0x7f25c0078ab0 msgr2=0x7f25c00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.706+0000 7f25c7043700 1 -- 172.21.15.26:0/745394950 shutdown_connections 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.707+0000 7f25c7043700 1 -- 172.21.15.26:0/745394950 wait complete. 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 Processor -- start 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 -- start start 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c0111960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25c01084c0 0x7f25c01103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0113620 con 0x7f25c0102c40 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01134a0 con 0x7f25c01084c0 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.708+0000 7f25c7043700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01137a0 con 0x7f25c0103600 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59570/0 (socket says 172.21.15.26:59570) 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 -- 172.21.15.26:0/146165098 learned_addr learned my addr 172.21.15.26:0/146165098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25c55e0700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25c01084c0 0x7f25c01103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25c4ddf700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c0111960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 -- 172.21.15.26:0/146165098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25c01084c0 msgr2=0x7f25c01103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25c01084c0 0x7f25c01103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 -- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 msgr2=0x7f25c0111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c0111960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 -- 172.21.15.26:0/146165098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c01ac580 con 0x7f25c0103600 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25bffff700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010fd70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005910 tx=0x7f25ac0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.709+0000 7f25c4ddf700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c0111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.710+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac027070 con 0x7f25c0103600 2024-05-20T20:57:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.710+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25ac00fcc0 con 0x7f25c0103600 2024-05-20T20:57:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.710+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c01ac7b0 con 0x7f25c0103600 2024-05-20T20:57:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.710+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac022d40 con 0x7f25c0103600 2024-05-20T20:57:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.710+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c01acc00 con 0x7f25c0103600 2024-05-20T20:57:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.711+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c006a230 con 0x7f25c0103600 2024-05-20T20:57:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.714+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f25ac02a070 con 0x7f25c0103600 2024-05-20T20:57:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.715+0000 7f25bdffb700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25b00b2900 0x7f25b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:42.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.715+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f25ac0e2e70 con 0x7f25c0103600 2024-05-20T20:57:42.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.715+0000 7f25c4ddf700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25b00b2900 0x7f25b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.716+0000 7f25c4ddf700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25b00b2900 0x7f25b00b4dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005e90 tx=0x7f25b4005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:42.719+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25ac013070 con 0x7f25c0103600 2024-05-20T20:57:42.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:42 smithi026 ceph-mon[31990]: pgmap v435: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T20:57:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 -- 172.21.15.26:0/4012399380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 msgr2=0x7f913010af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 --2- 172.21.15.26:0/4012399380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f913010af80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9128009cd0 tx=0x7f912802a8e0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 -- 172.21.15.26:0/4012399380 shutdown_connections 2024-05-20T20:57:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 --2- 172.21.15.26:0/4012399380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f913010af80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 --2- 172.21.15.26:0/4012399380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 0x7f91301041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 --2- 172.21.15.26:0/4012399380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9130100d60 0x7f9130101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 -- 172.21.15.26:0/4012399380 >> 172.21.15.26:0/4012399380 conn(0x7f91300fc6b0 msgr2=0x7f91300fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 -- 172.21.15.26:0/4012399380 shutdown_connections 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.114+0000 7f913594d640 1 -- 172.21.15.26:0/4012399380 wait complete. 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 Processor -- start 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 -- start start 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9130100d60 0x7f9130103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 0x7f9130101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f9130102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f913010e4a0 con 0x7f91301046f0 2024-05-20T20:57:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f913010e320 con 0x7f9130100d60 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f913594d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f913010e620 con 0x7f9130103d30 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f912f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f9130102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.115+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 0x7f9130101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 0x7f9130101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59592/0 (socket says 172.21.15.26:59592) 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f9130102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37504/0 (socket says 172.21.15.26:37504) 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912e7fc640 1 -- 172.21.15.26:0/23223556 learned_addr learned my addr 172.21.15.26:0/23223556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912effd640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9130100d60 0x7f9130103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 -- 172.21.15.26:0/23223556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 msgr2=0x7f9130101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 0x7f9130101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 -- 172.21.15.26:0/23223556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9130100d60 msgr2=0x7f9130103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9130100d60 0x7f9130103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 -- 172.21.15.26:0/23223556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9130102ae0 con 0x7f91301046f0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f912f7fe640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f9130102220 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9128009bf0 tx=0x7f9128003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f913494b640 1 -- 172.21.15.26:0/23223556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128004040 con 0x7f91301046f0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.116+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91301a2c80 con 0x7f91301046f0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.117+0000 7f913494b640 1 -- 172.21.15.26:0/23223556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9128004670 con 0x7f91301046f0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.117+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91301a31a0 con 0x7f91301046f0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.117+0000 7f913494b640 1 -- 172.21.15.26:0/23223556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128030aa0 con 0x7f91301046f0 2024-05-20T20:57:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.120+0000 7f913494b640 1 -- 172.21.15.26:0/23223556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9128030c40 con 0x7f91301046f0 2024-05-20T20:57:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.120+0000 7f9138661740 1 Processor -- start 2024-05-20T20:57:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.120+0000 7f9138661740 1 -- start start 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f9138661740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 0x5617578edae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f9138661740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f9138661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f9138661740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56175785ec30 con 0x5617578ee0b0 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f9138661740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56175784e7d0 con 0x5617578ed6c0 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f9138661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561757852810 con 0x5617578f3220 2024-05-20T20:57:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f913494b640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91180b2970 0x7f91180b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f912effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f912effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37510/0 (socket says 172.21.15.26:37510) 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f912effd640 1 -- 172.21.15.26:0/3450978344 learned_addr learned my addr 172.21.15.26:0/3450978344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f912f7fe640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91180b2970 0x7f91180b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f912e7fc640 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f913494b640 1 -- 172.21.15.26:0/23223556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f91281030a0 con 0x7f91301046f0 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.121+0000 7f912effd640 1 -- 172.21.15.26:0/3450978344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 msgr2=0x5617578f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912effd640 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912effd640 1 -- 172.21.15.26:0/3450978344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 msgr2=0x5617578edae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912effd640 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 0x5617578edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912effd640 1 -- 172.21.15.26:0/3450978344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561757563210 con 0x5617578ee0b0 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912e7fc640 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912f7fe640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91180b2970 0x7f91180b4e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f91280ca300 tx=0x7f91280beed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f912effd640 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578f2bb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9120009ca0 tx=0x7f9120011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafbac03b5c946ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f9115ffb640 1 -- 172.21.15.26:0/3450978344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120004220 con 0x5617578ee0b0 2024-05-20T20:57:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f9115ffb640 1 -- 172.21.15.26:0/3450978344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91200043c0 con 0x5617578ee0b0 2024-05-20T20:57:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.122+0000 7f9115ffb640 1 -- 172.21.15.26:0/3450978344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91200197a0 con 0x5617578ee0b0 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 -- 172.21.15.26:0/3450978344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 msgr2=0x5617578f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578f2bb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9120009ca0 tx=0x7f9120011810 comp rx=0 tx=0).stop 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 -- 172.21.15.26:0/3450978344 shutdown_connections 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578f2bb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 --2- 172.21.15.26:0/3450978344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 0x5617578edae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 -- 172.21.15.26:0/3450978344 >> 172.21.15.26:0/3450978344 conn(0x5617578e8d30 msgr2=0x5617578e9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.123+0000 7f9138661740 1 -- 172.21.15.26:0/3450978344 shutdown_connections 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.124+0000 7f9138661740 1 -- 172.21.15.26:0/3450978344 wait complete. 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.124+0000 7f9138661740 1 -- start start 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.124+0000 7f9138661740 1 Processor -- start 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.124+0000 7f9138661740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 0x5617578f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f9138661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ee0b0 0x5617578f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f9138661740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 0x5617578fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 0x5617578fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 0x5617578fedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37518/0 (socket says 172.21.15.26:37518) 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f912effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ee0b0 0x5617578f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f912e7fc640 1 -- 172.21.15.26:0/4187913067 learned_addr learned my addr 172.21.15.26:0/4187913067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561757534790 con 0x5617578f3220 2024-05-20T20:57:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56175780a890 con 0x5617578ed6c0 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.125+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5617578f61d0 con 0x5617578ee0b0 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912e7fc640 1 -- 172.21.15.26:0/4187913067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ee0b0 msgr2=0x5617578f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912e7fc640 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ee0b0 0x5617578f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912e7fc640 1 -- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 msgr2=0x5617578f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912e7fc640 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 0x5617578f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912e7fc640 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56175762f620 con 0x5617578f3220 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912effd640 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ee0b0 0x5617578f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f9138661740 10 client.0 _ll_register_callbacks cb 0x561757562a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91200043c0 con 0x5617578f3220 2024-05-20T20:57:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f912e7fc640 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 0x5617578fedd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9124010a90 tx=0x7f9124010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea51e91da5fb3d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.126+0000 7f910effd640 10 client.15813 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.127+0000 7f910effd640 1 -- 172.21.15.26:0/4187913067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912401f060 con 0x5617578f3220 2024-05-20T20:57:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.127+0000 7f910effd640 1 -- 172.21.15.26:0/4187913067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f912400fc20 con 0x5617578f3220 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.127+0000 7f910effd640 1 -- 172.21.15.26:0/4187913067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f912400fe80 con 0x5617578f3220 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.127+0000 7f9138661740 10 client.15813 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.127+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=26}) -- 0x561757901d00 con 0x5617578f3220 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f910effd640 1 -- 172.21.15.26:0/4187913067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9124019030 con 0x5617578f3220 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f9138661740 10 client.15813 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f9138661740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f9138661740 20 client.15813 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f9138661740 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757905840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f9138661740 4 client.15813 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.128+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561757860b80 con 0x5617579033d0 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.129+0000 7f912effd640 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757905840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.129+0000 7f910effd640 10 client.15813 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.129+0000 7f912effd640 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757905840 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab1c42b8ec8577a server_cookie=648e4583deb411e6 in_seq=0 out_seq=0 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.130+0000 7f910effd640 1 -- 172.21.15.26:0/4187913067 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561757860b80 con 0x5617579033d0 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.130+0000 7f910effd640 10 client.15813 handle_command_reply: tid=0 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.130+0000 7f9138661740 1 client.15813 shutdown 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.130+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 msgr2=0x5617578fedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.130+0000 7f9138661740 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 0x5617578fedd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9124010a90 tx=0x7f9124010f50 comp rx=0 tx=0).stop 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 shutdown_connections 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757905840 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578f3220 0x5617578fedd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ee0b0 0x5617578f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 --2- 172.21.15.26:0/4187913067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ed6c0 0x5617578f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f910effd640 0 client.15813 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 >> 172.21.15.26:0/4187913067 conn(0x5617578e8d30 msgr2=0x5617578e7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 shutdown_connections 2024-05-20T20:57:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 1 -- 172.21.15.26:0/4187913067 wait complete. 2024-05-20T20:57:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.131+0000 7f9138661740 20 client.15813 trim_cache size 0 max 16384 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 Processor -- start 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 -- start start 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 0x5617578e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578f3220 0x5617578ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91301ae9a0 con 0x5617578ee0b0 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f9138661740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91301ac270 con 0x5617578f3220 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f912effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.162+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 0x5617578e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 0x5617578e8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59612/0 (socket says 172.21.15.26:59612) 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578fed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37530/0 (socket says 172.21.15.26:37530) 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 -- 172.21.15.26:0/2251655526 learned_addr learned my addr 172.21.15.26:0/2251655526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f9138661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91301aa0c0 con 0x5617578ed6c0 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578f3220 0x5617578ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 -- 172.21.15.26:0/2251655526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578f3220 msgr2=0x5617578ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578f3220 0x5617578ebc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 -- 172.21.15.26:0/2251655526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 msgr2=0x5617578fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 -- 172.21.15.26:0/2251655526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617578f8980 con 0x5617578ed6c0 2024-05-20T20:57:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912effd640 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.163+0000 7f912e7fc640 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 0x5617578e8730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9124005d80 tx=0x7f9124014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d99d2ba6d4c882fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f91157fa640 1 -- 172.21.15.26:0/2251655526 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912401f070 con 0x5617578ed6c0 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f91157fa640 1 -- 172.21.15.26:0/2251655526 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91200043c0 con 0x5617578ed6c0 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f91157fa640 1 -- 172.21.15.26:0/2251655526 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124000e60 con 0x5617578ed6c0 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 -- 172.21.15.26:0/2251655526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 msgr2=0x5617578e8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 0x5617578e8730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9124005d80 tx=0x7f9124014000 comp rx=0 tx=0).stop 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 -- 172.21.15.26:0/2251655526 shutdown_connections 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578f3220 0x5617578ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ee0b0 0x5617578fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 --2- 172.21.15.26:0/2251655526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578ed6c0 0x5617578e8730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.164+0000 7f9138661740 1 -- 172.21.15.26:0/2251655526 >> 172.21.15.26:0/2251655526 conn(0x5617578e8d30 msgr2=0x5617578e3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.165+0000 7f9138661740 1 -- 172.21.15.26:0/2251655526 shutdown_connections 2024-05-20T20:57:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.165+0000 7f9138661740 1 -- 172.21.15.26:0/2251655526 wait complete. 2024-05-20T20:57:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.165+0000 7f9138661740 1 -- start start 2024-05-20T20:57:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.165+0000 7f9138661740 1 Processor -- start 2024-05-20T20:57:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.165+0000 7f9138661740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 0x5617578fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.165+0000 7f9138661740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ee0b0 0x5617578fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f9138661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f9138661740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91301aa730 con 0x5617578ed6c0 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f9138661740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91301abba0 con 0x5617578ee0b0 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f9138661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91301aae00 con 0x5617578f3220 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 0x5617578fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f9138661740 10 client.0 _ll_register_callbacks cb 0x7f91240098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 0x5617578fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37536/0 (socket says 172.21.15.26:37536) 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 -- 172.21.15.26:0/991051054 learned_addr learned my addr 172.21.15.26:0/991051054 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912effd640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ee0b0 0x5617578fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 -- 172.21.15.26:0/991051054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 msgr2=0x5617578fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 -- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ee0b0 msgr2=0x5617578fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ee0b0 0x5617578fd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 -- 172.21.15.26:0/991051054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617578fe620 con 0x5617578ed6c0 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912effd640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ee0b0 0x5617578fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f912e7fc640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 0x5617578fcae0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f912401a470 tx=0x7f912401a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa89f3af03ba75e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.166+0000 7f910effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f910effd640 1 -- 172.21.15.26:0/991051054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912401f070 con 0x5617578ed6c0 2024-05-20T20:57:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f910effd640 1 -- 172.21.15.26:0/991051054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9124005070 con 0x5617578ed6c0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91200043c0 con 0x5617578ed6c0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f910effd640 1 -- 172.21.15.26:0/991051054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9124005270 con 0x5617578ed6c0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f9138661740 10 client.15819 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56175796e5a0 con 0x5617578ed6c0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.167+0000 7f910effd640 1 -- 172.21.15.26:0/991051054 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91200043c0 con 0x5617578ed6c0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f910effd640 1 -- 172.21.15.26:0/991051054 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9124019040 con 0x5617578ed6c0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f9138661740 10 client.15819 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f9138661740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f9138661740 20 client.15819 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f9138661740 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757970f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f9138661740 4 client.15819 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56175785a360 con 0x5617579033d0 2024-05-20T20:57:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.168+0000 7f912effd640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757970f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.169+0000 7f912effd640 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757970f90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0fcc7fb23a0f38 server_cookie=d213f5e5709cecd1 in_seq=0 out_seq=0 2024-05-20T20:57:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.169+0000 7f910effd640 10 client.15819 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f910effd640 1 -- 172.21.15.26:0/991051054 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56175785a360 con 0x5617579033d0 2024-05-20T20:57:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f910effd640 10 client.15819 handle_command_reply: tid=0 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 client.15819 shutdown 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 msgr2=0x5617578fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 0x5617578fcae0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f912401a470 tx=0x7f912401a6d0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 shutdown_connections 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5617579033d0 0x561757970f90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617578f3220 0x5617578fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5617578ee0b0 0x5617578fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 --2- 172.21.15.26:0/991051054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617578ed6c0 0x5617578fcae0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 >> 172.21.15.26:0/991051054 conn(0x5617578e8d30 msgr2=0x5617578f2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f910effd640 0 client.15819 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 shutdown_connections 2024-05-20T20:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 1 -- 172.21.15.26:0/991051054 wait complete. 2024-05-20T20:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.173+0000 7f9138661740 20 client.15819 trim_cache size 0 max 16384 2024-05-20T20:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91180b2970 msgr2=0x7f91180b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91180b2970 0x7f91180b4e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f91280ca300 tx=0x7f91280beed0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 msgr2=0x7f9130102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f9130102220 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9128009bf0 tx=0x7f9128003ba0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 shutdown_connections 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91180b2970 0x7f91180b4e30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91301046f0 0x7f9130102220 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9130103d30 0x7f9130101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 --2- 172.21.15.26:0/23223556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9130100d60 0x7f9130103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 >> 172.21.15.26:0/23223556 conn(0x7f91300fc6b0 msgr2=0x7f9130107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 shutdown_connections 2024-05-20T20:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:43.174+0000 7f913594d640 1 -- 172.21.15.26:0/23223556 wait complete. 2024-05-20T20:57:43.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:42 smithi055 ceph-mon[35213]: pgmap v435: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T20:57:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.250+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25c0103c90 con 0x7f25c0103600 2024-05-20T20:57:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.252+0000 7f25bdffb700 1 -- 172.21.15.26:0/146165098 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f25ac0aabc0 con 0x7f25c0103600 2024-05-20T20:57:43.252 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:57:43.253 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:57:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25b00b2900 msgr2=0x7f25b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25b00b2900 0x7f25b00b4dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005e90 tx=0x7f25b4005d40 comp rx=0 tx=0).stop 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 msgr2=0x7f25c010fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010fd70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005910 tx=0x7f25ac0047b0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 shutdown_connections 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25b00b2900 0x7f25b00b4dc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c0102c40 0x7f25c0111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25c01084c0 0x7f25c01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 --2- 172.21.15.26:0/146165098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c0103600 0x7f25c010fd70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.255+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 >> 172.21.15.26:0/146165098 conn(0x7f25c0078ab0 msgr2=0x7f25c0100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.256+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 shutdown_connections 2024-05-20T20:57:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:43.256+0000 7f25c7043700 1 -- 172.21.15.26:0/146165098 wait complete. 2024-05-20T20:57:43.258 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:57:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:42 smithi144 ceph-mon[35660]: pgmap v435: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T20:57:44.013 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 96e2b476 --release --await' 2024-05-20T20:57:44.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 96e2b476 --release --await 2024-05-20T20:57:44.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:43 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/146165098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:44.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:43 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/146165098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:44.281 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:57:44.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:57:44.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:43 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/146165098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:44.398 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:44.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:44 smithi026 ceph-mon[31990]: pgmap v436: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:45.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:44 smithi055 ceph-mon[35213]: pgmap v436: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:45.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:44 smithi144 ceph-mon[35660]: pgmap v436: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:46.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.925+0000 7f96db9e1700 1 -- 172.21.15.26:0/3895346325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 msgr2=0x7f96d41088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.925+0000 7f96db9e1700 1 --2- 172.21.15.26:0/3895346325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d41088a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009990 tx=0x7f96d0009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.926+0000 7f96db9e1700 1 -- 172.21.15.26:0/3895346325 shutdown_connections 2024-05-20T20:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.926+0000 7f96db9e1700 1 --2- 172.21.15.26:0/3895346325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d41088a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.926+0000 7f96db9e1700 1 --2- 172.21.15.26:0/3895346325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.926+0000 7f96db9e1700 1 --2- 172.21.15.26:0/3895346325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96d4102c40 0x7f96d41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.926+0000 7f96db9e1700 1 -- 172.21.15.26:0/3895346325 >> 172.21.15.26:0/3895346325 conn(0x7f96d4078ab0 msgr2=0x7f96d40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.927+0000 7f96db9e1700 1 -- 172.21.15.26:0/3895346325 shutdown_connections 2024-05-20T20:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.927+0000 7f96db9e1700 1 -- 172.21.15.26:0/3895346325 wait complete. 2024-05-20T20:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.927+0000 7f96db9e1700 1 Processor -- start 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.927+0000 7f96db9e1700 1 -- start start 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96db9e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96d4102c40 0x7f96d419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96db9e1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96db9e1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96db9e1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41124d0 con 0x7f96d41084c0 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96db9e1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4112350 con 0x7f96d4103600 2024-05-20T20:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96db9e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4112650 con 0x7f96d4102c40 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42016/0 (socket says 172.21.15.26:42016) 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 -- 172.21.15.26:0/2301582747 learned_addr learned my addr 172.21.15.26:0/2301582747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 -- 172.21.15.26:0/2301582747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96d4102c40 msgr2=0x7f96d419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96d4102c40 0x7f96d419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 -- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 msgr2=0x7f96d4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d4199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d8f7c700 1 -- 172.21.15.26:0/2301582747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d419a0c0 con 0x7f96d4103600 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.929+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d419fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96c4007f30 tx=0x7f96c400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.929+0000 7f96ca7fc700 1 -- 172.21.15.26:0/2301582747 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c400de40 con 0x7f96d4103600 2024-05-20T20:57:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.929+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d419a3b0 con 0x7f96d4103600 2024-05-20T20:57:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.928+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.929+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d4104040 con 0x7f96d4103600 2024-05-20T20:57:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.929+0000 7f96ca7fc700 1 -- 172.21.15.26:0/2301582747 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96c40044b0 con 0x7f96d4103600 2024-05-20T20:57:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.929+0000 7f96ca7fc700 1 -- 172.21.15.26:0/2301582747 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4005280 con 0x7f96d4103600 2024-05-20T20:57:46.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.933+0000 7f96ca7fc700 1 -- 172.21.15.26:0/2301582747 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f96c40049a0 con 0x7f96d4103600 2024-05-20T20:57:46.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.934+0000 7f96ca7fc700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f96c00b2980 0x7f96c00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.934+0000 7f96ca7fc700 1 -- 172.21.15.26:0/2301582747 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f96c40ea1c0 con 0x7f96d4103600 2024-05-20T20:57:46.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.934+0000 7f96d977d700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f96c00b2980 0x7f96c00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 Processor -- start 2024-05-20T20:57:46.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 -- start start 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f45eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f45a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f466620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d79f3b7e70 con 0x55d79f45e720 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d79f3a7a10 con 0x55d79f459bf0 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96e6277b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d79f3aba50 con 0x55d79f45a5b0 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f45a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f45a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42024/0 (socket says 172.21.15.26:42024) 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3517683729 learned_addr learned my addr 172.21.15.26:0/3517683729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.935+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f466620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3517683729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 msgr2=0x55d79f466620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f466620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3517683729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 msgr2=0x55d79f45eb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f45eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3517683729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79f3510b0 con 0x55d79f459bf0 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f45a070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96c40ad7a0 tx=0x7f96c40ad710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a22029d75438fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96d977d700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f96c00b2980 0x7f96c00b4e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f96d00098e0 tx=0x7f96d000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96bf7fe700 1 -- 172.21.15.26:0/3517683729 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4025070 con 0x55d79f459bf0 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96bf7fe700 1 -- 172.21.15.26:0/3517683729 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96c40b8e50 con 0x55d79f459bf0 2024-05-20T20:57:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.936+0000 7f96bf7fe700 1 -- 172.21.15.26:0/3517683729 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40992b0 con 0x55d79f459bf0 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 -- 172.21.15.26:0/3517683729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 msgr2=0x55d79f45a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f45a070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96c40ad7a0 tx=0x7f96c40ad710 comp rx=0 tx=0).stop 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 -- 172.21.15.26:0/3517683729 shutdown_connections 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f45eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f466620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3517683729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f45a070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 -- 172.21.15.26:0/3517683729 >> 172.21.15.26:0/3517683729 conn(0x55d79f385340 msgr2=0x55d79f4548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 -- 172.21.15.26:0/3517683729 shutdown_connections 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.937+0000 7f96e6277b80 1 -- 172.21.15.26:0/3517683729 wait complete. 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.938+0000 7f96e6277b80 1 -- start start 2024-05-20T20:57:46.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.938+0000 7f96e6277b80 1 Processor -- start 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f466230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f463530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f463a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d79f2ae0a0 con 0x55d79f45e720 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d79f30fea0 con 0x55d79f459bf0 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d79f466c20 con 0x55d79f45a5b0 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96e6277b80 10 client.0 _ll_register_callbacks cb 0x55d79f342a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f463530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f463530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59658/0 (socket says 172.21.15.26:59658) 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3025481620 learned_addr learned my addr 172.21.15.26:0/3025481620 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f463a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.939+0000 7f96d977d700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f466230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 msgr2=0x55d79f466230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f466230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3025481620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 msgr2=0x55d79f463a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f463a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d977d700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f466230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3025481620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79f2f79d0 con 0x55d79f45a5b0 2024-05-20T20:57:46.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f463530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96c40ea130 tx=0x7f96c40b1f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5782b7433115e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96bcff9700 1 -- 172.21.15.26:0/3025481620 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4025050 con 0x55d79f45a5b0 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96bcff9700 1 -- 172.21.15.26:0/3025481620 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f96c40992c0 con 0x55d79f45a5b0 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.940+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c40b8e50 con 0x55d79f45a5b0 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96bcff9700 1 -- 172.21.15.26:0/3025481620 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96c40ba0f0 con 0x55d79f45a5b0 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96e6277b80 10 client.25342 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:46.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d79f361860 con 0x55d79f45a5b0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96bcff9700 1 -- 172.21.15.26:0/3025481620 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96c40ba290 con 0x55d79f45a5b0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96bcff9700 1 -- 172.21.15.26:0/3025481620 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f96c4005520 con 0x55d79f45a5b0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96e6277b80 10 client.25342 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96e6277b80 10 client.25342 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96e6277b80 20 client.25342 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.941+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4735a0 0x55d79f475990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.942+0000 7f96e6277b80 4 client.25342 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.942+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f96c40b8e50 con 0x55d79f4735a0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.942+0000 7f96d977d700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4735a0 0x55d79f475990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.942+0000 7f96bcff9700 10 client.25342 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.943+0000 7f96d977d700 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4735a0 0x55d79f475990 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf71baade9a4395 server_cookie=9582ee6b37dc527b in_seq=0 out_seq=0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.943+0000 7f96bcff9700 1 -- 172.21.15.26:0/3025481620 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f96c40b8e50 con 0x55d79f4735a0 2024-05-20T20:57:46.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.943+0000 7f96bcff9700 10 client.25342 handle_command_reply: tid=0 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.943+0000 7f96e6277b80 1 client.25342 shutdown 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 msgr2=0x55d79f463530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f463530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96c40ea130 tx=0x7f96c40b1f20 comp rx=0 tx=0).stop 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 shutdown_connections 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4735a0 0x55d79f475990 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f45e720 0x55d79f463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96bcff9700 0 client.25342 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f459bf0 0x55d79f466230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3025481620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45a5b0 0x55d79f463530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 >> 172.21.15.26:0/3025481620 conn(0x55d79f385340 msgr2=0x55d79f458be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.944+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 shutdown_connections 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.945+0000 7f96e6277b80 1 -- 172.21.15.26:0/3025481620 wait complete. 2024-05-20T20:57:46.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.945+0000 7f96e6277b80 20 client.25342 trim_cache size 0 max 16384 2024-05-20T20:57:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 Processor -- start 2024-05-20T20:57:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 -- start start 2024-05-20T20:57:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f44ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f45dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f45c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41ae480 con 0x55d79f459bf0 2024-05-20T20:57:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41abd10 con 0x55d79f45a5b0 2024-05-20T20:57:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96e6277b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41a9b80 con 0x55d79f45e720 2024-05-20T20:57:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96d977d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f44ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96d977d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f44ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37584/0 (socket says 172.21.15.26:37584) 2024-05-20T20:57:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.966+0000 7f96d977d700 1 -- 172.21.15.26:0/2156972409 learned_addr learned my addr 172.21.15.26:0/2156972409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f45c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d977d700 1 -- 172.21.15.26:0/2156972409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 msgr2=0x55d79f45c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d977d700 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f45c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d977d700 1 -- 172.21.15.26:0/2156972409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 msgr2=0x55d79f45dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d8f7c700 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f45dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d977d700 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f45dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d977d700 1 -- 172.21.15.26:0/2156972409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79f45e290 con 0x55d79f459bf0 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.967+0000 7f96d977d700 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f44ee70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009990 tx=0x7f96d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba4a99da097b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.968+0000 7f96beffd700 1 -- 172.21.15.26:0/2156972409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0036020 con 0x55d79f459bf0 2024-05-20T20:57:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.968+0000 7f96beffd700 1 -- 172.21.15.26:0/2156972409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96c40b8e50 con 0x55d79f459bf0 2024-05-20T20:57:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.968+0000 7f96beffd700 1 -- 172.21.15.26:0/2156972409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0031690 con 0x55d79f459bf0 2024-05-20T20:57:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.968+0000 7f96e6277b80 1 -- 172.21.15.26:0/2156972409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 msgr2=0x55d79f44ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.968+0000 7f96e6277b80 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f44ee70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009990 tx=0x7f96d0010000 comp rx=0 tx=0).stop 2024-05-20T20:57:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 -- 172.21.15.26:0/2156972409 shutdown_connections 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f44ee70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f45c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 --2- 172.21.15.26:0/2156972409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f45dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 -- 172.21.15.26:0/2156972409 >> 172.21.15.26:0/2156972409 conn(0x55d79f385340 msgr2=0x55d79f45d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 -- 172.21.15.26:0/2156972409 shutdown_connections 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 -- 172.21.15.26:0/2156972409 wait complete. 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 -- start start 2024-05-20T20:57:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.969+0000 7f96e6277b80 1 Processor -- start 2024-05-20T20:57:46.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f476f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f4774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f4d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41aa1d0 con 0x55d79f459bf0 2024-05-20T20:57:46.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41ab640 con 0x55d79f45a5b0 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41aa8a0 con 0x55d79f45e720 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96d977d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f476f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96e6277b80 10 client.0 _ll_register_callbacks cb 0x55d79f342a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f4774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96d8f7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f4774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42054/0 (socket says 172.21.15.26:42054) 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96d8f7c700 1 -- 172.21.15.26:0/3199251302 learned_addr learned my addr 172.21.15.26:0/3199251302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.970+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f4d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d9f7e700 1 -- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 msgr2=0x55d79f4774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f4774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d9f7e700 1 -- 172.21.15.26:0/3199251302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 msgr2=0x55d79f476f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f476f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d9f7e700 1 -- 172.21.15.26:0/3199251302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79f4d7460 con 0x55d79f45e720 2024-05-20T20:57:46.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d977d700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f476f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96d9f7e700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f4d6e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96cc007fb0 tx=0x7f96cc009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d55d8cdc8c329d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96bcff9700 1 -- 172.21.15.26:0/3199251302 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc017c50 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96bcff9700 1 -- 172.21.15.26:0/3199251302 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f96cc017df0 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.971+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c40b8e50 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.972+0000 7f96bcff9700 1 -- 172.21.15.26:0/3199251302 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96cc021470 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.972+0000 7f96e6277b80 10 client.25348 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.972+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d79f4de890 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.972+0000 7f96bcff9700 1 -- 172.21.15.26:0/3199251302 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc021aa0 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96bcff9700 1 -- 172.21.15.26:0/3199251302 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f96cc02a050 con 0x55d79f45e720 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96e6277b80 10 client.25348 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96e6277b80 10 client.25348 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:57:46.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96e6277b80 20 client.25348 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:46.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4def80 0x55d79f4e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:46.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96e6277b80 4 client.25348 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "96e2b476", "release": true, "await": true}] 2024-05-20T20:57:46.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "96e2b476", "release": true, "await": true}) -- 0x7f96c40b8e50 con 0x55d79f4def80 2024-05-20T20:57:46.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.973+0000 7f96d977d700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4def80 0x55d79f4e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:46.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.974+0000 7f96bcff9700 10 client.25348 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:46.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.974+0000 7f96d977d700 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4def80 0x55d79f4e13a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef04ff3aa64d02c server_cookie=f3d50e7a8202ab28 in_seq=0 out_seq=0 2024-05-20T20:57:46.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.975+0000 7f96bcff9700 1 -- 172.21.15.26:0/3199251302 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f96c40b8e50 con 0x55d79f4def80 2024-05-20T20:57:46.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.975+0000 7f96bcff9700 10 client.25348 handle_command_reply: tid=0 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.975+0000 7f96e6277b80 1 client.25348 shutdown 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 msgr2=0x55d79f4d6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f4d6e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96cc007fb0 tx=0x7f96cc009f80 comp rx=0 tx=0).stop 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 shutdown_connections 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d79f4def80 0x55d79f4e13a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d79f459bf0 0x55d79f476f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96bcff9700 0 client.25348 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d79f45e720 0x55d79f4d6e20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 --2- 172.21.15.26:0/3199251302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d79f45a5b0 0x55d79f4774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 >> 172.21.15.26:0/3199251302 conn(0x55d79f385340 msgr2=0x55d79f464d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 shutdown_connections 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.976+0000 7f96e6277b80 1 -- 172.21.15.26:0/3199251302 wait complete. 2024-05-20T20:57:46.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96e6277b80 20 client.25348 trim_cache size 0 max 16384 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 8, 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "96e2b476": { 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 8, 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:57:46.980 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 138.8, 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:57:46.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:57:46.982 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f96c00b2980 msgr2=0x7f96c00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f96c00b2980 0x7f96c00b4e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f96d00098e0 tx=0x7f96d000b610 comp rx=0 tx=0).stop 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 msgr2=0x7f96d419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d419fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96c4007f30 tx=0x7f96c400dc60 comp rx=0 tx=0).stop 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 shutdown_connections 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f96c00b2980 0x7f96c00b4e40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d41084c0 0x7f96d4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.977+0000 7f96db9e1700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96d4102c40 0x7f96d419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.978+0000 7f96db9e1700 1 --2- 172.21.15.26:0/2301582747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96d4103600 0x7f96d419fc90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.978+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 >> 172.21.15.26:0/2301582747 conn(0x7f96d4078ab0 msgr2=0x7f96d41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.978+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 shutdown_connections 2024-05-20T20:57:46.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:46.978+0000 7f96db9e1700 1 -- 172.21.15.26:0/2301582747 wait complete. 2024-05-20T20:57:47.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:46 smithi055 ceph-mon[35213]: pgmap v437: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-20T20:57:47.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:46 smithi026 ceph-mon[31990]: pgmap v437: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-20T20:57:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:46 smithi144 ceph-mon[35660]: pgmap v437: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-20T20:57:48.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.338+0000 7f8bcdfce700 1 -- 172.21.15.26:0/1982622019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc806c470 msgr2=0x7f8bc81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.338+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/1982622019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc806c470 0x7f8bc81103c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000b0a0 tx=0x7f8bc000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.338+0000 7f8bcdfce700 1 -- 172.21.15.26:0/1982622019 shutdown_connections 2024-05-20T20:57:48.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.338+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/1982622019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc806c470 0x7f8bc81103c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.339+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/1982622019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc81095a0 0x7f8bc8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.339+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/1982622019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc806bab0 0x7f8bc806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.339+0000 7f8bcdfce700 1 -- 172.21.15.26:0/1982622019 >> 172.21.15.26:0/1982622019 conn(0x7f8bc80793b0 msgr2=0x7f8bc80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:48.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.339+0000 7f8bcdfce700 1 -- 172.21.15.26:0/1982622019 shutdown_connections 2024-05-20T20:57:48.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.339+0000 7f8bcdfce700 1 -- 172.21.15.26:0/1982622019 wait complete. 2024-05-20T20:57:48.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.340+0000 7f8bcdfce700 1 Processor -- start 2024-05-20T20:57:48.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.341+0000 7f8bcdfce700 1 -- start start 2024-05-20T20:57:48.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.341+0000 7f8bcdfce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 0x7f8bc819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.341+0000 7f8bcdfce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc806c470 0x7f8bc819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.341+0000 7f8bcdfce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc81095a0 0x7f8bc8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.341+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 0x7f8bc819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.341+0000 7f8bcdfce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81135c0 con 0x7f8bc81095a0 2024-05-20T20:57:48.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bcdfce700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8113440 con 0x7f8bc806bab0 2024-05-20T20:57:48.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bcdfce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8113740 con 0x7f8bc806c470 2024-05-20T20:57:48.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 0x7f8bc819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42068/0 (socket says 172.21.15.26:42068) 2024-05-20T20:57:48.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 -- 172.21.15.26:0/361260517 learned_addr learned my addr 172.21.15.26:0/361260517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc806c470 0x7f8bc819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc81095a0 0x7f8bc8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 -- 172.21.15.26:0/361260517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc806c470 msgr2=0x7f8bc819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc806c470 0x7f8bc819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 -- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc81095a0 msgr2=0x7f8bc8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc81095a0 0x7f8bc8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.342+0000 7f8bc77fe700 1 -- 172.21.15.26:0/361260517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc819a0a0 con 0x7f8bc806bab0 2024-05-20T20:57:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.343+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 0x7f8bc819f780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8007b10 tx=0x7f8bb8007e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.343+0000 7f8bc4ff9700 1 -- 172.21.15.26:0/361260517 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8004020 con 0x7f8bc806bab0 2024-05-20T20:57:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.343+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc819a330 con 0x7f8bc806bab0 2024-05-20T20:57:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.343+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc8102430 con 0x7f8bc806bab0 2024-05-20T20:57:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.343+0000 7f8bc4ff9700 1 -- 172.21.15.26:0/361260517 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bb8004830 con 0x7f8bc806bab0 2024-05-20T20:57:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.343+0000 7f8bc4ff9700 1 -- 172.21.15.26:0/361260517 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8005480 con 0x7f8bc806bab0 2024-05-20T20:57:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.347+0000 7f8bc4ff9700 1 -- 172.21.15.26:0/361260517 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8bb8021020 con 0x7f8bc806bab0 2024-05-20T20:57:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.348+0000 7f8bc4ff9700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b2980 0x7f8bb00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.348+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b2980 0x7f8bb00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.348+0000 7f8bc4ff9700 1 -- 172.21.15.26:0/361260517 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f8bb80e5450 con 0x7f8bc806bab0 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.348+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b2980 0x7f8bb00b4e40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8079af0 tx=0x7f8bbc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.348+0000 7f8bd8864b80 1 Processor -- start 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 -- start start 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88222ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b8821d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8821dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560b88172df0 con 0x560b882234a0 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560b88162990 con 0x560b8821d750 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bd8864b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b881669d0 con 0x560b88222af0 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b8821d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88222ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88222ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59714/0 (socket says 172.21.15.26:59714) 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2676881412 learned_addr learned my addr 172.21.15.26:0/2676881412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.349+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8821dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2676881412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 msgr2=0x560b8821dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8821dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2676881412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 msgr2=0x560b8821d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b8821d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2676881412 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8810c030 con 0x560b88222af0 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b8821d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88222ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80ac940 tx=0x7f8bb8004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9814dc8522e276a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8badffb700 1 -- 172.21.15.26:0/2676881412 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb801e040 con 0x560b88222af0 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.350+0000 7f8badffb700 1 -- 172.21.15.26:0/2676881412 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bb80ad0b0 con 0x560b88222af0 2024-05-20T20:57:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8badffb700 1 -- 172.21.15.26:0/2676881412 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80cf110 con 0x560b88222af0 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2676881412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 msgr2=0x560b88222ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88222ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80ac940 tx=0x7f8bb8004fb0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2676881412 shutdown_connections 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8821dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88222ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2676881412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b8821d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2676881412 >> 172.21.15.26:0/2676881412 conn(0x560b881402c0 msgr2=0x560b8820f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2676881412 shutdown_connections 2024-05-20T20:57:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.351+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2676881412 wait complete. 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.352+0000 7f8bd8864b80 1 -- start start 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.352+0000 7f8bd8864b80 1 Processor -- start 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.352+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b88229200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.352+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88226500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.352+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b88226a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b88229200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b88229200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42086/0 (socket says 172.21.15.26:42086) 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2750808099 learned_addr learned my addr 172.21.15.26:0/2750808099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b88226a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88226500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560b88069020 con 0x560b882234a0 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560b880cae20 con 0x560b8821d750 2024-05-20T20:57:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b88229f70 con 0x560b88222af0 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bd8864b80 10 client.0 _ll_register_callbacks cb 0x560b880fd9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2750808099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 msgr2=0x560b88226500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88226500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2750808099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 msgr2=0x560b88226a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b88226a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b880b2950 con 0x560b8821d750 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b88226a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b88229200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005e40 tx=0x7f8bb8004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9baa2eed16a42d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb80ad0b0 con 0x560b8821d750 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.353+0000 7f8ba6ffd700 10 client.25403 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/2750808099 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80c5070 con 0x560b8821d750 2024-05-20T20:57:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/2750808099 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f8bb80a8fa0 con 0x560b8821d750 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/2750808099 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bb80ac9b0 con 0x560b8821d750 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 10 client.25403 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560b8811c7e0 con 0x560b8821d750 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/2750808099 <== mon.2 v2:172.21.15.55:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8bb80adce0 con 0x560b8821d750 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 10 client.25403 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 20 client.25403 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882368a0 0x560b88238c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 4 client.25403 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.354+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8bb80ad0b0 con 0x560b882368a0 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.355+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882368a0 0x560b88238c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.355+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882368a0 0x560b88238c90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4050f77e0644aa8 server_cookie=6b14f93caf9bb712 in_seq=0 out_seq=0 2024-05-20T20:57:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.355+0000 7f8ba6ffd700 10 client.25403 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/2750808099 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f8bb80ad0b0 con 0x560b882368a0 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8ba6ffd700 10 client.25403 handle_command_reply: tid=0 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 client.25403 shutdown 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 msgr2=0x560b88229200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b88229200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005e40 tx=0x7f8bb8004f10 comp rx=0 tx=0).stop 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 shutdown_connections 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882368a0 0x560b88238c90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b882234a0 0x560b88226a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b88229200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/2750808099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b88222af0 0x560b88226500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 >> 172.21.15.26:0/2750808099 conn(0x560b881402c0 msgr2=0x560b8820eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.356+0000 7f8ba6ffd700 0 client.25403 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.357+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 shutdown_connections 2024-05-20T20:57:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.357+0000 7f8bd8864b80 1 -- 172.21.15.26:0/2750808099 wait complete. 2024-05-20T20:57:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.357+0000 7f8bd8864b80 20 client.25403 trim_cache size 0 max 16384 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.392+0000 7f8bd8864b80 1 Processor -- start 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.392+0000 7f8bd8864b80 1 -- start start 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.392+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8820f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.392+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 0x560b88221d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.392+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b88222240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 0x560b88221d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 0x560b88221d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37634/0 (socket says 172.21.15.26:37634) 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8820f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8820f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42100/0 (socket says 172.21.15.26:42100) 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc6ffd700 1 -- 172.21.15.26:0/3071741758 learned_addr learned my addr 172.21.15.26:0/3071741758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ae350 con 0x560b88222af0 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81abbe0 con 0x560b8821d750 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81a9a30 con 0x560b882234a0 2024-05-20T20:57:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b88222240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc7fff700 1 -- 172.21.15.26:0/3071741758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 msgr2=0x560b8820f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8820f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc7fff700 1 -- 172.21.15.26:0/3071741758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 msgr2=0x560b88221d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 0x560b88221d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc7fff700 1 -- 172.21.15.26:0/3071741758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b88210b00 con 0x560b882234a0 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.393+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 0x560b88221d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.394+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b88222240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009fd0 tx=0x7f8bc0003c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d710bb91bb74dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.394+0000 7f8bad7fa700 1 -- 172.21.15.26:0/3071741758 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0009490 con 0x560b882234a0 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.394+0000 7f8bad7fa700 1 -- 172.21.15.26:0/3071741758 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bc0003db0 con 0x560b882234a0 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.394+0000 7f8bad7fa700 1 -- 172.21.15.26:0/3071741758 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0005370 con 0x560b882234a0 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 msgr2=0x560b88222240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b88222240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009fd0 tx=0x7f8bc0003c40 comp rx=0 tx=0).stop 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 shutdown_connections 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b88222240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b8821d750 0x560b8820f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/3071741758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b88222af0 0x560b88221d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 >> 172.21.15.26:0/3071741758 conn(0x560b881402c0 msgr2=0x560b882215f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 shutdown_connections 2024-05-20T20:57:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 -- 172.21.15.26:0/3071741758 wait complete. 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.395+0000 7f8bd8864b80 1 -- start start 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.396+0000 7f8bd8864b80 1 Processor -- start 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.396+0000 7f8bd8864b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 0x560b8823bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.396+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 0x560b8823bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.396+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 0x560b8823bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37638/0 (socket says 172.21.15.26:37638) 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc77fe700 1 -- 172.21.15.26:0/1425591192 learned_addr learned my addr 172.21.15.26:0/1425591192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b88222af0 0x560b8823a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b8823a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81aa0a0 con 0x560b8821d750 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ab510 con 0x560b88222af0 2024-05-20T20:57:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81aa770 con 0x560b882234a0 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b88222af0 0x560b8823a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b8823a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bd8864b80 10 client.0 _ll_register_callbacks cb 0x560b880fd9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc77fe700 1 -- 172.21.15.26:0/1425591192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 msgr2=0x560b8823a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b8823a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc77fe700 1 -- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b88222af0 msgr2=0x560b8823a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b88222af0 0x560b8823a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc77fe700 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8823ab80 con 0x560b8821d750 2024-05-20T20:57:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.397+0000 7f8bc7fff700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b8823a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc0003db0 con 0x560b8821d750 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8bc77fe700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 0x560b8823bd20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b0e30 tx=0x7f8bb800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526811e0933c6785 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8ba6ffd700 10 client.15852 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/1425591192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb801e070 con 0x560b8821d750 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/1425591192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f8bb80c1e00 con 0x560b8821d750 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/1425591192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bb80a8870 con 0x560b8821d750 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8bd8864b80 10 client.15852 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.398+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560b882a1d00 con 0x560b8821d750 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/1425591192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8bb80abc10 con 0x560b8821d750 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bd8864b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bd8864b80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bd8864b80 20 client.15852 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882a2120 0x560b882a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bd8864b80 4 client.15852 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:57:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b88174d40 con 0x560b882a2120 2024-05-20T20:57:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.399+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882a2120 0x560b882a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.400+0000 7f8bc6ffd700 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882a2120 0x560b882a45c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f740c7bf9ef48b9c server_cookie=f92ffde596cfa694 in_seq=0 out_seq=0 2024-05-20T20:57:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.400+0000 7f8ba6ffd700 10 client.15852 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.401+0000 7f8ba6ffd700 1 -- 172.21.15.26:0/1425591192 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b88174d40 con 0x560b882a2120 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.401+0000 7f8ba6ffd700 10 client.15852 handle_command_reply: tid=0 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.401+0000 7f8bd8864b80 1 client.15852 shutdown 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.401+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 msgr2=0x560b8823bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.401+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 0x560b8823bd20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b0e30 tx=0x7f8bb800fa10 comp rx=0 tx=0).stop 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 shutdown_connections 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560b882a2120 0x560b882a45c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b8821d750 0x560b8823bd20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b882234a0 0x560b8823a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 --2- 172.21.15.26:0/1425591192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b88222af0 0x560b8823a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8ba6ffd700 0 client.15852 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 >> 172.21.15.26:0/1425591192 conn(0x560b881402c0 msgr2=0x560b88221e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 shutdown_connections 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 1 -- 172.21.15.26:0/1425591192 wait complete. 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.402+0000 7f8bd8864b80 20 client.15852 trim_cache size 0 max 16384 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "4273a831-30d5-403f-97bc-42420f4127f0", 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:57:48.405 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b2980 msgr2=0x7f8bb00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b2980 0x7f8bb00b4e40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8079af0 tx=0x7f8bbc00b410 comp rx=0 tx=0).stop 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 msgr2=0x7f8bc819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 0x7f8bc819f780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8007b10 tx=0x7f8bb8007e20 comp rx=0 tx=0).stop 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 shutdown_connections 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b2980 0x7f8bb00b4e40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc81095a0 0x7f8bc8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc806bab0 0x7f8bc819f780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 --2- 172.21.15.26:0/361260517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc806c470 0x7f8bc819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.403+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 >> 172.21.15.26:0/361260517 conn(0x7f8bc80793b0 msgr2=0x7f8bc8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.404+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 shutdown_connections 2024-05-20T20:57:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:48.404+0000 7f8bcdfce700 1 -- 172.21.15.26:0/361260517 wait complete. 2024-05-20T20:57:48.840 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '96e2b476', total seconds elapsed: 31.5 2024-05-20T20:57:49.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:48 smithi026 ceph-mon[31990]: pgmap v438: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:49.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:48 smithi055 ceph-mon[35213]: pgmap v438: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:49.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:48 smithi144 ceph-mon[35660]: pgmap v438: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T20:57:49.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:57:49.769 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:50.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:49 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:57:50.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:49 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:50.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:49 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:57:50.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:49 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:49 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:57:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:49 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:49 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:57:50.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:49 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:49 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:57:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:49 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:49 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:57:50.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:49 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:57:51.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:50 smithi055 ceph-mon[35213]: pgmap v439: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:51.201 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:50 smithi144 ceph-mon[35660]: pgmap v439: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:50 smithi026 ceph-mon[31990]: pgmap v439: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:52.023 INFO:tasks.workunit.client.0.smithi026.stderr:Updating files: 90% (11050/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-20T20:57:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 -- 172.21.15.26:0/279019745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 msgr2=0x7ff2d81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 --2- 172.21.15.26:0/279019745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d81081f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0099c0 tx=0x7ff2cc009cd0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 -- 172.21.15.26:0/279019745 shutdown_connections 2024-05-20T20:57:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 --2- 172.21.15.26:0/279019745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 0x7ff2d81104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 --2- 172.21.15.26:0/279019745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2d8103850 0x7ff2d8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 --2- 172.21.15.26:0/279019745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d81081f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 -- 172.21.15.26:0/279019745 >> 172.21.15.26:0/279019745 conn(0x7ff2d807a5d0 msgr2=0x7ff2d807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 -- 172.21.15.26:0/279019745 shutdown_connections 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.025+0000 7ff2de945700 1 -- 172.21.15.26:0/279019745 wait complete. 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.026+0000 7ff2de945700 1 Processor -- start 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.026+0000 7ff2de945700 1 -- start start 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2de945700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2d8103850 0x7ff2d819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2de945700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2de945700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 0x7ff2d81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2de945700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8113590 con 0x7ff2d8104200 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2de945700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8113410 con 0x7ff2d8108920 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2de945700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8113710 con 0x7ff2d8103850 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51784/0 (socket says 172.21.15.26:51784) 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 -- 172.21.15.26:0/1687640996 learned_addr learned my addr 172.21.15.26:0/1687640996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2d8103850 0x7ff2d819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2dcee2700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 0x7ff2d81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 -- 172.21.15.26:0/1687640996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2d8103850 msgr2=0x7ff2d819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2d8103850 0x7ff2d819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 -- 172.21.15.26:0/1687640996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 msgr2=0x7ff2d81998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 0x7ff2d81998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 -- 172.21.15.26:0/1687640996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d819a010 con 0x7ff2d8104200 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2dcee2700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 0x7ff2d81998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.027+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d819fbe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc006010 tx=0x7ff2cc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.028+0000 7ff2d57fa700 1 -- 172.21.15.26:0/1687640996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc026070 con 0x7ff2d8104200 2024-05-20T20:57:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.028+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d819a2a0 con 0x7ff2d8104200 2024-05-20T20:57:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.028+0000 7ff2d57fa700 1 -- 172.21.15.26:0/1687640996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2cc00f460 con 0x7ff2d8104200 2024-05-20T20:57:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.028+0000 7ff2d57fa700 1 -- 172.21.15.26:0/1687640996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc009e60 con 0x7ff2d8104200 2024-05-20T20:57:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.028+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81a6570 con 0x7ff2d8104200 2024-05-20T20:57:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.030+0000 7ff2d57fa700 1 -- 172.21.15.26:0/1687640996 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff2cc029070 con 0x7ff2d8104200 2024-05-20T20:57:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.031+0000 7ff2d57fa700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff2c00b2880 0x7ff2c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.031+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff2c00b2880 0x7ff2c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.031+0000 7ff2d57fa700 1 -- 172.21.15.26:0/1687640996 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff2cc0e2b30 con 0x7ff2d8104200 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff2c00b2880 0x7ff2c00b4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2d81083d0 tx=0x7ff2c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 Processor -- start 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 -- start start 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 0x55a2351d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cccb0 0x55a2351cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351cd670 0x55a2351d9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a235126d70 con 0x55a2351d26c0 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a235116910 con 0x55a2351cccb0 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a23511a950 con 0x55a2351cd670 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 0x55a2351d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 0x55a2351d2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51794/0 (socket says 172.21.15.26:51794) 2024-05-20T20:57:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2095052275 learned_addr learned my addr 172.21.15.26:0/2095052275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cccb0 0x55a2351cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2dcee2700 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351cd670 0x55a2351d9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2095052275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351cd670 msgr2=0x55a2351d9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351cd670 0x55a2351d9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2095052275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cccb0 msgr2=0x55a2351cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cccb0 0x55a2351cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2095052275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2350bffb0 con 0x55a2351d26c0 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.032+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cccb0 0x55a2351cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.033+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 0x55a2351d2aa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8010f70 tx=0x7ff2c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf06027eab30018 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.033+0000 7ff2be7fc700 1 -- 172.21.15.26:0/2095052275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800be60 con 0x55a2351d26c0 2024-05-20T20:57:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.033+0000 7ff2be7fc700 1 -- 172.21.15.26:0/2095052275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2c8011db0 con 0x55a2351d26c0 2024-05-20T20:57:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.033+0000 7ff2be7fc700 1 -- 172.21.15.26:0/2095052275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c801c630 con 0x55a2351d26c0 2024-05-20T20:57:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2095052275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 msgr2=0x55a2351d2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 0x55a2351d2aa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8010f70 tx=0x7ff2c8013000 comp rx=0 tx=0).stop 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2095052275 shutdown_connections 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351d26c0 0x55a2351d2aa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351cd670 0x55a2351d9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2095052275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cccb0 0x55a2351cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2095052275 >> 172.21.15.26:0/2095052275 conn(0x55a2350f4240 msgr2=0x55a2351c33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2095052275 shutdown_connections 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2095052275 wait complete. 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 -- start start 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.034+0000 7ff2e91dbb80 1 Processor -- start 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351d6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a23501cfa0 con 0x55a2351cccb0 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a23507eda0 con 0x55a2351cd670 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2351d9b50 con 0x55a2351d26c0 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2e91dbb80 10 client.0 _ll_register_callbacks cb 0x55a2350b1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51806/0 (socket says 172.21.15.26:51806) 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2051177203 learned_addr learned my addr 172.21.15.26:0/2051177203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2dcee2700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351d6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2051177203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 msgr2=0x55a2351d6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351d6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 msgr2=0x55a2351d63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.035+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2051177203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2350668d0 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d90f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff2c801e680 tx=0x7ff2c801e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742656d73187202e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2b77fe700 1 -- 172.21.15.26:0/2051177203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8035060 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2b77fe700 1 -- 172.21.15.26:0/2051177203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff2c8011bb0 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c8011db0 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2b77fe700 1 -- 172.21.15.26:0/2051177203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2c801ec80 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2e91dbb80 10 client.15864 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2350d0760 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.036+0000 7ff2b77fe700 1 -- 172.21.15.26:0/2051177203 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2c8011db0 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2b77fe700 1 -- 172.21.15.26:0/2051177203 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff2c8039030 con 0x55a2351cccb0 2024-05-20T20:57:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2e91dbb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2e91dbb80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2e91dbb80 20 client.15864 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2351e6600 0x55a2351e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2e91dbb80 4 client.15864 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a235128cc0 con 0x55a2351e6600 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2351e6600 0x55a2351e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2b77fe700 10 client.15864 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.037+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2351e6600 0x55a2351e89f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d0556927efc33b server_cookie=178eb1bfd434fb51 in_seq=0 out_seq=0 2024-05-20T20:57:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2b77fe700 1 -- 172.21.15.26:0/2051177203 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a235128cc0 con 0x55a2351e6600 2024-05-20T20:57:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2b77fe700 10 client.15864 handle_command_reply: tid=0 2024-05-20T20:57:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 client.15864 shutdown 2024-05-20T20:57:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 msgr2=0x55a2351d90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d90f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff2c801e680 tx=0x7ff2c801e760 comp rx=0 tx=0).stop 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 shutdown_connections 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2351e6600 0x55a2351e89f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d90f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351d6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2051177203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2b77fe700 0 client.15864 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.038+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 >> 172.21.15.26:0/2051177203 conn(0x55a2350f4240 msgr2=0x55a2351c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.039+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 shutdown_connections 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.039+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2051177203 wait complete. 2024-05-20T20:57:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.039+0000 7ff2e91dbb80 20 client.15864 trim_cache size 0 max 16384 2024-05-20T20:57:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.062+0000 7ff2e91dbb80 1 Processor -- start 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.063+0000 7ff2e91dbb80 1 -- start start 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81ae1c0 con 0x55a2351cccb0 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81aba50 con 0x55a2351cd670 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81a98a0 con 0x55a2351d26c0 2024-05-20T20:57:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51820/0 (socket says 172.21.15.26:51820) 2024-05-20T20:57:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.064+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2233948936 learned_addr learned my addr 172.21.15.26:0/2233948936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2dcee2700 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2233948936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 msgr2=0x55a2351c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2233948936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 msgr2=0x55a2351d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d7fff700 1 -- 172.21.15.26:0/2233948936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2351d5720 con 0x55a2351cccb0 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d0980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005d90 tx=0x7ff2c8004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de74199bb6adafc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2bdffb700 1 -- 172.21.15.26:0/2233948936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8035040 con 0x55a2351cccb0 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2bdffb700 1 -- 172.21.15.26:0/2233948936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2c8011bb0 con 0x55a2351cccb0 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.065+0000 7ff2bdffb700 1 -- 172.21.15.26:0/2233948936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8005420 con 0x55a2351cccb0 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2233948936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 msgr2=0x55a2351d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d0980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005d90 tx=0x7ff2c8004da0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2233948936 shutdown_connections 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351d0980 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/2233948936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2233948936 >> 172.21.15.26:0/2233948936 conn(0x55a2350f4240 msgr2=0x55a2351d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2233948936 shutdown_connections 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/2233948936 wait complete. 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.066+0000 7ff2e91dbb80 1 -- start start 2024-05-20T20:57:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 Processor -- start 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81a9f10 con 0x55a2351cccb0 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81ab380 con 0x55a2351cd670 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81aa5e0 con 0x55a2351d26c0 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2e91dbb80 10 client.0 _ll_register_callbacks cb 0x55a2350b1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51822/0 (socket says 172.21.15.26:51822) 2024-05-20T20:57:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2d7fff700 1 -- 172.21.15.26:0/4241402947 learned_addr learned my addr 172.21.15.26:0/4241402947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.067+0000 7ff2dcee2700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2d7fff700 1 -- 172.21.15.26:0/4241402947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 msgr2=0x55a2351cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2d7fff700 1 -- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 msgr2=0x55a2351cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2d7fff700 1 -- 172.21.15.26:0/4241402947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2351ea910 con 0x55a2351cccb0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2d7fff700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351caf00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff2c801e7a0 tx=0x7ff2c80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f026f0b95885377 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2b77fe700 1 -- 172.21.15.26:0/4241402947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8047060 con 0x55a2351cccb0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c8011bb0 con 0x55a2351cccb0 2024-05-20T20:57:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2b77fe700 1 -- 172.21.15.26:0/4241402947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff2c80304b0 con 0x55a2351cccb0 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2b77fe700 1 -- 172.21.15.26:0/4241402947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2c8035070 con 0x55a2351cccb0 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2e91dbb80 10 client.15870 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2351ebc20 con 0x55a2351cccb0 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.068+0000 7ff2b77fe700 1 -- 172.21.15.26:0/4241402947 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2c8011bb0 con 0x55a2351cccb0 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2b77fe700 1 -- 172.21.15.26:0/4241402947 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff2c804b030 con 0x55a2351cccb0 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2e91dbb80 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2e91dbb80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2e91dbb80 20 client.15870 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a235255620 0x55a235257a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2e91dbb80 4 client.15870 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2351224a0 con 0x55a235255620 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a235255620 0x55a235257a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2d77fe700 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a235255620 0x55a235257a10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8801080c576cce84 server_cookie=a9ca55499e23ff19 in_seq=0 out_seq=0 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.069+0000 7ff2b77fe700 10 client.15870 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2b77fe700 1 -- 172.21.15.26:0/4241402947 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2351224a0 con 0x55a235255620 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2b77fe700 10 client.15870 handle_command_reply: tid=0 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 client.15870 shutdown 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 msgr2=0x55a2351caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351caf00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff2c801e7a0 tx=0x7ff2c80052d0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 shutdown_connections 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a235255620 0x55a235257a10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2351cccb0 0x55a2351caf00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2351d26c0 0x55a2351cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2b77fe700 0 client.15870 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 --2- 172.21.15.26:0/4241402947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2351cd670 0x55a2351cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 >> 172.21.15.26:0/4241402947 conn(0x55a2350f4240 msgr2=0x55a2351d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 shutdown_connections 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 1 -- 172.21.15.26:0/4241402947 wait complete. 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.070+0000 7ff2e91dbb80 20 client.15870 trim_cache size 0 max 16384 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:57:52.076 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff2c00b2880 msgr2=0x7ff2c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff2c00b2880 0x7ff2c00b4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2d81083d0 tx=0x7ff2c800b410 comp rx=0 tx=0).stop 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 msgr2=0x7ff2d819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d819fbe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc006010 tx=0x7ff2cc0045e0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 shutdown_connections 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2d8108920 0x7ff2d81998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2d8103850 0x7ff2d819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff2c00b2880 0x7ff2c00b4d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 --2- 172.21.15.26:0/1687640996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d8104200 0x7ff2d819fbe0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 >> 172.21.15.26:0/1687640996 conn(0x7ff2d807a5d0 msgr2=0x7ff2d8101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 shutdown_connections 2024-05-20T20:57:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:52.071+0000 7ff2de945700 1 -- 172.21.15.26:0/1687640996 wait complete. 2024-05-20T20:57:52.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:51 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:57:52.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:51 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:57:52.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:51 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr:Note: switching to '4ebd1d5361d457738957dc0455bba90102296634'. 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr:state without impacting any branches by switching back to a branch. 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.390 INFO:tasks.workunit.client.0.smithi026.stderr: git switch -c 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr:Or undo this operation with: 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr: git switch - 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-05-20T20:57:52.391 INFO:tasks.workunit.client.0.smithi026.stderr:HEAD is now at 4ebd1d5361d mds/queisce-db: collect acks while bootstrapping 2024-05-20T20:57:52.396 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-20T20:57:52.422 INFO:tasks.workunit.client.0.smithi026.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-20T20:57:52.425 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-20T20:57:52.425 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-20T20:57:53.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:57:53.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:57:53.181 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:52 smithi026 ceph-mon[31990]: pgmap v440: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T20:57:53.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:52 smithi055 ceph-mon[35213]: pgmap v440: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T20:57:53.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:52 smithi144 ceph-mon[35660]: pgmap v440: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T20:57:53.414 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:54.050 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-20T20:57:54.148 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-20T20:57:54.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:53 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:57:54.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:53 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:57:54.202 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-20T20:57:54.204 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-20T20:57:54.204 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-20T20:57:54.245 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-20T20:57:54.249 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T20:57:54.249 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-20T20:57:54.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:53 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:57:54.306 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-20T20:57:54.308 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-20T20:57:54.308 DEBUG:teuthology.orchestra.run.smithi026:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ebd1d5361d457738957dc0455bba90102296634 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-20T20:57:54.380 INFO:tasks.workunit.client.0.smithi026.stderr:+ dbench 1 2024-05-20T20:57:54.431 INFO:tasks.workunit.client.0.smithi026.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-20T20:57:54.431 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-05-20T20:57:54.431 INFO:tasks.workunit.client.0.smithi026.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-20T20:57:54.448 INFO:tasks.workunit.client.0.smithi026.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-20T20:57:54.548 INFO:tasks.workunit.client.0.smithi026.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-20T20:57:54.548 INFO:tasks.workunit.client.0.smithi026.stdout:releasing clients 2024-05-20T20:57:55.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:54 smithi055 ceph-mon[35213]: pgmap v441: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:55.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:57:55.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:57:55.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:54 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:57:55.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:54 smithi026 ceph-mon[31990]: pgmap v441: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:55.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:57:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:57:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:54 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:57:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:54 smithi144 ceph-mon[35660]: pgmap v441: 97 pgs: 97 active+clean; 672 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T20:57:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:57:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:57:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:54 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:57:55.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 -- 172.21.15.26:0/388483878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54881096a0 msgr2=0x7f5488109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:55.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 --2- 172.21.15.26:0/388483878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54881096a0 0x7f5488109a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5478009990 tx=0x7f5478009ca0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 -- 172.21.15.26:0/388483878 shutdown_connections 2024-05-20T20:57:55.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 --2- 172.21.15.26:0/388483878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54881096a0 0x7f5488109a80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 --2- 172.21.15.26:0/388483878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5488103f50 0x7f5488110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 --2- 172.21.15.26:0/388483878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5488103590 0x7f5488103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 -- 172.21.15.26:0/388483878 >> 172.21.15.26:0/388483878 conn(0x7f548807a6f0 msgr2=0x7f548807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:55.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 -- 172.21.15.26:0/388483878 shutdown_connections 2024-05-20T20:57:55.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.528+0000 7f548e0ab700 1 -- 172.21.15.26:0/388483878 wait complete. 2024-05-20T20:57:55.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 Processor -- start 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 -- start start 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 0x7f5488075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5488103f50 0x7f5488075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54881096a0 0x7f54880762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54881135c0 con 0x7f5488103590 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5488113440 con 0x7f54881096a0 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f548e0ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5488113740 con 0x7f5488103f50 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f54877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 0x7f5488075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f54877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 0x7f5488075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51840/0 (socket says 172.21.15.26:51840) 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f54877fe700 1 -- 172.21.15.26:0/3977002329 learned_addr learned my addr 172.21.15.26:0/3977002329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.529+0000 7f5487fff700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54881096a0 0x7f54880762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f5486ffd700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5488103f50 0x7f5488075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f54877fe700 1 -- 172.21.15.26:0/3977002329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5488103f50 msgr2=0x7f5488075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:55.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f54877fe700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5488103f50 0x7f5488075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f54877fe700 1 -- 172.21.15.26:0/3977002329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54881096a0 msgr2=0x7f54880762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f54877fe700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54881096a0 0x7f54880762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f54877fe700 1 -- 172.21.15.26:0/3977002329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54881a7fc0 con 0x7f5488103590 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f5487fff700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54881096a0 0x7f54880762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f54877fe700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 0x7f5488075840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5478006010 tx=0x7f547800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478027070 con 0x7f5488103590 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5478022470 con 0x7f5488103590 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478017650 con 0x7f5488103590 2024-05-20T20:57:55.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54881a8250 con 0x7f5488103590 2024-05-20T20:57:55.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.530+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881a8700 con 0x7f5488103590 2024-05-20T20:57:55.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.532+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548806a230 con 0x7f5488103590 2024-05-20T20:57:55.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.533+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5478009d10 con 0x7f5488103590 2024-05-20T20:57:55.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.534+0000 7f5484ff9700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54740b2900 0x7f54740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:55.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.534+0000 7f5486ffd700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54740b2900 0x7f54740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:55.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.534+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f54780e3030 con 0x7f5488103590 2024-05-20T20:57:55.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.534+0000 7f5486ffd700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54740b2900 0x7f54740b4dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f548807acf0 tx=0x7f547000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:55.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.538+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5478013070 con 0x7f5488103590 2024-05-20T20:57:55.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 929 21.78 MB/sec warmup 1 sec latency 17.133 ms 2024-05-20T20:57:55.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.840+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f548802e720 con 0x7f5488103590 2024-05-20T20:57:55.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.841+0000 7f5484ff9700 1 -- 172.21.15.26:0/3977002329 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f54780aad80 con 0x7f5488103590 2024-05-20T20:57:55.841 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:57:55.841 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:57:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54740b2900 msgr2=0x7f54740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54740b2900 0x7f54740b4dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f548807acf0 tx=0x7f547000b410 comp rx=0 tx=0).stop 2024-05-20T20:57:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 msgr2=0x7f5488075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 0x7f5488075840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5478006010 tx=0x7f547800bfc0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 shutdown_connections 2024-05-20T20:57:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54740b2900 0x7f54740b4dc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5488103590 0x7f5488075840 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54881096a0 0x7f54880762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 --2- 172.21.15.26:0/3977002329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5488103f50 0x7f5488075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 >> 172.21.15.26:0/3977002329 conn(0x7f548807a6f0 msgr2=0x7f5488102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 shutdown_connections 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:55.844+0000 7f548e0ab700 1 -- 172.21.15.26:0/3977002329 wait complete. 2024-05-20T20:57:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:57:56.170 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:55 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3977002329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:56.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:55 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3977002329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 -- 172.21.15.26:0/3461088365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a41019b0 msgr2=0x7f84a4101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 --2- 172.21.15.26:0/3461088365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a41019b0 0x7f84a4101dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8494009900 tx=0x7f849402a410 comp rx=0 tx=0).stop 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 -- 172.21.15.26:0/3461088365 shutdown_connections 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 --2- 172.21.15.26:0/3461088365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 0x7f84a4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 --2- 172.21.15.26:0/3461088365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a4102ba0 0x7f84a4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 --2- 172.21.15.26:0/3461088365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a41019b0 0x7f84a4101dd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 -- 172.21.15.26:0/3461088365 >> 172.21.15.26:0/3461088365 conn(0x7f84a40fcf20 msgr2=0x7f84a40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 -- 172.21.15.26:0/3461088365 shutdown_connections 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.272+0000 7f84a9854640 1 -- 172.21.15.26:0/3461088365 wait complete. 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 Processor -- start 2024-05-20T20:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 -- start start 2024-05-20T20:57:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a41019b0 0x7f84a4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 0x7f84a4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 0x7f84a419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410d380 con 0x7f84a4102ba0 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410d200 con 0x7f84a4103560 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a9854640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410d500 con 0x7f84a41019b0 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 0x7f84a4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 0x7f84a4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51868/0 (socket says 172.21.15.26:51868) 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a27fc640 1 -- 172.21.15.26:0/1390868964 learned_addr learned my addr 172.21.15.26:0/1390868964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.273+0000 7f84a37fe640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 0x7f84a419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a41019b0 0x7f84a4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a27fc640 1 -- 172.21.15.26:0/1390868964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a41019b0 msgr2=0x7f84a4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a27fc640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a41019b0 0x7f84a4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a27fc640 1 -- 172.21.15.26:0/1390868964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 msgr2=0x7f84a419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a27fc640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 0x7f84a419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a27fc640 1 -- 172.21.15.26:0/1390868964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419f6f0 con 0x7f84a4102ba0 2024-05-20T20:57:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a37fe640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 0x7f84a419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a27fc640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 0x7f84a4198060 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f849800d770 tx=0x7f849800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a8852640 1 -- 172.21.15.26:0/1390868964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498017070 con 0x7f84a4102ba0 2024-05-20T20:57:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a8852640 1 -- 172.21.15.26:0/1390868964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84980054b0 con 0x7f84a4102ba0 2024-05-20T20:57:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419f9e0 con 0x7f84a4102ba0 2024-05-20T20:57:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a8852640 1 -- 172.21.15.26:0/1390868964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498004c60 con 0x7f84a4102ba0 2024-05-20T20:57:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.274+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a419fee0 con 0x7f84a4102ba0 2024-05-20T20:57:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84a8852640 1 -- 172.21.15.26:0/1390868964 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8498004e00 con 0x7f84a4102ba0 2024-05-20T20:57:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 Processor -- start 2024-05-20T20:57:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 -- start start 2024-05-20T20:57:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda486cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda48bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda48e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559cda3f5c30 con 0x559cda4872c0 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559cda3e57d0 con 0x559cda48c430 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84ac56b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559cda3e9810 con 0x559cda4868d0 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.277+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda48bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda48bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51882/0 (socket says 172.21.15.26:51882) 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a2ffd640 1 -- 172.21.15.26:0/3001507371 learned_addr learned my addr 172.21.15.26:0/3001507371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a27fc640 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda48e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a37fe640 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda486cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a37fe640 1 -- 172.21.15.26:0/3001507371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 msgr2=0x559cda48e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a37fe640 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda48e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a37fe640 1 -- 172.21.15.26:0/3001507371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 msgr2=0x559cda48bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a37fe640 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda48bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a37fe640 1 -- 172.21.15.26:0/3001507371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cda0fa210 con 0x559cda4868d0 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a8852640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f848c0b28a0 0x7f848c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a8852640 1 -- 172.21.15.26:0/1390868964 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f84980e1cc0 con 0x7f84a4102ba0 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.278+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f848c0b28a0 0x7f848c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.279+0000 7f84a37fe640 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda486cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f849c005fd0 tx=0x7f849c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f232dd480a0651c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.279+0000 7f8489ffb640 1 -- 172.21.15.26:0/3001507371 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c004d20 con 0x559cda4868d0 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.279+0000 7f8489ffb640 1 -- 172.21.15.26:0/3001507371 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f849c00f9a0 con 0x559cda4868d0 2024-05-20T20:57:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.279+0000 7f8489ffb640 1 -- 172.21.15.26:0/3001507371 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0043d0 con 0x559cda4868d0 2024-05-20T20:57:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 -- 172.21.15.26:0/3001507371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 msgr2=0x559cda486cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f848c0b28a0 0x7f848c0b4d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x559cda487d30 tx=0x7f8494005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda486cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f849c005fd0 tx=0x7f849c011bf0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 -- 172.21.15.26:0/3001507371 shutdown_connections 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda48e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda48bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 --2- 172.21.15.26:0/3001507371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda486cf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 -- 172.21.15.26:0/3001507371 >> 172.21.15.26:0/3001507371 conn(0x559cda47fd30 msgr2=0x559cda480130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 -- 172.21.15.26:0/3001507371 shutdown_connections 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 -- 172.21.15.26:0/3001507371 wait complete. 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 -- start start 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.280+0000 7f84ac56b740 1 Processor -- start 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda495480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda4979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda499fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559cda0cb790 con 0x559cda4872c0 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559cda3a1890 con 0x559cda48c430 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559cda4846e0 con 0x559cda4868d0 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84ac56b740 10 client.0 _ll_register_callbacks cb 0x559cda0f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda4979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda4979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51892/0 (socket says 172.21.15.26:51892) 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 -- 172.21.15.26:0/1816738688 learned_addr learned my addr 172.21.15.26:0/1816738688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda495480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a37fe640 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda499fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 -- 172.21.15.26:0/1816738688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 msgr2=0x559cda495480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda495480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 -- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 msgr2=0x559cda499fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda499fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 -- 172.21.15.26:0/1816738688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cda1c6620 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f84a27fc640 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda4979d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f849800d740 tx=0x7f849800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e1cc9c7ef9e4e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f8482ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.281+0000 7f8482ffd640 1 -- 172.21.15.26:0/1816738688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980e3070 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f8482ffd640 1 -- 172.21.15.26:0/1816738688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f84980050f0 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849c00f9a0 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f8482ffd640 1 -- 172.21.15.26:0/1816738688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84980c3050 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559cda49d230 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f8482ffd640 1 -- 172.21.15.26:0/1816738688 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f849c00f9a0 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f8482ffd640 1 -- 172.21.15.26:0/1816738688 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84980a4bf0 con 0x559cda4872c0 2024-05-20T20:57:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 20 client.15888 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda4a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 4 client.15888 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cda3f7b80 con 0x559cda49e920 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.282+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda4a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.283+0000 7f8482ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.283+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda4a0d10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878bd4563c15b970 server_cookie=e8844532e5a8da62 in_seq=0 out_seq=0 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f8482ffd640 1 -- 172.21.15.26:0/1816738688 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559cda3f7b80 con 0x559cda49e920 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f8482ffd640 10 client.15888 handle_command_reply: tid=0 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 client.15888 shutdown 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 msgr2=0x559cda4979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda4979d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f849800d740 tx=0x7f849800de80 comp rx=0 tx=0).stop 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 shutdown_connections 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda4a0d10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda48c430 0x559cda499fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f8482ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4872c0 0x559cda4979d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 --2- 172.21.15.26:0/1816738688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda495480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 >> 172.21.15.26:0/1816738688 conn(0x559cda47fd30 msgr2=0x559cda47efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 shutdown_connections 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 1 -- 172.21.15.26:0/1816738688 wait complete. 2024-05-20T20:57:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.284+0000 7f84ac56b740 20 client.15888 trim_cache size 0 max 16384 2024-05-20T20:57:56.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:55 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3977002329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 Processor -- start 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 -- start start 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 0x559cda48a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda480c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda48c430 0x559cda481140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41aa5e0 con 0x559cda4868d0 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41a7eb0 con 0x559cda4872c0 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84ac56b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41a5d00 con 0x559cda48c430 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 0x559cda48a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda480c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 0x559cda48a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51900/0 (socket says 172.21.15.26:51900) 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a2ffd640 1 -- 172.21.15.26:0/2532336996 learned_addr learned my addr 172.21.15.26:0/2532336996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a37fe640 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda48c430 0x559cda481140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a27fc640 1 -- 172.21.15.26:0/2532336996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda48c430 msgr2=0x559cda481140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a27fc640 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda48c430 0x559cda481140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a27fc640 1 -- 172.21.15.26:0/2532336996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 msgr2=0x559cda48a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a27fc640 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 0x559cda48a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a27fc640 1 -- 172.21.15.26:0/2532336996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cda4a1080 con 0x559cda4872c0 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a37fe640 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda48c430 0x559cda481140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.302+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 0x559cda48a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84a27fc640 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda480c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8498004770 tx=0x7f84980e5810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aeb7829469062f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84897fa640 1 -- 172.21.15.26:0/2532336996 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498020070 con 0x559cda4872c0 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84897fa640 1 -- 172.21.15.26:0/2532336996 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84980a8d90 con 0x559cda4872c0 2024-05-20T20:57:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84897fa640 1 -- 172.21.15.26:0/2532336996 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980b20e0 con 0x559cda4872c0 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 -- 172.21.15.26:0/2532336996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 msgr2=0x559cda480c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda480c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8498004770 tx=0x7f84980e5810 comp rx=0 tx=0).stop 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 -- 172.21.15.26:0/2532336996 shutdown_connections 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda48c430 0x559cda481140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda480c00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 --2- 172.21.15.26:0/2532336996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda4868d0 0x559cda48a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.303+0000 7f84ac56b740 1 -- 172.21.15.26:0/2532336996 >> 172.21.15.26:0/2532336996 conn(0x559cda47fd30 msgr2=0x559cda48a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 -- 172.21.15.26:0/2532336996 shutdown_connections 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 -- 172.21.15.26:0/2532336996 wait complete. 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 -- start start 2024-05-20T20:57:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 Processor -- start 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda480e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda481370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 0x559cda4b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41a6370 con 0x559cda48c430 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41a77e0 con 0x559cda4872c0 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41a6a40 con 0x559cda4868d0 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84ac56b740 10 client.0 _ll_register_callbacks cb 0x7f84980046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 0x559cda4b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda481370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84a27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda481370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35996/0 (socket says 172.21.15.26:35996) 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 0x559cda4b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51912/0 (socket says 172.21.15.26:51912) 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.304+0000 7f84a27fc640 1 -- 172.21.15.26:0/982095865 learned_addr learned my addr 172.21.15.26:0/982095865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda480e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a37fe640 1 -- 172.21.15.26:0/982095865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 msgr2=0x559cda480e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a37fe640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda480e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a37fe640 1 -- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 msgr2=0x559cda481370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a37fe640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda481370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a37fe640 1 -- 172.21.15.26:0/982095865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cda481c20 con 0x559cda48c430 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a27fc640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda481370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84a37fe640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 0x559cda4b11a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f849c003fd0 tx=0x7f849c005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddb435274d87377 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f8482ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f8482ffd640 1 -- 172.21.15.26:0/982095865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c01e040 con 0x559cda48c430 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f8482ffd640 1 -- 172.21.15.26:0/982095865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f849c00f9a0 con 0x559cda48c430 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84980a8d90 con 0x559cda48c430 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f8482ffd640 1 -- 172.21.15.26:0/982095865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f849c00fbe0 con 0x559cda48c430 2024-05-20T20:57:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84ac56b740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.305+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559cda4b2820 con 0x559cda48c430 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f8482ffd640 1 -- 172.21.15.26:0/982095865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84980a8d90 con 0x559cda48c430 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f8482ffd640 1 -- 172.21.15.26:0/982095865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f849c018070 con 0x559cda48c430 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84ac56b740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84ac56b740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84ac56b740 20 client.15894 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84ac56b740 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda50b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84ac56b740 4 client.15894 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cda3f1360 con 0x559cda49e920 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda50b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f8482ffd640 10 client.15894 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.306+0000 7f84a2ffd640 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda50b2e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e7f408d0ef8bf0 server_cookie=91b4fde23c3eabfa in_seq=0 out_seq=0 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.345+0000 7f8482ffd640 1 -- 172.21.15.26:0/982095865 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559cda3f1360 con 0x559cda49e920 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.345+0000 7f8482ffd640 10 client.15894 handle_command_reply: tid=0 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.345+0000 7f84ac56b740 1 client.15894 shutdown 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.345+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 msgr2=0x559cda4b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.345+0000 7f84ac56b740 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 0x559cda4b11a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f849c003fd0 tx=0x7f849c005ee0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 shutdown_connections 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559cda49e920 0x559cda50b2e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cda48c430 0x559cda4b11a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f8482ffd640 0 client.15894 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559cda4872c0 0x559cda481370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 --2- 172.21.15.26:0/982095865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cda4868d0 0x559cda480e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 >> 172.21.15.26:0/982095865 conn(0x559cda47fd30 msgr2=0x559cda48aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 shutdown_connections 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 1 -- 172.21.15.26:0/982095865 wait complete. 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84ac56b740 20 client.15894 trim_cache size 0 max 16384 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:57:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:57:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:57:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f848c0b28a0 msgr2=0x7f848c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f848c0b28a0 0x7f848c0b4d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x559cda487d30 tx=0x7f8494005a20 comp rx=0 tx=0).stop 2024-05-20T20:57:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 msgr2=0x7f84a4198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 0x7f84a4198060 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f849800d770 tx=0x7f849800dc30 comp rx=0 tx=0).stop 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 shutdown_connections 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f848c0b28a0 0x7f848c0b4d60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84a4103560 0x7f84a419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84a4102ba0 0x7f84a4198060 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 --2- 172.21.15.26:0/1390868964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a41019b0 0x7f84a4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.346+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 >> 172.21.15.26:0/1390868964 conn(0x7f84a40fcf20 msgr2=0x7f84a4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.347+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 shutdown_connections 2024-05-20T20:57:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:57:56.347+0000 7f84a9854640 1 -- 172.21.15.26:0/1390868964 wait complete. 2024-05-20T20:57:56.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 1612 13.08 MB/sec warmup 2 sec latency 37.548 ms 2024-05-20T20:57:56.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:57:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:57:57.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:56 smithi055 ceph-mon[35213]: pgmap v442: 97 pgs: 97 active+clean; 8.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 686 KiB/s wr, 10 op/s 2024-05-20T20:57:57.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:56 smithi026 ceph-mon[31990]: pgmap v442: 97 pgs: 97 active+clean; 8.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 686 KiB/s wr, 10 op/s 2024-05-20T20:57:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:56 smithi144 ceph-mon[35660]: pgmap v442: 97 pgs: 97 active+clean; 8.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 686 KiB/s wr, 10 op/s 2024-05-20T20:57:57.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 2334 9.28 MB/sec warmup 3 sec latency 18.331 ms 2024-05-20T20:57:58.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 3237 8.62 MB/sec warmup 4 sec latency 17.780 ms 2024-05-20T20:57:58.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 -- 172.21.15.26:0/1164353238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 msgr2=0x7f34f810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 --2- 172.21.15.26:0/1164353238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f810f2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b0a0 tx=0x7f34f000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 -- 172.21.15.26:0/1164353238 shutdown_connections 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 --2- 172.21.15.26:0/1164353238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f810f2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 --2- 172.21.15.26:0/1164353238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8108520 0x7f34f8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 --2- 172.21.15.26:0/1164353238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8102440 0x7f34f81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 -- 172.21.15.26:0/1164353238 >> 172.21.15.26:0/1164353238 conn(0x7f34f80793f0 msgr2=0x7f34f8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.792+0000 7f3500256700 1 -- 172.21.15.26:0/1164353238 shutdown_connections 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.793+0000 7f3500256700 1 -- 172.21.15.26:0/1164353238 wait complete. 2024-05-20T20:57:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.793+0000 7f3500256700 1 Processor -- start 2024-05-20T20:57:58.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.793+0000 7f3500256700 1 -- start start 2024-05-20T20:57:58.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f3500256700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8102440 0x7f34f819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f3500256700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f3500256700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 0x7f34f81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f3500256700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81124f0 con 0x7f34f8102e00 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f3500256700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8112370 con 0x7f34f8102440 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f3500256700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8112670 con 0x7f34f8108520 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 0x7f34f81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fd7f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8102440 0x7f34f819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 0x7f34f81a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:54310/0 (socket says 172.21.15.26:54310) 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 -- 172.21.15.26:0/1121915576 learned_addr learned my addr 172.21.15.26:0/1121915576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 -- 172.21.15.26:0/1121915576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8102440 msgr2=0x7f34f819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8102440 0x7f34f819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 -- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 msgr2=0x7f34f819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fe7f3700 1 -- 172.21.15.26:0/1121915576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f81a5f30 con 0x7f34f8108520 2024-05-20T20:57:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fdff2700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8102440 0x7f34f819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.794+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.795+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 0x7f34f81a59f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b070 tx=0x7f34f0004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.795+0000 7f34eeffd700 1 -- 172.21.15.26:0/1121915576 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0021070 con 0x7f34f8108520 2024-05-20T20:57:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.795+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81a6100 con 0x7f34f8108520 2024-05-20T20:57:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.795+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81a65b0 con 0x7f34f8108520 2024-05-20T20:57:58.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.795+0000 7f34eeffd700 1 -- 172.21.15.26:0/1121915576 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34f0007b80 con 0x7f34f8108520 2024-05-20T20:57:58.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.795+0000 7f34eeffd700 1 -- 172.21.15.26:0/1121915576 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004300 con 0x7f34f8108520 2024-05-20T20:57:58.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.798+0000 7f34eeffd700 1 -- 172.21.15.26:0/1121915576 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f34f00044a0 con 0x7f34f8108520 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 Processor -- start 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34eeffd700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34e40b2880 0x7f34e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34e40b2880 0x7f34e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 -- start start 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34eeffd700 1 -- 172.21.15.26:0/1121915576 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f34f00e2e20 con 0x7f34f8108520 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 0x55d4e2c4bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4c210 0x55d4e2c4f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4e2baac90 con 0x55d4e2c4b860 2024-05-20T20:57:58.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d4e2b9a830 con 0x55d4e2c4c210 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f350aaecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e2b9e870 con 0x55d4e2c4ff60 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 0x55d4e2c4bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34e40b2880 0x7f34e40b4d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34f8079a60 tx=0x7f34e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 0x55d4e2c4bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51940/0 (socket says 172.21.15.26:51940) 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34fdff2700 1 -- 172.21.15.26:0/2484858053 learned_addr learned my addr 172.21.15.26:0/2484858053 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.799+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4c210 0x55d4e2c4f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fd7f1700 1 -- 172.21.15.26:0/2484858053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 msgr2=0x55d4e2c503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fd7f1700 1 -- 172.21.15.26:0/2484858053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 msgr2=0x55d4e2c4bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 0x55d4e2c4bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fd7f1700 1 -- 172.21.15.26:0/2484858053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e2aea8d0 con 0x55d4e2c4c210 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:57:58.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fdff2700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 0x55d4e2c4bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:58.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4c210 0x55d4e2c4f940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34e80042b0 tx=0x7f34e8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0aedb480cc503d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:58.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34e3fff700 1 -- 172.21.15.26:0/2484858053 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e802c070 con 0x55d4e2c4c210 2024-05-20T20:57:58.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34e3fff700 1 -- 172.21.15.26:0/2484858053 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34e801e0d0 con 0x55d4e2c4c210 2024-05-20T20:57:58.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.800+0000 7f34e3fff700 1 -- 172.21.15.26:0/2484858053 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e801cb00 con 0x55d4e2c4c210 2024-05-20T20:57:58.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 -- 172.21.15.26:0/2484858053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4c210 msgr2=0x55d4e2c4f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4c210 0x55d4e2c4f940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34e80042b0 tx=0x7f34e8014000 comp rx=0 tx=0).stop 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 -- 172.21.15.26:0/2484858053 shutdown_connections 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4b860 0x55d4e2c4bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2484858053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4c210 0x55d4e2c4f940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 -- 172.21.15.26:0/2484858053 >> 172.21.15.26:0/2484858053 conn(0x55d4e2b79080 msgr2=0x55d4e2c45e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 -- 172.21.15.26:0/2484858053 shutdown_connections 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.801+0000 7f350aaecb80 1 -- 172.21.15.26:0/2484858053 wait complete. 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 -- start start 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 Processor -- start 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4c210 0x55d4e2c546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c54c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4e2aa0fa0 con 0x55d4e2c4ff60 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d4e2b3c7c0 con 0x55d4e2c4b860 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e2c58120 con 0x55d4e2c4c210 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f350aaecb80 10 client.0 _ll_register_callbacks cb 0x55d4e2b35920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f34fe7f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c54c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.802+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36048/0 (socket says 172.21.15.26:36048) 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fe7f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c54c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51952/0 (socket says 172.21.15.26:51952) 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 -- 172.21.15.26:0/1716806333 learned_addr learned my addr 172.21.15.26:0/1716806333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4c210 0x55d4e2c546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 -- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4c210 msgr2=0x55d4e2c546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4c210 0x55d4e2c546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 -- 172.21.15.26:0/1716806333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 msgr2=0x55d4e2c54c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c54c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 -- 172.21.15.26:0/1716806333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e2b39690 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c54c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34fdff2700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c573c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34f4011e10 tx=0x7f34f400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac1ed65b88b97a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34e17fa700 1 -- 172.21.15.26:0/1716806333 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f400fe50 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f34e17fa700 1 -- 172.21.15.26:0/1716806333 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f34f4011050 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.803+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e801e0d0 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f34e17fa700 1 -- 172.21.15.26:0/1716806333 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34f4011250 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 10 client.25463 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4e2c62910 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f34e17fa700 1 -- 172.21.15.26:0/1716806333 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34e801e0d0 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f34e17fa700 1 -- 172.21.15.26:0/1716806333 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f34f4003b90 con 0x55d4e2c4b860 2024-05-20T20:57:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 10 client.25463 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 10 client.25463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:57:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 20 client.25463 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2c64ff0 0x55d4e2c67430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 4 client.25463 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:57:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.804+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e2c179e0 con 0x55d4e2c64ff0 2024-05-20T20:57:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.805+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2c64ff0 0x55d4e2c67430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.805+0000 7f34e17fa700 10 client.25463 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.805+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2c64ff0 0x55d4e2c67430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c522244bf474bcab server_cookie=70f395ddcc1ba847 in_seq=0 out_seq=0 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f34e17fa700 1 -- 172.21.15.26:0/1716806333 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d4e2c179e0 con 0x55d4e2c64ff0 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f34e17fa700 10 client.25463 handle_command_reply: tid=0 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 client.25463 shutdown 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 msgr2=0x55d4e2c573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c573c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34f4011e10 tx=0x7f34f400f6a0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 shutdown_connections 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2c64ff0 0x55d4e2c67430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c54c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c573c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 --2- 172.21.15.26:0/1716806333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4c210 0x55d4e2c546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 >> 172.21.15.26:0/1716806333 conn(0x55d4e2b79080 msgr2=0x55d4e2c51350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f34e17fa700 0 client.25463 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 shutdown_connections 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 1 -- 172.21.15.26:0/1716806333 wait complete. 2024-05-20T20:57:58.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.806+0000 7f350aaecb80 20 client.25463 trim_cache size 0 max 16384 2024-05-20T20:57:58.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 Processor -- start 2024-05-20T20:57:58.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 -- start start 2024-05-20T20:57:58.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4b860 0x55d4e2c6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c6d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c4d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81ae250 con 0x55d4e2c4c210 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81abae0 con 0x55d4e2c4ff60 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.822+0000 7f350aaecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81a9930 con 0x55d4e2c4b860 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c6d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c6d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51958/0 (socket says 172.21.15.26:51958) 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 -- 172.21.15.26:0/222696744 learned_addr learned my addr 172.21.15.26:0/222696744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fdff2700 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4b860 0x55d4e2c6cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c4d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 -- 172.21.15.26:0/222696744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4b860 msgr2=0x55d4e2c6cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4b860 0x55d4e2c6cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 -- 172.21.15.26:0/222696744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4ff60 msgr2=0x55d4e2c4d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c4d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 -- 172.21.15.26:0/222696744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e2c52c30 con 0x55d4e2c4c210 2024-05-20T20:57:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fdff2700 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4b860 0x55d4e2c6cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:57:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c6d240 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009c30 tx=0x7f34e801e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a95fc078f7f7b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34e37fe700 1 -- 172.21.15.26:0/222696744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8037030 con 0x55d4e2c4c210 2024-05-20T20:57:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34e37fe700 1 -- 172.21.15.26:0/222696744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34e801e0d0 con 0x55d4e2c4c210 2024-05-20T20:57:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.823+0000 7f34e37fe700 1 -- 172.21.15.26:0/222696744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e802c040 con 0x55d4e2c4c210 2024-05-20T20:57:58.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 -- 172.21.15.26:0/222696744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 msgr2=0x55d4e2c6d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c6d240 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009c30 tx=0x7f34e801e9c0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 -- 172.21.15.26:0/222696744 shutdown_connections 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c4d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4b860 0x55d4e2c6cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 --2- 172.21.15.26:0/222696744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c6d240 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 -- 172.21.15.26:0/222696744 >> 172.21.15.26:0/222696744 conn(0x55d4e2b79080 msgr2=0x55d4e2c44c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 -- 172.21.15.26:0/222696744 shutdown_connections 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.824+0000 7f350aaecb80 1 -- 172.21.15.26:0/222696744 wait complete. 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 -- start start 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 Processor -- start 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c5d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c5ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c5b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81a9fa0 con 0x55d4e2c4c210 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81ab410 con 0x55d4e2c4b860 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81aa670 con 0x55d4e2c4ff60 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.825+0000 7f350aaecb80 10 client.0 _ll_register_callbacks cb 0x55d4e2b35920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c5d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c5d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36074/0 (socket says 172.21.15.26:36074) 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 -- 172.21.15.26:0/2628258470 learned_addr learned my addr 172.21.15.26:0/2628258470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c5ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fe7f3700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 -- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 msgr2=0x55d4e2c5b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c5b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 -- 172.21.15.26:0/2628258470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 msgr2=0x55d4e2c5ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c5ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 -- 172.21.15.26:0/2628258470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e2c5b6f0 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34fdff2700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c5d850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34f4011b30 tx=0x7f34f40057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ec56d212c1e829 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34e17fa700 1 -- 172.21.15.26:0/2628258470 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f400fe50 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e801e0d0 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.826+0000 7f34e17fa700 1 -- 172.21.15.26:0/2628258470 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f34f4003680 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f34e17fa700 1 -- 172.21.15.26:0/2628258470 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34f4003880 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 10 client.25469 fetch_fsmap learned FSMap version 26 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4e2ccf940 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f34e17fa700 1 -- 172.21.15.26:0/2628258470 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34e801e0d0 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f34e17fa700 1 -- 172.21.15.26:0/2628258470 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f34f4010990 con 0x55d4e2c4b860 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 10 client.25469 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 10 client.25469 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 20 client.25469 populate_metadata read hostname 'smithi026' 2024-05-20T20:57:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2cd00a0 0x55d4e2cd24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 4 client.25469 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4e2bacbe0 con 0x55d4e2cd00a0 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.827+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2cd00a0 0x55d4e2cd24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f34fd7f1700 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2cd00a0 0x55d4e2cd24c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a321314b23ea630 server_cookie=900126783706e5f1 in_seq=0 out_seq=0 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f34e17fa700 10 client.25469 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f34e17fa700 1 -- 172.21.15.26:0/2628258470 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4e2bacbe0 con 0x55d4e2cd00a0 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f34e17fa700 10 client.25469 handle_command_reply: tid=0 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 client.25469 shutdown 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 msgr2=0x55d4e2c5d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c5d850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34f4011b30 tx=0x7f34f40057e0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 shutdown_connections 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55d4e2cd00a0 0x55d4e2cd24c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e2c4ff60 0x55d4e2c5b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f34e17fa700 0 client.25469 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4e2c4b860 0x55d4e2c5d850 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 --2- 172.21.15.26:0/2628258470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4e2c4c210 0x55d4e2c5ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 >> 172.21.15.26:0/2628258470 conn(0x55d4e2b79080 msgr2=0x55d4e2c4a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 shutdown_connections 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 1 -- 172.21.15.26:0/2628258470 wait complete. 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.828+0000 7f350aaecb80 20 client.25469 trim_cache size 0 max 16384 2024-05-20T20:57:58.833 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34e40b2880 msgr2=0x7f34e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34e40b2880 0x7f34e40b4d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34f8079a60 tx=0x7f34e8009260 comp rx=0 tx=0).stop 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 msgr2=0x7f34f81a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 0x7f34f81a59f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b070 tx=0x7f34f0004c60 comp rx=0 tx=0).stop 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 shutdown_connections 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f8108520 0x7f34f81a59f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34f8102440 0x7f34f819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34e40b2880 0x7f34e40b4d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 --2- 172.21.15.26:0/1121915576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8102e00 0x7f34f819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:57:58.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 >> 172.21.15.26:0/1121915576 conn(0x7f34f80793f0 msgr2=0x7f34f8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:57:58.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 shutdown_connections 2024-05-20T20:57:58.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:57:58.829+0000 7f3500256700 1 -- 172.21.15.26:0/1121915576 wait complete. 2024-05-20T20:57:59.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:57:58 smithi055 ceph-mon[35213]: pgmap v443: 97 pgs: 97 active+clean; 14 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 MiB/s wr, 23 op/s 2024-05-20T20:57:59.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:57:58 smithi026 ceph-mon[31990]: pgmap v443: 97 pgs: 97 active+clean; 14 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 MiB/s wr, 23 op/s 2024-05-20T20:57:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:57:58 smithi144 ceph-mon[35660]: pgmap v443: 97 pgs: 97 active+clean; 14 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 MiB/s wr, 23 op/s 2024-05-20T20:57:59.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 3758 7.95 MB/sec warmup 5 sec latency 18.099 ms 2024-05-20T20:57:59.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:57:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:00.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 4162 6.68 MB/sec warmup 6 sec latency 12.569 ms 2024-05-20T20:58:01.083 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:00 smithi026 ceph-mon[31990]: pgmap v444: 97 pgs: 97 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.0 MiB/s wr, 54 op/s 2024-05-20T20:58:01.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:00 smithi055 ceph-mon[35213]: pgmap v444: 97 pgs: 97 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.0 MiB/s wr, 54 op/s 2024-05-20T20:58:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:00 smithi144 ceph-mon[35660]: pgmap v444: 97 pgs: 97 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.0 MiB/s wr, 54 op/s 2024-05-20T20:58:01.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 4770 6.02 MB/sec warmup 7 sec latency 25.934 ms 2024-05-20T20:58:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 -- 172.21.15.26:0/4202130042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 msgr2=0x7fbe6c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202130042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c108900 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009990 tx=0x7fbe5c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 -- 172.21.15.26:0/4202130042 shutdown_connections 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202130042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c108900 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202130042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe6c102e00 0x7fbe6c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202130042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6c102440 0x7fbe6c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 -- 172.21.15.26:0/4202130042 >> 172.21.15.26:0/4202130042 conn(0x7fbe6c0793f0 msgr2=0x7fbe6c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 -- 172.21.15.26:0/4202130042 shutdown_connections 2024-05-20T20:58:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.740+0000 7fbe71743700 1 -- 172.21.15.26:0/4202130042 wait complete. 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.741+0000 7fbe71743700 1 Processor -- start 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.741+0000 7fbe71743700 1 -- start start 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe71743700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6c102440 0x7fbe6c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe71743700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe6c102e00 0x7fbe6c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe71743700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe71743700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1124a0 con 0x7fbe6c108520 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe71743700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c112320 con 0x7fbe6c102e00 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe71743700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c112620 con 0x7fbe6c102440 2024-05-20T20:58:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59694/0 (socket says 172.21.15.26:59694) 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe6c102e00 0x7fbe6c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/4202107258 learned_addr learned my addr 172.21.15.26:0/4202107258 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6affd700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6c102440 0x7fbe6c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/4202107258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6c102440 msgr2=0x7fbe6c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6c102440 0x7fbe6c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/4202107258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe6c102e00 msgr2=0x7fbe6c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe6c102e00 0x7fbe6c198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.742+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/4202107258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c1a8040 con 0x7fbe6c108520 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.743+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c199250 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400ef30 tx=0x7fbe6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.743+0000 7fbe53fff700 1 -- 172.21.15.26:0/4202107258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640092e0 con 0x7fbe6c108520 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.743+0000 7fbe53fff700 1 -- 172.21.15.26:0/4202107258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe64004bb0 con 0x7fbe6c108520 2024-05-20T20:58:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.743+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c1a82d0 con 0x7fbe6c108520 2024-05-20T20:58:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.743+0000 7fbe53fff700 1 -- 172.21.15.26:0/4202107258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64004020 con 0x7fbe6c108520 2024-05-20T20:58:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.743+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c1a87c0 con 0x7fbe6c108520 2024-05-20T20:58:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.746+0000 7fbe53fff700 1 -- 172.21.15.26:0/4202107258 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbe6401b030 con 0x7fbe6c108520 2024-05-20T20:58:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.746+0000 7fbe53fff700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe540b2880 0x7fbe540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.747+0000 7fbe6affd700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe540b2880 0x7fbe540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.747+0000 7fbe53fff700 1 -- 172.21.15.26:0/4202107258 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbe640e1c60 con 0x7fbe6c108520 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.747+0000 7fbe6affd700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe540b2880 0x7fbe540b4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00b2e0 tx=0x7fbe5c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.747+0000 7fbe7bfd9b80 1 Processor -- start 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.747+0000 7fbe7bfd9b80 1 -- start start 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.747+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eecac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eece940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eecf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56303ee29c90 con 0x56303eecb210 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56303ee19830 con 0x56303eeca860 2024-05-20T20:58:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56303ee1d870 con 0x56303eecef60 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eece940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eece940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59704/0 (socket says 172.21.15.26:59704) 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/3322706868 learned_addr learned my addr 172.21.15.26:0/3322706868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6affd700 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eecac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eecf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/3322706868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 msgr2=0x56303eecf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eecf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/3322706868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 msgr2=0x56303eecac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eecac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/3322706868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303ed698d0 con 0x56303eecb210 2024-05-20T20:58:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.748+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eece940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b700 tx=0x7fbe6000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b5bb3733579277 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe517fa700 1 -- 172.21.15.26:0/3322706868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60004a30 con 0x56303eecb210 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe517fa700 1 -- 172.21.15.26:0/3322706868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe60015070 con 0x56303eecb210 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe517fa700 1 -- 172.21.15.26:0/3322706868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60003980 con 0x56303eecb210 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/3322706868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 msgr2=0x56303eece940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eece940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b700 tx=0x7fbe6000ba10 comp rx=0 tx=0).stop 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/3322706868 shutdown_connections 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eecf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eecac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/3322706868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eece940 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.749+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/3322706868 >> 172.21.15.26:0/3322706868 conn(0x56303edf8080 msgr2=0x56303eec4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/3322706868 shutdown_connections 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/3322706868 wait complete. 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 -- start start 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 Processor -- start 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eed3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eed3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56303ed1ffa0 con 0x56303eecb210 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56303edbb7c0 con 0x56303eeca860 2024-05-20T20:58:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56303eed7120 con 0x56303eecef60 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.750+0000 7fbe7bfd9b80 10 client.0 _ll_register_callbacks cb 0x56303edb4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eed3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eed3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59716/0 (socket says 172.21.15.26:59716) 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/2192919700 learned_addr learned my addr 172.21.15.26:0/2192919700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eed3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/2192919700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 msgr2=0x56303eed3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eed3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 msgr2=0x56303eed63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 -- 172.21.15.26:0/2192919700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303edb8690 con 0x56303eecb210 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eed3720 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe60003ab0 tx=0x7fbe60003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f7bc2514e71f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2192919700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60027060 con 0x56303eecb210 2024-05-20T20:58:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe60015070 con 0x56303eecb210 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.751+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2192919700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbe60005030 con 0x56303eecb210 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2192919700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe60005230 con 0x56303eecb210 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 10 client.15924 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56303eee1880 con 0x56303eecb210 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2192919700 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe60015070 con 0x56303eecb210 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2192919700 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbe6002b030 con 0x56303eecb210 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 20 client.15924 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303eee61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 4 client.15924 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.752+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303ee969e0 con 0x56303eee3db0 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.753+0000 7fbe6affd700 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303eee61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.753+0000 7fbe4eff5700 10 client.15924 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.753+0000 7fbe6affd700 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303eee61a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54ae5a53b2a075be server_cookie=ee37db8c7182229a in_seq=0 out_seq=0 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2192919700 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56303ee969e0 con 0x56303eee3db0 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe4eff5700 10 client.15924 handle_command_reply: tid=0 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 client.15924 shutdown 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 msgr2=0x56303eed3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eed3720 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe60003ab0 tx=0x7fbe60003b90 comp rx=0 tx=0).stop 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 shutdown_connections 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303eee61a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe4eff5700 0 client.15924 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecef60 0x56303eed3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2192919700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecb210 0x56303eed3720 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 >> 172.21.15.26:0/2192919700 conn(0x56303edf8080 msgr2=0x56303eed0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.754+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 shutdown_connections 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.755+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2192919700 wait complete. 2024-05-20T20:58:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.755+0000 7fbe7bfd9b80 20 client.15924 trim_cache size 0 max 16384 2024-05-20T20:58:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.772+0000 7fbe7bfd9b80 1 Processor -- start 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 -- start start 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eec9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eeeb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1b2840 con 0x56303eecef60 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1b00d0 con 0x56303eeca860 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1adf20 con 0x56303eecb210 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eeeb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eeeb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59722/0 (socket says 172.21.15.26:59722) 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/1954816537 learned_addr learned my addr 172.21.15.26:0/1954816537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eec9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/1954816537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 msgr2=0x56303eec9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eec9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/1954816537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 msgr2=0x56303eed28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.773+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/1954816537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303eed4600 con 0x56303eecef60 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.774+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eeeb7a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbe64004b80 tx=0x7fbe640a8a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d5c3ba80f6cc95 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.774+0000 7fbe50ff9700 1 -- 172.21.15.26:0/1954816537 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640c3070 con 0x56303eecef60 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.774+0000 7fbe50ff9700 1 -- 172.21.15.26:0/1954816537 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe640a99b0 con 0x56303eecef60 2024-05-20T20:58:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.774+0000 7fbe50ff9700 1 -- 172.21.15.26:0/1954816537 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640a9770 con 0x56303eecef60 2024-05-20T20:58:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.774+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/1954816537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 msgr2=0x56303eeeb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.774+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eeeb7a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbe64004b80 tx=0x7fbe640a8a70 comp rx=0 tx=0).stop 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/1954816537 shutdown_connections 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eeeb7a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eed28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/1954816537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eec9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/1954816537 >> 172.21.15.26:0/1954816537 conn(0x56303edf8080 msgr2=0x56303eec9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/1954816537 shutdown_connections 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/1954816537 wait complete. 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- start start 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 Processor -- start 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eedc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eedc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eecbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1ae590 con 0x56303eecef60 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1afa00 con 0x56303eeca860 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1aec60 con 0x56303eecb210 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.775+0000 7fbe7bfd9b80 10 client.0 _ll_register_callbacks cb 0x56303edb4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eecbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eecbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59726/0 (socket says 172.21.15.26:59726) 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/2158753917 learned_addr learned my addr 172.21.15.26:0/2158753917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6affd700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eedc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6a7fc700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eedc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/2158753917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 msgr2=0x56303eedc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eedc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 msgr2=0x56303eedc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eedc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 -- 172.21.15.26:0/2158753917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303eecc320 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe6b7fe700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eecbca0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbe640a5330 tx=0x7fbe640ad660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aee16785e53eeac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2158753917 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6406d060 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2158753917 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbe640b1e00 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.776+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe640a99b0 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2158753917 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe640c3040 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 10 client.15930 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56303ef4dfd0 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2158753917 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe640a92b0 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2158753917 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbe640a8680 con 0x56303eecef60 2024-05-20T20:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 10 client.15930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 20 client.15930 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303ef50530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 4 client.15930 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbe640a99b0 con 0x56303eee3db0 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.777+0000 7fbe6affd700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303ef50530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe6affd700 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303ef50530 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9b27bd502f99cd5 server_cookie=f42f11ab2ba96909 in_seq=0 out_seq=0 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe4eff5700 10 client.15930 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe4eff5700 1 -- 172.21.15.26:0/2158753917 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbe640a99b0 con 0x56303eee3db0 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe4eff5700 10 client.15930 handle_command_reply: tid=0 2024-05-20T20:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 client.15930 shutdown 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 msgr2=0x56303eecbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eecbca0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbe640a5330 tx=0x7fbe640ad660 comp rx=0 tx=0).stop 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 shutdown_connections 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56303eee3db0 0x56303ef50530 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56303eecef60 0x56303eecbca0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56303eeca860 0x56303eedc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 --2- 172.21.15.26:0/2158753917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56303eecb210 0x56303eedc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 >> 172.21.15.26:0/2158753917 conn(0x56303edf8080 msgr2=0x56303eec9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe4eff5700 0 client.15930 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 shutdown_connections 2024-05-20T20:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.778+0000 7fbe7bfd9b80 1 -- 172.21.15.26:0/2158753917 wait complete. 2024-05-20T20:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe7bfd9b80 20 client.15930 trim_cache size 0 max 16384 2024-05-20T20:58:01.784 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe540b2880 msgr2=0x7fbe540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe540b2880 0x7fbe540b4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00b2e0 tx=0x7fbe5c005fb0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 msgr2=0x7fbe6c199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c199250 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400ef30 tx=0x7fbe6400c3a0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 shutdown_connections 2024-05-20T20:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbe540b2880 0x7fbe540b4d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe6c108520 0x7fbe6c199250 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6c102440 0x7fbe6c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 --2- 172.21.15.26:0/4202107258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe6c102e00 0x7fbe6c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 >> 172.21.15.26:0/4202107258 conn(0x7fbe6c0793f0 msgr2=0x7fbe6c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.779+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 shutdown_connections 2024-05-20T20:58:01.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:01.780+0000 7fbe71743700 1 -- 172.21.15.26:0/4202107258 wait complete. 2024-05-20T20:58:02.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 5540 5.70 MB/sec warmup 8 sec latency 16.273 ms 2024-05-20T20:58:02.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:58:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:03.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:02 smithi026 ceph-mon[31990]: pgmap v445: 97 pgs: 97 active+clean; 24 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 62 op/s 2024-05-20T20:58:03.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:02 smithi055 ceph-mon[35213]: pgmap v445: 97 pgs: 97 active+clean; 24 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 62 op/s 2024-05-20T20:58:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:02 smithi144 ceph-mon[35660]: pgmap v445: 97 pgs: 97 active+clean; 24 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 62 op/s 2024-05-20T20:58:03.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 6167 5.35 MB/sec warmup 9 sec latency 19.398 ms 2024-05-20T20:58:04.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 6891 5.45 MB/sec warmup 10 sec latency 18.966 ms 2024-05-20T20:58:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 -- 172.21.15.26:0/4116168242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a410ffe0 msgr2=0x7fe3a41123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 --2- 172.21.15.26:0/4116168242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a410ffe0 0x7fe3a41123d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b0a0 tx=0x7fe39c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 -- 172.21.15.26:0/4116168242 shutdown_connections 2024-05-20T20:58:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 --2- 172.21.15.26:0/4116168242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a410ffe0 0x7fe3a41123d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 --2- 172.21.15.26:0/4116168242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406bae0 0x7fe3a406bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 --2- 172.21.15.26:0/4116168242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a406c400 0x7fe3a410faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 -- 172.21.15.26:0/4116168242 >> 172.21.15.26:0/4116168242 conn(0x7fe3a40793b0 msgr2=0x7fe3a40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 -- 172.21.15.26:0/4116168242 shutdown_connections 2024-05-20T20:58:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.662+0000 7fe3aa179700 1 -- 172.21.15.26:0/4116168242 wait complete. 2024-05-20T20:58:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 Processor -- start 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 -- start start 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 0x7fe3a41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406c400 0x7fe3a41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a410ffe0 0x7fe3a419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4115790 con 0x7fe3a406bae0 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4115610 con 0x7fe3a406c400 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3aa179700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4115910 con 0x7fe3a410ffe0 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 0x7fe3a41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406c400 0x7fe3a41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 0x7fe3a41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59746/0 (socket says 172.21.15.26:59746) 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.663+0000 7fe3a37fe700 1 -- 172.21.15.26:0/1466628718 learned_addr learned my addr 172.21.15.26:0/1466628718 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a410ffe0 0x7fe3a419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a37fe700 1 -- 172.21.15.26:0/1466628718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a410ffe0 msgr2=0x7fe3a419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a410ffe0 0x7fe3a419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a37fe700 1 -- 172.21.15.26:0/1466628718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406c400 msgr2=0x7fe3a41a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406c400 0x7fe3a41a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a37fe700 1 -- 172.21.15.26:0/1466628718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a419c250 con 0x7fe3a406bae0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406c400 0x7fe3a41a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 0x7fe3a41a1930 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe39400e9a0 tx=0x7fe39400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a0ff9700 1 -- 172.21.15.26:0/1466628718 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400ca10 con 0x7fe3a406bae0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a419c4e0 con 0x7fe3a406bae0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a0ff9700 1 -- 172.21.15.26:0/1466628718 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe39400cbb0 con 0x7fe3a406bae0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3a0ff9700 1 -- 172.21.15.26:0/1466628718 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394018680 con 0x7fe3a406bae0 2024-05-20T20:58:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.664+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41a8800 con 0x7fe3a406bae0 2024-05-20T20:58:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.667+0000 7fe3a0ff9700 1 -- 172.21.15.26:0/1466628718 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe394010430 con 0x7fe3a406bae0 2024-05-20T20:58:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.668+0000 7fe3a0ff9700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe38c0b2880 0x7fe38c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.668+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe38c0b2880 0x7fe38c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.668+0000 7fe3a0ff9700 1 -- 172.21.15.26:0/1466628718 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe3940e2410 con 0x7fe3a406bae0 2024-05-20T20:58:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.668+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe38c0b2880 0x7fe38c0b4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3a41039f0 tx=0x7fe39800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.668+0000 7fe3b4a0fb80 1 Processor -- start 2024-05-20T20:58:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 -- start start 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca187320 0x560fca187700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 0x560fca181d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560fca0dfd70 con 0x560fca182260 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560fca0cf910 con 0x560fca1818a0 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560fca0d3950 con 0x560fca187320 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 0x560fca181d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 0x560fca181d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:58744/0 (socket says 172.21.15.26:58744) 2024-05-20T20:58:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/2478088299 learned_addr learned my addr 172.21.15.26:0/2478088299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca187320 0x560fca187700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/2478088299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca187320 msgr2=0x560fca187700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca187320 0x560fca187700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/2478088299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 msgr2=0x560fca18e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/2478088299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fca078fb0 con 0x560fca1818a0 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.669+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca187320 0x560fca187700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.670+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 0x560fca181d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe398011790 tx=0x7fe398013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7600f59599f43f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.670+0000 7fe389ffb700 1 -- 172.21.15.26:0/2478088299 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39801e420 con 0x560fca1818a0 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.670+0000 7fe389ffb700 1 -- 172.21.15.26:0/2478088299 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe39801ea50 con 0x560fca1818a0 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.670+0000 7fe389ffb700 1 -- 172.21.15.26:0/2478088299 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980269e0 con 0x560fca1818a0 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2478088299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 msgr2=0x560fca181d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 0x560fca181d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe398011790 tx=0x7fe398013000 comp rx=0 tx=0).stop 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2478088299 shutdown_connections 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca187320 0x560fca187700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2478088299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca1818a0 0x560fca181d20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2478088299 >> 172.21.15.26:0/2478088299 conn(0x560fca0ad240 msgr2=0x560fca17c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2478088299 shutdown_connections 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2478088299 wait complete. 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 -- start start 2024-05-20T20:58:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.671+0000 7fe3b4a0fb80 1 Processor -- start 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca1818a0 0x560fca18dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560fc9fd5fa0 con 0x560fca182260 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560fca037da0 con 0x560fca187320 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560fca18e6e0 con 0x560fca1818a0 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3b4a0fb80 10 client.0 _ll_register_callbacks cb 0x560fca06a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:58752/0 (socket says 172.21.15.26:58752) 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a3fff700 1 -- 172.21.15.26:0/80710211 learned_addr learned my addr 172.21.15.26:0/80710211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca1818a0 0x560fca18dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/80710211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca1818a0 msgr2=0x560fca18dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca1818a0 0x560fca18dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 msgr2=0x560fca18b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a2ffd700 1 -- 172.21.15.26:0/80710211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fca01f8d0 con 0x560fca182260 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.672+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18b030 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe398011760 tx=0x7fe39800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcc583a4e392948 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe382ffd700 1 -- 172.21.15.26:0/80710211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398036060 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe382ffd700 1 -- 172.21.15.26:0/80710211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe39801e420 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39801ea50 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe382ffd700 1 -- 172.21.15.26:0/80710211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe39801e620 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe3b4a0fb80 10 client.15942 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560fca089760 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe382ffd700 1 -- 172.21.15.26:0/80710211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe39801ea50 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.673+0000 7fe382ffd700 1 -- 172.21.15.26:0/80710211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe39803a030 con 0x560fca182260 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3b4a0fb80 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3b4a0fb80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3b4a0fb80 20 client.15942 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca19b1e0 0x560fca19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3b4a0fb80 4 client.15942 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fca0e1cc0 con 0x560fca19b1e0 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca19b1e0 0x560fca19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe382ffd700 10 client.15942 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.674+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca19b1e0 0x560fca19d5d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2718f739154538 server_cookie=8a8493e3e202c589 in_seq=0 out_seq=0 2024-05-20T20:58:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe382ffd700 1 -- 172.21.15.26:0/80710211 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560fca0e1cc0 con 0x560fca19b1e0 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe382ffd700 10 client.15942 handle_command_reply: tid=0 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 client.15942 shutdown 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 msgr2=0x560fca18b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18b030 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe398011760 tx=0x7fe39800bd30 comp rx=0 tx=0).stop 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 shutdown_connections 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca19b1e0 0x560fca19d5d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca1818a0 0x560fca18dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/80710211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca182260 0x560fca18b030 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 >> 172.21.15.26:0/80710211 conn(0x560fca0ad240 msgr2=0x560fca17baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe382ffd700 0 client.15942 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.675+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 shutdown_connections 2024-05-20T20:58:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.676+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/80710211 wait complete. 2024-05-20T20:58:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.676+0000 7fe3b4a0fb80 20 client.15942 trim_cache size 0 max 16384 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 Processor -- start 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 -- start start 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca17e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca1867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41b2740 con 0x560fca1818a0 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41affd0 con 0x560fca187320 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41ade20 con 0x560fca182260 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca17e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca17e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59772/0 (socket says 172.21.15.26:59772) 2024-05-20T20:58:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a37fe700 1 -- 172.21.15.26:0/3024966092 learned_addr learned my addr 172.21.15.26:0/3024966092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca1867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a3fff700 1 -- 172.21.15.26:0/3024966092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 msgr2=0x560fca1867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca1867d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a3fff700 1 -- 172.21.15.26:0/3024966092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 msgr2=0x560fca17e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca17e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a3fff700 1 -- 172.21.15.26:0/3024966092 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fca189020 con 0x560fca187320 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.692+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca17e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.693+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18c860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009610 tx=0x7fe39c00b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d749c3507a481417 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.693+0000 7fe3897fa700 1 -- 172.21.15.26:0/3024966092 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c003c00 con 0x560fca187320 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.693+0000 7fe3897fa700 1 -- 172.21.15.26:0/3024966092 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe39c003da0 con 0x560fca187320 2024-05-20T20:58:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.693+0000 7fe3897fa700 1 -- 172.21.15.26:0/3024966092 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c005280 con 0x560fca187320 2024-05-20T20:58:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/3024966092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 msgr2=0x560fca18c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18c860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009610 tx=0x7fe39c00b720 comp rx=0 tx=0).stop 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/3024966092 shutdown_connections 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca17e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca18c860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/3024966092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca1867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/3024966092 >> 172.21.15.26:0/3024966092 conn(0x560fca0ad240 msgr2=0x560fca181120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/3024966092 shutdown_connections 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/3024966092 wait complete. 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 -- start start 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.694+0000 7fe3b4a0fb80 1 Processor -- start 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca19b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41ae490 con 0x560fca1818a0 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41af900 con 0x560fca187320 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41aeb60 con 0x560fca182260 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3b4a0fb80 10 client.0 _ll_register_callbacks cb 0x560fca06a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:58766/0 (socket says 172.21.15.26:58766) 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 -- 172.21.15.26:0/2283848612 learned_addr learned my addr 172.21.15.26:0/2283848612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca1a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59784/0 (socket says 172.21.15.26:59784) 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a2ffd700 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 -- 172.21.15.26:0/2283848612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 msgr2=0x560fca19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 -- 172.21.15.26:0/2283848612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 msgr2=0x560fca1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.695+0000 7fe3a3fff700 1 -- 172.21.15.26:0/2283848612 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fca19bf90 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe3a3fff700 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca19b720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b010 tx=0x7fe39c0076d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1c13f15c54c70f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe382ffd700 1 -- 172.21.15.26:0/2283848612 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c02d070 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe382ffd700 1 -- 172.21.15.26:0/2283848612 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe39c027440 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39c003da0 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe382ffd700 1 -- 172.21.15.26:0/2283848612 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe39c007db0 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe3b4a0fb80 10 client.25502 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560fca204a80 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe382ffd700 1 -- 172.21.15.26:0/2283848612 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe39c003da0 con 0x560fca187320 2024-05-20T20:58:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe382ffd700 1 -- 172.21.15.26:0/2283848612 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe39c025070 con 0x560fca187320 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe3b4a0fb80 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.696+0000 7fe3b4a0fb80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe3b4a0fb80 20 client.25502 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca2050f0 0x560fca207510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe3b4a0fb80 4 client.25502 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fca0db4a0 con 0x560fca2050f0 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca2050f0 0x560fca207510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe382ffd700 10 client.25502 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.697+0000 7fe3a37fe700 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca2050f0 0x560fca207510 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79afd8aa28ed8d4 server_cookie=b0e79cbd282ac848 in_seq=0 out_seq=0 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe382ffd700 1 -- 172.21.15.26:0/2283848612 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fca0db4a0 con 0x560fca2050f0 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe382ffd700 10 client.25502 handle_command_reply: tid=0 2024-05-20T20:58:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 client.25502 shutdown 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 msgr2=0x560fca19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca19b720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b010 tx=0x7fe39c0076d0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 shutdown_connections 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560fca2050f0 0x560fca207510 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fca1818a0 0x560fca1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560fca187320 0x560fca19b720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 --2- 172.21.15.26:0/2283848612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560fca182260 0x560fca19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 >> 172.21.15.26:0/2283848612 conn(0x560fca0ad240 msgr2=0x560fca184ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe382ffd700 0 client.25502 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 shutdown_connections 2024-05-20T20:58:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 1 -- 172.21.15.26:0/2283848612 wait complete. 2024-05-20T20:58:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.698+0000 7fe3b4a0fb80 20 client.25502 trim_cache size 0 max 16384 2024-05-20T20:58:04.704 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe38c0b2880 msgr2=0x7fe38c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe38c0b2880 0x7fe38c0b4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3a41039f0 tx=0x7fe39800b3c0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 msgr2=0x7fe3a41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 0x7fe3a41a1930 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe39400e9a0 tx=0x7fe39400ecb0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 shutdown_connections 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe38c0b2880 0x7fe38c0b4d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3a406bae0 0x7fe3a41a1930 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a410ffe0 0x7fe3a419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 --2- 172.21.15.26:0/1466628718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3a406c400 0x7fe3a41a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 >> 172.21.15.26:0/1466628718 conn(0x7fe3a40793b0 msgr2=0x7fe3a41114f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 shutdown_connections 2024-05-20T20:58:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:04.699+0000 7fe3aa179700 1 -- 172.21.15.26:0/1466628718 wait complete. 2024-05-20T20:58:05.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:04 smithi055 ceph-mon[35213]: pgmap v446: 97 pgs: 97 active+clean; 23 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.5 MiB/s wr, 74 op/s 2024-05-20T20:58:05.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:04 smithi026 ceph-mon[31990]: pgmap v446: 97 pgs: 97 active+clean; 23 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.5 MiB/s wr, 74 op/s 2024-05-20T20:58:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:04 smithi144 ceph-mon[35660]: pgmap v446: 97 pgs: 97 active+clean; 23 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.5 MiB/s wr, 74 op/s 2024-05-20T20:58:05.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 7503 5.38 MB/sec warmup 11 sec latency 11.991 ms 2024-05-20T20:58:06.519 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:58:06.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:58:06.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 8048 5.00 MB/sec warmup 12 sec latency 19.091 ms 2024-05-20T20:58:06.799 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:07.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:06 smithi026 ceph-mon[31990]: pgmap v447: 97 pgs: 97 active+clean; 25 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 2.9 MiB/s wr, 95 op/s 2024-05-20T20:58:07.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:06 smithi055 ceph-mon[35213]: pgmap v447: 97 pgs: 97 active+clean; 25 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 2.9 MiB/s wr, 95 op/s 2024-05-20T20:58:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:06 smithi144 ceph-mon[35660]: pgmap v447: 97 pgs: 97 active+clean; 25 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 2.9 MiB/s wr, 95 op/s 2024-05-20T20:58:07.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 8774 4.96 MB/sec warmup 13 sec latency 12.868 ms 2024-05-20T20:58:08.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 9505 4.73 MB/sec warmup 14 sec latency 17.199 ms 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 -- 172.21.15.26:0/1586385276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90108520 msgr2=0x7fed90108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 --2- 172.21.15.26:0/1586385276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90108520 0x7fed90108900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009990 tx=0x7fed8c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 -- 172.21.15.26:0/1586385276 shutdown_connections 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 --2- 172.21.15.26:0/1586385276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90108520 0x7fed90108900 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 --2- 172.21.15.26:0/1586385276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102e00 0x7fed9010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 --2- 172.21.15.26:0/1586385276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed90102440 0x7fed901028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 -- 172.21.15.26:0/1586385276 >> 172.21.15.26:0/1586385276 conn(0x7fed900793f0 msgr2=0x7fed90079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 -- 172.21.15.26:0/1586385276 shutdown_connections 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.657+0000 7fed978b4700 1 -- 172.21.15.26:0/1586385276 wait complete. 2024-05-20T20:58:08.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 Processor -- start 2024-05-20T20:58:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 -- start start 2024-05-20T20:58:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102440 0x7fed901044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 0x7fed90104a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed90108520 0x7fed90104f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed901124f0 con 0x7fed90102e00 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed90112370 con 0x7fed90102440 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed978b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed90112670 con 0x7fed90108520 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed94e4f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 0x7fed90104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed94e4f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 0x7fed90104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59796/0 (socket says 172.21.15.26:59796) 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed94e4f700 1 -- 172.21.15.26:0/40721019 learned_addr learned my addr 172.21.15.26:0/40721019 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.658+0000 7fed95650700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102440 0x7fed901044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed95e51700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed90108520 0x7fed90104f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed94e4f700 1 -- 172.21.15.26:0/40721019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed90108520 msgr2=0x7fed90104f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed94e4f700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed90108520 0x7fed90104f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed94e4f700 1 -- 172.21.15.26:0/40721019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102440 msgr2=0x7fed901044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed94e4f700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102440 0x7fed901044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed94e4f700 1 -- 172.21.15.26:0/40721019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed901a80f0 con 0x7fed90102e00 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed95650700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102440 0x7fed901044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed94e4f700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 0x7fed90104a10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fed8000e9c0 tx=0x7fed8000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed867fc700 1 -- 172.21.15.26:0/40721019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8000cbd0 con 0x7fed90102e00 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed867fc700 1 -- 172.21.15.26:0/40721019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed8000ee60 con 0x7fed90102e00 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901a83e0 con 0x7fed90102e00 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.659+0000 7fed867fc700 1 -- 172.21.15.26:0/40721019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed800054b0 con 0x7fed90102e00 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.660+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901a8920 con 0x7fed90102e00 2024-05-20T20:58:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.663+0000 7fed867fc700 1 -- 172.21.15.26:0/40721019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fed80004750 con 0x7fed90102e00 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.663+0000 7fed867fc700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fed7c0b2900 0x7fed7c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.663+0000 7feda214ab80 1 Processor -- start 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 -- start start 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed867fc700 1 -- 172.21.15.26:0/40721019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fed800e24c0 con 0x7fed90102e00 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95650700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fed7c0b2900 0x7fed7c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc654f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 0x55cc65500100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc65451df0 con 0x55cc654f42e0 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cc65441990 con 0x55cc654f93a0 2024-05-20T20:58:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7feda214ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc654459d0 con 0x55cc654f3920 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 0x55cc65500100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 0x55cc65500100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59806/0 (socket says 172.21.15.26:59806) 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 -- 172.21.15.26:0/674178955 learned_addr learned my addr 172.21.15.26:0/674178955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed94e4f700 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc654f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95650700 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95650700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fed7c0b2900 0x7fed7c0b4dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b2e0 tx=0x7fed8c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 -- 172.21.15.26:0/674178955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 msgr2=0x55cc654f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc654f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 -- 172.21.15.26:0/674178955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 msgr2=0x55cc654f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.664+0000 7fed95e51700 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.665+0000 7fed95e51700 1 -- 172.21.15.26:0/674178955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc653eb030 con 0x55cc654f42e0 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.665+0000 7fed95e51700 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 0x55cc65500100 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fed88011680 tx=0x7fed88009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4af98c275f065d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.665+0000 7fed7b7fe700 1 -- 172.21.15.26:0/674178955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88019030 con 0x55cc654f42e0 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.665+0000 7fed7b7fe700 1 -- 172.21.15.26:0/674178955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed880092e0 con 0x55cc654f42e0 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.665+0000 7fed7b7fe700 1 -- 172.21.15.26:0/674178955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88007500 con 0x55cc654f42e0 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 -- 172.21.15.26:0/674178955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 msgr2=0x55cc65500100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 0x55cc65500100 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fed88011680 tx=0x7fed88009920 comp rx=0 tx=0).stop 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 -- 172.21.15.26:0/674178955 shutdown_connections 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f42e0 0x55cc65500100 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 --2- 172.21.15.26:0/674178955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc654f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 -- 172.21.15.26:0/674178955 >> 172.21.15.26:0/674178955 conn(0x55cc6541f2c0 msgr2=0x55cc654ee460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 -- 172.21.15.26:0/674178955 shutdown_connections 2024-05-20T20:58:08.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 -- 172.21.15.26:0/674178955 wait complete. 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 -- start start 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.666+0000 7feda214ab80 1 Processor -- start 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f42e0 0x55cc654fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc65348020 con 0x55cc654f3920 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cc653a9e20 con 0x55cc654f93a0 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc65500760 con 0x55cc654f42e0 2024-05-20T20:58:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7feda214ab80 10 client.0 _ll_register_callbacks cb 0x55cc653dc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59812/0 (socket says 172.21.15.26:59812) 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 -- 172.21.15.26:0/4147674191 learned_addr learned my addr 172.21.15.26:0/4147674191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95e51700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed94e4f700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f42e0 0x55cc654fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 -- 172.21.15.26:0/4147674191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f42e0 msgr2=0x55cc654fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f42e0 0x55cc654fd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 -- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 msgr2=0x55cc654fd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654fd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95650700 1 -- 172.21.15.26:0/4147674191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc65391950 con 0x55cc654f3920 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.667+0000 7fed95e51700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed95650700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654ffd20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009990 tx=0x7fed8c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07ebd539f87dafd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed78ff9700 1 -- 172.21.15.26:0/4147674191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c035060 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed78ff9700 1 -- 172.21.15.26:0/4147674191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fed8c00f460 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed880092e0 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed78ff9700 1 -- 172.21.15.26:0/4147674191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed8c00f660 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 10 client.15960 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc653fb7e0 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed78ff9700 1 -- 172.21.15.26:0/4147674191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed880092e0 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7fed78ff9700 1 -- 172.21.15.26:0/4147674191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fed8c039030 con 0x55cc654f3920 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 20 client.15960 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc6550d260 0x55cc6550f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 4 client.15960 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.668+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc65453d40 con 0x55cc6550d260 2024-05-20T20:58:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.669+0000 7fed94e4f700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc6550d260 0x55cc6550f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.669+0000 7fed94e4f700 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc6550d260 0x55cc6550f650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42260abc96b0dbf server_cookie=9dc529197f09e53 in_seq=0 out_seq=0 2024-05-20T20:58:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.669+0000 7fed78ff9700 10 client.15960 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7fed78ff9700 1 -- 172.21.15.26:0/4147674191 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cc65453d40 con 0x55cc6550d260 2024-05-20T20:58:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7fed78ff9700 10 client.15960 handle_command_reply: tid=0 2024-05-20T20:58:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 client.15960 shutdown 2024-05-20T20:58:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 msgr2=0x55cc654ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654ffd20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009990 tx=0x7fed8c009310 comp rx=0 tx=0).stop 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 shutdown_connections 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc6550d260 0x55cc6550f650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654ffd20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f93a0 0x55cc654fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 --2- 172.21.15.26:0/4147674191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f42e0 0x55cc654fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 >> 172.21.15.26:0/4147674191 conn(0x55cc6541f2c0 msgr2=0x55cc654edb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.672+0000 7fed78ff9700 0 client.15960 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.673+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 shutdown_connections 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.673+0000 7feda214ab80 1 -- 172.21.15.26:0/4147674191 wait complete. 2024-05-20T20:58:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.673+0000 7feda214ab80 20 client.15960 trim_cache size 0 max 16384 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.696+0000 7feda214ab80 1 Processor -- start 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.696+0000 7feda214ab80 1 -- start start 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7feda214ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7feda214ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc654ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7feda214ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f93a0 0x55cc654fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7feda214ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed901b27e0 con 0x55cc654f3920 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7feda214ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed901b0070 con 0x55cc654f42e0 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7feda214ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed901adec0 con 0x55cc654f93a0 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59814/0 (socket says 172.21.15.26:59814) 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 -- 172.21.15.26:0/1933912336 learned_addr learned my addr 172.21.15.26:0/1933912336 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed94e4f700 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc654ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95e51700 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f93a0 0x55cc654fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 -- 172.21.15.26:0/1933912336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f93a0 msgr2=0x55cc654fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f93a0 0x55cc654fef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 -- 172.21.15.26:0/1933912336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 msgr2=0x55cc654ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc654ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 -- 172.21.15.26:0/1933912336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc654ff960 con 0x55cc654f3920 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.697+0000 7fed95650700 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654fc1a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00f5f0 tx=0x7fed8c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfb2dd71f13f8ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7fed7affd700 1 -- 172.21.15.26:0/1933912336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c035030 con 0x55cc654f3920 2024-05-20T20:58:08.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7fed7affd700 1 -- 172.21.15.26:0/1933912336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed8c00fc90 con 0x55cc654f3920 2024-05-20T20:58:08.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7fed7affd700 1 -- 172.21.15.26:0/1933912336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c00e710 con 0x55cc654f3920 2024-05-20T20:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 -- 172.21.15.26:0/1933912336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 msgr2=0x55cc654fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654fc1a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00f5f0 tx=0x7fed8c005fb0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 -- 172.21.15.26:0/1933912336 shutdown_connections 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f3920 0x55cc654fc1a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f93a0 0x55cc654fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 --2- 172.21.15.26:0/1933912336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc654ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 -- 172.21.15.26:0/1933912336 >> 172.21.15.26:0/1933912336 conn(0x55cc6541f2c0 msgr2=0x55cc654fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 -- 172.21.15.26:0/1933912336 shutdown_connections 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.698+0000 7feda214ab80 1 -- 172.21.15.26:0/1933912336 wait complete. 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 -- start start 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 Processor -- start 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc65514b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc655150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 0x55cc655155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed901ae530 con 0x55cc654f93a0 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed901af9a0 con 0x55cc654f42e0 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.699+0000 7feda214ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed901aec00 con 0x55cc654f3920 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7feda214ab80 10 client.0 _ll_register_callbacks cb 0x55cc653dc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 0x55cc655155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 0x55cc655155e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59830/0 (socket says 172.21.15.26:59830) 2024-05-20T20:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed94e4f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc655150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 -- 172.21.15.26:0/2168953049 learned_addr learned my addr 172.21.15.26:0/2168953049 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95650700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc65514b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 -- 172.21.15.26:0/2168953049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 msgr2=0x55cc65514b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc65514b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 -- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 msgr2=0x55cc655150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc655150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 -- 172.21.15.26:0/2168953049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc65576ed0 con 0x55cc654f93a0 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed94e4f700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc655150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed95e51700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 0x55cc655155e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed8800afe0 tx=0x7fed88004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29afdcd98ad9d16e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed78ff9700 1 -- 172.21.15.26:0/2168953049 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88003c10 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7fed78ff9700 1 -- 172.21.15.26:0/2168953049 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fed88003db0 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.700+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed8c00fc90 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7fed78ff9700 1 -- 172.21.15.26:0/2168953049 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed88003fb0 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 10 client.15966 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc655781f0 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7fed78ff9700 1 -- 172.21.15.26:0/2168953049 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed8c00fc90 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7fed78ff9700 1 -- 172.21.15.26:0/2168953049 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fed8801f480 con 0x55cc654f93a0 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 20 client.15966 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc65578710 0x55cc6557ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 4 client.15966 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.701+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cc6544d520 con 0x55cc65578710 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.702+0000 7fed95650700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc65578710 0x55cc6557ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.702+0000 7fed95650700 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc65578710 0x55cc6557ab30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66924a42824cba2 server_cookie=b10d79f913526fc7 in_seq=0 out_seq=0 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.702+0000 7fed78ff9700 10 client.15966 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7fed78ff9700 1 -- 172.21.15.26:0/2168953049 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cc6544d520 con 0x55cc65578710 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7fed78ff9700 10 client.15966 handle_command_reply: tid=0 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 client.15966 shutdown 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 msgr2=0x55cc655155e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 0x55cc655155e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed8800afe0 tx=0x7fed88004ef0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 shutdown_connections 2024-05-20T20:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc65578710 0x55cc6557ab30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc654f93a0 0x55cc655155e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc654f3920 0x55cc65514b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 --2- 172.21.15.26:0/2168953049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc654f42e0 0x55cc655150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 >> 172.21.15.26:0/2168953049 conn(0x55cc6541f2c0 msgr2=0x55cc654f31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.703+0000 7fed78ff9700 0 client.15966 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 shutdown_connections 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7feda214ab80 1 -- 172.21.15.26:0/2168953049 wait complete. 2024-05-20T20:58:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7feda214ab80 20 client.15966 trim_cache size 0 max 16384 2024-05-20T20:58:08.710 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:58:08.710 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:58:08.710 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "d7e3a5dd-157a-44f7-8383-f2ab42df6064", 2024-05-20T20:58:08.710 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:58:08.710 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fed7c0b2900 msgr2=0x7fed7c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7fed978b4700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fed7c0b2900 0x7fed7c0b4dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b2e0 tx=0x7fed8c01a040 comp rx=0 tx=0).stop 2024-05-20T20:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 msgr2=0x7fed90104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.704+0000 7fed978b4700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 0x7fed90104a10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fed8000e9c0 tx=0x7fed8000ecd0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 shutdown_connections 2024-05-20T20:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed90108520 0x7fed90104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed90102440 0x7fed901044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fed7c0b2900 0x7fed7c0b4dc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 --2- 172.21.15.26:0/40721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90102e00 0x7fed90104a10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 >> 172.21.15.26:0/40721019 conn(0x7fed900793f0 msgr2=0x7fed90106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 shutdown_connections 2024-05-20T20:58:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:08.705+0000 7fed978b4700 1 -- 172.21.15.26:0/40721019 wait complete. 2024-05-20T20:58:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:08 smithi144 ceph-mon[35660]: pgmap v448: 97 pgs: 97 active+clean; 26 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.3 MiB/s wr, 100 op/s 2024-05-20T20:58:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 -- 172.21.15.26:0/240854434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa128103560 msgr2=0x7fa128109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 --2- 172.21.15.26:0/240854434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa128103560 0x7fa128109df0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa120009ca0 tx=0x7fa12002a8c0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 -- 172.21.15.26:0/240854434 shutdown_connections 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 --2- 172.21.15.26:0/240854434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa128103560 0x7fa128109df0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 --2- 172.21.15.26:0/240854434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128102ba0 0x7fa128103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 --2- 172.21.15.26:0/240854434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1281019b0 0x7fa128101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 -- 172.21.15.26:0/240854434 >> 172.21.15.26:0/240854434 conn(0x7fa1280fcf20 msgr2=0x7fa1280ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 -- 172.21.15.26:0/240854434 shutdown_connections 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 -- 172.21.15.26:0/240854434 wait complete. 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.446+0000 7fa12f0da640 1 Processor -- start 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 -- start start 2024-05-20T20:58:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 0x7fa12806ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa128102ba0 0x7fa12819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128103560 0x7fa12819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810d3c0 con 0x7fa1281019b0 2024-05-20T20:58:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810d240 con 0x7fa128102ba0 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12f0da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810d540 con 0x7fa128103560 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 0x7fa12806ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 0x7fa12806ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59846/0 (socket says 172.21.15.26:59846) 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 -- 172.21.15.26:0/3540382662 learned_addr learned my addr 172.21.15.26:0/3540382662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa127fff640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa128102ba0 0x7fa12819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12d677640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128103560 0x7fa12819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 -- 172.21.15.26:0/3540382662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128103560 msgr2=0x7fa12819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128103560 0x7fa12819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 -- 172.21.15.26:0/3540382662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa128102ba0 msgr2=0x7fa12819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa128102ba0 0x7fa12819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 -- 172.21.15.26:0/3540382662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281a07b0 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12d677640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128103560 0x7fa12819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa12ce76640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 0x7fa12806ac70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e9c0 tx=0x7fa11c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.447+0000 7fa125ffb640 1 -- 172.21.15.26:0/3540382662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00cd80 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.448+0000 7fa125ffb640 1 -- 172.21.15.26:0/3540382662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa11c0044b0 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.448+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1281a0a40 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.448+0000 7fa125ffb640 1 -- 172.21.15.26:0/3540382662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c010430 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.448+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1281a0fe0 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.450+0000 7fa125ffb640 1 -- 172.21.15.26:0/3540382662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa11c003f00 con 0x7fa1281019b0 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 Processor -- start 2024-05-20T20:58:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 -- start start 2024-05-20T20:58:09.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:08 smithi055 ceph-mon[35213]: pgmap v448: 97 pgs: 97 active+clean; 26 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.3 MiB/s wr, 100 op/s 2024-05-20T20:58:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e5ad0 0x5567bf8e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8eb630 0x5567bf8edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567bf852c30 con 0x5567bf8eb630 2024-05-20T20:58:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5567bf8427d0 con 0x5567bf8e5ad0 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa131dfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567bf846810 con 0x5567bf8e64c0 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa125ffb640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1180b2920 0x7fa1180b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8eb630 0x5567bf8edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8eb630 0x5567bf8edac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59854/0 (socket says 172.21.15.26:59854) 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa127fff640 1 -- 172.21.15.26:0/4229713979 learned_addr learned my addr 172.21.15.26:0/4229713979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa12d677640 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e5ad0 0x5567bf8e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa12ce76640 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.451+0000 7fa12d677640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1180b2920 0x7fa1180b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa125ffb640 1 -- 172.21.15.26:0/3540382662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa11c0e2690 con 0x7fa1281019b0 2024-05-20T20:58:09.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:08 smithi026 ceph-mon[31990]: pgmap v448: 97 pgs: 97 active+clean; 26 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.3 MiB/s wr, 100 op/s 2024-05-20T20:58:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 -- 172.21.15.26:0/4229713979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 msgr2=0x5567bf8eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 -- 172.21.15.26:0/4229713979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8eb630 msgr2=0x5567bf8edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8eb630 0x5567bf8edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 -- 172.21.15.26:0/4229713979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567bf557210 con 0x5567bf8e5ad0 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12ce76640 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e5ad0 0x5567bf8e5ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa120006010 tx=0x7fa120032ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27adc265825c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa12d677640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1180b2920 0x7fa1180b4de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa12003c000 tx=0x7fa120048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa112ffd640 1 -- 172.21.15.26:0/4229713979 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120004410 con 0x5567bf8e5ad0 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa112ffd640 1 -- 172.21.15.26:0/4229713979 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa120036040 con 0x5567bf8e5ad0 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.452+0000 7fa112ffd640 1 -- 172.21.15.26:0/4229713979 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120030e10 con 0x5567bf8e5ad0 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 -- 172.21.15.26:0/4229713979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e5ad0 msgr2=0x5567bf8e5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e5ad0 0x5567bf8e5ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa120006010 tx=0x7fa120032ba0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 -- 172.21.15.26:0/4229713979 shutdown_connections 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8eb630 0x5567bf8edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 --2- 172.21.15.26:0/4229713979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e5ad0 0x5567bf8e5ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 -- 172.21.15.26:0/4229713979 >> 172.21.15.26:0/4229713979 conn(0x5567bf8dcd30 msgr2=0x5567bf8dd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 -- 172.21.15.26:0/4229713979 shutdown_connections 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 -- 172.21.15.26:0/4229713979 wait complete. 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.453+0000 7fa131dfb740 1 -- start start 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 Processor -- start 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf8efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e64c0 0x5567bf8effb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8eb630 0x5567bf8f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567bf528790 con 0x5567bf8e5ad0 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5567bf7fe890 con 0x5567bf8e64c0 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567bf8ee400 con 0x5567bf8eb630 2024-05-20T20:58:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa131dfb740 10 client.0 _ll_register_callbacks cb 0x5567bf556a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf8efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf8efa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59864/0 (socket says 172.21.15.26:59864) 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 -- 172.21.15.26:0/1204848228 learned_addr learned my addr 172.21.15.26:0/1204848228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa127fff640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e64c0 0x5567bf8effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12d677640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8eb630 0x5567bf8f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 -- 172.21.15.26:0/1204848228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8eb630 msgr2=0x5567bf8f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8eb630 0x5567bf8f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 -- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e64c0 msgr2=0x5567bf8effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.454+0000 7fa12ce76640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e64c0 0x5567bf8effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa12ce76640 1 -- 172.21.15.26:0/1204848228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bf623620 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa127fff640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e64c0 0x5567bf8effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa12ce76640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf8efa70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0a9be0 tx=0x7fa11c0a95a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c7445a9195cd96 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa107fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa107fff640 1 -- 172.21.15.26:0/1204848228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0b23f0 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa107fff640 1 -- 172.21.15.26:0/1204848228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa11c0c42d0 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa120036040 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa107fff640 1 -- 172.21.15.26:0/1204848228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa11c0a9760 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa131dfb740 10 client.15978 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567bf8fa090 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa107fff640 1 -- 172.21.15.26:0/1204848228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa11c069e50 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.455+0000 7fa107fff640 1 -- 172.21.15.26:0/1204848228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa11c0a8eb0 con 0x5567bf8e5ad0 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa131dfb740 10 client.15978 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa131dfb740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa131dfb740 20 client.15978 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa131dfb740 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf8fb7d0 0x5567bf8fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa131dfb740 4 client.15978 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567bf854b80 con 0x5567bf8fb7d0 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa127fff640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf8fb7d0 0x5567bf8fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa107fff640 10 client.15978 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.456+0000 7fa127fff640 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf8fb7d0 0x5567bf8fdbc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4d288f9e128c15 server_cookie=bc058f48b1f09428 in_seq=0 out_seq=0 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa107fff640 1 -- 172.21.15.26:0/1204848228 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5567bf854b80 con 0x5567bf8fb7d0 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa107fff640 10 client.15978 handle_command_reply: tid=0 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 client.15978 shutdown 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 msgr2=0x5567bf8efa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf8efa70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0a9be0 tx=0x7fa11c0a95a0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 shutdown_connections 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf8fb7d0 0x5567bf8fdbc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8eb630 0x5567bf8f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8e64c0 0x5567bf8effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa107fff640 0 client.15978 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 --2- 172.21.15.26:0/1204848228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf8efa70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.457+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 >> 172.21.15.26:0/1204848228 conn(0x5567bf8dcd30 msgr2=0x5567bf8dbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.458+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 shutdown_connections 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.458+0000 7fa131dfb740 1 -- 172.21.15.26:0/1204848228 wait complete. 2024-05-20T20:58:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.458+0000 7fa131dfb740 20 client.15978 trim_cache size 0 max 16384 2024-05-20T20:58:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 Processor -- start 2024-05-20T20:58:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 -- start start 2024-05-20T20:58:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf903490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf9039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281aa4d0 con 0x5567bf8e5ad0 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a7da0 con 0x5567bf8eb630 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa131dfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a5bf0 con 0x5567bf8e64c0 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.475+0000 7fa12ce76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf903490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf903490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59876/0 (socket says 172.21.15.26:59876) 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 -- 172.21.15.26:0/2057292842 learned_addr learned my addr 172.21.15.26:0/2057292842 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12d677640 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf9039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa127fff640 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 -- 172.21.15.26:0/2057292842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 msgr2=0x5567bf8f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 -- 172.21.15.26:0/2057292842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 msgr2=0x5567bf9039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf9039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 -- 172.21.15.26:0/2057292842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567bf8dcb60 con 0x5567bf8e5ad0 2024-05-20T20:58:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa12ce76640 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf903490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0a9900 tx=0x7fa11c0a96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fec8a05a50dda8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa1127fc640 1 -- 172.21.15.26:0/2057292842 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0a9a20 con 0x5567bf8e5ad0 2024-05-20T20:58:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa1127fc640 1 -- 172.21.15.26:0/2057292842 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa11c069e50 con 0x5567bf8e5ad0 2024-05-20T20:58:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.476+0000 7fa1127fc640 1 -- 172.21.15.26:0/2057292842 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0a9c50 con 0x5567bf8e5ad0 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 -- 172.21.15.26:0/2057292842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 msgr2=0x5567bf903490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf903490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0a9900 tx=0x7fa11c0a96d0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 -- 172.21.15.26:0/2057292842 shutdown_connections 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf9039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e64c0 0x5567bf8f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 --2- 172.21.15.26:0/2057292842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e5ad0 0x5567bf903490 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 -- 172.21.15.26:0/2057292842 >> 172.21.15.26:0/2057292842 conn(0x5567bf8dcd30 msgr2=0x5567bf8dfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 -- 172.21.15.26:0/2057292842 shutdown_connections 2024-05-20T20:58:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 -- 172.21.15.26:0/2057292842 wait complete. 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 -- start start 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.477+0000 7fa131dfb740 1 Processor -- start 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e5ad0 0x5567bf8f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 0x5567bf8f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf8f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a6260 con 0x5567bf8e64c0 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a76d0 con 0x5567bf8eb630 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a6930 con 0x5567bf8e5ad0 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa131dfb740 10 client.0 _ll_register_callbacks cb 0x7fa11c0038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 0x5567bf8f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 0x5567bf8f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59892/0 (socket says 172.21.15.26:59892) 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 -- 172.21.15.26:0/3782815999 learned_addr learned my addr 172.21.15.26:0/3782815999 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa12d677640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf8f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa12ce76640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e5ad0 0x5567bf8f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 -- 172.21.15.26:0/3782815999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e5ad0 msgr2=0x5567bf8f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e5ad0 0x5567bf8f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 -- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 msgr2=0x5567bf8f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf8f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 -- 172.21.15.26:0/3782815999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bf8ee540 con 0x5567bf8e64c0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa127fff640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 0x5567bf8f4e30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa114005d00 tx=0x7fa114011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7d161cdc7b0666 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.478+0000 7fa107fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa107fff640 1 -- 172.21.15.26:0/3782815999 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114004220 con 0x5567bf8e64c0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa107fff640 1 -- 172.21.15.26:0/3782815999 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa1140043c0 con 0x5567bf8e64c0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c069e50 con 0x5567bf8e64c0 2024-05-20T20:58:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa107fff640 1 -- 172.21.15.26:0/3782815999 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa11401d780 con 0x5567bf8e64c0 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567bf966430 con 0x5567bf8e64c0 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa107fff640 1 -- 172.21.15.26:0/3782815999 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa11c069e50 con 0x5567bf8e64c0 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa107fff640 1 -- 172.21.15.26:0/3782815999 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa114030050 con 0x5567bf8e64c0 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 20 client.15984 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf966de0 0x5567bf969200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 4 client.15984 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567bf84e360 con 0x5567bf966de0 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.479+0000 7fa12ce76640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf966de0 0x5567bf969200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.480+0000 7fa107fff640 10 client.15984 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.480+0000 7fa12ce76640 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf966de0 0x5567bf969200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20168ca8d0a9d67 server_cookie=5d870bb7a409f1b9 in_seq=0 out_seq=0 2024-05-20T20:58:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa107fff640 1 -- 172.21.15.26:0/3782815999 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5567bf84e360 con 0x5567bf966de0 2024-05-20T20:58:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa107fff640 10 client.15984 handle_command_reply: tid=0 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 client.15984 shutdown 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 msgr2=0x5567bf8f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 0x5567bf8f4e30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa114005d00 tx=0x7fa114011940 comp rx=0 tx=0).stop 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 shutdown_connections 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5567bf966de0 0x5567bf969200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567bf8eb630 0x5567bf8f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa107fff640 0 client.15984 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567bf8e64c0 0x5567bf8f4e30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 --2- 172.21.15.26:0/3782815999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567bf8e5ad0 0x5567bf8f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 >> 172.21.15.26:0/3782815999 conn(0x5567bf8dcd30 msgr2=0x5567bf8db720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 shutdown_connections 2024-05-20T20:58:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 1 -- 172.21.15.26:0/3782815999 wait complete. 2024-05-20T20:58:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.511+0000 7fa131dfb740 20 client.15984 trim_cache size 0 max 16384 2024-05-20T20:58:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:58:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:58:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:58:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1180b2920 msgr2=0x7fa1180b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1180b2920 0x7fa1180b4de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa12003c000 tx=0x7fa120048000 comp rx=0 tx=0).stop 2024-05-20T20:58:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 msgr2=0x7fa12806ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 0x7fa12806ac70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e9c0 tx=0x7fa11c00ee80 comp rx=0 tx=0).stop 2024-05-20T20:58:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 shutdown_connections 2024-05-20T20:58:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa1180b2920 0x7fa1180b4de0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa128103560 0x7fa12819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa128102ba0 0x7fa12819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 --2- 172.21.15.26:0/3540382662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1281019b0 0x7fa12806ac70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 >> 172.21.15.26:0/3540382662 conn(0x7fa1280fcf20 msgr2=0x7fa128105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 shutdown_connections 2024-05-20T20:58:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:09.512+0000 7fa12f0da640 1 -- 172.21.15.26:0/3540382662 wait complete. 2024-05-20T20:58:09.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:58:09.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 10380 4.91 MB/sec warmup 15 sec latency 19.260 ms 2024-05-20T20:58:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:10.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:09 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:58:10.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:09 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:10.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:09 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:58:10.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:09 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:09 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:58:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:09 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:09 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:58:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:09 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:10.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:09 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:58:10.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:09 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:10.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:09 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:58:10.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:09 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:10.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 10879 4.88 MB/sec warmup 16 sec latency 15.910 ms 2024-05-20T20:58:11.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:10 smithi026 ceph-mon[31990]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 2.2 MiB/s wr, 113 op/s 2024-05-20T20:58:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:10 smithi144 ceph-mon[35660]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 2.2 MiB/s wr, 113 op/s 2024-05-20T20:58:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:10 smithi055 ceph-mon[35213]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 2.2 MiB/s wr, 113 op/s 2024-05-20T20:58:11.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 11306 4.61 MB/sec warmup 17 sec latency 17.071 ms 2024-05-20T20:58:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 -- 172.21.15.26:0/2153615786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde006bab0 msgr2=0x7fcde006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 --2- 172.21.15.26:0/2153615786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde006bab0 0x7fcde006bf30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009990 tx=0x7fcdd4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 -- 172.21.15.26:0/2153615786 shutdown_connections 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 --2- 172.21.15.26:0/2153615786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006c470 0x7fcde01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 --2- 172.21.15.26:0/2153615786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde01095a0 0x7fcde0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 --2- 172.21.15.26:0/2153615786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde006bab0 0x7fcde006bf30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 -- 172.21.15.26:0/2153615786 >> 172.21.15.26:0/2153615786 conn(0x7fcde00793b0 msgr2=0x7fcde00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 -- 172.21.15.26:0/2153615786 shutdown_connections 2024-05-20T20:58:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.610+0000 7fcde576f700 1 -- 172.21.15.26:0/2153615786 wait complete. 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.611+0000 7fcde576f700 1 Processor -- start 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.611+0000 7fcde576f700 1 -- start start 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.611+0000 7fcde576f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006bab0 0x7fcde0102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcde576f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde006c470 0x7fcde0102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcde576f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde01095a0 0x7fcde0102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcde576f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01135e0 con 0x7fcde01095a0 2024-05-20T20:58:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcde576f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0113460 con 0x7fcde006c470 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcde576f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0113760 con 0x7fcde006bab0 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006bab0 0x7fcde0102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde01095a0 0x7fcde0102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006bab0 0x7fcde0102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46570/0 (socket says 172.21.15.26:46570) 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddeffd700 1 -- 172.21.15.26:0/2906896330 learned_addr learned my addr 172.21.15.26:0/2906896330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde006c470 0x7fcde0102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 -- 172.21.15.26:0/2906896330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006bab0 msgr2=0x7fcde0102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006bab0 0x7fcde0102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 -- 172.21.15.26:0/2906896330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde006c470 msgr2=0x7fcde0102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde006c470 0x7fcde0102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 -- 172.21.15.26:0/2906896330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01a80d0 con 0x7fcde01095a0 2024-05-20T20:58:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.612+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde01095a0 0x7fcde0102f80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8007f10 tx=0x7fcdd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.613+0000 7fcdc7fff700 1 -- 172.21.15.26:0/2906896330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8004db0 con 0x7fcde01095a0 2024-05-20T20:58:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.613+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01a83c0 con 0x7fcde01095a0 2024-05-20T20:58:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.613+0000 7fcdc7fff700 1 -- 172.21.15.26:0/2906896330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd8009e20 con 0x7fcde01095a0 2024-05-20T20:58:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.613+0000 7fcdc7fff700 1 -- 172.21.15.26:0/2906896330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8015400 con 0x7fcde01095a0 2024-05-20T20:58:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.613+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a88d0 con 0x7fcde01095a0 2024-05-20T20:58:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.615+0000 7fcdc7fff700 1 -- 172.21.15.26:0/2906896330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcdd8005240 con 0x7fcde01095a0 2024-05-20T20:58:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.616+0000 7fcdc7fff700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdc80b2880 0x7fcdc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.616+0000 7fcddeffd700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdc80b2880 0x7fcdc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.616+0000 7fcdc7fff700 1 -- 172.21.15.26:0/2906896330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fcdd80e1bd0 con 0x7fcde01095a0 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 Processor -- start 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcddeffd700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdc80b2880 0x7fcdc80b4d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009ca0 tx=0x7fcdd0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 -- start start 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e9971f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9971fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 0x55e9971fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e997158c90 con 0x55e9971fdf60 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e997148830 con 0x55e9971f9860 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e99714c870 con 0x55e9971fa210 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 0x55e9971fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 0x55e9971fe3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58308/0 (socket says 172.21.15.26:58308) 2024-05-20T20:58:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcddf7fe700 1 -- 172.21.15.26:0/3526119922 learned_addr learned my addr 172.21.15.26:0/3526119922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcddeffd700 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e9971f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.617+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9971fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcddf7fe700 1 -- 172.21.15.26:0/3526119922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 msgr2=0x55e9971fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9971fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcddf7fe700 1 -- 172.21.15.26:0/3526119922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 msgr2=0x55e9971f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e9971f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcddf7fe700 1 -- 172.21.15.26:0/3526119922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9970988d0 con 0x55e9971fdf60 2024-05-20T20:58:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 0x55e9971fe3a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80a5210 tx=0x7fcdd80a89d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6148ddbb68887516 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcdc57fa700 1 -- 172.21.15.26:0/3526119922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80ad540 con 0x55e9971fdf60 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcdc57fa700 1 -- 172.21.15.26:0/3526119922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd80b1ff0 con 0x55e9971fdf60 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.618+0000 7fcdc57fa700 1 -- 172.21.15.26:0/3526119922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80b1b60 con 0x55e9971fdf60 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3526119922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 msgr2=0x55e9971fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 0x55e9971fe3a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80a5210 tx=0x7fcdd80a89d0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3526119922 shutdown_connections 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fdf60 0x55e9971fe3a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e9971f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3526119922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9971fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3526119922 >> 172.21.15.26:0/3526119922 conn(0x55e997127080 msgr2=0x55e9971f3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3526119922 shutdown_connections 2024-05-20T20:58:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.619+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3526119922 wait complete. 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 -- start start 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 Processor -- start 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9972026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e997202be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e99704efa0 con 0x55e9971f9860 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e9970ea7c0 con 0x55e9971fdf60 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e997206120 con 0x55e9971fa210 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdf0005b80 10 client.0 _ll_register_callbacks cb 0x55e9970e3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58322/0 (socket says 172.21.15.26:58322) 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e997202be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcddeffd700 1 -- 172.21.15.26:0/3112727920 learned_addr learned my addr 172.21.15.26:0/3112727920 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.620+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9972026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddeffd700 1 -- 172.21.15.26:0/3112727920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 msgr2=0x55e9972026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddeffd700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9972026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddeffd700 1 -- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 msgr2=0x55e997202be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddeffd700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e997202be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddeffd700 1 -- 172.21.15.26:0/3112727920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9970e7690 con 0x55e9971f9860 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e997202be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcddeffd700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972053a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcdd001dc80 tx=0x7fcdd00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b013d75b8cf2d4c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/3112727920 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000e9d0 con 0x55e9971f9860 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/3112727920 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fcdd0009040 con 0x55e9971f9860 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd80b1ff0 con 0x55e9971f9860 2024-05-20T20:58:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/3112727920 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcdd000eb70 con 0x55e9971f9860 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdf0005b80 10 client.15996 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.621+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e997210810 con 0x55e9971f9860 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/3112727920 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd80b1ff0 con 0x55e9971f9860 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/3112727920 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcdd002f020 con 0x55e9971f9860 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdf0005b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdf0005b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdf0005b80 20 client.15996 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e997212df0 0x55e997215220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdf0005b80 4 client.15996 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9971c59e0 con 0x55e997212df0 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e997212df0 0x55e997215220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e997212df0 0x55e997215220 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673c722945e5acfd server_cookie=abfb66cf5c4aa415 in_seq=0 out_seq=0 2024-05-20T20:58:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.622+0000 7fcdc2ff5700 10 client.15996 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.623+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/3112727920 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9971c59e0 con 0x55e997212df0 2024-05-20T20:58:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.623+0000 7fcdc2ff5700 10 client.15996 handle_command_reply: tid=0 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.623+0000 7fcdf0005b80 1 client.15996 shutdown 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 msgr2=0x55e9972053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972053a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcdd001dc80 tx=0x7fcdd00047c0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 shutdown_connections 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e997212df0 0x55e997215220 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972053a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e997202be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdc2ff5700 0 client.15996 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/3112727920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e9972026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 >> 172.21.15.26:0/3112727920 conn(0x55e997127080 msgr2=0x55e9971ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 shutdown_connections 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 1 -- 172.21.15.26:0/3112727920 wait complete. 2024-05-20T20:58:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.624+0000 7fcdf0005b80 20 client.15996 trim_cache size 0 max 16384 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 Processor -- start 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 -- start start 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e99721aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e99721b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b28c0 con 0x55e9971f9860 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b0150 con 0x55e9971fdf60 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01adfa0 con 0x55e9971fa210 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58324/0 (socket says 172.21.15.26:58324) 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 -- 172.21.15.26:0/1305322936 learned_addr learned my addr 172.21.15.26:0/1305322936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e99721b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e99721aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 -- 172.21.15.26:0/1305322936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 msgr2=0x55e99721aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e99721aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 -- 172.21.15.26:0/1305322936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 msgr2=0x55e99721b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e99721b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddeffd700 1 -- 172.21.15.26:0/1305322936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9972158a0 con 0x55e9971f9860 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.647+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e99721b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcddeffd700 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972045e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0004790 tx=0x7fcdd0009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b25a9931ec1f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcdc4ff9700 1 -- 172.21.15.26:0/1305322936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000e9d0 con 0x55e9971f9860 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcdc4ff9700 1 -- 172.21.15.26:0/1305322936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd0036070 con 0x55e9971f9860 2024-05-20T20:58:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcdc4ff9700 1 -- 172.21.15.26:0/1305322936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0031460 con 0x55e9971f9860 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcdf0005b80 1 -- 172.21.15.26:0/1305322936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 msgr2=0x55e9972045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972045e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0004790 tx=0x7fcdd0009350 comp rx=0 tx=0).stop 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.648+0000 7fcdf0005b80 1 -- 172.21.15.26:0/1305322936 shutdown_connections 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971f9860 0x55e9972045e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971fdf60 0x55e99721b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/1305322936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fa210 0x55e99721aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- 172.21.15.26:0/1305322936 >> 172.21.15.26:0/1305322936 conn(0x55e997127080 msgr2=0x55e997203ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- 172.21.15.26:0/1305322936 shutdown_connections 2024-05-20T20:58:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- 172.21.15.26:0/1305322936 wait complete. 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- start start 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 Processor -- start 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e99720b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 0x55e99720b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fdf60 0x55e997208380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ae610 con 0x55e9971fa210 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01afa80 con 0x55e9971f9860 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01aece0 con 0x55e9971fdf60 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdf0005b80 10 client.0 _ll_register_callbacks cb 0x55e9970e3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdde7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 0x55e99720b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcddeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e99720b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdde7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 0x55e99720b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58338/0 (socket says 172.21.15.26:58338) 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.649+0000 7fcdde7fc700 1 -- 172.21.15.26:0/2343365953 learned_addr learned my addr 172.21.15.26:0/2343365953 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcddf7fe700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fdf60 0x55e997208380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdde7fc700 1 -- 172.21.15.26:0/2343365953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fdf60 msgr2=0x55e997208380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fdf60 0x55e997208380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdde7fc700 1 -- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 msgr2=0x55e99720b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e99720b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdde7fc700 1 -- 172.21.15.26:0/2343365953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9972089c0 con 0x55e9971fa210 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcddeffd700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e99720b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdde7fc700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 0x55e99720b8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40098e0 tx=0x7fcdd4005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60331d7476da8a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/2343365953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4003a90 con 0x55e9971fa210 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/2343365953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fcdd4003c30 con 0x55e9971fa210 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd0036070 con 0x55e9971fa210 2024-05-20T20:58:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.650+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/2343365953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcdd4003e30 con 0x55e9971fa210 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 10 client.16002 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e99727e370 con 0x55e9971fa210 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/2343365953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd0036070 con 0x55e9971fa210 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/2343365953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcdd4037020 con 0x55e9971fa210 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 20 client.16002 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e99727eab0 0x55e997280ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 4 client.16002 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e99715abe0 con 0x55e99727eab0 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.651+0000 7fcddeffd700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e99727eab0 0x55e997280ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdc2ff5700 10 client.16002 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcddeffd700 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e99727eab0 0x55e997280ed0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c805f4eee0d32a server_cookie=84541a0d49a68209 in_seq=0 out_seq=0 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdc2ff5700 1 -- 172.21.15.26:0/2343365953 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e99715abe0 con 0x55e99727eab0 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdc2ff5700 10 client.16002 handle_command_reply: tid=0 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 client.16002 shutdown 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 msgr2=0x55e99720b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 0x55e99720b8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40098e0 tx=0x7fcdd4005450 comp rx=0 tx=0).stop 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 shutdown_connections 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e99727eab0 0x55e997280ed0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9971fdf60 0x55e997208380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9971f9860 0x55e99720b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 --2- 172.21.15.26:0/2343365953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9971fa210 0x55e99720b8f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 >> 172.21.15.26:0/2343365953 conn(0x55e997127080 msgr2=0x55e997203ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdc2ff5700 0 client.16002 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 shutdown_connections 2024-05-20T20:58:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 1 -- 172.21.15.26:0/2343365953 wait complete. 2024-05-20T20:58:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.652+0000 7fcdf0005b80 20 client.16002 trim_cache size 0 max 16384 2024-05-20T20:58:11.658 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:58:11.658 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:58:11.658 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:58:11.658 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdc80b2880 msgr2=0x7fcdc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdc80b2880 0x7fcdc80b4d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009ca0 tx=0x7fcdd0009260 comp rx=0 tx=0).stop 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde01095a0 msgr2=0x7fcde0102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde01095a0 0x7fcde0102f80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8007f10 tx=0x7fcdd800d3b0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 shutdown_connections 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdc80b2880 0x7fcdc80b4d40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde01095a0 0x7fcde0102f80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde006bab0 0x7fcde0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 --2- 172.21.15.26:0/2906896330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde006c470 0x7fcde0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 >> 172.21.15.26:0/2906896330 conn(0x7fcde00793b0 msgr2=0x7fcde0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 shutdown_connections 2024-05-20T20:58:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:11.653+0000 7fcde576f700 1 -- 172.21.15.26:0/2906896330 wait complete. 2024-05-20T20:58:12.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:11 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:58:12.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:11 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:58:12.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:11 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:58:12.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 12040 4.63 MB/sec warmup 18 sec latency 11.477 ms 2024-05-20T20:58:12.583 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:58:12.583 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:58:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:12 smithi144 ceph-mon[35660]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-20T20:58:13.314 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:12 smithi026 ceph-mon[31990]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-20T20:58:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:12 smithi055 ceph-mon[35213]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-20T20:58:13.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 12734 4.42 MB/sec warmup 19 sec latency 17.383 ms 2024-05-20T20:58:14.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 13434 4.47 MB/sec warmup 20 sec latency 17.010 ms 2024-05-20T20:58:14.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 -- 172.21.15.26:0/1231521558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102de0 msgr2=0x7f0d6010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 --2- 172.21.15.26:0/1231521558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102de0 0x7f0d6010f2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b0a0 tx=0x7f0d5800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 -- 172.21.15.26:0/1231521558 shutdown_connections 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 --2- 172.21.15.26:0/1231521558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102de0 0x7f0d6010f2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 --2- 172.21.15.26:0/1231521558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d601084b0 0x7f0d60108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 --2- 172.21.15.26:0/1231521558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d60102420 0x7f0d601028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 -- 172.21.15.26:0/1231521558 >> 172.21.15.26:0/1231521558 conn(0x7f0d60079440 msgr2=0x7f0d60079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 -- 172.21.15.26:0/1231521558 shutdown_connections 2024-05-20T20:58:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.779+0000 7f0d669f1700 1 -- 172.21.15.26:0/1231521558 wait complete. 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 Processor -- start 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 -- start start 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 0x7f0d6019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d60102de0 0x7f0d6019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d601084b0 0x7f0d601a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601124a0 con 0x7f0d60102420 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112320 con 0x7f0d60102de0 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d669f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112620 con 0x7f0d601084b0 2024-05-20T20:58:14.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 0x7f0d6019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.780+0000 7f0d5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 0x7f0d6019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58360/0 (socket says 172.21.15.26:58360) 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 -- 172.21.15.26:0/654188113 learned_addr learned my addr 172.21.15.26:0/654188113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d64f8e700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d601084b0 0x7f0d601a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d57fff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d60102de0 0x7f0d6019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 -- 172.21.15.26:0/654188113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d601084b0 msgr2=0x7f0d601a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d601084b0 0x7f0d601a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 -- 172.21.15.26:0/654188113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d60102de0 msgr2=0x7f0d6019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d60102de0 0x7f0d6019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 -- 172.21.15.26:0/654188113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d601a5ee0 con 0x7f0d60102420 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d57fff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d60102de0 0x7f0d6019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5ffff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 0x7f0d6019a920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000e9a0 tx=0x7f0d5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000cbd0 con 0x7f0d60102420 2024-05-20T20:58:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d601a6110 con 0x7f0d60102420 2024-05-20T20:58:14.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d5000cd70 con 0x7f0d60102420 2024-05-20T20:58:14.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a6670 con 0x7f0d60102420 2024-05-20T20:58:14.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.781+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50018680 con 0x7f0d60102420 2024-05-20T20:58:14.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.782+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6006a230 con 0x7f0d60102420 2024-05-20T20:58:14.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.784+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0d50010950 con 0x7f0d60102420 2024-05-20T20:58:14.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.784+0000 7f0d5dffb700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d4c0b2900 0x7f0d4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:14.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.785+0000 7f0d57fff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d4c0b2900 0x7f0d4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:14.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.785+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0d500e26c0 con 0x7f0d60102420 2024-05-20T20:58:14.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.785+0000 7f0d57fff700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d4c0b2900 0x7f0d4c0b4dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d60079a20 tx=0x7f0d48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:14.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:14.789+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d500aa380 con 0x7f0d60102420 2024-05-20T20:58:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.103+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d6002e560 con 0x7f0d60102420 2024-05-20T20:58:15.103 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:14 smithi026 ceph-mon[31990]: pgmap v451: 97 pgs: 97 active+clean; 27 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T20:58:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.103+0000 7f0d5dffb700 1 -- 172.21.15.26:0/654188113 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f0d500a9c80 con 0x7f0d60102420 2024-05-20T20:58:15.104 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:58:15.104 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:58:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.106+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d4c0b2900 msgr2=0x7f0d4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.106+0000 7f0d669f1700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d4c0b2900 0x7f0d4c0b4dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d60079a20 tx=0x7f0d48009260 comp rx=0 tx=0).stop 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 msgr2=0x7f0d6019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 0x7f0d6019a920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000e9a0 tx=0x7f0d5000ed60 comp rx=0 tx=0).stop 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 shutdown_connections 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0d4c0b2900 0x7f0d4c0b4dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d60102420 0x7f0d6019a920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d601084b0 0x7f0d601a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 --2- 172.21.15.26:0/654188113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d60102de0 0x7f0d6019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 >> 172.21.15.26:0/654188113 conn(0x7f0d60079440 msgr2=0x7f0d60100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 shutdown_connections 2024-05-20T20:58:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:15.107+0000 7f0d669f1700 1 -- 172.21.15.26:0/654188113 wait complete. 2024-05-20T20:58:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:58:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:14 smithi144 ceph-mon[35660]: pgmap v451: 97 pgs: 97 active+clean; 27 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T20:58:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:14 smithi055 ceph-mon[35213]: pgmap v451: 97 pgs: 97 active+clean; 27 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T20:58:15.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 14042 4.42 MB/sec warmup 21 sec latency 19.293 ms 2024-05-20T20:58:15.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:58:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:16.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:15 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/654188113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:58:16.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:15 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/654188113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:58:16.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:15 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/654188113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:58:16.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 14638 4.43 MB/sec warmup 22 sec latency 16.152 ms 2024-05-20T20:58:17.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:16 smithi144 ceph-mon[35660]: pgmap v452: 97 pgs: 97 active+clean; 28 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-05-20T20:58:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:16 smithi055 ceph-mon[35213]: pgmap v452: 97 pgs: 97 active+clean; 28 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-05-20T20:58:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:16 smithi026 ceph-mon[31990]: pgmap v452: 97 pgs: 97 active+clean; 28 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-05-20T20:58:17.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 15242 4.32 MB/sec warmup 23 sec latency 25.406 ms 2024-05-20T20:58:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 -- 172.21.15.26:0/2842571104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 msgr2=0x7f2814103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 --2- 172.21.15.26:0/2842571104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 0x7f2814103a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f28040099c0 tx=0x7f2804009cd0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 -- 172.21.15.26:0/2842571104 shutdown_connections 2024-05-20T20:58:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 --2- 172.21.15.26:0/2842571104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814103fa0 0x7f2814110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 --2- 172.21.15.26:0/2842571104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814109680 0x7f2814109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 --2- 172.21.15.26:0/2842571104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 0x7f2814103a60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 -- 172.21.15.26:0/2842571104 >> 172.21.15.26:0/2842571104 conn(0x7f281407a6f0 msgr2=0x7f281407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.512+0000 7f281ba27700 1 -- 172.21.15.26:0/2842571104 shutdown_connections 2024-05-20T20:58:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.513+0000 7f281ba27700 1 -- 172.21.15.26:0/2842571104 wait complete. 2024-05-20T20:58:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 Processor -- start 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 -- start start 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 0x7f281419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814103fa0 0x7f281419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 0x7f2814199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2814113690 con 0x7f28141035e0 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2814113510 con 0x7f2814109680 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f281ba27700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2814113810 con 0x7f2814103fa0 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f2819fc4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 0x7f2814199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f28197c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 0x7f281419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f2819fc4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 0x7f2814199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53688/0 (socket says 172.21.15.26:53688) 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f2819fc4700 1 -- 172.21.15.26:0/3507689787 learned_addr learned my addr 172.21.15.26:0/3507689787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.514+0000 7f2818fc2700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814103fa0 0x7f281419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2819fc4700 1 -- 172.21.15.26:0/3507689787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814103fa0 msgr2=0x7f281419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2819fc4700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814103fa0 0x7f281419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2819fc4700 1 -- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 msgr2=0x7f281419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2819fc4700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 0x7f281419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2819fc4700 1 -- 172.21.15.26:0/3507689787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281419a070 con 0x7f2814109680 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2818fc2700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814103fa0 0x7f281419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f2819fc4700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 0x7f2814199910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f280c00ef30 tx=0x7f280c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f280a7fc700 1 -- 172.21.15.26:0/3507689787 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0092e0 con 0x7f2814109680 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f280a7fc700 1 -- 172.21.15.26:0/3507689787 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f280c004bb0 con 0x7f2814109680 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281419a300 con 0x7f2814109680 2024-05-20T20:58:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f280a7fc700 1 -- 172.21.15.26:0/3507689787 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c004020 con 0x7f2814109680 2024-05-20T20:58:18.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.515+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2814075770 con 0x7f2814109680 2024-05-20T20:58:18.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.518+0000 7f280a7fc700 1 -- 172.21.15.26:0/3507689787 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f280c01b030 con 0x7f2814109680 2024-05-20T20:58:18.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.518+0000 7f280a7fc700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28000b2880 0x7f28000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.519+0000 7f28197c3700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28000b2880 0x7f28000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.519+0000 7f280a7fc700 1 -- 172.21.15.26:0/3507689787 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f280c0e1e00 con 0x7f2814109680 2024-05-20T20:58:18.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.519+0000 7f28197c3700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28000b2880 0x7f28000b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f281419aa50 tx=0x7f281000f820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:18.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.519+0000 7f28262bdb80 1 Processor -- start 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.519+0000 7f28262bdb80 1 -- start start 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737bd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28262bdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559b73718c90 con 0x559b737ba210 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28262bdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559b73708830 con 0x559b737b9860 2024-05-20T20:58:18.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28262bdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559b7370c870 con 0x559b737bdf60 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737bd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f28197c3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737bd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58386/0 (socket says 172.21.15.26:58386) 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 -- 172.21.15.26:0/2036458407 learned_addr learned my addr 172.21.15.26:0/2036458407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 -- 172.21.15.26:0/2036458407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 msgr2=0x559b737be3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 -- 172.21.15.26:0/2036458407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 msgr2=0x559b737b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737b9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 -- 172.21.15.26:0/2036458407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b736588d0 con 0x559b737ba210 2024-05-20T20:58:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.520+0000 7f2818fc2700 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737bd940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f28040098e0 tx=0x7f2804003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ba3bc3b46afb02 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:18.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f27ff7fe700 1 -- 172.21.15.26:0/2036458407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804027070 con 0x559b737ba210 2024-05-20T20:58:18.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f27ff7fe700 1 -- 172.21.15.26:0/2036458407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2804003e50 con 0x559b737ba210 2024-05-20T20:58:18.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f27ff7fe700 1 -- 172.21.15.26:0/2036458407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400b9f0 con 0x559b737ba210 2024-05-20T20:58:18.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f28262bdb80 1 -- 172.21.15.26:0/2036458407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 msgr2=0x559b737bd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737bd940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f28040098e0 tx=0x7f2804003680 comp rx=0 tx=0).stop 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f28262bdb80 1 -- 172.21.15.26:0/2036458407 shutdown_connections 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737be3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.521+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2036458407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737bd940 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- 172.21.15.26:0/2036458407 >> 172.21.15.26:0/2036458407 conn(0x559b736e7080 msgr2=0x559b737b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- 172.21.15.26:0/2036458407 shutdown_connections 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- 172.21.15.26:0/2036458407 wait complete. 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- start start 2024-05-20T20:58:18.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 Processor -- start 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559b7360efa0 con 0x559b737ba210 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559b736aa7c0 con 0x559b737b9860 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.522+0000 7f28262bdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559b737c60c0 con 0x559b737bdf60 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f28262bdb80 10 client.0 _ll_register_callbacks cb 0x559b736a3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58392/0 (socket says 172.21.15.26:58392) 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 -- 172.21.15.26:0/2629351442 learned_addr learned my addr 172.21.15.26:0/2629351442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f28197c3700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 -- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 msgr2=0x559b737c2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2819fc4700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 -- 172.21.15.26:0/2629351442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 msgr2=0x559b737c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 -- 172.21.15.26:0/2629351442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b736a7690 con 0x559b737ba210 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2819fc4700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f28197c3700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f2818fc2700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737c2750 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2804009d10 tx=0x7f2804005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4457b998a8ac30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f27fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f27fcff9700 1 -- 172.21.15.26:0/2629351442 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804032060 con 0x559b737ba210 2024-05-20T20:58:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2804003e50 con 0x559b737ba210 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.523+0000 7f27fcff9700 1 -- 172.21.15.26:0/2629351442 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f280400b790 con 0x559b737ba210 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f27fcff9700 1 -- 172.21.15.26:0/2629351442 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2804027070 con 0x559b737ba210 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 10 client.16020 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b737d08a0 con 0x559b737ba210 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f27fcff9700 1 -- 172.21.15.26:0/2629351442 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2804003e50 con 0x559b737ba210 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f27fcff9700 1 -- 172.21.15.26:0/2629351442 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2804021070 con 0x559b737ba210 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 10 client.16020 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 20 client.16020 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b737d2d70 0x559b737d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 4 client.16020 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.524+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b737859e0 con 0x559b737d2d70 2024-05-20T20:58:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.525+0000 7f28197c3700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b737d2d70 0x559b737d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.525+0000 7f28197c3700 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b737d2d70 0x559b737d5160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11002e3901b4da91 server_cookie=15ec99a6e905376a in_seq=0 out_seq=0 2024-05-20T20:58:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.525+0000 7f27fcff9700 10 client.16020 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f27fcff9700 1 -- 172.21.15.26:0/2629351442 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559b737859e0 con 0x559b737d2d70 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f27fcff9700 10 client.16020 handle_command_reply: tid=0 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 client.16020 shutdown 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 msgr2=0x559b737c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737c2750 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2804009d10 tx=0x7f2804005810 comp rx=0 tx=0).stop 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 shutdown_connections 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b737d2d70 0x559b737d5160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 --2- 172.21.15.26:0/2629351442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737ba210 0x559b737c2750 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 >> 172.21.15.26:0/2629351442 conn(0x559b736e7080 msgr2=0x559b737bf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f27fcff9700 0 client.16020 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 shutdown_connections 2024-05-20T20:58:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 1 -- 172.21.15.26:0/2629351442 wait complete. 2024-05-20T20:58:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.526+0000 7f28262bdb80 20 client.16020 trim_cache size 0 max 16384 2024-05-20T20:58:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.543+0000 7f28262bdb80 1 Processor -- start 2024-05-20T20:58:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.543+0000 7f28262bdb80 1 -- start start 2024-05-20T20:58:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 0x559b737b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28262bdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28141ae230 con 0x559b737c1750 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28262bdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f28141abac0 con 0x559b737b9860 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28262bdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a9930 con 0x559b737bdf60 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f2819fc4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 0x559b737b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53710/0 (socket says 172.21.15.26:53710) 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f2819fc4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 0x559b737b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58408/0 (socket says 172.21.15.26:58408) 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 -- 172.21.15.26:0/3165313452 learned_addr learned my addr 172.21.15.26:0/3165313452 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f2818fc2700 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 -- 172.21.15.26:0/3165313452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 msgr2=0x559b737c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 -- 172.21.15.26:0/3165313452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 msgr2=0x559b737b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 0x559b737b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 -- 172.21.15.26:0/3165313452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b737c3190 con 0x559b737b9860 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f2819fc4700 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 0x559b737b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.544+0000 7f28197c3700 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c2710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2810005d90 tx=0x7f2810017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29307131b222cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f27feffd700 1 -- 172.21.15.26:0/3165313452 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810009e30 con 0x559b737b9860 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f27feffd700 1 -- 172.21.15.26:0/3165313452 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f281001bd60 con 0x559b737b9860 2024-05-20T20:58:18.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f27feffd700 1 -- 172.21.15.26:0/3165313452 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810029500 con 0x559b737b9860 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 -- 172.21.15.26:0/3165313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 msgr2=0x559b737c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c2710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2810005d90 tx=0x7f2810017000 comp rx=0 tx=0).stop 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 -- 172.21.15.26:0/3165313452 shutdown_connections 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737c1750 0x559b737b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737b9860 0x559b737c2710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 --2- 172.21.15.26:0/3165313452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737bdf60 0x559b737c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 -- 172.21.15.26:0/3165313452 >> 172.21.15.26:0/3165313452 conn(0x559b736e7080 msgr2=0x559b737c1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 -- 172.21.15.26:0/3165313452 shutdown_connections 2024-05-20T20:58:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.545+0000 7f28262bdb80 1 -- 172.21.15.26:0/3165313452 wait complete. 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.546+0000 7f28262bdb80 1 -- start start 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.546+0000 7f28262bdb80 1 Processor -- start 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.546+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 0x559b73835f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737bdf60 0x559b73836470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28262bdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737c1750 0x559b737d7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28262bdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a9f80 con 0x559b737b9860 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28262bdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f28141ab3f0 con 0x559b737bdf60 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28262bdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28141aa650 con 0x559b737c1750 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28262bdb80 10 client.0 _ll_register_callbacks cb 0x559b736a3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 0x559b73835f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 0x559b73835f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58422/0 (socket says 172.21.15.26:58422) 2024-05-20T20:58:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 -- 172.21.15.26:0/934314945 learned_addr learned my addr 172.21.15.26:0/934314945 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f2818fc2700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737bdf60 0x559b73836470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f2819fc4700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737c1750 0x559b737d7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 -- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737c1750 msgr2=0x559b737d7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737c1750 0x559b737d7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 -- 172.21.15.26:0/934314945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737bdf60 msgr2=0x559b73836470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737bdf60 0x559b73836470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 -- 172.21.15.26:0/934314945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b737d8490 con 0x559b737b9860 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f2818fc2700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737bdf60 0x559b73836470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f28197c3700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 0x559b73835f30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2810005810 tx=0x7f281001b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7b2c4dad482f72 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.547+0000 7f27fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f27fcff9700 1 -- 172.21.15.26:0/934314945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281000f070 con 0x559b737b9860 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f27fcff9700 1 -- 172.21.15.26:0/934314945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f2810036050 con 0x559b737b9860 2024-05-20T20:58:18.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281001bd60 con 0x559b737b9860 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f27fcff9700 1 -- 172.21.15.26:0/934314945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f281000f210 con 0x559b737b9860 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 10 client.16026 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b73845f70 con 0x559b737b9860 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f27fcff9700 1 -- 172.21.15.26:0/934314945 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f281001bd60 con 0x559b737b9860 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f27fcff9700 1 -- 172.21.15.26:0/934314945 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2810029710 con 0x559b737b9860 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 10 client.16026 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 20 client.16026 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b73846640 0x559b73848a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 4 client.16026 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.548+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b7371abe0 con 0x559b73846640 2024-05-20T20:58:18.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.549+0000 7f2818fc2700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b73846640 0x559b73848a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:18.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 15887 4.28 MB/sec warmup 24 sec latency 39.052 ms 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.549+0000 7f27fcff9700 10 client.16026 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.549+0000 7f2818fc2700 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b73846640 0x559b73848a60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cf03d354cc577e5 server_cookie=c418321190a0c731 in_seq=0 out_seq=0 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.549+0000 7f27fcff9700 1 -- 172.21.15.26:0/934314945 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b7371abe0 con 0x559b73846640 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.549+0000 7f27fcff9700 10 client.16026 handle_command_reply: tid=0 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.549+0000 7f28262bdb80 1 client.16026 shutdown 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 msgr2=0x559b73835f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 0x559b73835f30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2810005810 tx=0x7f281001b7a0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 shutdown_connections 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559b73846640 0x559b73848a60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f27fcff9700 0 client.16026 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559b737b9860 0x559b73835f30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b737c1750 0x559b737d7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 --2- 172.21.15.26:0/934314945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b737bdf60 0x559b73836470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 >> 172.21.15.26:0/934314945 conn(0x559b736e7080 msgr2=0x559b737c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 shutdown_connections 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 1 -- 172.21.15.26:0/934314945 wait complete. 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.550+0000 7f28262bdb80 20 client.16026 trim_cache size 0 max 16384 2024-05-20T20:58:18.555 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28000b2880 msgr2=0x7f28000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28000b2880 0x7f28000b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f281419aa50 tx=0x7f281000f820 comp rx=0 tx=0).stop 2024-05-20T20:58:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 msgr2=0x7f2814199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 0x7f2814199910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f280c00ef30 tx=0x7f280c00c3a0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 shutdown_connections 2024-05-20T20:58:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28000b2880 0x7f28000b4d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28141035e0 0x7f281419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2814109680 0x7f2814199910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 --2- 172.21.15.26:0/3507689787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2814103fa0 0x7f281419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 >> 172.21.15.26:0/3507689787 conn(0x7f281407a6f0 msgr2=0x7f2814102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 shutdown_connections 2024-05-20T20:58:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:18.551+0000 7f281ba27700 1 -- 172.21.15.26:0/3507689787 wait complete. 2024-05-20T20:58:19.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:18 smithi026 ceph-mon[31990]: pgmap v453: 97 pgs: 97 active+clean; 29 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-05-20T20:58:19.253 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:18 smithi055 ceph-mon[35213]: pgmap v453: 97 pgs: 97 active+clean; 29 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-05-20T20:58:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:18 smithi144 ceph-mon[35660]: pgmap v453: 97 pgs: 97 active+clean; 29 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-05-20T20:58:19.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:58:19.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16567 4.18 MB/sec warmup 25 sec latency 17.492 ms 2024-05-20T20:58:19.678 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:20.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 17467 4.30 MB/sec warmup 26 sec latency 17.917 ms 2024-05-20T20:58:21.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:20 smithi026 ceph-mon[31990]: pgmap v454: 97 pgs: 97 active+clean; 29 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T20:58:21.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:20 smithi144 ceph-mon[35660]: pgmap v454: 97 pgs: 97 active+clean; 29 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T20:58:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:20 smithi055 ceph-mon[35213]: pgmap v454: 97 pgs: 97 active+clean; 29 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T20:58:21.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 -- 172.21.15.26:0/3807180654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1037f0 msgr2=0x7f26cc103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 --2- 172.21.15.26:0/3807180654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1037f0 0x7f26cc103bd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099c0 tx=0x7f26c0009cd0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 -- 172.21.15.26:0/3807180654 shutdown_connections 2024-05-20T20:58:21.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 --2- 172.21.15.26:0/3807180654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1037f0 0x7f26cc103bd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 --2- 172.21.15.26:0/3807180654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1088f0 0x7f26cc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 --2- 172.21.15.26:0/3807180654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1041a0 0x7f26cc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 -- 172.21.15.26:0/3807180654 >> 172.21.15.26:0/3807180654 conn(0x7f26cc07a6a0 msgr2=0x7f26cc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 -- 172.21.15.26:0/3807180654 shutdown_connections 2024-05-20T20:58:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.488+0000 7f26d2d12700 1 -- 172.21.15.26:0/3807180654 wait complete. 2024-05-20T20:58:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.489+0000 7f26d2d12700 1 Processor -- start 2024-05-20T20:58:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.489+0000 7f26d2d12700 1 -- start start 2024-05-20T20:58:21.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.489+0000 7f26d2d12700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 0x7f26cc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.489+0000 7f26d2d12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1041a0 0x7f26cc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.489+0000 7f26d2d12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1088f0 0x7f26cc199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d2d12700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1135e0 con 0x7f26cc1088f0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d2d12700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc113460 con 0x7f26cc1037f0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d2d12700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc113760 con 0x7f26cc1041a0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 0x7f26cc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d12af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1088f0 0x7f26cc199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 0x7f26cc19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59276/0 (socket says 172.21.15.26:59276) 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 -- 172.21.15.26:0/829546070 learned_addr learned my addr 172.21.15.26:0/829546070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26cbfff700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1041a0 0x7f26cc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 -- 172.21.15.26:0/829546070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1041a0 msgr2=0x7f26cc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1041a0 0x7f26cc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 -- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1088f0 msgr2=0x7f26cc199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1088f0 0x7f26cc199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 -- 172.21.15.26:0/829546070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc19a0e0 con 0x7f26cc1037f0 2024-05-20T20:58:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26cbfff700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1041a0 0x7f26cc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d12af700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1088f0 0x7f26cc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26d0aae700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 0x7f26cc19f7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005910 tx=0x7f26c0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.490+0000 7f26c9ffb700 1 -- 172.21.15.26:0/829546070 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0026070 con 0x7f26cc1037f0 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.491+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc19a370 con 0x7f26cc1037f0 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.491+0000 7f26c9ffb700 1 -- 172.21.15.26:0/829546070 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26c0003c50 con 0x7f26cc1037f0 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.491+0000 7f26c9ffb700 1 -- 172.21.15.26:0/829546070 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c000faa0 con 0x7f26cc1037f0 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.491+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc1a6690 con 0x7f26cc1037f0 2024-05-20T20:58:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.493+0000 7f26c9ffb700 1 -- 172.21.15.26:0/829546070 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f26c0029070 con 0x7f26cc1037f0 2024-05-20T20:58:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.494+0000 7f26c9ffb700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26bc0b2880 0x7f26bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.494+0000 7f26cbfff700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26bc0b2880 0x7f26bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.494+0000 7f26c9ffb700 1 -- 172.21.15.26:0/829546070 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f26c00e2ab0 con 0x7f26cc1037f0 2024-05-20T20:58:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 Processor -- start 2024-05-20T20:58:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26cbfff700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26bc0b2880 0x7f26bc0b4d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f26cc1031e0 tx=0x7f26b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 -- start start 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2046700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c20408a0 0x55f7c2040d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 0x55f7c204d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1f9ed70 con 0x55f7c2041260 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1f8e910 con 0x55f7c20408a0 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1f92950 con 0x55f7c2046320 2024-05-20T20:58:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26d12af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 0x55f7c204d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26d12af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 0x55f7c204d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39842/0 (socket says 172.21.15.26:39842) 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26d12af700 1 -- 172.21.15.26:0/2206728967 learned_addr learned my addr 172.21.15.26:0/2206728967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26d0aae700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2046700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.495+0000 7f26cbfff700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c20408a0 0x55f7c2040d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d12af700 1 -- 172.21.15.26:0/2206728967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 msgr2=0x55f7c2046700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d12af700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2046700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d12af700 1 -- 172.21.15.26:0/2206728967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c20408a0 msgr2=0x55f7c2040d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d12af700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c20408a0 0x55f7c2040d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d12af700 1 -- 172.21.15.26:0/2206728967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c1f37fb0 con 0x55f7c2041260 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26cbfff700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c20408a0 0x55f7c2040d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d0aae700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2046700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26d12af700 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 0x55f7c204d080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f26c4003eb0 tx=0x7f26c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f227e2d6e89f61ad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26b6ffd700 1 -- 172.21.15.26:0/2206728967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4019020 con 0x55f7c2041260 2024-05-20T20:58:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26b6ffd700 1 -- 172.21.15.26:0/2206728967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26c40092d0 con 0x55f7c2041260 2024-05-20T20:58:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.496+0000 7f26b6ffd700 1 -- 172.21.15.26:0/2206728967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c400e6f0 con 0x55f7c2041260 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/2206728967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 msgr2=0x55f7c204d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 0x55f7c204d080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f26c4003eb0 tx=0x7f26c4009920 comp rx=0 tx=0).stop 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/2206728967 shutdown_connections 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2041260 0x55f7c204d080 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2046700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/2206728967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c20408a0 0x55f7c2040d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/2206728967 >> 172.21.15.26:0/2206728967 conn(0x55f7c1f6c240 msgr2=0x55f7c203b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/2206728967 shutdown_connections 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/2206728967 wait complete. 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 -- start start 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.497+0000 7f26dd5a8b80 1 Processor -- start 2024-05-20T20:58:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c204ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2049ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c204a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1e94fa0 con 0x55f7c2046320 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1ef6da0 con 0x55f7c2041260 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7c204d6e0 con 0x55f7c20408a0 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26dd5a8b80 10 client.0 _ll_register_callbacks cb 0x55f7c1f29950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26d12af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c204a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2049ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2049ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59296/0 (socket says 172.21.15.26:59296) 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26d12af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c204a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39844/0 (socket says 172.21.15.26:39844) 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 -- 172.21.15.26:0/1804598980 learned_addr learned my addr 172.21.15.26:0/1804598980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26d0aae700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c204ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 -- 172.21.15.26:0/1804598980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 msgr2=0x55f7c204ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c204ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 -- 172.21.15.26:0/1804598980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 msgr2=0x55f7c204a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c204a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26cbfff700 1 -- 172.21.15.26:0/1804598980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c1ede8d0 con 0x55f7c2041260 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.498+0000 7f26d12af700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c204a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26cbfff700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2049ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26b8003930 tx=0x7f26b80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9317d453b6c4647 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26abfff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26abfff700 1 -- 172.21.15.26:0/1804598980 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8012860 con 0x55f7c2041260 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26abfff700 1 -- 172.21.15.26:0/1804598980 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f26b8012a00 con 0x55f7c2041260 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c40092d0 con 0x55f7c2041260 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26abfff700 1 -- 172.21.15.26:0/1804598980 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26b8011e90 con 0x55f7c2041260 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26dd5a8b80 10 client.25592 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7c1f48760 con 0x55f7c2041260 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.499+0000 7f26abfff700 1 -- 172.21.15.26:0/1804598980 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26c40092d0 con 0x55f7c2041260 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26abfff700 1 -- 172.21.15.26:0/1804598980 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26b8032a30 con 0x55f7c2041260 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26dd5a8b80 10 client.25592 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26dd5a8b80 10 client.25592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26dd5a8b80 20 client.25592 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c205a6b0 0x55f7c205cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26dd5a8b80 4 client.25592 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c1fa0cc0 con 0x55f7c205a6b0 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26d0aae700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c205a6b0 0x55f7c205cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26d0aae700 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c205a6b0 0x55f7c205cad0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd2b845c9352fb2e server_cookie=2a7623163a1b1536 in_seq=0 out_seq=0 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.500+0000 7f26abfff700 10 client.25592 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26abfff700 1 -- 172.21.15.26:0/1804598980 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f7c1fa0cc0 con 0x55f7c205a6b0 2024-05-20T20:58:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26abfff700 10 client.25592 handle_command_reply: tid=0 2024-05-20T20:58:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 client.25592 shutdown 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 msgr2=0x55f7c2049ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2049ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26b8003930 tx=0x7f26b80047c0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 shutdown_connections 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c205a6b0 0x55f7c205cad0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c204a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c204ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/1804598980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2049ff0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 >> 172.21.15.26:0/1804598980 conn(0x55f7c1f6c240 msgr2=0x55f7c203aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.501+0000 7f26abfff700 0 client.25592 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.502+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 shutdown_connections 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.502+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/1804598980 wait complete. 2024-05-20T20:58:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.502+0000 7f26dd5a8b80 20 client.25592 trim_cache size 0 max 16384 2024-05-20T20:58:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.520+0000 7f26dd5a8b80 1 Processor -- start 2024-05-20T20:58:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.520+0000 7f26dd5a8b80 1 -- start start 2024-05-20T20:58:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 0x55f7c2046700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c204a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2048f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1b04e0 con 0x55f7c20408a0 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1add70 con 0x55f7c2041260 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1abbc0 con 0x55f7c2046320 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 0x55f7c2046700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 0x55f7c2046700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39858/0 (socket says 172.21.15.26:39858) 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 -- 172.21.15.26:0/3015216904 learned_addr learned my addr 172.21.15.26:0/3015216904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26cbfff700 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c204a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d12af700 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2048f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 -- 172.21.15.26:0/3015216904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 msgr2=0x55f7c2048f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2048f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 -- 172.21.15.26:0/3015216904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 msgr2=0x55f7c204a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c204a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 -- 172.21.15.26:0/3015216904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c2048560 con 0x55f7c20408a0 2024-05-20T20:58:21.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.521+0000 7f26d0aae700 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 0x55f7c2046700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009fd0 tx=0x7f26c000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d054a30807e79e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.522+0000 7f26b67fc700 1 -- 172.21.15.26:0/3015216904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00c4070 con 0x55f7c20408a0 2024-05-20T20:58:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.522+0000 7f26b67fc700 1 -- 172.21.15.26:0/3015216904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26c00aa100 con 0x55f7c20408a0 2024-05-20T20:58:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.522+0000 7f26b67fc700 1 -- 172.21.15.26:0/3015216904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00ae150 con 0x55f7c20408a0 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.522+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/3015216904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 msgr2=0x55f7c2046700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.522+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 0x55f7c2046700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009fd0 tx=0x7f26c000b5b0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/3015216904 shutdown_connections 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c20408a0 0x55f7c2046700 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c2046320 0x55f7c2048f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/3015216904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c204a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/3015216904 >> 172.21.15.26:0/3015216904 conn(0x55f7c1f6c240 msgr2=0x55f7c203f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/3015216904 shutdown_connections 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/3015216904 wait complete. 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- start start 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 Processor -- start 2024-05-20T20:58:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c2062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2061390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c20618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ac230 con 0x55f7c2046320 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ad6a0 con 0x55f7c2041260 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ac900 con 0x55f7c20408a0 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.523+0000 7f26dd5a8b80 10 client.0 _ll_register_callbacks cb 0x55f7c1f29950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26d0aae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c2062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2061390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26d0aae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c2062740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48746/0 (socket says 172.21.15.26:48746) 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26d0aae700 1 -- 172.21.15.26:0/4081831134 learned_addr learned my addr 172.21.15.26:0/4081831134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26d12af700 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c20618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 -- 172.21.15.26:0/4081831134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 msgr2=0x55f7c2062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c2062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 -- 172.21.15.26:0/4081831134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 msgr2=0x55f7c20618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c20618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 -- 172.21.15.26:0/4081831134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c2061e10 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26cbfff700 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2061390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26b8011a50 tx=0x7f26b80126e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711442398dc04d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26abfff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26abfff700 1 -- 172.21.15.26:0/4081831134 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8037040 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26abfff700 1 -- 172.21.15.26:0/4081831134 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f26b8012c00 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.524+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c00aa100 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26abfff700 1 -- 172.21.15.26:0/4081831134 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26b8011e90 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 10 client.25598 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7c2052650 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26abfff700 1 -- 172.21.15.26:0/4081831134 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26c00aa100 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26abfff700 1 -- 172.21.15.26:0/4081831134 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26b803a070 con 0x55f7c2041260 2024-05-20T20:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 10 client.25598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 20 client.25598 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c2052d30 0x55f7c2053140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 4 client.25598 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.525+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7c1f9a4a0 con 0x55f7c2052d30 2024-05-20T20:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.526+0000 7f26d0aae700 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c2052d30 0x55f7c2053140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.526+0000 7f26abfff700 10 client.25598 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.526+0000 7f26d0aae700 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c2052d30 0x55f7c2053140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80e41fdbfe411959 server_cookie=333dbb0be68fed54 in_seq=0 out_seq=0 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.526+0000 7f26abfff700 1 -- 172.21.15.26:0/4081831134 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7c1f9a4a0 con 0x55f7c2052d30 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.526+0000 7f26abfff700 10 client.25598 handle_command_reply: tid=0 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.526+0000 7f26dd5a8b80 1 client.25598 shutdown 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 msgr2=0x55f7c2061390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2061390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26b8011a50 tx=0x7f26b80126e0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 shutdown_connections 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7c2052d30 0x55f7c2053140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7c2046320 0x55f7c20618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7c20408a0 0x55f7c2062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 --2- 172.21.15.26:0/4081831134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7c2041260 0x55f7c2061390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 >> 172.21.15.26:0/4081831134 conn(0x55f7c1f6c240 msgr2=0x55f7c2043ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26abfff700 0 client.25598 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 shutdown_connections 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 1 -- 172.21.15.26:0/4081831134 wait complete. 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.527+0000 7f26dd5a8b80 20 client.25598 trim_cache size 0 max 16384 2024-05-20T20:58:21.531 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26bc0b2880 msgr2=0x7f26bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26bc0b2880 0x7f26bc0b4d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f26cc1031e0 tx=0x7f26b8009260 comp rx=0 tx=0).stop 2024-05-20T20:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 msgr2=0x7f26cc19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 0x7f26cc19f7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005910 tx=0x7f26c0004270 comp rx=0 tx=0).stop 2024-05-20T20:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 shutdown_connections 2024-05-20T20:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f26bc0b2880 0x7f26bc0b4d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26cc1088f0 0x7f26cc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26cc1037f0 0x7f26cc19f7c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 --2- 172.21.15.26:0/829546070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26cc1041a0 0x7f26cc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 >> 172.21.15.26:0/829546070 conn(0x7f26cc07a6a0 msgr2=0x7f26cc101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 shutdown_connections 2024-05-20T20:58:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:21.528+0000 7f26d2d12700 1 -- 172.21.15.26:0/829546070 wait complete. 2024-05-20T20:58:21.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 17951 4.31 MB/sec warmup 27 sec latency 17.823 ms 2024-05-20T20:58:22.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 18365 4.16 MB/sec warmup 28 sec latency 15.177 ms 2024-05-20T20:58:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 -- 172.21.15.26:0/3166140265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc102ac0 msgr2=0x7fd5fc102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 --2- 172.21.15.26:0/3166140265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc102ee0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009900 tx=0x7fd5f802a410 comp rx=0 tx=0).stop 2024-05-20T20:58:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 -- 172.21.15.26:0/3166140265 shutdown_connections 2024-05-20T20:58:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 --2- 172.21.15.26:0/3166140265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5fc104670 0x7fd5fc10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 --2- 172.21.15.26:0/3166140265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 --2- 172.21.15.26:0/3166140265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc102ee0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.610+0000 7fd6032e3640 1 -- 172.21.15.26:0/3166140265 >> 172.21.15.26:0/3166140265 conn(0x7fd5fc0fe030 msgr2=0x7fd5fc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 -- 172.21.15.26:0/3166140265 shutdown_connections 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 -- 172.21.15.26:0/3166140265 wait complete. 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 Processor -- start 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 -- start start 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5fc104670 0x7fd5fc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10e490 con 0x7fd5fc103cb0 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10e310 con 0x7fd5fc104670 2024-05-20T20:58:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd6032e3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10e610 con 0x7fd5fc102ac0 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.611+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39870/0 (socket says 172.21.15.26:39870) 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 -- 172.21.15.26:0/2396078356 learned_addr learned my addr 172.21.15.26:0/2396078356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd601880640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5fc104670 0x7fd5fc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60107f640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 -- 172.21.15.26:0/2396078356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc102ac0 msgr2=0x7fd5fc076800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc076800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 -- 172.21.15.26:0/2396078356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5fc104670 msgr2=0x7fd5fc075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5fc104670 0x7fd5fc075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 -- 172.21.15.26:0/2396078356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc075c10 con 0x7fd5fc103cb0 2024-05-20T20:58:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60107f640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:58:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd60087e640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc074e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d770 tx=0x7fd5ec00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd5f27fc640 1 -- 172.21.15.26:0/2396078356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec017070 con 0x7fd5fc103cb0 2024-05-20T20:58:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd5f27fc640 1 -- 172.21.15.26:0/2396078356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5ec009820 con 0x7fd5fc103cb0 2024-05-20T20:58:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1a2ce0 con 0x7fd5fc103cb0 2024-05-20T20:58:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd5f27fc640 1 -- 172.21.15.26:0/2396078356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec004b20 con 0x7fd5fc103cb0 2024-05-20T20:58:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.612+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a3180 con 0x7fd5fc103cb0 2024-05-20T20:58:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd5f27fc640 1 -- 172.21.15.26:0/2396078356 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd5ec0099d0 con 0x7fd5fc103cb0 2024-05-20T20:58:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 Processor -- start 2024-05-20T20:58:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 -- start start 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a19ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 0x5590f7a1ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1f220 0x5590f7a216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5590f798ac30 con 0x5590f7a1f220 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5590f797a7d0 con 0x5590f7a1a0b0 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.615+0000 7fd605ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590f797e810 con 0x5590f7a196c0 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 0x5590f7a1ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1f220 0x5590f7a216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 0x5590f7a1ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59354/0 (socket says 172.21.15.26:59354) 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 -- 172.21.15.26:0/460959992 learned_addr learned my addr 172.21.15.26:0/460959992 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd601880640 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a19ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 -- 172.21.15.26:0/460959992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 msgr2=0x5590f7a19ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a19ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 -- 172.21.15.26:0/460959992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1f220 msgr2=0x5590f7a216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1f220 0x5590f7a216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 -- 172.21.15.26:0/460959992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f768f210 con 0x5590f7a1a0b0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd60107f640 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 0x5590f7a1ebb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8006010 tx=0x7fd5f8031590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2926470a7b9f217 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd5f27fc640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5e80b28a0 0x7fd5e80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd601880640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5e80b28a0 0x7fd5e80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd5e77fe640 1 -- 172.21.15.26:0/460959992 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8035030 con 0x5590f7a1a0b0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd5f27fc640 1 -- 172.21.15.26:0/2396078356 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd5ec0e1c70 con 0x7fd5fc103cb0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd5e77fe640 1 -- 172.21.15.26:0/460959992 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f80044c0 con 0x5590f7a1a0b0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.616+0000 7fd5e77fe640 1 -- 172.21.15.26:0/460959992 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8003bd0 con 0x5590f7a1a0b0 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 -- 172.21.15.26:0/460959992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 msgr2=0x5590f7a1ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 0x5590f7a1ebb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8006010 tx=0x7fd5f8031590 comp rx=0 tx=0).stop 2024-05-20T20:58:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd601880640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5e80b28a0 0x7fd5e80b4d60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5590f7a1a910 tx=0x7fd5f4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 -- 172.21.15.26:0/460959992 shutdown_connections 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1f220 0x5590f7a216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1a0b0 0x5590f7a1ebb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 --2- 172.21.15.26:0/460959992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a19ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 -- 172.21.15.26:0/460959992 >> 172.21.15.26:0/460959992 conn(0x5590f7a14d30 msgr2=0x5590f7a15130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 -- 172.21.15.26:0/460959992 shutdown_connections 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.617+0000 7fd605ffd740 1 -- 172.21.15.26:0/460959992 wait complete. 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 -- start start 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 Processor -- start 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a23d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a2ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5590f7660790 con 0x5590f7a1a0b0 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5590f7936890 con 0x5590f7a196c0 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590f7a22170 con 0x5590f7a1f220 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a23d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd60107f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd605ffd740 10 client.0 _ll_register_callbacks cb 0x5590f768ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd60107f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a237e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59358/0 (socket says 172.21.15.26:59358) 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.618+0000 7fd60107f640 1 -- 172.21.15.26:0/1017331817 learned_addr learned my addr 172.21.15.26:0/1017331817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd601880640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a2ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60087e640 1 -- 172.21.15.26:0/1017331817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 msgr2=0x5590f7a2ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60087e640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a2ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60087e640 1 -- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 msgr2=0x5590f7a237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60087e640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60087e640 1 -- 172.21.15.26:0/1017331817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f775b620 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60107f640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd60087e640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a23d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0df760 tx=0x7fd5ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a42b4bf79c6fcea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd5e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1017331817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec020070 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1017331817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd5ec0a54c0 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590f798cb80 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1017331817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec0a9930 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd605ffd740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5590f7a2ddb0 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.619+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1017331817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5590f798cb80 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1017331817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd5ec0e9040 con 0x5590f7a1a0b0 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd605ffd740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd605ffd740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd605ffd740 20 client.16056 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a2f420 0x5590f7a31850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd605ffd740 4 client.16056 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590f7986360 con 0x5590f7a2f420 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd60107f640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a2f420 0x5590f7a31850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd60107f640 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a2f420 0x5590f7a31850 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45fc3392498e9a server_cookie=2295a477126b162c in_seq=0 out_seq=0 2024-05-20T20:58:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.620+0000 7fd5e4ff9640 10 client.16056 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1017331817 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5590f7986360 con 0x5590f7a2f420 2024-05-20T20:58:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd5e4ff9640 10 client.16056 handle_command_reply: tid=0 2024-05-20T20:58:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 client.16056 shutdown 2024-05-20T20:58:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 msgr2=0x5590f7a23d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a23d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0df760 tx=0x7fd5ec0057f0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 shutdown_connections 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a2f420 0x5590f7a31850 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a2ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a23d20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd5e4ff9640 0 client.16056 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1017331817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 >> 172.21.15.26:0/1017331817 conn(0x5590f7a14d30 msgr2=0x5590f7a13fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 shutdown_connections 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.621+0000 7fd605ffd740 1 -- 172.21.15.26:0/1017331817 wait complete. 2024-05-20T20:58:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.622+0000 7fd605ffd740 20 client.16056 trim_cache size 0 max 16384 2024-05-20T20:58:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.639+0000 7fd605ffd740 1 Processor -- start 2024-05-20T20:58:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 -- start start 2024-05-20T20:58:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a13e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1aea60 con 0x5590f7a1a0b0 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ac330 con 0x5590f7a196c0 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd605ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1aa180 con 0x5590f7a1f220 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a13e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a2ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39896/0 (socket says 172.21.15.26:39896) 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a13e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59366/0 (socket says 172.21.15.26:59366) 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60087e640 1 -- 172.21.15.26:0/3881555996 learned_addr learned my addr 172.21.15.26:0/3881555996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd601880640 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 -- 172.21.15.26:0/3881555996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 msgr2=0x5590f7a36150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a36150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 -- 172.21.15.26:0/3881555996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 msgr2=0x5590f7a2ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a2ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.640+0000 7fd60107f640 1 -- 172.21.15.26:0/3881555996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f7a14b60 con 0x5590f7a196c0 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.641+0000 7fd60107f640 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a13e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005ab0 tx=0x7fd5f8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83213a53e875e8af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.641+0000 7fd5e6ffd640 1 -- 172.21.15.26:0/3881555996 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8035040 con 0x5590f7a196c0 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.641+0000 7fd5e6ffd640 1 -- 172.21.15.26:0/3881555996 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f8031cf0 con 0x5590f7a196c0 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.641+0000 7fd5e6ffd640 1 -- 172.21.15.26:0/3881555996 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8050690 con 0x5590f7a196c0 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 -- 172.21.15.26:0/3881555996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 msgr2=0x5590f7a13e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a13e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005ab0 tx=0x7fd5f8005bd0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 -- 172.21.15.26:0/3881555996 shutdown_connections 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a1f220 0x5590f7a36150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 --2- 172.21.15.26:0/3881555996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a196c0 0x5590f7a13e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 -- 172.21.15.26:0/3881555996 >> 172.21.15.26:0/3881555996 conn(0x5590f7a14d30 msgr2=0x5590f7a13720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 -- 172.21.15.26:0/3881555996 shutdown_connections 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 -- 172.21.15.26:0/3881555996 wait complete. 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 -- start start 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.642+0000 7fd605ffd740 1 Processor -- start 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a28b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 0x5590f7a29b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1aa7f0 con 0x5590f7a1a0b0 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1abc60 con 0x5590f7a1f220 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1aaec0 con 0x5590f7a196c0 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd605ffd740 10 client.0 _ll_register_callbacks cb 0x7fd5ec0a50f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd60107f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 0x5590f7a29b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd60087e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a28b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 0x5590f7a29b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59382/0 (socket says 172.21.15.26:59382) 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 -- 172.21.15.26:0/1755664809 learned_addr learned my addr 172.21.15.26:0/1755664809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd60107f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a285d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48818/0 (socket says 172.21.15.26:48818) 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 -- 172.21.15.26:0/1755664809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 msgr2=0x5590f7a285d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a285d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 -- 172.21.15.26:0/1755664809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 msgr2=0x5590f7a28b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a28b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 -- 172.21.15.26:0/1755664809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f7a2a110 con 0x5590f7a1f220 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd60087e640 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a28b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd601880640 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 0x5590f7a29b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400faa0 tx=0x7fd5f4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966d0f52ca604cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.643+0000 7fd5e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1755664809 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4004b20 con 0x5590f7a1f220 2024-05-20T20:58:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1755664809 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd5f4004cc0 con 0x5590f7a1f220 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f8031cf0 con 0x5590f7a1f220 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1755664809 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5f401d8a0 con 0x5590f7a1f220 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 10 client.25616 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5590f7a9a270 con 0x5590f7a1f220 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1755664809 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f8031cf0 con 0x5590f7a1f220 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1755664809 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd5f4005610 con 0x5590f7a1f220 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 10 client.25616 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 20 client.25616 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a9acc0 0x5590f7a9d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 4 client.25616 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590f79005f0 con 0x5590f7a9acc0 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.644+0000 7fd60107f640 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a9acc0 0x5590f7a9d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.645+0000 7fd5e4ff9640 10 client.25616 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.645+0000 7fd60107f640 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a9acc0 0x5590f7a9d0e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dce8339aadc421 server_cookie=8bc252bf568644c2 in_seq=0 out_seq=0 2024-05-20T20:58:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.683+0000 7fd5e4ff9640 1 -- 172.21.15.26:0/1755664809 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5590f79005f0 con 0x5590f7a9acc0 2024-05-20T20:58:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.683+0000 7fd5e4ff9640 10 client.25616 handle_command_reply: tid=0 2024-05-20T20:58:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 client.25616 shutdown 2024-05-20T20:58:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 msgr2=0x5590f7a29b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 0x5590f7a29b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400faa0 tx=0x7fd5f4004a40 comp rx=0 tx=0).stop 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 shutdown_connections 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5590f7a9acc0 0x5590f7a9d0e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5590f7a1f220 0x5590f7a29b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590f7a1a0b0 0x5590f7a28b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 --2- 172.21.15.26:0/1755664809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590f7a196c0 0x5590f7a285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd5e4ff9640 0 client.25616 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 >> 172.21.15.26:0/1755664809 conn(0x5590f7a14d30 msgr2=0x5590f7a13fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 shutdown_connections 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 1 -- 172.21.15.26:0/1755664809 wait complete. 2024-05-20T20:58:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.684+0000 7fd605ffd740 20 client.25616 trim_cache size 0 max 16384 2024-05-20T20:58:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:58:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:58:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5e80b28a0 msgr2=0x7fd5e80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5e80b28a0 0x7fd5e80b4d60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5590f7a1a910 tx=0x7fd5f4009500 comp rx=0 tx=0).stop 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 msgr2=0x7fd5fc074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc074e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d770 tx=0x7fd5ec00db70 comp rx=0 tx=0).stop 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 shutdown_connections 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd5e80b28a0 0x7fd5e80b4d60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5fc104670 0x7fd5fc075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc103cb0 0x7fd5fc074e40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 --2- 172.21.15.26:0/2396078356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc102ac0 0x7fd5fc076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 >> 172.21.15.26:0/2396078356 conn(0x7fd5fc0fe030 msgr2=0x7fd5fc107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 shutdown_connections 2024-05-20T20:58:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:22.685+0000 7fd6032e3640 1 -- 172.21.15.26:0/2396078356 wait complete. 2024-05-20T20:58:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:22 smithi144 ceph-mon[35660]: pgmap v455: 97 pgs: 97 active+clean; 28 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-05-20T20:58:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:22 smithi055 ceph-mon[35213]: pgmap v455: 97 pgs: 97 active+clean; 28 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-05-20T20:58:23.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:22 smithi026 ceph-mon[31990]: pgmap v455: 97 pgs: 97 active+clean; 28 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-05-20T20:58:23.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 18957 4.09 MB/sec warmup 29 sec latency 33.182 ms 2024-05-20T20:58:24.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 19693 4.07 MB/sec warmup 30 sec latency 45.215 ms 2024-05-20T20:58:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:24 smithi144 ceph-mon[35660]: pgmap v456: 97 pgs: 97 active+clean; 30 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T20:58:25.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:24 smithi055 ceph-mon[35213]: pgmap v456: 97 pgs: 97 active+clean; 30 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T20:58:25.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:24 smithi026 ceph-mon[31990]: pgmap v456: 97 pgs: 97 active+clean; 30 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T20:58:25.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20337 4.02 MB/sec warmup 31 sec latency 47.583 ms 2024-05-20T20:58:26.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 21085 4.09 MB/sec warmup 32 sec latency 105.980 ms 2024-05-20T20:58:27.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:58:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:26 smithi144 ceph-mon[35660]: pgmap v457: 97 pgs: 97 active+clean; 26 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-20T20:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:27.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:26 smithi026 ceph-mon[31990]: pgmap v457: 97 pgs: 97 active+clean; 26 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-20T20:58:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:26 smithi055 ceph-mon[35213]: pgmap v457: 97 pgs: 97 active+clean; 26 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-20T20:58:27.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 21612 4.10 MB/sec warmup 33 sec latency 40.818 ms 2024-05-20T20:58:28.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 21938 3.99 MB/sec warmup 34 sec latency 87.465 ms 2024-05-20T20:58:29.110 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:28 smithi026 ceph-mon[31990]: pgmap v458: 97 pgs: 97 active+clean; 26 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-20T20:58:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:28 smithi144 ceph-mon[35660]: pgmap v458: 97 pgs: 97 active+clean; 26 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-20T20:58:29.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:28 smithi055 ceph-mon[35213]: pgmap v458: 97 pgs: 97 active+clean; 26 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-20T20:58:29.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 22535 3.94 MB/sec warmup 35 sec latency 18.852 ms 2024-05-20T20:58:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 -- 172.21.15.26:0/1788717104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc10ff40 msgr2=0x7ff6cc112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 --2- 172.21.15.26:0/1788717104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc10ff40 0x7ff6cc112330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400b0a0 tx=0x7ff6c400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 -- 172.21.15.26:0/1788717104 shutdown_connections 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 --2- 172.21.15.26:0/1788717104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc10ff40 0x7ff6cc112330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 --2- 172.21.15.26:0/1788717104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6cc075770 0x7ff6cc075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 --2- 172.21.15.26:0/1788717104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6cc076090 0x7ff6cc10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 -- 172.21.15.26:0/1788717104 >> 172.21.15.26:0/1788717104 conn(0x7ff6cc07a6f0 msgr2=0x7ff6cc07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 -- 172.21.15.26:0/1788717104 shutdown_connections 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.352+0000 7ff6d3316700 1 -- 172.21.15.26:0/1788717104 wait complete. 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 Processor -- start 2024-05-20T20:58:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 -- start start 2024-05-20T20:58:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6cc075770 0x7ff6cc19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 0x7ff6cc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6cc10ff40 0x7ff6cc1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1157f0 con 0x7ff6cc076090 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc115670 con 0x7ff6cc075770 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d3316700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc115970 con 0x7ff6cc10ff40 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 0x7ff6cc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 0x7ff6cc19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33406/0 (socket says 172.21.15.26:33406) 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 -- 172.21.15.26:0/4175655918 learned_addr learned my addr 172.21.15.26:0/4175655918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6cc10ff40 0x7ff6cc1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6cc075770 0x7ff6cc19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 -- 172.21.15.26:0/4175655918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6cc10ff40 msgr2=0x7ff6cc1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6cc10ff40 0x7ff6cc1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 -- 172.21.15.26:0/4175655918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6cc075770 msgr2=0x7ff6cc19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.353+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6cc075770 0x7ff6cc19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6d08b1700 1 -- 172.21.15.26:0/4175655918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc1a0d50 con 0x7ff6cc076090 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 0x7ff6cc19c280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d980 tx=0x7ff6bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6c27fc700 1 -- 172.21.15.26:0/4175655918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc017070 con 0x7ff6cc076090 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6cc1a0fe0 con 0x7ff6cc076090 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6c27fc700 1 -- 172.21.15.26:0/4175655918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc005020 con 0x7ff6cc076090 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6c27fc700 1 -- 172.21.15.26:0/4175655918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00f980 con 0x7ff6cc076090 2024-05-20T20:58:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.354+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6cc1a88c0 con 0x7ff6cc076090 2024-05-20T20:58:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.357+0000 7ff6c27fc700 1 -- 172.21.15.26:0/4175655918 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff6bc004a30 con 0x7ff6cc076090 2024-05-20T20:58:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.358+0000 7ff6c27fc700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff6b80b2880 0x7ff6b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.358+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff6b80b2880 0x7ff6b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.358+0000 7ff6c27fc700 1 -- 172.21.15.26:0/4175655918 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff6bc0e2030 con 0x7ff6cc076090 2024-05-20T20:58:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.358+0000 7ff6ddbacb80 1 Processor -- start 2024-05-20T20:58:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.358+0000 7ff6ddbacb80 1 -- start start 2024-05-20T20:58:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b08700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 0x55dd79b02d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dd79a60d70 con 0x55dd79b028a0 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dd79a50910 con 0x55dd79b03260 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dd79a54950 con 0x55dd79b08320 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 0x55dd79b02d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 0x55dd79b02d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33420/0 (socket says 172.21.15.26:33420) 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 -- 172.21.15.26:0/1918768899 learned_addr learned my addr 172.21.15.26:0/1918768899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff6b80b2880 0x7ff6b80b4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009c60 tx=0x7ff6c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b08700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 -- 172.21.15.26:0/1918768899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 msgr2=0x55dd79b08700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b08700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 -- 172.21.15.26:0/1918768899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 msgr2=0x55dd79b0f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 -- 172.21.15.26:0/1918768899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd799f9fb0 con 0x55dd79b028a0 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 0x55dd79b02d20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0a6860 tx=0x7ff6bc0a9100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b06cebfcbc56778 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6b77fe700 1 -- 172.21.15.26:0/1918768899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0a95f0 con 0x55dd79b028a0 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6b77fe700 1 -- 172.21.15.26:0/1918768899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc0ad9a0 con 0x55dd79b028a0 2024-05-20T20:58:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.359+0000 7ff6b77fe700 1 -- 172.21.15.26:0/1918768899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0b2220 con 0x55dd79b028a0 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/1918768899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 msgr2=0x55dd79b02d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 0x55dd79b02d20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0a6860 tx=0x7ff6bc0a9100 comp rx=0 tx=0).stop 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/1918768899 shutdown_connections 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b08700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/1918768899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b028a0 0x55dd79b02d20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/1918768899 >> 172.21.15.26:0/1918768899 conn(0x55dd79a2e240 msgr2=0x55dd79afd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/1918768899 shutdown_connections 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.360+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/1918768899 wait complete. 2024-05-20T20:58:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 -- start start 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 Processor -- start 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b03260 0x55dd79b0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b08320 0x55dd79b0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 0x55dd79b0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dd79956fa0 con 0x55dd79b08320 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dd799b8da0 con 0x55dd79b0c760 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dd79b0f620 con 0x55dd79b03260 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6ddbacb80 10 client.0 _ll_register_callbacks cb 0x55dd799eb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.361+0000 7ff6d08b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b08320 0x55dd79b0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 0x55dd79b0cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 0x55dd79b0cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54970/0 (socket says 172.21.15.26:54970) 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 -- 172.21.15.26:0/201601950 learned_addr learned my addr 172.21.15.26:0/201601950 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b03260 0x55dd79b0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 -- 172.21.15.26:0/201601950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b03260 msgr2=0x55dd79b0ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b03260 0x55dd79b0ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 -- 172.21.15.26:0/201601950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b08320 msgr2=0x55dd79b0c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b08320 0x55dd79b0c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 -- 172.21.15.26:0/201601950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd799a08d0 con 0x55dd79b0c760 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b08320 0x55dd79b0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 0x55dd79b0cbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4015000 tx=0x7ff6c400bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec86230902b7805a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/201601950 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4003c00 con 0x55dd79b0c760 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6bc0ad9a0 con 0x55dd79b0c760 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.362+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/201601950 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff6c4003da0 con 0x55dd79b0c760 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/201601950 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6c40060e0 con 0x55dd79b0c760 2024-05-20T20:58:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 10 client.25628 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd79a0a760 con 0x55dd79b0c760 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/201601950 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc0ad9a0 con 0x55dd79b0c760 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/201601950 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff6c402f050 con 0x55dd79b0c760 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 10 client.25628 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 20 client.25628 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b1c6e0 0x55dd79b1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 4 client.25628 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd79a62cc0 con 0x55dd79b1c6e0 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.363+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b1c6e0 0x55dd79b1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.364+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b1c6e0 0x55dd79b1ead0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac12f7377201e92 server_cookie=3a5275e90e2d0f79 in_seq=0 out_seq=0 2024-05-20T20:58:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.364+0000 7ff6b4ff9700 10 client.25628 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.364+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/201601950 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dd79a62cc0 con 0x55dd79b1c6e0 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.364+0000 7ff6b4ff9700 10 client.25628 handle_command_reply: tid=0 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.364+0000 7ff6ddbacb80 1 client.25628 shutdown 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 msgr2=0x55dd79b0cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 0x55dd79b0cbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4015000 tx=0x7ff6c400bd50 comp rx=0 tx=0).stop 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 shutdown_connections 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b1c6e0 0x55dd79b1ead0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b0c760 0x55dd79b0cbe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b03260 0x55dd79b0ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/201601950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b08320 0x55dd79b0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 >> 172.21.15.26:0/201601950 conn(0x55dd79a2e240 msgr2=0x55dd79afcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6b4ff9700 0 client.25628 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 shutdown_connections 2024-05-20T20:58:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/201601950 wait complete. 2024-05-20T20:58:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.365+0000 7ff6ddbacb80 20 client.25628 trim_cache size 0 max 16384 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 Processor -- start 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 -- start start 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79afd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b01890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1b2700 con 0x55dd79b0c760 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1aff90 con 0x55dd79b03260 2024-05-20T20:58:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1adde0 con 0x55dd79b08320 2024-05-20T20:58:30.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6d10b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6d18b3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b01890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6d10b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54978/0 (socket says 172.21.15.26:54978) 2024-05-20T20:58:30.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6d18b3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b01890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33434/0 (socket says 172.21.15.26:33434) 2024-05-20T20:58:30.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6d10b2700 1 -- 172.21.15.26:0/3975561427 learned_addr learned my addr 172.21.15.26:0/3975561427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.382+0000 7ff6d08b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79afd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6d10b2700 1 -- 172.21.15.26:0/3975561427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 msgr2=0x55dd79afd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79afd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6d10b2700 1 -- 172.21.15.26:0/3975561427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 msgr2=0x55dd79b01890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b01890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6d10b2700 1 -- 172.21.15.26:0/3975561427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd79b07f40 con 0x55dd79b03260 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0c1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005950 tx=0x7ff6c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a869336a83bb1639 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6b6ffd700 1 -- 172.21.15.26:0/3975561427 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8003ce0 con 0x55dd79b03260 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6b6ffd700 1 -- 172.21.15.26:0/3975561427 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c8003e80 con 0x55dd79b03260 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.383+0000 7ff6b6ffd700 1 -- 172.21.15.26:0/3975561427 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c801d670 con 0x55dd79b03260 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3975561427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 msgr2=0x55dd79b0c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0c1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005950 tx=0x7ff6c8014000 comp rx=0 tx=0).stop 2024-05-20T20:58:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3975561427 shutdown_connections 2024-05-20T20:58:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b01890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b0c1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3975561427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79afd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3975561427 >> 172.21.15.26:0/3975561427 conn(0x55dd79a2e240 msgr2=0x55dd79b0bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3975561427 shutdown_connections 2024-05-20T20:58:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.384+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3975561427 wait complete. 2024-05-20T20:58:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 -- start start 2024-05-20T20:58:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 Processor -- start 2024-05-20T20:58:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b23930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b13870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1ae450 con 0x55dd79b0c760 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1af8c0 con 0x55dd79b03260 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1aeb20 con 0x55dd79b08320 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6ddbacb80 10 client.0 _ll_register_callbacks cb 0x55dd799eb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6d18b3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b13870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6d10b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6d18b3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b13870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33438/0 (socket says 172.21.15.26:33438) 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6d18b3700 1 -- 172.21.15.26:0/3479785170 learned_addr learned my addr 172.21.15.26:0/3479785170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.385+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b23930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d18b3700 1 -- 172.21.15.26:0/3479785170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 msgr2=0x55dd79b23930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b23930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d18b3700 1 -- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 msgr2=0x55dd79b233f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:30.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b233f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d18b3700 1 -- 172.21.15.26:0/3479785170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd79b13db0 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d08b1700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b23930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6d18b3700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b13870 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40062a0 tx=0x7ff6c40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e9870cf39d6e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/3479785170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4035060 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c8003e80 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/3479785170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7ff6c400f040 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/3479785170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6c4003d90 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6ddbacb80 10 client.16080 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.386+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd79b150b0 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/3479785170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c8003e80 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/3479785170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff6c4039030 con 0x55dd79b0c760 2024-05-20T20:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6ddbacb80 10 client.16080 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6ddbacb80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6ddbacb80 20 client.16080 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b871f0 0x55dd79b895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6ddbacb80 4 client.16080 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd79a5c4a0 con 0x55dd79b871f0 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b871f0 0x55dd79b895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6d10b2700 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b871f0 0x55dd79b895e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17734f5c48f19ff2 server_cookie=4d76b46da2a58635 in_seq=0 out_seq=0 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.387+0000 7ff6b4ff9700 10 client.16080 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6b4ff9700 1 -- 172.21.15.26:0/3479785170 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd79a5c4a0 con 0x55dd79b871f0 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6b4ff9700 10 client.16080 handle_command_reply: tid=0 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 client.16080 shutdown 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 msgr2=0x55dd79b13870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b13870 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40062a0 tx=0x7ff6c40099e0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 shutdown_connections 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dd79b871f0 0x55dd79b895e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dd79b0c760 0x55dd79b13870 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd79b03260 0x55dd79b233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 --2- 172.21.15.26:0/3479785170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd79b08320 0x55dd79b23930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 >> 172.21.15.26:0/3479785170 conn(0x55dd79a2e240 msgr2=0x55dd79b01890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6b4ff9700 0 client.16080 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 shutdown_connections 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 1 -- 172.21.15.26:0/3479785170 wait complete. 2024-05-20T20:58:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.388+0000 7ff6ddbacb80 20 client.16080 trim_cache size 0 max 16384 2024-05-20T20:58:30.393 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff6b80b2880 msgr2=0x7ff6b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff6b80b2880 0x7ff6b80b4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009c60 tx=0x7ff6c8009260 comp rx=0 tx=0).stop 2024-05-20T20:58:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 msgr2=0x7ff6cc19c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 0x7ff6cc19c280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d980 tx=0x7ff6bc00dc90 comp rx=0 tx=0).stop 2024-05-20T20:58:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 shutdown_connections 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6cc10ff40 0x7ff6cc1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6cc075770 0x7ff6cc19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff6b80b2880 0x7ff6b80b4d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 --2- 172.21.15.26:0/4175655918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6cc076090 0x7ff6cc19c280 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 >> 172.21.15.26:0/4175655918 conn(0x7ff6cc07a6f0 msgr2=0x7ff6cc100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 shutdown_connections 2024-05-20T20:58:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:30.389+0000 7ff6d3316700 1 -- 172.21.15.26:0/4175655918 wait complete. 2024-05-20T20:58:30.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 23238 3.92 MB/sec warmup 36 sec latency 47.787 ms 2024-05-20T20:58:31.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:30 smithi026 ceph-mon[31990]: pgmap v459: 97 pgs: 97 active+clean; 27 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T20:58:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:30 smithi144 ceph-mon[35660]: pgmap v459: 97 pgs: 97 active+clean; 27 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T20:58:31.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:30 smithi055 ceph-mon[35213]: pgmap v459: 97 pgs: 97 active+clean; 27 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T20:58:31.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 23905 3.88 MB/sec warmup 37 sec latency 19.407 ms 2024-05-20T20:58:32.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:58:32.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:58:32.517 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:32.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 24657 3.95 MB/sec warmup 38 sec latency 13.587 ms 2024-05-20T20:58:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:32 smithi026 ceph-mon[31990]: pgmap v460: 97 pgs: 97 active+clean; 28 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T20:58:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:32 smithi144 ceph-mon[35660]: pgmap v460: 97 pgs: 97 active+clean; 28 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T20:58:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:32 smithi055 ceph-mon[35213]: pgmap v460: 97 pgs: 97 active+clean; 28 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T20:58:33.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 25262 3.97 MB/sec warmup 39 sec latency 17.544 ms 2024-05-20T20:58:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 -- 172.21.15.26:0/3490459652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc1077d0 msgr2=0x7f5ecc10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 --2- 172.21.15.26:0/3490459652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc10f370 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400b0d0 tx=0x7f5ec400b3e0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 -- 172.21.15.26:0/3490459652 shutdown_connections 2024-05-20T20:58:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 --2- 172.21.15.26:0/3490459652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc10f370 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 --2- 172.21.15.26:0/3490459652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc102a80 0x7f5ecc102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 --2- 172.21.15.26:0/3490459652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 0x7f5ecc1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 -- 172.21.15.26:0/3490459652 >> 172.21.15.26:0/3490459652 conn(0x7f5ecc0793f0 msgr2=0x7f5ecc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 -- 172.21.15.26:0/3490459652 shutdown_connections 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.416+0000 7f5ed159a700 1 -- 172.21.15.26:0/3490459652 wait complete. 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 Processor -- start 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 -- start start 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 0x7f5ecc104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 0x7f5ecc104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc112540 con 0x7f5ecc102a80 2024-05-20T20:58:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1123c0 con 0x7f5ecc103430 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ed159a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1126c0 con 0x7f5ecc1077d0 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 0x7f5ecc104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 0x7f5ecc104140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33464/0 (socket says 172.21.15.26:33464) 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ec27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 0x7f5ecc104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ecaffd700 1 -- 172.21.15.26:0/1340338445 learned_addr learned my addr 172.21.15.26:0/1340338445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.417+0000 7f5ecb7fe700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecaffd700 1 -- 172.21.15.26:0/1340338445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1077d0 msgr2=0x7f5ecc104bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc104bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecaffd700 1 -- 172.21.15.26:0/1340338445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 msgr2=0x7f5ecc104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 0x7f5ecc104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecaffd700 1 -- 172.21.15.26:0/1340338445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc1a80d0 con 0x7f5ecc102a80 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 0x7f5ecc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecb7fe700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 0x7f5ecc104140 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d950 tx=0x7f5ebc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ec8ff9700 1 -- 172.21.15.26:0/1340338445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc016070 con 0x7f5ecc102a80 2024-05-20T20:58:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc1a83c0 con 0x7f5ecc102a80 2024-05-20T20:58:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a8950 con 0x7f5ecc102a80 2024-05-20T20:58:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ec8ff9700 1 -- 172.21.15.26:0/1340338445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ebc00fd20 con 0x7f5ecc102a80 2024-05-20T20:58:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.418+0000 7f5ec8ff9700 1 -- 172.21.15.26:0/1340338445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00e3b0 con 0x7f5ecc102a80 2024-05-20T20:58:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.421+0000 7f5ec8ff9700 1 -- 172.21.15.26:0/1340338445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5ebc019070 con 0x7f5ecc102a80 2024-05-20T20:58:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.422+0000 7f5ec8ff9700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5eb80b2900 0x7f5eb80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.422+0000 7f5ec8ff9700 1 -- 172.21.15.26:0/1340338445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f5ebc0e1ef0 con 0x7f5ecc102a80 2024-05-20T20:58:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.422+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5eb80b2900 0x7f5eb80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.422+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5eb80b2900 0x7f5eb80b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc1026f0 tx=0x7f5eb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.422+0000 7f5edbe30b80 1 Processor -- start 2024-05-20T20:58:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 -- start start 2024-05-20T20:58:34.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 0x55bd27792cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 0x55bd27797420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd276f1d10 con 0x55bd27796fe0 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bd276e18b0 con 0x55bd277928e0 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd276e58f0 con 0x55bd27793290 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5ecb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 0x55bd27797420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 0x55bd27792cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5ecb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 0x55bd27797420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33480/0 (socket says 172.21.15.26:33480) 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5ecb7fe700 1 -- 172.21.15.26:0/386713049 learned_addr learned my addr 172.21.15.26:0/386713049 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 0x55bd27792cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55004/0 (socket says 172.21.15.26:55004) 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.423+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecaffd700 1 -- 172.21.15.26:0/386713049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 msgr2=0x55bd277969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecaffd700 1 -- 172.21.15.26:0/386713049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 msgr2=0x55bd27797420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 0x55bd27797420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecaffd700 1 -- 172.21.15.26:0/386713049 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd27631950 con 0x55bd277928e0 2024-05-20T20:58:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecb7fe700 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 0x55bd27797420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 0x55bd27792cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0dfd00 tx=0x7f5ebc0a9010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76601cc166ba876 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ec17fa700 1 -- 172.21.15.26:0/386713049 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc020070 con 0x55bd277928e0 2024-05-20T20:58:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ec17fa700 1 -- 172.21.15.26:0/386713049 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ebc0dfe50 con 0x55bd277928e0 2024-05-20T20:58:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.424+0000 7f5ec17fa700 1 -- 172.21.15.26:0/386713049 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0b0e40 con 0x55bd277928e0 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 -- 172.21.15.26:0/386713049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 msgr2=0x55bd27792cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 0x55bd27792cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0dfd00 tx=0x7f5ebc0a9010 comp rx=0 tx=0).stop 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 -- 172.21.15.26:0/386713049 shutdown_connections 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27796fe0 0x55bd27797420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd277928e0 0x55bd27792cc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/386713049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 -- 172.21.15.26:0/386713049 >> 172.21.15.26:0/386713049 conn(0x55bd276c0100 msgr2=0x55bd2778cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 -- 172.21.15.26:0/386713049 shutdown_connections 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 -- 172.21.15.26:0/386713049 wait complete. 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.425+0000 7f5edbe30b80 1 -- start start 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 Processor -- start 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd277928e0 0x55bd2779e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 0x55bd2779b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5ec27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 0x55bd2779b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5ec27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 0x55bd2779b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33488/0 (socket says 172.21.15.26:33488) 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5ec27fc700 1 -- 172.21.15.26:0/3133810993 learned_addr learned my addr 172.21.15.26:0/3133810993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd2779bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd277928e0 0x55bd2779e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd275e8020 con 0x55bd27793290 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bd27683840 con 0x55bd27796fe0 2024-05-20T20:58:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd2779f0e0 con 0x55bd277928e0 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.426+0000 7f5edbe30b80 10 client.0 _ll_register_callbacks cb 0x55bd2767c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ecb7fe700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd2779bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ec27fc700 1 -- 172.21.15.26:0/3133810993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd277928e0 msgr2=0x55bd2779e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd277928e0 0x55bd2779e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ec27fc700 1 -- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 msgr2=0x55bd2779bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd2779bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ec27fc700 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd27680710 con 0x55bd27793290 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 0x55bd2779b790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400eb10 tx=0x7f5eb40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90366d060987f534 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5eb27fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.427+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3133810993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb401fca0 con 0x55bd27793290 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3133810993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f5eb40273e0 con 0x55bd27793290 2024-05-20T20:58:34.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ebc0dfe50 con 0x55bd27793290 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3133810993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5eb401fe40 con 0x55bd27793290 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5edbe30b80 10 client.16092 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd277b2120 con 0x55bd27793290 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3133810993 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ebc0dfe50 con 0x55bd27793290 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3133810993 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5eb4030020 con 0x55bd27793290 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5edbe30b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5edbe30b80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.428+0000 7f5edbe30b80 20 client.16092 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.429+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd277b4750 0x55bd277b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.429+0000 7f5edbe30b80 4 client.16092 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.429+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd2775ea60 con 0x55bd277b4750 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.429+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd277b4750 0x55bd277b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.429+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd277b4750 0x55bd277b6b70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd18756ecd606bb server_cookie=f1168c213a3d2df8 in_seq=0 out_seq=0 2024-05-20T20:58:34.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.429+0000 7f5eb27fc700 10 client.16092 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:34.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3133810993 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bd2775ea60 con 0x55bd277b4750 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5eb27fc700 10 client.16092 handle_command_reply: tid=0 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5edbe30b80 1 client.16092 shutdown 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 msgr2=0x55bd2779b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 0x55bd2779b790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400eb10 tx=0x7f5eb40047c0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 shutdown_connections 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.431+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd277b4750 0x55bd277b6b70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd2779bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd277928e0 0x55bd2779e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3133810993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd27793290 0x55bd2779b790 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 >> 172.21.15.26:0/3133810993 conn(0x55bd276c0100 msgr2=0x55bd2778c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5eb27fc700 0 client.16092 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 shutdown_connections 2024-05-20T20:58:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3133810993 wait complete. 2024-05-20T20:58:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.432+0000 7f5edbe30b80 20 client.16092 trim_cache size 0 max 16384 2024-05-20T20:58:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.457+0000 7f5edbe30b80 1 Processor -- start 2024-05-20T20:58:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.457+0000 7f5edbe30b80 1 -- start start 2024-05-20T20:58:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.457+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd2779d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27793290 0x55bd2778cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27796fe0 0x55bd2778d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00feb0 con 0x55bd277928e0 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00d740 con 0x55bd27793290 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00b590 con 0x55bd27796fe0 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd2779d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd2779d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33496/0 (socket says 172.21.15.26:33496) 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3715194358 learned_addr learned my addr 172.21.15.26:0/3715194358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27793290 0x55bd2778cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecb7fe700 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27796fe0 0x55bd2778d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3715194358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27796fe0 msgr2=0x55bd2778d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27796fe0 0x55bd2778d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3715194358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27793290 msgr2=0x55bd2778cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27793290 0x55bd2778cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3715194358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd27799d00 con 0x55bd277928e0 2024-05-20T20:58:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd2779d6d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0df9e0 tx=0x7f5ebc0b2490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f6e560d8c053eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.458+0000 7f5ec0ff9700 1 -- 172.21.15.26:0/3715194358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0c2070 con 0x55bd277928e0 2024-05-20T20:58:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5ec0ff9700 1 -- 172.21.15.26:0/3715194358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ebc0dfe50 con 0x55bd277928e0 2024-05-20T20:58:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5ec0ff9700 1 -- 172.21.15.26:0/3715194358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc020040 con 0x55bd277928e0 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3715194358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 msgr2=0x55bd2779d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd2779d6d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0df9e0 tx=0x7f5ebc0b2490 comp rx=0 tx=0).stop 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3715194358 shutdown_connections 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd2779d6d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27796fe0 0x55bd2778d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3715194358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27793290 0x55bd2778cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.459+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3715194358 >> 172.21.15.26:0/3715194358 conn(0x55bd276c0100 msgr2=0x55bd2779cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3715194358 shutdown_connections 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3715194358 wait complete. 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 -- start start 2024-05-20T20:58:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 Processor -- start 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd277accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd277ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00bc00 con 0x55bd277928e0 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00d070 con 0x55bd27796fe0 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00c2d0 con 0x55bd27793290 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.460+0000 7f5edbe30b80 10 client.0 _ll_register_callbacks cb 0x55bd2767c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd277accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd277accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33510/0 (socket says 172.21.15.26:33510) 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3970519132 learned_addr learned my addr 172.21.15.26:0/3970519132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecb7fe700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd277ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3970519132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 msgr2=0x55bd277ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 msgr2=0x55bd277ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd277ab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 -- 172.21.15.26:0/3970519132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd277ab8c0 con 0x55bd277928e0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5ecaffd700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd277accb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0a6800 tx=0x7f5ebc0dfa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503e56db18788536 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5eb27fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3970519132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc020050 con 0x55bd277928e0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3970519132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f5ebc08fbe0 con 0x55bd277928e0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ebc0dfe50 con 0x55bd277928e0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3970519132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc0c2070 con 0x55bd277928e0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5edbe30b80 10 client.16098 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.461+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd2781fe40 con 0x55bd277928e0 2024-05-20T20:58:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3970519132 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ebc0dfe50 con 0x55bd277928e0 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3970519132 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5ebc0a8880 con 0x55bd277928e0 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5edbe30b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5edbe30b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5edbe30b80 20 client.16098 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd278200c0 0x55bd278224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5edbe30b80 4 client.16098 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd276f3c60 con 0x55bd278200c0 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.462+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd278200c0 0x55bd278224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.463+0000 7f5eb27fc700 10 client.16098 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.463+0000 7f5ec27fc700 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd278200c0 0x55bd278224b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3e55cd4def5461 server_cookie=17f7078b5644eb0f in_seq=0 out_seq=0 2024-05-20T20:58:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.463+0000 7f5eb27fc700 1 -- 172.21.15.26:0/3970519132 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd276f3c60 con 0x55bd278200c0 2024-05-20T20:58:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.463+0000 7f5eb27fc700 10 client.16098 handle_command_reply: tid=0 2024-05-20T20:58:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.463+0000 7f5edbe30b80 1 client.16098 shutdown 2024-05-20T20:58:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 msgr2=0x55bd277accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd277accb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0a6800 tx=0x7f5ebc0dfa10 comp rx=0 tx=0).stop 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 shutdown_connections 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd278200c0 0x55bd278224b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd277928e0 0x55bd277accb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd27796fe0 0x55bd277ab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 --2- 172.21.15.26:0/3970519132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd27793290 0x55bd277ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 >> 172.21.15.26:0/3970519132 conn(0x55bd276c0100 msgr2=0x55bd277961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5eb27fc700 0 client.16098 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 shutdown_connections 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 1 -- 172.21.15.26:0/3970519132 wait complete. 2024-05-20T20:58:34.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.464+0000 7f5edbe30b80 20 client.16098 trim_cache size 0 max 16384 2024-05-20T20:58:34.468 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:58:34.468 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:58:34.468 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "797da025-377c-452b-9e99-c40df438bffc", 2024-05-20T20:58:34.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:58:34.468 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5eb80b2900 msgr2=0x7f5eb80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5eb80b2900 0x7f5eb80b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc1026f0 tx=0x7f5eb4009260 comp rx=0 tx=0).stop 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 msgr2=0x7f5ecc104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 0x7f5ecc104140 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d950 tx=0x7f5ebc00dc60 comp rx=0 tx=0).stop 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 shutdown_connections 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5eb80b2900 0x7f5eb80b4dc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ecc102a80 0x7f5ecc104140 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1077d0 0x7f5ecc104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 --2- 172.21.15.26:0/1340338445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ecc103430 0x7f5ecc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 >> 172.21.15.26:0/1340338445 conn(0x7f5ecc0793f0 msgr2=0x7f5ecc06c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 shutdown_connections 2024-05-20T20:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:34.465+0000 7f5ed159a700 1 -- 172.21.15.26:0/1340338445 wait complete. 2024-05-20T20:58:34.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 25777 3.89 MB/sec warmup 40 sec latency 46.505 ms 2024-05-20T20:58:35.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:34 smithi026 ceph-mon[31990]: pgmap v461: 97 pgs: 97 active+clean; 29 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T20:58:35.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:58:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:34 smithi144 ceph-mon[35660]: pgmap v461: 97 pgs: 97 active+clean; 29 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T20:58:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:34 smithi055 ceph-mon[35213]: pgmap v461: 97 pgs: 97 active+clean; 29 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T20:58:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:35.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 26414 3.90 MB/sec warmup 41 sec latency 16.582 ms 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.791+0000 7f1205578640 1 -- 172.21.15.26:0/1357576472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1200105f00 msgr2=0x7f12001082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.791+0000 7f1205578640 1 --2- 172.21.15.26:0/1357576472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1200105f00 0x7f12001082f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009900 tx=0x7f11e802a410 comp rx=0 tx=0).stop 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 -- 172.21.15.26:0/1357576472 shutdown_connections 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 --2- 172.21.15.26:0/1357576472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200108830 0x7f120010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 --2- 172.21.15.26:0/1357576472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1200105f00 0x7f12001082f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 --2- 172.21.15.26:0/1357576472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12001035d0 0x7f12001059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 -- 172.21.15.26:0/1357576472 >> 172.21.15.26:0/1357576472 conn(0x7f12000fcf60 msgr2=0x7f12000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 -- 172.21.15.26:0/1357576472 shutdown_connections 2024-05-20T20:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 -- 172.21.15.26:0/1357576472 wait complete. 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 Processor -- start 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.792+0000 7f1205578640 1 -- start start 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 0x7f1200198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200105f00 0x7f12001990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1200108830 0x7f12001a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f120010e4a0 con 0x7f12001035d0 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f120010e320 con 0x7f1200108830 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f120010e620 con 0x7f1200105f00 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 0x7f1200198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 0x7f1200198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33534/0 (socket says 172.21.15.26:33534) 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 -- 172.21.15.26:0/1518458461 learned_addr learned my addr 172.21.15.26:0/1518458461 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1200108830 0x7f12001a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11f7fff640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200105f00 0x7f12001990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 -- 172.21.15.26:0/1518458461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200105f00 msgr2=0x7f12001990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200105f00 0x7f12001990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 -- 172.21.15.26:0/1518458461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1200108830 msgr2=0x7f12001a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1200108830 0x7f12001a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 -- 172.21.15.26:0/1518458461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12001a07d0 con 0x7f12001035d0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11f7fff640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200105f00 0x7f12001990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11feffd640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 0x7f1200198bb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f11f000e9f0 tx=0x7f11f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11fcff9640 1 -- 172.21.15.26:0/1518458461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f000cd80 con 0x7f12001035d0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11fcff9640 1 -- 172.21.15.26:0/1518458461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f00108c0 con 0x7f12001035d0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12001a0ac0 con 0x7f12001035d0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.793+0000 7f11fcff9640 1 -- 172.21.15.26:0/1518458461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00187c0 con 0x7f12001035d0 2024-05-20T20:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.794+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12001a0fe0 con 0x7f12001035d0 2024-05-20T20:58:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.796+0000 7f11fcff9640 1 -- 172.21.15.26:0/1518458461 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f11f0010430 con 0x7f12001035d0 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 Processor -- start 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 -- start start 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a94fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a954bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a955220 0x559e3a9576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559e3a8c0c30 con 0x559e3a94f6c0 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559e3a8b07d0 con 0x559e3a955220 2024-05-20T20:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f120836a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e3a8b4810 con 0x559e3a9500b0 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a94fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a955220 0x559e3a9576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a94fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33544/0 (socket says 172.21.15.26:33544) 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2844545848 learned_addr learned my addr 172.21.15.26:0/2844545848 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.797+0000 7f11feffd640 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a954bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2844545848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 msgr2=0x559e3a954bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a954bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2844545848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a955220 msgr2=0x559e3a9576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a955220 0x559e3a9576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2844545848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3a5c5210 con 0x559e3a94f6c0 2024-05-20T20:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11fcff9640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11ec0b28a0 0x7f11ec0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a94fae0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f11f80039f0 tx=0x7f11f8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ad2a314cba609 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11feffd640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11ec0b28a0 0x7f11ec0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11fcff9640 1 -- 172.21.15.26:0/1518458461 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f11f00e26c0 con 0x7f12001035d0 2024-05-20T20:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11f57fa640 1 -- 172.21.15.26:0/2844545848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f801a470 con 0x559e3a94f6c0 2024-05-20T20:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11f57fa640 1 -- 172.21.15.26:0/2844545848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f8015070 con 0x559e3a94f6c0 2024-05-20T20:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.798+0000 7f11f57fa640 1 -- 172.21.15.26:0/2844545848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8010770 con 0x559e3a94f6c0 2024-05-20T20:58:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.799+0000 7f11feffd640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11ec0b28a0 0x7f11ec0b4d60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11f00e04d0 tx=0x7f11f0010cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.799+0000 7f120836a740 1 -- 172.21.15.26:0/2844545848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 msgr2=0x559e3a94fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.799+0000 7f120836a740 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a94fae0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f11f80039f0 tx=0x7f11f8003eb0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 -- 172.21.15.26:0/2844545848 shutdown_connections 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a955220 0x559e3a9576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a954bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 --2- 172.21.15.26:0/2844545848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a94fae0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 -- 172.21.15.26:0/2844545848 >> 172.21.15.26:0/2844545848 conn(0x559e3a94ad30 msgr2=0x559e3a94b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 -- 172.21.15.26:0/2844545848 shutdown_connections 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.800+0000 7f120836a740 1 -- 172.21.15.26:0/2844545848 wait complete. 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.801+0000 7f120836a740 1 -- start start 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.801+0000 7f120836a740 1 Processor -- start 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.801+0000 7f120836a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a9597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f120836a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a959d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f120836a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a960d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f120836a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559e3a596790 con 0x559e3a94f6c0 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f120836a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559e3a86c890 con 0x559e3a9500b0 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f120836a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e3a9581d0 con 0x559e3a955220 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f120836a740 10 client.0 _ll_register_callbacks cb 0x559e3a5c4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a9597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a9597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33550/0 (socket says 172.21.15.26:33550) 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a959d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 -- 172.21.15.26:0/903083006 learned_addr learned my addr 172.21.15.26:0/903083006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 -- 172.21.15.26:0/903083006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 msgr2=0x559e3a960d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a960d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 -- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 msgr2=0x559e3a959d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a959d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 -- 172.21.15.26:0/903083006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3a691620 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11f7fff640 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a959d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a9597f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f11f8011e70 tx=0x7f11f8011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc82abc9dbb1db8a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.802+0000 7f11e67fc640 1 -- 172.21.15.26:0/903083006 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f801a470 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f11e67fc640 1 -- 172.21.15.26:0/903083006 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f11f801a610 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f8015070 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f11e67fc640 1 -- 172.21.15.26:0/903083006 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11f801a810 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 10 client.16110 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e3a963b40 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f11e67fc640 1 -- 172.21.15.26:0/903083006 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f8015070 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f11e67fc640 1 -- 172.21.15.26:0/903083006 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11f80239a0 con 0x559e3a94f6c0 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 10 client.16110 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 20 client.16110 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a964f40 0x559e3a967330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 4 client.16110 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3a8c2b80 con 0x559e3a964f40 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.803+0000 7f11f7fff640 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a964f40 0x559e3a967330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.804+0000 7f11e67fc640 10 client.16110 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.804+0000 7f11f7fff640 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a964f40 0x559e3a967330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35d4ed5f8540737 server_cookie=40a71edc351ac882 in_seq=0 out_seq=0 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f11e67fc640 1 -- 172.21.15.26:0/903083006 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e3a8c2b80 con 0x559e3a964f40 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f11e67fc640 10 client.16110 handle_command_reply: tid=0 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 client.16110 shutdown 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 msgr2=0x559e3a9597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a9597f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f11f8011e70 tx=0x7f11f8011650 comp rx=0 tx=0).stop 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 shutdown_connections 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a964f40 0x559e3a967330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a960d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a959d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f11e67fc640 0 client.16110 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 --2- 172.21.15.26:0/903083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a9597f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 >> 172.21.15.26:0/903083006 conn(0x559e3a94ad30 msgr2=0x559e3a949fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 shutdown_connections 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 1 -- 172.21.15.26:0/903083006 wait complete. 2024-05-20T20:58:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.805+0000 7f120836a740 20 client.16110 trim_cache size 0 max 16384 2024-05-20T20:58:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.825+0000 7f120836a740 1 Processor -- start 2024-05-20T20:58:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.825+0000 7f120836a740 1 -- start start 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f120836a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a959c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f120836a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a960d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f120836a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a94e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f120836a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ac8d0 con 0x559e3a94f6c0 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f120836a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12001aa1a0 con 0x559e3a9500b0 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f120836a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12001a7ff0 con 0x559e3a955220 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a959c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a960d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a960d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55066/0 (socket says 172.21.15.26:55066) 2024-05-20T20:58:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 -- 172.21.15.26:0/346680972 learned_addr learned my addr 172.21.15.26:0/346680972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a959c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33566/0 (socket says 172.21.15.26:33566) 2024-05-20T20:58:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a94e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 -- 172.21.15.26:0/346680972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 msgr2=0x559e3a94e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a94e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 -- 172.21.15.26:0/346680972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 msgr2=0x559e3a959c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a959c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 -- 172.21.15.26:0/346680972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3a9676a0 con 0x559e3a9500b0 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a94e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.826+0000 7f11f7fff640 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a960d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f11e8000c00 tx=0x7f11e80058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22fa1cb918ff9fa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.827+0000 7f11f4ff9640 1 -- 172.21.15.26:0/346680972 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e804b070 con 0x559e3a9500b0 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.827+0000 7f11f4ff9640 1 -- 172.21.15.26:0/346680972 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e8031d50 con 0x559e3a9500b0 2024-05-20T20:58:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.827+0000 7f11f4ff9640 1 -- 172.21.15.26:0/346680972 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8035040 con 0x559e3a9500b0 2024-05-20T20:58:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 -- 172.21.15.26:0/346680972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 msgr2=0x559e3a960d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a960d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f11e8000c00 tx=0x7f11e80058b0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 -- 172.21.15.26:0/346680972 shutdown_connections 2024-05-20T20:58:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a955220 0x559e3a94e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a9500b0 0x559e3a960d30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 --2- 172.21.15.26:0/346680972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a94f6c0 0x559e3a959c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 -- 172.21.15.26:0/346680972 >> 172.21.15.26:0/346680972 conn(0x559e3a94ad30 msgr2=0x559e3a94da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 -- 172.21.15.26:0/346680972 shutdown_connections 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 -- 172.21.15.26:0/346680972 wait complete. 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 -- start start 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 Processor -- start 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.828+0000 7f120836a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a94f6c0 0x559e3a95e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a95e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 0x559e3a95f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12001a8660 con 0x559e3a955220 2024-05-20T20:58:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12001a9ad0 con 0x559e3a94f6c0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12001a8d30 con 0x559e3a9500b0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 10 client.0 _ll_register_callbacks cb 0x7f11f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 0x559e3a95f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 0x559e3a95f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33568/0 (socket says 172.21.15.26:33568) 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2127928823 learned_addr learned my addr 172.21.15.26:0/2127928823 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11f7fff640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a95e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a94f6c0 0x559e3a95e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2127928823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 msgr2=0x559e3a95e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a95e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a94f6c0 msgr2=0x559e3a95e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a94f6c0 0x559e3a95e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 -- 172.21.15.26:0/2127928823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3a95fd80 con 0x559e3a955220 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11ff7fe640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 0x559e3a95f790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f11f8000f80 tx=0x7f11f8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3c188735ef8bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11e67fc640 1 -- 172.21.15.26:0/2127928823 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f801abb0 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11e67fc640 1 -- 172.21.15.26:0/2127928823 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f11f801ad50 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e8031d50 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.829+0000 7f11e67fc640 1 -- 172.21.15.26:0/2127928823 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11f8009c50 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 10 client.16116 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e3a961000 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f11e67fc640 1 -- 172.21.15.26:0/2127928823 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e8031d50 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f11e67fc640 1 -- 172.21.15.26:0/2127928823 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11f8023470 con 0x559e3a955220 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 20 client.16116 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a9d01c0 0x559e3a9d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 4 client.16116 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:58:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e3a8bc360 con 0x559e3a9d01c0 2024-05-20T20:58:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f11f7fff640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a9d01c0 0x559e3a9d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f11e67fc640 10 client.16116 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.830+0000 7f11f7fff640 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a9d01c0 0x559e3a9d25e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be16b19e8f7a1c4 server_cookie=d7f26f8a7b2b8b88 in_seq=0 out_seq=0 2024-05-20T20:58:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.879+0000 7f11e67fc640 1 -- 172.21.15.26:0/2127928823 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e3a8bc360 con 0x559e3a9d01c0 2024-05-20T20:58:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.879+0000 7f11e67fc640 10 client.16116 handle_command_reply: tid=0 2024-05-20T20:58:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.879+0000 7f120836a740 1 client.16116 shutdown 2024-05-20T20:58:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 msgr2=0x559e3a95f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 0x559e3a95f790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f11f8000f80 tx=0x7f11f8003eb0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 shutdown_connections 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e3a9d01c0 0x559e3a9d25e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f11e67fc640 0 client.16116 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e3a955220 0x559e3a95f790 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3a9500b0 0x559e3a95e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 --2- 172.21.15.26:0/2127928823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e3a94f6c0 0x559e3a95e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 >> 172.21.15.26:0/2127928823 conn(0x559e3a94ad30 msgr2=0x559e3a94e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 shutdown_connections 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 1 -- 172.21.15.26:0/2127928823 wait complete. 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f120836a740 20 client.16116 trim_cache size 0 max 16384 2024-05-20T20:58:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:58:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:58:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:58:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11ec0b28a0 msgr2=0x7f11ec0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f1205578640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11ec0b28a0 0x7f11ec0b4d60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11f00e04d0 tx=0x7f11f0010cc0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 msgr2=0x7f1200198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.880+0000 7f1205578640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 0x7f1200198bb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f11f000e9f0 tx=0x7f11f000eeb0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 shutdown_connections 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11ec0b28a0 0x7f11ec0b4d60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1200108830 0x7f12001a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1200105f00 0x7f12001990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 --2- 172.21.15.26:0/1518458461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12001035d0 0x7f1200198bb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 >> 172.21.15.26:0/1518458461 conn(0x7f12000fcf60 msgr2=0x7f1200103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 shutdown_connections 2024-05-20T20:58:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:35.881+0000 7f1205578640 1 -- 172.21.15.26:0/1518458461 wait complete. 2024-05-20T20:58:36.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:35 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:58:36.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:35 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:36.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:35 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:58:36.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:35 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:35 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:58:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:36 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:36 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:58:36.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:36 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:36.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:36 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:58:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:36 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:36 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:58:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:36 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:36.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 27183 3.85 MB/sec warmup 42 sec latency 29.938 ms 2024-05-20T20:58:37.258 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:37 smithi026 ceph-mon[31990]: pgmap v462: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T20:58:37.259 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:37 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:58:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:37 smithi144 ceph-mon[35660]: pgmap v462: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T20:58:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:37 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.376+0000 7f390c091700 1 -- 172.21.15.26:0/694379284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 msgr2=0x7f3904109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.376+0000 7f390c091700 1 --2- 172.21.15.26:0/694379284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904109900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f39000099c0 tx=0x7f3900009cd0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 -- 172.21.15.26:0/694379284 shutdown_connections 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 --2- 172.21.15.26:0/694379284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904109900 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 --2- 172.21.15.26:0/694379284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39041024f0 0x7f39041103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 --2- 172.21.15.26:0/694379284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 0x7f3904101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 -- 172.21.15.26:0/694379284 >> 172.21.15.26:0/694379284 conn(0x7f3904078ab0 msgr2=0x7f39040ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 -- 172.21.15.26:0/694379284 shutdown_connections 2024-05-20T20:58:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.377+0000 7f390c091700 1 -- 172.21.15.26:0/694379284 wait complete. 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 Processor -- start 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 -- start start 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 0x7f3904102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39041024f0 0x7f3904103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39041135b0 con 0x7f3904109520 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3904113430 con 0x7f3904101b30 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390c091700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3904113730 con 0x7f39041024f0 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f3909e2d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 0x7f3904102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f3909e2d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 0x7f3904102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55100/0 (socket says 172.21.15.26:55100) 2024-05-20T20:58:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f3909e2d700 1 -- 172.21.15.26:0/3939236354 learned_addr learned my addr 172.21.15.26:0/3939236354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.378+0000 7f390962c700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39041024f0 0x7f3904103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390a62e700 1 -- 172.21.15.26:0/3939236354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39041024f0 msgr2=0x7f3904103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390a62e700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39041024f0 0x7f3904103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390a62e700 1 -- 172.21.15.26:0/3939236354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 msgr2=0x7f3904102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390a62e700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 0x7f3904102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390a62e700 1 -- 172.21.15.26:0/3939236354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39041a8010 con 0x7f3904109520 2024-05-20T20:58:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390a62e700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904103790 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007f10 tx=0x7f38fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f38faffd700 1 -- 172.21.15.26:0/3939236354 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc004db0 con 0x7f3904109520 2024-05-20T20:58:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39041a8300 con 0x7f3904109520 2024-05-20T20:58:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f38faffd700 1 -- 172.21.15.26:0/3939236354 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38fc009e20 con 0x7f3904109520 2024-05-20T20:58:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f38faffd700 1 -- 172.21.15.26:0/3939236354 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc015400 con 0x7f3904109520 2024-05-20T20:58:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.379+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041a8840 con 0x7f3904109520 2024-05-20T20:58:37.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.382+0000 7f38faffd700 1 -- 172.21.15.26:0/3939236354 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f38fc0155a0 con 0x7f3904109520 2024-05-20T20:58:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.383+0000 7f38faffd700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38f00b2880 0x7f38f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.383+0000 7f3909e2d700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38f00b2880 0x7f38f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.383+0000 7f38faffd700 1 -- 172.21.15.26:0/3939236354 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f38fc0e1b70 con 0x7f3904109520 2024-05-20T20:58:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.383+0000 7f3916927b80 1 Processor -- start 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 -- start start 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3909e2d700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38f00b2880 0x7f38f00b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3900009990 tx=0x7f3900005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b98983c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98987940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557b988e2c90 con 0x557b98987f60 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557b988d2830 con 0x557b98984210 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3916927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557b988d6870 con 0x557b98983860 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390962c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98987940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f3909e2d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b98983c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989883a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33606/0 (socket says 172.21.15.26:33606) 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390a62e700 1 -- 172.21.15.26:0/1288097478 learned_addr learned my addr 172.21.15.26:0/1288097478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390962c700 1 -- 172.21.15.26:0/1288097478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 msgr2=0x557b98983c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390962c700 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b98983c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390962c700 1 -- 172.21.15.26:0/1288097478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 msgr2=0x557b989883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390962c700 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989883a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390962c700 1 -- 172.21.15.26:0/1288097478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b988228d0 con 0x557b98984210 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.384+0000 7f390a62e700 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.385+0000 7f390962c700 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98987940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39041047a0 tx=0x7f38f4009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b648447badff39 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.385+0000 7f38effff700 1 -- 172.21.15.26:0/1288097478 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011c80 con 0x557b98984210 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.385+0000 7f38effff700 1 -- 172.21.15.26:0/1288097478 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38f4011e20 con 0x557b98984210 2024-05-20T20:58:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.385+0000 7f38effff700 1 -- 172.21.15.26:0/1288097478 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4005280 con 0x557b98984210 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 -- 172.21.15.26:0/1288097478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 msgr2=0x557b98987940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98987940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39041047a0 tx=0x7f38f4009f60 comp rx=0 tx=0).stop 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 -- 172.21.15.26:0/1288097478 shutdown_connections 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b98983c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 --2- 172.21.15.26:0/1288097478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98987940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 -- 172.21.15.26:0/1288097478 >> 172.21.15.26:0/1288097478 conn(0x557b988b1080 msgr2=0x557b9897de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 -- 172.21.15.26:0/1288097478 shutdown_connections 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 -- 172.21.15.26:0/1288097478 wait complete. 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.386+0000 7f3916927b80 1 -- start start 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 Processor -- start 2024-05-20T20:58:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9898c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b9898cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b9898cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557b987d8fa0 con 0x557b98987f60 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b9898cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33620/0 (socket says 172.21.15.26:33620) 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f390a62e700 1 -- 172.21.15.26:0/2801017072 learned_addr learned my addr 172.21.15.26:0/2801017072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557b988747c0 con 0x557b98983860 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f390962c700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9898c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557b98990120 con 0x557b98984210 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.387+0000 7f3909e2d700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f3916927b80 10 client.0 _ll_register_callbacks cb 0x557b9886d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f390962c700 1 -- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 msgr2=0x557b9898f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f390962c700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f390962c700 1 -- 172.21.15.26:0/2801017072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 msgr2=0x557b9898cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f390962c700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b9898cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f390962c700 1 -- 172.21.15.26:0/2801017072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b98871690 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f3909e2d700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f390962c700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9898c690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38f4019000 tx=0x7f38f4004890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d11f35856c121b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f38ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f38ed7fa700 1 -- 172.21.15.26:0/2801017072 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4004b60 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f4011e20 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f38ed7fa700 1 -- 172.21.15.26:0/2801017072 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f38f4004260 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f38ed7fa700 1 -- 172.21.15.26:0/2801017072 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38f4004460 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f3916927b80 10 client.25624 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.388+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b9899a7c0 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f38ed7fa700 1 -- 172.21.15.26:0/2801017072 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38f4011e20 con 0x557b98984210 2024-05-20T20:58:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f38ed7fa700 1 -- 172.21.15.26:0/2801017072 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f38f4018070 con 0x557b98984210 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3916927b80 10 client.25624 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3916927b80 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3916927b80 20 client.25624 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3916927b80 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b9899f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3916927b80 4 client.25624 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b9894f9e0 con 0x557b9899cc90 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.389+0000 7f3909e2d700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b9899f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f38ed7fa700 10 client.25624 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3909e2d700 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b9899f080 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa4496c571386d0 server_cookie=62352e849a2eecc8 in_seq=0 out_seq=0 2024-05-20T20:58:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f38ed7fa700 1 -- 172.21.15.26:0/2801017072 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557b9894f9e0 con 0x557b9899cc90 2024-05-20T20:58:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f38ed7fa700 10 client.25624 handle_command_reply: tid=0 2024-05-20T20:58:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3916927b80 1 client.25624 shutdown 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 msgr2=0x557b9898c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3916927b80 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9898c690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38f4019000 tx=0x7f38f4004890 comp rx=0 tx=0).stop 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 shutdown_connections 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3916927b80 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b9899f080 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.390+0000 7f3916927b80 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b9898cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f38ed7fa700 0 client.25624 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f3916927b80 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f3916927b80 1 --2- 172.21.15.26:0/2801017072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9898c690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 >> 172.21.15.26:0/2801017072 conn(0x557b988b1080 msgr2=0x557b9897d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 shutdown_connections 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f3916927b80 1 -- 172.21.15.26:0/2801017072 wait complete. 2024-05-20T20:58:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.391+0000 7f3916927b80 20 client.25624 trim_cache size 0 max 16384 2024-05-20T20:58:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.414+0000 7f3916927b80 1 Processor -- start 2024-05-20T20:58:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.414+0000 7f3916927b80 1 -- start start 2024-05-20T20:58:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3916927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3916927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9897dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3916927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3916927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39041b2820 con 0x557b98987f60 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3916927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f39041b00b0 con 0x557b98983860 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3916927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39041adf00 con 0x557b98984210 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33622/0 (socket says 172.21.15.26:33622) 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 -- 172.21.15.26:0/3497642387 learned_addr learned my addr 172.21.15.26:0/3497642387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f3909e2d700 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390962c700 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9897dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 -- 172.21.15.26:0/3497642387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 msgr2=0x557b9897dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9897dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 -- 172.21.15.26:0/3497642387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 msgr2=0x557b9898e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 -- 172.21.15.26:0/3497642387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b9897ed40 con 0x557b98987f60 2024-05-20T20:58:37.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f390a62e700 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989a46e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000f80 tx=0x7f38fc005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a831cd009a4f3d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:37.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.415+0000 7f38ef7fe700 1 -- 172.21.15.26:0/3497642387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc06d020 con 0x557b98987f60 2024-05-20T20:58:37.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f38ef7fe700 1 -- 172.21.15.26:0/3497642387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38fc0b1f90 con 0x557b98987f60 2024-05-20T20:58:37.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f38ef7fe700 1 -- 172.21.15.26:0/3497642387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0a9390 con 0x557b98987f60 2024-05-20T20:58:37.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 -- 172.21.15.26:0/3497642387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 msgr2=0x557b989a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989a46e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000f80 tx=0x7f38fc005b60 comp rx=0 tx=0).stop 2024-05-20T20:58:37.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 -- 172.21.15.26:0/3497642387 shutdown_connections 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989a46e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98983860 0x557b9898e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 --2- 172.21.15.26:0/3497642387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98984210 0x557b9897dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 -- 172.21.15.26:0/3497642387 >> 172.21.15.26:0/3497642387 conn(0x557b988b1080 msgr2=0x557b9898dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.416+0000 7f3916927b80 1 -- 172.21.15.26:0/3497642387 shutdown_connections 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 -- 172.21.15.26:0/3497642387 wait complete. 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 -- start start 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 Processor -- start 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b989953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98995930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ae570 con 0x557b98987f60 2024-05-20T20:58:37.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f39041af9e0 con 0x557b98984210 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39041aec40 con 0x557b98983860 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f3916927b80 10 client.0 _ll_register_callbacks cb 0x557b9886d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.417+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33634/0 (socket says 172.21.15.26:33634) 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 -- 172.21.15.26:0/1422611162 learned_addr learned my addr 172.21.15.26:0/1422611162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390962c700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98995930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f3909e2d700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b989953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 -- 172.21.15.26:0/1422611162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 msgr2=0x557b989953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b989953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 -- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 msgr2=0x557b98995930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98995930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 -- 172.21.15.26:0/1422611162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b989940a0 con 0x557b98987f60 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f390a62e700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989939a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0df980 tx=0x7f38fc0a92b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce72e59c79f62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f38ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f38ed7fa700 1 -- 172.21.15.26:0/1422611162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0c8060 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f38ed7fa700 1 -- 172.21.15.26:0/1422611162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f38fc0a9590 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.418+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38fc0b1f90 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f38ed7fa700 1 -- 172.21.15.26:0/1422611162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38fc06d070 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 10 client.16134 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b98a08170 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f38ed7fa700 1 -- 172.21.15.26:0/1422611162 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38fc0b1f90 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f38ed7fa700 1 -- 172.21.15.26:0/1422611162 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f38fc0a4aa0 con 0x557b98987f60 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 20 client.16134 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b98a0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:37.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 4 client.16134 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:58:37.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b988e4be0 con 0x557b9899cc90 2024-05-20T20:58:37.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.419+0000 7f3909e2d700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b98a0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f3909e2d700 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b98a0a8a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e7b423a42ad67 server_cookie=a70cf3f5ea981fa7 in_seq=0 out_seq=0 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f38ed7fa700 10 client.16134 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f38ed7fa700 1 -- 172.21.15.26:0/1422611162 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557b988e4be0 con 0x557b9899cc90 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f38ed7fa700 10 client.16134 handle_command_reply: tid=0 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f3916927b80 1 client.16134 shutdown 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 msgr2=0x557b989939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f3916927b80 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989939a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0df980 tx=0x7f38fc0a92b0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 shutdown_connections 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.420+0000 7f3916927b80 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557b9899cc90 0x557b98a0a8a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b98987f60 0x557b989939a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b98983860 0x557b989953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f38ed7fa700 0 client.16134 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 1 --2- 172.21.15.26:0/1422611162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557b98984210 0x557b98995930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 >> 172.21.15.26:0/1422611162 conn(0x557b988b1080 msgr2=0x557b9898e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 shutdown_connections 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 1 -- 172.21.15.26:0/1422611162 wait complete. 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f3916927b80 20 client.16134 trim_cache size 0 max 16384 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:58:37.424 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:58:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38f00b2880 msgr2=0x7f38f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38f00b2880 0x7f38f00b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3900009990 tx=0x7f3900005960 comp rx=0 tx=0).stop 2024-05-20T20:58:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 msgr2=0x7f3904103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904103790 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007f10 tx=0x7f38fc00d3b0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 shutdown_connections 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38f00b2880 0x7f38f00b4d40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3904109520 0x7f3904103790 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3904101b30 0x7f3904102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 --2- 172.21.15.26:0/3939236354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39041024f0 0x7f3904103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.421+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 >> 172.21.15.26:0/3939236354 conn(0x7f3904078ab0 msgr2=0x7f3904100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.422+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 shutdown_connections 2024-05-20T20:58:37.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:37.422+0000 7f390c091700 1 -- 172.21.15.26:0/3939236354 wait complete. 2024-05-20T20:58:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:37 smithi055 ceph-mon[35213]: pgmap v462: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T20:58:37.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:37 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:58:37.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 27902 3.90 MB/sec warmup 43 sec latency 27.490 ms 2024-05-20T20:58:38.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:58:38.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:58:38.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 28534 3.92 MB/sec warmup 44 sec latency 18.283 ms 2024-05-20T20:58:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:39 smithi144 ceph-mon[35660]: pgmap v463: 97 pgs: 97 active+clean; 25 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-05-20T20:58:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:39 smithi055 ceph-mon[35213]: pgmap v463: 97 pgs: 97 active+clean; 25 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-05-20T20:58:39.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:39 smithi026 ceph-mon[31990]: pgmap v463: 97 pgs: 97 active+clean; 25 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-05-20T20:58:39.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 28942 3.87 MB/sec warmup 45 sec latency 17.831 ms 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 -- 172.21.15.26:0/4237672925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0108520 msgr2=0x7f6ea0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 --2- 172.21.15.26:0/4237672925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0108520 0x7f6ea0108900 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009990 tx=0x7f6e9c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 -- 172.21.15.26:0/4237672925 shutdown_connections 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 --2- 172.21.15.26:0/4237672925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0108520 0x7f6ea0108900 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 --2- 172.21.15.26:0/4237672925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea0102e00 0x7f6ea010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 --2- 172.21.15.26:0/4237672925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0102440 0x7f6ea01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.427+0000 7f6ea7523700 1 -- 172.21.15.26:0/4237672925 >> 172.21.15.26:0/4237672925 conn(0x7f6ea00793f0 msgr2=0x7f6ea0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.428+0000 7f6ea7523700 1 -- 172.21.15.26:0/4237672925 shutdown_connections 2024-05-20T20:58:40.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.428+0000 7f6ea7523700 1 -- 172.21.15.26:0/4237672925 wait complete. 2024-05-20T20:58:40.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.428+0000 7f6ea7523700 1 Processor -- start 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.428+0000 7f6ea7523700 1 -- start start 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea7523700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea0102440 0x7f6ea019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea7523700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 0x7f6ea019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea7523700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0108520 0x7f6ea01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea7523700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea01124d0 con 0x7f6ea0102e00 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea7523700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea0112350 con 0x7f6ea0108520 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea7523700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea0112650 con 0x7f6ea0102440 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea4abe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 0x7f6ea019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea4abe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 0x7f6ea019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43382/0 (socket says 172.21.15.26:43382) 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea4abe700 1 -- 172.21.15.26:0/3832249021 learned_addr learned my addr 172.21.15.26:0/3832249021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea5ac0700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0108520 0x7f6ea01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea52bf700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea0102440 0x7f6ea019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea5ac0700 1 -- 172.21.15.26:0/3832249021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea0102440 msgr2=0x7f6ea019e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea5ac0700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea0102440 0x7f6ea019e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea5ac0700 1 -- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 msgr2=0x7f6ea019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea5ac0700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 0x7f6ea019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea5ac0700 1 -- 172.21.15.26:0/3832249021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea0199010 con 0x7f6ea0108520 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.429+0000 7f6ea4abe700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 0x7f6ea019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.430+0000 7f6ea5ac0700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0108520 0x7f6ea01988b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800efc0 tx=0x7f6e9800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.430+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e980092e0 con 0x7f6ea0108520 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.430+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e98004bb0 con 0x7f6ea0108520 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.430+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea01992a0 con 0x7f6ea0108520 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.430+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98004020 con 0x7f6ea0108520 2024-05-20T20:58:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.430+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea006b220 con 0x7f6ea0108520 2024-05-20T20:58:40.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.431+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea006a230 con 0x7f6ea0108520 2024-05-20T20:58:40.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.433+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6e9801b030 con 0x7f6ea0108520 2024-05-20T20:58:40.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.434+0000 7f6e967fc700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:40.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.434+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f6e980e1d10 con 0x7f6ea0108520 2024-05-20T20:58:40.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.434+0000 7f6ea52bf700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.435+0000 7f6ea52bf700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00b2e0 tx=0x7f6e9c009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.440+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e98022020 con 0x7f6ea0108520 2024-05-20T20:58:40.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 29538 3.83 MB/sec warmup 46 sec latency 67.367 ms 2024-05-20T20:58:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.753+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ea006bc50 con 0x7f6ea0108520 2024-05-20T20:58:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.753+0000 7f6e967fc700 1 -- 172.21.15.26:0/3832249021 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f6e98010040 con 0x7f6ea0108520 2024-05-20T20:58:40.753 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:58:40.753 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:58:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e8c0b2900 msgr2=0x7f6e8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00b2e0 tx=0x7f6e9c009650 comp rx=0 tx=0).stop 2024-05-20T20:58:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0108520 msgr2=0x7f6ea01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0108520 0x7f6ea01988b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800efc0 tx=0x7f6e9800c3a0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 shutdown_connections 2024-05-20T20:58:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea0108520 0x7f6ea01988b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea0102440 0x7f6ea019e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 --2- 172.21.15.26:0/3832249021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0102e00 0x7f6ea019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 >> 172.21.15.26:0/3832249021 conn(0x7f6ea00793f0 msgr2=0x7f6ea01013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 shutdown_connections 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:40.756+0000 7f6ea7523700 1 -- 172.21.15.26:0/3832249021 wait complete. 2024-05-20T20:58:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:58:41.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:41 smithi026 ceph-mon[31990]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-05-20T20:58:41.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:41 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3832249021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:58:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:41 smithi144 ceph-mon[35660]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-05-20T20:58:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:41 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3832249021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:58:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:41 smithi055 ceph-mon[35213]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-05-20T20:58:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:41 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3832249021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:58:41.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:58:41.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 30249 3.82 MB/sec warmup 47 sec latency 17.686 ms 2024-05-20T20:58:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:42.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 30860 3.77 MB/sec warmup 48 sec latency 18.507 ms 2024-05-20T20:58:43.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:43 smithi026 ceph-mon[31990]: pgmap v465: 97 pgs: 97 active+clean; 26 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T20:58:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:43 smithi144 ceph-mon[35660]: pgmap v465: 97 pgs: 97 active+clean; 26 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T20:58:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:43 smithi055 ceph-mon[35213]: pgmap v465: 97 pgs: 97 active+clean; 26 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.513+0000 7fa940872700 1 -- 172.21.15.26:0/2744168137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c10b750 msgr2=0x7fa93c10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.513+0000 7fa940872700 1 --2- 172.21.15.26:0/2744168137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c10b750 0x7fa93c10bb30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 -- 172.21.15.26:0/2744168137 shutdown_connections 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 --2- 172.21.15.26:0/2744168137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c10b750 0x7fa93c10bb30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 --2- 172.21.15.26:0/2744168137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c106110 0x7fa93c112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 --2- 172.21.15.26:0/2744168137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa93c105750 0x7fa93c105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 -- 172.21.15.26:0/2744168137 >> 172.21.15.26:0/2744168137 conn(0x7fa93c101240 msgr2=0x7fa93c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 -- 172.21.15.26:0/2744168137 shutdown_connections 2024-05-20T20:58:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.514+0000 7fa940872700 1 -- 172.21.15.26:0/2744168137 wait complete. 2024-05-20T20:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 Processor -- start 2024-05-20T20:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 -- start start 2024-05-20T20:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c105750 0x7fa93c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c106110 0x7fa93c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa93c10b750 0x7fa93c19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1157e0 con 0x7fa93c106110 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c115660 con 0x7fa93c105750 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa940872700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c115960 con 0x7fa93c10b750 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa939d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c106110 0x7fa93c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c105750 0x7fa93c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa939d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c106110 0x7fa93c1a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43398/0 (socket says 172.21.15.26:43398) 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa939d9b700 1 -- 172.21.15.26:0/3825950745 learned_addr learned my addr 172.21.15.26:0/3825950745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa93c10b750 0x7fa93c19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93a59c700 1 -- 172.21.15.26:0/3825950745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa93c10b750 msgr2=0x7fa93c19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93a59c700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa93c10b750 0x7fa93c19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93a59c700 1 -- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c106110 msgr2=0x7fa93c1a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93a59c700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c106110 0x7fa93c1a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.515+0000 7fa93a59c700 1 -- 172.21.15.26:0/3825950745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c19c200 con 0x7fa93c105750 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.516+0000 7fa93a59c700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c105750 0x7fa93c1a18e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.516+0000 7fa92b7fe700 1 -- 172.21.15.26:0/3825950745 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c026070 con 0x7fa93c105750 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.516+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c19c490 con 0x7fa93c105750 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.516+0000 7fa92b7fe700 1 -- 172.21.15.26:0/3825950745 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c021410 con 0x7fa93c105750 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.516+0000 7fa92b7fe700 1 -- 172.21.15.26:0/3825950745 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c017650 con 0x7fa93c105750 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.516+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c1a87b0 con 0x7fa93c105750 2024-05-20T20:58:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.519+0000 7fa92b7fe700 1 -- 172.21.15.26:0/3825950745 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa92c029070 con 0x7fa93c105750 2024-05-20T20:58:43.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.519+0000 7fa92b7fe700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9240b2880 0x7fa9240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.520+0000 7fa939d9b700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9240b2880 0x7fa9240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.520+0000 7fa92b7fe700 1 -- 172.21.15.26:0/3825950745 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa92c0e3000 con 0x7fa93c105750 2024-05-20T20:58:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.520+0000 7fa94b108b80 1 Processor -- start 2024-05-20T20:58:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.520+0000 7fa939d9b700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9240b2880 0x7fa9240b4d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa93c1038c0 tx=0x7fa930005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.520+0000 7fa94b108b80 1 -- start start 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69e8a0 0x5567aa69ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 0x5567aa6ab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa94b108b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567aa5fcd70 con 0x5567aa69f260 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa94b108b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5567aa5ec910 con 0x5567aa69e8a0 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa94b108b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567aa5f0950 con 0x5567aa6a4320 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa93ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 0x5567aa6ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa93ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 0x5567aa6ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43410/0 (socket says 172.21.15.26:43410) 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa93ad9d700 1 -- 172.21.15.26:0/1889224792 learned_addr learned my addr 172.21.15.26:0/1889224792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69e8a0 0x5567aa69ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa93a59c700 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 -- 172.21.15.26:0/1889224792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 msgr2=0x5567aa6a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 -- 172.21.15.26:0/1889224792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 msgr2=0x5567aa6ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 0x5567aa6ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 -- 172.21.15.26:0/1889224792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aa595fb0 con 0x5567aa69e8a0 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa93a59c700 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 0x5567aa6ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.521+0000 7fa939d9b700 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69e8a0 0x5567aa69ed20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa93000ff60 tx=0x7fa930012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8f1fee3ab65d957 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa928ff9700 1 -- 172.21.15.26:0/1889224792 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930016070 con 0x5567aa69e8a0 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa928ff9700 1 -- 172.21.15.26:0/1889224792 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa93001b420 con 0x5567aa69e8a0 2024-05-20T20:58:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa928ff9700 1 -- 172.21.15.26:0/1889224792 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93001c680 con 0x5567aa69e8a0 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 -- 172.21.15.26:0/1889224792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69e8a0 msgr2=0x5567aa69ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69e8a0 0x5567aa69ed20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa93000ff60 tx=0x7fa930012000 comp rx=0 tx=0).stop 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 -- 172.21.15.26:0/1889224792 shutdown_connections 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69f260 0x5567aa6ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1889224792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69e8a0 0x5567aa69ed20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.522+0000 7fa94b108b80 1 -- 172.21.15.26:0/1889224792 >> 172.21.15.26:0/1889224792 conn(0x5567aa5ca240 msgr2=0x5567aa6993e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 -- 172.21.15.26:0/1889224792 shutdown_connections 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 -- 172.21.15.26:0/1889224792 wait complete. 2024-05-20T20:58:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 -- start start 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 Processor -- start 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6aae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6a81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567aa4f2fa0 con 0x5567aa69e8a0 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5567aa554da0 con 0x5567aa69f260 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567aa6ab6e0 con 0x5567aa6a4320 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.523+0000 7fa94b108b80 10 client.0 _ll_register_callbacks cb 0x5567aa587950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa93a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6a81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6a81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53474/0 (socket says 172.21.15.26:53474) 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 -- 172.21.15.26:0/170789637 learned_addr learned my addr 172.21.15.26:0/170789637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 -- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 msgr2=0x5567aa6a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 -- 172.21.15.26:0/170789637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 msgr2=0x5567aa6aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 -- 172.21.15.26:0/170789637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567aa53c8d0 con 0x5567aa69f260 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa93a59c700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa939d9b700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6a81e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa93001bd30 tx=0x7fa93000ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f898f943a3c9aacb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa921ffb700 1 -- 172.21.15.26:0/170789637 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930010cd0 con 0x5567aa69f260 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa921ffb700 1 -- 172.21.15.26:0/170789637 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa930005040 con 0x5567aa69f260 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.524+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa93001b420 con 0x5567aa69f260 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa921ffb700 1 -- 172.21.15.26:0/170789637 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa930005280 con 0x5567aa69f260 2024-05-20T20:58:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 10 client.25706 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567aa5a6760 con 0x5567aa69f260 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa921ffb700 1 -- 172.21.15.26:0/170789637 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa93001b420 con 0x5567aa69f260 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa921ffb700 1 -- 172.21.15.26:0/170789637 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa930037020 con 0x5567aa69f260 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 10 client.25706 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 10 client.25706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 20 client.25706 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b8230 0x5567aa6ba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 4 client.25706 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567aa5fecc0 con 0x5567aa6b8230 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.525+0000 7fa93a59c700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b8230 0x5567aa6ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.526+0000 7fa93a59c700 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b8230 0x5567aa6ba620 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4ededb22d6c25f1 server_cookie=d8df09e7d28e0402 in_seq=0 out_seq=0 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.526+0000 7fa921ffb700 10 client.25706 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa921ffb700 1 -- 172.21.15.26:0/170789637 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5567aa5fecc0 con 0x5567aa6b8230 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa921ffb700 10 client.25706 handle_command_reply: tid=0 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 client.25706 shutdown 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 msgr2=0x5567aa6a81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6a81e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa93001bd30 tx=0x7fa93000ab90 comp rx=0 tx=0).stop 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 shutdown_connections 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b8230 0x5567aa6ba620 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa6a4320 0x5567aa6a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 --2- 172.21.15.26:0/170789637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6a81e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa921ffb700 0 client.25706 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 >> 172.21.15.26:0/170789637 conn(0x5567aa5ca240 msgr2=0x5567aa698af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 shutdown_connections 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 1 -- 172.21.15.26:0/170789637 wait complete. 2024-05-20T20:58:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.527+0000 7fa94b108b80 20 client.25706 trim_cache size 0 max 16384 2024-05-20T20:58:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 Processor -- start 2024-05-20T20:58:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 -- start start 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69f260 0x5567aa699270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa6a4320 0x5567aa6a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1b2580 con 0x5567aa69e8a0 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1afe10 con 0x5567aa6a4320 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa94b108b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1adc60 con 0x5567aa69f260 2024-05-20T20:58:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa93a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa93a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43434/0 (socket says 172.21.15.26:43434) 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa93a59c700 1 -- 172.21.15.26:0/2537782629 learned_addr learned my addr 172.21.15.26:0/2537782629 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.546+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa6a4320 0x5567aa6a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa939d9b700 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69f260 0x5567aa699270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa93a59c700 1 -- 172.21.15.26:0/2537782629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69f260 msgr2=0x5567aa699270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa93a59c700 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69f260 0x5567aa699270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa93a59c700 1 -- 172.21.15.26:0/2537782629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa6a4320 msgr2=0x5567aa6a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa93a59c700 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa6a4320 0x5567aa6a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa93a59c700 1 -- 172.21.15.26:0/2537782629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aa6a3f40 con 0x5567aa69e8a0 2024-05-20T20:58:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa93a59c700 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6b4770 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0e0b60 tx=0x7fa92c0e0a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b824b2d604fe62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa923fff700 1 -- 172.21.15.26:0/2537782629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0c4070 con 0x5567aa69e8a0 2024-05-20T20:58:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa923fff700 1 -- 172.21.15.26:0/2537782629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c0aadf0 con 0x5567aa69e8a0 2024-05-20T20:58:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.547+0000 7fa923fff700 1 -- 172.21.15.26:0/2537782629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c092310 con 0x5567aa69e8a0 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 -- 172.21.15.26:0/2537782629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 msgr2=0x5567aa6b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6b4770 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0e0b60 tx=0x7fa92c0e0a10 comp rx=0 tx=0).stop 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 -- 172.21.15.26:0/2537782629 shutdown_connections 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa69e8a0 0x5567aa6b4770 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa6a4320 0x5567aa6a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 --2- 172.21.15.26:0/2537782629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69f260 0x5567aa699270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 -- 172.21.15.26:0/2537782629 >> 172.21.15.26:0/2537782629 conn(0x5567aa5ca240 msgr2=0x5567aa6a6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 -- 172.21.15.26:0/2537782629 shutdown_connections 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.548+0000 7fa94b108b80 1 -- 172.21.15.26:0/2537782629 wait complete. 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 -- start start 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 Processor -- start 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69e8a0 0x5567aa6c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 0x5567aa6bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1ae2d0 con 0x5567aa6a4320 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1af740 con 0x5567aa69f260 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1ae9a0 con 0x5567aa69e8a0 2024-05-20T20:58:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa94b108b80 10 client.0 _ll_register_callbacks cb 0x7fa934009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa93ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 0x5567aa6bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa93ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 0x5567aa6bf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43440/0 (socket says 172.21.15.26:43440) 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa939d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa93ad9d700 1 -- 172.21.15.26:0/1101383253 learned_addr learned my addr 172.21.15.26:0/1101383253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.549+0000 7fa93a59c700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69e8a0 0x5567aa6c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa93ad9d700 1 -- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69e8a0 msgr2=0x5567aa6c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69e8a0 0x5567aa6c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa93ad9d700 1 -- 172.21.15.26:0/1101383253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 msgr2=0x5567aa6bedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6bedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa93ad9d700 1 -- 172.21.15.26:0/1101383253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567aa6bfb70 con 0x5567aa6a4320 2024-05-20T20:58:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa939d9b700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6bedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa93ad9d700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 0x5567aa6bf330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa934007f00 tx=0x7fa9340176f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b33f002d0ddad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:43.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 31705 3.84 MB/sec warmup 49 sec latency 32.171 ms 2024-05-20T20:58:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa921ffb700 1 -- 172.21.15.26:0/1101383253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934017d60 con 0x5567aa6a4320 2024-05-20T20:58:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa921ffb700 1 -- 172.21.15.26:0/1101383253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fa934003a00 con 0x5567aa6a4320 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92c0aadf0 con 0x5567aa6a4320 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa921ffb700 1 -- 172.21.15.26:0/1101383253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa934003c00 con 0x5567aa6a4320 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa94b108b80 10 client.16158 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567aa6b0a50 con 0x5567aa6a4320 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.550+0000 7fa921ffb700 1 -- 172.21.15.26:0/1101383253 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c0aadf0 con 0x5567aa6a4320 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa921ffb700 1 -- 172.21.15.26:0/1101383253 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa93402b410 con 0x5567aa6a4320 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa94b108b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa94b108b80 10 client.16158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa94b108b80 20 client.16158 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b1150 0x5567aa724cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa94b108b80 4 client.16158 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567aa5f84a0 con 0x5567aa6b1150 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.551+0000 7fa93a59c700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b1150 0x5567aa724cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa93a59c700 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b1150 0x5567aa724cc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfa4a39fe555a224 server_cookie=cc04210d161e9684 in_seq=0 out_seq=0 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa921ffb700 10 client.16158 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa921ffb700 1 -- 172.21.15.26:0/1101383253 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567aa5f84a0 con 0x5567aa6b1150 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa921ffb700 10 client.16158 handle_command_reply: tid=0 2024-05-20T20:58:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 client.16158 shutdown 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 msgr2=0x5567aa6bf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 0x5567aa6bf330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa934007f00 tx=0x7fa9340176f0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 shutdown_connections 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5567aa6b1150 0x5567aa724cc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567aa6a4320 0x5567aa6bf330 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567aa69e8a0 0x5567aa6c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 --2- 172.21.15.26:0/1101383253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567aa69f260 0x5567aa6bedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa921ffb700 0 client.16158 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 >> 172.21.15.26:0/1101383253 conn(0x5567aa5ca240 msgr2=0x5567aa69e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 shutdown_connections 2024-05-20T20:58:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 1 -- 172.21.15.26:0/1101383253 wait complete. 2024-05-20T20:58:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.552+0000 7fa94b108b80 20 client.16158 trim_cache size 0 max 16384 2024-05-20T20:58:43.562 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9240b2880 msgr2=0x7fa9240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9240b2880 0x7fa9240b4d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa93c1038c0 tx=0x7fa930005f90 comp rx=0 tx=0).stop 2024-05-20T20:58:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c105750 msgr2=0x7fa93c1a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c105750 0x7fa93c1a18e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c0057c0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 shutdown_connections 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa93c10b750 0x7fa93c19baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c105750 0x7fa93c1a18e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9240b2880 0x7fa9240b4d40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 --2- 172.21.15.26:0/3825950745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa93c106110 0x7fa93c1a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.553+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 >> 172.21.15.26:0/3825950745 conn(0x7fa93c101240 msgr2=0x7fa93c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.554+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 shutdown_connections 2024-05-20T20:58:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:43.554+0000 7fa940872700 1 -- 172.21.15.26:0/3825950745 wait complete. 2024-05-20T20:58:44.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:58:44.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 32197 3.86 MB/sec warmup 50 sec latency 19.016 ms 2024-05-20T20:58:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:45.217 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:45 smithi026 ceph-mon[31990]: pgmap v466: 97 pgs: 97 active+clean; 26 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T20:58:45.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:45 smithi144 ceph-mon[35660]: pgmap v466: 97 pgs: 97 active+clean; 26 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T20:58:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:45 smithi055 ceph-mon[35213]: pgmap v466: 97 pgs: 97 active+clean; 26 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T20:58:45.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 32621 3.79 MB/sec warmup 51 sec latency 16.679 ms 2024-05-20T20:58:46.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 33249 3.76 MB/sec warmup 52 sec latency 24.042 ms 2024-05-20T20:58:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 -- 172.21.15.26:0/2786617500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102440 msgr2=0x7fd2581028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 --2- 172.21.15.26:0/2786617500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102440 0x7fd2581028c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd240009990 tx=0x7fd240009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 -- 172.21.15.26:0/2786617500 shutdown_connections 2024-05-20T20:58:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 --2- 172.21.15.26:0/2786617500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd258102e00 0x7fd25810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 --2- 172.21.15.26:0/2786617500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd258108520 0x7fd258108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 --2- 172.21.15.26:0/2786617500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102440 0x7fd2581028c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 -- 172.21.15.26:0/2786617500 >> 172.21.15.26:0/2786617500 conn(0x7fd2580793f0 msgr2=0x7fd258079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 -- 172.21.15.26:0/2786617500 shutdown_connections 2024-05-20T20:58:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.715+0000 7fd25d48e700 1 -- 172.21.15.26:0/2786617500 wait complete. 2024-05-20T20:58:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 Processor -- start 2024-05-20T20:58:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 -- start start 2024-05-20T20:58:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd258102440 0x7fd258074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 0x7fd25806df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd258108520 0x7fd25806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581124a0 con 0x7fd258102e00 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd258112320 con 0x7fd258108520 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd25d48e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd258112620 con 0x7fd258102440 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd24e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 0x7fd25806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd24e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 0x7fd25806df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43454/0 (socket says 172.21.15.26:43454) 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd24e7fc700 1 -- 172.21.15.26:0/446455211 learned_addr learned my addr 172.21.15.26:0/446455211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.716+0000 7fd256ffd700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd258102440 0x7fd258074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd2577fe700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd258108520 0x7fd25806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd24e7fc700 1 -- 172.21.15.26:0/446455211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd258102440 msgr2=0x7fd258074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd258102440 0x7fd258074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd24e7fc700 1 -- 172.21.15.26:0/446455211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd258108520 msgr2=0x7fd25806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd258108520 0x7fd25806e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd24e7fc700 1 -- 172.21.15.26:0/446455211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25806eb40 con 0x7fd258102e00 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 0x7fd25806df20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd240006010 tx=0x7fd2400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd254ff9700 1 -- 172.21.15.26:0/446455211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240027070 con 0x7fd258102e00 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd258072920 con 0x7fd258102e00 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd254ff9700 1 -- 172.21.15.26:0/446455211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd240022470 con 0x7fd258102e00 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd254ff9700 1 -- 172.21.15.26:0/446455211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240017650 con 0x7fd258102e00 2024-05-20T20:58:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.717+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd258072cc0 con 0x7fd258102e00 2024-05-20T20:58:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.721+0000 7fd254ff9700 1 -- 172.21.15.26:0/446455211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd24002a070 con 0x7fd258102e00 2024-05-20T20:58:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd267d24b80 1 Processor -- start 2024-05-20T20:58:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd254ff9700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd2440b2880 0x7fd2440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd267d24b80 1 -- start start 2024-05-20T20:58:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd256ffd700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd2440b2880 0x7fd2440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd254ff9700 1 -- 172.21.15.26:0/446455211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd2400e3120 con 0x7fd258102e00 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d649fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.722+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d649fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd267d24b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5635d63fec90 con 0x5635d64a0210 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd267d24b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5635d63ee830 con 0x5635d64a3f60 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd267d24b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5635d63f2870 con 0x5635d649f860 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd2577fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d649fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48016/0 (socket says 172.21.15.26:48016) 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd256ffd700 1 -- 172.21.15.26:0/428171726 learned_addr learned my addr 172.21.15.26:0/428171726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd24e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd24e7fc700 1 -- 172.21.15.26:0/428171726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 msgr2=0x5635d649fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d649fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd24e7fc700 1 -- 172.21.15.26:0/428171726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 msgr2=0x5635d64a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd24e7fc700 1 -- 172.21.15.26:0/428171726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d633e8d0 con 0x5635d64a0210 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.723+0000 7fd256ffd700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd2440b2880 0x7fd2440b4d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd248011630 tx=0x7fd2480119f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.724+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a3940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd2400e0b60 tx=0x7fd2400aa220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319686105c1de86f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.724+0000 7fd24d7fa700 1 -- 172.21.15.26:0/428171726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400aa710 con 0x5635d64a0210 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.724+0000 7fd24d7fa700 1 -- 172.21.15.26:0/428171726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2400aeac0 con 0x5635d64a0210 2024-05-20T20:58:46.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.724+0000 7fd24d7fa700 1 -- 172.21.15.26:0/428171726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240092310 con 0x5635d64a0210 2024-05-20T20:58:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 -- 172.21.15.26:0/428171726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 msgr2=0x5635d64a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a3940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd2400e0b60 tx=0x7fd2400aa220 comp rx=0 tx=0).stop 2024-05-20T20:58:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 -- 172.21.15.26:0/428171726 shutdown_connections 2024-05-20T20:58:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d649fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 --2- 172.21.15.26:0/428171726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a3940 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 -- 172.21.15.26:0/428171726 >> 172.21.15.26:0/428171726 conn(0x5635d63cd080 msgr2=0x5635d6499e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 -- 172.21.15.26:0/428171726 shutdown_connections 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 -- 172.21.15.26:0/428171726 wait complete. 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 -- start start 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.725+0000 7fd267d24b80 1 Processor -- start 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 0x5635d64ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a0210 0x5635d64a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 0x5635d64a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5635d62f4fa0 con 0x5635d64a3f60 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5635d63907c0 con 0x5635d649f860 2024-05-20T20:58:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5635d64b46f0 con 0x5635d64a0210 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd267d24b80 10 client.0 _ll_register_callbacks cb 0x5635d6389920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 0x5635d64ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd2577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 0x5635d64a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 0x5635d64ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53542/0 (socket says 172.21.15.26:53542) 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd2577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 0x5635d64a8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43476/0 (socket says 172.21.15.26:43476) 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 -- 172.21.15.26:0/2627291985 learned_addr learned my addr 172.21.15.26:0/2627291985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a0210 0x5635d64a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 -- 172.21.15.26:0/2627291985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a0210 msgr2=0x5635d64a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a0210 0x5635d64a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 -- 172.21.15.26:0/2627291985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 msgr2=0x5635d64a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 0x5635d64a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 -- 172.21.15.26:0/2627291985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d638d690 con 0x5635d649f860 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd2577fe700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 0x5635d64a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd256ffd700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 0x5635d64ab440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd248020a90 tx=0x7fd2480047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254610c420d5ce30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd23e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.726+0000 7fd23e7fc700 1 -- 172.21.15.26:0/2627291985 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248024070 con 0x5635d649f860 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2400aeac0 con 0x5635d649f860 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd23e7fc700 1 -- 172.21.15.26:0/2627291985 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd24800f7d0 con 0x5635d649f860 2024-05-20T20:58:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd23e7fc700 1 -- 172.21.15.26:0/2627291985 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd24801a040 con 0x5635d649f860 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 10 client.25724 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5635d64bedc0 con 0x5635d649f860 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd23e7fc700 1 -- 172.21.15.26:0/2627291985 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2400aeac0 con 0x5635d649f860 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd23e7fc700 1 -- 172.21.15.26:0/2627291985 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd24801d030 con 0x5635d649f860 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 10 client.25724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 20 client.25724 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d64c1350 0x5635d64c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 4 client.25724 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635d646b9e0 con 0x5635d64c1350 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.727+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d64c1350 0x5635d64c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.728+0000 7fd23e7fc700 10 client.25724 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.728+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d64c1350 0x5635d64c3740 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae6100168b184a10 server_cookie=e6031dbb15210720 in_seq=0 out_seq=0 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd23e7fc700 1 -- 172.21.15.26:0/2627291985 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5635d646b9e0 con 0x5635d64c1350 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd23e7fc700 10 client.25724 handle_command_reply: tid=0 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 client.25724 shutdown 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 msgr2=0x5635d64ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 0x5635d64ab440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd248020a90 tx=0x7fd2480047c0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 shutdown_connections 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d64c1350 0x5635d64c3740 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a3f60 0x5635d64a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d649f860 0x5635d64ab440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd23e7fc700 0 client.25724 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 --2- 172.21.15.26:0/2627291985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a0210 0x5635d64a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 >> 172.21.15.26:0/2627291985 conn(0x5635d63cd080 msgr2=0x5635d6499510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 shutdown_connections 2024-05-20T20:58:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 1 -- 172.21.15.26:0/2627291985 wait complete. 2024-05-20T20:58:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.729+0000 7fd267d24b80 20 client.25724 trim_cache size 0 max 16384 2024-05-20T20:58:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 Processor -- start 2024-05-20T20:58:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 -- start start 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d64bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b6ac0 con 0x5635d64a0210 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b4350 con 0x5635d64a3f60 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd267d24b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b21a0 con 0x5635d649f860 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd24e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd24e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43484/0 (socket says 172.21.15.26:43484) 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd24e7fc700 1 -- 172.21.15.26:0/1105298810 learned_addr learned my addr 172.21.15.26:0/1105298810 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd2577fe700 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.749+0000 7fd256ffd700 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d64bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24e7fc700 1 -- 172.21.15.26:0/1105298810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 msgr2=0x5635d64bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d64bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24e7fc700 1 -- 172.21.15.26:0/1105298810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 msgr2=0x5635d64c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24e7fc700 1 -- 172.21.15.26:0/1105298810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d649fe90 con 0x5635d64a0210 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd2577fe700 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a2fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd2400a6790 tx=0x7fd2400cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6254bc01c8b4ca0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24cff9700 1 -- 172.21.15.26:0/1105298810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400094b0 con 0x5635d64a0210 2024-05-20T20:58:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24cff9700 1 -- 172.21.15.26:0/1105298810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd240022920 con 0x5635d64a0210 2024-05-20T20:58:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.750+0000 7fd24cff9700 1 -- 172.21.15.26:0/1105298810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400aa970 con 0x5635d64a0210 2024-05-20T20:58:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 -- 172.21.15.26:0/1105298810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 msgr2=0x5635d64a2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a2fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd2400a6790 tx=0x7fd2400cd040 comp rx=0 tx=0).stop 2024-05-20T20:58:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 -- 172.21.15.26:0/1105298810 shutdown_connections 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a3f60 0x5635d64c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d649f860 0x5635d64bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1105298810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d64a0210 0x5635d64a2fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 -- 172.21.15.26:0/1105298810 >> 172.21.15.26:0/1105298810 conn(0x5635d63cd080 msgr2=0x5635d64a28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 -- 172.21.15.26:0/1105298810 shutdown_connections 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 -- 172.21.15.26:0/1105298810 wait complete. 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.751+0000 7fd267d24b80 1 -- start start 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 Processor -- start 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 0x5635d64b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a0210 0x5635d64b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a3f60 0x5635d6523310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b2810 con 0x5635d649f860 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b3c80 con 0x5635d64a0210 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b2ee0 con 0x5635d64a3f60 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd267d24b80 10 client.0 _ll_register_callbacks cb 0x5635d6389920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 0x5635d64b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 0x5635d64b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43500/0 (socket says 172.21.15.26:43500) 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 -- 172.21.15.26:0/1875531275 learned_addr learned my addr 172.21.15.26:0/1875531275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a0210 0x5635d64b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd2577fe700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a3f60 0x5635d6523310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 -- 172.21.15.26:0/1875531275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a3f60 msgr2=0x5635d6523310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a3f60 0x5635d6523310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 -- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a0210 msgr2=0x5635d64b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a0210 0x5635d64b99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.752+0000 7fd256ffd700 1 -- 172.21.15.26:0/1875531275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d65239f0 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd256ffd700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 0x5635d64b94a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c00 tx=0x7fd2480037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cc2e8d00d9ec71 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd23e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd23e7fc700 1 -- 172.21.15.26:0/1875531275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248024040 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd23e7fc700 1 -- 172.21.15.26:0/1875531275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd248009040 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd240022920 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd23e7fc700 1 -- 172.21.15.26:0/1875531275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd248040050 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd267d24b80 10 client.16176 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5635d652bf60 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd23e7fc700 1 -- 172.21.15.26:0/1875531275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd240022920 con 0x5635d649f860 2024-05-20T20:58:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.753+0000 7fd23e7fc700 1 -- 172.21.15.26:0/1875531275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd24800fb20 con 0x5635d649f860 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 10 client.16176 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 10 client.16176 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 20 client.16176 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d652c6c0 0x5635d652eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 4 client.16176 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635d6400be0 con 0x5635d652c6c0 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d652c6c0 0x5635d652eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd24e7fc700 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d652c6c0 0x5635d652eae0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1503918fe6851ad5 server_cookie=3309738b96dc8079 in_seq=0 out_seq=0 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd23e7fc700 10 client.16176 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd23e7fc700 1 -- 172.21.15.26:0/1875531275 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635d6400be0 con 0x5635d652c6c0 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd23e7fc700 10 client.16176 handle_command_reply: tid=0 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 1 client.16176 shutdown 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 msgr2=0x5635d64b94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.754+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 0x5635d64b94a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c00 tx=0x7fd2480037d0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 shutdown_connections 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5635d652c6c0 0x5635d652eae0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635d649f860 0x5635d64b94a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd23e7fc700 0 client.16176 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635d64a3f60 0x5635d6523310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 --2- 172.21.15.26:0/1875531275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5635d64a0210 0x5635d64b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 >> 172.21.15.26:0/1875531275 conn(0x5635d63cd080 msgr2=0x5635d64a3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 shutdown_connections 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 1 -- 172.21.15.26:0/1875531275 wait complete. 2024-05-20T20:58:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd267d24b80 20 client.16176 trim_cache size 0 max 16384 2024-05-20T20:58:46.761 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.755+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd2440b2880 msgr2=0x7fd2440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd2440b2880 0x7fd2440b4d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd248011630 tx=0x7fd2480119f0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 msgr2=0x7fd25806df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 0x7fd25806df20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd240006010 tx=0x7fd2400047b0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 shutdown_connections 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd258108520 0x7fd25806e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd258102440 0x7fd258074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd2440b2880 0x7fd2440b4d40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 --2- 172.21.15.26:0/446455211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd258102e00 0x7fd25806df20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 >> 172.21.15.26:0/446455211 conn(0x7fd2580793f0 msgr2=0x7fd258100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 shutdown_connections 2024-05-20T20:58:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:46.756+0000 7fd25d48e700 1 -- 172.21.15.26:0/446455211 wait complete. 2024-05-20T20:58:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:47 smithi144 ceph-mon[35660]: pgmap v467: 97 pgs: 97 active+clean; 27 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-05-20T20:58:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:47 smithi055 ceph-mon[35213]: pgmap v467: 97 pgs: 97 active+clean; 27 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-05-20T20:58:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:47 smithi026 ceph-mon[31990]: pgmap v467: 97 pgs: 97 active+clean; 27 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-05-20T20:58:47.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 34026 3.75 MB/sec warmup 53 sec latency 12.573 ms 2024-05-20T20:58:47.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:58:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:48.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 34726 3.78 MB/sec warmup 54 sec latency 15.412 ms 2024-05-20T20:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 -- 172.21.15.26:0/3946744762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc102ba0 msgr2=0x7f7fdc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/3946744762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc102ba0 0x7f7fdc103020 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009900 tx=0x7f7fcc02a410 comp rx=0 tx=0).stop 2024-05-20T20:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 -- 172.21.15.26:0/3946744762 shutdown_connections 2024-05-20T20:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/3946744762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc103560 0x7f7fdc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/3946744762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc102ba0 0x7f7fdc103020 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/3946744762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 -- 172.21.15.26:0/3946744762 >> 172.21.15.26:0/3946744762 conn(0x7f7fdc0fcf20 msgr2=0x7f7fdc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 -- 172.21.15.26:0/3946744762 shutdown_connections 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.980+0000 7f7fe42c4640 1 -- 172.21.15.26:0/3946744762 wait complete. 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 Processor -- start 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 -- start start 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc105a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc102ba0 0x7f7fdc1040d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fdc103560 0x7f7fdc104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d370 con 0x7f7fdc1019b0 2024-05-20T20:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d1f0 con 0x7f7fdc102ba0 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe42c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d4f0 con 0x7f7fdc103560 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc105a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc105a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43514/0 (socket says 172.21.15.26:43514) 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 -- 172.21.15.26:0/383304149 learned_addr learned my addr 172.21.15.26:0/383304149 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe185f640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc102ba0 0x7f7fdc1040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2861640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fdc103560 0x7f7fdc104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 -- 172.21.15.26:0/383304149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fdc103560 msgr2=0x7f7fdc104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fdc103560 0x7f7fdc104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 -- 172.21.15.26:0/383304149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc102ba0 msgr2=0x7f7fdc1040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc102ba0 0x7f7fdc1040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.981+0000 7f7fe2060640 1 -- 172.21.15.26:0/383304149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc104ea0 con 0x7f7fdc1019b0 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.982+0000 7f7fe2060640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc105a90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c970 tx=0x7f7fd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.982+0000 7f7fd37fe640 1 -- 172.21.15.26:0/383304149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8007c10 con 0x7f7fdc1019b0 2024-05-20T20:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.982+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc1a2be0 con 0x7f7fdc1019b0 2024-05-20T20:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.982+0000 7f7fd37fe640 1 -- 172.21.15.26:0/383304149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fd8007db0 con 0x7f7fdc1019b0 2024-05-20T20:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.982+0000 7f7fd37fe640 1 -- 172.21.15.26:0/383304149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800f7d0 con 0x7f7fdc1019b0 2024-05-20T20:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.982+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1a3100 con 0x7f7fdc1019b0 2024-05-20T20:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 Processor -- start 2024-05-20T20:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 -- start start 2024-05-20T20:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 0x55c3c96b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96bbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96bc220 0x55c3c96be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c3c9627c30 con 0x55c3c96bc220 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c3c96177d0 con 0x55c3c96b66c0 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c3c961b810 con 0x55c3c96b70b0 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.985+0000 7f7fd37fe640 1 -- 172.21.15.26:0/383304149 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7fd8018480 con 0x7f7fdc1019b0 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 0x55c3c96b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 0x55c3c96b6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53578/0 (socket says 172.21.15.26:53578) 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 -- 172.21.15.26:0/424470898 learned_addr learned my addr 172.21.15.26:0/424470898 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe185f640 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96bc220 0x55c3c96be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2060640 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96bbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 -- 172.21.15.26:0/424470898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 msgr2=0x55c3c96bbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96bbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 -- 172.21.15.26:0/424470898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96bc220 msgr2=0x55c3c96be6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96bc220 0x55c3c96be6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 -- 172.21.15.26:0/424470898 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c932c210 con 0x55c3c96b66c0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2060640 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96bbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fe2861640 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 0x55c3c96b6ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009fd0 tx=0x7f7fd40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52df49beb74969d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fd0bfe640 1 -- 172.21.15.26:0/424470898 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4004d20 con 0x55c3c96b66c0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fd0bfe640 1 -- 172.21.15.26:0/424470898 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fd4004050 con 0x55c3c96b66c0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.986+0000 7f7fd0bfe640 1 -- 172.21.15.26:0/424470898 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4005260 con 0x55c3c96b66c0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fd37fe640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fc80b28a0 0x7f7fc80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe2060640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fc80b28a0 0x7f7fc80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fd37fe640 1 -- 172.21.15.26:0/383304149 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7fd80e2530 con 0x7f7fdc1019b0 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/424470898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 msgr2=0x55c3c96b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe2060640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fc80b28a0 0x7f7fc80b4d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80e0340 tx=0x7f7fd8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 0x55c3c96b6ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009fd0 tx=0x7f7fd40118d0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/424470898 shutdown_connections 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96bc220 0x55c3c96be6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96bbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/424470898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96b66c0 0x55c3c96b6ae0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/424470898 >> 172.21.15.26:0/424470898 conn(0x55c3c96b1d30 msgr2=0x55c3c96b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.987+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/424470898 shutdown_connections 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/424470898 wait complete. 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 -- start start 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 Processor -- start 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c3c92fd790 con 0x55c3c96b66c0 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c3c95d3890 con 0x55c3c96bc220 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c3c96bf1d0 con 0x55c3c96b70b0 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe2861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe6fdb740 10 client.0 _ll_register_callbacks cb 0x55c3c932ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe2861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43528/0 (socket says 172.21.15.26:43528) 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe2861640 1 -- 172.21.15.26:0/3914510290 learned_addr learned my addr 172.21.15.26:0/3914510290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.988+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe2861640 1 -- 172.21.15.26:0/3914510290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 msgr2=0x55c3c96c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe2861640 1 -- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 msgr2=0x55c3c96c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe2861640 1 -- 172.21.15.26:0/3914510290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c93f8620 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0780 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009fa0 tx=0x7f7fd4004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf79348adeef22c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3914510290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400f6e0 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3914510290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7fd400f880 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd4004050 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3914510290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fd40042a0 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe6fdb740 10 client.16188 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.989+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3c96cac50 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3914510290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fd4004050 con 0x55c3c96b66c0 2024-05-20T20:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3914510290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7fd4005470 con 0x55c3c96b66c0 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe6fdb740 10 client.16188 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe6fdb740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe6fdb740 20 client.16188 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c96cc190 0x55c3c96ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe6fdb740 4 client.16188 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3c9629b80 con 0x55c3c96cc190 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c96cc190 0x55c3c96ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fc5ffb640 10 client.16188 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.990+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c96cc190 0x55c3c96ce580 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63c8d5e49db5300 server_cookie=b3d633fa9886fb91 in_seq=0 out_seq=0 2024-05-20T20:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3914510290 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c3c9629b80 con 0x55c3c96cc190 2024-05-20T20:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fc5ffb640 10 client.16188 handle_command_reply: tid=0 2024-05-20T20:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 client.16188 shutdown 2024-05-20T20:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 msgr2=0x55c3c96c0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0780 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009fa0 tx=0x7f7fd4004840 comp rx=0 tx=0).stop 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 shutdown_connections 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c96cc190 0x55c3c96ce580 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3914510290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0780 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fc5ffb640 0 client.16188 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.991+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 >> 172.21.15.26:0/3914510290 conn(0x55c3c96b1d30 msgr2=0x55c3c96b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.992+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 shutdown_connections 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.992+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3914510290 wait complete. 2024-05-20T20:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:48.992+0000 7f7fe6fdb740 20 client.16188 trim_cache size 0 max 16384 2024-05-20T20:58:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 Processor -- start 2024-05-20T20:58:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 -- start start 2024-05-20T20:58:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 0x55c3c96c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1ae960 con 0x55c3c96b70b0 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1ac230 con 0x55c3c96bc220 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1aa080 con 0x55c3c96b66c0 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe185f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 0x55c3c96c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe185f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 0x55c3c96c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43530/0 (socket says 172.21.15.26:43530) 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.008+0000 7f7fe185f640 1 -- 172.21.15.26:0/3435038418 learned_addr learned my addr 172.21.15.26:0/3435038418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe185f640 1 -- 172.21.15.26:0/3435038418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b66c0 msgr2=0x55c3c96c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe185f640 1 -- 172.21.15.26:0/3435038418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 msgr2=0x55c3c96b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe185f640 1 -- 172.21.15.26:0/3435038418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c96cebb0 con 0x55c3c96b70b0 2024-05-20T20:58:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 0x55c3c96c7cc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc02a930 tx=0x7f7fcc004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c29c6dca4d8086e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fc7fff640 1 -- 172.21.15.26:0/3435038418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc035040 con 0x55c3c96b70b0 2024-05-20T20:58:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fc7fff640 1 -- 172.21.15.26:0/3435038418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fcc031cd0 con 0x55c3c96b70b0 2024-05-20T20:58:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.009+0000 7f7fc7fff640 1 -- 172.21.15.26:0/3435038418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc003dc0 con 0x55c3c96b70b0 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3435038418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 msgr2=0x55c3c96c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 0x55c3c96c7cc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc02a930 tx=0x7f7fcc004310 comp rx=0 tx=0).stop 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3435038418 shutdown_connections 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b70b0 0x55c3c96c7cc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3435038418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b66c0 0x55c3c96c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3435038418 >> 172.21.15.26:0/3435038418 conn(0x55c3c96b1d30 msgr2=0x55c3c96b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3435038418 shutdown_connections 2024-05-20T20:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3435038418 wait complete. 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 -- start start 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.010+0000 7f7fe6fdb740 1 Processor -- start 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1aa6f0 con 0x55c3c96b66c0 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1abb60 con 0x55c3c96bc220 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1aadc0 con 0x55c3c96b70b0 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe6fdb740 10 client.0 _ll_register_callbacks cb 0x7f7fcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43532/0 (socket says 172.21.15.26:43532) 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 -- 172.21.15.26:0/3356057247 learned_addr learned my addr 172.21.15.26:0/3356057247 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 -- 172.21.15.26:0/3356057247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 msgr2=0x55c3c96c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 -- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 msgr2=0x55c3c96c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 -- 172.21.15.26:0/3356057247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c96b6300 con 0x55c3c96b66c0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fe2861640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c4dd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4011dd0 tx=0x7f7fd4009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b99a290b8aea68 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.011+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3356057247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4003f30 con 0x55c3c96b66c0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.012+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3356057247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7fd4019740 con 0x55c3c96b66c0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.012+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc031cd0 con 0x55c3c96b66c0 2024-05-20T20:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.012+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3356057247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fd40040d0 con 0x55c3c96b66c0 2024-05-20T20:58:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.012+0000 7f7fe6fdb740 10 client.16194 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.012+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3c96b1b60 con 0x55c3c96b66c0 2024-05-20T20:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3356057247 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fcc031cd0 con 0x55c3c96b66c0 2024-05-20T20:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3356057247 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7fd403c040 con 0x55c3c96b66c0 2024-05-20T20:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe6fdb740 10 client.16194 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe6fdb740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe6fdb740 20 client.16194 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c9736cb0 0x55c3c97390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe6fdb740 4 client.16194 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:58:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3c9623360 con 0x55c3c9736cb0 2024-05-20T20:58:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c9736cb0 0x55c3c97390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fc5ffb640 10 client.16194 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.016+0000 7f7fe185f640 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c9736cb0 0x55c3c97390a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e6328abe698170 server_cookie=3e5146cf9abbab59 in_seq=0 out_seq=0 2024-05-20T20:58:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.059+0000 7f7fc5ffb640 1 -- 172.21.15.26:0/3356057247 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3c9623360 con 0x55c3c9736cb0 2024-05-20T20:58:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.059+0000 7f7fc5ffb640 10 client.16194 handle_command_reply: tid=0 2024-05-20T20:58:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.059+0000 7f7fe6fdb740 1 client.16194 shutdown 2024-05-20T20:58:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 msgr2=0x55c3c96c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c4dd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4011dd0 tx=0x7f7fd4009d50 comp rx=0 tx=0).stop 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 shutdown_connections 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c3c9736cb0 0x55c3c97390a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c3c96bc220 0x55c3c96c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3c96b70b0 0x55c3c96c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 --2- 172.21.15.26:0/3356057247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3c96b66c0 0x55c3c96c4dd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 >> 172.21.15.26:0/3356057247 conn(0x55c3c96b1d30 msgr2=0x55c3c96c0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fc5ffb640 0 client.16194 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 shutdown_connections 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 1 -- 172.21.15.26:0/3356057247 wait complete. 2024-05-20T20:58:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe6fdb740 20 client.16194 trim_cache size 0 max 16384 2024-05-20T20:58:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:58:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:58:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:58:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fc80b28a0 msgr2=0x7f7fc80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fc80b28a0 0x7f7fc80b4d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80e0340 tx=0x7f7fd8000bc0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 msgr2=0x7f7fdc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.060+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc105a90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c970 tx=0x7f7fd800ce30 comp rx=0 tx=0).stop 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 shutdown_connections 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fc80b28a0 0x7f7fc80b4d60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fdc103560 0x7f7fdc104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc102ba0 0x7f7fdc1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 --2- 172.21.15.26:0/383304149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fdc1019b0 0x7f7fdc105a90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 >> 172.21.15.26:0/383304149 conn(0x7f7fdc0fcf20 msgr2=0x7f7fdc06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 shutdown_connections 2024-05-20T20:58:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:58:49.061+0000 7f7fe42c4640 1 -- 172.21.15.26:0/383304149 wait complete. 2024-05-20T20:58:49.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:49 smithi144 ceph-mon[35660]: pgmap v468: 97 pgs: 97 active+clean; 28 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-05-20T20:58:49.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:49 smithi026 ceph-mon[31990]: pgmap v468: 97 pgs: 97 active+clean; 28 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-05-20T20:58:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:49 smithi055 ceph-mon[35213]: pgmap v468: 97 pgs: 97 active+clean; 28 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-05-20T20:58:49.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 35352 3.78 MB/sec warmup 55 sec latency 38.950 ms 2024-05-20T20:58:49.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 -- 172.21.15.26:0/3055041973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 msgr2=0x7f461c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 --2- 172.21.15.26:0/3055041973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c108900 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4618009990 tx=0x7f4618009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 -- 172.21.15.26:0/3055041973 shutdown_connections 2024-05-20T20:58:49.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 --2- 172.21.15.26:0/3055041973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c108900 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 --2- 172.21.15.26:0/3055041973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102e00 0x7f461c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 --2- 172.21.15.26:0/3055041973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f461c102440 0x7f461c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 -- 172.21.15.26:0/3055041973 >> 172.21.15.26:0/3055041973 conn(0x7f461c0793f0 msgr2=0x7f461c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 -- 172.21.15.26:0/3055041973 shutdown_connections 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.915+0000 7f462426e700 1 -- 172.21.15.26:0/3055041973 wait complete. 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 Processor -- start 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 -- start start 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 0x7f461c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f461c102e00 0x7f461c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462200a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 0x7f461c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462280b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462200a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 0x7f461c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51182/0 (socket says 172.21.15.26:51182) 2024-05-20T20:58:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462280b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c1a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42280/0 (socket says 172.21.15.26:42280) 2024-05-20T20:58:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f461c112480 con 0x7f461c108520 2024-05-20T20:58:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f461c112300 con 0x7f461c102440 2024-05-20T20:58:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462426e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f461c112600 con 0x7f461c102e00 2024-05-20T20:58:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.916+0000 7f462200a700 1 -- 172.21.15.26:0/1657775300 learned_addr learned my addr 172.21.15.26:0/1657775300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462280b700 1 -- 172.21.15.26:0/1657775300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f461c102e00 msgr2=0x7f461c19af20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462280b700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f461c102e00 0x7f461c19af20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462280b700 1 -- 172.21.15.26:0/1657775300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 msgr2=0x7f461c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462280b700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 0x7f461c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462280b700 1 -- 172.21.15.26:0/1657775300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c1a5fa0 con 0x7f461c108520 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462200a700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 0x7f461c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462280b700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c1a5a60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f461400efc0 tx=0x7f461400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f46137fe700 1 -- 172.21.15.26:0/1657775300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46140092e0 con 0x7f461c108520 2024-05-20T20:58:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c1a6170 con 0x7f461c108520 2024-05-20T20:58:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f46137fe700 1 -- 172.21.15.26:0/1657775300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4614004bb0 con 0x7f461c108520 2024-05-20T20:58:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.917+0000 7f46137fe700 1 -- 172.21.15.26:0/1657775300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614004020 con 0x7f461c108520 2024-05-20T20:58:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.918+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c1a6570 con 0x7f461c108520 2024-05-20T20:58:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.920+0000 7f46137fe700 1 -- 172.21.15.26:0/1657775300 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f461401b030 con 0x7f461c108520 2024-05-20T20:58:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.921+0000 7f46137fe700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46080b2900 0x7f46080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462200a700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46080b2900 0x7f46080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f46137fe700 1 -- 172.21.15.26:0/1657775300 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f46140e1c40 con 0x7f461c108520 2024-05-20T20:58:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462eb04b80 1 Processor -- start 2024-05-20T20:58:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462eb04b80 1 -- start start 2024-05-20T20:58:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462200a700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46080b2900 0x7f46080b4dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4618009750 tx=0x7f461801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c2297700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 0x55c0c2291d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c229e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.922+0000 7f462eb04b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c0c21efd70 con 0x55c0c2297320 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f462eb04b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c0c21df910 con 0x55c0c22918a0 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f462eb04b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c0c21e3950 con 0x55c0c2292260 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f462200a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c2297700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 0x55c0c2291d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f462200a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c2297700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42296/0 (socket says 172.21.15.26:42296) 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f462200a700 1 -- 172.21.15.26:0/2606898145 learned_addr learned my addr 172.21.15.26:0/2606898145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 0x55c0c2291d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51190/0 (socket says 172.21.15.26:51190) 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f462280b700 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c229e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 -- 172.21.15.26:0/2606898145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 msgr2=0x55c0c229e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c229e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 -- 172.21.15.26:0/2606898145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 msgr2=0x55c0c2297700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c2297700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 -- 172.21.15.26:0/2606898145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c2188fb0 con 0x55c0c22918a0 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4621809700 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 0x55c0c2291d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f461c06bd70 tx=0x7f460c00ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e11cf046e2db0b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4610d70700 1 -- 172.21.15.26:0/2606898145 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c00c9e0 con 0x55c0c22918a0 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4610d70700 1 -- 172.21.15.26:0/2606898145 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f460c00cb80 con 0x55c0c22918a0 2024-05-20T20:58:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.923+0000 7f4610d70700 1 -- 172.21.15.26:0/2606898145 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c018400 con 0x55c0c22918a0 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 -- 172.21.15.26:0/2606898145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 msgr2=0x55c0c2291d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 0x55c0c2291d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f461c06bd70 tx=0x7f460c00ac00 comp rx=0 tx=0).stop 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 -- 172.21.15.26:0/2606898145 shutdown_connections 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c2297700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c229e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2606898145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c22918a0 0x55c0c2291d20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 -- 172.21.15.26:0/2606898145 >> 172.21.15.26:0/2606898145 conn(0x55c0c21bd240 msgr2=0x55c0c228c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 -- 172.21.15.26:0/2606898145 shutdown_connections 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.924+0000 7f462eb04b80 1 -- 172.21.15.26:0/2606898145 wait complete. 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 -- start start 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 Processor -- start 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c229dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 0x55c0c229b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c229b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c0c20e5fa0 con 0x55c0c2297320 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2147da0 con 0x55c0c2292260 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462eb04b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c0c229e6e0 con 0x55c0c22918a0 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f4621809700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 0x55c0c229b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f4621809700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 0x55c0c229b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51200/0 (socket says 172.21.15.26:51200) 2024-05-20T20:58:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f4621809700 1 -- 172.21.15.26:0/2585323219 learned_addr learned my addr 172.21.15.26:0/2585323219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462200a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c229dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.925+0000 7f462280b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c229b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f462eb04b80 10 client.0 _ll_register_callbacks cb 0x55c0c217a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4621809700 1 -- 172.21.15.26:0/2585323219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 msgr2=0x55c0c229dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4621809700 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c229dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4621809700 1 -- 172.21.15.26:0/2585323219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 msgr2=0x55c0c229b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4621809700 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c229b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4621809700 1 -- 172.21.15.26:0/2585323219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c212f8d0 con 0x55c0c2292260 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f462280b700 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c229b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4621809700 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 0x55c0c229b030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460c00fc10 tx=0x7f460c00ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646bed01417bb48a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4605ffb700 1 -- 172.21.15.26:0/2585323219 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01d070 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f4605ffb700 1 -- 172.21.15.26:0/2585323219 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f460c00f960 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.926+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460c00cb80 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f4605ffb700 1 -- 172.21.15.26:0/2585323219 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f460c005040 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 10 client.25760 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0c2199760 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f4605ffb700 1 -- 172.21.15.26:0/2585323219 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f460c00cb80 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f4605ffb700 1 -- 172.21.15.26:0/2585323219 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f460c004cd0 con 0x55c0c2292260 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 20 client.25760 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c22ab140 0x55c0c22ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 4 client.25760 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c21f1cc0 con 0x55c0c22ab140 2024-05-20T20:58:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.927+0000 7f462200a700 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c22ab140 0x55c0c22ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.928+0000 7f462200a700 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c22ab140 0x55c0c22ad530 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0aaa54563e918ff server_cookie=9cf6d49b2e277400 in_seq=0 out_seq=0 2024-05-20T20:58:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.928+0000 7f4605ffb700 10 client.25760 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f4605ffb700 1 -- 172.21.15.26:0/2585323219 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c0c21f1cc0 con 0x55c0c22ab140 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f4605ffb700 10 client.25760 handle_command_reply: tid=0 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 client.25760 shutdown 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 msgr2=0x55c0c229b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 0x55c0c229b030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460c00fc10 tx=0x7f460c00ca10 comp rx=0 tx=0).stop 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 shutdown_connections 2024-05-20T20:58:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c22ab140 0x55c0c22ad530 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2297320 0x55c0c229b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c229dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 --2- 172.21.15.26:0/2585323219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2292260 0x55c0c229b030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f4605ffb700 0 client.25760 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 >> 172.21.15.26:0/2585323219 conn(0x55c0c21bd240 msgr2=0x55c0c228baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 shutdown_connections 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.929+0000 7f462eb04b80 1 -- 172.21.15.26:0/2585323219 wait complete. 2024-05-20T20:58:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.930+0000 7f462eb04b80 20 client.25760 trim_cache size 0 max 16384 2024-05-20T20:58:49.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.947+0000 7f462eb04b80 1 Processor -- start 2024-05-20T20:58:49.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.947+0000 7f462eb04b80 1 -- start start 2024-05-20T20:58:49.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.947+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c228c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.947+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 0x55c0c229cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c2298010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462eb04b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1b05e0 con 0x55c0c2292260 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462eb04b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ade70 con 0x55c0c2297320 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462eb04b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1abcc0 con 0x55c0c22918a0 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 0x55c0c229cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 0x55c0c229cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42304/0 (socket says 172.21.15.26:42304) 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 -- 172.21.15.26:0/3448568506 learned_addr learned my addr 172.21.15.26:0/3448568506 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462200a700 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c228c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462280b700 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c2298010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 -- 172.21.15.26:0/3448568506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 msgr2=0x55c0c228c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c228c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 -- 172.21.15.26:0/3448568506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 msgr2=0x55c0c2298010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c2298010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 -- 172.21.15.26:0/3448568506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c229e530 con 0x55c0c2292260 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f462280b700 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c2298010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4621809700 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 0x55c0c229cf70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f460c00a870 tx=0x7f460c00ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23822419dc5259d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.948+0000 7f4607fff700 1 -- 172.21.15.26:0/3448568506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01d040 con 0x55c0c2292260 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f4607fff700 1 -- 172.21.15.26:0/3448568506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f460c026070 con 0x55c0c2292260 2024-05-20T20:58:49.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f4607fff700 1 -- 172.21.15.26:0/3448568506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c005440 con 0x55c0c2292260 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 -- 172.21.15.26:0/3448568506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 msgr2=0x55c0c229cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 0x55c0c229cf70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f460c00a870 tx=0x7f460c00ac00 comp rx=0 tx=0).stop 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 -- 172.21.15.26:0/3448568506 shutdown_connections 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c2298010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c22918a0 0x55c0c228c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 --2- 172.21.15.26:0/3448568506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c2292260 0x55c0c229cf70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 -- 172.21.15.26:0/3448568506 >> 172.21.15.26:0/3448568506 conn(0x55c0c21bd240 msgr2=0x55c0c229c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 -- 172.21.15.26:0/3448568506 shutdown_connections 2024-05-20T20:58:49.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.949+0000 7f462eb04b80 1 -- 172.21.15.26:0/3448568506 wait complete. 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 -- start start 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 Processor -- start 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c22918a0 0x55c0c22b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c22b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c22b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ac330 con 0x55c0c22918a0 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ad7a0 con 0x55c0c2297320 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1aca00 con 0x55c0c2292260 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462eb04b80 10 client.0 _ll_register_callbacks cb 0x55c0c217a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462200a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c22918a0 0x55c0c22b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462200a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c22918a0 0x55c0c22b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42312/0 (socket says 172.21.15.26:42312) 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462200a700 1 -- 172.21.15.26:0/92195343 learned_addr learned my addr 172.21.15.26:0/92195343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:49.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f462280b700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c22b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.950+0000 7f4621809700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c22b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462280b700 1 -- 172.21.15.26:0/92195343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 msgr2=0x55c0c22b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462280b700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c22b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462280b700 1 -- 172.21.15.26:0/92195343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c22918a0 msgr2=0x55c0c22b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462280b700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c22918a0 0x55c0c22b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462280b700 1 -- 172.21.15.26:0/92195343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c22a2be0 con 0x55c0c2297320 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462280b700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c22b25d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46140df730 tx=0x7f4614004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56753bbcfb765aea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f4605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f4605ffb700 1 -- 172.21.15.26:0/92195343 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46140a8da0 con 0x55c0c2297320 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f4605ffb700 1 -- 172.21.15.26:0/92195343 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f46140ad360 con 0x55c0c2297320 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460c026070 con 0x55c0c2297320 2024-05-20T20:58:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f4605ffb700 1 -- 172.21.15.26:0/92195343 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46140a9990 con 0x55c0c2297320 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462eb04b80 10 client.25766 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.951+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0c22a3ee0 con 0x55c0c2297320 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f4605ffb700 1 -- 172.21.15.26:0/92195343 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f460c026070 con 0x55c0c2297320 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f4605ffb700 1 -- 172.21.15.26:0/92195343 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f46140a4c00 con 0x55c0c2297320 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462eb04b80 10 client.25766 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462eb04b80 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462eb04b80 20 client.25766 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462eb04b80 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c2315a30 0x55c0c2317e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462eb04b80 4 client.25766 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0c21eb4a0 con 0x55c0c2315a30 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.952+0000 7f462200a700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c2315a30 0x55c0c2317e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f4605ffb700 10 client.25766 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462200a700 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c2315a30 0x55c0c2317e20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a72d5a808876967 server_cookie=f8586b11885cb86f in_seq=0 out_seq=0 2024-05-20T20:58:49.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f4605ffb700 1 -- 172.21.15.26:0/92195343 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0c21eb4a0 con 0x55c0c2315a30 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f4605ffb700 10 client.25766 handle_command_reply: tid=0 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 client.25766 shutdown 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 msgr2=0x55c0c22b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c22b25d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46140df730 tx=0x7f4614004f90 comp rx=0 tx=0).stop 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 shutdown_connections 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c0c2315a30 0x55c0c2317e20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c22918a0 0x55c0c22b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c0c2297320 0x55c0c22b25d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f4605ffb700 0 client.25766 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 --2- 172.21.15.26:0/92195343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0c2292260 0x55c0c22b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 >> 172.21.15.26:0/92195343 conn(0x55c0c21bd240 msgr2=0x55c0c229d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.953+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 shutdown_connections 2024-05-20T20:58:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462eb04b80 1 -- 172.21.15.26:0/92195343 wait complete. 2024-05-20T20:58:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462eb04b80 20 client.25766 trim_cache size 0 max 16384 2024-05-20T20:58:49.959 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46080b2900 msgr2=0x7f46080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46080b2900 0x7f46080b4dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4618009750 tx=0x7f461801a040 comp rx=0 tx=0).stop 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 msgr2=0x7f461c1a5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c1a5a60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f461400efc0 tx=0x7f461400c3a0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 shutdown_connections 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46080b2900 0x7f46080b4dc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f461c108520 0x7f461c1a5a60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f461c102440 0x7f461c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 --2- 172.21.15.26:0/1657775300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f461c102e00 0x7f461c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 >> 172.21.15.26:0/1657775300 conn(0x7f461c0793f0 msgr2=0x7f461c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.954+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 shutdown_connections 2024-05-20T20:58:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:49.955+0000 7f462426e700 1 -- 172.21.15.26:0/1657775300 wait complete. 2024-05-20T20:58:50.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 35957 3.79 MB/sec warmup 56 sec latency 12.604 ms 2024-05-20T20:58:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:51 smithi144 ceph-mon[35660]: pgmap v469: 97 pgs: 97 active+clean; 30 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-20T20:58:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:51 smithi055 ceph-mon[35213]: pgmap v469: 97 pgs: 97 active+clean; 30 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-20T20:58:51.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:51 smithi026 ceph-mon[31990]: pgmap v469: 97 pgs: 97 active+clean; 30 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-20T20:58:51.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 36576 3.76 MB/sec warmup 57 sec latency 18.928 ms 2024-05-20T20:58:51.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:58:51.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:58:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:52.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 37328 3.75 MB/sec warmup 58 sec latency 15.283 ms 2024-05-20T20:58:53.279 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:53 smithi026 ceph-mon[31990]: pgmap v470: 97 pgs: 97 active+clean; 24 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-20T20:58:53.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:53 smithi144 ceph-mon[35660]: pgmap v470: 97 pgs: 97 active+clean; 24 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-20T20:58:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:53 smithi055 ceph-mon[35213]: pgmap v470: 97 pgs: 97 active+clean; 24 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-20T20:58:53.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 37993 3.73 MB/sec warmup 59 sec latency 19.533 ms 2024-05-20T20:58:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 -- 172.21.15.26:0/1722774784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 msgr2=0x7fd68c1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 --2- 172.21.15.26:0/1722774784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c1030c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009990 tx=0x7fd67c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 -- 172.21.15.26:0/1722774784 shutdown_connections 2024-05-20T20:58:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 --2- 172.21.15.26:0/1722774784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 0x7fd68c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 --2- 172.21.15.26:0/1722774784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 0x7fd68c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 --2- 172.21.15.26:0/1722774784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c1030c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 -- 172.21.15.26:0/1722774784 >> 172.21.15.26:0/1722774784 conn(0x7fd68c078ab0 msgr2=0x7fd68c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 -- 172.21.15.26:0/1722774784 shutdown_connections 2024-05-20T20:58:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.860+0000 7fd693652700 1 -- 172.21.15.26:0/1722774784 wait complete. 2024-05-20T20:58:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 Processor -- start 2024-05-20T20:58:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 -- start start 2024-05-20T20:58:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 0x7fd68c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 0x7fd68c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c112480 con 0x7fd68c102c40 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd6913ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c112300 con 0x7fd68c1084c0 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd693652700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c112600 con 0x7fd68c103600 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.861+0000 7fd6913ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42334/0 (socket says 172.21.15.26:42334) 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd6913ee700 1 -- 172.21.15.26:0/429855030 learned_addr learned my addr 172.21.15.26:0/429855030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 0x7fd68c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd690bed700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 0x7fd68c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 -- 172.21.15.26:0/429855030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 msgr2=0x7fd68c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 0x7fd68c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 -- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 msgr2=0x7fd68c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 -- 172.21.15.26:0/429855030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c19a090 con 0x7fd68c1084c0 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd690bed700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 0x7fd68c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd691bef700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 0x7fd68c199930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd68400ab40 tx=0x7fd68400ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd6827fc700 1 -- 172.21.15.26:0/429855030 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684017020 con 0x7fd68c1084c0 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c19a380 con 0x7fd68c1084c0 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.862+0000 7fd6827fc700 1 -- 172.21.15.26:0/429855030 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd684004bb0 con 0x7fd68c1084c0 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.863+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1a65f0 con 0x7fd68c1084c0 2024-05-20T20:58:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.863+0000 7fd6827fc700 1 -- 172.21.15.26:0/429855030 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684005280 con 0x7fd68c1084c0 2024-05-20T20:58:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.865+0000 7fd6827fc700 1 -- 172.21.15.26:0/429855030 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd68400c580 con 0x7fd68c1084c0 2024-05-20T20:58:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.866+0000 7fd6827fc700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6780b2900 0x7fd6780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.866+0000 7fd6913ee700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6780b2900 0x7fd6780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.866+0000 7fd6827fc700 1 -- 172.21.15.26:0/429855030 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd6840e1760 con 0x7fd68c1084c0 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 Processor -- start 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 -- start start 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd6913ee700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6780b2900 0x7fd6780b4dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1075d0 tx=0x7fd688009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 0x55c176b98780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b932e0 0x55c176b9f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c176af0df0 con 0x55c176b983a0 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c176ae0990 con 0x55c176b932e0 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c176ae49d0 con 0x55c176b92920 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd6913ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 0x55c176b98780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd6913ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 0x55c176b98780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42350/0 (socket says 172.21.15.26:42350) 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd6913ee700 1 -- 172.21.15.26:0/2452946971 learned_addr learned my addr 172.21.15.26:0/2452946971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd691bef700 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b932e0 0x55c176b9f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd690bed700 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd691bef700 1 -- 172.21.15.26:0/2452946971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 msgr2=0x55c176b92da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd691bef700 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b92da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd691bef700 1 -- 172.21.15.26:0/2452946971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 msgr2=0x55c176b98780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd691bef700 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 0x55c176b98780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd691bef700 1 -- 172.21.15.26:0/2452946971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c176a8a030 con 0x55c176b932e0 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.867+0000 7fd6913ee700 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 0x55c176b98780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.868+0000 7fd691bef700 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b932e0 0x55c176b9f100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6840df200 tx=0x7fd6840a8cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=214c51aeaddc8d65 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.868+0000 7fd6777fe700 1 -- 172.21.15.26:0/2452946971 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840bf030 con 0x55c176b932e0 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.868+0000 7fd6777fe700 1 -- 172.21.15.26:0/2452946971 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6840ac020 con 0x55c176b932e0 2024-05-20T20:58:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.868+0000 7fd6777fe700 1 -- 172.21.15.26:0/2452946971 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684004e60 con 0x55c176b932e0 2024-05-20T20:58:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 -- 172.21.15.26:0/2452946971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b932e0 msgr2=0x55c176b9f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b932e0 0x55c176b9f100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6840df200 tx=0x7fd6840a8cb0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 -- 172.21.15.26:0/2452946971 shutdown_connections 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b983a0 0x55c176b98780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b932e0 0x55c176b9f100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/2452946971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b92da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 -- 172.21.15.26:0/2452946971 >> 172.21.15.26:0/2452946971 conn(0x55c176abe2c0 msgr2=0x55c176b8d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 -- 172.21.15.26:0/2452946971 shutdown_connections 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 -- 172.21.15.26:0/2452946971 wait complete. 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 -- start start 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.869+0000 7fd69dee8b80 1 Processor -- start 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b9ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c1769e7020 con 0x55c176b932e0 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c176a48e20 con 0x55c176b983a0 2024-05-20T20:58:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c176b9f760 con 0x55c176b92920 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd69dee8b80 10 client.0 _ll_register_callbacks cb 0x55c176a7b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd690bed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd691bef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd690bed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42362/0 (socket says 172.21.15.26:42362) 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd690bed700 1 -- 172.21.15.26:0/1857598107 learned_addr learned my addr 172.21.15.26:0/1857598107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd6913ee700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b9ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd691bef700 1 -- 172.21.15.26:0/1857598107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 msgr2=0x55c176b9ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd691bef700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b9ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd691bef700 1 -- 172.21.15.26:0/1857598107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 msgr2=0x55c176b9c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd691bef700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd691bef700 1 -- 172.21.15.26:0/1857598107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c176a30950 con 0x55c176b983a0 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.870+0000 7fd690bed700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd691bef700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9c620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd684007fd0 tx=0x7fd6840ad230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2496be77370a9c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd674ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd674ff9700 1 -- 172.21.15.26:0/1857598107 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840c7070 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd674ff9700 1 -- 172.21.15.26:0/1857598107 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd6840a53a0 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6840ac020 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd674ff9700 1 -- 172.21.15.26:0/1857598107 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6840bf030 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd69dee8b80 10 client.25778 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c176a9a7e0 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd674ff9700 1 -- 172.21.15.26:0/1857598107 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6840ac020 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.871+0000 7fd674ff9700 1 -- 172.21.15.26:0/1857598107 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd68408b020 con 0x55c176b983a0 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd69dee8b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd69dee8b80 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd69dee8b80 20 client.25778 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176bae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd69dee8b80 4 client.25778 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c176af2d40 con 0x55c176bac170 2024-05-20T20:58:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd6913ee700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176bae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd674ff9700 10 client.25778 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.872+0000 7fd6913ee700 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176bae560 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6c6169ab2edf88 server_cookie=d3fde88fb56a5391 in_seq=0 out_seq=0 2024-05-20T20:58:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd674ff9700 1 -- 172.21.15.26:0/1857598107 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c176af2d40 con 0x55c176bac170 2024-05-20T20:58:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd674ff9700 10 client.25778 handle_command_reply: tid=0 2024-05-20T20:58:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 client.25778 shutdown 2024-05-20T20:58:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 msgr2=0x55c176b9c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9c620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd684007fd0 tx=0x7fd6840ad230 comp rx=0 tx=0).stop 2024-05-20T20:58:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 shutdown_connections 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176bae560 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9c620 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd674ff9700 0 client.25778 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176b9ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1857598107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.873+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 >> 172.21.15.26:0/1857598107 conn(0x55c176abe2c0 msgr2=0x55c176b8cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.874+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 shutdown_connections 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.874+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1857598107 wait complete. 2024-05-20T20:58:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.874+0000 7fd69dee8b80 20 client.25778 trim_cache size 0 max 16384 2024-05-20T20:58:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 Processor -- start 2024-05-20T20:58:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 -- start start 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 0x55c176b8d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b932e0 0x55c176b94dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b95310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1ae320 con 0x55c176b92920 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1abbb0 con 0x55c176b983a0 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1a9a00 con 0x55c176b932e0 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd6913ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 0x55c176b8d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd6913ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 0x55c176b8d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42364/0 (socket says 172.21.15.26:42364) 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd6913ee700 1 -- 172.21.15.26:0/3896692786 learned_addr learned my addr 172.21.15.26:0/3896692786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd691bef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b95310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd690bed700 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b932e0 0x55c176b94dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd691bef700 1 -- 172.21.15.26:0/3896692786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b932e0 msgr2=0x55c176b94dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd691bef700 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b932e0 0x55c176b94dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd691bef700 1 -- 172.21.15.26:0/3896692786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 msgr2=0x55c176b8d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd691bef700 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 0x55c176b8d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd691bef700 1 -- 172.21.15.26:0/3896692786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c176b9cfe0 con 0x55c176b983a0 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.900+0000 7fd6913ee700 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 0x55c176b8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.901+0000 7fd691bef700 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b95310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd68400cba0 tx=0x7fd68400a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2471f393d8958bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.901+0000 7fd676ffd700 1 -- 172.21.15.26:0/3896692786 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840c7030 con 0x55c176b983a0 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.901+0000 7fd676ffd700 1 -- 172.21.15.26:0/3896692786 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6840ac070 con 0x55c176b983a0 2024-05-20T20:58:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.901+0000 7fd676ffd700 1 -- 172.21.15.26:0/3896692786 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840a9010 con 0x55c176b983a0 2024-05-20T20:58:53.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 -- 172.21.15.26:0/3896692786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 msgr2=0x55c176b95310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b95310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd68400cba0 tx=0x7fd68400a370 comp rx=0 tx=0).stop 2024-05-20T20:58:53.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 -- 172.21.15.26:0/3896692786 shutdown_connections 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b92920 0x55c176b8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b95310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/3896692786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b932e0 0x55c176b94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 -- 172.21.15.26:0/3896692786 >> 172.21.15.26:0/3896692786 conn(0x55c176abe2c0 msgr2=0x55c176b921a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 -- 172.21.15.26:0/3896692786 shutdown_connections 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 -- 172.21.15.26:0/3896692786 wait complete. 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 -- start start 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.902+0000 7fd69dee8b80 1 Processor -- start 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176ba5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1aa070 con 0x55c176b932e0 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1ab4e0 con 0x55c176b983a0 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1aa740 con 0x55c176b92920 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd69dee8b80 10 client.0 _ll_register_callbacks cb 0x55c176a7b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd691bef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd690bed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd691bef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51256/0 (socket says 172.21.15.26:51256) 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd691bef700 1 -- 172.21.15.26:0/1608321628 learned_addr learned my addr 172.21.15.26:0/1608321628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.903+0000 7fd6913ee700 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176ba5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd691bef700 1 -- 172.21.15.26:0/1608321628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 msgr2=0x55c176ba5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd691bef700 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176ba5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd691bef700 1 -- 172.21.15.26:0/1608321628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 msgr2=0x55c176b9f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd691bef700 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd691bef700 1 -- 172.21.15.26:0/1608321628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c176c16a30 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd691bef700 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9ffa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6840a6020 tx=0x7fd6840a8c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde8a1e9701c19fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd674ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd674ff9700 1 -- 172.21.15.26:0/1608321628 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840c7030 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6840ac070 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd674ff9700 1 -- 172.21.15.26:0/1608321628 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd6840a53a0 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd674ff9700 1 -- 172.21.15.26:0/1608321628 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6840b1b80 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd69dee8b80 10 client.25784 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c176c16a30 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.904+0000 7fd674ff9700 1 -- 172.21.15.26:0/1608321628 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6840ac070 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd674ff9700 1 -- 172.21.15.26:0/1608321628 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd68408b070 con 0x55c176b983a0 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd69dee8b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:53.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd69dee8b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd69dee8b80 20 client.25784 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176c19e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd69dee8b80 4 client.25784 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c176aec520 con 0x55c176bac170 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd6913ee700 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176c19e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd674ff9700 10 client.25784 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.905+0000 7fd6913ee700 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176c19e40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c74d59b8315b6e2 server_cookie=cd2fe3fe51868607 in_seq=0 out_seq=0 2024-05-20T20:58:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.906+0000 7fd674ff9700 1 -- 172.21.15.26:0/1608321628 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c176aec520 con 0x55c176bac170 2024-05-20T20:58:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.906+0000 7fd674ff9700 10 client.25784 handle_command_reply: tid=0 2024-05-20T20:58:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.906+0000 7fd69dee8b80 1 client.25784 shutdown 2024-05-20T20:58:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 msgr2=0x55c176b9ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9ffa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6840a6020 tx=0x7fd6840a8c50 comp rx=0 tx=0).stop 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 shutdown_connections 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c176bac170 0x55c176c19e40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c176b983a0 0x55c176b9ffa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c176b92920 0x55c176ba5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd674ff9700 0 client.25784 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 --2- 172.21.15.26:0/1608321628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c176b932e0 0x55c176b9f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 >> 172.21.15.26:0/1608321628 conn(0x55c176abe2c0 msgr2=0x55c176b95d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 shutdown_connections 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 1 -- 172.21.15.26:0/1608321628 wait complete. 2024-05-20T20:58:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.907+0000 7fd69dee8b80 20 client.25784 trim_cache size 0 max 16384 2024-05-20T20:58:53.912 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:58:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:58:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "ffe90122-dff6-4469-acb6-e26aa1aabfbb", 2024-05-20T20:58:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:58:53.912 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6780b2900 msgr2=0x7fd6780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6780b2900 0x7fd6780b4dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1075d0 tx=0x7fd688009260 comp rx=0 tx=0).stop 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 msgr2=0x7fd68c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 0x7fd68c199930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd68400ab40 tx=0x7fd68400ae50 comp rx=0 tx=0).stop 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 shutdown_connections 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6780b2900 0x7fd6780b4dc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd68c102c40 0x7fd68c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd68c1084c0 0x7fd68c199930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 --2- 172.21.15.26:0/429855030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103600 0x7fd68c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 >> 172.21.15.26:0/429855030 conn(0x7fd68c078ab0 msgr2=0x7fd68c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 shutdown_connections 2024-05-20T20:58:53.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:53.908+0000 7fd693652700 1 -- 172.21.15.26:0/429855030 wait complete. 2024-05-20T20:58:54.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 38810 3.78 MB/sec warmup 60 sec latency 23.256 ms 2024-05-20T20:58:54.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:58:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:55 smithi026 ceph-mon[31990]: pgmap v471: 97 pgs: 97 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-05-20T20:58:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:55 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:58:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:55 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:55 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:58:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:55 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:55.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:55 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:58:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:55 smithi144 ceph-mon[35660]: pgmap v471: 97 pgs: 97 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-05-20T20:58:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:55 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:58:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:55 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:55 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:58:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:55 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:55 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:58:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:55 smithi055 ceph-mon[35213]: pgmap v471: 97 pgs: 97 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-05-20T20:58:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:55 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:58:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:55 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:55 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:58:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:55 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:58:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:55 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:58:55.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 39279 3.79 MB/sec warmup 61 sec latency 22.295 ms 2024-05-20T20:58:56.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:58:56.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:58:56.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:58:56.429 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:58:56.430 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:58:56.430 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:58:56.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:58:56.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:58:56.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:58:56.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 39658 3.73 MB/sec warmup 62 sec latency 85.659 ms 2024-05-20T20:58:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 -- 172.21.15.26:0/3398935817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94102930 msgr2=0x7fef94110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 --2- 172.21.15.26:0/3398935817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94102930 0x7fef94110430 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b0a0 tx=0x7fef8c00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 -- 172.21.15.26:0/3398935817 shutdown_connections 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 --2- 172.21.15.26:0/3398935817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94102930 0x7fef94110430 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 --2- 172.21.15.26:0/3398935817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94109720 0x7fef94109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 --2- 172.21.15.26:0/3398935817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94101f70 0x7fef941023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 -- 172.21.15.26:0/3398935817 >> 172.21.15.26:0/3398935817 conn(0x7fef940776b0 msgr2=0x7fef94077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 -- 172.21.15.26:0/3398935817 shutdown_connections 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 -- 172.21.15.26:0/3398935817 wait complete. 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.070+0000 7fef993c1700 1 Processor -- start 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 -- start start 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94101f70 0x7fef94103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 0x7fef94103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 0x7fef94103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef94113600 con 0x7fef94109720 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef94113480 con 0x7fef94102930 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef993c1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef94113780 con 0x7fef94101f70 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 0x7fef94103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef927fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 0x7fef94103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef92ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94101f70 0x7fef94103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 0x7fef94103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42390/0 (socket says 172.21.15.26:42390) 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef927fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 0x7fef94103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51266/0 (socket says 172.21.15.26:51266) 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 -- 172.21.15.26:0/1745614196 learned_addr learned my addr 172.21.15.26:0/1745614196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 -- 172.21.15.26:0/1745614196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94101f70 msgr2=0x7fef94103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94101f70 0x7fef94103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 -- 172.21.15.26:0/1745614196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 msgr2=0x7fef94103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 0x7fef94103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.071+0000 7fef937fe700 1 -- 172.21.15.26:0/1745614196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef941a80e0 con 0x7fef94109720 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef927fc700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 0x7fef94103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef937fe700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 0x7fef94103bd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b780 tx=0x7fef8c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef7bfff700 1 -- 172.21.15.26:0/1745614196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c021070 con 0x7fef94109720 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef941a8370 con 0x7fef94109720 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef7bfff700 1 -- 172.21.15.26:0/1745614196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef8c007b80 con 0x7fef94109720 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef7bfff700 1 -- 172.21.15.26:0/1745614196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0052a0 con 0x7fef94109720 2024-05-20T20:58:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.072+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941a8820 con 0x7fef94109720 2024-05-20T20:58:57.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.075+0000 7fef7bfff700 1 -- 172.21.15.26:0/1745614196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fef8c024070 con 0x7fef94109720 2024-05-20T20:58:57.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fef7bfff700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef7c0b2880 0x7fef7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef7c0b2880 0x7fef7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fef7bfff700 1 -- 172.21.15.26:0/1745614196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fef8c00f020 con 0x7fef94109720 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fefa3c57b80 1 Processor -- start 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fefa3c57b80 1 -- start start 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef7c0b2880 0x7fef7c0b4d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef84007b60 tx=0x7fef840073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a26052f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a260529d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.076+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260536080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a260487d70 con 0x55a26052a260 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a260477910 con 0x55a2605298a0 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a26047b950 con 0x55a26052f320 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a260529d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260536080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a260529d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51274/0 (socket says 172.21.15.26:51274) 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 -- 172.21.15.26:0/3953655603 learned_addr learned my addr 172.21.15.26:0/3953655603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef92ffd700 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a26052f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 -- 172.21.15.26:0/3953655603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 msgr2=0x55a26052f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a26052f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 -- 172.21.15.26:0/3953655603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 msgr2=0x55a260536080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260536080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 -- 172.21.15.26:0/3953655603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a260420fb0 con 0x55a2605298a0 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef937fe700 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260536080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.077+0000 7fef927fc700 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a260529d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef88011540 tx=0x7fef88011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed0d11e6360c83b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fef797fa700 1 -- 172.21.15.26:0/3953655603 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88004a30 con 0x55a2605298a0 2024-05-20T20:58:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fef797fa700 1 -- 172.21.15.26:0/3953655603 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef880054b0 con 0x55a2605298a0 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fef797fa700 1 -- 172.21.15.26:0/3953655603 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e910 con 0x55a2605298a0 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 -- 172.21.15.26:0/3953655603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 msgr2=0x55a260529d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a260529d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef88011540 tx=0x7fef88011850 comp rx=0 tx=0).stop 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 -- 172.21.15.26:0/3953655603 shutdown_connections 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260536080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a26052f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/3953655603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a260529d20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.078+0000 7fefa3c57b80 1 -- 172.21.15.26:0/3953655603 >> 172.21.15.26:0/3953655603 conn(0x55a260455240 msgr2=0x55a2605243e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 -- 172.21.15.26:0/3953655603 shutdown_connections 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 -- 172.21.15.26:0/3953655603 wait complete. 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 -- start start 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 Processor -- start 2024-05-20T20:58:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260535d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260533000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260533540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a26037dfa0 con 0x55a26052a260 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a2603dfda0 con 0x55a26052f320 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2605366e0 con 0x55a2605298a0 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.079+0000 7fefa3c57b80 10 client.0 _ll_register_callbacks cb 0x55a260412950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef927fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260533000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef927fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260533000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42412/0 (socket says 172.21.15.26:42412) 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef927fc700 1 -- 172.21.15.26:0/1044101004 learned_addr learned my addr 172.21.15.26:0/1044101004 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260535d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef937fe700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260533540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 -- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 msgr2=0x55a260533540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260533540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 -- 172.21.15.26:0/1044101004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 msgr2=0x55a260533000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260533000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 -- 172.21.15.26:0/1044101004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2603c78d0 con 0x55a2605298a0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef927fc700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260533000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef937fe700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260533540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef92ffd700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260535d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef84007fd0 tx=0x7fef840047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af65c00744b019c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.080+0000 7fef76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fef76ff5700 1 -- 172.21.15.26:0/1044101004 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84011e00 con 0x55a2605298a0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fef76ff5700 1 -- 172.21.15.26:0/1044101004 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fef84028470 con 0x55a2605298a0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef880054b0 con 0x55a2605298a0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fef76ff5700 1 -- 172.21.15.26:0/1044101004 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef840286b0 con 0x55a2605298a0 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 10 client.25720 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a260431760 con 0x55a2605298a0 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fef76ff5700 1 -- 172.21.15.26:0/1044101004 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef880054b0 con 0x55a2605298a0 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fef76ff5700 1 -- 172.21.15.26:0/1044101004 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef84031050 con 0x55a2605298a0 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 10 client.25720 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 20 client.25720 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a260543200 0x55a260545640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 4 client.25720 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.081+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a260489cc0 con 0x55a260543200 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.082+0000 7fef927fc700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a260543200 0x55a260545640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.082+0000 7fef927fc700 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a260543200 0x55a260545640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404318b7e90c540e server_cookie=c4557b80feec102b in_seq=0 out_seq=0 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.082+0000 7fef76ff5700 10 client.25720 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fef76ff5700 1 -- 172.21.15.26:0/1044101004 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a260489cc0 con 0x55a260543200 2024-05-20T20:58:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fef76ff5700 10 client.25720 handle_command_reply: tid=0 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 client.25720 shutdown 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 msgr2=0x55a260535d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260535d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef84007fd0 tx=0x7fef840047c0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 shutdown_connections 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a260543200 0x55a260545640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260533540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260535d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fef76ff5700 0 client.25720 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/1044101004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260533000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 >> 172.21.15.26:0/1044101004 conn(0x55a260455240 msgr2=0x55a260523af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 shutdown_connections 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 1 -- 172.21.15.26:0/1044101004 wait complete. 2024-05-20T20:58:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.083+0000 7fefa3c57b80 20 client.25720 trim_cache size 0 max 16384 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 Processor -- start 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 -- start start 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260529120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260524270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260530010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef941b26d0 con 0x55a26052a260 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef941aff60 con 0x55a26052f320 2024-05-20T20:58:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef941addb0 con 0x55a2605298a0 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260524270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260524270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42422/0 (socket says 172.21.15.26:42422) 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 -- 172.21.15.26:0/2571607042 learned_addr learned my addr 172.21.15.26:0/2571607042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef92ffd700 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260529120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef937fe700 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260530010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 -- 172.21.15.26:0/2571607042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 msgr2=0x55a260529120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260529120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 -- 172.21.15.26:0/2571607042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 msgr2=0x55a260530010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260530010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.106+0000 7fef927fc700 1 -- 172.21.15.26:0/2571607042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2605338c0 con 0x55a26052a260 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.107+0000 7fef927fc700 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260524270 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef88011460 tx=0x7fef88011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39009f36cba7e8ab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.107+0000 7fef78ff9700 1 -- 172.21.15.26:0/2571607042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88031070 con 0x55a26052a260 2024-05-20T20:58:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.107+0000 7fef78ff9700 1 -- 172.21.15.26:0/2571607042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88003680 con 0x55a26052a260 2024-05-20T20:58:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.107+0000 7fef78ff9700 1 -- 172.21.15.26:0/2571607042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88004fa0 con 0x55a26052a260 2024-05-20T20:58:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 -- 172.21.15.26:0/2571607042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 msgr2=0x55a260524270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260524270 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef88011460 tx=0x7fef88011850 comp rx=0 tx=0).stop 2024-05-20T20:58:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 -- 172.21.15.26:0/2571607042 shutdown_connections 2024-05-20T20:58:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a26052f320 0x55a260530010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2605298a0 0x55a260529120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/2571607042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a260524270 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 -- 172.21.15.26:0/2571607042 >> 172.21.15.26:0/2571607042 conn(0x55a260455240 msgr2=0x55a26052e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 -- 172.21.15.26:0/2571607042 shutdown_connections 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 -- 172.21.15.26:0/2571607042 wait complete. 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 -- start start 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.108+0000 7fefa3c57b80 1 Processor -- start 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a26053b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a26053be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a2605368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef941ae420 con 0x55a26052a260 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef941af890 con 0x55a2605298a0 2024-05-20T20:58:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef941aeaf0 con 0x55a26052f320 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef927fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a26053be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fefa3c57b80 10 client.0 _ll_register_callbacks cb 0x55a260412950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef927fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a26053be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42438/0 (socket says 172.21.15.26:42438) 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef927fc700 1 -- 172.21.15.26:0/600043017 learned_addr learned my addr 172.21.15.26:0/600043017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a26053b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef937fe700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a2605368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 -- 172.21.15.26:0/600043017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 msgr2=0x55a2605368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a2605368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 -- 172.21.15.26:0/600043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 msgr2=0x55a26053be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a26053be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 -- 172.21.15.26:0/600043017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a260536e30 con 0x55a2605298a0 2024-05-20T20:58:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef927fc700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a26053be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef92ffd700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a26053b940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef8401fb20 tx=0x7fef84007660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf37d363d752183d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.109+0000 7fef76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fef76ff5700 1 -- 172.21.15.26:0/600043017 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8401f630 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef88003680 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fef76ff5700 1 -- 172.21.15.26:0/600043017 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fef84004030 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fef76ff5700 1 -- 172.21.15.26:0/600043017 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef84004230 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 10 client.25802 fetch_fsmap learned FSMap version 26 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2605ae4f0 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fef76ff5700 1 -- 172.21.15.26:0/600043017 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88003680 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fef76ff5700 1 -- 172.21.15.26:0/600043017 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef84031040 con 0x55a2605298a0 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 20 client.25802 populate_metadata read hostname 'smithi026' 2024-05-20T20:58:57.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2605aebd0 0x55a2605b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 4 client.25802 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.110+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2604834a0 con 0x55a2605aebd0 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.111+0000 7fef927fc700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2605aebd0 0x55a2605b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.111+0000 7fef927fc700 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2605aebd0 0x55a2605b0ff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7dd44045fc6c097 server_cookie=fc97948a14d7a4c5 in_seq=0 out_seq=0 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.111+0000 7fef76ff5700 10 client.25802 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.111+0000 7fef76ff5700 1 -- 172.21.15.26:0/600043017 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2604834a0 con 0x55a2605aebd0 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.111+0000 7fef76ff5700 10 client.25802 handle_command_reply: tid=0 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.111+0000 7fefa3c57b80 1 client.25802 shutdown 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 msgr2=0x55a26053b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a26053b940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef8401fb20 tx=0x7fef84007660 comp rx=0 tx=0).stop 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 shutdown_connections 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a2605aebd0 0x55a2605b0ff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a26052f320 0x55a2605368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fef76ff5700 0 client.25802 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2605298a0 0x55a26053b940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 --2- 172.21.15.26:0/600043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26052a260 0x55a26053be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 >> 172.21.15.26:0/600043017 conn(0x55a260455240 msgr2=0x55a260533140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 shutdown_connections 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 1 -- 172.21.15.26:0/600043017 wait complete. 2024-05-20T20:58:57.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fefa3c57b80 20 client.25802 trim_cache size 0 max 16384 2024-05-20T20:58:57.118 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:58:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:58:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:58:57.118 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef7c0b2880 msgr2=0x7fef7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fef993c1700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef7c0b2880 0x7fef7c0b4d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef84007b60 tx=0x7fef840073d0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 msgr2=0x7fef94103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.112+0000 7fef993c1700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 0x7fef94103bd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b780 tx=0x7fef8c004d00 comp rx=0 tx=0).stop 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 shutdown_connections 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef7c0b2880 0x7fef7c0b4d40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef94109720 0x7fef94103bd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94101f70 0x7fef94103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 --2- 172.21.15.26:0/1745614196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef94102930 0x7fef94103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 >> 172.21.15.26:0/1745614196 conn(0x7fef940776b0 msgr2=0x7fef94100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 shutdown_connections 2024-05-20T20:58:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:58:57.113+0000 7fef993c1700 1 -- 172.21.15.26:0/1745614196 wait complete. 2024-05-20T20:58:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:57 smithi026 ceph-mon[31990]: pgmap v472: 97 pgs: 97 active+clean; 25 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-05-20T20:58:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:57 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:58:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:57 smithi144 ceph-mon[35660]: pgmap v472: 97 pgs: 97 active+clean; 25 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-05-20T20:58:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:57 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:58:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:57 smithi055 ceph-mon[35213]: pgmap v472: 97 pgs: 97 active+clean; 25 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-05-20T20:58:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:57 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:58:57.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 40306 3.71 MB/sec warmup 63 sec latency 19.612 ms 2024-05-20T20:58:58.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:58:58.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:58:58.310 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:58:58.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 41039 3.70 MB/sec warmup 64 sec latency 17.690 ms 2024-05-20T20:58:58.855 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-20T20:58:58.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:58:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:58:59 smithi144 ceph-mon[35660]: pgmap v473: 97 pgs: 97 active+clean; 26 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T20:58:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:58:59 smithi055 ceph-mon[35213]: pgmap v473: 97 pgs: 97 active+clean; 26 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T20:58:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:58:59 smithi026 ceph-mon[31990]: pgmap v473: 97 pgs: 97 active+clean; 26 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T20:58:59.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 41660 3.68 MB/sec warmup 65 sec latency 50.507 ms 2024-05-20T20:59:00.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 42366 3.72 MB/sec warmup 66 sec latency 85.838 ms 2024-05-20T20:59:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:01 smithi144 ceph-mon[35660]: pgmap v474: 97 pgs: 97 active+clean; 27 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-20T20:59:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:01 smithi055 ceph-mon[35213]: pgmap v474: 97 pgs: 97 active+clean; 27 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-20T20:59:01.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:01 smithi026 ceph-mon[31990]: pgmap v474: 97 pgs: 97 active+clean; 27 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-20T20:59:01.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 42890 3.73 MB/sec warmup 67 sec latency 60.614 ms 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- 172.21.15.26:0/2769383916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0103610 msgr2=0x7f28f0109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/2769383916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0103610 0x7f28f0109ea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009ca0 tx=0x7f28e802a8c0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- 172.21.15.26:0/2769383916 shutdown_connections 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/2769383916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0103610 0x7f28f0109ea0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/2769383916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0102c50 0x7f28f01030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/2769383916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0101a60 0x7f28f0101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- 172.21.15.26:0/2769383916 >> 172.21.15.26:0/2769383916 conn(0x7f28f00fcfd0 msgr2=0x7f28f00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- 172.21.15.26:0/2769383916 shutdown_connections 2024-05-20T20:59:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- 172.21.15.26:0/2769383916 wait complete. 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 Processor -- start 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- start start 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 0x7f28f0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0102c50 0x7f28f0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0103610 0x7f28f01a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.153+0000 7f28f5cd1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010d420 con 0x7f28f0101a60 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28f5cd1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010d2a0 con 0x7f28f0103610 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28f5cd1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010d5a0 con 0x7f28f0102c50 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 0x7f28f0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 0x7f28f0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34720/0 (socket says 172.21.15.26:34720) 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 -- 172.21.15.26:0/1234523098 learned_addr learned my addr 172.21.15.26:0/1234523098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 -- 172.21.15.26:0/1234523098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0102c50 msgr2=0x7f28f0199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28effff640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0103610 0x7f28f01a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28e7fff640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0102c50 0x7f28f0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0102c50 0x7f28f0199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 -- 172.21.15.26:0/1234523098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0103610 msgr2=0x7f28f01a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0103610 0x7f28f01a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 -- 172.21.15.26:0/1234523098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f01a0750 con 0x7f28f0101a60 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28effff640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0103610 0x7f28f01a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28e7fff640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0102c50 0x7f28f0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ef7fe640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 0x7f28f0198b30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f28e000d950 tx=0x7f28e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ed7fa640 1 -- 172.21.15.26:0/1234523098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0017070 con 0x7f28f0101a60 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01a0a40 con 0x7f28f0101a60 2024-05-20T20:59:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ed7fa640 1 -- 172.21.15.26:0/1234523098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28e0009c30 con 0x7f28f0101a60 2024-05-20T20:59:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28ed7fa640 1 -- 172.21.15.26:0/1234523098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e000f620 con 0x7f28f0101a60 2024-05-20T20:59:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.154+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a0f40 con 0x7f28f0101a60 2024-05-20T20:59:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28ed7fa640 1 -- 172.21.15.26:0/1234523098 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f28e001a070 con 0x7f28f0101a60 2024-05-20T20:59:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 Processor -- start 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 -- start start 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db611ad0 0x5636db611ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db6124c0 0x5636db616fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 0x5636db619ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636db57ec30 con 0x5636db617630 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5636db56e7d0 con 0x5636db6124c0 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28f89ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636db572810 con 0x5636db611ad0 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 0x5636db619ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28effff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db611ad0 0x5636db611ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 0x5636db619ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34722/0 (socket says 172.21.15.26:34722) 2024-05-20T20:59:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28e7fff640 1 -- 172.21.15.26:0/280621010 learned_addr learned my addr 172.21.15.26:0/280621010 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.157+0000 7f28ef7fe640 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db6124c0 0x5636db616fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28effff640 1 -- 172.21.15.26:0/280621010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db6124c0 msgr2=0x5636db616fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28effff640 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db6124c0 0x5636db616fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28effff640 1 -- 172.21.15.26:0/280621010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 msgr2=0x5636db619ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28effff640 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 0x5636db619ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28effff640 1 -- 172.21.15.26:0/280621010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636db283210 con 0x5636db611ad0 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28e7fff640 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 0x5636db619ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28ed7fa640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28dc0b28a0 0x7f28dc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28effff640 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db611ad0 0x5636db611ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28e8005e30 tx=0x7f28e8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f4cf0120b576c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28ef7fe640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28dc0b28a0 0x7f28dc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28ed7fa640 1 -- 172.21.15.26:0/1234523098 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f28e00e1d80 con 0x7f28f0101a60 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28e5ffb640 1 -- 172.21.15.26:0/280621010 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8031a80 con 0x5636db611ad0 2024-05-20T20:59:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28e5ffb640 1 -- 172.21.15.26:0/280621010 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28e8036070 con 0x5636db611ad0 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.158+0000 7f28e5ffb640 1 -- 172.21.15.26:0/280621010 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004460 con 0x5636db611ad0 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 -- 172.21.15.26:0/280621010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db611ad0 msgr2=0x5636db611ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28ef7fe640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28dc0b28a0 0x7f28dc0b4d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f28e00dfb90 tx=0x7f28e000bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db611ad0 0x5636db611ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28e8005e30 tx=0x7f28e8009a20 comp rx=0 tx=0).stop 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 -- 172.21.15.26:0/280621010 shutdown_connections 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db617630 0x5636db619ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db6124c0 0x5636db616fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 --2- 172.21.15.26:0/280621010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db611ad0 0x5636db611ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 -- 172.21.15.26:0/280621010 >> 172.21.15.26:0/280621010 conn(0x5636db608d30 msgr2=0x5636db609130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 -- 172.21.15.26:0/280621010 shutdown_connections 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 -- 172.21.15.26:0/280621010 wait complete. 2024-05-20T20:59:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 -- start start 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.159+0000 7f28f89ea740 1 Processor -- start 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db61bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db61c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db623110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636db254790 con 0x5636db611ad0 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5636db52a890 con 0x5636db617630 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636db61a580 con 0x5636db6124c0 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 10 client.0 _ll_register_callbacks cb 0x5636db282a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db61bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db61bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34736/0 (socket says 172.21.15.26:34736) 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 -- 172.21.15.26:0/2500819876 learned_addr learned my addr 172.21.15.26:0/2500819876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db623110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28e7fff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db61c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 -- 172.21.15.26:0/2500819876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 msgr2=0x5636db61c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db61c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 -- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 msgr2=0x5636db623110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db623110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 -- 172.21.15.26:0/2500819876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636db34f620 con 0x5636db611ad0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28e7fff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db61c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28effff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db61bb90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f28e8004af0 tx=0x7f28e80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5a1eed6fe43dba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28ceffd640 1 -- 172.21.15.26:0/2500819876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8057060 con 0x5636db611ad0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28ceffd640 1 -- 172.21.15.26:0/2500819876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f28e8004d40 con 0x5636db611ad0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.160+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e8036070 con 0x5636db611ad0 2024-05-20T20:59:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28ceffd640 1 -- 172.21.15.26:0/2500819876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28e8049040 con 0x5636db611ad0 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 10 client.16260 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636db6260f0 con 0x5636db611ad0 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28ceffd640 1 -- 172.21.15.26:0/2500819876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28e8036070 con 0x5636db611ad0 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28ceffd640 1 -- 172.21.15.26:0/2500819876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28e804f070 con 0x5636db611ad0 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 10 client.16260 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 20 client.16260 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db627440 0x5636db629830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 4 client.16260 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636db580b80 con 0x5636db627440 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.161+0000 7f28e7fff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db627440 0x5636db629830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28e7fff640 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db627440 0x5636db629830 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89c42e5b36a8793 server_cookie=d64f042ee1feaee7 in_seq=0 out_seq=0 2024-05-20T20:59:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28ceffd640 10 client.16260 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28ceffd640 1 -- 172.21.15.26:0/2500819876 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636db580b80 con 0x5636db627440 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28ceffd640 10 client.16260 handle_command_reply: tid=0 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 client.16260 shutdown 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 msgr2=0x5636db61bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db61bb90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f28e8004af0 tx=0x7f28e80095c0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 shutdown_connections 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db627440 0x5636db629830 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db623110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db61c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28ceffd640 0 client.16260 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.162+0000 7f28f89ea740 1 --2- 172.21.15.26:0/2500819876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db61bb90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.163+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 >> 172.21.15.26:0/2500819876 conn(0x5636db608d30 msgr2=0x5636db607fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.163+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 shutdown_connections 2024-05-20T20:59:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.163+0000 7f28f89ea740 1 -- 172.21.15.26:0/2500819876 wait complete. 2024-05-20T20:59:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.163+0000 7f28f89ea740 20 client.16260 trim_cache size 0 max 16384 2024-05-20T20:59:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 Processor -- start 2024-05-20T20:59:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 -- start start 2024-05-20T20:59:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db611ad0 0x5636db61c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 0x5636db6230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db617630 0x5636db60bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01aa530 con 0x5636db6124c0 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01a7e00 con 0x5636db611ad0 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28f89ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01a5c50 con 0x5636db617630 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 0x5636db6230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 0x5636db6230a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34740/0 (socket says 172.21.15.26:34740) 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 -- 172.21.15.26:0/3389262917 learned_addr learned my addr 172.21.15.26:0/3389262917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28effff640 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db611ad0 0x5636db61c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28effff640 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db617630 0x5636db60bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 -- 172.21.15.26:0/3389262917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db617630 msgr2=0x5636db60bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db617630 0x5636db60bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 -- 172.21.15.26:0/3389262917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db611ad0 msgr2=0x5636db61c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db611ad0 0x5636db61c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 -- 172.21.15.26:0/3389262917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636db629ba0 con 0x5636db6124c0 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.178+0000 7f28e7fff640 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 0x5636db6230a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x5636db62a000 tx=0x7f28d800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38109240afa55a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28e57fa640 1 -- 172.21.15.26:0/3389262917 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d801c6f0 con 0x5636db6124c0 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28e57fa640 1 -- 172.21.15.26:0/3389262917 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28d800fe50 con 0x5636db6124c0 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28e57fa640 1 -- 172.21.15.26:0/3389262917 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d800ec00 con 0x5636db6124c0 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 -- 172.21.15.26:0/3389262917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 msgr2=0x5636db6230a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 0x5636db6230a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x5636db62a000 tx=0x7f28d800f7f0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 -- 172.21.15.26:0/3389262917 shutdown_connections 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db617630 0x5636db60bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db6124c0 0x5636db6230a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 --2- 172.21.15.26:0/3389262917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db611ad0 0x5636db61c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 -- 172.21.15.26:0/3389262917 >> 172.21.15.26:0/3389262917 conn(0x5636db608d30 msgr2=0x5636db603e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 -- 172.21.15.26:0/3389262917 shutdown_connections 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.179+0000 7f28f89ea740 1 -- 172.21.15.26:0/3389262917 wait complete. 2024-05-20T20:59:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 -- start start 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 Processor -- start 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db6201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db6206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db621700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01a62c0 con 0x5636db611ad0 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01a7730 con 0x5636db617630 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01a6990 con 0x5636db6124c0 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28f89ea740 10 client.0 _ll_register_callbacks cb 0x7f28d800b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db6201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db6201b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34748/0 (socket says 172.21.15.26:34748) 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 -- 172.21.15.26:0/1192494402 learned_addr learned my addr 172.21.15.26:0/1192494402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db621700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 -- 172.21.15.26:0/1192494402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 msgr2=0x5636db6206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db6206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 -- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 msgr2=0x5636db621700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db621700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.180+0000 7f28effff640 1 -- 172.21.15.26:0/1192494402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636db621cf0 con 0x5636db611ad0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28effff640 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db6201b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009960 tx=0x7f28e80316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6357aca694edc4f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28ceffd640 1 -- 172.21.15.26:0/1192494402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80309d0 con 0x5636db611ad0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28ceffd640 1 -- 172.21.15.26:0/1192494402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f28e8030b70 con 0x5636db611ad0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d800fe50 con 0x5636db611ad0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28ceffd640 1 -- 172.21.15.26:0/1192494402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28e806d050 con 0x5636db611ad0 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 10 client.16266 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636db6230a0 con 0x5636db611ad0 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28ceffd640 1 -- 172.21.15.26:0/1192494402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28d800fe50 con 0x5636db611ad0 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28ceffd640 1 -- 172.21.15.26:0/1192494402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28e8065020 con 0x5636db611ad0 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 10 client.16266 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 20 client.16266 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db692440 0x5636db694860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 4 client.16266 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636db57a360 con 0x5636db692440 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.181+0000 7f28e7fff640 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db692440 0x5636db694860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.182+0000 7f28e7fff640 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db692440 0x5636db694860 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ea254a44c6b1e7 server_cookie=22e8499874cef6b0 in_seq=0 out_seq=0 2024-05-20T20:59:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.182+0000 7f28ceffd640 10 client.16266 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28ceffd640 1 -- 172.21.15.26:0/1192494402 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5636db57a360 con 0x5636db692440 2024-05-20T20:59:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28ceffd640 10 client.16266 handle_command_reply: tid=0 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 client.16266 shutdown 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 msgr2=0x5636db6201b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db6201b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009960 tx=0x7f28e80316b0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 shutdown_connections 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636db692440 0x5636db694860 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636db617630 0x5636db621700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636db6124c0 0x5636db6206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28ceffd640 0 client.16266 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 --2- 172.21.15.26:0/1192494402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636db611ad0 0x5636db6201b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 >> 172.21.15.26:0/1192494402 conn(0x5636db608d30 msgr2=0x5636db60bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 shutdown_connections 2024-05-20T20:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 1 -- 172.21.15.26:0/1192494402 wait complete. 2024-05-20T20:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.225+0000 7f28f89ea740 20 client.16266 trim_cache size 0 max 16384 2024-05-20T20:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28dc0b28a0 msgr2=0x7f28dc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28dc0b28a0 0x7f28dc0b4d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f28e00dfb90 tx=0x7f28e000bf60 comp rx=0 tx=0).stop 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 msgr2=0x7f28f0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 0x7f28f0198b30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f28e000d950 tx=0x7f28e000de10 comp rx=0 tx=0).stop 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 shutdown_connections 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28dc0b28a0 0x7f28dc0b4d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28f0103610 0x7f28f01a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28f0102c50 0x7f28f0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 --2- 172.21.15.26:0/1234523098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0101a60 0x7f28f0198b30 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 >> 172.21.15.26:0/1234523098 conn(0x7f28f00fcfd0 msgr2=0x7f28f00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 shutdown_connections 2024-05-20T20:59:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:02.226+0000 7f28f5cd1640 1 -- 172.21.15.26:0/1234523098 wait complete. 2024-05-20T20:59:02.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 43209 3.68 MB/sec warmup 68 sec latency 129.235 ms 2024-05-20T20:59:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:03 smithi144 ceph-mon[35660]: pgmap v475: 97 pgs: 97 active+clean; 28 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T20:59:03.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:03 smithi055 ceph-mon[35213]: pgmap v475: 97 pgs: 97 active+clean; 28 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T20:59:03.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:03 smithi026 ceph-mon[31990]: pgmap v475: 97 pgs: 97 active+clean; 28 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T20:59:03.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 43815 3.66 MB/sec warmup 69 sec latency 29.236 ms 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/974563340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6806bab0 msgr2=0x7f4a6806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/974563340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6806bab0 0x7f4a6806bf30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009990 tx=0x7f4a54009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/974563340 shutdown_connections 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/974563340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 0x7f4a681103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/974563340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a681095a0 0x7f4a68109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/974563340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6806bab0 0x7f4a6806bf30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/974563340 >> 172.21.15.26:0/974563340 conn(0x7f4a680793b0 msgr2=0x7f4a680797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/974563340 shutdown_connections 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.216+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/974563340 wait complete. 2024-05-20T20:59:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.217+0000 7f4a6ec8d700 1 Processor -- start 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 -- start start 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a6806bab0 0x7f4a6819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 0x7f4a6819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 0x7f4a68199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113580 con 0x7f4a681095a0 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113400 con 0x7f4a6806c470 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ec8d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113700 con 0x7f4a6806bab0 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6d22a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 0x7f4a68199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 0x7f4a6819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6d22a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 0x7f4a68199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34778/0 (socket says 172.21.15.26:34778) 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6d22a700 1 -- 172.21.15.26:0/15685699 learned_addr learned my addr 172.21.15.26:0/15685699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6ca29700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a6806bab0 0x7f4a6819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a67fff700 1 -- 172.21.15.26:0/15685699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a6806bab0 msgr2=0x7f4a6819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a67fff700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a6806bab0 0x7f4a6819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a67fff700 1 -- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 msgr2=0x7f4a68199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a67fff700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 0x7f4a68199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a67fff700 1 -- 172.21.15.26:0/15685699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819a0a0 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.218+0000 7f4a6d22a700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 0x7f4a68199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.219+0000 7f4a67fff700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 0x7f4a6819fcc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000c00 tx=0x7f4a540047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.219+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54026070 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.219+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6819a330 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.219+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a54021410 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.219+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54017650 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.219+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681a6650 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.220+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6806a230 con 0x7f4a6806c470 2024-05-20T20:59:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.222+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4a54029070 con 0x7f4a6806c470 2024-05-20T20:59:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.222+0000 7f4a65ffb700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4a580b2880 0x7f4a580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.223+0000 7f4a6ca29700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4a580b2880 0x7f4a580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.223+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f4a540e37f0 con 0x7f4a6806c470 2024-05-20T20:59:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.223+0000 7f4a6ca29700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4a580b2880 0x7f4a580b4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005950 tx=0x7f4a5c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.226+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a54030070 con 0x7f4a6806c470 2024-05-20T20:59:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:04.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 44576 3.66 MB/sec warmup 70 sec latency 21.273 ms 2024-05-20T20:59:04.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.569+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a6802e560 con 0x7f4a6806c470 2024-05-20T20:59:04.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.570+0000 7f4a65ffb700 1 -- 172.21.15.26:0/15685699 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f4a54013070 con 0x7f4a6806c470 2024-05-20T20:59:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:59:04.570 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4a580b2880 msgr2=0x7f4a580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4a580b2880 0x7f4a580b4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005950 tx=0x7f4a5c00a300 comp rx=0 tx=0).stop 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 msgr2=0x7f4a6819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 0x7f4a6819fcc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000c00 tx=0x7f4a540047b0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 shutdown_connections 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4a580b2880 0x7f4a580b4d40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a681095a0 0x7f4a68199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a6806bab0 0x7f4a6819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 --2- 172.21.15.26:0/15685699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6806c470 0x7f4a6819fcc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 >> 172.21.15.26:0/15685699 conn(0x7f4a680793b0 msgr2=0x7f4a68101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 shutdown_connections 2024-05-20T20:59:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:04.573+0000 7f4a6ec8d700 1 -- 172.21.15.26:0/15685699 wait complete. 2024-05-20T20:59:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:59:05.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:05 smithi026 ceph-mon[31990]: pgmap v476: 97 pgs: 97 active+clean; 23 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T20:59:05.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:05 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/15685699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:05 smithi144 ceph-mon[35660]: pgmap v476: 97 pgs: 97 active+clean; 23 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T20:59:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:05 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/15685699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:05 smithi055 ceph-mon[35213]: pgmap v476: 97 pgs: 97 active+clean; 23 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T20:59:05.504 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:05 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/15685699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:05.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 45215 3.64 MB/sec warmup 71 sec latency 12.885 ms 2024-05-20T20:59:06.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:59:06.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 45962 3.68 MB/sec warmup 72 sec latency 17.413 ms 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 -- 172.21.15.26:0/3705371778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 msgr2=0x7f403810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 --2- 172.21.15.26:0/3705371778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403810f2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f403000b0a0 tx=0x7f403000b3b0 comp rx=0 tx=0).stop 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 -- 172.21.15.26:0/3705371778 shutdown_connections 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 --2- 172.21.15.26:0/3705371778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403810f2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 --2- 172.21.15.26:0/3705371778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 0x7f4038108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 --2- 172.21.15.26:0/3705371778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4038102440 0x7f40381028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 -- 172.21.15.26:0/3705371778 >> 172.21.15.26:0/3705371778 conn(0x7f40380793f0 msgr2=0x7f4038079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 -- 172.21.15.26:0/3705371778 shutdown_connections 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.891+0000 7f403de1a700 1 -- 172.21.15.26:0/3705371778 wait complete. 2024-05-20T20:59:06.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 Processor -- start 2024-05-20T20:59:06.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 -- start start 2024-05-20T20:59:06.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4038102440 0x7f403819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:06.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:06.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 0x7f4038199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:06.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112480 con 0x7f4038102e00 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112300 con 0x7f4038108520 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f403de1a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112600 con 0x7f4038102440 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f4036ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f4036ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34798/0 (socket says 172.21.15.26:34798) 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f4036ffd700 1 -- 172.21.15.26:0/2363969911 learned_addr learned my addr 172.21.15.26:0/2363969911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.892+0000 7f4037fff700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 0x7f4038199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f40377fe700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4038102440 0x7f403819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4036ffd700 1 -- 172.21.15.26:0/2363969911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4038102440 msgr2=0x7f403819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4036ffd700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4038102440 0x7f403819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4036ffd700 1 -- 172.21.15.26:0/2363969911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 msgr2=0x7f4038199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4036ffd700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 0x7f4038199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4036ffd700 1 -- 172.21.15.26:0/2363969911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403819a080 con 0x7f4038102e00 2024-05-20T20:59:06.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4037fff700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 0x7f4038199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4036ffd700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403819fc50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f402c00d950 tx=0x7f402c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c016070 con 0x7f4038102e00 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f402c009dd0 con 0x7f4038102e00 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403819a310 con 0x7f4038102e00 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c00f3c0 con 0x7f4038102e00 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.893+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40381a65e0 con 0x7f4038102e00 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.894+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403806a230 con 0x7f4038102e00 2024-05-20T20:59:06.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.896+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f402c019070 con 0x7f4038102e00 2024-05-20T20:59:06.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.897+0000 7f4034ff9700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f40200b2880 0x7f40200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:06.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.897+0000 7f40377fe700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f40200b2880 0x7f40200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:06.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.897+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f402c0e1ea0 con 0x7f4038102e00 2024-05-20T20:59:06.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.898+0000 7f40377fe700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f40200b2880 0x7f40200b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4038107910 tx=0x7f4028009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:06.902+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f402c0a9b60 con 0x7f4038102e00 2024-05-20T20:59:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:07.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.245+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f403802e560 con 0x7f4038102e00 2024-05-20T20:59:07.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.246+0000 7f4034ff9700 1 -- 172.21.15.26:0/2363969911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f402c0a9460 con 0x7f4038102e00 2024-05-20T20:59:07.246 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:59:07.247 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f40200b2880 msgr2=0x7f40200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f40200b2880 0x7f40200b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4038107910 tx=0x7f4028009260 comp rx=0 tx=0).stop 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 msgr2=0x7f403819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403819fc50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f402c00d950 tx=0x7f402c00dc60 comp rx=0 tx=0).stop 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 shutdown_connections 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4038108520 0x7f4038199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4038102440 0x7f403819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f40200b2880 0x7f40200b4d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:07.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 --2- 172.21.15.26:0/2363969911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4038102e00 0x7f403819fc50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:07.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 >> 172.21.15.26:0/2363969911 conn(0x7f40380793f0 msgr2=0x7f403806b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:07.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 shutdown_connections 2024-05-20T20:59:07.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:07.249+0000 7f403de1a700 1 -- 172.21.15.26:0/2363969911 wait complete. 2024-05-20T20:59:07.250 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:59:07.291 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:07 smithi026 ceph-mon[31990]: pgmap v477: 97 pgs: 97 active+clean; 24 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T20:59:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:07 smithi144 ceph-mon[35660]: pgmap v477: 97 pgs: 97 active+clean; 24 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T20:59:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:07 smithi055 ceph-mon[35213]: pgmap v477: 97 pgs: 97 active+clean; 24 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T20:59:07.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 46567 3.69 MB/sec warmup 73 sec latency 13.789 ms 2024-05-20T20:59:08.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:08 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2363969911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:08.429 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:08 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2363969911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:08.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:08 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2363969911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:08.457 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-20T20:59:08.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-20T20:59:08.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 47127 3.65 MB/sec warmup 74 sec latency 18.591 ms 2024-05-20T20:59:09.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:09 smithi026 ceph-mon[31990]: pgmap v478: 97 pgs: 97 active+clean; 24 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T20:59:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:09 smithi144 ceph-mon[35660]: pgmap v478: 97 pgs: 97 active+clean; 24 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T20:59:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:09 smithi055 ceph-mon[35213]: pgmap v478: 97 pgs: 97 active+clean; 24 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T20:59:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.522+0000 7fb650356700 1 -- 172.21.15.26:0/1019382026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 msgr2=0x7fb648103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.522+0000 7fb650356700 1 --2- 172.21.15.26:0/1019382026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648103c30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6440099c0 tx=0x7fb644009cd0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 -- 172.21.15.26:0/1019382026 shutdown_connections 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 --2- 172.21.15.26:0/1019382026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb648108920 0x7fb6481104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 --2- 172.21.15.26:0/1019382026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648103c30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 --2- 172.21.15.26:0/1019382026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648104200 0x7fb6481081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 -- 172.21.15.26:0/1019382026 >> 172.21.15.26:0/1019382026 conn(0x7fb64807a5d0 msgr2=0x7fb64807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 -- 172.21.15.26:0/1019382026 shutdown_connections 2024-05-20T20:59:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 -- 172.21.15.26:0/1019382026 wait complete. 2024-05-20T20:59:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 Processor -- start 2024-05-20T20:59:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.523+0000 7fb650356700 1 -- start start 2024-05-20T20:59:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb650356700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648077430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb650356700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb648104200 0x7fb648075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb650356700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648108920 0x7fb648075ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb650356700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb648113610 con 0x7fb648108920 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb650356700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb648113490 con 0x7fb648103850 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb650356700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb648113790 con 0x7fb648104200 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e0f2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648077430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e0f2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648077430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54820/0 (socket says 172.21.15.26:54820) 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e0f2700 1 -- 172.21.15.26:0/3617103549 learned_addr learned my addr 172.21.15.26:0/3617103549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648108920 0x7fb648075ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb648104200 0x7fb648075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3617103549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb648104200 msgr2=0x7fb648075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb648104200 0x7fb648075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3617103549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 msgr2=0x7fb648077430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648077430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.524+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3617103549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6481ac4d0 con 0x7fb648108920 2024-05-20T20:59:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.525+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648108920 0x7fb648075ea0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb640007f10 tx=0x7fb64000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.525+0000 7fb63f7fe700 1 -- 172.21.15.26:0/3617103549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640004db0 con 0x7fb648108920 2024-05-20T20:59:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.525+0000 7fb63f7fe700 1 -- 172.21.15.26:0/3617103549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb640009e20 con 0x7fb648108920 2024-05-20T20:59:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.525+0000 7fb63f7fe700 1 -- 172.21.15.26:0/3617103549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640015400 con 0x7fb648108920 2024-05-20T20:59:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.525+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6481ac760 con 0x7fb648108920 2024-05-20T20:59:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.525+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6481acb60 con 0x7fb648108920 2024-05-20T20:59:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.528+0000 7fb63f7fe700 1 -- 172.21.15.26:0/3617103549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb640005240 con 0x7fb648108920 2024-05-20T20:59:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.529+0000 7fb63f7fe700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6340b29d0 0x7fb6340b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.529+0000 7fb63f7fe700 1 -- 172.21.15.26:0/3617103549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb6400e1ad0 con 0x7fb648108920 2024-05-20T20:59:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.529+0000 7fb65abecb80 1 Processor -- start 2024-05-20T20:59:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6340b29d0 0x7fb6340b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 -- start start 2024-05-20T20:59:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e0369ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6340b29d0 0x7fb6340b4e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb644003760 tx=0x7fb644005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e035fdc90 con 0x560e0369e860 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560e035ed830 con 0x560e0369f210 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb65abecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560e035f1870 con 0x560e036a2f60 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb64d8f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb64e0f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e0369ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb64d8f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54832/0 (socket says 172.21.15.26:54832) 2024-05-20T20:59:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.530+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1020331417 learned_addr learned my addr 172.21.15.26:0/1020331417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1020331417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 msgr2=0x560e036a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1020331417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 msgr2=0x560e0369ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e0369ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1020331417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0353d8d0 con 0x560e0369f210 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a2940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb638009cb0 tx=0x7fb638011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10ea0cbaa856021 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb63cd70700 1 -- 172.21.15.26:0/1020331417 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638011d80 con 0x560e0369f210 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb63cd70700 1 -- 172.21.15.26:0/1020331417 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6380044e0 con 0x560e0369f210 2024-05-20T20:59:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.531+0000 7fb63cd70700 1 -- 172.21.15.26:0/1020331417 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638005280 con 0x560e0369f210 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 -- 172.21.15.26:0/1020331417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 msgr2=0x560e036a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a2940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb638009cb0 tx=0x7fb638011440 comp rx=0 tx=0).stop 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 -- 172.21.15.26:0/1020331417 shutdown_connections 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e0369ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1020331417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a2940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 -- 172.21.15.26:0/1020331417 >> 172.21.15.26:0/1020331417 conn(0x560e035cc080 msgr2=0x560e03698e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.532+0000 7fb65abecb80 1 -- 172.21.15.26:0/1020331417 shutdown_connections 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 -- 172.21.15.26:0/1020331417 wait complete. 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 -- start start 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 Processor -- start 2024-05-20T20:59:09.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 0x560e036a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e034f3fa0 con 0x560e0369f210 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560e0358f7c0 con 0x560e0369e860 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.533+0000 7fb65abecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560e036ab120 con 0x560e036a2f60 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 0x560e036a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 0x560e036a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34830/0 (socket says 172.21.15.26:34830) 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1182325841 learned_addr learned my addr 172.21.15.26:0/1182325841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb65abecb80 10 client.0 _ll_register_callbacks cb 0x560e03588920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 msgr2=0x560e036a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1182325841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 msgr2=0x560e036aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 -- 172.21.15.26:0/1182325841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0358c690 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 0x560e036a76e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb638009840 tx=0x7fb638004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36faff69d541ce56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.534+0000 7fb631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb631ffb700 1 -- 172.21.15.26:0/1182325841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638026020 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6380044e0 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb631ffb700 1 -- 172.21.15.26:0/1182325841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb638004960 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb631ffb700 1 -- 172.21.15.26:0/1182325841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb638031050 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 10 client.16290 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e036b5860 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb631ffb700 1 -- 172.21.15.26:0/1182325841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6380044e0 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb631ffb700 1 -- 172.21.15.26:0/1182325841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb638018070 con 0x560e0369f210 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 10 client.16290 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 10 client.16290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:59:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 20 client.16290 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e036b8120 0x560e036ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 4 client.16290 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.535+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e0366a9e0 con 0x560e036b8120 2024-05-20T20:59:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.536+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e036b8120 0x560e036ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.536+0000 7fb631ffb700 10 client.16290 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.536+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e036b8120 0x560e036ba5a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92d4618b9f50f6fa server_cookie=b3bdfa722c158083 in_seq=0 out_seq=0 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb631ffb700 1 -- 172.21.15.26:0/1182325841 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560e0366a9e0 con 0x560e036b8120 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb631ffb700 10 client.16290 handle_command_reply: tid=0 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 client.16290 shutdown 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 msgr2=0x560e036a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 0x560e036a76e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb638009840 tx=0x7fb638004050 comp rx=0 tx=0).stop 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 shutdown_connections 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e036b8120 0x560e036ba5a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1182325841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369f210 0x560e036a76e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 >> 172.21.15.26:0/1182325841 conn(0x560e035cc080 msgr2=0x560e036a4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb631ffb700 0 client.16290 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 shutdown_connections 2024-05-20T20:59:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.537+0000 7fb65abecb80 1 -- 172.21.15.26:0/1182325841 wait complete. 2024-05-20T20:59:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.538+0000 7fb65abecb80 20 client.16290 trim_cache size 0 max 16384 2024-05-20T20:59:09.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 47770 3.66 MB/sec warmup 75 sec latency 18.359 ms 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 Processor -- start 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 -- start start 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e036a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481b6c00 con 0x560e0369e860 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481b4490 con 0x560e0369f210 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb65abecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481b22e0 con 0x560e036a2f60 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e036a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e036a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34840/0 (socket says 172.21.15.26:34840) 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 -- 172.21.15.26:0/1635314515 learned_addr learned my addr 172.21.15.26:0/1635314515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 -- 172.21.15.26:0/1635314515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 msgr2=0x560e036bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 -- 172.21.15.26:0/1635314515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 msgr2=0x560e036a7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 -- 172.21.15.26:0/1635314515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e036a74b0 con 0x560e0369e860 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.560+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e036a76a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb644009990 tx=0x7fb64401a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e804ff87a04aedd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.561+0000 7fb633fff700 1 -- 172.21.15.26:0/1635314515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400b790 con 0x560e0369e860 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.561+0000 7fb633fff700 1 -- 172.21.15.26:0/1635314515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb64400bdc0 con 0x560e0369e860 2024-05-20T20:59:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.561+0000 7fb633fff700 1 -- 172.21.15.26:0/1635314515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644023b90 con 0x560e0369e860 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- 172.21.15.26:0/1635314515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 msgr2=0x560e036a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e036a76a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb644009990 tx=0x7fb64401a070 comp rx=0 tx=0).stop 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- 172.21.15.26:0/1635314515 shutdown_connections 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0369e860 0x560e036a76a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e036a2f60 0x560e036bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- 172.21.15.26:0/1635314515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369f210 0x560e036a7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- 172.21.15.26:0/1635314515 >> 172.21.15.26:0/1635314515 conn(0x560e035cc080 msgr2=0x560e036a6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- 172.21.15.26:0/1635314515 shutdown_connections 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- 172.21.15.26:0/1635314515 wait complete. 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- start start 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 Processor -- start 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e0369f210 0x560e036aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 0x560e03720d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481b2950 con 0x560e036a2f60 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481b3dc0 con 0x560e0369e860 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481b3020 con 0x560e0369f210 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.562+0000 7fb65abecb80 10 client.0 _ll_register_callbacks cb 0x560e03588920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 0x560e03720d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 0x560e03720d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34846/0 (socket says 172.21.15.26:34846) 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3142106950 learned_addr learned my addr 172.21.15.26:0/3142106950 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64d8f1700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e0369f210 0x560e036aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e0369f210 msgr2=0x560e036aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e0369f210 0x560e036aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3142106950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 msgr2=0x560e036ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 -- 172.21.15.26:0/3142106950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e037212b0 con 0x560e036a2f60 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb64e8f3700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 0x560e03720d40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb640005590 tx=0x7fb6400a9730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6766cefaffa3f7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb631ffb700 1 -- 172.21.15.26:0/3142106950 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400c8060 con 0x560e036a2f60 2024-05-20T20:59:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64400bdc0 con 0x560e036a2f60 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb631ffb700 1 -- 172.21.15.26:0/3142106950 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb6400a94a0 con 0x560e036a2f60 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.563+0000 7fb631ffb700 1 -- 172.21.15.26:0/3142106950 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6400b1e90 con 0x560e036a2f60 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 10 client.16296 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e037225d0 con 0x560e036a2f60 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb631ffb700 1 -- 172.21.15.26:0/3142106950 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb64400bdc0 con 0x560e036a2f60 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb631ffb700 1 -- 172.21.15.26:0/3142106950 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb64000dcd0 con 0x560e036a2f60 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 10 client.16296 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 10 client.16296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 20 client.16296 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e03722c30 0x560e03725050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 4 client.16296 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:59:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e035ffbe0 con 0x560e03722c30 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.564+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e03722c30 0x560e03725050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb631ffb700 10 client.16296 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb64e0f2700 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e03722c30 0x560e03725050 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32e9edbb6c466239 server_cookie=527afe30c7a1e85e in_seq=0 out_seq=0 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb631ffb700 1 -- 172.21.15.26:0/3142106950 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e035ffbe0 con 0x560e03722c30 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb631ffb700 10 client.16296 handle_command_reply: tid=0 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 client.16296 shutdown 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 msgr2=0x560e03720d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 0x560e03720d40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb640005590 tx=0x7fb6400a9730 comp rx=0 tx=0).stop 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 shutdown_connections 2024-05-20T20:59:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x560e03722c30 0x560e03725050 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e036a2f60 0x560e03720d40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560e0369e860 0x560e036ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb631ffb700 0 client.16296 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.565+0000 7fb65abecb80 1 --2- 172.21.15.26:0/3142106950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e0369f210 0x560e036aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 >> 172.21.15.26:0/3142106950 conn(0x560e035cc080 msgr2=0x560e036a5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 shutdown_connections 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb65abecb80 1 -- 172.21.15.26:0/3142106950 wait complete. 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb65abecb80 20 client.16296 trim_cache size 0 max 16384 2024-05-20T20:59:09.570 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6340b29d0 msgr2=0x7fb6340b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6340b29d0 0x7fb6340b4e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb644003760 tx=0x7fb644005fd0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648108920 msgr2=0x7fb648075ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648108920 0x7fb648075ea0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb640007f10 tx=0x7fb64000d3b0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 shutdown_connections 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6340b29d0 0x7fb6340b4e90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb648108920 0x7fb648075ea0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.566+0000 7fb650356700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb648103850 0x7fb648077430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.567+0000 7fb650356700 1 --2- 172.21.15.26:0/3617103549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb648104200 0x7fb648075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.567+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 >> 172.21.15.26:0/3617103549 conn(0x7fb64807a5d0 msgr2=0x7fb648101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.567+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 shutdown_connections 2024-05-20T20:59:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:09.567+0000 7fb650356700 1 -- 172.21.15.26:0/3617103549 wait complete. 2024-05-20T20:59:10.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 48485 3.64 MB/sec warmup 76 sec latency 27.699 ms 2024-05-20T20:59:11.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:59:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:11 smithi144 ceph-mon[35660]: pgmap v479: 97 pgs: 97 active+clean; 26 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-05-20T20:59:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:11 smithi055 ceph-mon[35213]: pgmap v479: 97 pgs: 97 active+clean; 26 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-05-20T20:59:11.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:11 smithi026 ceph-mon[31990]: pgmap v479: 97 pgs: 97 active+clean; 26 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-05-20T20:59:11.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49331 3.67 MB/sec warmup 77 sec latency 41.323 ms 2024-05-20T20:59:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 -- 172.21.15.26:0/2562596171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c1028c0 msgr2=0x7fbc8c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2562596171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c1028c0 0x7fbc8c110330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400b0a0 tx=0x7fbc8400b3b0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 -- 172.21.15.26:0/2562596171 shutdown_connections 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2562596171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c1028c0 0x7fbc8c110330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2562596171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c108620 0x7fbc8c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2562596171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c101f00 0x7fbc8c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 -- 172.21.15.26:0/2562596171 >> 172.21.15.26:0/2562596171 conn(0x7fbc8c077780 msgr2=0x7fbc8c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 -- 172.21.15.26:0/2562596171 shutdown_connections 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.219+0000 7fbc947c5700 1 -- 172.21.15.26:0/2562596171 wait complete. 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 Processor -- start 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 -- start start 2024-05-20T20:59:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c101f00 0x7fbc8c104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c1028c0 0x7fbc8c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 0x7fbc8c103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c113610 con 0x7fbc8c108620 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c113490 con 0x7fbc8c101f00 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.220+0000 7fbc947c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c113790 con 0x7fbc8c1028c0 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92d62700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 0x7fbc8c103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c101f00 0x7fbc8c104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92d62700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 0x7fbc8c103740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44360/0 (socket says 172.21.15.26:44360) 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92d62700 1 -- 172.21.15.26:0/2480292522 learned_addr learned my addr 172.21.15.26:0/2480292522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c1028c0 0x7fbc8c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 -- 172.21.15.26:0/2480292522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c1028c0 msgr2=0x7fbc8c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c1028c0 0x7fbc8c1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 -- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 msgr2=0x7fbc8c103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 0x7fbc8c103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 -- 172.21.15.26:0/2480292522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c1ac3d0 con 0x7fbc8c101f00 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92d62700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 0x7fbc8c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc92561700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c101f00 0x7fbc8c104cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e9c0 tx=0x7fbc8800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc837fe700 1 -- 172.21.15.26:0/2480292522 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88004220 con 0x7fbc8c101f00 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc837fe700 1 -- 172.21.15.26:0/2480292522 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc880043c0 con 0x7fbc8c101f00 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c1ac600 con 0x7fbc8c101f00 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.221+0000 7fbc837fe700 1 -- 172.21.15.26:0/2480292522 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88003b40 con 0x7fbc8c101f00 2024-05-20T20:59:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.222+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1acb40 con 0x7fbc8c101f00 2024-05-20T20:59:12.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.225+0000 7fbc837fe700 1 -- 172.21.15.26:0/2480292522 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbc88018430 con 0x7fbc8c101f00 2024-05-20T20:59:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.226+0000 7fbc837fe700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbc780b29b0 0x7fbc780b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.226+0000 7fbc9f05bb80 1 Processor -- start 2024-05-20T20:59:12.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.226+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbc780b29b0 0x7fbc780b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.226+0000 7fbc837fe700 1 -- 172.21.15.26:0/2480292522 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbc880e23f0 con 0x7fbc8c101f00 2024-05-20T20:59:12.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 -- start start 2024-05-20T20:59:12.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 0x55f998813720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880d970 0x55f99880ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880e330 0x55f99881a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f99876ce20 con 0x55f998813340 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f99875c9c0 con 0x55f99880d970 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f998760a00 con 0x55f99880e330 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 0x55f998813720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbc780b29b0 0x7fbc780b4e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c104850 tx=0x7fbc7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 0x55f998813720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44368/0 (socket says 172.21.15.26:44368) 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 -- 172.21.15.26:0/1513666207 learned_addr learned my addr 172.21.15.26:0/1513666207 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc91d60700 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880d970 0x55f99880ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92d62700 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880e330 0x55f99881a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 -- 172.21.15.26:0/1513666207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880e330 msgr2=0x55f99881a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880e330 0x55f99881a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 -- 172.21.15.26:0/1513666207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880d970 msgr2=0x55f99880ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880d970 0x55f99880ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.227+0000 7fbc92561700 1 -- 172.21.15.26:0/1513666207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9986aca60 con 0x55f998813340 2024-05-20T20:59:12.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.228+0000 7fbc92561700 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 0x55f998813720 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009230 tx=0x7fbc880a9850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0327859022fbac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.228+0000 7fbc80ff9700 1 -- 172.21.15.26:0/1513666207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880c3070 con 0x55f998813340 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.228+0000 7fbc80ff9700 1 -- 172.21.15.26:0/1513666207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc880b20f0 con 0x55f998813340 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.228+0000 7fbc80ff9700 1 -- 172.21.15.26:0/1513666207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880b27b0 con 0x55f998813340 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/1513666207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 msgr2=0x55f998813720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 0x55f998813720 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009230 tx=0x7fbc880a9850 comp rx=0 tx=0).stop 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/1513666207 shutdown_connections 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f998813340 0x55f998813720 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880e330 0x55f99881a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/1513666207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880d970 0x55f99880ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/1513666207 >> 172.21.15.26:0/1513666207 conn(0x55f99873b210 msgr2=0x55f998808280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/1513666207 shutdown_connections 2024-05-20T20:59:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.229+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/1513666207 wait complete. 2024-05-20T20:59:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.230+0000 7fbc9f05bb80 1 -- start start 2024-05-20T20:59:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.230+0000 7fbc9f05bb80 1 Processor -- start 2024-05-20T20:59:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.230+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998819ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.230+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f9988171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.230+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998817710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f998663130 con 0x55f99880d970 2024-05-20T20:59:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f9986fe950 con 0x55f99880e330 2024-05-20T20:59:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f99881a770 con 0x55f998813340 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc9f05bb80 10 client.0 _ll_register_callbacks cb 0x55f9986f7ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998817710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998817710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:53712/0 (socket says 172.21.15.26:53712) 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 -- 172.21.15.26:0/2419198926 learned_addr learned my addr 172.21.15.26:0/2419198926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f9988171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92561700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998819ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 -- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 msgr2=0x55f9988171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f9988171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 -- 172.21.15.26:0/2419198926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 msgr2=0x55f998819ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998819ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc92d62700 1 -- 172.21.15.26:0/2419198926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9986fb820 con 0x55f998813340 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.231+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f9988171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc92d62700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998817710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009e60 tx=0x7fbc84003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef8543e1ee460d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2419198926 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc840077b0 con 0x55f998813340 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9987d9b70 con 0x55f998813340 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2419198926 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbc84007950 con 0x55f998813340 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2419198926 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc84007b50 con 0x55f998813340 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc9f05bb80 10 client.25771 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f998824f40 con 0x55f998813340 2024-05-20T20:59:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.232+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2419198926 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f9987d9b70 con 0x55f998813340 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2419198926 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbc84011070 con 0x55f998813340 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc9f05bb80 10 client.25771 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc9f05bb80 10 client.25771 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc9f05bb80 20 client.25771 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9988275d0 0x55f9988299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc9f05bb80 4 client.25771 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f99876ed70 con 0x55f9988275d0 2024-05-20T20:59:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc92561700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9988275d0 0x55f9988299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc75ffb700 10 client.25771 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.233+0000 7fbc92561700 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9988275d0 0x55f9988299f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29cb41e717d535 server_cookie=af7453883e617e79 in_seq=0 out_seq=0 2024-05-20T20:59:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.234+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2419198926 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f99876ed70 con 0x55f9988275d0 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.234+0000 7fbc75ffb700 10 client.25771 handle_command_reply: tid=0 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.234+0000 7fbc9f05bb80 1 client.25771 shutdown 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 msgr2=0x55f998817710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998817710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009e60 tx=0x7fbc84003ce0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 shutdown_connections 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9988275d0 0x55f9988299f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998819ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998817710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc75ffb700 0 client.25771 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2419198926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f9988171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 >> 172.21.15.26:0/2419198926 conn(0x55f99873b210 msgr2=0x55f998802d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 shutdown_connections 2024-05-20T20:59:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2419198926 wait complete. 2024-05-20T20:59:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.235+0000 7fbc9f05bb80 20 client.25771 trim_cache size 0 max 16384 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.261+0000 7fbc9f05bb80 1 Processor -- start 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 -- start start 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998823810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f998816150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998819a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1b6a80 con 0x55f99880d970 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1b4310 con 0x55f99880e330 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1b2160 con 0x55f998813340 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f998816150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc92561700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998823810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f998816150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35206/0 (socket says 172.21.15.26:35206) 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 -- 172.21.15.26:0/3055649548 learned_addr learned my addr 172.21.15.26:0/3055649548 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 -- 172.21.15.26:0/3055649548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 msgr2=0x55f998819a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998819a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 -- 172.21.15.26:0/3055649548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 msgr2=0x55f998823810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998823810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.262+0000 7fbc91d60700 1 -- 172.21.15.26:0/3055649548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99882a050 con 0x55f99880e330 2024-05-20T20:59:12.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.263+0000 7fbc91d60700 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f998816150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c011df0 tx=0x7fbc7c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4d1866d7c72706 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:12.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.263+0000 7fbc77fff700 1 -- 172.21.15.26:0/3055649548 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c01dd80 con 0x55f99880e330 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.263+0000 7fbc77fff700 1 -- 172.21.15.26:0/3055649548 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc7c02c070 con 0x55f99880e330 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.263+0000 7fbc77fff700 1 -- 172.21.15.26:0/3055649548 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c01c400 con 0x55f99880e330 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/3055649548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 msgr2=0x55f998816150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f998816150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c011df0 tx=0x7fbc7c014000 comp rx=0 tx=0).stop 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/3055649548 shutdown_connections 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880d970 0x55f998823810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f998813340 0x55f998819a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/3055649548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f99880e330 0x55f998816150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/3055649548 >> 172.21.15.26:0/3055649548 conn(0x55f99873b210 msgr2=0x55f99880d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/3055649548 shutdown_connections 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/3055649548 wait complete. 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.264+0000 7fbc9f05bb80 1 -- start start 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 Processor -- start 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880d970 0x55f99881caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 0x55f99881d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f998813340 0x55f998892180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc91d60700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 0x55f99881d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1b27d0 con 0x55f99880e330 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1b3c40 con 0x55f998813340 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc91d60700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 0x55f99881d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44404/0 (socket says 172.21.15.26:44404) 2024-05-20T20:59:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc91d60700 1 -- 172.21.15.26:0/2071960265 learned_addr learned my addr 172.21.15.26:0/2071960265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1b2ea0 con 0x55f99880d970 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc92d62700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f998813340 0x55f998892180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.265+0000 7fbc92561700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880d970 0x55f99881caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc91d60700 1 -- 172.21.15.26:0/2071960265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880d970 msgr2=0x55f99881caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880d970 0x55f99881caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc91d60700 1 -- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f998813340 msgr2=0x55f998892180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f998813340 0x55f998892180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc91d60700 1 -- 172.21.15.26:0/2071960265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9988926c0 con 0x55f99880e330 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc92d62700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f998813340 0x55f998892180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc92561700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880d970 0x55f99881caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc91d60700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 0x55f99881d030 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009b20 tx=0x7fbc7c01de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264214a25fa9875e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:12.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc9f05bb80 10 client.0 _ll_register_callbacks cb 0x55f9986f7ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7c02c070 con 0x55f99880e330 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.266+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2071960265 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c034070 con 0x55f99880e330 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2071960265 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fbc7c004750 con 0x55f99880e330 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2071960265 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc7c004970 con 0x55f99880e330 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 10 client.16314 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f998893700 con 0x55f99880e330 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2071960265 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbc7c004b10 con 0x55f99880e330 2024-05-20T20:59:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 10 client.16314 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 10 client.16314 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:59:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 20 client.16314 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f998893c20 0x55f998896040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 4 client.16314 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-20T20:59:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7fbc7c02c070 con 0x55f998893c20 2024-05-20T20:59:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.267+0000 7fbc92561700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f998893c20 0x55f998896040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.268+0000 7fbc75ffb700 10 client.16314 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.268+0000 7fbc92561700 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f998893c20 0x55f998896040 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220c296f10b86d4f server_cookie=4c11738aeb2cdbce in_seq=0 out_seq=0 2024-05-20T20:59:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc75ffb700 1 -- 172.21.15.26:0/2071960265 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fbc7c02c070 con 0x55f998893c20 2024-05-20T20:59:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc75ffb700 10 client.16314 handle_command_reply: tid=0 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 client.16314 shutdown 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 msgr2=0x55f99881d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 0x55f99881d030 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009b20 tx=0x7fbc7c01de30 comp rx=0 tx=0).stop 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 shutdown_connections 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f998893c20 0x55f998896040 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f998813340 0x55f998892180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc75ffb700 0 client.16314 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f99880d970 0x55f99881caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 --2- 172.21.15.26:0/2071960265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f99880e330 0x55f99881d030 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 >> 172.21.15.26:0/2071960265 conn(0x55f99873b210 msgr2=0x55f9988024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 shutdown_connections 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 1 -- 172.21.15.26:0/2071960265 wait complete. 2024-05-20T20:59:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.431+0000 7fbc9f05bb80 20 client.16314 trim_cache size 0 max 16384 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 10, 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "d2603423": { 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 10, 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:59:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.0, 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:59:12.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbc780b29b0 msgr2=0x7fbc780b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbc780b29b0 0x7fbc780b4e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c104850 tx=0x7fbc7c009260 comp rx=0 tx=0).stop 2024-05-20T20:59:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c101f00 msgr2=0x7fbc8c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c101f00 0x7fbc8c104cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e9c0 tx=0x7fbc8800ecd0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 shutdown_connections 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbc780b29b0 0x7fbc780b4e70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c108620 0x7fbc8c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc8c101f00 0x7fbc8c104cd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 --2- 172.21.15.26:0/2480292522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c1028c0 0x7fbc8c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 >> 172.21.15.26:0/2480292522 conn(0x7fbc8c077780 msgr2=0x7fbc8c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 shutdown_connections 2024-05-20T20:59:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:12.432+0000 7fbc947c5700 1 -- 172.21.15.26:0/2480292522 wait complete. 2024-05-20T20:59:12.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.68 MB/sec warmup 78 sec latency 269.469 ms 2024-05-20T20:59:13.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:13 smithi026 ceph-mon[31990]: pgmap v480: 97 pgs: 97 active+clean; 27 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T20:59:13.181 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd2603423', quiesce took 14.3 seconds. Will release after: -4.300000000000001 2024-05-20T20:59:13.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:59:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:13 smithi144 ceph-mon[35660]: pgmap v480: 97 pgs: 97 active+clean; 27 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T20:59:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:13 smithi055 ceph-mon[35213]: pgmap v480: 97 pgs: 97 active+clean; 27 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T20:59:13.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.63 MB/sec warmup 79 sec latency 1269.552 ms 2024-05-20T20:59:14.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.59 MB/sec warmup 80 sec latency 2269.593 ms 2024-05-20T20:59:14.833 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/948377823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102440 msgr2=0x7f0ad01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/948377823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102440 0x7f0ad01028c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009990 tx=0x7f0ac4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/948377823 shutdown_connections 2024-05-20T20:59:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/948377823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad0102e00 0x7f0ad010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/948377823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad0108520 0x7f0ad0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/948377823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102440 0x7f0ad01028c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/948377823 >> 172.21.15.26:0/948377823 conn(0x7f0ad00793f0 msgr2=0x7f0ad0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/948377823 shutdown_connections 2024-05-20T20:59:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.841+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/948377823 wait complete. 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.842+0000 7f0ad6a0e700 1 Processor -- start 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.842+0000 7f0ad6a0e700 1 -- start start 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad6a0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad0102440 0x7f0ad019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad6a0e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 0x7f0ad019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad6a0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad0108520 0x7f0ad01a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad6a0e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01124d0 con 0x7f0ad0102e00 2024-05-20T20:59:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad6a0e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad0112350 con 0x7f0ad0108520 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad6a0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad0112650 con 0x7f0ad0102440 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 0x7f0ad019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 0x7f0ad019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44432/0 (socket says 172.21.15.26:44432) 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 -- 172.21.15.26:0/4245376029 learned_addr learned my addr 172.21.15.26:0/4245376029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acffff700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad0102440 0x7f0ad019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad0108520 0x7f0ad01a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 -- 172.21.15.26:0/4245376029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad0102440 msgr2=0x7f0ad019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad0102440 0x7f0ad019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 -- 172.21.15.26:0/4245376029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad0108520 msgr2=0x7f0ad01a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad0108520 0x7f0ad01a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.843+0000 7f0acf7fe700 1 -- 172.21.15.26:0/4245376029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad01a5f90 con 0x7f0ad0102e00 2024-05-20T20:59:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.844+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 0x7f0ad019af10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4000c00 tx=0x7f0ac40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.844+0000 7f0acd7fa700 1 -- 172.21.15.26:0/4245376029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4027070 con 0x7f0ad0102e00 2024-05-20T20:59:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.844+0000 7f0acd7fa700 1 -- 172.21.15.26:0/4245376029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac400fb00 con 0x7f0ad0102e00 2024-05-20T20:59:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.844+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad01a6160 con 0x7f0ad0102e00 2024-05-20T20:59:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.844+0000 7f0acd7fa700 1 -- 172.21.15.26:0/4245376029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4022a00 con 0x7f0ad0102e00 2024-05-20T20:59:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.844+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad01a6610 con 0x7f0ad0102e00 2024-05-20T20:59:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.847+0000 7f0acd7fa700 1 -- 172.21.15.26:0/4245376029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0ac402a070 con 0x7f0ad0102e00 2024-05-20T20:59:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.848+0000 7f0acd7fa700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0ab80b2900 0x7f0ab80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.848+0000 7f0ae12a4b80 1 Processor -- start 2024-05-20T20:59:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.848+0000 7f0acffff700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0ab80b2900 0x7f0ab80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.848+0000 7f0acd7fa700 1 -- 172.21.15.26:0/4245376029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0ac40e31c0 con 0x7f0ad0102e00 2024-05-20T20:59:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 -- start start 2024-05-20T20:59:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb058a0 0x563e6eb05d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e6ea63d70 con 0x563e6eb0b320 2024-05-20T20:59:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563e6ea53910 con 0x563e6eb06260 2024-05-20T20:59:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e6ea57950 con 0x563e6eb058a0 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44444/0 (socket says 172.21.15.26:44444) 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 -- 172.21.15.26:0/3009916360 learned_addr learned my addr 172.21.15.26:0/3009916360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0ab80b2900 0x7f0ab80b4dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0007b60 tx=0x7f0ac00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb058a0 0x563e6eb05d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 -- 172.21.15.26:0/3009916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb058a0 msgr2=0x563e6eb05d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb058a0 0x563e6eb05d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 -- 172.21.15.26:0/3009916360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 msgr2=0x563e6eb12080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb12080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.849+0000 7f0acffff700 1 -- 172.21.15.26:0/3009916360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6e9fcfb0 con 0x563e6eb0b320 2024-05-20T20:59:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.850+0000 7f0acffff700 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0b700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0010db0 tx=0x7f0ac0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314f57505a9e24a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.850+0000 7f0ab67fc700 1 -- 172.21.15.26:0/3009916360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0011c60 con 0x563e6eb0b320 2024-05-20T20:59:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.850+0000 7f0ab67fc700 1 -- 172.21.15.26:0/3009916360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac0011e00 con 0x563e6eb0b320 2024-05-20T20:59:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.850+0000 7f0ab67fc700 1 -- 172.21.15.26:0/3009916360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac001d6a0 con 0x563e6eb0b320 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3009916360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 msgr2=0x563e6eb0b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0b700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0010db0 tx=0x7f0ac0013000 comp rx=0 tx=0).stop 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3009916360 shutdown_connections 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0b700 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb12080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3009916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb058a0 0x563e6eb05d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3009916360 >> 172.21.15.26:0/3009916360 conn(0x563e6ea31240 msgr2=0x563e6eb003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3009916360 shutdown_connections 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3009916360 wait complete. 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.851+0000 7f0ae12a4b80 1 -- start start 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 Processor -- start 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 0x563e6eb11d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e6e959fa0 con 0x563e6eb0b320 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563e6e9bbda0 con 0x563e6eb058a0 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e6eb126e0 con 0x563e6eb06260 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ae12a4b80 10 client.0 _ll_register_callbacks cb 0x563e6e9ee950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ad4fab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0acffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 0x563e6eb11d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0acffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 0x563e6eb11d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35236/0 (socket says 172.21.15.26:35236) 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0acffff700 1 -- 172.21.15.26:0/3626379118 learned_addr learned my addr 172.21.15.26:0/3626379118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44456/0 (socket says 172.21.15.26:44456) 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.852+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ad4fab700 1 -- 172.21.15.26:0/3626379118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 msgr2=0x563e6eb0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ad4fab700 1 -- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 msgr2=0x563e6eb11d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 0x563e6eb11d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ad4fab700 1 -- 172.21.15.26:0/3626379118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6e9a38d0 con 0x563e6eb0b320 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0acffff700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 0x563e6eb11d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0f570 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80040c0 tx=0x7f0ac80043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5898d76cd26c6dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0aaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/3626379118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac801a020 con 0x563e6eb0b320 2024-05-20T20:59:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/3626379118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0ac8007910 con 0x563e6eb0b320 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac0011e00 con 0x563e6eb0b320 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/3626379118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ac8025050 con 0x563e6eb0b320 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ae12a4b80 10 client.16326 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.853+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e6ea0d760 con 0x563e6eb0b320 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/3626379118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac0011e00 con 0x563e6eb0b320 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/3626379118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0ac80153c0 con 0x563e6eb0b320 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0ae12a4b80 10 client.16326 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0ae12a4b80 10 client.16326 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0ae12a4b80 20 client.16326 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb25b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0ae12a4b80 4 client.16326 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6ea65cc0 con 0x563e6eb236b0 2024-05-20T20:59:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.854+0000 7f0acffff700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb25b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.855+0000 7f0acffff700 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb25b00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93077bfcd49eb07e server_cookie=4ca52b2e2cd85a83 in_seq=0 out_seq=0 2024-05-20T20:59:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.855+0000 7f0aaf7fe700 10 client.16326 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/3626379118 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563e6ea65cc0 con 0x563e6eb236b0 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0aaf7fe700 10 client.16326 handle_command_reply: tid=0 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 client.16326 shutdown 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 msgr2=0x563e6eb0f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0f570 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80040c0 tx=0x7f0ac80043d0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 shutdown_connections 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb25b00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb0b320 0x563e6eb0f570 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0aaf7fe700 0 client.16326 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb058a0 0x563e6eb11d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3626379118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 >> 172.21.15.26:0/3626379118 conn(0x563e6ea31240 msgr2=0x563e6eaffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 shutdown_connections 2024-05-20T20:59:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.856+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3626379118 wait complete. 2024-05-20T20:59:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.857+0000 7f0ae12a4b80 20 client.16326 trim_cache size 0 max 16384 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.878+0000 7f0ae12a4b80 1 Processor -- start 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.878+0000 7f0ae12a4b80 1 -- start start 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb0a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb0b320 0x563e6eb00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01ae2a0 con 0x563e6eb058a0 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01abb30 con 0x563e6eb0b320 2024-05-20T20:59:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01a9980 con 0x563e6eb06260 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb0a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb0a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44472/0 (socket says 172.21.15.26:44472) 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 -- 172.21.15.26:0/3275394423 learned_addr learned my addr 172.21.15.26:0/3275394423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb0b320 0x563e6eb00270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 -- 172.21.15.26:0/3275394423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 msgr2=0x563e6eb0e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 -- 172.21.15.26:0/3275394423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb0b320 msgr2=0x563e6eb00270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb0b320 0x563e6eb00270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.879+0000 7f0acffff700 1 -- 172.21.15.26:0/3275394423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6eb12510 con 0x563e6eb058a0 2024-05-20T20:59:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.880+0000 7f0acffff700 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb0a7c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0ac001efd0 tx=0x7f0ac0007480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a5d02607750692 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.880+0000 7f0ab5ffb700 1 -- 172.21.15.26:0/3275394423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0005340 con 0x563e6eb058a0 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.880+0000 7f0ab5ffb700 1 -- 172.21.15.26:0/3275394423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac00054e0 con 0x563e6eb058a0 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.880+0000 7f0ab5ffb700 1 -- 172.21.15.26:0/3275394423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac001e6f0 con 0x563e6eb058a0 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3275394423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 msgr2=0x563e6eb0a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb0a7c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0ac001efd0 tx=0x7f0ac0007480 comp rx=0 tx=0).stop 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3275394423 shutdown_connections 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb0a7c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb0b320 0x563e6eb00270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/3275394423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb06260 0x563e6eb0e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3275394423 >> 172.21.15.26:0/3275394423 conn(0x563e6ea31240 msgr2=0x563e6eb05120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3275394423 shutdown_connections 2024-05-20T20:59:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.881+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/3275394423 wait complete. 2024-05-20T20:59:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 -- start start 2024-05-20T20:59:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 Processor -- start 2024-05-20T20:59:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb2a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb0b320 0x563e6eb2ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01a9ff0 con 0x563e6eb058a0 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01ab460 con 0x563e6eb06260 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01aa6c0 con 0x563e6eb0b320 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ae12a4b80 10 client.0 _ll_register_callbacks cb 0x563e6e9ee950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0acffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0acffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb2a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44480/0 (socket says 172.21.15.26:44480) 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0acffff700 1 -- 172.21.15.26:0/2233198285 learned_addr learned my addr 172.21.15.26:0/2233198285 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb2a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.882+0000 7f0ad4fab700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb0b320 0x563e6eb2ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acffff700 1 -- 172.21.15.26:0/2233198285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb0b320 msgr2=0x563e6eb2ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acffff700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb0b320 0x563e6eb2ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acffff700 1 -- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 msgr2=0x563e6eb2a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acffff700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb2a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acffff700 1 -- 172.21.15.26:0/2233198285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6eb16930 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb2a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0acffff700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb2a3c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0021040 tx=0x7f0ac0010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafb2cadb47b1359 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0aaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/2233198285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0005240 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/2233198285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f0ac0011be0 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac00054e0 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/2233198285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ac0011de0 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0ae12a4b80 10 client.16332 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e6eb17c30 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.883+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/2233198285 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac00054e0 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/2233198285 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0ac001e720 con 0x563e6eb058a0 2024-05-20T20:59:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0ae12a4b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0ae12a4b80 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0ae12a4b80 20 client.16332 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb18190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0ae12a4b80 4 client.16332 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e6ea5f4a0 con 0x563e6eb236b0 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb18190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0acf7fe700 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb18190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83dd13bbbf02f980 server_cookie=aac4c2690c5ea9b5 in_seq=0 out_seq=0 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.884+0000 7f0aaf7fe700 10 client.16332 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0aaf7fe700 1 -- 172.21.15.26:0/2233198285 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e6ea5f4a0 con 0x563e6eb236b0 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0aaf7fe700 10 client.16332 handle_command_reply: tid=0 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 client.16332 shutdown 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 msgr2=0x563e6eb2a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb2a3c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0021040 tx=0x7f0ac0010dc0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 shutdown_connections 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x563e6eb236b0 0x563e6eb18190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6eb058a0 0x563e6eb2a3c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6eb0b320 0x563e6eb2ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0aaf7fe700 0 client.16332 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 --2- 172.21.15.26:0/2233198285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e6eb06260 0x563e6eb2a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 >> 172.21.15.26:0/2233198285 conn(0x563e6ea31240 msgr2=0x563e6eb08ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 shutdown_connections 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 1 -- 172.21.15.26:0/2233198285 wait complete. 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.885+0000 7f0ae12a4b80 20 client.16332 trim_cache size 0 max 16384 2024-05-20T20:59:14.889 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:59:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0ab80b2900 msgr2=0x7f0ab80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0ab80b2900 0x7f0ab80b4dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0007b60 tx=0x7f0ac00073d0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 msgr2=0x7f0ad019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 0x7f0ad019af10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4000c00 tx=0x7f0ac40057a0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 shutdown_connections 2024-05-20T20:59:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad0108520 0x7f0ad01a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad0102440 0x7f0ad019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f0ab80b2900 0x7f0ab80b4dc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 --2- 172.21.15.26:0/4245376029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad0102e00 0x7f0ad019af10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 >> 172.21.15.26:0/4245376029 conn(0x7f0ad00793f0 msgr2=0x7f0ad0100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 shutdown_connections 2024-05-20T20:59:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:14.886+0000 7f0ad6a0e700 1 -- 172.21.15.26:0/4245376029 wait complete. 2024-05-20T20:59:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:15 smithi026 ceph-mon[31990]: pgmap v481: 97 pgs: 97 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 -- 172.21.15.26:0/1880559355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 msgr2=0x7f7530109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 --2- 172.21.15.26:0/1880559355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f7530109ea0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7528009ca0 tx=0x7f752802a8c0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 -- 172.21.15.26:0/1880559355 shutdown_connections 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 --2- 172.21.15.26:0/1880559355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f7530109ea0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 --2- 172.21.15.26:0/1880559355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 0x7f75301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 --2- 172.21.15.26:0/1880559355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7530101a60 0x7f7530101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 -- 172.21.15.26:0/1880559355 >> 172.21.15.26:0/1880559355 conn(0x7f75300fcfd0 msgr2=0x7f75300ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 -- 172.21.15.26:0/1880559355 shutdown_connections 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.320+0000 7f7537008640 1 -- 172.21.15.26:0/1880559355 wait complete. 2024-05-20T20:59:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 Processor -- start 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 -- start start 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7530101a60 0x7f7530198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 0x7f7530199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f75301a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f753010d400 con 0x7f7530103610 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f753010d280 con 0x7f7530102c50 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7537008640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f753010d580 con 0x7f7530101a60 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f75301a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f75301a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44484/0 (socket says 172.21.15.26:44484) 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 -- 172.21.15.26:0/64425211 learned_addr learned my addr 172.21.15.26:0/64425211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f7534da4640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7530101a60 0x7f7530198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f752ffff640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 0x7f7530199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 -- 172.21.15.26:0/64425211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7530101a60 msgr2=0x7f7530198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7530101a60 0x7f7530198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 -- 172.21.15.26:0/64425211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 msgr2=0x7f7530199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 0x7f7530199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f75355a5640 1 -- 172.21.15.26:0/64425211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75301a0710 con 0x7f7530103610 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.321+0000 7f752ffff640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 0x7f7530199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.322+0000 7f75355a5640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f75301a00a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f752802aeb0 tx=0x7f7528003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.322+0000 7f752dffb640 1 -- 172.21.15.26:0/64425211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528032ed0 con 0x7f7530103610 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.322+0000 7f752dffb640 1 -- 172.21.15.26:0/64425211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7528003e10 con 0x7f7530103610 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.322+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75301a09a0 con 0x7f7530103610 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.322+0000 7f752dffb640 1 -- 172.21.15.26:0/64425211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528030cd0 con 0x7f7530103610 2024-05-20T20:59:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.322+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75301a0ef0 con 0x7f7530103610 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 Processor -- start 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f752dffb640 1 -- 172.21.15.26:0/64425211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7528004240 con 0x7f7530103610 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 -- start start 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 0x557ee2d40bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d41220 0x557ee2d436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ee2cacc30 con 0x557ee2d3c0b0 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557ee2c9c7d0 con 0x557ee2d3b6c0 2024-05-20T20:59:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7539df5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ee2ca0810 con 0x557ee2d41220 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 0x557ee2d40bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 0x557ee2d40bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44490/0 (socket says 172.21.15.26:44490) 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 -- 172.21.15.26:0/3993219865 learned_addr learned my addr 172.21.15.26:0/3993219865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f752ffff640 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 -- 172.21.15.26:0/3993219865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d41220 msgr2=0x557ee2d436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d41220 0x557ee2d436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 -- 172.21.15.26:0/3993219865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 msgr2=0x557ee2d3bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.325+0000 7f7534da4640 1 -- 172.21.15.26:0/3993219865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee29b1210 con 0x557ee2d3c0b0 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f752dffb640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75200b28a0 0x7f75200b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f7534da4640 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 0x557ee2d40bb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f752400b7d0 tx=0x7f7524004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdf1e6fa5128c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f752ffff640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75200b28a0 0x7f75200b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f752dffb640 1 -- 172.21.15.26:0/64425211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7528103ab0 con 0x7f7530103610 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f751affd640 1 -- 172.21.15.26:0/3993219865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524003dd0 con 0x557ee2d3c0b0 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f751affd640 1 -- 172.21.15.26:0/3993219865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7524015070 con 0x557ee2d3c0b0 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f751affd640 1 -- 172.21.15.26:0/3993219865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240116a0 con 0x557ee2d3c0b0 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f7539df5740 1 -- 172.21.15.26:0/3993219865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 msgr2=0x557ee2d40bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f752ffff640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75200b28a0 0x7f75200b4d60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557ee2d3c910 tx=0x7f751c00e400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.326+0000 7f7539df5740 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 0x557ee2d40bb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f752400b7d0 tx=0x7f7524004610 comp rx=0 tx=0).stop 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- 172.21.15.26:0/3993219865 shutdown_connections 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d41220 0x557ee2d436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d3c0b0 0x557ee2d40bb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 --2- 172.21.15.26:0/3993219865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- 172.21.15.26:0/3993219865 >> 172.21.15.26:0/3993219865 conn(0x557ee2d36d30 msgr2=0x557ee2d37130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- 172.21.15.26:0/3993219865 shutdown_connections 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- 172.21.15.26:0/3993219865 wait complete. 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- start start 2024-05-20T20:59:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 Processor -- start 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d45dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ee2982790 con 0x557ee2d41220 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557ee2c58890 con 0x557ee2d3b6c0 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.327+0000 7f7539df5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ee2d441d0 con 0x557ee2d3c0b0 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f7539df5740 10 client.0 _ll_register_callbacks cb 0x557ee29b0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44498/0 (socket says 172.21.15.26:44498) 2024-05-20T20:59:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 -- 172.21.15.26:0/3434356650 learned_addr learned my addr 172.21.15.26:0/3434356650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f7534da4640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f752ffff640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d45dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 -- 172.21.15.26:0/3434356650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 msgr2=0x557ee2d45dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d45dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 -- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 msgr2=0x557ee2d45880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d45880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 -- 172.21.15.26:0/3434356650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee2a7d620 con 0x557ee2d41220 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f752ffff640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d45dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f75355a5640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4cd90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f75280cb000 tx=0x7f75280cac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183eddc7ada9ce56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f750ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f750ffff640 1 -- 172.21.15.26:0/3434356650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75280e5060 con 0x557ee2d41220 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f750ffff640 1 -- 172.21.15.26:0/3434356650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f75280cf0d0 con 0x557ee2d41220 2024-05-20T20:59:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7524015070 con 0x557ee2d41220 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f750ffff640 1 -- 172.21.15.26:0/3434356650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75280cf420 con 0x557ee2d41220 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f7539df5740 10 client.16344 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.328+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ee2d4fd70 con 0x557ee2d41220 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f750ffff640 1 -- 172.21.15.26:0/3434356650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7524015070 con 0x557ee2d41220 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f750ffff640 1 -- 172.21.15.26:0/3434356650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f75280d3a40 con 0x557ee2d41220 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7539df5740 10 client.16344 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7539df5740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7539df5740 20 client.16344 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7539df5740 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2d514f0 0x557ee2d53930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7539df5740 4 client.16344 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ee2caeb80 con 0x557ee2d514f0 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7534da4640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2d514f0 0x557ee2d53930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f750ffff640 10 client.16344 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.329+0000 7f7534da4640 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2d514f0 0x557ee2d53930 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb1cab1461ed17d server_cookie=27b2ecd988ec6f64 in_seq=0 out_seq=0 2024-05-20T20:59:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f750ffff640 1 -- 172.21.15.26:0/3434356650 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ee2caeb80 con 0x557ee2d514f0 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f750ffff640 10 client.16344 handle_command_reply: tid=0 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 client.16344 shutdown 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 msgr2=0x557ee2d4cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4cd90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f75280cb000 tx=0x7f75280cac40 comp rx=0 tx=0).stop 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 shutdown_connections 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2d514f0 0x557ee2d53930 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4cd90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d45dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 --2- 172.21.15.26:0/3434356650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d45880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 >> 172.21.15.26:0/3434356650 conn(0x557ee2d36d30 msgr2=0x557ee2d35fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.330+0000 7f750ffff640 0 client.16344 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.331+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 shutdown_connections 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.331+0000 7f7539df5740 1 -- 172.21.15.26:0/3434356650 wait complete. 2024-05-20T20:59:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.331+0000 7f7539df5740 20 client.16344 trim_cache size 0 max 16384 2024-05-20T20:59:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 Processor -- start 2024-05-20T20:59:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 -- start start 2024-05-20T20:59:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d3d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75301ac740 con 0x557ee2d41220 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f75301aa010 con 0x557ee2d3c0b0 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7539df5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75301a7e60 con 0x557ee2d3b6c0 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d3d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d3d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44500/0 (socket says 172.21.15.26:44500) 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 -- 172.21.15.26:0/4222424895 learned_addr learned my addr 172.21.15.26:0/4222424895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f752ffff640 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f7534da4640 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 -- 172.21.15.26:0/4222424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3b6c0 msgr2=0x557ee2d3d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 -- 172.21.15.26:0/4222424895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3c0b0 msgr2=0x557ee2d4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f75355a5640 1 -- 172.21.15.26:0/4222424895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee2d3c940 con 0x557ee2d41220 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.347+0000 7f752ffff640 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.348+0000 7f75355a5640 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d3d870 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f752802ae80 tx=0x7f75280099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffc0e8a3a4d75ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.348+0000 7f751a7fc640 1 -- 172.21.15.26:0/4222424895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75280cb2d0 con 0x557ee2d41220 2024-05-20T20:59:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.348+0000 7f751a7fc640 1 -- 172.21.15.26:0/4222424895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7528036020 con 0x557ee2d41220 2024-05-20T20:59:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.348+0000 7f751a7fc640 1 -- 172.21.15.26:0/4222424895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75280c7300 con 0x557ee2d41220 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 -- 172.21.15.26:0/4222424895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 msgr2=0x557ee2d3d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d3d870 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f752802ae80 tx=0x7f75280099f0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 -- 172.21.15.26:0/4222424895 shutdown_connections 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d3d870 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 --2- 172.21.15.26:0/4222424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3b6c0 0x557ee2d3d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 -- 172.21.15.26:0/4222424895 >> 172.21.15.26:0/4222424895 conn(0x557ee2d36d30 msgr2=0x557ee2d45880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 -- 172.21.15.26:0/4222424895 shutdown_connections 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 -- 172.21.15.26:0/4222424895 wait complete. 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 -- start start 2024-05-20T20:59:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.349+0000 7f7539df5740 1 Processor -- start 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75301a84d0 con 0x557ee2d41220 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f75301a9940 con 0x557ee2d3b6c0 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75301a8ba0 con 0x557ee2d3c0b0 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7539df5740 10 client.0 _ll_register_callbacks cb 0x7f75280095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f75355a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44504/0 (socket says 172.21.15.26:44504) 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f75355a5640 1 -- 172.21.15.26:0/2450534805 learned_addr learned my addr 172.21.15.26:0/2450534805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f7534da4640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.350+0000 7f752ffff640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f75355a5640 1 -- 172.21.15.26:0/2450534805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 msgr2=0x557ee2d4a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f75355a5640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f75355a5640 1 -- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 msgr2=0x557ee2d4a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f75355a5640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d4a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f75355a5640 1 -- 172.21.15.26:0/2450534805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee2d4be00 con 0x557ee2d41220 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f752ffff640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f75355a5640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4b810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7528004d10 tx=0x7f75280cf350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390943ae3a8d607 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f750ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f750ffff640 1 -- 172.21.15.26:0/2450534805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75280cb2d0 con 0x557ee2d41220 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f750ffff640 1 -- 172.21.15.26:0/2450534805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f7528004530 con 0x557ee2d41220 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7528036020 con 0x557ee2d41220 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f750ffff640 1 -- 172.21.15.26:0/2450534805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7528004770 con 0x557ee2d41220 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f7539df5740 10 client.16350 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.351+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ee2d4d110 con 0x557ee2d41220 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f750ffff640 1 -- 172.21.15.26:0/2450534805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f752808aae0 con 0x557ee2d41220 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f750ffff640 1 -- 172.21.15.26:0/2450534805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f75280c6b70 con 0x557ee2d41220 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7539df5740 10 client.16350 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7539df5740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7539df5740 20 client.16350 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7539df5740 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2dbc640 0x557ee2dbea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7539df5740 4 client.16350 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7528036020 con 0x557ee2dbc640 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.352+0000 7f7534da4640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2dbc640 0x557ee2dbea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.353+0000 7f750ffff640 10 client.16350 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.353+0000 7f7534da4640 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2dbc640 0x557ee2dbea60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48041255e6909f54 server_cookie=7bfe3c417054b6dd in_seq=0 out_seq=0 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f750ffff640 1 -- 172.21.15.26:0/2450534805 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f7528036020 con 0x557ee2dbc640 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f750ffff640 10 client.16350 handle_command_reply: tid=0 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 client.16350 shutdown 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 msgr2=0x557ee2d4b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4b810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7528004d10 tx=0x7f75280cf350 comp rx=0 tx=0).stop 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 shutdown_connections 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ee2dbc640 0x557ee2dbea60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ee2d41220 0x557ee2d4b810 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ee2d3c0b0 0x557ee2d4a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f750ffff640 0 client.16350 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 --2- 172.21.15.26:0/2450534805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ee2d3b6c0 0x557ee2d4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.383+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 >> 172.21.15.26:0/2450534805 conn(0x557ee2d36d30 msgr2=0x557ee2d45880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 shutdown_connections 2024-05-20T20:59:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7539df5740 1 -- 172.21.15.26:0/2450534805 wait complete. 2024-05-20T20:59:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7539df5740 20 client.16350 trim_cache size 0 max 16384 2024-05-20T20:59:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:59:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:59:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:59:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75200b28a0 msgr2=0x7f75200b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75200b28a0 0x7f75200b4d60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557ee2d3c910 tx=0x7f751c00e400 comp rx=0 tx=0).stop 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 msgr2=0x7f75301a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f75301a00a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f752802aeb0 tx=0x7f7528003c20 comp rx=0 tx=0).stop 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 shutdown_connections 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f75200b28a0 0x7f75200b4d60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7530103610 0x7f75301a00a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7530102c50 0x7f7530199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 --2- 172.21.15.26:0/64425211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7530101a60 0x7f7530198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 >> 172.21.15.26:0/64425211 conn(0x7f75300fcfd0 msgr2=0x7f75300fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.384+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 shutdown_connections 2024-05-20T20:59:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:15.385+0000 7f7537008640 1 -- 172.21.15.26:0/64425211 wait complete. 2024-05-20T20:59:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:15 smithi055 ceph-mon[35213]: pgmap v481: 97 pgs: 97 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T20:59:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:15 smithi144 ceph-mon[35660]: pgmap v481: 97 pgs: 97 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T20:59:15.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.54 MB/sec warmup 81 sec latency 3269.692 ms 2024-05-20T20:59:15.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:59:16.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.50 MB/sec warmup 82 sec latency 4269.783 ms 2024-05-20T20:59:16.950 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:17.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:17 smithi026 ceph-mon[31990]: pgmap v482: 97 pgs: 97 active+clean; 26 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T20:59:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:17 smithi055 ceph-mon[35213]: pgmap v482: 97 pgs: 97 active+clean; 26 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T20:59:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:17 smithi144 ceph-mon[35660]: pgmap v482: 97 pgs: 97 active+clean; 26 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T20:59:17.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.46 MB/sec warmup 83 sec latency 5269.811 ms 2024-05-20T20:59:17.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 -- 172.21.15.26:0/2915698692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001041a0 msgr2=0x7fc7001081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:17.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 --2- 172.21.15.26:0/2915698692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001041a0 0x7fc7001081c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80099c0 tx=0x7fc6e8009cd0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 -- 172.21.15.26:0/2915698692 shutdown_connections 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 --2- 172.21.15.26:0/2915698692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7001088f0 0x7fc700110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 --2- 172.21.15.26:0/2915698692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7001037f0 0x7fc700103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 --2- 172.21.15.26:0/2915698692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001041a0 0x7fc7001081c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 -- 172.21.15.26:0/2915698692 >> 172.21.15.26:0/2915698692 conn(0x7fc70007a6a0 msgr2=0x7fc70007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 -- 172.21.15.26:0/2915698692 shutdown_connections 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 -- 172.21.15.26:0/2915698692 wait complete. 2024-05-20T20:59:17.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.669+0000 7fc705657700 1 Processor -- start 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 -- start start 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 0x7fc70019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7001041a0 0x7fc70019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7001088f0 0x7fc7001998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc700113580 con 0x7fc7001037f0 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc700113400 con 0x7fc7001088f0 2024-05-20T20:59:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc705657700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc700113700 con 0x7fc7001041a0 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 0x7fc70019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 0x7fc70019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44534/0 (socket says 172.21.15.26:44534) 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 -- 172.21.15.26:0/3607668371 learned_addr learned my addr 172.21.15.26:0/3607668371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6ff7fe700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7001088f0 0x7fc7001998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6fe7fc700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7001041a0 0x7fc70019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 -- 172.21.15.26:0/3607668371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7001041a0 msgr2=0x7fc70019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7001041a0 0x7fc70019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 -- 172.21.15.26:0/3607668371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7001088f0 msgr2=0x7fc7001998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7001088f0 0x7fc7001998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.670+0000 7fc6feffd700 1 -- 172.21.15.26:0/3607668371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70019a040 con 0x7fc7001037f0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.671+0000 7fc6feffd700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 0x7fc70019f720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000d980 tx=0x7fc6f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.671+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0017070 con 0x7fc7001037f0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.671+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70019a330 con 0x7fc7001037f0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.671+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6f0005020 con 0x7fc7001037f0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.671+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000f980 con 0x7fc7001037f0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.671+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7001a65f0 con 0x7fc7001037f0 2024-05-20T20:59:17.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.672+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc700104c80 con 0x7fc7001037f0 2024-05-20T20:59:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.673+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc6f0004a30 con 0x7fc7001037f0 2024-05-20T20:59:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.674+0000 7fc6f7fff700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc6ec0b2880 0x7fc6ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.674+0000 7fc6fe7fc700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc6ec0b2880 0x7fc6ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.674+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fc6f00e24a0 con 0x7fc7001037f0 2024-05-20T20:59:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.674+0000 7fc6fe7fc700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc6ec0b2880 0x7fc6ec0b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8000c00 tx=0x7fc6e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.678+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6f0020020 con 0x7fc7001037f0 2024-05-20T20:59:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.945+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc70019ae10 con 0x7fc7001037f0 2024-05-20T20:59:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.946+0000 7fc6f7fff700 1 -- 172.21.15.26:0/3607668371 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fc6f00aa160 con 0x7fc7001037f0 2024-05-20T20:59:17.946 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:59:17.946 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:59:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc6ec0b2880 msgr2=0x7fc6ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc6ec0b2880 0x7fc6ec0b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8000c00 tx=0x7fc6e8005fb0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 msgr2=0x7fc70019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 0x7fc70019f720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000d980 tx=0x7fc6f000dc90 comp rx=0 tx=0).stop 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 shutdown_connections 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc6ec0b2880 0x7fc6ec0b4d40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7001037f0 0x7fc70019f720 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7001088f0 0x7fc7001998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 --2- 172.21.15.26:0/3607668371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7001041a0 0x7fc70019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 >> 172.21.15.26:0/3607668371 conn(0x7fc70007a6a0 msgr2=0x7fc700101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 shutdown_connections 2024-05-20T20:59:17.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:17.948+0000 7fc705657700 1 -- 172.21.15.26:0/3607668371 wait complete. 2024-05-20T20:59:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:59:18.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:18 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3607668371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:18.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:18 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3607668371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:18 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3607668371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:18.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.42 MB/sec warmup 84 sec latency 6269.908 ms 2024-05-20T20:59:19.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:19 smithi026 ceph-mon[31990]: pgmap v483: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-05-20T20:59:19.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:19 smithi055 ceph-mon[35213]: pgmap v483: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-05-20T20:59:19.461 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id d2603423 --release --await' 2024-05-20T20:59:19.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id d2603423 --release --await 2024-05-20T20:59:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:19 smithi144 ceph-mon[35660]: pgmap v483: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-05-20T20:59:19.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.38 MB/sec warmup 85 sec latency 7269.969 ms 2024-05-20T20:59:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 -- 172.21.15.26:0/2925983990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102440 msgr2=0x7f32b81028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/2925983990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102440 0x7f32b81028c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009990 tx=0x7f32a8009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 -- 172.21.15.26:0/2925983990 shutdown_connections 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/2925983990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b8102e00 0x7f32b810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/2925983990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32b8108520 0x7f32b8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/2925983990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102440 0x7f32b81028c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 -- 172.21.15.26:0/2925983990 >> 172.21.15.26:0/2925983990 conn(0x7f32b80793f0 msgr2=0x7f32b8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 -- 172.21.15.26:0/2925983990 shutdown_connections 2024-05-20T20:59:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 -- 172.21.15.26:0/2925983990 wait complete. 2024-05-20T20:59:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.075+0000 7f32bf9f3700 1 Processor -- start 2024-05-20T20:59:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 -- start start 2024-05-20T20:59:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b8102440 0x7f32b819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 0x7f32b819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32b8108520 0x7f32b8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112480 con 0x7f32b8102e00 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112300 con 0x7f32b8108520 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bf9f3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112600 con 0x7f32b8102440 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 0x7f32b819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 0x7f32b819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40270/0 (socket says 172.21.15.26:40270) 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 -- 172.21.15.26:0/4225876192 learned_addr learned my addr 172.21.15.26:0/4225876192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bdf90700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32b8108520 0x7f32b8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bd78f700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b8102440 0x7f32b819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 -- 172.21.15.26:0/4225876192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b8102440 msgr2=0x7f32b819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b8102440 0x7f32b819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 -- 172.21.15.26:0/4225876192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32b8108520 msgr2=0x7f32b8199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32b8108520 0x7f32b8199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.076+0000 7f32bcf8e700 1 -- 172.21.15.26:0/4225876192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b819a0b0 con 0x7f32b8102e00 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.077+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 0x7f32b819fcd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f32a80057c0 tx=0x7f32a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.077+0000 7f32ae7fc700 1 -- 172.21.15.26:0/4225876192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8027070 con 0x7f32b8102e00 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.077+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b819a340 con 0x7f32b8102e00 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.077+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81a6660 con 0x7f32b8102e00 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.077+0000 7f32ae7fc700 1 -- 172.21.15.26:0/4225876192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32a80095d0 con 0x7f32b8102e00 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.077+0000 7f32ae7fc700 1 -- 172.21.15.26:0/4225876192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80176e0 con 0x7f32b8102e00 2024-05-20T20:59:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.080+0000 7f32ae7fc700 1 -- 172.21.15.26:0/4225876192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f32a802a070 con 0x7f32b8102e00 2024-05-20T20:59:20.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.080+0000 7f32ae7fc700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f32a40b2880 0x7f32a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ae7fc700 1 -- 172.21.15.26:0/4225876192 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f32a80e2e10 con 0x7f32b8102e00 2024-05-20T20:59:20.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32bd78f700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f32a40b2880 0x7f32a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 Processor -- start 2024-05-20T20:59:20.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 -- start start 2024-05-20T20:59:20.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32bd78f700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f32a40b2880 0x7f32a40b4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32b819aa90 tx=0x7f32b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94795aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b94790130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b9479c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b946e9d70 con 0x561b947956c0 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561b946d9910 con 0x561b94790670 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32ca289b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b946dd950 con 0x561b9478fcb0 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.081+0000 7f32bd78f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94795aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94795aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40274/0 (socket says 172.21.15.26:40274) 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 -- 172.21.15.26:0/1337759869 learned_addr learned my addr 172.21.15.26:0/1337759869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:20.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b94790130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bdf90700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b9479c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 -- 172.21.15.26:0/1337759869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 msgr2=0x561b94790130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b94790130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 -- 172.21.15.26:0/1337759869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 msgr2=0x561b9479c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b9479c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 -- 172.21.15.26:0/1337759869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b94682fb0 con 0x561b947956c0 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bdf90700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b9479c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b94790130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32bd78f700 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94795aa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f32b4010e00 tx=0x7f32b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1022d2e4fbfc7766 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32a37fe700 1 -- 172.21.15.26:0/1337759869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b401e420 con 0x561b947956c0 2024-05-20T20:59:20.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32a37fe700 1 -- 172.21.15.26:0/1337759869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b401ea50 con 0x561b947956c0 2024-05-20T20:59:20.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.082+0000 7f32a37fe700 1 -- 172.21.15.26:0/1337759869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b401ca10 con 0x561b947956c0 2024-05-20T20:59:20.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 -- 172.21.15.26:0/1337759869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 msgr2=0x561b94795aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94795aa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f32b4010e00 tx=0x7f32b4013000 comp rx=0 tx=0).stop 2024-05-20T20:59:20.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 -- 172.21.15.26:0/1337759869 shutdown_connections 2024-05-20T20:59:20.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94795aa0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b9479c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1337759869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b94790130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 -- 172.21.15.26:0/1337759869 >> 172.21.15.26:0/1337759869 conn(0x561b946b7240 msgr2=0x561b947863e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 -- 172.21.15.26:0/1337759869 shutdown_connections 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.083+0000 7f32ca289b80 1 -- 172.21.15.26:0/1337759869 wait complete. 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 -- start start 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 Processor -- start 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b9479c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b94799310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94799850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b945dffa0 con 0x561b947956c0 2024-05-20T20:59:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561b94641da0 con 0x561b94790670 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b9479cb50 con 0x561b9478fcb0 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.084+0000 7f32ca289b80 10 client.0 _ll_register_callbacks cb 0x561b94674950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94799850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bd78f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b9479c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94799850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40284/0 (socket says 172.21.15.26:40284) 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 -- 172.21.15.26:0/1141155578 learned_addr learned my addr 172.21.15.26:0/1141155578 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b94799310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 -- 172.21.15.26:0/1141155578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 msgr2=0x561b9479c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b9479c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 -- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 msgr2=0x561b94799310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b94799310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 -- 172.21.15.26:0/1141155578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b946298d0 con 0x561b947956c0 2024-05-20T20:59:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b94799310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32bdf90700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94799850 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f32b0011cb0 tx=0x7f32b000f1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754d17f5db8d8fad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.085+0000 7f32a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32a0ff9700 1 -- 172.21.15.26:0/1141155578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b001b020 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b401ea50 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32a0ff9700 1 -- 172.21.15.26:0/1141155578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f32b000fb10 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32a0ff9700 1 -- 172.21.15.26:0/1141155578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32b0026050 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32ca289b80 10 client.16368 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b94693760 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.086+0000 7f32a0ff9700 1 -- 172.21.15.26:0/1141155578 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b401ea50 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32a0ff9700 1 -- 172.21.15.26:0/1141155578 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f32b0015430 con 0x561b947956c0 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32ca289b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32ca289b80 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32ca289b80 20 client.16368 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b947a9ac0 0x561b947abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32ca289b80 4 client.16368 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b946ebcc0 con 0x561b947a9ac0 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.087+0000 7f32bd78f700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b947a9ac0 0x561b947abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.088+0000 7f32a0ff9700 10 client.16368 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.088+0000 7f32bd78f700 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b947a9ac0 0x561b947abee0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7371670c66b6ada server_cookie=1d4b5fc109c9d07d in_seq=0 out_seq=0 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32a0ff9700 1 -- 172.21.15.26:0/1141155578 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561b946ebcc0 con 0x561b947a9ac0 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32a0ff9700 10 client.16368 handle_command_reply: tid=0 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 client.16368 shutdown 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 msgr2=0x561b94799850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94799850 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f32b0011cb0 tx=0x7f32b000f1d0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 shutdown_connections 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b947a9ac0 0x561b947abee0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94799850 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b9478fcb0 0x561b9479c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 --2- 172.21.15.26:0/1141155578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b94790670 0x561b94799310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 >> 172.21.15.26:0/1141155578 conn(0x561b946b7240 msgr2=0x561b94785af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32a0ff9700 0 client.16368 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 shutdown_connections 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 1 -- 172.21.15.26:0/1141155578 wait complete. 2024-05-20T20:59:20.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.089+0000 7f32ca289b80 20 client.16368 trim_cache size 0 max 16384 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 Processor -- start 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 -- start start 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b94798270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b94794ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94786270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f329c00fdf0 con 0x561b947956c0 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f329c00d680 con 0x561b9478fcb0 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32ca289b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f329c00b4d0 con 0x561b94790670 2024-05-20T20:59:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bdf90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94786270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bdf90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94786270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40300/0 (socket says 172.21.15.26:40300) 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bdf90700 1 -- 172.21.15.26:0/903821790 learned_addr learned my addr 172.21.15.26:0/903821790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b94794ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bd78f700 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b94798270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bdf90700 1 -- 172.21.15.26:0/903821790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 msgr2=0x561b94794ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bdf90700 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b94794ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.114+0000 7f32bdf90700 1 -- 172.21.15.26:0/903821790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 msgr2=0x561b94798270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32bdf90700 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b94798270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32bdf90700 1 -- 172.21.15.26:0/903821790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b947a70e0 con 0x561b947956c0 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32bd78f700 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b94798270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32bdf90700 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94786270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f32b0015a40 tx=0x7f32b0007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22267c7d10d49976 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32a2ffd700 1 -- 172.21.15.26:0/903821790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b001b070 con 0x561b947956c0 2024-05-20T20:59:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32a2ffd700 1 -- 172.21.15.26:0/903821790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b0015630 con 0x561b947956c0 2024-05-20T20:59:20.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.115+0000 7f32a2ffd700 1 -- 172.21.15.26:0/903821790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0017480 con 0x561b947956c0 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 -- 172.21.15.26:0/903821790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 msgr2=0x561b94786270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94786270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f32b0015a40 tx=0x7f32b0007c00 comp rx=0 tx=0).stop 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 -- 172.21.15.26:0/903821790 shutdown_connections 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b94786270 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b94798270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 --2- 172.21.15.26:0/903821790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b94794ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 -- 172.21.15.26:0/903821790 >> 172.21.15.26:0/903821790 conn(0x561b946b7240 msgr2=0x561b947930f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 -- 172.21.15.26:0/903821790 shutdown_connections 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 -- 172.21.15.26:0/903821790 wait complete. 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 -- start start 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.116+0000 7f32ca289b80 1 Processor -- start 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b947a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b947a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b947b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f329c00bb40 con 0x561b947956c0 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f329c00cfb0 con 0x561b9478fcb0 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f329c00c210 con 0x561b94790670 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32bd78f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b947a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32bdf90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b947b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32ca289b80 10 client.0 _ll_register_callbacks cb 0x561b94674950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32bd78f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b947a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44014/0 (socket says 172.21.15.26:44014) 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32bd78f700 1 -- 172.21.15.26:0/3377820077 learned_addr learned my addr 172.21.15.26:0/3377820077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.117+0000 7f32bcf8e700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b947a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bdf90700 1 -- 172.21.15.26:0/3377820077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 msgr2=0x561b947a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bdf90700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b947a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bdf90700 1 -- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 msgr2=0x561b947a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bdf90700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b947a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bdf90700 1 -- 172.21.15.26:0/3377820077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b947b0d30 con 0x561b947956c0 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bd78f700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b947a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32bdf90700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b947b07f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f32b0015850 tx=0x7f32b000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4474ea2b9da13fe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32a0ff9700 1 -- 172.21.15.26:0/3377820077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b003b060 con 0x561b947956c0 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32a0ff9700 1 -- 172.21.15.26:0/3377820077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f32b000fd20 con 0x561b947956c0 2024-05-20T20:59:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b0015630 con 0x561b947956c0 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32a0ff9700 1 -- 172.21.15.26:0/3377820077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32b001b070 con 0x561b947956c0 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32ca289b80 10 client.16374 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b9478b8a0 con 0x561b947956c0 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.118+0000 7f32a0ff9700 1 -- 172.21.15.26:0/3377820077 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b0015630 con 0x561b947956c0 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32a0ff9700 1 -- 172.21.15.26:0/3377820077 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f32b0017480 con 0x561b947956c0 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32ca289b80 10 client.16374 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32ca289b80 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32ca289b80 20 client.16374 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32ca289b80 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b9478bf60 0x561b9478e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32ca289b80 4 client.16374 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b946e54a0 con 0x561b9478bf60 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32bd78f700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b9478bf60 0x561b9478e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32a0ff9700 10 client.16374 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32bd78f700 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b9478bf60 0x561b9478e380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcb6542e59f4fe1 server_cookie=52af3e261376c465 in_seq=0 out_seq=0 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32a0ff9700 1 -- 172.21.15.26:0/3377820077 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b946e54a0 con 0x561b9478bf60 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.119+0000 7f32a0ff9700 10 client.16374 handle_command_reply: tid=0 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 client.16374 shutdown 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 msgr2=0x561b947b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b947b07f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f32b0015850 tx=0x7f32b000f710 comp rx=0 tx=0).stop 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 shutdown_connections 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b9478bf60 0x561b9478e380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b947956c0 0x561b947b07f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b9478fcb0 0x561b947a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 --2- 172.21.15.26:0/3377820077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b94790670 0x561b947a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 >> 172.21.15.26:0/3377820077 conn(0x561b946b7240 msgr2=0x561b94794d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32a0ff9700 0 client.16374 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 shutdown_connections 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 1 -- 172.21.15.26:0/3377820077 wait complete. 2024-05-20T20:59:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.120+0000 7f32ca289b80 20 client.16374 trim_cache size 0 max 16384 2024-05-20T20:59:20.126 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f32a40b2880 msgr2=0x7f32a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f32a40b2880 0x7f32a40b4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32b819aa90 tx=0x7f32b400b410 comp rx=0 tx=0).stop 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 msgr2=0x7f32b819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 0x7f32b819fcd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f32a80057c0 tx=0x7f32a80057f0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 shutdown_connections 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32b8108520 0x7f32b8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b8102440 0x7f32b819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f32a40b2880 0x7f32a40b4d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 --2- 172.21.15.26:0/4225876192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32b8102e00 0x7f32b819fcd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 >> 172.21.15.26:0/4225876192 conn(0x7f32b80793f0 msgr2=0x7f32b806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 shutdown_connections 2024-05-20T20:59:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:20.121+0000 7f32bf9f3700 1 -- 172.21.15.26:0/4225876192 wait complete. 2024-05-20T20:59:20.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.34 MB/sec warmup 86 sec latency 8270.054 ms 2024-05-20T20:59:21.166 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:21 smithi026 ceph-mon[31990]: pgmap v484: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1015 KiB/s wr, 74 op/s 2024-05-20T20:59:21.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:21 smithi144 ceph-mon[35660]: pgmap v484: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1015 KiB/s wr, 74 op/s 2024-05-20T20:59:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:21 smithi055 ceph-mon[35213]: pgmap v484: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1015 KiB/s wr, 74 op/s 2024-05-20T20:59:21.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.30 MB/sec warmup 87 sec latency 9270.100 ms 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 -- 172.21.15.26:0/3215702522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 msgr2=0x7f208c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 --2- 172.21.15.26:0/3215702522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c06b630 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2088009990 tx=0x7f2088009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 -- 172.21.15.26:0/3215702522 shutdown_connections 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 --2- 172.21.15.26:0/3215702522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c06b630 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 --2- 172.21.15.26:0/3215702522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f208c10fff0 0x7f208c1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 --2- 172.21.15.26:0/3215702522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f208c06bb70 0x7f208c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 -- 172.21.15.26:0/3215702522 >> 172.21.15.26:0/3215702522 conn(0x7f208c0793f0 msgr2=0x7f208c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 -- 172.21.15.26:0/3215702522 shutdown_connections 2024-05-20T20:59:22.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.532+0000 7f20943ae700 1 -- 172.21.15.26:0/3215702522 wait complete. 2024-05-20T20:59:22.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 Processor -- start 2024-05-20T20:59:22.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 -- start start 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f208c06bb70 0x7f208c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f208c10fff0 0x7f208c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f208c115820 con 0x7f208c06b250 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1156a0 con 0x7f208c10fff0 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f20943ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1159a0 con 0x7f208c06bb70 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.533+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40326/0 (socket says 172.21.15.26:40326) 2024-05-20T20:59:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 -- 172.21.15.26:0/3571561034 learned_addr learned my addr 172.21.15.26:0/3571561034 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f2091949700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f208c06bb70 0x7f208c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209294b700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f208c10fff0 0x7f208c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 -- 172.21.15.26:0/3571561034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f208c06bb70 msgr2=0x7f208c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f208c06bb70 0x7f208c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 -- 172.21.15.26:0/3571561034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f208c10fff0 msgr2=0x7f208c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f208c10fff0 0x7f208c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 -- 172.21.15.26:0/3571561034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c1a0da0 con 0x7f208c06b250 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f209214a700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c19bd90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2088000c00 tx=0x7f20880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f20837fe700 1 -- 172.21.15.26:0/3571561034 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088027070 con 0x7f208c06b250 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c1a1030 con 0x7f208c06b250 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f20837fe700 1 -- 172.21.15.26:0/3571561034 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f208800fb80 con 0x7f208c06b250 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f20837fe700 1 -- 172.21.15.26:0/3571561034 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088022d00 con 0x7f208c06b250 2024-05-20T20:59:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.534+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c1a8910 con 0x7f208c06b250 2024-05-20T20:59:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.537+0000 7f20837fe700 1 -- 172.21.15.26:0/3571561034 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f208802a070 con 0x7f208c06b250 2024-05-20T20:59:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.538+0000 7f20837fe700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20780b2880 0x7f20780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.538+0000 7f2091949700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20780b2880 0x7f20780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.538+0000 7f20837fe700 1 -- 172.21.15.26:0/3571561034 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f20880e3050 con 0x7f208c06b250 2024-05-20T20:59:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.538+0000 7f209ec44b80 1 Processor -- start 2024-05-20T20:59:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.538+0000 7f209ec44b80 1 -- start start 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.538+0000 7f2091949700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20780b2880 0x7f20780b4d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f208c101710 tx=0x7f207c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1dbf0 0x56118bd1e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 0x56118bd2a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209ec44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56118bc7be70 con 0x56118bd1e5b0 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209ec44b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56118bc6ba10 con 0x56118bd22720 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209ec44b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56118bc6fa50 con 0x56118bd1dbf0 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 0x56118bd2a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 0x56118bd2a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40330/0 (socket says 172.21.15.26:40330) 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 -- 172.21.15.26:0/2689951914 learned_addr learned my addr 172.21.15.26:0/2689951914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f2091949700 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1dbf0 0x56118bd1e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209214a700 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 -- 172.21.15.26:0/2689951914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1dbf0 msgr2=0x56118bd1e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1dbf0 0x56118bd1e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 -- 172.21.15.26:0/2689951914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 msgr2=0x56118bd22b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd22b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 -- 172.21.15.26:0/2689951914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118bc150b0 con 0x56118bd1e5b0 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209214a700 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f209294b700 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 0x56118bd2a620 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2084009ef0 tx=0x7f2084009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dff9afdb849982 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f2080d70700 1 -- 172.21.15.26:0/2689951914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208401a070 con 0x56118bd1e5b0 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.539+0000 7f2080d70700 1 -- 172.21.15.26:0/2689951914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2084015410 con 0x56118bd1e5b0 2024-05-20T20:59:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f2080d70700 1 -- 172.21.15.26:0/2689951914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208400a600 con 0x56118bd1e5b0 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 -- 172.21.15.26:0/2689951914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 msgr2=0x56118bd2a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 0x56118bd2a620 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2084009ef0 tx=0x7f2084009f20 comp rx=0 tx=0).stop 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 -- 172.21.15.26:0/2689951914 shutdown_connections 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1e5b0 0x56118bd2a620 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 --2- 172.21.15.26:0/2689951914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1dbf0 0x56118bd1e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 -- 172.21.15.26:0/2689951914 >> 172.21.15.26:0/2689951914 conn(0x56118bc49340 msgr2=0x56118bd188e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 -- 172.21.15.26:0/2689951914 shutdown_connections 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.540+0000 7f209ec44b80 1 -- 172.21.15.26:0/2689951914 wait complete. 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 -- start start 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 Processor -- start 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd27b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56118bb720a0 con 0x56118bd1dbf0 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56118bbd3ea0 con 0x56118bd1e5b0 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56118bd2ac80 con 0x56118bd22720 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209ec44b80 10 client.0 _ll_register_callbacks cb 0x56118bc06a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40336/0 (socket says 172.21.15.26:40336) 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209214a700 1 -- 172.21.15.26:0/715190399 learned_addr learned my addr 172.21.15.26:0/715190399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f2091949700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.541+0000 7f209294b700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd27b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209214a700 1 -- 172.21.15.26:0/715190399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 msgr2=0x56118bd27b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209214a700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd27b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209214a700 1 -- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 msgr2=0x56118bd275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209214a700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209214a700 1 -- 172.21.15.26:0/715190399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118bbbb9d0 con 0x56118bd1dbf0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209214a700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2a2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2088016ff0 tx=0x7f20880227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c8c837e281efdd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f2075ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f2075ffb700 1 -- 172.21.15.26:0/715190399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880a4e60 con 0x56118bd1dbf0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f2075ffb700 1 -- 172.21.15.26:0/715190399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f2088092300 con 0x56118bd1dbf0 2024-05-20T20:59:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2084015410 con 0x56118bd1dbf0 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f2075ffb700 1 -- 172.21.15.26:0/715190399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20880aa740 con 0x56118bd1dbf0 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209ec44b80 10 client.16386 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56118bc25860 con 0x56118bd1dbf0 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f2075ffb700 1 -- 172.21.15.26:0/715190399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2084015410 con 0x56118bd1dbf0 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.542+0000 7f2075ffb700 1 -- 172.21.15.26:0/715190399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f20880a99a0 con 0x56118bd1dbf0 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f209ec44b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f209ec44b80 10 client.16386 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f209ec44b80 20 client.16386 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f209ec44b80 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bd39a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f209ec44b80 4 client.16386 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56118bc7ddc0 con 0x56118bd37640 2024-05-20T20:59:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f2091949700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bd39a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f2091949700 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bd39a30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad891678d023292c server_cookie=160a4b8da3a49e5 in_seq=0 out_seq=0 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.543+0000 7f2075ffb700 10 client.16386 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f2075ffb700 1 -- 172.21.15.26:0/715190399 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56118bc7ddc0 con 0x56118bd37640 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f2075ffb700 10 client.16386 handle_command_reply: tid=0 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 client.16386 shutdown 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 msgr2=0x56118bd2a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2a2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2088016ff0 tx=0x7f20880227e0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 shutdown_connections 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bd39a30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f2075ffb700 0 client.16386 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2a2f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd27b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 --2- 172.21.15.26:0/715190399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 >> 172.21.15.26:0/715190399 conn(0x56118bc49340 msgr2=0x56118bd1cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:22.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 shutdown_connections 2024-05-20T20:59:22.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 1 -- 172.21.15.26:0/715190399 wait complete. 2024-05-20T20:59:22.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.544+0000 7f209ec44b80 20 client.16386 trim_cache size 0 max 16384 2024-05-20T20:59:22.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49634 3.26 MB/sec warmup 88 sec latency 10270.159 ms 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 Processor -- start 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 -- start start 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd23ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1b2740 con 0x56118bd1dbf0 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1affd0 con 0x56118bd1e5b0 2024-05-20T20:59:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209ec44b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1ade20 con 0x56118bd22720 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd12e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40346/0 (socket says 172.21.15.26:40346) 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209214a700 1 -- 172.21.15.26:0/143326221 learned_addr learned my addr 172.21.15.26:0/143326221 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f2091949700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd23ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.565+0000 7f209294b700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209214a700 1 -- 172.21.15.26:0/143326221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 msgr2=0x56118bd24010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209214a700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd24010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209214a700 1 -- 172.21.15.26:0/143326221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 msgr2=0x56118bd23ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209214a700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd23ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209214a700 1 -- 172.21.15.26:0/143326221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118bd22e70 con 0x56118bd1dbf0 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f2091949700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209294b700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd24010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209214a700 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd12e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20880aa440 tx=0x7f208800ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abd896f581e77df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f2077fff700 1 -- 172.21.15.26:0/143326221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880a4e60 con 0x56118bd1dbf0 2024-05-20T20:59:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f2077fff700 1 -- 172.21.15.26:0/143326221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20880aeaf0 con 0x56118bd1dbf0 2024-05-20T20:59:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f2077fff700 1 -- 172.21.15.26:0/143326221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880a60b0 con 0x56118bd1dbf0 2024-05-20T20:59:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209ec44b80 1 -- 172.21.15.26:0/143326221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 msgr2=0x56118bd12e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.566+0000 7f209ec44b80 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd12e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20880aa440 tx=0x7f208800ff40 comp rx=0 tx=0).stop 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- 172.21.15.26:0/143326221 shutdown_connections 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd12e70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd22720 0x56118bd24010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 --2- 172.21.15.26:0/143326221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd1e5b0 0x56118bd23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- 172.21.15.26:0/143326221 >> 172.21.15.26:0/143326221 conn(0x56118bc49340 msgr2=0x56118bd233c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- 172.21.15.26:0/143326221 shutdown_connections 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- 172.21.15.26:0/143326221 wait complete. 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- start start 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 Processor -- start 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1e5b0 0x56118bd2b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd2b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1ae490 con 0x56118bd1dbf0 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1af900 con 0x56118bd22720 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1aeb60 con 0x56118bd1e5b0 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209ec44b80 10 client.0 _ll_register_callbacks cb 0x56118bc06a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.567+0000 7f209214a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40358/0 (socket says 172.21.15.26:40358) 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 -- 172.21.15.26:0/1855575360 learned_addr learned my addr 172.21.15.26:0/1855575360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209294b700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd2b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f2091949700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1e5b0 0x56118bd2b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 -- 172.21.15.26:0/1855575360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1e5b0 msgr2=0x56118bd2b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1e5b0 0x56118bd2b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 -- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 msgr2=0x56118bd2b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd2b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 -- 172.21.15.26:0/1855575360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118bda2260 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209214a700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2ae90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f208800ffd0 tx=0x7f20880b3310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b89a4e3852ac438 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f2075ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f2075ffb700 1 -- 172.21.15.26:0/1855575360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880a4e60 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f2075ffb700 1 -- 172.21.15.26:0/1855575360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f2088092300 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20880aeaf0 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f2075ffb700 1 -- 172.21.15.26:0/1855575360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f208800f900 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209ec44b80 10 client.16392 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56118bda2260 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.568+0000 7f2075ffb700 1 -- 172.21.15.26:0/1855575360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20880aeaf0 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f2075ffb700 1 -- 172.21.15.26:0/1855575360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f20880bfce0 con 0x56118bd1dbf0 2024-05-20T20:59:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f209ec44b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f209ec44b80 10 client.16392 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f209ec44b80 20 client.16392 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f209ec44b80 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bda58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f209ec44b80 4 client.16392 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d2603423", "release": true, "await": true}] 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d2603423", "release": true, "await": true}) -- 0x56118bc775a0 con 0x56118bd37640 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f2091949700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bda58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f2091949700 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bda58a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aacae1232fdcb97f server_cookie=b4f8c5679cefcd3d in_seq=0 out_seq=0 2024-05-20T20:59:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.569+0000 7f2075ffb700 10 client.16392 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f2075ffb700 1 -- 172.21.15.26:0/1855575360 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56118bc775a0 con 0x56118bd37640 2024-05-20T20:59:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f2075ffb700 10 client.16392 handle_command_reply: tid=0 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 client.16392 shutdown 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 msgr2=0x56118bd2ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2ae90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f208800ffd0 tx=0x7f20880b3310 comp rx=0 tx=0).stop 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 shutdown_connections 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56118bd37640 0x56118bda58a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56118bd1dbf0 0x56118bd2ae90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56118bd22720 0x56118bd2b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f2075ffb700 0 client.16392 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 --2- 172.21.15.26:0/1855575360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56118bd1e5b0 0x56118bd2b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 >> 172.21.15.26:0/1855575360 conn(0x56118bc49340 msgr2=0x56118bd24e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 shutdown_connections 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 1 -- 172.21.15.26:0/1855575360 wait complete. 2024-05-20T20:59:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.582+0000 7f209ec44b80 20 client.16392 trim_cache size 0 max 16384 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 12, 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "d2603423": { 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 12, 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:59:22.584 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.0, 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:22.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:22.586 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:22.586 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:22.586 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:22.586 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20780b2880 msgr2=0x7f20780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20780b2880 0x7f20780b4d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f208c101710 tx=0x7f207c00b410 comp rx=0 tx=0).stop 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 msgr2=0x7f208c19bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c19bd90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2088000c00 tx=0x7f20880047b0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 shutdown_connections 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20780b2880 0x7f20780b4d40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b250 0x7f208c19bd90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f208c10fff0 0x7f208c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 --2- 172.21.15.26:0/3571561034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f208c06bb70 0x7f208c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 >> 172.21.15.26:0/3571561034 conn(0x7f208c0793f0 msgr2=0x7f208c1114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 shutdown_connections 2024-05-20T20:59:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:22.583+0000 7f20943ae700 1 -- 172.21.15.26:0/3571561034 wait complete. 2024-05-20T20:59:22.588 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:59:22.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:59:23.383 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:23 smithi026 ceph-mon[31990]: pgmap v485: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 670 KiB/s wr, 50 op/s 2024-05-20T20:59:23.412 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd2603423', total seconds elapsed: 24.6 2024-05-20T20:59:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:23 smithi055 ceph-mon[35213]: pgmap v485: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 670 KiB/s wr, 50 op/s 2024-05-20T20:59:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:23 smithi144 ceph-mon[35660]: pgmap v485: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 670 KiB/s wr, 50 op/s 2024-05-20T20:59:23.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 50126 3.24 MB/sec warmup 89 sec latency 10295.073 ms 2024-05-20T20:59:24.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 50735 3.23 MB/sec warmup 90 sec latency 19.007 ms 2024-05-20T20:59:25.418 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:25 smithi026 ceph-mon[31990]: pgmap v486: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 490 KiB/s wr, 46 op/s 2024-05-20T20:59:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 -- 172.21.15.26:0/872392590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06b220 msgr2=0x7f948c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 --2- 172.21.15.26:0/872392590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06b220 0x7f948c06b6a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9478009990 tx=0x7f9478009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 -- 172.21.15.26:0/872392590 shutdown_connections 2024-05-20T20:59:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 --2- 172.21.15.26:0/872392590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c06bbe0 0x7f948c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 --2- 172.21.15.26:0/872392590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c1095b0 0x7f948c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 --2- 172.21.15.26:0/872392590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06b220 0x7f948c06b6a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 -- 172.21.15.26:0/872392590 >> 172.21.15.26:0/872392590 conn(0x7f948c0793f0 msgr2=0x7f948c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 -- 172.21.15.26:0/872392590 shutdown_connections 2024-05-20T20:59:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.440+0000 7f9492b65700 1 -- 172.21.15.26:0/872392590 wait complete. 2024-05-20T20:59:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 Processor -- start 2024-05-20T20:59:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 -- start start 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c06b220 0x7f948c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 0x7f948c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1095b0 0x7f948c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1135f0 con 0x7f948c06bbe0 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f948c113470 con 0x7f948c06b220 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9492b65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f948c113770 con 0x7f948c1095b0 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 0x7f948c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9490901700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c06b220 0x7f948c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 0x7f948c102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40372/0 (socket says 172.21.15.26:40372) 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f948bfff700 1 -- 172.21.15.26:0/2774343154 learned_addr learned my addr 172.21.15.26:0/2774343154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.441+0000 7f9491102700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1095b0 0x7f948c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9490901700 1 -- 172.21.15.26:0/2774343154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1095b0 msgr2=0x7f948c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9490901700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1095b0 0x7f948c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9490901700 1 -- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 msgr2=0x7f948c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9490901700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 0x7f948c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9490901700 1 -- 172.21.15.26:0/2774343154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c1a8090 con 0x7f948c06b220 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9491102700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1095b0 0x7f948c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f948bfff700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 0x7f948c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9490901700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c06b220 0x7f948c102510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f948000e9f0 tx=0x7f948000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9489ffb700 1 -- 172.21.15.26:0/2774343154 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948000cbd0 con 0x7f948c06b220 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c1a8380 con 0x7f948c06b220 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9489ffb700 1 -- 172.21.15.26:0/2774343154 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f948000cd70 con 0x7f948c06b220 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9489ffb700 1 -- 172.21.15.26:0/2774343154 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480018680 con 0x7f948c06b220 2024-05-20T20:59:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.442+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c1a8910 con 0x7f948c06b220 2024-05-20T20:59:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.445+0000 7f9489ffb700 1 -- 172.21.15.26:0/2774343154 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9480010430 con 0x7f948c06b220 2024-05-20T20:59:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f9489ffb700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f947c0b2880 0x7f947c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f948bfff700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f947c0b2880 0x7f947c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f9489ffb700 1 -- 172.21.15.26:0/2774343154 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f94800e2390 con 0x7f948c06b220 2024-05-20T20:59:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 Processor -- start 2024-05-20T20:59:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 -- start start 2024-05-20T20:59:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f948bfff700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f947c0b2880 0x7f947c0b4d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f948c079ad0 tx=0x7f947801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd974da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd981100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e8dd8d2df0 con 0x55e8dd974920 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e8dd8c2990 con 0x55e8dd9752e0 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.446+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8dd8c69d0 con 0x55e8dd97a3a0 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd974da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd981100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd974da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40386/0 (socket says 172.21.15.26:40386) 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f948bfff700 1 -- 172.21.15.26:0/3153120259 learned_addr learned my addr 172.21.15.26:0/3153120259 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9490901700 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 -- 172.21.15.26:0/3153120259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 msgr2=0x55e8dd97a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 -- 172.21.15.26:0/3153120259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 msgr2=0x55e8dd974da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd974da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 -- 172.21.15.26:0/3153120259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8dd86c030 con 0x55e8dd9752e0 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9490901700 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f948bfff700 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd974da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9491102700 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd981100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f94840093a0 tx=0x7f9484009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75660c9b28221a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9476ffd700 1 -- 172.21.15.26:0/3153120259 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484019030 con 0x55e8dd9752e0 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9476ffd700 1 -- 172.21.15.26:0/3153120259 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9484004bb0 con 0x55e8dd9752e0 2024-05-20T20:59:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.447+0000 7f9476ffd700 1 -- 172.21.15.26:0/3153120259 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484005020 con 0x55e8dd9752e0 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3153120259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 msgr2=0x55e8dd981100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd981100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f94840093a0 tx=0x7f9484009920 comp rx=0 tx=0).stop 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3153120259 shutdown_connections 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd981100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3153120259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd974da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3153120259 >> 172.21.15.26:0/3153120259 conn(0x55e8dd8a02c0 msgr2=0x55e8dd96f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3153120259 shutdown_connections 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.448+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3153120259 wait complete. 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 -- start start 2024-05-20T20:59:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 Processor -- start 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 0x55e8dd980d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd97e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e8dd7c9020 con 0x55e8dd9752e0 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e8dd82ae20 con 0x55e8dd974920 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8dd981760 con 0x55e8dd97a3a0 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f949d3fbb80 10 client.0 _ll_register_callbacks cb 0x55e8dd85d9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd97e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f9490901700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 0x55e8dd980d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f9490901700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 0x55e8dd980d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44100/0 (socket says 172.21.15.26:44100) 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.449+0000 7f9490901700 1 -- 172.21.15.26:0/4149248434 learned_addr learned my addr 172.21.15.26:0/4149248434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9491102700 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9490901700 1 -- 172.21.15.26:0/4149248434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 msgr2=0x55e8dd97e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9490901700 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9490901700 1 -- 172.21.15.26:0/4149248434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 msgr2=0x55e8dd97e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9490901700 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd97e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9490901700 1 -- 172.21.15.26:0/4149248434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8dd812950 con 0x55e8dd974920 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f9490901700 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 0x55e8dd980d20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94800e01a0 tx=0x7f94800a9e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5c0a51cfe314f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f946bfff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f946bfff700 1 -- 172.21.15.26:0/4149248434 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800c3060 con 0x55e8dd974920 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f946bfff700 1 -- 172.21.15.26:0/4149248434 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f9480010930 con 0x55e8dd974920 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9484004bb0 con 0x55e8dd974920 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f946bfff700 1 -- 172.21.15.26:0/4149248434 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94800b20f0 con 0x55e8dd974920 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.450+0000 7f949d3fbb80 10 client.25946 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8dd87c7e0 con 0x55e8dd974920 2024-05-20T20:59:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f946bfff700 1 -- 172.21.15.26:0/4149248434 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9484004bb0 con 0x55e8dd974920 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f946bfff700 1 -- 172.21.15.26:0/4149248434 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f94800e1040 con 0x55e8dd974920 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 10 client.25946 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 20 client.25946 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd98e290 0x55e8dd990680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 4 client.25946 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8dd8d4d40 con 0x55e8dd98e290 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.451+0000 7f948bfff700 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd98e290 0x55e8dd990680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.452+0000 7f948bfff700 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd98e290 0x55e8dd990680 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0597f62263dcc2 server_cookie=441a563a29bde738 in_seq=0 out_seq=0 2024-05-20T20:59:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.452+0000 7f946bfff700 10 client.25946 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:25.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:25 smithi055 ceph-mon[35213]: pgmap v486: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 490 KiB/s wr, 46 op/s 2024-05-20T20:59:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f946bfff700 1 -- 172.21.15.26:0/4149248434 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e8dd8d4d40 con 0x55e8dd98e290 2024-05-20T20:59:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f946bfff700 10 client.25946 handle_command_reply: tid=0 2024-05-20T20:59:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 client.25946 shutdown 2024-05-20T20:59:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 msgr2=0x55e8dd980d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 0x55e8dd980d20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94800e01a0 tx=0x7f94800a9e90 comp rx=0 tx=0).stop 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 shutdown_connections 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd98e290 0x55e8dd990680 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd974920 0x55e8dd980d20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/4149248434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd97e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 >> 172.21.15.26:0/4149248434 conn(0x55e8dd8a02c0 msgr2=0x55e8dd96eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f946bfff700 0 client.25946 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 shutdown_connections 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 1 -- 172.21.15.26:0/4149248434 wait complete. 2024-05-20T20:59:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.453+0000 7f949d3fbb80 20 client.25946 trim_cache size 0 max 16384 2024-05-20T20:59:25.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.480+0000 7f949d3fbb80 1 Processor -- start 2024-05-20T20:59:25.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.480+0000 7f949d3fbb80 1 -- start start 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd974920 0x55e8dd97d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd96f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b27c0 con 0x55e8dd9752e0 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b0050 con 0x55e8dd97a3a0 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1adea0 con 0x55e8dd974920 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd96f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd96f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40402/0 (socket says 172.21.15.26:40402) 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f9491102700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 -- 172.21.15.26:0/3417832270 learned_addr learned my addr 172.21.15.26:0/3417832270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f9490901700 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd974920 0x55e8dd97d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 -- 172.21.15.26:0/3417832270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd974920 msgr2=0x55e8dd97d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd974920 0x55e8dd97d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 -- 172.21.15.26:0/3417832270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd97a3a0 msgr2=0x55e8dd97ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f948bfff700 1 -- 172.21.15.26:0/3417832270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8dd980960 con 0x55e8dd9752e0 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.481+0000 7f9491102700 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.482+0000 7f948bfff700 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd96f2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9478009720 tx=0x7f947801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc41b7a8cbfe0445 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.482+0000 7f94767fc700 1 -- 172.21.15.26:0/3417832270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478003880 con 0x55e8dd9752e0 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.482+0000 7f94767fc700 1 -- 172.21.15.26:0/3417832270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9478033450 con 0x55e8dd9752e0 2024-05-20T20:59:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.482+0000 7f94767fc700 1 -- 172.21.15.26:0/3417832270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478026690 con 0x55e8dd9752e0 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3417832270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 msgr2=0x55e8dd96f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd96f2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9478009720 tx=0x7f947801f000 comp rx=0 tx=0).stop 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3417832270 shutdown_connections 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd97a3a0 0x55e8dd97ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd974920 0x55e8dd97d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/3417832270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd9752e0 0x55e8dd96f2f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3417832270 >> 172.21.15.26:0/3417832270 conn(0x55e8dd8a02c0 msgr2=0x55e8dd97f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3417832270 shutdown_connections 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 -- 172.21.15.26:0/3417832270 wait complete. 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 -- start start 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.483+0000 7f949d3fbb80 1 Processor -- start 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd9869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd986ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd983980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ae510 con 0x55e8dd974920 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1af980 con 0x55e8dd9752e0 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1aebe0 con 0x55e8dd97a3a0 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f949d3fbb80 10 client.0 _ll_register_callbacks cb 0x55e8dd85d9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f9491102700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd983980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f9491102700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd983980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:52628/0 (socket says 172.21.15.26:52628) 2024-05-20T20:59:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.484+0000 7f9491102700 1 -- 172.21.15.26:0/1153390323 learned_addr learned my addr 172.21.15.26:0/1153390323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9490901700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd9869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f948bfff700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd986ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9491102700 1 -- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 msgr2=0x55e8dd986ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9491102700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd986ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9491102700 1 -- 172.21.15.26:0/1153390323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 msgr2=0x55e8dd9869b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9491102700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd9869b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9491102700 1 -- 172.21.15.26:0/1153390323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8dd983fc0 con 0x55e8dd97a3a0 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9490901700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd9869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f9491102700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd983980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94840093a0 tx=0x7f9484004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94504eb0f77db5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.485+0000 7f946bfff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T20:59:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f946bfff700 1 -- 172.21.15.26:0/1153390323 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484015910 con 0x55e8dd97a3a0 2024-05-20T20:59:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f946bfff700 1 -- 172.21.15.26:0/1153390323 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f948401f450 con 0x55e8dd97a3a0 2024-05-20T20:59:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9478033450 con 0x55e8dd97a3a0 2024-05-20T20:59:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f946bfff700 1 -- 172.21.15.26:0/1153390323 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9484032050 con 0x55e8dd97a3a0 2024-05-20T20:59:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f949d3fbb80 10 client.25858 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8dd9f9750 con 0x55e8dd97a3a0 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.486+0000 7f946bfff700 1 -- 172.21.15.26:0/1153390323 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9478033450 con 0x55e8dd97a3a0 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f946bfff700 1 -- 172.21.15.26:0/1153390323 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9484027020 con 0x55e8dd97a3a0 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f949d3fbb80 10 client.25858 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f949d3fbb80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f949d3fbb80 20 client.25858 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd9f9c90 0x55e8dd9fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f949d3fbb80 4 client.25858 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:59:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.487+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e8dd8ce520 con 0x55e8dd9f9c90 2024-05-20T20:59:25.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.488+0000 7f9490901700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd9f9c90 0x55e8dd9fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:25.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.490+0000 7f9490901700 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd9f9c90 0x55e8dd9fc0b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23dd5c98a489c5b server_cookie=7ec91ffc11833ec9 in_seq=0 out_seq=0 2024-05-20T20:59:25.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.490+0000 7f946bfff700 10 client.25858 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.492+0000 7f946bfff700 1 -- 172.21.15.26:0/1153390323 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e8dd8ce520 con 0x55e8dd9f9c90 2024-05-20T20:59:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.492+0000 7f946bfff700 10 client.25858 handle_command_reply: tid=0 2024-05-20T20:59:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.492+0000 7f949d3fbb80 1 client.25858 shutdown 2024-05-20T20:59:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 msgr2=0x55e8dd983980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd983980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94840093a0 tx=0x7f9484004d50 comp rx=0 tx=0).stop 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 shutdown_connections 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e8dd9f9c90 0x55e8dd9fc0b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f946bfff700 0 client.25858 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8dd974920 0x55e8dd9869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8dd97a3a0 0x55e8dd983980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 --2- 172.21.15.26:0/1153390323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e8dd9752e0 0x55e8dd986ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 >> 172.21.15.26:0/1153390323 conn(0x55e8dd8a02c0 msgr2=0x55e8dd9741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 shutdown_connections 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 1 -- 172.21.15.26:0/1153390323 wait complete. 2024-05-20T20:59:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.493+0000 7f949d3fbb80 20 client.25858 trim_cache size 0 max 16384 2024-05-20T20:59:25.495 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:59:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "b494f55a-7711-41e3-a4eb-fad5c893b11f", 2024-05-20T20:59:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:59:25.495 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f947c0b2880 msgr2=0x7f947c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f947c0b2880 0x7f947c0b4d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f948c079ad0 tx=0x7f947801a040 comp rx=0 tx=0).stop 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c06b220 msgr2=0x7f948c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c06b220 0x7f948c102510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f948000e9f0 tx=0x7f948000ed00 comp rx=0 tx=0).stop 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 shutdown_connections 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c1095b0 0x7f948c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f948c06b220 0x7f948c102510 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f947c0b2880 0x7f947c0b4d40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 --2- 172.21.15.26:0/2774343154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06bbe0 0x7f948c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.494+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 >> 172.21.15.26:0/2774343154 conn(0x7f948c0793f0 msgr2=0x7f948c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.495+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 shutdown_connections 2024-05-20T20:59:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:25.495+0000 7f9492b65700 1 -- 172.21.15.26:0/2774343154 wait complete. 2024-05-20T20:59:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:25 smithi144 ceph-mon[35660]: pgmap v486: 97 pgs: 97 active+clean; 26 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 490 KiB/s wr, 46 op/s 2024-05-20T20:59:25.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 51349 3.23 MB/sec warmup 91 sec latency 56.781 ms 2024-05-20T20:59:26.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:59:26.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 52063 3.21 MB/sec warmup 92 sec latency 20.409 ms 2024-05-20T20:59:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:27 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:27 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:27 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:27 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:27 smithi055 ceph-mon[35213]: pgmap v487: 97 pgs: 97 active+clean; 28 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 435 KiB/s wr, 47 op/s 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:27 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:27 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:59:27.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:27 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:27 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:59:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:27 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:27 smithi026 ceph-mon[31990]: pgmap v487: 97 pgs: 97 active+clean; 28 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 435 KiB/s wr, 47 op/s 2024-05-20T20:59:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:27 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:59:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:27 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:59:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:27 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:27 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:59:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:27 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:27 smithi144 ceph-mon[35660]: pgmap v487: 97 pgs: 97 active+clean; 28 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 435 KiB/s wr, 47 op/s 2024-05-20T20:59:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:27 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:59:27.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 52968 3.26 MB/sec warmup 93 sec latency 17.971 ms 2024-05-20T20:59:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 -- 172.21.15.26:0/3167342548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 msgr2=0x7fae84108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3167342548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae84108900 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 -- 172.21.15.26:0/3167342548 shutdown_connections 2024-05-20T20:59:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3167342548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae84108900 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3167342548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 0x7fae8410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3167342548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae84102440 0x7fae841028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 -- 172.21.15.26:0/3167342548 >> 172.21.15.26:0/3167342548 conn(0x7fae840793f0 msgr2=0x7fae84079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 -- 172.21.15.26:0/3167342548 shutdown_connections 2024-05-20T20:59:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.439+0000 7fae8bd65700 1 -- 172.21.15.26:0/3167342548 wait complete. 2024-05-20T20:59:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.440+0000 7fae8bd65700 1 Processor -- start 2024-05-20T20:59:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.440+0000 7fae8bd65700 1 -- start start 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.440+0000 7fae8bd65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae84102440 0x7fae84075010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.440+0000 7fae8bd65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 0x7fae8406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.440+0000 7fae8bd65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae8406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae8bd65700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae841124d0 con 0x7fae84108520 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae8bd65700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112350 con 0x7fae84102e00 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae8bd65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112650 con 0x7fae84102440 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae8a302700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae8406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae8a302700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae8406d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40418/0 (socket says 172.21.15.26:40418) 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae8a302700 1 -- 172.21.15.26:0/3089659709 learned_addr learned my addr 172.21.15.26:0/3089659709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89300700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 0x7fae8406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89b01700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae84102440 0x7fae84075010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89300700 1 -- 172.21.15.26:0/3089659709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae84102440 msgr2=0x7fae84075010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89300700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae84102440 0x7fae84075010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89300700 1 -- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 msgr2=0x7fae8406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89300700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae8406d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.441+0000 7fae89300700 1 -- 172.21.15.26:0/3089659709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8406ddb0 con 0x7fae84102e00 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.442+0000 7fae89300700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 0x7fae8406d030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae7400d950 tx=0x7fae7400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.442+0000 7fae7affd700 1 -- 172.21.15.26:0/3089659709 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74016070 con 0x7fae84102e00 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.442+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae84104010 con 0x7fae84102e00 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.442+0000 7fae7affd700 1 -- 172.21.15.26:0/3089659709 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae7400fd20 con 0x7fae84102e00 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.442+0000 7fae7affd700 1 -- 172.21.15.26:0/3089659709 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7400e3b0 con 0x7fae84102e00 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.442+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841044f0 con 0x7fae84102e00 2024-05-20T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.445+0000 7fae7affd700 1 -- 172.21.15.26:0/3089659709 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fae74019070 con 0x7fae84102e00 2024-05-20T20:59:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.445+0000 7fae7affd700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae700b2900 0x7fae700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.446+0000 7fae7affd700 1 -- 172.21.15.26:0/3089659709 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fae740e2de0 con 0x7fae84102e00 2024-05-20T20:59:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.446+0000 7fae89b01700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae700b2900 0x7fae700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.447+0000 7fae89b01700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae700b2900 0x7fae700b4dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae80003760 tx=0x7fae80005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.447+0000 7fae965fbb80 1 Processor -- start 2024-05-20T20:59:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 -- start start 2024-05-20T20:59:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf55940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f24beb0c90 con 0x55f24bf51860 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f24bea0830 con 0x55f24bf55f60 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae965fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f24bea4870 con 0x55f24bf52210 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae89b01700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae89b01700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf51c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40424/0 (socket says 172.21.15.26:40424) 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae89b01700 1 -- 172.21.15.26:0/2254663413 learned_addr learned my addr 172.21.15.26:0/2254663413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae8a302700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.448+0000 7fae89300700 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf55940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89b01700 1 -- 172.21.15.26:0/2254663413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 msgr2=0x55f24bf55940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89b01700 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf55940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89b01700 1 -- 172.21.15.26:0/2254663413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 msgr2=0x55f24bf563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89b01700 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89b01700 1 -- 172.21.15.26:0/2254663413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f24bdf08d0 con 0x55f24bf51860 2024-05-20T20:59:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae8a302700 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89300700 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf55940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae89b01700 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf51c40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ba60 tx=0x7fae8001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8156cc56cddefba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae6ffff700 1 -- 172.21.15.26:0/2254663413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80034070 con 0x55f24bf51860 2024-05-20T20:59:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae6ffff700 1 -- 172.21.15.26:0/2254663413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae8000f8f0 con 0x55f24bf51860 2024-05-20T20:59:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.449+0000 7fae6ffff700 1 -- 172.21.15.26:0/2254663413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8002f870 con 0x55f24bf51860 2024-05-20T20:59:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 -- 172.21.15.26:0/2254663413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 msgr2=0x55f24bf51c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf51c40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ba60 tx=0x7fae8001f000 comp rx=0 tx=0).stop 2024-05-20T20:59:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 -- 172.21.15.26:0/2254663413 shutdown_connections 2024-05-20T20:59:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf51c40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2254663413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf55940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.450+0000 7fae965fbb80 1 -- 172.21.15.26:0/2254663413 >> 172.21.15.26:0/2254663413 conn(0x55f24be7f080 msgr2=0x55f24bf4be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.451+0000 7fae965fbb80 1 -- 172.21.15.26:0/2254663413 shutdown_connections 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.451+0000 7fae965fbb80 1 -- 172.21.15.26:0/2254663413 wait complete. 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.451+0000 7fae965fbb80 1 -- start start 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.451+0000 7fae965fbb80 1 Processor -- start 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf5acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f24bda6fa0 con 0x55f24bf51860 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f24be427c0 con 0x55f24bf55f60 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f24bf5e0c0 con 0x55f24bf52210 2024-05-20T20:59:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae965fbb80 10 client.0 _ll_register_callbacks cb 0x55f24be3b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae89300700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae89b01700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae89300700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:52674/0 (socket says 172.21.15.26:52674) 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae89300700 1 -- 172.21.15.26:0/2239275169 learned_addr learned my addr 172.21.15.26:0/2239275169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.452+0000 7fae8a302700 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf5acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae8a302700 1 -- 172.21.15.26:0/2239275169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 msgr2=0x55f24bf5a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae8a302700 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae8a302700 1 -- 172.21.15.26:0/2239275169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 msgr2=0x55f24bf5d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae8a302700 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf5d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae8a302700 1 -- 172.21.15.26:0/2239275169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f24be3f690 con 0x55f24bf55f60 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae8a302700 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf5acb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00f970 tx=0x7fae7c00fc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26407b8f0dde8edf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae6d7fa700 1 -- 172.21.15.26:0/2239275169 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c003c10 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae6d7fa700 1 -- 172.21.15.26:0/2239275169 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fae7c010040 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.453+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8000f8f0 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae6d7fa700 1 -- 172.21.15.26:0/2239275169 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae7c003db0 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 10 client.25961 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f24bf688f0 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae6d7fa700 1 -- 172.21.15.26:0/2239275169 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae8000f8f0 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae6d7fa700 1 -- 172.21.15.26:0/2239275169 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fae7c0226b0 con 0x55f24bf55f60 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 10 client.25961 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 20 client.25961 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bf6d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 4 client.25961 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f24bf1d9e0 con 0x55f24bf6af30 2024-05-20T20:59:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.454+0000 7fae89b01700 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bf6d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.455+0000 7fae6d7fa700 10 client.25961 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.455+0000 7fae89b01700 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bf6d320 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ea9bb2717fb94a server_cookie=f91139e04f9c6e03 in_seq=0 out_seq=0 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae6d7fa700 1 -- 172.21.15.26:0/2239275169 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f24bf1d9e0 con 0x55f24bf6af30 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae6d7fa700 10 client.25961 handle_command_reply: tid=0 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 client.25961 shutdown 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 msgr2=0x55f24bf5acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf5acb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00f970 tx=0x7fae7c00fc80 comp rx=0 tx=0).stop 2024-05-20T20:59:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 shutdown_connections 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bf6d320 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae6d7fa700 0 client.25961 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf5d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf5acb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2239275169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 >> 172.21.15.26:0/2239275169 conn(0x55f24be7f080 msgr2=0x55f24bf4b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 shutdown_connections 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.456+0000 7fae965fbb80 1 -- 172.21.15.26:0/2239275169 wait complete. 2024-05-20T20:59:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.457+0000 7fae965fbb80 20 client.25961 trim_cache size 0 max 16384 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 Processor -- start 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 -- start start 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf67160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf4bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b6b10 con 0x55f24bf51860 2024-05-20T20:59:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b43a0 con 0x55f24bf55f60 2024-05-20T20:59:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae965fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b2210 con 0x55f24bf52210 2024-05-20T20:59:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae8a302700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf4bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae8a302700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf4bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44154/0 (socket says 172.21.15.26:44154) 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae8a302700 1 -- 172.21.15.26:0/2767605029 learned_addr learned my addr 172.21.15.26:0/2767605029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.497+0000 7fae89b01700 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf67160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.498+0000 7fae89300700 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.498+0000 7fae8a302700 1 -- 172.21.15.26:0/2767605029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 msgr2=0x55f24bf5c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.498+0000 7fae8a302700 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.498+0000 7fae8a302700 1 -- 172.21.15.26:0/2767605029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 msgr2=0x55f24bf67160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.498+0000 7fae8a302700 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf67160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.498+0000 7fae8a302700 1 -- 172.21.15.26:0/2767605029 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f24bf5a540 con 0x55f24bf55f60 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.499+0000 7fae8a302700 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf4bc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae7c022aa0 tx=0x7fae7c0100a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39594fed47b86147 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.499+0000 7fae6f7fe700 1 -- 172.21.15.26:0/2767605029 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00e0c0 con 0x55f24bf55f60 2024-05-20T20:59:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.499+0000 7fae6f7fe700 1 -- 172.21.15.26:0/2767605029 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae7c022420 con 0x55f24bf55f60 2024-05-20T20:59:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.499+0000 7fae6f7fe700 1 -- 172.21.15.26:0/2767605029 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0055a0 con 0x55f24bf55f60 2024-05-20T20:59:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 -- 172.21.15.26:0/2767605029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 msgr2=0x55f24bf4bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf4bc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae7c022aa0 tx=0x7fae7c0100a0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 -- 172.21.15.26:0/2767605029 shutdown_connections 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf51860 0x55f24bf67160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf55f60 0x55f24bf4bc90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 --2- 172.21.15.26:0/2767605029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf52210 0x55f24bf5c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 -- 172.21.15.26:0/2767605029 >> 172.21.15.26:0/2767605029 conn(0x55f24be7f080 msgr2=0x55f24bf5bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 -- 172.21.15.26:0/2767605029 shutdown_connections 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 -- 172.21.15.26:0/2767605029 wait complete. 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.500+0000 7fae965fbb80 1 -- start start 2024-05-20T20:59:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 Processor -- start 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 0x55f24bf63810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf52210 0x55f24bf60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf55f60 0x55f24bf610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b2860 con 0x55f24bf52210 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b3cd0 con 0x55f24bf51860 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae965fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b2f30 con 0x55f24bf55f60 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae89b01700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 0x55f24bf63810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae89b01700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 0x55f24bf63810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44156/0 (socket says 172.21.15.26:44156) 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.501+0000 7fae89b01700 1 -- 172.21.15.26:0/3343305455 learned_addr learned my addr 172.21.15.26:0/3343305455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89300700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf52210 0x55f24bf60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae965fbb80 10 client.0 _ll_register_callbacks cb 0x55f24be3b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae8a302700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf55f60 0x55f24bf610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89b01700 1 -- 172.21.15.26:0/3343305455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf55f60 msgr2=0x55f24bf610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89b01700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf55f60 0x55f24bf610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89b01700 1 -- 172.21.15.26:0/3343305455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf52210 msgr2=0x55f24bf60bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89b01700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf52210 0x55f24bf60bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89b01700 1 -- 172.21.15.26:0/3343305455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f24bf61730 con 0x55f24bf51860 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae8a302700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf55f60 0x55f24bf610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae89b01700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 0x55f24bf63810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae80005f50 tx=0x7fae800058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee96f26835b110ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae6d7fa700 1 -- 172.21.15.26:0/3343305455 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80034040 con 0x55f24bf51860 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae6d7fa700 1 -- 172.21.15.26:0/3343305455 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fae8000f980 con 0x55f24bf51860 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.502+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae7c022420 con 0x55f24bf51860 2024-05-20T20:59:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae6d7fa700 1 -- 172.21.15.26:0/3343305455 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae8000fb80 con 0x55f24bf51860 2024-05-20T20:59:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 10 client.25967 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f24bfd58d0 con 0x55f24bf51860 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae6d7fa700 1 -- 172.21.15.26:0/3343305455 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae7c022420 con 0x55f24bf51860 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae6d7fa700 1 -- 172.21.15.26:0/3343305455 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fae8004b020 con 0x55f24bf51860 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 10 client.25967 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 20 client.25967 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bfd7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 4 client.25967 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.503+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f24beb2be0 con 0x55f24bf6af30 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.504+0000 7fae89300700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bfd7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.504+0000 7fae89300700 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bfd7b10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea7ba84d0f4929 server_cookie=121cee598a55069a in_seq=0 out_seq=0 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.504+0000 7fae6d7fa700 10 client.25967 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.504+0000 7fae6d7fa700 1 -- 172.21.15.26:0/3343305455 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f24beb2be0 con 0x55f24bf6af30 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.504+0000 7fae6d7fa700 10 client.25967 handle_command_reply: tid=0 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.504+0000 7fae965fbb80 1 client.25967 shutdown 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 msgr2=0x55f24bf63810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 0x55f24bf63810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae80005f50 tx=0x7fae800058e0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 shutdown_connections 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f24bf6af30 0x55f24bfd7b10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae6d7fa700 0 client.25967 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24bf55f60 0x55f24bf610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f24bf51860 0x55f24bf63810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 --2- 172.21.15.26:0/3343305455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f24bf52210 0x55f24bf60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 >> 172.21.15.26:0/3343305455 conn(0x55f24be7f080 msgr2=0x55f24bf50f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 shutdown_connections 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 1 -- 172.21.15.26:0/3343305455 wait complete. 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.505+0000 7fae965fbb80 20 client.25967 trim_cache size 0 max 16384 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T20:59:28.508 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae700b2900 msgr2=0x7fae700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae700b2900 0x7fae700b4dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae80003760 tx=0x7fae80005fd0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 msgr2=0x7fae8406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 0x7fae8406d030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae7400d950 tx=0x7fae7400dc60 comp rx=0 tx=0).stop 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 shutdown_connections 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae700b2900 0x7fae700b4dc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84108520 0x7fae8406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae84102440 0x7fae84075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 --2- 172.21.15.26:0/3089659709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae84102e00 0x7fae8406d030 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 >> 172.21.15.26:0/3089659709 conn(0x7fae840793f0 msgr2=0x7fae8406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 shutdown_connections 2024-05-20T20:59:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:28.506+0000 7fae8bd65700 1 -- 172.21.15.26:0/3089659709 wait complete. 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 -- 172.21.15.26:0/443807943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988105f00 msgr2=0x7fe9881082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 --2- 172.21.15.26:0/443807943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988105f00 0x7fe9881082f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe978009900 tx=0x7fe97802a410 comp rx=0 tx=0).stop 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 -- 172.21.15.26:0/443807943 shutdown_connections 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 --2- 172.21.15.26:0/443807943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe988108830 0x7fe98810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 --2- 172.21.15.26:0/443807943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988105f00 0x7fe9881082f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 --2- 172.21.15.26:0/443807943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9881035d0 0x7fe9881059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 -- 172.21.15.26:0/443807943 >> 172.21.15.26:0/443807943 conn(0x7fe9880fcf40 msgr2=0x7fe9880ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 -- 172.21.15.26:0/443807943 shutdown_connections 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.507+0000 7fe98efe4640 1 -- 172.21.15.26:0/443807943 wait complete. 2024-05-20T20:59:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.508+0000 7fe98efe4640 1 Processor -- start 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.508+0000 7fe98efe4640 1 -- start start 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98efe4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9881035d0 0x7fe988102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98efe4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe988105f00 0x7fe988101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98efe4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988108830 0x7fe988101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98efe4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe98810e4d0 con 0x7fe9881035d0 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98efe4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe98810e350 con 0x7fe988108830 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98efe4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe98810e650 con 0x7fe988105f00 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9881035d0 0x7fe988102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9881035d0 0x7fe988102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40474/0 (socket says 172.21.15.26:40474) 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98cd80640 1 -- 172.21.15.26:0/2019409345 learned_addr learned my addr 172.21.15.26:0/2019409345 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98d581640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988108830 0x7fe988101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe987fff640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe988105f00 0x7fe988101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98d581640 1 -- 172.21.15.26:0/2019409345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe988105f00 msgr2=0x7fe988101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98d581640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe988105f00 0x7fe988101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98d581640 1 -- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9881035d0 msgr2=0x7fe988102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98d581640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9881035d0 0x7fe988102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.509+0000 7fe98d581640 1 -- 172.21.15.26:0/2019409345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe988102350 con 0x7fe988108830 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.510+0000 7fe98d581640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988108830 0x7fe988101ac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe980009fd0 tx=0x7fe98000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.510+0000 7fe985ffb640 1 -- 172.21.15.26:0/2019409345 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000ce20 con 0x7fe988108830 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.510+0000 7fe985ffb640 1 -- 172.21.15.26:0/2019409345 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe980005020 con 0x7fe988108830 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.510+0000 7fe985ffb640 1 -- 172.21.15.26:0/2019409345 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980010690 con 0x7fe988108830 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.510+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9881a2c90 con 0x7fe988108830 2024-05-20T20:59:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.510+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881a3190 con 0x7fe988108830 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.513+0000 7fe991dda740 1 Processor -- start 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe985ffb640 1 -- 172.21.15.26:0/2019409345 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe980004a40 con 0x7fe988108830 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 -- start start 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a30bed0 0x55968a30c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a3113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 0x55968a313ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55968a274c30 con 0x55968a30c8c0 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55968a2647d0 con 0x55968a311a30 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe991dda740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55968a268810 con 0x55968a30bed0 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe987fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 0x55968a313ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe987fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 0x55968a313ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44184/0 (socket says 172.21.15.26:44184) 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.514+0000 7fe987fff640 1 -- 172.21.15.26:0/367456376 learned_addr learned my addr 172.21.15.26:0/367456376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a30bed0 0x55968a30c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe98cd80640 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a3113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 -- 172.21.15.26:0/367456376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a30bed0 msgr2=0x55968a30c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a30bed0 0x55968a30c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 -- 172.21.15.26:0/367456376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 msgr2=0x55968a3113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a3113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 -- 172.21.15.26:0/367456376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559689f79210 con 0x55968a311a30 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe98cd80640 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a3113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe987fff640 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 0x55968a313ec0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe978005ec0 tx=0x7fe978005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba779d871c6d65a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe985ffb640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9740b2920 0x7fe9740b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe97effd640 1 -- 172.21.15.26:0/367456376 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978035020 con 0x55968a311a30 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe97effd640 1 -- 172.21.15.26:0/367456376 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9780044d0 con 0x55968a311a30 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.515+0000 7fe97effd640 1 -- 172.21.15.26:0/367456376 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978004d60 con 0x55968a311a30 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe985ffb640 1 -- 172.21.15.26:0/2019409345 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe9800e2960 con 0x7fe988108830 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe98cd80640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9740b2920 0x7fe9740b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe98cd80640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9740b2920 0x7fe9740b4de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55968a30ccc0 tx=0x7fe97000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe991dda740 1 -- 172.21.15.26:0/367456376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 msgr2=0x55968a313ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe991dda740 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 0x55968a313ec0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe978005ec0 tx=0x7fe978005720 comp rx=0 tx=0).stop 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe991dda740 1 -- 172.21.15.26:0/367456376 shutdown_connections 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe991dda740 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a311a30 0x55968a313ec0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe991dda740 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a3113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.516+0000 7fe991dda740 1 --2- 172.21.15.26:0/367456376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a30bed0 0x55968a30c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.517+0000 7fe991dda740 1 -- 172.21.15.26:0/367456376 >> 172.21.15.26:0/367456376 conn(0x55968a307540 msgr2=0x55968a307940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.517+0000 7fe991dda740 1 -- 172.21.15.26:0/367456376 shutdown_connections 2024-05-20T20:59:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.517+0000 7fe991dda740 1 -- 172.21.15.26:0/367456376 wait complete. 2024-05-20T20:59:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.517+0000 7fe991dda740 1 -- start start 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.517+0000 7fe991dda740 1 Processor -- start 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 0x55968a315fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30c8c0 0x55968a316510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559689f4a790 con 0x55968a30bed0 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 0x55968a315fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55968a220890 con 0x55968a30c8c0 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 0x55968a315fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40496/0 (socket says 172.21.15.26:40496) 2024-05-20T20:59:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe98cd80640 1 -- 172.21.15.26:0/1769625426 learned_addr learned my addr 172.21.15.26:0/1769625426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55968a314980 con 0x55968a311a30 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe991dda740 10 client.0 _ll_register_callbacks cb 0x559689f78a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.518+0000 7fe98d581640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe98cd80640 1 -- 172.21.15.26:0/1769625426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 msgr2=0x55968a31d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe98cd80640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe98cd80640 1 -- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30c8c0 msgr2=0x55968a316510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe987fff640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30c8c0 0x55968a316510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe98cd80640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30c8c0 0x55968a316510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe98cd80640 1 -- 172.21.15.26:0/1769625426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968a045620 con 0x55968a30bed0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe98cd80640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 0x55968a315fd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe9700107c0 tx=0x7fe9700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4366b0b30853dfe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe967fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe967fff640 1 -- 172.21.15.26:0/1769625426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970023070 con 0x55968a30bed0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe967fff640 1 -- 172.21.15.26:0/1769625426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe97000fac0 con 0x55968a30bed0 2024-05-20T20:59:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9780044d0 con 0x55968a30bed0 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe967fff640 1 -- 172.21.15.26:0/1769625426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe97002f050 con 0x55968a30bed0 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe991dda740 10 client.16440 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.519+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55968a3205e0 con 0x55968a30bed0 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe967fff640 1 -- 172.21.15.26:0/1769625426 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9780044d0 con 0x55968a30bed0 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe967fff640 1 -- 172.21.15.26:0/1769625426 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe9700273b0 con 0x55968a30bed0 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe991dda740 10 client.16440 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe991dda740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe991dda740 20 client.16440 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe991dda740 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a324190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe991dda740 4 client.16440 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55968a276b80 con 0x55968a321d40 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.520+0000 7fe987fff640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a324190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.521+0000 7fe967fff640 10 client.16440 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.521+0000 7fe987fff640 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a324190 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ba4c5fd2ef8aaa server_cookie=31f3c75daf7d3010 in_seq=0 out_seq=0 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe967fff640 1 -- 172.21.15.26:0/1769625426 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55968a276b80 con 0x55968a321d40 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe967fff640 10 client.16440 handle_command_reply: tid=0 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 client.16440 shutdown 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 msgr2=0x55968a315fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 0x55968a315fd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe9700107c0 tx=0x7fe9700046c0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 shutdown_connections 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a324190 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe967fff640 0 client.16440 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30c8c0 0x55968a316510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 --2- 172.21.15.26:0/1769625426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30bed0 0x55968a315fd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 >> 172.21.15.26:0/1769625426 conn(0x55968a307540 msgr2=0x55968a2fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 shutdown_connections 2024-05-20T20:59:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.522+0000 7fe991dda740 1 -- 172.21.15.26:0/1769625426 wait complete. 2024-05-20T20:59:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.523+0000 7fe991dda740 20 client.16440 trim_cache size 0 max 16384 2024-05-20T20:59:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.545+0000 7fe991dda740 1 Processor -- start 2024-05-20T20:59:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.545+0000 7fe991dda740 1 -- start start 2024-05-20T20:59:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe991dda740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a3162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe991dda740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a3162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a3162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44208/0 (socket says 172.21.15.26:44208) 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe98cd80640 1 -- 172.21.15.26:0/4102632140 learned_addr learned my addr 172.21.15.26:0/4102632140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe991dda740 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a2fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe987fff640 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881aa610 con 0x55968a30c8c0 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a7ee0 con 0x55968a30bed0 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.546+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a5d30 con 0x55968a311a30 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98d581640 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a2fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98cd80640 1 -- 172.21.15.26:0/4102632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 msgr2=0x55968a2fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98cd80640 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a2fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98cd80640 1 -- 172.21.15.26:0/4102632140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 msgr2=0x55968a31d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98cd80640 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98cd80640 1 -- 172.21.15.26:0/4102632140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968a30ba30 con 0x55968a30bed0 2024-05-20T20:59:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe987fff640 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe98cd80640 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a3162f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9700277a0 tx=0x7fe970012d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189498baa51c31ec server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe97e7fc640 1 -- 172.21.15.26:0/4102632140 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970023040 con 0x55968a30bed0 2024-05-20T20:59:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe97e7fc640 1 -- 172.21.15.26:0/4102632140 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe970027a10 con 0x55968a30bed0 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.547+0000 7fe97e7fc640 1 -- 172.21.15.26:0/4102632140 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9700286a0 con 0x55968a30bed0 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.548+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 msgr2=0x55968a3162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.548+0000 7fe991dda740 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a3162f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9700277a0 tx=0x7fe970012d60 comp rx=0 tx=0).stop 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 shutdown_connections 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a2fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 --2- 172.21.15.26:0/4102632140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a3162f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 >> 172.21.15.26:0/4102632140 conn(0x55968a307540 msgr2=0x55968a315be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 shutdown_connections 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.549+0000 7fe991dda740 1 -- 172.21.15.26:0/4102632140 wait complete. 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.550+0000 7fe991dda740 1 -- start start 2024-05-20T20:59:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.550+0000 7fe991dda740 1 Processor -- start 2024-05-20T20:59:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.550+0000 7fe991dda740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a31ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.550+0000 7fe991dda740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.550+0000 7fe991dda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a31ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe991dda740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a63a0 con 0x55968a30c8c0 2024-05-20T20:59:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe991dda740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a7810 con 0x55968a30bed0 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe991dda740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a6a70 con 0x55968a311a30 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a31ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44220/0 (socket says 172.21.15.26:44220) 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 -- 172.21.15.26:0/3705660641 learned_addr learned my addr 172.21.15.26:0/3705660641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe987fff640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe991dda740 10 client.0 _ll_register_callbacks cb 0x7fe970010440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 -- 172.21.15.26:0/3705660641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 msgr2=0x55968a31c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 -- 172.21.15.26:0/3705660641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 msgr2=0x55968a31b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 -- 172.21.15.26:0/3705660641 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968a31c600 con 0x55968a30bed0 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe987fff640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe98cd80640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a31ab10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9700278c0 tx=0x7fe970005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3773eb54590f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe967fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe967fff640 1 -- 172.21.15.26:0/3705660641 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970040020 con 0x55968a30bed0 2024-05-20T20:59:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe970027a10 con 0x55968a30bed0 2024-05-20T20:59:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe967fff640 1 -- 172.21.15.26:0/3705660641 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fe97000f990 con 0x55968a30bed0 2024-05-20T20:59:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.551+0000 7fe967fff640 1 -- 172.21.15.26:0/3705660641 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe97004b050 con 0x55968a30bed0 2024-05-20T20:59:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 10 client.25982 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55968a31d8f0 con 0x55968a30bed0 2024-05-20T20:59:28.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 53446 3.27 MB/sec warmup 94 sec latency 16.293 ms 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe967fff640 1 -- 172.21.15.26:0/3705660641 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe970027a10 con 0x55968a30bed0 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe967fff640 1 -- 172.21.15.26:0/3705660641 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe970028440 con 0x55968a30bed0 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 10 client.25982 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 20 client.25982 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a38f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 4 client.25982 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:59:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55968a30d650 con 0x55968a321d40 2024-05-20T20:59:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.552+0000 7fe987fff640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a38f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.553+0000 7fe987fff640 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a38f0f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ebee778f3bc8e9 server_cookie=7be380993a938032 in_seq=0 out_seq=0 2024-05-20T20:59:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.553+0000 7fe967fff640 10 client.25982 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.587+0000 7fe967fff640 1 -- 172.21.15.26:0/3705660641 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55968a30d650 con 0x55968a321d40 2024-05-20T20:59:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.587+0000 7fe967fff640 10 client.25982 handle_command_reply: tid=0 2024-05-20T20:59:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 client.25982 shutdown 2024-05-20T20:59:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 msgr2=0x55968a31ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a31ab10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9700278c0 tx=0x7fe970005ee0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 shutdown_connections 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968a321d40 0x55968a38f0f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968a311a30 0x55968a31c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968a30c8c0 0x55968a31b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe967fff640 0 client.25982 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 --2- 172.21.15.26:0/3705660641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968a30bed0 0x55968a31ab10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 >> 172.21.15.26:0/3705660641 conn(0x55968a307540 msgr2=0x55968a2fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 shutdown_connections 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 1 -- 172.21.15.26:0/3705660641 wait complete. 2024-05-20T20:59:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.588+0000 7fe991dda740 20 client.25982 trim_cache size 0 max 16384 2024-05-20T20:59:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:59:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:59:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9740b2920 msgr2=0x7fe9740b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9740b2920 0x7fe9740b4de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55968a30ccc0 tx=0x7fe97000b410 comp rx=0 tx=0).stop 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988108830 msgr2=0x7fe988101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988108830 0x7fe988101ac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe980009fd0 tx=0x7fe98000ef10 comp rx=0 tx=0).stop 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 shutdown_connections 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe9740b2920 0x7fe9740b4de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe988108830 0x7fe988101ac0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe988105f00 0x7fe988101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 --2- 172.21.15.26:0/2019409345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9881035d0 0x7fe988102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 >> 172.21.15.26:0/2019409345 conn(0x7fe9880fcf40 msgr2=0x7fe988103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 shutdown_connections 2024-05-20T20:59:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:28.589+0000 7fe98efe4640 1 -- 172.21.15.26:0/2019409345 wait complete. 2024-05-20T20:59:29.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:29 smithi055 ceph-mon[35213]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 413 KiB/s wr, 45 op/s 2024-05-20T20:59:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:29 smithi026 ceph-mon[31990]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 413 KiB/s wr, 45 op/s 2024-05-20T20:59:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:29 smithi144 ceph-mon[35660]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 413 KiB/s wr, 45 op/s 2024-05-20T20:59:29.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 53839 3.24 MB/sec warmup 95 sec latency 15.785 ms 2024-05-20T20:59:29.686 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T20:59:29.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:59:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:30.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 54467 3.23 MB/sec warmup 96 sec latency 18.386 ms 2024-05-20T20:59:31.445 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:31 smithi026 ceph-mon[31990]: pgmap v489: 97 pgs: 97 active+clean; 23 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 563 KiB/s wr, 51 op/s 2024-05-20T20:59:31.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:31 smithi055 ceph-mon[35213]: pgmap v489: 97 pgs: 97 active+clean; 23 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 563 KiB/s wr, 51 op/s 2024-05-20T20:59:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:31 smithi144 ceph-mon[35660]: pgmap v489: 97 pgs: 97 active+clean; 23 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 563 KiB/s wr, 51 op/s 2024-05-20T20:59:31.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 55215 3.23 MB/sec warmup 97 sec latency 17.385 ms 2024-05-20T20:59:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 -- 172.21.15.26:0/3895492956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406bc90 msgr2=0x7fd8b406c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 --2- 172.21.15.26:0/3895492956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406bc90 0x7fd8b406c110 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009990 tx=0x7fd8a4009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 -- 172.21.15.26:0/3895492956 shutdown_connections 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 --2- 172.21.15.26:0/3895492956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406c650 0x7fd8b41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 --2- 172.21.15.26:0/3895492956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41095b0 0x7fd8b4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 --2- 172.21.15.26:0/3895492956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406bc90 0x7fd8b406c110 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 -- 172.21.15.26:0/3895492956 >> 172.21.15.26:0/3895492956 conn(0x7fd8b4079410 msgr2=0x7fd8b4079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 -- 172.21.15.26:0/3895492956 shutdown_connections 2024-05-20T20:59:31.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.854+0000 7fd8bb714700 1 -- 172.21.15.26:0/3895492956 wait complete. 2024-05-20T20:59:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 Processor -- start 2024-05-20T20:59:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 -- start start 2024-05-20T20:59:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 0x7fd8b4102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406c650 0x7fd8b4102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41095b0 0x7fd8b4102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4113570 con 0x7fd8b406c650 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.855+0000 7fd8bb714700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41133f0 con 0x7fd8b406bc90 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b94b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 0x7fd8b4102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8bb714700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41136f0 con 0x7fd8b41095b0 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b94b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 0x7fd8b4102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:52864/0 (socket says 172.21.15.26:52864) 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406c650 0x7fd8b4102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b94b0700 1 -- 172.21.15.26:0/79837515 learned_addr learned my addr 172.21.15.26:0/79837515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b9cb1700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41095b0 0x7fd8b4102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 -- 172.21.15.26:0/79837515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41095b0 msgr2=0x7fd8b4102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41095b0 0x7fd8b4102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 -- 172.21.15.26:0/79837515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 msgr2=0x7fd8b4102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 0x7fd8b4102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 -- 172.21.15.26:0/79837515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41a8050 con 0x7fd8b406c650 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b94b0700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 0x7fd8b4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8b8caf700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406c650 0x7fd8b4102a80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009990 tx=0x7fd8a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400fb10 con 0x7fd8b406c650 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8a4004cc0 con 0x7fd8b406c650 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40054b0 con 0x7fd8b406c650 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.856+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41a82e0 con 0x7fd8b406c650 2024-05-20T20:59:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.857+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a8680 con 0x7fd8b406c650 2024-05-20T20:59:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.858+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b406a230 con 0x7fd8b406c650 2024-05-20T20:59:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.859+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd8a4009d10 con 0x7fd8b406c650 2024-05-20T20:59:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.860+0000 7fd8aa7fc700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a00b2880 0x7fd8a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.860+0000 7fd8b94b0700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a00b2880 0x7fd8a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.860+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fd8a40e2f30 con 0x7fd8b406c650 2024-05-20T20:59:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.861+0000 7fd8b94b0700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a00b2880 0x7fd8a00b4d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005950 tx=0x7fd8b000b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:31.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:31.866+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8a4013070 con 0x7fd8b406c650 2024-05-20T20:59:32.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.170+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8b402e560 con 0x7fd8b406c650 2024-05-20T20:59:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.171+0000 7fd8aa7fc700 1 -- 172.21.15.26:0/79837515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fd8a40aac80 con 0x7fd8b406c650 2024-05-20T20:59:32.170 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:59:32.170 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:55:45.467827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/539511953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":539511953},{"type":"v1","addr":"172.21.15.144:6837","nonce":539511953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:59:32.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a00b2880 msgr2=0x7fd8a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:32.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a00b2880 0x7fd8a00b4d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005950 tx=0x7fd8b000b750 comp rx=0 tx=0).stop 2024-05-20T20:59:32.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406c650 msgr2=0x7fd8b4102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406c650 0x7fd8b4102a80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009990 tx=0x7fd8a40047b0 comp rx=0 tx=0).stop 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 shutdown_connections 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41095b0 0x7fd8b4102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8b406bc90 0x7fd8b4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd8a00b2880 0x7fd8a00b4d40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 --2- 172.21.15.26:0/79837515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b406c650 0x7fd8b4102a80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 >> 172.21.15.26:0/79837515 conn(0x7fd8b4079410 msgr2=0x7fd8b4101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 shutdown_connections 2024-05-20T20:59:32.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:32.173+0000 7fd8bb714700 1 -- 172.21.15.26:0/79837515 wait complete. 2024-05-20T20:59:32.174 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-05-20T20:59:32.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 55797 3.22 MB/sec warmup 98 sec latency 47.351 ms 2024-05-20T20:59:32.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T20:59:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:33 smithi026 ceph-mon[31990]: pgmap v490: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-05-20T20:59:33.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:33 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/79837515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:33 smithi055 ceph-mon[35213]: pgmap v490: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-05-20T20:59:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:33 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/79837515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:33 smithi144 ceph-mon[35660]: pgmap v490: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-05-20T20:59:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:33 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/79837515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:33.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 56557 3.25 MB/sec warmup 99 sec latency 18.142 ms 2024-05-20T20:59:34.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 57102 3.27 MB/sec warmup 100 sec latency 24.691 ms 2024-05-20T20:59:35.355 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:35 smithi026 ceph-mon[31990]: pgmap v491: 97 pgs: 97 active+clean; 24 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-20T20:59:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:35 smithi055 ceph-mon[35213]: pgmap v491: 97 pgs: 97 active+clean; 24 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-20T20:59:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:35 smithi144 ceph-mon[35660]: pgmap v491: 97 pgs: 97 active+clean; 24 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-20T20:59:35.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 57504 3.24 MB/sec warmup 101 sec latency 68.546 ms 2024-05-20T20:59:36.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 58060 3.23 MB/sec warmup 102 sec latency 79.763 ms 2024-05-20T20:59:37.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:37 smithi026 ceph-mon[31990]: pgmap v492: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-05-20T20:59:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:37 smithi055 ceph-mon[35213]: pgmap v492: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-05-20T20:59:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:37 smithi144 ceph-mon[35660]: pgmap v492: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-05-20T20:59:37.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 58710 3.23 MB/sec warmup 103 sec latency 66.531 ms 2024-05-20T20:59:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3769990224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 msgr2=0x7fb2c4110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3769990224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c4110430 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00b0a0 tx=0x7fb2bc00b3b0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3769990224 shutdown_connections 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3769990224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c4110430 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3769990224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41096a0 0x7fb2c4109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3769990224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2c4103590 0x7fb2c4103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3769990224 >> 172.21.15.26:0/3769990224 conn(0x7fb2c407a6f0 msgr2=0x7fb2c407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3769990224 shutdown_connections 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.903+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3769990224 wait complete. 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 Processor -- start 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 -- start start 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c4103590 0x7fb2c419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2c41096a0 0x7fb2c4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4113620 con 0x7fb2c4103f50 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41134a0 con 0x7fb2c41096a0 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2cb80a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41137a0 con 0x7fb2c4103590 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59130/0 (socket says 172.21.15.26:59130) 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 -- 172.21.15.26:0/3362931360 learned_addr learned my addr 172.21.15.26:0/3362931360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c4103590 0x7fb2c419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2c41096a0 0x7fb2c4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 -- 172.21.15.26:0/3362931360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c4103590 msgr2=0x7fb2c419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c4103590 0x7fb2c419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 -- 172.21.15.26:0/3362931360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2c41096a0 msgr2=0x7fb2c4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2c41096a0 0x7fb2c4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.904+0000 7fb2c8da5700 1 -- 172.21.15.26:0/3362931360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c419a020 con 0x7fb2c4103f50 2024-05-20T20:59:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.905+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c419fc00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400eb30 tx=0x7fb2b400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.905+0000 7fb2ba7fc700 1 -- 172.21.15.26:0/3362931360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400cbd0 con 0x7fb2c4103f50 2024-05-20T20:59:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.905+0000 7fb2ba7fc700 1 -- 172.21.15.26:0/3362931360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2b40044b0 con 0x7fb2c4103f50 2024-05-20T20:59:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.905+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c419a2b0 con 0x7fb2c4103f50 2024-05-20T20:59:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.905+0000 7fb2ba7fc700 1 -- 172.21.15.26:0/3362931360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4003dc0 con 0x7fb2c4103f50 2024-05-20T20:59:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.905+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a6520 con 0x7fb2c4103f50 2024-05-20T20:59:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.908+0000 7fb2ba7fc700 1 -- 172.21.15.26:0/3362931360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb2b4003f60 con 0x7fb2c4103f50 2024-05-20T20:59:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.909+0000 7fb2ba7fc700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2b00b2880 0x7fb2b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.909+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2b00b2880 0x7fb2b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.909+0000 7fb2ba7fc700 1 -- 172.21.15.26:0/3362931360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb2b40e25b0 con 0x7fb2c4103f50 2024-05-20T20:59:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.909+0000 7fb2d60a0b80 1 Processor -- start 2024-05-20T20:59:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 -- start start 2024-05-20T20:59:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2b00b2880 0x7fb2b00b4d40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009680 tx=0x7fb2c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb73320 0x5625cdb73700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb6dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6e260 0x5625cdb7a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5625cdacbd70 con 0x5625cdb6d8a0 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5625cdabb910 con 0x5625cdb73320 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625cdabf950 con 0x5625cdb6e260 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c95a6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb73320 0x5625cdb73700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb6dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c95a6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb73320 0x5625cdb73700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:52878/0 (socket says 172.21.15.26:52878) 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c95a6700 1 -- 172.21.15.26:0/4106047639 learned_addr learned my addr 172.21.15.26:0/4106047639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6e260 0x5625cdb7a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c9da7700 1 -- 172.21.15.26:0/4106047639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb73320 msgr2=0x5625cdb73700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb73320 0x5625cdb73700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c9da7700 1 -- 172.21.15.26:0/4106047639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 msgr2=0x5625cdb6dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb6dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.910+0000 7fb2c9da7700 1 -- 172.21.15.26:0/4106047639 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625cda64fb0 con 0x5625cdb6e260 2024-05-20T20:59:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.911+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6e260 0x5625cdb7a080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00b070 tx=0x7fb2bc0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa6202338778d05a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.911+0000 7fb2af7fe700 1 -- 172.21.15.26:0/4106047639 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc020070 con 0x5625cdb6e260 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.911+0000 7fb2af7fe700 1 -- 172.21.15.26:0/4106047639 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2bc006020 con 0x5625cdb6e260 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.911+0000 7fb2af7fe700 1 -- 172.21.15.26:0/4106047639 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc007af0 con 0x5625cdb6e260 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/4106047639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6e260 msgr2=0x5625cdb7a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6e260 0x5625cdb7a080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00b070 tx=0x7fb2bc0092e0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/4106047639 shutdown_connections 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6e260 0x5625cdb7a080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb73320 0x5625cdb73700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/4106047639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb6dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/4106047639 >> 172.21.15.26:0/4106047639 conn(0x5625cda99240 msgr2=0x5625cdb683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/4106047639 shutdown_connections 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/4106047639 wait complete. 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 -- start start 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.912+0000 7fb2d60a0b80 1 Processor -- start 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb79cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb76ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb77530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5625cd9c1fa0 con 0x5625cdb6d8a0 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5625cda23da0 con 0x5625cdb6e260 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625cdb7a680 con 0x5625cdb73320 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2d60a0b80 10 client.0 _ll_register_callbacks cb 0x5625cda56950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c95a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb79cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c95a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb79cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59144/0 (socket says 172.21.15.26:59144) 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c95a6700 1 -- 172.21.15.26:0/390242772 learned_addr learned my addr 172.21.15.26:0/390242772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb76ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb77530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c8da5700 1 -- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 msgr2=0x5625cdb77530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb77530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c8da5700 1 -- 172.21.15.26:0/390242772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 msgr2=0x5625cdb79cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb79cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c8da5700 1 -- 172.21.15.26:0/390242772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625cda0b8d0 con 0x5625cdb6e260 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.913+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb79cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb76ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40e00a0 tx=0x7fb2b40a94c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832b45090ed7e1b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2acff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2acff9700 1 -- 172.21.15.26:0/390242772 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40a5300 con 0x5625cdb6e260 2024-05-20T20:59:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2acff9700 1 -- 172.21.15.26:0/390242772 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb2b40a6160 con 0x5625cdb6e260 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2bc006020 con 0x5625cdb6e260 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2acff9700 1 -- 172.21.15.26:0/390242772 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2b400ce90 con 0x5625cdb6e260 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2d60a0b80 10 client.26000 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625cda75760 con 0x5625cdb6e260 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2acff9700 1 -- 172.21.15.26:0/390242772 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2bc006020 con 0x5625cdb6e260 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.914+0000 7fb2acff9700 1 -- 172.21.15.26:0/390242772 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb2b40b2540 con 0x5625cdb6e260 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2d60a0b80 10 client.26000 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2d60a0b80 10 client.26000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2d60a0b80 20 client.26000 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdb87140 0x5625cdb89530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2d60a0b80 4 client.26000 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625cdacdcc0 con 0x5625cdb87140 2024-05-20T20:59:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdb87140 0x5625cdb89530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2acff9700 10 client.26000 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.915+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdb87140 0x5625cdb89530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c493d9587b1a928 server_cookie=3da71dca568dfe93 in_seq=0 out_seq=0 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2acff9700 1 -- 172.21.15.26:0/390242772 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5625cdacdcc0 con 0x5625cdb87140 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2acff9700 10 client.26000 handle_command_reply: tid=0 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 client.26000 shutdown 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 msgr2=0x5625cdb76ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb76ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40e00a0 tx=0x7fb2b40a94c0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 shutdown_connections 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdb87140 0x5625cdb89530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb79cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb77530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/390242772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb76ff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 >> 172.21.15.26:0/390242772 conn(0x5625cda99240 msgr2=0x5625cdb67af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2acff9700 0 client.26000 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.916+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 shutdown_connections 2024-05-20T20:59:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.917+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/390242772 wait complete. 2024-05-20T20:59:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.917+0000 7fb2d60a0b80 20 client.26000 trim_cache size 0 max 16384 2024-05-20T20:59:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 Processor -- start 2024-05-20T20:59:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 -- start start 2024-05-20T20:59:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb8ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41b0400 con 0x5625cdb6d8a0 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41adc90 con 0x5625cdb6e260 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41abae0 con 0x5625cdb73320 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2c95a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb68270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:52892/0 (socket says 172.21.15.26:52892) 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2c8da5700 1 -- 172.21.15.26:0/391068882 learned_addr learned my addr 172.21.15.26:0/391068882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.937+0000 7fb2c95a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb72830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59150/0 (socket says 172.21.15.26:59150) 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb8ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c8da5700 1 -- 172.21.15.26:0/391068882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 msgr2=0x5625cdb8ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb8ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c8da5700 1 -- 172.21.15.26:0/391068882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 msgr2=0x5625cdb72830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb72830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c8da5700 1 -- 172.21.15.26:0/391068882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625cdb75560 con 0x5625cdb6e260 2024-05-20T20:59:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb72830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb68270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40a9870 tx=0x7fb2b4004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9e14137ffa855d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2aeffd700 1 -- 172.21.15.26:0/391068882 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400ce90 con 0x5625cdb6e260 2024-05-20T20:59:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2aeffd700 1 -- 172.21.15.26:0/391068882 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2b40a9680 con 0x5625cdb6e260 2024-05-20T20:59:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.938+0000 7fb2aeffd700 1 -- 172.21.15.26:0/391068882 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40a9b70 con 0x5625cdb6e260 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/391068882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 msgr2=0x5625cdb68270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb68270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40a9870 tx=0x7fb2b4004660 comp rx=0 tx=0).stop 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/391068882 shutdown_connections 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb6d8a0 0x5625cdb72830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb73320 0x5625cdb8ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/391068882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb68270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/391068882 >> 172.21.15.26:0/391068882 conn(0x5625cda99240 msgr2=0x5625cdb72120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/391068882 shutdown_connections 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/391068882 wait complete. 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.939+0000 7fb2d60a0b80 1 -- start start 2024-05-20T20:59:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 Processor -- start 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6d8a0 0x5625cdb7fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb8e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb73320 0x5625cdb8e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41ac150 con 0x5625cdb73320 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41ad5c0 con 0x5625cdb6e260 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41ac820 con 0x5625cdb6d8a0 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2d60a0b80 10 client.0 _ll_register_callbacks cb 0x5625cda56950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb8e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2c8da5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb8e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:52908/0 (socket says 172.21.15.26:52908) 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2c8da5700 1 -- 172.21.15.26:0/633130107 learned_addr learned my addr 172.21.15.26:0/633130107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6d8a0 0x5625cdb7fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.940+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb73320 0x5625cdb8e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c8da5700 1 -- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6d8a0 msgr2=0x5625cdb7fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6d8a0 0x5625cdb7fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c8da5700 1 -- 172.21.15.26:0/633130107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb73320 msgr2=0x5625cdb8e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb73320 0x5625cdb8e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c8da5700 1 -- 172.21.15.26:0/633130107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625cdb8efa0 con 0x5625cdb6e260 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c9da7700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb73320 0x5625cdb8e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6d8a0 0x5625cdb7fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2acff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2c8da5700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb8e470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4000c00 tx=0x7fb2b40a5b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82d3480a6ef242a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2acff9700 1 -- 172.21.15.26:0/633130107 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40c9070 con 0x5625cdb6e260 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2acff9700 1 -- 172.21.15.26:0/633130107 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fb2b40e2320 con 0x5625cdb6e260 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b40a9680 con 0x5625cdb6e260 2024-05-20T20:59:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2acff9700 1 -- 172.21.15.26:0/633130107 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2b400ce90 con 0x5625cdb6e260 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2d60a0b80 10 client.26006 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.941+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625cdbf1af0 con 0x5625cdb6e260 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2acff9700 1 -- 172.21.15.26:0/633130107 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2b40a9680 con 0x5625cdb6e260 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2acff9700 1 -- 172.21.15.26:0/633130107 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb2b40a8840 con 0x5625cdb6e260 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2d60a0b80 10 client.26006 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2d60a0b80 10 client.26006 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2d60a0b80 20 client.26006 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdbf1f80 0x5625cdbf43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2d60a0b80 4 client.26006 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625cdac74a0 con 0x5625cdbf1f80 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.942+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdbf1f80 0x5625cdbf43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2acff9700 10 client.26006 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2c95a6700 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdbf1f80 0x5625cdbf43a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a700f240c23fc0a3 server_cookie=55bb4a039f048e0c in_seq=0 out_seq=0 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2acff9700 1 -- 172.21.15.26:0/633130107 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625cdac74a0 con 0x5625cdbf1f80 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2acff9700 10 client.26006 handle_command_reply: tid=0 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 client.26006 shutdown 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 msgr2=0x5625cdb8e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb8e470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4000c00 tx=0x7fb2b40a5b60 comp rx=0 tx=0).stop 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 shutdown_connections 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5625cdbf1f80 0x5625cdbf43a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625cdb73320 0x5625cdb8e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2acff9700 0 client.26006 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625cdb6d8a0 0x5625cdb7fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 --2- 172.21.15.26:0/633130107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625cdb6e260 0x5625cdb8e470 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 >> 172.21.15.26:0/633130107 conn(0x5625cda99240 msgr2=0x5625cdb70d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 shutdown_connections 2024-05-20T20:59:37.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 1 -- 172.21.15.26:0/633130107 wait complete. 2024-05-20T20:59:37.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.943+0000 7fb2d60a0b80 20 client.26006 trim_cache size 0 max 16384 2024-05-20T20:59:37.948 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:59:37.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2b00b2880 msgr2=0x7fb2b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2b00b2880 0x7fb2b00b4d40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009680 tx=0x7fb2c0009260 comp rx=0 tx=0).stop 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 msgr2=0x7fb2c419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c419fc00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400eb30 tx=0x7fb2b400ee40 comp rx=0 tx=0).stop 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 shutdown_connections 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2c41096a0 0x7fb2c4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c4103590 0x7fb2c419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2b00b2880 0x7fb2b00b4d40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 --2- 172.21.15.26:0/3362931360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c4103f50 0x7fb2c419fc00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 >> 172.21.15.26:0/3362931360 conn(0x7fb2c407a6f0 msgr2=0x7fb2c4102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 shutdown_connections 2024-05-20T20:59:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:37.944+0000 7fb2cb80a700 1 -- 172.21.15.26:0/3362931360 wait complete. 2024-05-20T20:59:38.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 59379 3.22 MB/sec warmup 104 sec latency 15.066 ms 2024-05-20T20:59:38.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T20:59:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:39.339 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:39 smithi026 ceph-mon[31990]: pgmap v493: 97 pgs: 97 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-20T20:59:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:39 smithi055 ceph-mon[35213]: pgmap v493: 97 pgs: 97 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-20T20:59:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:39 smithi144 ceph-mon[35660]: pgmap v493: 97 pgs: 97 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-20T20:59:39.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 60101 3.25 MB/sec warmup 105 sec latency 17.750 ms 2024-05-20T20:59:40.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 60664 3.26 MB/sec warmup 106 sec latency 17.699 ms 2024-05-20T20:59:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 -- 172.21.15.26:0/1986476752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30103f90 msgr2=0x7fce30110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1986476752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30103f90 0x7fce30110470 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fce2800b0a0 tx=0x7fce2800b3b0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 -- 172.21.15.26:0/1986476752 shutdown_connections 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1986476752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30103f90 0x7fce30110470 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1986476752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce30108500 0x7fce301088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1986476752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce301035f0 0x7fce30103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 -- 172.21.15.26:0/1986476752 >> 172.21.15.26:0/1986476752 conn(0x7fce3007a6a0 msgr2=0x7fce3007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 -- 172.21.15.26:0/1986476752 shutdown_connections 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.942+0000 7fce34fc4700 1 -- 172.21.15.26:0/1986476752 wait complete. 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 Processor -- start 2024-05-20T20:59:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 -- start start 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce301035f0 0x7fce3019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 0x7fce3019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 0x7fce301998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce30113690 con 0x7fce30108500 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fce30113510 con 0x7fce30103f90 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce34fc4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce30113810 con 0x7fce301035f0 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 0x7fce301998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 0x7fce3019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 0x7fce301998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43956/0 (socket says 172.21.15.26:43956) 2024-05-20T20:59:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 0x7fce3019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35208/0 (socket says 172.21.15.26:35208) 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2ed9d700 1 -- 172.21.15.26:0/1394637324 learned_addr learned my addr 172.21.15.26:0/1394637324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2e59c700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce301035f0 0x7fce3019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2dd9b700 1 -- 172.21.15.26:0/1394637324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce301035f0 msgr2=0x7fce3019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.943+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce301035f0 0x7fce3019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce2dd9b700 1 -- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 msgr2=0x7fce301998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 0x7fce301998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce2dd9b700 1 -- 172.21.15.26:0/1394637324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3019a030 con 0x7fce30103f90 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce2e59c700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce301035f0 0x7fce3019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 0x7fce301998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 0x7fce3019fc50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce2400e9f0 tx=0x7fce2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce1f7fe700 1 -- 172.21.15.26:0/1394637324 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2400cb70 con 0x7fce30103f90 2024-05-20T20:59:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3019a2c0 con 0x7fce30103f90 2024-05-20T20:59:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce1f7fe700 1 -- 172.21.15.26:0/1394637324 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce240044b0 con 0x7fce30103f90 2024-05-20T20:59:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce1f7fe700 1 -- 172.21.15.26:0/1394637324 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24003ca0 con 0x7fce30103f90 2024-05-20T20:59:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.944+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce301a65e0 con 0x7fce30103f90 2024-05-20T20:59:40.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.946+0000 7fce1f7fe700 1 -- 172.21.15.26:0/1394637324 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fce24003f60 con 0x7fce30103f90 2024-05-20T20:59:40.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.947+0000 7fce1f7fe700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce180b2880 0x7fce180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.947+0000 7fce2e59c700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce180b2880 0x7fce180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.948+0000 7fce1f7fe700 1 -- 172.21.15.26:0/1394637324 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fce240e2560 con 0x7fce30103f90 2024-05-20T20:59:40.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.948+0000 7fce2e59c700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce180b2880 0x7fce180b4d40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce20005e90 tx=0x7fce20005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.948+0000 7fce3f85ab80 1 Processor -- start 2024-05-20T20:59:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.948+0000 7fce3f85ab80 1 -- start start 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bb8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bb2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bce1b10d70 con 0x55bce1bb8320 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bce1b00910 con 0x55bce1bb3260 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bce1b04950 con 0x55bce1bb28a0 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bb8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bb8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43962/0 (socket says 172.21.15.26:43962) 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2e59c700 1 -- 172.21.15.26:0/341629642 learned_addr learned my addr 172.21.15.26:0/341629642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bb2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2ed9d700 1 -- 172.21.15.26:0/341629642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 msgr2=0x55bce1bb2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bb2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2ed9d700 1 -- 172.21.15.26:0/341629642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 msgr2=0x55bce1bb8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bb8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.949+0000 7fce2ed9d700 1 -- 172.21.15.26:0/341629642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce1aa9fb0 con 0x55bce1bb3260 2024-05-20T20:59:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.950+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbf080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fce280062a0 tx=0x7fce28009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114b3ae77ee3d0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.950+0000 7fce1cff9700 1 -- 172.21.15.26:0/341629642 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28021070 con 0x55bce1bb3260 2024-05-20T20:59:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.950+0000 7fce1cff9700 1 -- 172.21.15.26:0/341629642 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce28009460 con 0x55bce1bb3260 2024-05-20T20:59:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.950+0000 7fce1cff9700 1 -- 172.21.15.26:0/341629642 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28025450 con 0x55bce1bb3260 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 -- 172.21.15.26:0/341629642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 msgr2=0x55bce1bbf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbf080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fce280062a0 tx=0x7fce28009dd0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 -- 172.21.15.26:0/341629642 shutdown_connections 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bb8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbf080 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/341629642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bb2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 -- 172.21.15.26:0/341629642 >> 172.21.15.26:0/341629642 conn(0x55bce1ade240 msgr2=0x55bce1bad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 -- 172.21.15.26:0/341629642 shutdown_connections 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 -- 172.21.15.26:0/341629642 wait complete. 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 -- start start 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.951+0000 7fce3f85ab80 1 Processor -- start 2024-05-20T20:59:40.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb8320 0x55bce1bbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43966/0 (socket says 172.21.15.26:43966) 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2e59c700 1 -- 172.21.15.26:0/2011670866 learned_addr learned my addr 172.21.15.26:0/2011670866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bce1a06fa0 con 0x55bce1bb28a0 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bce1a68da0 con 0x55bce1bb3260 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bce1bc3b30 con 0x55bce1bb8320 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce3f85ab80 10 client.0 _ll_register_callbacks cb 0x55bce1a9b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb8320 0x55bce1bbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2dd9b700 1 -- 172.21.15.26:0/2011670866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb8320 msgr2=0x55bce1bbc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb8320 0x55bce1bbc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2dd9b700 1 -- 172.21.15.26:0/2011670866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 msgr2=0x55bce1bbece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2dd9b700 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce1a508d0 con 0x55bce1bb3260 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb8320 0x55bce1bbc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.952+0000 7fce2e59c700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbc030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce240dffa0 tx=0x7fce240a95d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4847e3c9ae59457c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce15ffb700 1 -- 172.21.15.26:0/2011670866 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240c3060 con 0x55bce1bb3260 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce15ffb700 1 -- 172.21.15.26:0/2011670866 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fce2400cd10 con 0x55bce1bb3260 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce28009460 con 0x55bce1bb3260 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce15ffb700 1 -- 172.21.15.26:0/2011670866 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce240b2270 con 0x55bce1bb3260 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce3f85ab80 10 client.26018 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.953+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bce1aba760 con 0x55bce1bb3260 2024-05-20T20:59:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce15ffb700 1 -- 172.21.15.26:0/2011670866 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce28009460 con 0x55bce1bb3260 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce15ffb700 1 -- 172.21.15.26:0/2011670866 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fce240a9d20 con 0x55bce1bb3260 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce3f85ab80 10 client.26018 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce3f85ab80 10 client.26018 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce3f85ab80 20 client.26018 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1bd29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce3f85ab80 4 client.26018 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bce1b12cc0 con 0x55bce1bd05f0 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.954+0000 7fce2e59c700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1bd29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce15ffb700 10 client.26018 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce2e59c700 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1bd29e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e0f06f340e7b1e8 server_cookie=105da81f281ac866 in_seq=0 out_seq=0 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce15ffb700 1 -- 172.21.15.26:0/2011670866 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bce1b12cc0 con 0x55bce1bd05f0 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce15ffb700 10 client.26018 handle_command_reply: tid=0 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce3f85ab80 1 client.26018 shutdown 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 msgr2=0x55bce1bbc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbc030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce240dffa0 tx=0x7fce240a95d0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 shutdown_connections 2024-05-20T20:59:40.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.955+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1bd29e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb8320 0x55bce1bbc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/2011670866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bbc030 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce15ffb700 0 client.26018 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 >> 172.21.15.26:0/2011670866 conn(0x55bce1ade240 msgr2=0x55bce1bacaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 shutdown_connections 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 1 -- 172.21.15.26:0/2011670866 wait complete. 2024-05-20T20:59:40.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.956+0000 7fce3f85ab80 20 client.26018 trim_cache size 0 max 16384 2024-05-20T20:59:40.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 Processor -- start 2024-05-20T20:59:40.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 -- start start 2024-05-20T20:59:40.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb3260 0x55bce1bad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb8320 0x55bce1bb63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ae2c0 con 0x55bce1bb28a0 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fce301abb50 con 0x55bce1bb8320 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce301a99a0 con 0x55bce1bb3260 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43970/0 (socket says 172.21.15.26:43970) 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2e59c700 1 -- 172.21.15.26:0/736760564 learned_addr learned my addr 172.21.15.26:0/736760564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb8320 0x55bce1bb63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb3260 0x55bce1bad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2ed9d700 1 -- 172.21.15.26:0/736760564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb3260 msgr2=0x55bce1bad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb3260 0x55bce1bad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2ed9d700 1 -- 172.21.15.26:0/736760564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 msgr2=0x55bce1bbaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.973+0000 7fce2ed9d700 1 -- 172.21.15.26:0/736760564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce1bae090 con 0x55bce1bb8320 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.974+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb8320 0x55bce1bb63f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce28009e60 tx=0x7fce280061a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3693d7496eb57f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.974+0000 7fce17fff700 1 -- 172.21.15.26:0/736760564 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28021040 con 0x55bce1bb8320 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.974+0000 7fce17fff700 1 -- 172.21.15.26:0/736760564 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce28007930 con 0x55bce1bb8320 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.974+0000 7fce17fff700 1 -- 172.21.15.26:0/736760564 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28025990 con 0x55bce1bb8320 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 -- 172.21.15.26:0/736760564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb8320 msgr2=0x55bce1bb63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb8320 0x55bce1bb63f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce28009e60 tx=0x7fce280061a0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 -- 172.21.15.26:0/736760564 shutdown_connections 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb28a0 0x55bce1bbaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb8320 0x55bce1bb63f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/736760564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb3260 0x55bce1bad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 -- 172.21.15.26:0/736760564 >> 172.21.15.26:0/736760564 conn(0x55bce1ade240 msgr2=0x55bce1bb5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 -- 172.21.15.26:0/736760564 shutdown_connections 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 -- 172.21.15.26:0/736760564 wait complete. 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 -- start start 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.975+0000 7fce3f85ab80 1 Processor -- start 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bc93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bd7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bd7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aa010 con 0x55bce1bb8320 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ab480 con 0x55bce1bb3260 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aa6e0 con 0x55bce1bb28a0 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 10 client.0 _ll_register_callbacks cb 0x55bce1a9b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bd7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bd7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43980/0 (socket says 172.21.15.26:43980) 2024-05-20T20:59:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 -- 172.21.15.26:0/3310059357 learned_addr learned my addr 172.21.15.26:0/3310059357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bd7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2e59c700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bc93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 -- 172.21.15.26:0/3310059357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 msgr2=0x55bce1bc93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bc93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 -- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 msgr2=0x55bce1bd7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bd7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 -- 172.21.15.26:0/3310059357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce1bd8590 con 0x55bce1bb8320 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2e59c700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bc93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2dd9b700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bd7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce2ed9d700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bd7e60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fce28009e10 tx=0x7fce28007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bb480369feac88 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce15ffb700 1 -- 172.21.15.26:0/3310059357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28012700 con 0x55bce1bb8320 2024-05-20T20:59:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce28007930 con 0x55bce1bb8320 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.976+0000 7fce15ffb700 1 -- 172.21.15.26:0/3310059357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7fce280128a0 con 0x55bce1bb8320 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce15ffb700 1 -- 172.21.15.26:0/3310059357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce28012aa0 con 0x55bce1bb8320 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 10 client.16485 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bce1c3afd0 con 0x55bce1bb8320 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce15ffb700 1 -- 172.21.15.26:0/3310059357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce28007930 con 0x55bce1bb8320 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce15ffb700 1 -- 172.21.15.26:0/3310059357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fce28011070 con 0x55bce1bb8320 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 10 client.16485 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 10 client.16485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 20 client.16485 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1c3d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 4 client.16485 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bce1b0c4a0 con 0x55bce1bd05f0 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.977+0000 7fce2e59c700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1c3d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce15ffb700 10 client.16485 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce2e59c700 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1c3d1b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=764e49b9c25d3f5d server_cookie=973ae6c65b20af8c in_seq=0 out_seq=0 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce15ffb700 1 -- 172.21.15.26:0/3310059357 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bce1b0c4a0 con 0x55bce1bd05f0 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce15ffb700 10 client.16485 handle_command_reply: tid=0 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 client.16485 shutdown 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 msgr2=0x55bce1bd7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bd7e60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fce28009e10 tx=0x7fce28007780 comp rx=0 tx=0).stop 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 shutdown_connections 2024-05-20T20:59:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55bce1bd05f0 0x55bce1c3d1b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce1bb8320 0x55bce1bd7e60 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bce1bb28a0 0x55bce1bc93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 --2- 172.21.15.26:0/3310059357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bce1bb3260 0x55bce1bd7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 >> 172.21.15.26:0/3310059357 conn(0x55bce1ade240 msgr2=0x55bce1bb7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce15ffb700 0 client.16485 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.978+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 shutdown_connections 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce3f85ab80 1 -- 172.21.15.26:0/3310059357 wait complete. 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce3f85ab80 20 client.16485 trim_cache size 0 max 16384 2024-05-20T20:59:40.983 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:59:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce180b2880 msgr2=0x7fce180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce180b2880 0x7fce180b4d40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce20005e90 tx=0x7fce20005da0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 msgr2=0x7fce3019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 0x7fce3019fc50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce2400e9f0 tx=0x7fce2400ed00 comp rx=0 tx=0).stop 2024-05-20T20:59:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 shutdown_connections 2024-05-20T20:59:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce180b2880 0x7fce180b4d40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce30108500 0x7fce301998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce301035f0 0x7fce3019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 --2- 172.21.15.26:0/1394637324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce30103f90 0x7fce3019fc50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 >> 172.21.15.26:0/1394637324 conn(0x7fce3007a6a0 msgr2=0x7fce301016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 shutdown_connections 2024-05-20T20:59:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:40.979+0000 7fce34fc4700 1 -- 172.21.15.26:0/1394637324 wait complete. 2024-05-20T20:59:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:41 smithi055 ceph-mon[35213]: pgmap v494: 97 pgs: 97 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T20:59:41.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:41 smithi026 ceph-mon[31990]: pgmap v494: 97 pgs: 97 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T20:59:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:41 smithi144 ceph-mon[35660]: pgmap v494: 97 pgs: 97 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T20:59:41.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 61129 3.24 MB/sec warmup 107 sec latency 18.851 ms 2024-05-20T20:59:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 -- 172.21.15.26:0/69932342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac1019b0 msgr2=0x7f11ac101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 --2- 172.21.15.26:0/69932342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac1019b0 0x7f11ac101dd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009900 tx=0x7f11a802a410 comp rx=0 tx=0).stop 2024-05-20T20:59:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 -- 172.21.15.26:0/69932342 shutdown_connections 2024-05-20T20:59:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 --2- 172.21.15.26:0/69932342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac103560 0x7f11ac109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 --2- 172.21.15.26:0/69932342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac102ba0 0x7f11ac103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 --2- 172.21.15.26:0/69932342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac1019b0 0x7f11ac101dd0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 -- 172.21.15.26:0/69932342 >> 172.21.15.26:0/69932342 conn(0x7f11ac0fcf20 msgr2=0x7f11ac0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 -- 172.21.15.26:0/69932342 shutdown_connections 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.686+0000 7f11b46d1640 1 -- 172.21.15.26:0/69932342 wait complete. 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 Processor -- start 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 -- start start 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac1019b0 0x7f11ac198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac102ba0 0x7f11ac199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103560 0x7f11ac1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10d380 con 0x7f11ac102ba0 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10d200 con 0x7f11ac1019b0 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b46d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10d500 con 0x7f11ac103560 2024-05-20T20:59:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b1c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac102ba0 0x7f11ac199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b1c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac102ba0 0x7f11ac199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44002/0 (socket says 172.21.15.26:44002) 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b246d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac1019b0 0x7f11ac198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2738323674 learned_addr learned my addr 172.21.15.26:0/2738323674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103560 0x7f11ac1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b246d640 1 -- 172.21.15.26:0/2738323674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103560 msgr2=0x7f11ac1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b246d640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103560 0x7f11ac1a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b246d640 1 -- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac102ba0 msgr2=0x7f11ac199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.687+0000 7f11b246d640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac102ba0 0x7f11ac199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11b246d640 1 -- 172.21.15.26:0/2738323674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac1a07f0 con 0x7f11ac1019b0 2024-05-20T20:59:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11b246d640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac1019b0 0x7f11ac198bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009230 tx=0x7f11a80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11a37fe640 1 -- 172.21.15.26:0/2738323674 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a803f070 con 0x7f11ac1019b0 2024-05-20T20:59:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11a37fe640 1 -- 172.21.15.26:0/2738323674 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11a80044c0 con 0x7f11ac1019b0 2024-05-20T20:59:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac1a0a80 con 0x7f11ac1019b0 2024-05-20T20:59:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11a37fe640 1 -- 172.21.15.26:0/2738323674 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8035070 con 0x7f11ac1019b0 2024-05-20T20:59:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.688+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac1a0f70 con 0x7f11ac1019b0 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11a37fe640 1 -- 172.21.15.26:0/2738323674 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f11a80037f0 con 0x7f11ac1019b0 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 Processor -- start 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 -- start start 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a88ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a90ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b0799f5c30 con 0x55b079a88ad0 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b0799e57d0 con 0x55b079a8e630 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.691+0000 7f11b73ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0799e9810 con 0x55b079a894c0 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b2c6e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a88ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a90ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b2c6e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a88ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44010/0 (socket says 172.21.15.26:44010) 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b2c6e640 1 -- 172.21.15.26:0/2156554282 learned_addr learned my addr 172.21.15.26:0/2156554282 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b246d640 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11a37fe640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11980b28a0 0x7f11980b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2156554282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 msgr2=0x55b079a8dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a8dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2156554282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 msgr2=0x55b079a88ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a88ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2156554282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0796fa210 con 0x55b079a8e630 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a88ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11980b28a0 0x7f11980b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11a37fe640 1 -- 172.21.15.26:0/2738323674 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f11a8102590 con 0x7f11ac1019b0 2024-05-20T20:59:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a90ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f119c00bc80 tx=0x7f119c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121500d5f980d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11a0ff9640 1 -- 172.21.15.26:0/2156554282 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c005220 con 0x55b079a8e630 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11a0ff9640 1 -- 172.21.15.26:0/2156554282 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f119c00be30 con 0x55b079a8e630 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.692+0000 7f11a0ff9640 1 -- 172.21.15.26:0/2156554282 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0116a0 con 0x55b079a8e630 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11980b28a0 0x7f11980b4d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55b079a89d20 tx=0x7f11a4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 -- 172.21.15.26:0/2156554282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 msgr2=0x55b079a90ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a90ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f119c00bc80 tx=0x7f119c0107a0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 -- 172.21.15.26:0/2156554282 shutdown_connections 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a90ac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2156554282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a88ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 -- 172.21.15.26:0/2156554282 >> 172.21.15.26:0/2156554282 conn(0x55b079a7fd30 msgr2=0x55b079a80130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 -- 172.21.15.26:0/2156554282 shutdown_connections 2024-05-20T20:59:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 -- 172.21.15.26:0/2156554282 wait complete. 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.693+0000 7f11b73ed740 1 -- start start 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 Processor -- start 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 0x55b079a92bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a93100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a8e630 0x55b079a9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b0796cb790 con 0x55b079a8e630 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b0799a1890 con 0x55b079a88ad0 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b079a915e0 con 0x55b079a894c0 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b73ed740 10 client.0 _ll_register_callbacks cb 0x55b0796f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 0x55b079a92bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b2c6e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a8e630 0x55b079a9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 0x55b079a92bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35260/0 (socket says 172.21.15.26:35260) 2024-05-20T20:59:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 -- 172.21.15.26:0/3276276915 learned_addr learned my addr 172.21.15.26:0/3276276915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a93100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 -- 172.21.15.26:0/3276276915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 msgr2=0x55b079a93100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a93100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 -- 172.21.15.26:0/3276276915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a8e630 msgr2=0x55b079a9a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a8e630 0x55b079a9a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b246d640 1 -- 172.21.15.26:0/3276276915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0797c6620 con 0x55b079a88ad0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.694+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a8e630 0x55b079a9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f11b246d640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 0x55b079a92bc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f11a802a930 tx=0x7f11a803b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd5259bd1dc6568 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f1195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f1195ffb640 1 -- 172.21.15.26:0/3276276915 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8048040 con 0x55b079a88ad0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f1195ffb640 1 -- 172.21.15.26:0/3276276915 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f11a80c9e00 con 0x55b079a88ad0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c00be30 con 0x55b079a88ad0 2024-05-20T20:59:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f1195ffb640 1 -- 172.21.15.26:0/3276276915 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11a80c94f0 con 0x55b079a88ad0 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f11b73ed740 10 client.26036 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b079a9d120 con 0x55b079a88ad0 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.695+0000 7f1195ffb640 1 -- 172.21.15.26:0/3276276915 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f119c00be30 con 0x55b079a88ad0 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f1195ffb640 1 -- 172.21.15.26:0/3276276915 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11a804b060 con 0x55b079a88ad0 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b73ed740 10 client.26036 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b73ed740 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b73ed740 20 client.26036 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079a9e850 0x55b079aa0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b73ed740 4 client.26036 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0799f7b80 con 0x55b079a9e850 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079a9e850 0x55b079aa0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079a9e850 0x55b079aa0d10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac2628c349ebf37 server_cookie=c16386dbd4caa4ce in_seq=0 out_seq=0 2024-05-20T20:59:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.696+0000 7f1195ffb640 10 client.26036 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f1195ffb640 1 -- 172.21.15.26:0/3276276915 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b0799f7b80 con 0x55b079a9e850 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f1195ffb640 10 client.26036 handle_command_reply: tid=0 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 client.26036 shutdown 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 msgr2=0x55b079a92bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 0x55b079a92bc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f11a802a930 tx=0x7f11a803b690 comp rx=0 tx=0).stop 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 shutdown_connections 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079a9e850 0x55b079aa0d10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a8e630 0x55b079a9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a894c0 0x55b079a93100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f1195ffb640 0 client.26036 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3276276915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a88ad0 0x55b079a92bc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.697+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 >> 172.21.15.26:0/3276276915 conn(0x55b079a7fd30 msgr2=0x55b079a7efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.698+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 shutdown_connections 2024-05-20T20:59:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.698+0000 7f11b73ed740 1 -- 172.21.15.26:0/3276276915 wait complete. 2024-05-20T20:59:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.698+0000 7f11b73ed740 20 client.26036 trim_cache size 0 max 16384 2024-05-20T20:59:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 Processor -- start 2024-05-20T20:59:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 -- start start 2024-05-20T20:59:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a894c0 0x55b079a9a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a8e630 0x55b079a927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1aa510 con 0x55b079a88ad0 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1a7de0 con 0x55b079a894c0 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b73ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1a5c30 con 0x55b079a8e630 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b246d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b246d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a7ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44022/0 (socket says 172.21.15.26:44022) 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b246d640 1 -- 172.21.15.26:0/3585196547 learned_addr learned my addr 172.21.15.26:0/3585196547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.715+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a894c0 0x55b079a9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a8e630 0x55b079a927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b246d640 1 -- 172.21.15.26:0/3585196547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a8e630 msgr2=0x55b079a927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b246d640 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a8e630 0x55b079a927d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b246d640 1 -- 172.21.15.26:0/3585196547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a894c0 msgr2=0x55b079a9a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b246d640 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a894c0 0x55b079a9a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b246d640 1 -- 172.21.15.26:0/3585196547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b079a7fb60 con 0x55b079a88ad0 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a894c0 0x55b079a9a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f11b246d640 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a7ae40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f11a8006010 tx=0x7f11a8102810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5c0ef17a43ac10 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f1197fff640 1 -- 172.21.15.26:0/3585196547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8048070 con 0x55b079a88ad0 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f1197fff640 1 -- 172.21.15.26:0/3585196547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11a80c96b0 con 0x55b079a88ad0 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.716+0000 7f1197fff640 1 -- 172.21.15.26:0/3585196547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a80c5330 con 0x55b079a88ad0 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 -- 172.21.15.26:0/3585196547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 msgr2=0x55b079a7ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a7ae40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f11a8006010 tx=0x7f11a8102810 comp rx=0 tx=0).stop 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 -- 172.21.15.26:0/3585196547 shutdown_connections 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a8e630 0x55b079a927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a894c0 0x55b079a9a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 --2- 172.21.15.26:0/3585196547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a88ad0 0x55b079a7ae40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 -- 172.21.15.26:0/3585196547 >> 172.21.15.26:0/3585196547 conn(0x55b079a7fd30 msgr2=0x55b079a8dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 -- 172.21.15.26:0/3585196547 shutdown_connections 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 -- 172.21.15.26:0/3585196547 wait complete. 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.717+0000 7f11b73ed740 1 -- start start 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 Processor -- start 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a88ad0 0x55b079a97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 0x55b079a97e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a98e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1a62a0 con 0x55b079a894c0 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1a7710 con 0x55b079a8e630 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1a6970 con 0x55b079a88ad0 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b73ed740 10 client.0 _ll_register_callbacks cb 0x7f11a8105cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 0x55b079a97e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 0x55b079a97e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44030/0 (socket says 172.21.15.26:44030) 2024-05-20T20:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2064143511 learned_addr learned my addr 172.21.15.26:0/2064143511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b246d640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a88ad0 0x55b079a97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b2c6e640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a98e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2064143511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a88ad0 msgr2=0x55b079a97910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a88ad0 0x55b079a97910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 msgr2=0x55b079a98e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a98e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.718+0000 7f11b1c6c640 1 -- 172.21.15.26:0/2064143511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b079a99450 con 0x55b079a894c0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b1c6c640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 0x55b079a97e50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f119c010ca0 tx=0x7f119c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4f5ebcf66e7620 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f1195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f1195ffb640 1 -- 172.21.15.26:0/2064143511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c036060 con 0x55b079a894c0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f1195ffb640 1 -- 172.21.15.26:0/2064143511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8327+0+0 (secure 0 0 0) 0x7f119c0055b0 con 0x55b079a894c0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a80c96b0 con 0x55b079a894c0 2024-05-20T20:59:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f1195ffb640 1 -- 172.21.15.26:0/2064143511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f119c004660 con 0x55b079a894c0 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 10 client.16503 fetch_fsmap learned FSMap version 26 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b079b09550 con 0x55b079a894c0 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f1195ffb640 1 -- 172.21.15.26:0/2064143511 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11a80c96b0 con 0x55b079a894c0 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f1195ffb640 1 -- 172.21.15.26:0/2064143511 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f119c03a030 con 0x55b079a894c0 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 10 client.16503 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 20 client.16503 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079b09bc0 0x55b079b0bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 4 client.16503 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.719+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0799f1360 con 0x55b079b09bc0 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.720+0000 7f11b246d640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079b09bc0 0x55b079b0bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.720+0000 7f1195ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.720+0000 7f11b246d640 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079b09bc0 0x55b079b0bfb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12a275528529013 server_cookie=c5229e8f33285d09 in_seq=0 out_seq=0 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f1195ffb640 1 -- 172.21.15.26:0/2064143511 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b0799f1360 con 0x55b079b09bc0 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f1195ffb640 10 client.16503 handle_command_reply: tid=0 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 client.16503 shutdown 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 msgr2=0x55b079a97e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 0x55b079a97e50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f119c010ca0 tx=0x7f119c00b810 comp rx=0 tx=0).stop 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 shutdown_connections 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b079b09bc0 0x55b079b0bfb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b079a8e630 0x55b079a98e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b079a894c0 0x55b079a97e50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f1195ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 --2- 172.21.15.26:0/2064143511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b079a88ad0 0x55b079a97910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.755+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 >> 172.21.15.26:0/2064143511 conn(0x55b079a7fd30 msgr2=0x55b079a82d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 shutdown_connections 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b73ed740 1 -- 172.21.15.26:0/2064143511 wait complete. 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b73ed740 20 client.16503 trim_cache size 0 max 16384 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:59:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:59:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11980b28a0 msgr2=0x7f11980b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b46d1640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11980b28a0 0x7f11980b4d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55b079a89d20 tx=0x7f11a4004a40 comp rx=0 tx=0).stop 2024-05-20T20:59:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac1019b0 msgr2=0x7f11ac198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.756+0000 7f11b46d1640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac1019b0 0x7f11ac198bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009230 tx=0x7f11a80040c0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.757+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 shutdown_connections 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.757+0000 7f11b46d1640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11980b28a0 0x7f11980b4d60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.757+0000 7f11b46d1640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103560 0x7f11ac1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.757+0000 7f11b46d1640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ac102ba0 0x7f11ac199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.757+0000 7f11b46d1640 1 --2- 172.21.15.26:0/2738323674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11ac1019b0 0x7f11ac198bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.757+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 >> 172.21.15.26:0/2738323674 conn(0x7f11ac0fcf20 msgr2=0x7f11ac06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.758+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 shutdown_connections 2024-05-20T20:59:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:41.758+0000 7f11b46d1640 1 -- 172.21.15.26:0/2738323674 wait complete. 2024-05-20T20:59:41.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T20:59:41.984 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.1 2024-05-20T20:59:41.985 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T20:59:42.200 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:42.550 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:59:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: ignoring --setuser ceph since I am not root 2024-05-20T20:59:42.550 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:59:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: ignoring --setgroup ceph since I am not root 2024-05-20T20:59:42.550 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:59:42 smithi144 ceph-mds[65957]: -- 172.21.15.144:0/4153447249 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560297373a40 con 0x560297346800 2024-05-20T20:59:42.550 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 20:59:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: starting mds.c at 2024-05-20T20:59:42.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 61757 3.25 MB/sec warmup 108 sec latency 21.842 ms 2024-05-20T20:59:43.351 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:43 smithi026 ceph-mon[31990]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-05-20T20:59:43.351 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:43 smithi026 ceph-mon[31990]: daemon mds.c restarted 2024-05-20T20:59:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:43 smithi055 ceph-mon[35213]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-05-20T20:59:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:43 smithi055 ceph-mon[35213]: daemon mds.c restarted 2024-05-20T20:59:43.519 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:43 smithi144 ceph-mon[35660]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-05-20T20:59:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:43 smithi144 ceph-mon[35660]: daemon mds.c restarted 2024-05-20T20:59:43.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 62479 3.23 MB/sec warmup 109 sec latency 11.928 ms 2024-05-20T20:59:44.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:44 smithi026 ceph-mon[31990]: osdmap e83: 12 total, 12 up, 12 in 2024-05-20T20:59:44.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:44 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6836/1050250189,v1:172.21.15.144:6837/1050250189] up:boot 2024-05-20T20:59:44.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:44 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T20:59:44.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:44 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T20:59:44.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:44 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:59:44.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:44 smithi055 ceph-mon[35213]: osdmap e83: 12 total, 12 up, 12 in 2024-05-20T20:59:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:44 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6836/1050250189,v1:172.21.15.144:6837/1050250189] up:boot 2024-05-20T20:59:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:44 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T20:59:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:44 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T20:59:44.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:44 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:59:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:44 smithi144 ceph-mon[35660]: osdmap e83: 12 total, 12 up, 12 in 2024-05-20T20:59:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:44 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6836/1050250189,v1:172.21.15.144:6837/1050250189] up:boot 2024-05-20T20:59:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:44 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T20:59:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:44 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T20:59:44.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:44 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T20:59:44.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 63227 3.25 MB/sec warmup 110 sec latency 49.943 ms 2024-05-20T20:59:45.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:45 smithi026 ceph-mon[31990]: pgmap v497: 97 pgs: 97 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-05-20T20:59:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:45 smithi055 ceph-mon[35213]: pgmap v497: 97 pgs: 97 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 -- 172.21.15.26:0/2692971653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62881084c0 msgr2=0x7f62881088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 --2- 172.21.15.26:0/2692971653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62881084c0 0x7f62881088a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6278009990 tx=0x7f6278009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 -- 172.21.15.26:0/2692971653 shutdown_connections 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 --2- 172.21.15.26:0/2692971653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62881084c0 0x7f62881088a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 --2- 172.21.15.26:0/2692971653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6288103600 0x7f628810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 --2- 172.21.15.26:0/2692971653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 0x7f62881030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 -- 172.21.15.26:0/2692971653 >> 172.21.15.26:0/2692971653 conn(0x7f6288078ab0 msgr2=0x7f62880ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 -- 172.21.15.26:0/2692971653 shutdown_connections 2024-05-20T20:59:45.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.466+0000 7f628d6dd700 1 -- 172.21.15.26:0/2692971653 wait complete. 2024-05-20T20:59:45.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 Processor -- start 2024-05-20T20:59:45.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 -- start start 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 0x7f628819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288103600 0x7f628819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62881084c0 0x7f62881998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f62881124a0 con 0x7f6288103600 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6288112320 con 0x7f6288102c40 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f628d6dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6288112620 con 0x7f62881084c0 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.467+0000 7f62867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288103600 0x7f628819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 0x7f628819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f62867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288103600 0x7f628819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44042/0 (socket says 172.21.15.26:44042) 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f62867fc700 1 -- 172.21.15.26:0/1083910014 learned_addr learned my addr 172.21.15.26:0/1083910014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f62877fe700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62881084c0 0x7f62881998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 -- 172.21.15.26:0/1083910014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62881084c0 msgr2=0x7f62881998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62881084c0 0x7f62881998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 -- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288103600 msgr2=0x7f628819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288103600 0x7f628819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 -- 172.21.15.26:0/1083910014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628819a040 con 0x7f6288102c40 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f62877fe700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62881084c0 0x7f62881998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f6286ffd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 0x7f628819f720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6278000c00 tx=0x7f62780047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f626ffff700 1 -- 172.21.15.26:0/1083910014 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278026070 con 0x7f6288102c40 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.468+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628819a2d0 con 0x7f6288102c40 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.469+0000 7f626ffff700 1 -- 172.21.15.26:0/1083910014 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6278021410 con 0x7f6288102c40 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.469+0000 7f626ffff700 1 -- 172.21.15.26:0/1083910014 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278017650 con 0x7f6288102c40 2024-05-20T20:59:45.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.469+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62881a65f0 con 0x7f6288102c40 2024-05-20T20:59:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.471+0000 7f626ffff700 1 -- 172.21.15.26:0/1083910014 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6278029070 con 0x7f6288102c40 2024-05-20T20:59:45.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.472+0000 7f626ffff700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62700b2900 0x7f62700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.473+0000 7f62867fc700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62700b2900 0x7f62700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.473+0000 7f626ffff700 1 -- 172.21.15.26:0/1083910014 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f62780e2f70 con 0x7f6288102c40 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.473+0000 7f6297f73b80 1 Processor -- start 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.473+0000 7f6297f73b80 1 -- start start 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62867fc700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62700b2900 0x7f62700b4dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62881072a0 tx=0x7f627c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d69860 0x55a194d69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d6d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d6e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6297f73b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a194cc8c90 con 0x55a194d69860 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6297f73b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a194cb8830 con 0x55a194d6df60 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6297f73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a194cbc870 con 0x55a194d6a210 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d6e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d6e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35322/0 (socket says 172.21.15.26:35322) 2024-05-20T20:59:45.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 -- 172.21.15.26:0/388699686 learned_addr learned my addr 172.21.15.26:0/388699686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6286ffd700 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d69860 0x55a194d69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62867fc700 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d6d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 -- 172.21.15.26:0/388699686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 msgr2=0x55a194d6d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d6d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 -- 172.21.15.26:0/388699686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d69860 msgr2=0x55a194d69c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d69860 0x55a194d69c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f62877fe700 1 -- 172.21.15.26:0/388699686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a194c088d0 con 0x55a194d6df60 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.474+0000 7f6286ffd700 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d69860 0x55a194d69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.475+0000 7f62877fe700 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d6e3a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6280003eb0 tx=0x7f6280009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f020a1561c1425b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.475+0000 7f626d7fa700 1 -- 172.21.15.26:0/388699686 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280019020 con 0x55a194d6df60 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.475+0000 7f626d7fa700 1 -- 172.21.15.26:0/388699686 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62800092d0 con 0x55a194d6df60 2024-05-20T20:59:45.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.475+0000 7f626d7fa700 1 -- 172.21.15.26:0/388699686 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628000e870 con 0x55a194d6df60 2024-05-20T20:59:45.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 -- 172.21.15.26:0/388699686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 msgr2=0x55a194d6e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d6e3a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6280003eb0 tx=0x7f6280009920 comp rx=0 tx=0).stop 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 -- 172.21.15.26:0/388699686 shutdown_connections 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d69860 0x55a194d69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d6e3a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 --2- 172.21.15.26:0/388699686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d6d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 -- 172.21.15.26:0/388699686 >> 172.21.15.26:0/388699686 conn(0x55a194c97080 msgr2=0x55a194d63e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 -- 172.21.15.26:0/388699686 shutdown_connections 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.476+0000 7f6297f73b80 1 -- 172.21.15.26:0/388699686 wait complete. 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 -- start start 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 Processor -- start 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 0x55a194d72730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d72c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a194bbefa0 con 0x55a194d6df60 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a194c5a7c0 con 0x55a194d6a210 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a194d76120 con 0x55a194d69860 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6297f73b80 10 client.0 _ll_register_callbacks cb 0x55a194c53920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f62867fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 0x55a194d72730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6286ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d75430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f62877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d72c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f62867fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 0x55a194d72730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35328/0 (socket says 172.21.15.26:35328) 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f62867fc700 1 -- 172.21.15.26:0/1214456246 learned_addr learned my addr 172.21.15.26:0/1214456246 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.477+0000 7f6286ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d75430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:50858/0 (socket says 172.21.15.26:50858) 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f62867fc700 1 -- 172.21.15.26:0/1214456246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 msgr2=0x55a194d75430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f62867fc700 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d75430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f62867fc700 1 -- 172.21.15.26:0/1214456246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 msgr2=0x55a194d72c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f62867fc700 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d72c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f62867fc700 1 -- 172.21.15.26:0/1214456246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a194c57690 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f62867fc700 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 0x55a194d72730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f627c01dc10 tx=0x7f627c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d203c3dca253164 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f626aff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f626aff5700 1 -- 172.21.15.26:0/1214456246 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c00ea50 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f626aff5700 1 -- 172.21.15.26:0/1214456246 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f627c00ebf0 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62800092d0 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f626aff5700 1 -- 172.21.15.26:0/1214456246 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f627c00edf0 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f6297f73b80 10 client.26060 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.478+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a194d80900 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f626aff5700 1 -- 172.21.15.26:0/1214456246 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62800092d0 con 0x55a194d6a210 2024-05-20T20:59:45.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f626aff5700 1 -- 172.21.15.26:0/1214456246 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f627c01e4b0 con 0x55a194d6a210 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6297f73b80 10 client.26060 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6297f73b80 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6297f73b80 20 client.26060 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194d85380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6297f73b80 4 client.26060 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a194d359e0 con 0x55a194d82f90 2024-05-20T20:59:45.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6286ffd700 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194d85380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f6286ffd700 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194d85380 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2236d8ddf50bd16 server_cookie=4da4b41539c4234f in_seq=0 out_seq=0 2024-05-20T20:59:45.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.479+0000 7f626aff5700 10 client.26060 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f626aff5700 1 -- 172.21.15.26:0/1214456246 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a194d359e0 con 0x55a194d82f90 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f626aff5700 10 client.26060 handle_command_reply: tid=0 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f6297f73b80 1 client.26060 shutdown 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 msgr2=0x55a194d72730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 0x55a194d72730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f627c01dc10 tx=0x7f627c0047c0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 shutdown_connections 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.480+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194d85380 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f626aff5700 0 client.26060 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d72c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1214456246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6a210 0x55a194d72730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 >> 172.21.15.26:0/1214456246 conn(0x55a194c97080 msgr2=0x55a194d6f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 shutdown_connections 2024-05-20T20:59:45.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 1 -- 172.21.15.26:0/1214456246 wait complete. 2024-05-20T20:59:45.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.481+0000 7f6297f73b80 20 client.26060 trim_cache size 0 max 16384 2024-05-20T20:59:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.496+0000 7f6297f73b80 1 Processor -- start 2024-05-20T20:59:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 -- start start 2024-05-20T20:59:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d72cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 0x55a194d74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f62881b0410 con 0x55a194d6a210 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f62881adca0 con 0x55a194d6df60 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6297f73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f62881abaf0 con 0x55a194d69860 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 0x55a194d74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62877fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 0x55a194d74670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44064/0 (socket says 172.21.15.26:44064) 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62867fc700 1 -- 172.21.15.26:0/1062032264 learned_addr learned my addr 172.21.15.26:0/1062032264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f6286ffd700 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d72cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62877fe700 1 -- 172.21.15.26:0/1062032264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 msgr2=0x55a194d72cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62877fe700 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d72cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62877fe700 1 -- 172.21.15.26:0/1062032264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 msgr2=0x55a194d74670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62877fe700 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 0x55a194d74670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62877fe700 1 -- 172.21.15.26:0/1062032264 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a194d70cc0 con 0x55a194d6df60 2024-05-20T20:59:45.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.497+0000 7f62867fc700 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 0x55a194d74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:45.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.498+0000 7f62877fe700 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d8ac50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6280009b00 tx=0x7f6280009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5cee7680e55990 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.498+0000 7f626cff9700 1 -- 172.21.15.26:0/1062032264 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280019040 con 0x55a194d6df60 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.498+0000 7f626cff9700 1 -- 172.21.15.26:0/1062032264 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6280004d50 con 0x55a194d6df60 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.498+0000 7f626cff9700 1 -- 172.21.15.26:0/1062032264 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628000e750 con 0x55a194d6df60 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.498+0000 7f6297f73b80 1 -- 172.21.15.26:0/1062032264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 msgr2=0x55a194d8ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.498+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d8ac50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6280009b00 tx=0x7f6280009380 comp rx=0 tx=0).stop 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- 172.21.15.26:0/1062032264 shutdown_connections 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d6df60 0x55a194d8ac50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d69860 0x55a194d72cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 --2- 172.21.15.26:0/1062032264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6a210 0x55a194d74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- 172.21.15.26:0/1062032264 >> 172.21.15.26:0/1062032264 conn(0x55a194c97080 msgr2=0x55a194d73f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- 172.21.15.26:0/1062032264 shutdown_connections 2024-05-20T20:59:45.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- 172.21.15.26:0/1062032264 wait complete. 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- start start 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 Processor -- start 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 0x55a194d79280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f62881ac160 con 0x55a194d6df60 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.499+0000 7f6297f73b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f62881ad5d0 con 0x55a194d69860 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6297f73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f62881ac830 con 0x55a194d6a210 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6297f73b80 10 client.0 _ll_register_callbacks cb 0x55a194c53920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 0x55a194d79280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 0x55a194d79280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35342/0 (socket says 172.21.15.26:35342) 2024-05-20T20:59:45.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 -- 172.21.15.26:0/570447395 learned_addr learned my addr 172.21.15.26:0/570447395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f62877fe700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f62867fc700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 -- 172.21.15.26:0/570447395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 msgr2=0x55a194d797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d797c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 -- 172.21.15.26:0/570447395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 msgr2=0x55a194d79d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d79d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 -- 172.21.15.26:0/570447395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a194deccf0 con 0x55a194d69860 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6286ffd700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 0x55a194d79280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62780aa260 tx=0x7f6278009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c50ffd24254200d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f626aff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f626aff5700 1 -- 172.21.15.26:0/570447395 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62780aa560 con 0x55a194d69860 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f626aff5700 1 -- 172.21.15.26:0/570447395 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f62780aa7c0 con 0x55a194d69860 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.500+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6280004d50 con 0x55a194d69860 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f626aff5700 1 -- 172.21.15.26:0/570447395 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f62780ae910 con 0x55a194d69860 2024-05-20T20:59:45.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 10 client.26066 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a194dedfb0 con 0x55a194d69860 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f626aff5700 1 -- 172.21.15.26:0/570447395 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6280004d50 con 0x55a194d69860 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f626aff5700 1 -- 172.21.15.26:0/570447395 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f62780cb2b0 con 0x55a194d69860 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 10 client.26066 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 20 client.26066 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194df06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 4 client.26066 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a194ccabe0 con 0x55a194d82f90 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.501+0000 7f62867fc700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194df06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f626aff5700 10 client.26066 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f62867fc700 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194df06a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687918acaaa31ac5 server_cookie=d0b80cdc83cbf3d2 in_seq=0 out_seq=0 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f626aff5700 1 -- 172.21.15.26:0/570447395 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a194ccabe0 con 0x55a194d82f90 2024-05-20T20:59:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f626aff5700 10 client.26066 handle_command_reply: tid=0 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 client.26066 shutdown 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 msgr2=0x55a194d79280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 0x55a194d79280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62780aa260 tx=0x7f6278009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 shutdown_connections 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a194d82f90 0x55a194df06a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a194d6df60 0x55a194d79d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a194d69860 0x55a194d79280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 --2- 172.21.15.26:0/570447395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a194d6a210 0x55a194d797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f626aff5700 0 client.26066 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 >> 172.21.15.26:0/570447395 conn(0x55a194c97080 msgr2=0x55a194d68670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 shutdown_connections 2024-05-20T20:59:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 1 -- 172.21.15.26:0/570447395 wait complete. 2024-05-20T20:59:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.502+0000 7f6297f73b80 20 client.26066 trim_cache size 0 max 16384 2024-05-20T20:59:45.507 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62700b2900 msgr2=0x7f62700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62700b2900 0x7f62700b4dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62881072a0 tx=0x7f627c009260 comp rx=0 tx=0).stop 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 msgr2=0x7f628819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 0x7f628819f720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6278000c00 tx=0x7f62780047b0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 shutdown_connections 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62881084c0 0x7f62881998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6288102c40 0x7f628819f720 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f62700b2900 0x7f62700b4dc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 --2- 172.21.15.26:0/1083910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288103600 0x7f628819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 >> 172.21.15.26:0/1083910014 conn(0x7f6288078ab0 msgr2=0x7f62881008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 shutdown_connections 2024-05-20T20:59:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.503+0000 7f628d6dd700 1 -- 172.21.15.26:0/1083910014 wait complete. 2024-05-20T20:59:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:45 smithi144 ceph-mon[35660]: pgmap v497: 97 pgs: 97 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-05-20T20:59:45.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 63826 3.27 MB/sec warmup 111 sec latency 17.280 ms 2024-05-20T20:59:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 -- 172.21.15.26:0/10067538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102440 msgr2=0x7fd36c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 --2- 172.21.15.26:0/10067538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102440 0x7fd36c1028c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009990 tx=0x7fd35c009ca0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 -- 172.21.15.26:0/10067538 shutdown_connections 2024-05-20T20:59:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 --2- 172.21.15.26:0/10067538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36c102e00 0x7fd36c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 --2- 172.21.15.26:0/10067538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd36c108520 0x7fd36c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 --2- 172.21.15.26:0/10067538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102440 0x7fd36c1028c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 -- 172.21.15.26:0/10067538 >> 172.21.15.26:0/10067538 conn(0x7fd36c0793f0 msgr2=0x7fd36c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 -- 172.21.15.26:0/10067538 shutdown_connections 2024-05-20T20:59:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.974+0000 7fd373d42700 1 -- 172.21.15.26:0/10067538 wait complete. 2024-05-20T20:59:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 Processor -- start 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 -- start start 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd36c102440 0x7fd36c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 0x7fd36c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36c108520 0x7fd36c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c112480 con 0x7fd36c102e00 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c112300 con 0x7fd36c102440 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd373d42700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c112600 con 0x7fd36c108520 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd3712dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 0x7fd36c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd3712dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 0x7fd36c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44092/0 (socket says 172.21.15.26:44092) 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd3712dd700 1 -- 172.21.15.26:0/3207809801 learned_addr learned my addr 172.21.15.26:0/3207809801 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd371ade700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd36c102440 0x7fd36c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.975+0000 7fd3722df700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36c108520 0x7fd36c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd3712dd700 1 -- 172.21.15.26:0/3207809801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36c108520 msgr2=0x7fd36c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd3712dd700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36c108520 0x7fd36c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd3712dd700 1 -- 172.21.15.26:0/3207809801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd36c102440 msgr2=0x7fd36c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd3712dd700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd36c102440 0x7fd36c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd3712dd700 1 -- 172.21.15.26:0/3207809801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c19a0f0 con 0x7fd36c102e00 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd3712dd700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 0x7fd36c19fcc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd35c006010 tx=0x7fd35c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c027070 con 0x7fd36c102e00 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd35c022470 con 0x7fd36c102e00 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c19a380 con 0x7fd36c102e00 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1a6650 con 0x7fd36c102e00 2024-05-20T20:59:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.976+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c017650 con 0x7fd36c102e00 2024-05-20T20:59:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.978+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36c06a230 con 0x7fd36c102e00 2024-05-20T20:59:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.979+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd35c02a070 con 0x7fd36c102e00 2024-05-20T20:59:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.980+0000 7fd362ffd700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd3580b2900 0x7fd3580b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.980+0000 7fd371ade700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd3580b2900 0x7fd3580b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.980+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd35c0e3170 con 0x7fd36c102e00 2024-05-20T20:59:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.981+0000 7fd371ade700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd3580b2900 0x7fd3580b4dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd36c107a50 tx=0x7fd368009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:45.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:45.985+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd35c013070 con 0x7fd36c102e00 2024-05-20T20:59:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.289+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36c02e560 con 0x7fd36c102e00 2024-05-20T20:59:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.290+0000 7fd362ffd700 1 -- 172.21.15.26:0/3207809801 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7fd35c0aaec0 con 0x7fd36c102e00 2024-05-20T20:59:46.289 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T20:59:46.289 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T20:59:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd3580b2900 msgr2=0x7fd3580b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd3580b2900 0x7fd3580b4dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd36c107a50 tx=0x7fd368009260 comp rx=0 tx=0).stop 2024-05-20T20:59:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 msgr2=0x7fd36c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 0x7fd36c19fcc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd35c006010 tx=0x7fd35c0057c0 comp rx=0 tx=0).stop 2024-05-20T20:59:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 shutdown_connections 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36c108520 0x7fd36c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd36c102440 0x7fd36c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd3580b2900 0x7fd3580b4dc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 --2- 172.21.15.26:0/3207809801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36c102e00 0x7fd36c19fcc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 >> 172.21.15.26:0/3207809801 conn(0x7fd36c0793f0 msgr2=0x7fd36c06b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 shutdown_connections 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:46.292+0000 7fd373d42700 1 -- 172.21.15.26:0/3207809801 wait complete. 2024-05-20T20:59:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T20:59:46.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 64281 3.26 MB/sec warmup 112 sec latency 22.350 ms 2024-05-20T20:59:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:47 smithi055 ceph-mon[35213]: pgmap v498: 97 pgs: 97 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T20:59:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:47 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3207809801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:47 smithi026 ceph-mon[31990]: pgmap v498: 97 pgs: 97 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T20:59:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:47 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3207809801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:47 smithi144 ceph-mon[35660]: pgmap v498: 97 pgs: 97 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T20:59:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:47 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3207809801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T20:59:47.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 64843 3.24 MB/sec warmup 113 sec latency 18.587 ms 2024-05-20T20:59:47.570 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 143.1 --await-for 45.0' 2024-05-20T20:59:47.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 143.1 --await-for 45.0 2024-05-20T20:59:47.765 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T20:59:47.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T20:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:48.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 65493 3.25 MB/sec warmup 114 sec latency 22.690 ms 2024-05-20T20:59:49.123 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:49.336 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:49 smithi026 ceph-mon[31990]: pgmap v499: 97 pgs: 97 active+clean; 25 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-20T20:59:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:49 smithi055 ceph-mon[35213]: pgmap v499: 97 pgs: 97 active+clean; 25 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-20T20:59:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:49 smithi144 ceph-mon[35660]: pgmap v499: 97 pgs: 97 active+clean; 25 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-20T20:59:49.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66207 3.24 MB/sec warmup 115 sec latency 18.778 ms 2024-05-20T20:59:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 -- 172.21.15.26:0/191218785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 msgr2=0x7f705006b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 --2- 172.21.15.26:0/191218785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f705006b860 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f70400099c0 tx=0x7f7040009cd0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 -- 172.21.15.26:0/191218785 shutdown_connections 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 --2- 172.21.15.26:0/191218785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f705006b860 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 --2- 172.21.15.26:0/191218785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 0x7f7050110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 --2- 172.21.15.26:0/191218785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f705006be30 0x7f70501081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 -- 172.21.15.26:0/191218785 >> 172.21.15.26:0/191218785 conn(0x7f70500793e0 msgr2=0x7f70500797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 -- 172.21.15.26:0/191218785 shutdown_connections 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 -- 172.21.15.26:0/191218785 wait complete. 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.730+0000 7f70555c4700 1 Processor -- start 2024-05-20T20:59:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 -- start start 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f7050102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f705006be30 0x7f7050102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 0x7f7050102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7050113590 con 0x7f705006b480 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7050113410 con 0x7f7050108900 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f70555c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7050113710 con 0x7f705006be30 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f7050102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f7050102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58488/0 (socket says 172.21.15.26:58488) 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704effd700 1 -- 172.21.15.26:0/249185103 learned_addr learned my addr 172.21.15.26:0/249185103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704f7fe700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 0x7f7050102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704e7fc700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f705006be30 0x7f7050102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704f7fe700 1 -- 172.21.15.26:0/249185103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f705006be30 msgr2=0x7f7050102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704f7fe700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f705006be30 0x7f7050102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704f7fe700 1 -- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 msgr2=0x7f7050102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.731+0000 7f704f7fe700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f7050102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f704f7fe700 1 -- 172.21.15.26:0/249185103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70501a7f10 con 0x7f7050108900 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f704f7fe700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 0x7f7050102f90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f704800efc0 tx=0x7f704800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f7037fff700 1 -- 172.21.15.26:0/249185103 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70480092e0 con 0x7f7050108900 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f7037fff700 1 -- 172.21.15.26:0/249185103 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7048004bb0 con 0x7f7050108900 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70501a81a0 con 0x7f7050108900 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f7037fff700 1 -- 172.21.15.26:0/249185103 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048004020 con 0x7f7050108900 2024-05-20T20:59:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.732+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70501a86e0 con 0x7f7050108900 2024-05-20T20:59:49.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.735+0000 7f7037fff700 1 -- 172.21.15.26:0/249185103 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f704801b030 con 0x7f7050108900 2024-05-20T20:59:49.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.735+0000 7f7037fff700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f70380b2930 0x7f70380b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f704effd700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f70380b2930 0x7f70380b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f7037fff700 1 -- 172.21.15.26:0/249185103 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f70480e1c70 con 0x7f7050108900 2024-05-20T20:59:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f704effd700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f70380b2930 0x7f70380b4df0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7040009990 tx=0x7f7040005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 Processor -- start 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 -- start start 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859163b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x55585915f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915f640 0x55585916b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5558590bcf00 con 0x55585915ec80 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5558590acaa0 con 0x55585915f640 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.736+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5558590b0ae0 con 0x5558591637b0 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x55585915f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915f640 0x55585916b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915f640 0x55585916b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44202/0 (socket says 172.21.15.26:44202) 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x55585915f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58504/0 (socket says 172.21.15.26:58504) 2024-05-20T20:59:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704f7fe700 1 -- 172.21.15.26:0/1023662019 learned_addr learned my addr 172.21.15.26:0/1023662019 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704effd700 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859163b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 -- 172.21.15.26:0/1023662019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 msgr2=0x555859163b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859163b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 -- 172.21.15.26:0/1023662019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915f640 msgr2=0x55585916b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915f640 0x55585916b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 -- 172.21.15.26:0/1023662019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555859056140 con 0x55585915ec80 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f704e7fc700 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x55585915f100 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7050103fa0 tx=0x7f7044009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963c85e5e605cc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.737+0000 7f70357fa700 1 -- 172.21.15.26:0/1023662019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044011c80 con 0x55585915ec80 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f70357fa700 1 -- 172.21.15.26:0/1023662019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7044011e20 con 0x55585915ec80 2024-05-20T20:59:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f70357fa700 1 -- 172.21.15.26:0/1023662019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044005280 con 0x55585915ec80 2024-05-20T20:59:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 -- 172.21.15.26:0/1023662019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 msgr2=0x55585915f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x55585915f100 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7050103fa0 tx=0x7f7044009f60 comp rx=0 tx=0).stop 2024-05-20T20:59:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 -- 172.21.15.26:0/1023662019 shutdown_connections 2024-05-20T20:59:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915f640 0x55585916b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859163b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/1023662019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x55585915f100 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 -- 172.21.15.26:0/1023662019 >> 172.21.15.26:0/1023662019 conn(0x55585908a3d0 msgr2=0x555859159970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 -- 172.21.15.26:0/1023662019 shutdown_connections 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.738+0000 7f705fe5ab80 1 -- 172.21.15.26:0/1023662019 wait complete. 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 -- start start 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 Processor -- start 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 0x55585916b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x5558591685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859168b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555858fb3130 con 0x55585915f640 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555859014f30 con 0x55585915ec80 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55585916bd10 con 0x5558591637b0 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f705fe5ab80 10 client.0 _ll_register_callbacks cb 0x555859047ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f704effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 0x55585916b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f704f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859168b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f704effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 0x55585916b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44216/0 (socket says 172.21.15.26:44216) 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f704effd700 1 -- 172.21.15.26:0/898716026 learned_addr learned my addr 172.21.15.26:0/898716026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.739+0000 7f704e7fc700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x5558591685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704effd700 1 -- 172.21.15.26:0/898716026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 msgr2=0x555859168b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704effd700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859168b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704effd700 1 -- 172.21.15.26:0/898716026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 msgr2=0x5558591685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704effd700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x5558591685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704effd700 1 -- 172.21.15.26:0/898716026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555858ffca60 con 0x55585915ec80 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704e7fc700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x5558591685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f704effd700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 0x55585916b2a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7040005910 tx=0x7f7040009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc3042e65856e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f7032ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f7032ff5700 1 -- 172.21.15.26:0/898716026 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000bc20 con 0x55585915ec80 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f7032ff5700 1 -- 172.21.15.26:0/898716026 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f704000bdc0 con 0x55585915ec80 2024-05-20T20:59:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.740+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7044011e20 con 0x55585915ec80 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f7032ff5700 1 -- 172.21.15.26:0/898716026 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7040023460 con 0x55585915ec80 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 10 client.26084 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558590668f0 con 0x55585915ec80 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f7032ff5700 1 -- 172.21.15.26:0/898716026 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7044011e20 con 0x55585915ec80 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f7032ff5700 1 -- 172.21.15.26:0/898716026 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f7040038520 con 0x55585915ec80 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 10 client.26084 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 10 client.26084 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 20 client.26084 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591787f0 0x55585917ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 4 client.26084 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558590bee50 con 0x5558591787f0 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.741+0000 7f704e7fc700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591787f0 0x55585917ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.742+0000 7f704e7fc700 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591787f0 0x55585917ac60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37afbe560733c347 server_cookie=f79277c403a3babd in_seq=0 out_seq=0 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.742+0000 7f7032ff5700 10 client.26084 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f7032ff5700 1 -- 172.21.15.26:0/898716026 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5558590bee50 con 0x5558591787f0 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f7032ff5700 10 client.26084 handle_command_reply: tid=0 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 client.26084 shutdown 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 msgr2=0x55585916b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 0x55585916b2a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7040005910 tx=0x7f7040009cd0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 shutdown_connections 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591787f0 0x55585917ac60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558591637b0 0x555859168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55585915ec80 0x55585916b2a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/898716026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x5558591685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f7032ff5700 0 client.26084 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 >> 172.21.15.26:0/898716026 conn(0x55585908a3d0 msgr2=0x55585915dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 shutdown_connections 2024-05-20T20:59:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 1 -- 172.21.15.26:0/898716026 wait complete. 2024-05-20T20:59:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.743+0000 7f705fe5ab80 20 client.26084 trim_cache size 0 max 16384 2024-05-20T20:59:49.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.766+0000 7f705fe5ab80 1 Processor -- start 2024-05-20T20:59:49.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.766+0000 7f705fe5ab80 1 -- start start 2024-05-20T20:59:49.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x555859153f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915f640 0x555859167770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x555859180530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b26a0 con 0x55585915ec80 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f70501aff30 con 0x5558591637b0 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70501add80 con 0x55585915f640 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x555859153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x555859153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58528/0 (socket says 172.21.15.26:58528) 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x555859180530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 -- 172.21.15.26:0/2358651320 learned_addr learned my addr 172.21.15.26:0/2358651320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704e7fc700 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915f640 0x555859167770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 -- 172.21.15.26:0/2358651320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915f640 msgr2=0x555859167770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915f640 0x555859167770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 -- 172.21.15.26:0/2358651320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 msgr2=0x555859180530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x555859180530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704effd700 1 -- 172.21.15.26:0/2358651320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555859168eb0 con 0x55585915ec80 2024-05-20T20:59:49.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.767+0000 7f704f7fe700 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x555859180530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:49.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.768+0000 7f704effd700 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x555859153f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7040006010 tx=0x7f7040009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a4f6bf8bf6ad14 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.768+0000 7f7034ff9700 1 -- 172.21.15.26:0/2358651320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040046070 con 0x55585915ec80 2024-05-20T20:59:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.768+0000 7f7034ff9700 1 -- 172.21.15.26:0/2358651320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7040034070 con 0x55585915ec80 2024-05-20T20:59:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.768+0000 7f7034ff9700 1 -- 172.21.15.26:0/2358651320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040023980 con 0x55585915ec80 2024-05-20T20:59:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- 172.21.15.26:0/2358651320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 msgr2=0x555859153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x555859153f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7040006010 tx=0x7f7040009330 comp rx=0 tx=0).stop 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- 172.21.15.26:0/2358651320 shutdown_connections 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915ec80 0x555859153f00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x555859180530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/2358651320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915f640 0x555859167770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- 172.21.15.26:0/2358651320 >> 172.21.15.26:0/2358651320 conn(0x55585908a3d0 msgr2=0x555859162dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- 172.21.15.26:0/2358651320 shutdown_connections 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- 172.21.15.26:0/2358651320 wait complete. 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- start start 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 Processor -- start 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 0x55585916d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x55585916d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x5558591e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70501ae3f0 con 0x55585915f640 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f70501af860 con 0x5558591637b0 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.769+0000 7f705fe5ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70501aeac0 con 0x55585915ec80 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f705fe5ab80 10 client.0 _ll_register_callbacks cb 0x555859047ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x5558591e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 0x55585916d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x5558591e2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44232/0 (socket says 172.21.15.26:44232) 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 0x55585916d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48322/0 (socket says 172.21.15.26:48322) 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 -- 172.21.15.26:0/558905357 learned_addr learned my addr 172.21.15.26:0/558905357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704e7fc700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x55585916d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 -- 172.21.15.26:0/558905357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 msgr2=0x55585916d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 0x55585916d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 -- 172.21.15.26:0/558905357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 msgr2=0x55585916d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x55585916d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 -- 172.21.15.26:0/558905357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558591e2940 con 0x5558591637b0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704e7fc700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x55585916d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704effd700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 0x55585916d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f704f7fe700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x5558591e2400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70480a6530 tx=0x7f70480dfeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a2b46a455b0b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f7032ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f7032ff5700 1 -- 172.21.15.26:0/558905357 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704806d060 con 0x5558591637b0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f7032ff5700 1 -- 172.21.15.26:0/558905357 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f70480a8730 con 0x5558591637b0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.770+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7040034070 con 0x5558591637b0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f7032ff5700 1 -- 172.21.15.26:0/558905357 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70480a8dd0 con 0x5558591637b0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 10 client.26090 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558591e3b40 con 0x5558591637b0 2024-05-20T20:59:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f7032ff5700 1 -- 172.21.15.26:0/558905357 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7040034070 con 0x5558591637b0 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f7032ff5700 1 -- 172.21.15.26:0/558905357 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f70480ad0c0 con 0x5558591637b0 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 10 client.26090 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 10 client.26090 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 20 client.26090 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591e42c0 0x5558591e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 4 client.26090 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.1, "await_for": 45.0}] 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.1, "await_for": 45.0}) -- 0x5558590b8630 con 0x5558591e42c0 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.771+0000 7f704effd700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591e42c0 0x5558591e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.772+0000 7f7032ff5700 10 client.26090 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.772+0000 7f704effd700 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591e42c0 0x5558591e66e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e5ca0820d0e296 server_cookie=afbe87584c582a8b in_seq=0 out_seq=0 2024-05-20T20:59:49.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f7032ff5700 1 -- 172.21.15.26:0/558905357 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5558590b8630 con 0x5558591e42c0 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f7032ff5700 10 client.26090 handle_command_reply: tid=0 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 client.26090 shutdown 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 msgr2=0x5558591e2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x5558591e2400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70480a6530 tx=0x7f70480dfeb0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 shutdown_connections 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5558591e42c0 0x5558591e66e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5558591637b0 0x5558591e2400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55585915ec80 0x55585916d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 --2- 172.21.15.26:0/558905357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55585915f640 0x55585916d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f7032ff5700 0 client.26090 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 >> 172.21.15.26:0/558905357 conn(0x55585908a3d0 msgr2=0x555859167e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 shutdown_connections 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 1 -- 172.21.15.26:0/558905357 wait complete. 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.904+0000 7f705fe5ab80 20 client.26090 trim_cache size 0 max 16384 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 28, 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 14, 2024-05-20T20:59:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "be68fc79": { 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 14, 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 143.1, 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T20:59:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:49.908 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f70380b2930 msgr2=0x7f70380b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f70380b2930 0x7f70380b4df0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7040009990 tx=0x7f7040005960 comp rx=0 tx=0).stop 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 msgr2=0x7f7050102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 0x7f7050102f90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f704800efc0 tx=0x7f704800c3a0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 shutdown_connections 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f70380b2930 0x7f70380b4df0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705006b480 0x7f7050102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7050108900 0x7f7050102f90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 --2- 172.21.15.26:0/249185103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f705006be30 0x7f7050102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 >> 172.21.15.26:0/249185103 conn(0x7f70500793e0 msgr2=0x7f7050100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 shutdown_connections 2024-05-20T20:59:49.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:49.905+0000 7f70555c4700 1 -- 172.21.15.26:0/249185103 wait complete. 2024-05-20T20:59:50.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 3.21 MB/sec warmup 116 sec latency 631.895 ms 2024-05-20T20:59:51.368 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:51 smithi026 ceph-mon[31990]: pgmap v500: 97 pgs: 97 active+clean; 25 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-05-20T20:59:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:51 smithi055 ceph-mon[35213]: pgmap v500: 97 pgs: 97 active+clean; 25 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-05-20T20:59:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:51 smithi144 ceph-mon[35660]: pgmap v500: 97 pgs: 97 active+clean; 25 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-05-20T20:59:51.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 3.18 MB/sec warmup 117 sec latency 1631.945 ms 2024-05-20T20:59:51.808 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'be68fc79', quiesce took 9.8 seconds. Will release after: 13.3 2024-05-20T20:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 -- 172.21.15.26:0/1549405306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c109630 msgr2=0x7f796c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 --2- 172.21.15.26:0/1549405306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c109630 0x7f796c109a10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f79600099c0 tx=0x7f7960009cd0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 -- 172.21.15.26:0/1549405306 shutdown_connections 2024-05-20T20:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 --2- 172.21.15.26:0/1549405306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c109630 0x7f796c109a10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 --2- 172.21.15.26:0/1549405306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c06c6d0 0x7f796c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 --2- 172.21.15.26:0/1549405306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 0x7f796c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 -- 172.21.15.26:0/1549405306 >> 172.21.15.26:0/1549405306 conn(0x7f796c0794e0 msgr2=0x7f796c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 -- 172.21.15.26:0/1549405306 shutdown_connections 2024-05-20T20:59:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 -- 172.21.15.26:0/1549405306 wait complete. 2024-05-20T20:59:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 Processor -- start 2024-05-20T20:59:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.255+0000 7f7972f70700 1 -- start start 2024-05-20T20:59:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7972f70700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 0x7f796c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7972f70700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 0x7f796c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7972f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c109630 0x7f796c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7972f70700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f796c113600 con 0x7f796c06c6d0 2024-05-20T20:59:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7972f70700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f796c113480 con 0x7f796c06bd10 2024-05-20T20:59:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7972f70700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796c113780 con 0x7f796c109630 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 0x7f796c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 0x7f796c19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58560/0 (socket says 172.21.15.26:58560) 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 -- 172.21.15.26:0/4020840018 learned_addr learned my addr 172.21.15.26:0/4020840018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f797150d700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c109630 0x7f796c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7970d0c700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 0x7f796c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 -- 172.21.15.26:0/4020840018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c109630 msgr2=0x7f796c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c109630 0x7f796c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 -- 172.21.15.26:0/4020840018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 msgr2=0x7f796c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 0x7f796c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 -- 172.21.15.26:0/4020840018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c19a060 con 0x7f796c06c6d0 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7970d0c700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 0x7f796c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f796bfff700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 0x7f796c19fc30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f795800d950 tx=0x7f795800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.256+0000 7f7969ffb700 1 -- 172.21.15.26:0/4020840018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958016070 con 0x7f796c06c6d0 2024-05-20T20:59:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.257+0000 7f7969ffb700 1 -- 172.21.15.26:0/4020840018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7958009dd0 con 0x7f796c06c6d0 2024-05-20T20:59:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.257+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c19a350 con 0x7f796c06c6d0 2024-05-20T20:59:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.257+0000 7f7969ffb700 1 -- 172.21.15.26:0/4020840018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795800f3c0 con 0x7f796c06c6d0 2024-05-20T20:59:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.257+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c1a65c0 con 0x7f796c06c6d0 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.259+0000 7f7969ffb700 1 -- 172.21.15.26:0/4020840018 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7958019070 con 0x7f796c06c6d0 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.260+0000 7f7969ffb700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f795c0b2880 0x7f795c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.260+0000 7f7970d0c700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f795c0b2880 0x7f795c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.260+0000 7f7969ffb700 1 -- 172.21.15.26:0/4020840018 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f79580e1ee0 con 0x7f796c06c6d0 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 Processor -- start 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 -- start start 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f795c0b2880 0x7f795c0b4d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7960009990 tx=0x7f796001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f32420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5650e3e8cd10 con 0x5650e3f2d8e0 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5650e3e7c8b0 con 0x5650e3f31fe0 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797d806b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650e3e808f0 con 0x5650e3f2e290 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f2dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58572/0 (socket says 172.21.15.26:58572) 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 -- 172.21.15.26:0/149911221 learned_addr learned my addr 172.21.15.26:0/149911221 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f797150d700 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f32420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f796bfff700 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 -- 172.21.15.26:0/149911221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 msgr2=0x5650e3f319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 -- 172.21.15.26:0/149911221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 msgr2=0x5650e3f32420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f32420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f7970d0c700 1 -- 172.21.15.26:0/149911221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e3dcc950 con 0x5650e3f2d8e0 2024-05-20T20:59:52.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.261+0000 7f796bfff700 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:52.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.262+0000 7f7970d0c700 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f2dcc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f79600099c0 tx=0x7f796001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d934a06cb765132 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:52.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.262+0000 7f7956ffd700 1 -- 172.21.15.26:0/149911221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796000bc30 con 0x5650e3f2d8e0 2024-05-20T20:59:52.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.262+0000 7f7956ffd700 1 -- 172.21.15.26:0/149911221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f796000bdd0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.262+0000 7f7956ffd700 1 -- 172.21.15.26:0/149911221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796002f7e0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 -- 172.21.15.26:0/149911221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 msgr2=0x5650e3f2dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f2dcc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f79600099c0 tx=0x7f796001e000 comp rx=0 tx=0).stop 2024-05-20T20:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 -- 172.21.15.26:0/149911221 shutdown_connections 2024-05-20T20:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f2dcc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f32420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 --2- 172.21.15.26:0/149911221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 -- 172.21.15.26:0/149911221 >> 172.21.15.26:0/149911221 conn(0x5650e3e5b100 msgr2=0x5650e3f27f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 -- 172.21.15.26:0/149911221 shutdown_connections 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 -- 172.21.15.26:0/149911221 wait complete. 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 -- start start 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.263+0000 7f797d806b80 1 Processor -- start 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f36d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5650e3d83020 con 0x5650e3f2d8e0 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5650e3e1e840 con 0x5650e3f31fe0 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650e3f3a140 con 0x5650e3f2e290 2024-05-20T20:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797d806b80 10 client.0 _ll_register_callbacks cb 0x5650e3e179a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f394f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58574/0 (socket says 172.21.15.26:58574) 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 -- 172.21.15.26:0/3578414753 learned_addr learned my addr 172.21.15.26:0/3578414753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f796bfff700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797150d700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f36d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 -- 172.21.15.26:0/3578414753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 msgr2=0x5650e3f367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 -- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 msgr2=0x5650e3f36d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f36d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 -- 172.21.15.26:0/3578414753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e3e1b710 con 0x5650e3f2d8e0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f797150d700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f7970d0c700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f394f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f79600052a0 tx=0x7f796000bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14fe0b2e115f3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f794bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f794bfff700 1 -- 172.21.15.26:0/3578414753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796003c060 con 0x5650e3f2d8e0 2024-05-20T20:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.264+0000 7f794bfff700 1 -- 172.21.15.26:0/3578414753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f796002b070 con 0x5650e3f2d8e0 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f796000bdd0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f794bfff700 1 -- 172.21.15.26:0/3578414753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f796002f7b0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 10 client.16563 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650e3f448e0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f794bfff700 1 -- 172.21.15.26:0/3578414753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f796000bdd0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f794bfff700 1 -- 172.21.15.26:0/3578414753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f7960040030 con 0x5650e3f2d8e0 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 10 client.16563 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 20 client.16563 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3f46db0 0x5650e3f491a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 4 client.16563 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.265+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e3ef9a60 con 0x5650e3f46db0 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.266+0000 7f796bfff700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3f46db0 0x5650e3f491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.266+0000 7f794bfff700 10 client.16563 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.266+0000 7f796bfff700 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3f46db0 0x5650e3f491a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167a584f1a8e8e4 server_cookie=b35d78a12dc206b6 in_seq=0 out_seq=0 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f794bfff700 1 -- 172.21.15.26:0/3578414753 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5650e3ef9a60 con 0x5650e3f46db0 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f794bfff700 10 client.16563 handle_command_reply: tid=0 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 client.16563 shutdown 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 msgr2=0x5650e3f394f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f394f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f79600052a0 tx=0x7f796000bc30 comp rx=0 tx=0).stop 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 shutdown_connections 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3f46db0 0x5650e3f491a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3f394f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f794bfff700 0 client.16563 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 --2- 172.21.15.26:0/3578414753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2e290 0x5650e3f367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 >> 172.21.15.26:0/3578414753 conn(0x5650e3e5b100 msgr2=0x5650e3f27590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 shutdown_connections 2024-05-20T20:59:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 1 -- 172.21.15.26:0/3578414753 wait complete. 2024-05-20T20:59:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.267+0000 7f797d806b80 20 client.16563 trim_cache size 0 max 16384 2024-05-20T20:59:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.288+0000 7f797d806b80 1 Processor -- start 2024-05-20T20:59:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.288+0000 7f797d806b80 1 -- start start 2024-05-20T20:59:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.288+0000 7f797d806b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2d8e0 0x5650e3f311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.288+0000 7f797d806b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 0x5650e3f27d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.288+0000 7f797d806b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f28250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f797d806b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1b04d0 con 0x5650e3f2e290 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f797d806b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1add60 con 0x5650e3f31fe0 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f797d806b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1abbb0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 0x5650e3f27d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 0x5650e3f27d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58578/0 (socket says 172.21.15.26:58578) 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 -- 172.21.15.26:0/1325106794 learned_addr learned my addr 172.21.15.26:0/1325106794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f7970d0c700 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2d8e0 0x5650e3f311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f797150d700 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f28250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 -- 172.21.15.26:0/1325106794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2d8e0 msgr2=0x5650e3f311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2d8e0 0x5650e3f311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 -- 172.21.15.26:0/1325106794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 msgr2=0x5650e3f28250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f28250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 -- 172.21.15.26:0/1325106794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e3f37770 con 0x5650e3f2e290 2024-05-20T20:59:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f796bfff700 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 0x5650e3f27d10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f795800b2f0 tx=0x7f79580df850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8950db54119dfc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f79567fc700 1 -- 172.21.15.26:0/1325106794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958020040 con 0x5650e3f2e290 2024-05-20T20:59:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f79567fc700 1 -- 172.21.15.26:0/1325106794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79580b2300 con 0x5650e3f2e290 2024-05-20T20:59:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.289+0000 7f79567fc700 1 -- 172.21.15.26:0/1325106794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580b0e30 con 0x5650e3f2e290 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 -- 172.21.15.26:0/1325106794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 msgr2=0x5650e3f27d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 0x5650e3f27d10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f795800b2f0 tx=0x7f79580df850 comp rx=0 tx=0).stop 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 -- 172.21.15.26:0/1325106794 shutdown_connections 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f31fe0 0x5650e3f28250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f2d8e0 0x5650e3f311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 --2- 172.21.15.26:0/1325106794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2e290 0x5650e3f27d10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 -- 172.21.15.26:0/1325106794 >> 172.21.15.26:0/1325106794 conn(0x5650e3e5b100 msgr2=0x5650e3f35890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 -- 172.21.15.26:0/1325106794 shutdown_connections 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.290+0000 7f797d806b80 1 -- 172.21.15.26:0/1325106794 wait complete. 2024-05-20T20:59:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 -- start start 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 Processor -- start 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3fa5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f2e290 0x5650e3fa5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f31fe0 0x5650e3fa9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1ac220 con 0x5650e3f2d8e0 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1ad690 con 0x5650e3f2e290 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1ac8f0 con 0x5650e3f31fe0 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f797d806b80 10 client.0 _ll_register_callbacks cb 0x5650e3e179a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.291+0000 7f7970d0c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3fa5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f796bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f2e290 0x5650e3fa5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3fa5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58584/0 (socket says 172.21.15.26:58584) 2024-05-20T20:59:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 -- 172.21.15.26:0/1051334106 learned_addr learned my addr 172.21.15.26:0/1051334106 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f797150d700 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f31fe0 0x5650e3fa9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 -- 172.21.15.26:0/1051334106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f31fe0 msgr2=0x5650e3fa9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f31fe0 0x5650e3fa9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 -- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f2e290 msgr2=0x5650e3fa5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f2e290 0x5650e3fa5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 -- 172.21.15.26:0/1051334106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e3fa9f70 con 0x5650e3f2d8e0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f7970d0c700 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3fa5760 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7960009350 tx=0x7f7960009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6422d104c33e489 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f794bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f794bfff700 1 -- 172.21.15.26:0/1051334106 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796003c070 con 0x5650e3f2d8e0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f794bfff700 1 -- 172.21.15.26:0/1051334106 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f7960034040 con 0x5650e3f2d8e0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.292+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79580b2300 con 0x5650e3f2d8e0 2024-05-20T20:59:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f794bfff700 1 -- 172.21.15.26:0/1051334106 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f796000fdb0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 10 client.16569 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650e3fb24d0 con 0x5650e3f2d8e0 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f794bfff700 1 -- 172.21.15.26:0/1051334106 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f796000f460 con 0x5650e3f2d8e0 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f794bfff700 1 -- 172.21.15.26:0/1051334106 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f7960040070 con 0x5650e3f2d8e0 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 10 client.16569 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 20 client.16569 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3fb2930 0x5650e3fb4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 4 client.16569 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T20:59:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f79580b2300 con 0x5650e3fb2930 2024-05-20T20:59:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.293+0000 7f796bfff700 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3fb2930 0x5650e3fb4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.294+0000 7f794bfff700 10 client.16569 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.294+0000 7f796bfff700 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3fb2930 0x5650e3fb4d20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10415573219cde91 server_cookie=3cc49c302cc2b927 in_seq=0 out_seq=0 2024-05-20T20:59:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f794bfff700 1 -- 172.21.15.26:0/1051334106 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f79580b2300 con 0x5650e3fb2930 2024-05-20T20:59:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f794bfff700 10 client.16569 handle_command_reply: tid=0 2024-05-20T20:59:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 client.16569 shutdown 2024-05-20T20:59:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 msgr2=0x5650e3fa5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3fa5760 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7960009350 tx=0x7f7960009fd0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 shutdown_connections 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5650e3fb2930 0x5650e3fb4d20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650e3f2d8e0 0x5650e3fa5760 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e3f31fe0 0x5650e3fa9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 --2- 172.21.15.26:0/1051334106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650e3f2e290 0x5650e3fa5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f794bfff700 0 client.16569 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.295+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 >> 172.21.15.26:0/1051334106 conn(0x5650e3e5b100 msgr2=0x5650e3f311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 shutdown_connections 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f797d806b80 1 -- 172.21.15.26:0/1051334106 wait complete. 2024-05-20T20:59:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f797d806b80 20 client.16569 trim_cache size 0 max 16384 2024-05-20T20:59:52.301 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T20:59:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "2d88efa8-3a5b-4781-ba40-cec2b54b0182", 2024-05-20T20:59:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T20:59:52.301 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f795c0b2880 msgr2=0x7f795c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f7972f70700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f795c0b2880 0x7f795c0b4d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7960009990 tx=0x7f796001a040 comp rx=0 tx=0).stop 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 msgr2=0x7f796c19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.296+0000 7f7972f70700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 0x7f796c19fc30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f795800d950 tx=0x7f795800dc60 comp rx=0 tx=0).stop 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 shutdown_connections 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c109630 0x7f796c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f796c06bd10 0x7f796c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f795c0b2880 0x7f795c0b4d40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 --2- 172.21.15.26:0/4020840018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f796c06c6d0 0x7f796c19fc30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 >> 172.21.15.26:0/4020840018 conn(0x7f796c0794e0 msgr2=0x7f796c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 shutdown_connections 2024-05-20T20:59:52.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:52.297+0000 7f7972f70700 1 -- 172.21.15.26:0/4020840018 wait complete. 2024-05-20T20:59:52.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 3.16 MB/sec warmup 118 sec latency 2632.038 ms 2024-05-20T20:59:52.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T20:59:53.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:53 smithi026 ceph-mon[31990]: pgmap v501: 97 pgs: 97 active+clean; 24 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-05-20T20:59:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T20:59:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:53 smithi055 ceph-mon[35213]: pgmap v501: 97 pgs: 97 active+clean; 24 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-05-20T20:59:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:53 smithi144 ceph-mon[35660]: pgmap v501: 97 pgs: 97 active+clean; 24 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-05-20T20:59:53.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 3.13 MB/sec warmup 119 sec latency 3632.139 ms 2024-05-20T20:59:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:54 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T20:59:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:54 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:54 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T20:59:54.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:54 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:54.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:54 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T20:59:54.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:54 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:54.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:54 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T20:59:54.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:54 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:54 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T20:59:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:54 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:54 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T20:59:54.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:54 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 -- 172.21.15.26:0/3636757417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 msgr2=0x7f041c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 --2- 172.21.15.26:0/3636757417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c10ac20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0414009ca0 tx=0x7f041402a8c0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 -- 172.21.15.26:0/3636757417 shutdown_connections 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 --2- 172.21.15.26:0/3636757417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c10ac20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 --2- 172.21.15.26:0/3636757417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c105f00 0x7f041c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 --2- 172.21.15.26:0/3636757417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1035d0 0x7f041c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.856+0000 7f042431c640 1 -- 172.21.15.26:0/3636757417 >> 172.21.15.26:0/3636757417 conn(0x7f041c0fcf20 msgr2=0x7f041c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.857+0000 7f042431c640 1 -- 172.21.15.26:0/3636757417 shutdown_connections 2024-05-20T20:59:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.857+0000 7f042431c640 1 -- 172.21.15.26:0/3636757417 wait complete. 2024-05-20T20:59:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.857+0000 7f042431c640 1 Processor -- start 2024-05-20T20:59:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 -- start start 2024-05-20T20:59:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c1035d0 0x7f041c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c105f00 0x7f041c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10e4e0 con 0x7f041c108830 2024-05-20T20:59:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10e360 con 0x7f041c105f00 2024-05-20T20:59:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f042431c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10e660 con 0x7f041c1035d0 2024-05-20T20:59:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c1a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58594/0 (socket says 172.21.15.26:58594) 2024-05-20T20:59:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 -- 172.21.15.26:0/1134353605 learned_addr learned my addr 172.21.15.26:0/1134353605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04220b8640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c1035d0 0x7f041c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04218b7640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c105f00 0x7f041c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 -- 172.21.15.26:0/1134353605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c1035d0 msgr2=0x7f041c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c1035d0 0x7f041c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 -- 172.21.15.26:0/1134353605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c105f00 msgr2=0x7f041c19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c105f00 0x7f041c19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 -- 172.21.15.26:0/1134353605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a2970 con 0x7f041c108830 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04218b7640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c105f00 0x7f041c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.858+0000 7f04228b9640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c1a2300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f041402aeb0 tx=0x7f0414003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.859+0000 7f04137fe640 1 -- 172.21.15.26:0/1134353605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414003e10 con 0x7f041c108830 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.859+0000 7f04137fe640 1 -- 172.21.15.26:0/1134353605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04140044d0 con 0x7f041c108830 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.859+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a2c00 con 0x7f041c108830 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.859+0000 7f04137fe640 1 -- 172.21.15.26:0/1134353605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414030c90 con 0x7f041c108830 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.859+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a3150 con 0x7f041c108830 2024-05-20T20:59:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.862+0000 7f0427031740 1 Processor -- start 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.862+0000 7f04137fe640 1 -- 172.21.15.26:0/1134353605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0414004680 con 0x7f041c108830 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.862+0000 7f0427031740 1 -- start start 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f0427031740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f0427031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee570b0 0x55c61ee5bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f0427031740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee5c220 0x55c61ee5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f0427031740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c61edc7c30 con 0x55c61ee566c0 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f0427031740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c61edb77d0 con 0x55c61ee5c220 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f0427031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c61edbb810 con 0x55c61ee570b0 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee56ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58598/0 (socket says 172.21.15.26:58598) 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 -- 172.21.15.26:0/2667895674 learned_addr learned my addr 172.21.15.26:0/2667895674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04220b8640 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee570b0 0x55c61ee5bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 -- 172.21.15.26:0/2667895674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee570b0 msgr2=0x55c61ee5bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee570b0 0x55c61ee5bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 -- 172.21.15.26:0/2667895674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee5c220 msgr2=0x55c61ee5e6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee5c220 0x55c61ee5e6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 -- 172.21.15.26:0/2667895674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61eacc210 con 0x55c61ee566c0 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04220b8640 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee570b0 0x55c61ee5bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04137fe640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b28a0 0x7f04080b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04218b7640 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee56ae0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f040c00bcd0 tx=0x7f040c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d70075bb62748fa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.863+0000 7f04220b8640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b28a0 0x7f04080b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f04137fe640 1 -- 172.21.15.26:0/1134353605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f0414103880 con 0x7f041c108830 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f0410bfe640 1 -- 172.21.15.26:0/2667895674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c015070 con 0x55c61ee566c0 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f0410bfe640 1 -- 172.21.15.26:0/2667895674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c01a8f0 con 0x55c61ee566c0 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f0410bfe640 1 -- 172.21.15.26:0/2667895674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0118d0 con 0x55c61ee566c0 2024-05-20T20:59:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f0427031740 1 -- 172.21.15.26:0/2667895674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 msgr2=0x55c61ee56ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f04220b8640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b28a0 0x7f04080b4d60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55c61ee574b0 tx=0x7f0418005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.864+0000 7f0427031740 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee56ae0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f040c00bcd0 tx=0x7f040c00f730 comp rx=0 tx=0).stop 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- 172.21.15.26:0/2667895674 shutdown_connections 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee5c220 0x55c61ee5e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee570b0 0x55c61ee5bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 --2- 172.21.15.26:0/2667895674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee56ae0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- 172.21.15.26:0/2667895674 >> 172.21.15.26:0/2667895674 conn(0x55c61ee51d30 msgr2=0x55c61ee52130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- 172.21.15.26:0/2667895674 shutdown_connections 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- 172.21.15.26:0/2667895674 wait complete. 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- start start 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 Processor -- start 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee67e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c61ea9d790 con 0x55c61ee566c0 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c61ed73890 con 0x55c61ee570b0 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c61ee5f1d0 con 0x55c61ee5c220 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f0427031740 10 client.0 _ll_register_callbacks cb 0x55c61eacba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f04220b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f04220b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee60900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58610/0 (socket says 172.21.15.26:58610) 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.865+0000 7f04220b8640 1 -- 172.21.15.26:0/51210595 learned_addr learned my addr 172.21.15.26:0/51210595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04228b9640 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee67e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04220b8640 1 -- 172.21.15.26:0/51210595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 msgr2=0x55c61ee67e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04220b8640 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee67e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04220b8640 1 -- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 msgr2=0x55c61ee60e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04220b8640 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee60e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04220b8640 1 -- 172.21.15.26:0/51210595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61eb98620 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f04220b8640 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee60900 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0418011c10 tx=0x7f04180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d60b2a2795e30da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0405ffb640 1 -- 172.21.15.26:0/51210595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418010cb0 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0405ffb640 1 -- 172.21.15.26:0/51210595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f0418028400 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c01a8f0 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0405ffb640 1 -- 172.21.15.26:0/51210595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0418010e50 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0427031740 10 client.16581 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.866+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c61ee6ae40 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0405ffb640 1 -- 172.21.15.26:0/51210595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c01a8f0 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0405ffb640 1 -- 172.21.15.26:0/51210595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f0418031050 con 0x55c61ee566c0 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0427031740 10 client.16581 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0427031740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0427031740 20 client.16581 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0427031740 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61ee6c5a0 0x55c61ee6ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0427031740 4 client.16581 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61edc9b80 con 0x55c61ee6c5a0 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.867+0000 7f04218b7640 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61ee6c5a0 0x55c61ee6ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.868+0000 7f04218b7640 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61ee6c5a0 0x55c61ee6ea30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7742b4811ba7f2dd server_cookie=c0e6d03b5f49b869 in_seq=0 out_seq=0 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.868+0000 7f0405ffb640 10 client.16581 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.868+0000 7f0405ffb640 1 -- 172.21.15.26:0/51210595 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c61edc9b80 con 0x55c61ee6c5a0 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.868+0000 7f0405ffb640 10 client.16581 handle_command_reply: tid=0 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.868+0000 7f0427031740 1 client.16581 shutdown 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 msgr2=0x55c61ee60900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee60900 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0418011c10 tx=0x7f04180046c0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 shutdown_connections 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61ee6c5a0 0x55c61ee6ea30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee67e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee60e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0405ffb640 0 client.16581 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 --2- 172.21.15.26:0/51210595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee60900 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 >> 172.21.15.26:0/51210595 conn(0x55c61ee51d30 msgr2=0x55c61ee50fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 shutdown_connections 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 1 -- 172.21.15.26:0/51210595 wait complete. 2024-05-20T20:59:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.869+0000 7f0427031740 20 client.16581 trim_cache size 0 max 16384 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 Processor -- start 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 -- start start 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee566c0 0x55c61ee60c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee67e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 0x55c61ee54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ae940 con 0x55c61ee5c220 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ac210 con 0x55c61ee570b0 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f0427031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aa060 con 0x55c61ee566c0 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f04228b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 0x55c61ee54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f04228b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 0x55c61ee54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58618/0 (socket says 172.21.15.26:58618) 2024-05-20T20:59:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.884+0000 7f04228b9640 1 -- 172.21.15.26:0/3390277824 learned_addr learned my addr 172.21.15.26:0/3390277824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04220b8640 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee566c0 0x55c61ee60c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04228b9640 1 -- 172.21.15.26:0/3390277824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee566c0 msgr2=0x55c61ee60c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04228b9640 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee566c0 0x55c61ee60c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04218b7640 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee67e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04228b9640 1 -- 172.21.15.26:0/3390277824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 msgr2=0x55c61ee67e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04228b9640 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee67e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04228b9640 1 -- 172.21.15.26:0/3390277824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61ee56220 con 0x55c61ee5c220 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f04228b9640 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 0x55c61ee54c10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f04140cadd0 tx=0x7f04140ca850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f8c61b7d49ffac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f0407fff640 1 -- 172.21.15.26:0/3390277824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140cd070 con 0x55c61ee5c220 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f0407fff640 1 -- 172.21.15.26:0/3390277824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0414036040 con 0x55c61ee5c220 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f0407fff640 1 -- 172.21.15.26:0/3390277824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140c70d0 con 0x55c61ee5c220 2024-05-20T20:59:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f0427031740 1 -- 172.21.15.26:0/3390277824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 msgr2=0x55c61ee54c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.885+0000 7f0427031740 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 0x55c61ee54c10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f04140cadd0 tx=0x7f04140ca850 comp rx=0 tx=0).stop 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- 172.21.15.26:0/3390277824 shutdown_connections 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee5c220 0x55c61ee54c10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee67e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 --2- 172.21.15.26:0/3390277824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee566c0 0x55c61ee60c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- 172.21.15.26:0/3390277824 >> 172.21.15.26:0/3390277824 conn(0x55c61ee51d30 msgr2=0x55c61ee60510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- 172.21.15.26:0/3390277824 shutdown_connections 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- 172.21.15.26:0/3390277824 wait complete. 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- start start 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 Processor -- start 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee66a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aa6d0 con 0x55c61ee566c0 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1abb40 con 0x55c61ee570b0 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aada0 con 0x55c61ee5c220 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f0427031740 10 client.0 _ll_register_callbacks cb 0x7f0418009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f04220b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f04220b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58622/0 (socket says 172.21.15.26:58622) 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.886+0000 7f04220b8640 1 -- 172.21.15.26:0/2196844653 learned_addr learned my addr 172.21.15.26:0/2196844653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04218b7640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04228b9640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee66a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04220b8640 1 -- 172.21.15.26:0/2196844653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 msgr2=0x55c61ee66a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04220b8640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee66a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04220b8640 1 -- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 msgr2=0x55c61ee659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04220b8640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04220b8640 1 -- 172.21.15.26:0/2196844653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61ee66ff0 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f04220b8640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee654b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0418032760 tx=0x7f0418032620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc396069c4f5128 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.887+0000 7f0405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0405ffb640 1 -- 172.21.15.26:0/2196844653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418028600 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0405ffb640 1 -- 172.21.15.26:0/2196844653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f04180287a0 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0414036040 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0405ffb640 1 -- 172.21.15.26:0/2196844653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0418043050 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 10 client.16587 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c61eed71d0 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0405ffb640 1 -- 172.21.15.26:0/2196844653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0414036040 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0405ffb640 1 -- 172.21.15.26:0/2196844653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f0418031070 con 0x55c61ee566c0 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 10 client.16587 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 20 client.16587 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61eed7ba0 0x55c61eed9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 4 client.16587 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c61edc3360 con 0x55c61eed7ba0 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.888+0000 7f04218b7640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61eed7ba0 0x55c61eed9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.889+0000 7f04218b7640 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61eed7ba0 0x55c61eed9fc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf10741b4644503 server_cookie=471370e97e676f80 in_seq=0 out_seq=0 2024-05-20T20:59:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.889+0000 7f0405ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0405ffb640 1 -- 172.21.15.26:0/2196844653 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c61edc3360 con 0x55c61eed7ba0 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0405ffb640 10 client.16587 handle_command_reply: tid=0 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 client.16587 shutdown 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 msgr2=0x55c61ee654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee654b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0418032760 tx=0x7f0418032620 comp rx=0 tx=0).stop 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 shutdown_connections 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c61eed7ba0 0x55c61eed9fc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61ee5c220 0x55c61ee66a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0427031740 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c61ee570b0 0x55c61ee659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.916+0000 7f0405ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f0427031740 1 --2- 172.21.15.26:0/2196844653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c61ee566c0 0x55c61ee654b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 >> 172.21.15.26:0/2196844653 conn(0x55c61ee51d30 msgr2=0x55c61ee60d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 shutdown_connections 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f0427031740 1 -- 172.21.15.26:0/2196844653 wait complete. 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f0427031740 20 client.16587 trim_cache size 0 max 16384 2024-05-20T20:59:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T20:59:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T20:59:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b28a0 msgr2=0x7f04080b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b28a0 0x7f04080b4d60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55c61ee574b0 tx=0x7f0418005f90 comp rx=0 tx=0).stop 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 msgr2=0x7f041c1a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c1a2300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f041402aeb0 tx=0x7f0414003c20 comp rx=0 tx=0).stop 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 shutdown_connections 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b28a0 0x7f04080b4d60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c108830 0x7f041c1a2300 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c105f00 0x7f041c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 --2- 172.21.15.26:0/1134353605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c1035d0 0x7f041c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 >> 172.21.15.26:0/1134353605 conn(0x7f041c0fcf20 msgr2=0x7f041c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 shutdown_connections 2024-05-20T20:59:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T20:59:54.917+0000 7f042431c640 1 -- 172.21.15.26:0/1134353605 wait complete. 2024-05-20T20:59:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 -- 172.21.15.26:0/2682115339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901088f0 msgr2=0x7fd690110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 --2- 172.21.15.26:0/2682115339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901088f0 0x7fd690110470 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd68800b0d0 tx=0x7fd68800b3e0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 -- 172.21.15.26:0/2682115339 shutdown_connections 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 --2- 172.21.15.26:0/2682115339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901088f0 0x7fd690110470 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 --2- 172.21.15.26:0/2682115339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6901037f0 0x7fd690103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 --2- 172.21.15.26:0/2682115339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 0x7fd6901081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 -- 172.21.15.26:0/2682115339 >> 172.21.15.26:0/2682115339 conn(0x7fd69007a6a0 msgr2=0x7fd69007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 -- 172.21.15.26:0/2682115339 shutdown_connections 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 -- 172.21.15.26:0/2682115339 wait complete. 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 Processor -- start 2024-05-20T20:59:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.047+0000 7fd6986dc700 1 -- start start 2024-05-20T20:59:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6986dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 0x7fd69019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6986dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 0x7fd69019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6986dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6901088f0 0x7fd690199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6986dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd690113590 con 0x7fd6901037f0 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6986dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd690113410 con 0x7fd6901088f0 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6986dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd690113710 con 0x7fd6901041a0 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 0x7fd69019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 0x7fd69019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58652/0 (socket says 172.21.15.26:58652) 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 -- 172.21.15.26:0/1240552398 learned_addr learned my addr 172.21.15.26:0/1240552398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696c79700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6901088f0 0x7fd690199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd695c77700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 0x7fd69019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 -- 172.21.15.26:0/1240552398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 msgr2=0x7fd69019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 0x7fd69019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 -- 172.21.15.26:0/1240552398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6901088f0 msgr2=0x7fd690199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6901088f0 0x7fd690199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 -- 172.21.15.26:0/1240552398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69019a030 con 0x7fd6901037f0 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd695c77700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 0x7fd69019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd696478700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 0x7fd69019f6d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00d980 tx=0x7fd68c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.048+0000 7fd6877fe700 1 -- 172.21.15.26:0/1240552398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c016070 con 0x7fd6901037f0 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.049+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69019a320 con 0x7fd6901037f0 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.049+0000 7fd6877fe700 1 -- 172.21.15.26:0/1240552398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd68c004830 con 0x7fd6901037f0 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.049+0000 7fd6877fe700 1 -- 172.21.15.26:0/1240552398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c005440 con 0x7fd6901037f0 2024-05-20T20:59:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.049+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6901a6530 con 0x7fd6901037f0 2024-05-20T20:59:55.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.052+0000 7fd6877fe700 1 -- 172.21.15.26:0/1240552398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd68c0055e0 con 0x7fd6901037f0 2024-05-20T20:59:55.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.053+0000 7fd6877fe700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd67c0b2880 0x7fd67c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.053+0000 7fd695c77700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd67c0b2880 0x7fd67c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.053+0000 7fd6a2f72b80 1 Processor -- start 2024-05-20T20:59:55.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.053+0000 7fd6877fe700 1 -- 172.21.15.26:0/1240552398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd68c0e2120 con 0x7fd6901037f0 2024-05-20T20:59:55.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.053+0000 7fd6a2f72b80 1 -- start start 2024-05-20T20:59:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd67c0b2880 0x7fd67c0b4d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd680005fd0 tx=0x7fd680005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55647b91cd70 con 0x55647b9c8fc0 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55647b90c910 con 0x55647b9c8610 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55647b910950 con 0x55647b9c32d0 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58668/0 (socket says 172.21.15.26:58668) 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 -- 172.21.15.26:0/2923239575 learned_addr learned my addr 172.21.15.26:0/2923239575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd696c79700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd696478700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 -- 172.21.15.26:0/2923239575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 msgr2=0x55647b9c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9c3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 -- 172.21.15.26:0/2923239575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 msgr2=0x55647b9c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 -- 172.21.15.26:0/2923239575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647b8b5fb0 con 0x55647b9c8fc0 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd696c79700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd696478700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd695c77700 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9c2cb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd68000ec70 tx=0x7fd680011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7244647d8733023b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.054+0000 7fd684ff9700 1 -- 172.21.15.26:0/2923239575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680021070 con 0x55647b9c8fc0 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd684ff9700 1 -- 172.21.15.26:0/2923239575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd680010cd0 con 0x55647b9c8fc0 2024-05-20T20:59:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd684ff9700 1 -- 172.21.15.26:0/2923239575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680005690 con 0x55647b9c8fc0 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2923239575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 msgr2=0x55647b9c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9c2cb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd68000ec70 tx=0x7fd680011000 comp rx=0 tx=0).stop 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2923239575 shutdown_connections 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2923239575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9c2cb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2923239575 >> 172.21.15.26:0/2923239575 conn(0x55647b8ea240 msgr2=0x55647b9b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.055+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2923239575 shutdown_connections 2024-05-20T20:59:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2923239575 wait complete. 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 -- start start 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 Processor -- start 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 0x55647b9cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8fc0 0x55647b9cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55647b812fa0 con 0x55647b9c8610 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55647b874da0 con 0x55647b9c8fc0 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55647b9cfaa0 con 0x55647b9c32d0 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd6a2f72b80 10 client.0 _ll_register_callbacks cb 0x55647b8a7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd695c77700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 0x55647b9cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.056+0000 7fd695c77700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 0x55647b9cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58684/0 (socket says 172.21.15.26:58684) 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 -- 172.21.15.26:0/3504499493 learned_addr learned my addr 172.21.15.26:0/3504499493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd696478700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd696c79700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8fc0 0x55647b9cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 -- 172.21.15.26:0/3504499493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 msgr2=0x55647b9cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 -- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8fc0 msgr2=0x55647b9cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8fc0 0x55647b9cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 -- 172.21.15.26:0/3504499493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647b85c8d0 con 0x55647b9c8610 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd696478700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd695c77700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 0x55647b9cc0e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd680005950 tx=0x7fd68001dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b708987a4e858bd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd679ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd679ffb700 1 -- 172.21.15.26:0/3504499493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680033060 con 0x55647b9c8610 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd679ffb700 1 -- 172.21.15.26:0/3504499493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd680010ad0 con 0x55647b9c8610 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680010cd0 con 0x55647b9c8610 2024-05-20T20:59:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.057+0000 7fd679ffb700 1 -- 172.21.15.26:0/3504499493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd680021070 con 0x55647b9c8610 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 10 client.16599 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55647b8c6760 con 0x55647b9c8610 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd679ffb700 1 -- 172.21.15.26:0/3504499493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd680010cd0 con 0x55647b9c8610 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd679ffb700 1 -- 172.21.15.26:0/3504499493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fd680037030 con 0x55647b9c8610 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 20 client.16599 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647b9dc5c0 0x55647b9de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 4 client.16599 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55647b91ecc0 con 0x55647b9dc5c0 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.058+0000 7fd696478700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647b9dc5c0 0x55647b9de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.059+0000 7fd696478700 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647b9dc5c0 0x55647b9de9b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6ee378d47821a7 server_cookie=8e891f66deb72686 in_seq=0 out_seq=0 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.059+0000 7fd679ffb700 10 client.16599 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.059+0000 7fd679ffb700 1 -- 172.21.15.26:0/3504499493 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55647b91ecc0 con 0x55647b9dc5c0 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.059+0000 7fd679ffb700 10 client.16599 handle_command_reply: tid=0 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.059+0000 7fd6a2f72b80 1 client.16599 shutdown 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 msgr2=0x55647b9cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 0x55647b9cc0e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd680005950 tx=0x7fd68001dcc0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 shutdown_connections 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647b9dc5c0 0x55647b9de9b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8fc0 0x55647b9cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c32d0 0x55647b9cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/3504499493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8610 0x55647b9cc0e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd679ffb700 0 client.16599 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 >> 172.21.15.26:0/3504499493 conn(0x55647b8ea240 msgr2=0x55647b9b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 shutdown_connections 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/3504499493 wait complete. 2024-05-20T20:59:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.060+0000 7fd6a2f72b80 20 client.16599 trim_cache size 0 max 16384 2024-05-20T20:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 Processor -- start 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 -- start start 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c32d0 0x55647b9c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8610 0x55647b9b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901ae310 con 0x55647b9c8fc0 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901abba0 con 0x55647b9c32d0 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901a99f0 con 0x55647b9c8610 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9e19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58692/0 (socket says 172.21.15.26:58692) 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 -- 172.21.15.26:0/539787931 learned_addr learned my addr 172.21.15.26:0/539787931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696478700 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c32d0 0x55647b9c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd695c77700 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8610 0x55647b9b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 -- 172.21.15.26:0/539787931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8610 msgr2=0x55647b9b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8610 0x55647b9b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 -- 172.21.15.26:0/539787931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c32d0 msgr2=0x55647b9c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c32d0 0x55647b9c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 -- 172.21.15.26:0/539787931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647b9bac80 con 0x55647b9c8fc0 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd695c77700 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8610 0x55647b9b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.081+0000 7fd696c79700 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9e19d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd6880091b0 tx=0x7fd68800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4025dbc9ea94369 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd67bfff700 1 -- 172.21.15.26:0/539787931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880093c0 con 0x55647b9c8fc0 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd67bfff700 1 -- 172.21.15.26:0/539787931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6880074e0 con 0x55647b9c8fc0 2024-05-20T20:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd67bfff700 1 -- 172.21.15.26:0/539787931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880052f0 con 0x55647b9c8fc0 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/539787931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 msgr2=0x55647b9e19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9e19d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd6880091b0 tx=0x7fd68800b040 comp rx=0 tx=0).stop 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/539787931 shutdown_connections 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c8fc0 0x55647b9e19d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c32d0 0x55647b9c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/539787931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8610 0x55647b9b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/539787931 >> 172.21.15.26:0/539787931 conn(0x55647b8ea240 msgr2=0x55647b9c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/539787931 shutdown_connections 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.082+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/539787931 wait complete. 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 -- start start 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 Processor -- start 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 0x55647b9df800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8fc0 0x55647b9e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901aa060 con 0x55647b9c32d0 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901ab4d0 con 0x55647b9c8610 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901aa730 con 0x55647b9c8fc0 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd6a2f72b80 10 client.0 _ll_register_callbacks cb 0x55647b8a7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd696478700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 0x55647b9df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd696478700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 0x55647b9df800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58696/0 (socket says 172.21.15.26:58696) 2024-05-20T20:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd696478700 1 -- 172.21.15.26:0/2853059368 learned_addr learned my addr 172.21.15.26:0/2853059368 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd695c77700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.083+0000 7fd696c79700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8fc0 0x55647b9e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd696478700 1 -- 172.21.15.26:0/2853059368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8fc0 msgr2=0x55647b9e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd696478700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8fc0 0x55647b9e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd696478700 1 -- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 msgr2=0x55647b9dfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd696478700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9dfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd696478700 1 -- 172.21.15.26:0/2853059368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647ba45ed0 con 0x55647b9c32d0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd696478700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 0x55647b9df800 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00d950 tx=0x7fd68c0dfa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf77c9f544fc66c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd679ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd679ffb700 1 -- 172.21.15.26:0/2853059368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0ada30 con 0x55647b9c32d0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd679ffb700 1 -- 172.21.15.26:0/2853059368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd68c0049e0 con 0x55647b9c32d0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6880074e0 con 0x55647b9c32d0 2024-05-20T20:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd679ffb700 1 -- 172.21.15.26:0/2853059368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd68c0b24e0 con 0x55647b9c32d0 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd6a2f72b80 10 client.16605 fetch_fsmap learned FSMap version 28 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55647ba471b0 con 0x55647b9c32d0 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd679ffb700 1 -- 172.21.15.26:0/2853059368 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6880074e0 con 0x55647b9c32d0 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.084+0000 7fd679ffb700 1 -- 172.21.15.26:0/2853059368 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fd68c0b2050 con 0x55647b9c32d0 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 20 client.16605 populate_metadata read hostname 'smithi026' 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647ba47860 0x55647ba49c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 4 client.16605 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55647b9184a0 con 0x55647ba47860 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd695c77700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647ba47860 0x55647ba49c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T20:59:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd695c77700 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647ba47860 0x55647ba49c50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f05d3699308994 server_cookie=c069cb16c1848ddc in_seq=0 out_seq=0 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd679ffb700 10 client.16605 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd679ffb700 1 -- 172.21.15.26:0/2853059368 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55647b9184a0 con 0x55647ba47860 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd679ffb700 10 client.16605 handle_command_reply: tid=0 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.085+0000 7fd6a2f72b80 1 client.16605 shutdown 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 msgr2=0x55647b9df800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 0x55647b9df800 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00d950 tx=0x7fd68c0dfa30 comp rx=0 tx=0).stop 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 shutdown_connections 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55647ba47860 0x55647ba49c50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55647b9c32d0 0x55647b9df800 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647b9c8fc0 0x55647b9e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd679ffb700 0 client.16605 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T20:59:55.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 --2- 172.21.15.26:0/2853059368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55647b9c8610 0x55647b9dfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 >> 172.21.15.26:0/2853059368 conn(0x55647b8ea240 msgr2=0x55647b9bd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 shutdown_connections 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 1 -- 172.21.15.26:0/2853059368 wait complete. 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6a2f72b80 20 client.16605 trim_cache size 0 max 16384 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stdout: "2d88efa8-3a5b-4781-ba40-cec2b54b0182": { 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-05-20T20:59:55.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "2d88efa8-3a5b-4781-ba40-cec2b54b0182", 2024-05-20T20:59:55.093 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-05-20T20:59:55.093 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:55.093 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T20:59:55.093 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T20:59:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd67c0b2880 msgr2=0x7fd67c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6986dc700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd67c0b2880 0x7fd67c0b4d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd680005fd0 tx=0x7fd680005d90 comp rx=0 tx=0).stop 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 msgr2=0x7fd69019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.086+0000 7fd6986dc700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 0x7fd69019f6d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00d980 tx=0x7fd68c00dc90 comp rx=0 tx=0).stop 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 shutdown_connections 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd67c0b2880 0x7fd67c0b4d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6901037f0 0x7fd69019f6d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6901088f0 0x7fd690199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 --2- 172.21.15.26:0/1240552398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6901041a0 0x7fd69019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 >> 172.21.15.26:0/1240552398 conn(0x7fd69007a6a0 msgr2=0x7fd690101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 shutdown_connections 2024-05-20T20:59:55.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T20:59:55.087+0000 7fd6986dc700 1 -- 172.21.15.26:0/1240552398 wait complete. 2024-05-20T20:59:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:55 smithi055 ceph-mon[35213]: pgmap v502: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-20T20:59:55.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:55 smithi026 ceph-mon[31990]: pgmap v502: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-20T20:59:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:55 smithi144 ceph-mon[35660]: pgmap v502: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-20T20:59:55.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 1 sec latency 5632.212 ms 2024-05-20T20:59:55.780 INFO:tasks.cephfs.filesystem:scrub status for tag:2d88efa8-3a5b-4781-ba40-cec2b54b0182 - {'path': '/', 'tag': '2d88efa8-3a5b-4781-ba40-cec2b54b0182', 'options': 'recursive,force'} 2024-05-20T20:59:56.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:56 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:59:56.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:56 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:59:56.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:56 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T20:59:56.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 2 sec latency 6632.341 ms 2024-05-20T20:59:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:57 smithi055 ceph-mon[35213]: pgmap v503: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1024 KiB/s wr, 85 op/s 2024-05-20T20:59:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:57 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T20:59:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:59:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:59:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:59:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:57 smithi026 ceph-mon[31990]: pgmap v503: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1024 KiB/s wr, 85 op/s 2024-05-20T20:59:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:57 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T20:59:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:59:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:59:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:59:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:57 smithi144 ceph-mon[35660]: pgmap v503: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1024 KiB/s wr, 85 op/s 2024-05-20T20:59:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:57 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T20:59:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T20:59:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T20:59:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T20:59:57.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 3 sec latency 7632.442 ms 2024-05-20T20:59:58.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 4 sec latency 8632.532 ms 2024-05-20T20:59:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 20:59:59 smithi055 ceph-mon[35213]: pgmap v504: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 746 KiB/s wr, 77 op/s 2024-05-20T20:59:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 20:59:59 smithi026 ceph-mon[31990]: pgmap v504: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 746 KiB/s wr, 77 op/s 2024-05-20T20:59:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 20:59:59 smithi144 ceph-mon[35660]: pgmap v504: 97 pgs: 97 active+clean; 23 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 746 KiB/s wr, 77 op/s 2024-05-20T20:59:59.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 5 sec latency 9632.603 ms 2024-05-20T21:00:00.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:00 smithi055 ceph-mon[35213]: overall HEALTH_OK 2024-05-20T21:00:00.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:00 smithi026 ceph-mon[31990]: overall HEALTH_OK 2024-05-20T21:00:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:00 smithi144 ceph-mon[35660]: overall HEALTH_OK 2024-05-20T21:00:00.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 6 sec latency 10632.701 ms 2024-05-20T21:00:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:01 smithi055 ceph-mon[35213]: pgmap v505: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 504 KiB/s wr, 53 op/s 2024-05-20T21:00:01.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:01 smithi026 ceph-mon[31990]: pgmap v505: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 504 KiB/s wr, 53 op/s 2024-05-20T21:00:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:01 smithi144 ceph-mon[35660]: pgmap v505: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 504 KiB/s wr, 53 op/s 2024-05-20T21:00:01.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 7 sec latency 11632.791 ms 2024-05-20T21:00:02.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 8 sec latency 12632.903 ms 2024-05-20T21:00:03.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:03 smithi055 ceph-mon[35213]: pgmap v506: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-05-20T21:00:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:03 smithi026 ceph-mon[31990]: pgmap v506: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-05-20T21:00:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:03 smithi144 ceph-mon[35660]: pgmap v506: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-05-20T21:00:03.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 9 sec latency 13632.998 ms 2024-05-20T21:00:04.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 10 sec latency 14633.093 ms 2024-05-20T21:00:05.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:00:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:05 smithi055 ceph-mon[35213]: pgmap v507: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-05-20T21:00:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:05 smithi026 ceph-mon[31990]: pgmap v507: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-05-20T21:00:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:05 smithi144 ceph-mon[35660]: pgmap v507: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-05-20T21:00:05.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 11 sec latency 15633.132 ms 2024-05-20T21:00:06.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 12 sec latency 16633.166 ms 2024-05-20T21:00:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 -- 172.21.15.26:0/3136793475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 msgr2=0x7f545c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 --2- 172.21.15.26:0/3136793475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c108900 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f544c009990 tx=0x7f544c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 -- 172.21.15.26:0/3136793475 shutdown_connections 2024-05-20T21:00:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 --2- 172.21.15.26:0/3136793475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c108900 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 --2- 172.21.15.26:0/3136793475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c102e00 0x7f545c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 --2- 172.21.15.26:0/3136793475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 0x7f545c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 -- 172.21.15.26:0/3136793475 >> 172.21.15.26:0/3136793475 conn(0x7f545c0793f0 msgr2=0x7f545c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 -- 172.21.15.26:0/3136793475 shutdown_connections 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.233+0000 7f5462ad3700 1 -- 172.21.15.26:0/3136793475 wait complete. 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 Processor -- start 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 -- start start 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 0x7f545c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c102e00 0x7f545c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1124a0 con 0x7f545c108520 2024-05-20T21:00:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f545c112320 con 0x7f545c102440 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5462ad3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f545c112620 con 0x7f545c102e00 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5461070700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5461070700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39978/0 (socket says 172.21.15.26:39978) 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.234+0000 7f5461070700 1 -- 172.21.15.26:0/2297457343 learned_addr learned my addr 172.21.15.26:0/2297457343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f545bfff700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c102e00 0x7f545c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f546086f700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 0x7f545c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5461070700 1 -- 172.21.15.26:0/2297457343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c102e00 msgr2=0x7f545c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5461070700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c102e00 0x7f545c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5461070700 1 -- 172.21.15.26:0/2297457343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 msgr2=0x7f545c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5461070700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 0x7f545c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5461070700 1 -- 172.21.15.26:0/2297457343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19a0f0 con 0x7f545c108520 2024-05-20T21:00:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f546086f700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 0x7f545c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5461070700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c199990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5454007ee0 tx=0x7f545400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545400dbd0 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f545400dd70 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19a3e0 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54540058d0 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.235+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a66a0 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.236+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c06a230 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.238+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f545401a020 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.238+0000 7f5459ffb700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54480b2880 0x7f54480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.238+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f54540e2020 con 0x7f545c108520 2024-05-20T21:00:07.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.238+0000 7f546086f700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54480b2880 0x7f54480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.239+0000 7f546086f700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54480b2880 0x7f54480b4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f544c003760 tx=0x7f544c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.242+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5454010070 con 0x7f545c108520 2024-05-20T21:00:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:07 smithi055 ceph-mon[35213]: pgmap v508: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-05-20T21:00:07.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:07 smithi026 ceph-mon[31990]: pgmap v508: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-05-20T21:00:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.511+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f545c1a6870 con 0x7f545c108520 2024-05-20T21:00:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.512+0000 7f5459ffb700 1 -- 172.21.15.26:0/2297457343 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f54540a9ce0 con 0x7f545c108520 2024-05-20T21:00:07.512 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:00:07.513 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54480b2880 msgr2=0x7f54480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54480b2880 0x7f54480b4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f544c003760 tx=0x7f544c01a040 comp rx=0 tx=0).stop 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 msgr2=0x7f545c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c199990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5454007ee0 tx=0x7f545400d3b0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 shutdown_connections 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54480b2880 0x7f54480b4d40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f545c108520 0x7f545c199990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f545c102440 0x7f545c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 --2- 172.21.15.26:0/2297457343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c102e00 0x7f545c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 >> 172.21.15.26:0/2297457343 conn(0x7f545c0793f0 msgr2=0x7f545c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 shutdown_connections 2024-05-20T21:00:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:07.515+0000 7f5462ad3700 1 -- 172.21.15.26:0/2297457343 wait complete. 2024-05-20T21:00:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:00:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:07 smithi144 ceph-mon[35660]: pgmap v508: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-05-20T21:00:07.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 13 sec latency 17633.196 ms 2024-05-20T21:00:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/2802287396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44103cd0 msgr2=0x7f7a44104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/2802287396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44103cd0 0x7f7a44104150 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009900 tx=0x7f7a3402a410 comp rx=0 tx=0).stop 2024-05-20T21:00:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/2802287396 shutdown_connections 2024-05-20T21:00:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/2802287396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44104690 0x7f7a4410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/2802287396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44103cd0 0x7f7a44104150 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/2802287396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a44102ae0 0x7f7a44102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/2802287396 >> 172.21.15.26:0/2802287396 conn(0x7f7a440fe070 msgr2=0x7f7a44100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/2802287396 shutdown_connections 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.009+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/2802287396 wait complete. 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 Processor -- start 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 -- start start 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a44102ae0 0x7f7a44076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44103cd0 0x7f7a44074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 0x7f7a44075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4410e4a0 con 0x7f7a44104690 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4410e320 con 0x7f7a44102ae0 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a4b8dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4410e620 con 0x7f7a44103cd0 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 0x7f7a44075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 0x7f7a44075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40000/0 (socket says 172.21.15.26:40000) 2024-05-20T21:00:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 -- 172.21.15.26:0/3635299170 learned_addr learned my addr 172.21.15.26:0/3635299170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49678640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a44102ae0 0x7f7a44076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a48e77640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44103cd0 0x7f7a44074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 -- 172.21.15.26:0/3635299170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44103cd0 msgr2=0x7f7a44074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44103cd0 0x7f7a44074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 -- 172.21.15.26:0/3635299170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a44102ae0 msgr2=0x7f7a44076800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a44102ae0 0x7f7a44076800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.010+0000 7f7a49e79640 1 -- 172.21.15.26:0/3635299170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a44075c10 con 0x7f7a44104690 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.011+0000 7f7a49e79640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 0x7f7a44075380 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00cad0 tx=0x7f7a3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.011+0000 7f7a3a7fc640 1 -- 172.21.15.26:0/3635299170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00ae20 con 0x7f7a44104690 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.011+0000 7f7a3a7fc640 1 -- 172.21.15.26:0/3635299170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a3c005020 con 0x7f7a44104690 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.011+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a441a2c00 con 0x7f7a44104690 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.011+0000 7f7a3a7fc640 1 -- 172.21.15.26:0/3635299170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00f6b0 con 0x7f7a44104690 2024-05-20T21:00:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.011+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a441a30d0 con 0x7f7a44104690 2024-05-20T21:00:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.013+0000 7f7a3a7fc640 1 -- 172.21.15.26:0/3635299170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7a3c0051d0 con 0x7f7a44104690 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 Processor -- start 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 -- start start 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cba6c0 0x560dd3cbaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cbb0b0 0x560dd3cbfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cc26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560dd3c2bc30 con 0x560dd3cba6c0 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560dd3c1b7d0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560dd3c1f810 con 0x560dd3cc0220 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a48e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cba6c0 0x560dd3cbaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a48e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cba6c0 0x560dd3cbaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40002/0 (socket says 172.21.15.26:40002) 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a48e77640 1 -- 172.21.15.26:0/1548957270 learned_addr learned my addr 172.21.15.26:0/1548957270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49678640 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cbb0b0 0x560dd3cbfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a48e77640 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cc26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a3a7fc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a300b28a0 0x7f7a300b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49e79640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a300b28a0 0x7f7a300b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49678640 1 -- 172.21.15.26:0/1548957270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 msgr2=0x560dd3cc26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49678640 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cc26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49678640 1 -- 172.21.15.26:0/1548957270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cba6c0 msgr2=0x560dd3cbaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49678640 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cba6c0 0x560dd3cbaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a49678640 1 -- 172.21.15.26:0/1548957270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd3930210 con 0x560dd3cbb0b0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.014+0000 7f7a3a7fc640 1 -- 172.21.15.26:0/3635299170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f7a3c0e2730 con 0x7f7a44104690 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.015+0000 7f7a49678640 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cbb0b0 0x560dd3cbfbb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000cb70 tx=0x7f7a4000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d541066a6b17c585 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.015+0000 7f7a49e79640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a300b28a0 0x7f7a300b4d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0a5d70 tx=0x7f7a3c005b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.015+0000 7f7a2f7fe640 1 -- 172.21.15.26:0/1548957270 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40004220 con 0x560dd3cbb0b0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.015+0000 7f7a2f7fe640 1 -- 172.21.15.26:0/1548957270 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a400043c0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.015+0000 7f7a2f7fe640 1 -- 172.21.15.26:0/1548957270 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40004bc0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1548957270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cbb0b0 msgr2=0x560dd3cbfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cbb0b0 0x560dd3cbfbb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000cb70 tx=0x7f7a4000e720 comp rx=0 tx=0).stop 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1548957270 shutdown_connections 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cc26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cbb0b0 0x560dd3cbfbb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1548957270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cba6c0 0x560dd3cbaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1548957270 >> 172.21.15.26:0/1548957270 conn(0x560dd3cb5d30 msgr2=0x560dd3cb6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1548957270 shutdown_connections 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1548957270 wait complete. 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- start start 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 Processor -- start 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3cc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 0x560dd3ccbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.016+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560dd3901790 con 0x560dd3cc0220 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560dd3bd7890 con 0x560dd3cba6c0 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560dd3cc31d0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a4e5f3740 10 client.0 _ll_register_callbacks cb 0x560dd392fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 0x560dd3ccbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 0x560dd3ccbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40008/0 (socket says 172.21.15.26:40008) 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 -- 172.21.15.26:0/1568971402 learned_addr learned my addr 172.21.15.26:0/1568971402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3cc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a49678640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 -- 172.21.15.26:0/1568971402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cbb0b0 msgr2=0x560dd3cc4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 -- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 msgr2=0x560dd3cc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3cc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 -- 172.21.15.26:0/1568971402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd39fc620 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a48e77640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 0x560dd3ccbd60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7a3402a930 tx=0x7f7a34041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747c79f05e6310c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a2cff9640 1 -- 172.21.15.26:0/1568971402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34035070 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a2cff9640 1 -- 172.21.15.26:0/1568971402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f7a34004460 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a400043c0 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.017+0000 7f7a2cff9640 1 -- 172.21.15.26:0/1568971402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a3403e630 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 10 client.16623 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dd3ccedb0 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a2cff9640 1 -- 172.21.15.26:0/1568971402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a400043c0 con 0x560dd3cc0220 2024-05-20T21:00:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a2cff9640 1 -- 172.21.15.26:0/1568971402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f7a340039d0 con 0x560dd3cc0220 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 10 client.16623 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 20 client.16623 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3cd0480 0x560dd3cd28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 4 client.16623 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd3c2db80 con 0x560dd3cd0480 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.018+0000 7f7a49678640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3cd0480 0x560dd3cd28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.019+0000 7f7a2cff9640 10 client.16623 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.019+0000 7f7a49678640 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3cd0480 0x560dd3cd28b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff48a640f81729c8 server_cookie=6d837091ea9af8c8 in_seq=0 out_seq=0 2024-05-20T21:00:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.019+0000 7f7a2cff9640 1 -- 172.21.15.26:0/1568971402 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560dd3c2db80 con 0x560dd3cd0480 2024-05-20T21:00:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.019+0000 7f7a2cff9640 10 client.16623 handle_command_reply: tid=0 2024-05-20T21:00:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.019+0000 7f7a4e5f3740 1 client.16623 shutdown 2024-05-20T21:00:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 msgr2=0x560dd3ccbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 0x560dd3ccbd60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7a3402a930 tx=0x7f7a34041040 comp rx=0 tx=0).stop 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 shutdown_connections 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3cd0480 0x560dd3cd28b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cc0220 0x560dd3ccbd60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a2cff9640 0 client.16623 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/1568971402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3cc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 >> 172.21.15.26:0/1568971402 conn(0x560dd3cb5d30 msgr2=0x560dd3cb4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 shutdown_connections 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/1568971402 wait complete. 2024-05-20T21:00:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.020+0000 7f7a4e5f3740 20 client.16623 trim_cache size 0 max 16384 2024-05-20T21:00:08.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 Processor -- start 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 -- start start 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cba6c0 0x560dd3cb8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3ccbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cc0220 0x560dd3cb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441ae840 con 0x560dd3cbb0b0 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441ac110 con 0x560dd3cc0220 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441a9f60 con 0x560dd3cba6c0 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3ccbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3ccbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40018/0 (socket says 172.21.15.26:40018) 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 -- 172.21.15.26:0/2602304270 learned_addr learned my addr 172.21.15.26:0/2602304270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a48e77640 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cba6c0 0x560dd3cb8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 -- 172.21.15.26:0/2602304270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cba6c0 msgr2=0x560dd3cb8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cba6c0 0x560dd3cb8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 -- 172.21.15.26:0/2602304270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cc0220 msgr2=0x560dd3cb9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cc0220 0x560dd3cb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.035+0000 7f7a49678640 1 -- 172.21.15.26:0/2602304270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd3cbbd80 con 0x560dd3cbb0b0 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a49678640 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3ccbd20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000cac0 tx=0x7f7a4000c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e6802b403ac748 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a2effd640 1 -- 172.21.15.26:0/2602304270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400044d0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a2effd640 1 -- 172.21.15.26:0/2602304270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a4002e070 con 0x560dd3cbb0b0 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a2effd640 1 -- 172.21.15.26:0/2602304270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4001a6a0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/2602304270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 msgr2=0x560dd3ccbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3ccbd20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000cac0 tx=0x7f7a4000c4b0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/2602304270 shutdown_connections 2024-05-20T21:00:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cc0220 0x560dd3cb9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3ccbd20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/2602304270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cba6c0 0x560dd3cb8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/2602304270 >> 172.21.15.26:0/2602304270 conn(0x560dd3cb5d30 msgr2=0x560dd3cb0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/2602304270 shutdown_connections 2024-05-20T21:00:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.036+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/2602304270 wait complete. 2024-05-20T21:00:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 -- start start 2024-05-20T21:00:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 Processor -- start 2024-05-20T21:00:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3ccb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441aa5d0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441aba40 con 0x560dd3cba6c0 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441aaca0 con 0x560dd3cc0220 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a4e5f3740 10 client.0 _ll_register_callbacks cb 0x7f7a34032d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40034/0 (socket says 172.21.15.26:40034) 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 -- 172.21.15.26:0/505710939 learned_addr learned my addr 172.21.15.26:0/505710939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a48e77640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a48e77640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3ccb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 -- 172.21.15.26:0/505710939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 msgr2=0x560dd3cca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 -- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 msgr2=0x560dd3ccb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3ccb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.037+0000 7f7a49678640 1 -- 172.21.15.26:0/505710939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd3cb9760 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a48e77640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3ccb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a49678640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc9f10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c460 tx=0x7f7a40003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d6f7d630f87ab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a2cff9640 1 -- 172.21.15.26:0/505710939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400035f0 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a4002e070 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a2cff9640 1 -- 172.21.15.26:0/505710939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f7a40003790 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a2cff9640 1 -- 172.21.15.26:0/505710939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a40003990 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a4e5f3740 10 client.16629 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dd3cb5b60 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a2cff9640 1 -- 172.21.15.26:0/505710939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a4002e070 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a2cff9640 1 -- 172.21.15.26:0/505710939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f7a4001a460 con 0x560dd3cbb0b0 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a4e5f3740 10 client.16629 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a4e5f3740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.038+0000 7f7a4e5f3740 20 client.16629 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.039+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3d3c210 0x560dd3d3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.039+0000 7f7a4e5f3740 4 client.16629 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.039+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dd3c27360 con 0x560dd3d3c210 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.039+0000 7f7a48e77640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3d3c210 0x560dd3d3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.039+0000 7f7a48e77640 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3d3c210 0x560dd3d3e630 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4bab74b29c7963 server_cookie=60f56e1a9e4b0edc in_seq=0 out_seq=0 2024-05-20T21:00:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.039+0000 7f7a2cff9640 10 client.16629 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a2cff9640 1 -- 172.21.15.26:0/505710939 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560dd3c27360 con 0x560dd3d3c210 2024-05-20T21:00:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a2cff9640 10 client.16629 handle_command_reply: tid=0 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 client.16629 shutdown 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 msgr2=0x560dd3cc9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc9f10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c460 tx=0x7f7a40003f90 comp rx=0 tx=0).stop 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 shutdown_connections 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560dd3d3c210 0x560dd3d3e630 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd3cc0220 0x560dd3cca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a2cff9640 0 client.16629 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dd3cbb0b0 0x560dd3cc9f10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 --2- 172.21.15.26:0/505710939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560dd3cba6c0 0x560dd3ccb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 >> 172.21.15.26:0/505710939 conn(0x560dd3cb5d30 msgr2=0x560dd3cbf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 shutdown_connections 2024-05-20T21:00:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 1 -- 172.21.15.26:0/505710939 wait complete. 2024-05-20T21:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.055+0000 7f7a4e5f3740 20 client.16629 trim_cache size 0 max 16384 2024-05-20T21:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a300b28a0 msgr2=0x7f7a300b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a300b28a0 0x7f7a300b4d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0a5d70 tx=0x7f7a3c005b20 comp rx=0 tx=0).stop 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 msgr2=0x7f7a44075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 0x7f7a44075380 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00cad0 tx=0x7f7a3c00cf90 comp rx=0 tx=0).stop 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 shutdown_connections 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a300b28a0 0x7f7a300b4d60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a44104690 0x7f7a44075380 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44103cd0 0x7f7a44074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 --2- 172.21.15.26:0/3635299170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a44102ae0 0x7f7a44076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 >> 172.21.15.26:0/3635299170 conn(0x7f7a440fe070 msgr2=0x7f7a44106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 shutdown_connections 2024-05-20T21:00:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:08.056+0000 7f7a4b8dc640 1 -- 172.21.15.26:0/3635299170 wait complete. 2024-05-20T21:00:08.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id be68fc79 --release --await' 2024-05-20T21:00:08.231 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id be68fc79 --release --await 2024-05-20T21:00:08.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:08 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2297457343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:08.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:08 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2297457343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:08.480 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:08 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2297457343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:08.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 14 sec latency 18633.243 ms 2024-05-20T21:00:09.415 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:09 smithi144 ceph-mon[35660]: pgmap v509: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-05-20T21:00:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:09 smithi055 ceph-mon[35213]: pgmap v509: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-05-20T21:00:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:09 smithi026 ceph-mon[31990]: pgmap v509: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-05-20T21:00:09.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 15 sec latency 19633.600 ms 2024-05-20T21:00:10.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 16 sec latency 20633.695 ms 2024-05-20T21:00:11.429 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:11 smithi026 ceph-mon[31990]: pgmap v510: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-20T21:00:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:11 smithi055 ceph-mon[35213]: pgmap v510: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-20T21:00:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:11 smithi144 ceph-mon[35660]: pgmap v510: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-20T21:00:11.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66308 0.00 MB/sec execute 17 sec latency 21633.788 ms 2024-05-20T21:00:11.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 -- 172.21.15.26:0/593289471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841088f0 msgr2=0x7f6d84110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/593289471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841088f0 0x7f6d84110470 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099c0 tx=0x7f6d7c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 -- 172.21.15.26:0/593289471 shutdown_connections 2024-05-20T21:00:11.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/593289471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841088f0 0x7f6d84110470 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/593289471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d841037f0 0x7f6d84103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/593289471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d841041a0 0x7f6d841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 -- 172.21.15.26:0/593289471 >> 172.21.15.26:0/593289471 conn(0x7f6d8407a6a0 msgr2=0x7f6d8407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 -- 172.21.15.26:0/593289471 shutdown_connections 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.677+0000 7f6d8bd59700 1 -- 172.21.15.26:0/593289471 wait complete. 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 Processor -- start 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 -- start start 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 0x7f6d84075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d841041a0 0x7f6d84075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d841088f0 0x7f6d840762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841135f0 con 0x7f6d841037f0 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84113470 con 0x7f6d841041a0 2024-05-20T21:00:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d8bd59700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84113770 con 0x7f6d841088f0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d89af5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 0x7f6d84075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d89af5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 0x7f6d84075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54650/0 (socket says 172.21.15.26:54650) 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d89af5700 1 -- 172.21.15.26:0/545018177 learned_addr learned my addr 172.21.15.26:0/545018177 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.678+0000 7f6d892f4700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d841041a0 0x7f6d84075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d841088f0 0x7f6d840762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d89af5700 1 -- 172.21.15.26:0/545018177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d841088f0 msgr2=0x7f6d840762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d89af5700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d841088f0 0x7f6d840762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d89af5700 1 -- 172.21.15.26:0/545018177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d841041a0 msgr2=0x7f6d84075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d89af5700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d841041a0 0x7f6d84075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d89af5700 1 -- 172.21.15.26:0/545018177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841a7f80 con 0x7f6d841037f0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d89af5700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 0x7f6d84075840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba50 tx=0x7f6d7400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d82ffd700 1 -- 172.21.15.26:0/545018177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74019070 con 0x7f6d841037f0 2024-05-20T21:00:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d82ffd700 1 -- 172.21.15.26:0/545018177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7400ce40 con 0x7f6d841037f0 2024-05-20T21:00:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d82ffd700 1 -- 172.21.15.26:0/545018177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74007370 con 0x7f6d841037f0 2024-05-20T21:00:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841a8270 con 0x7f6d841037f0 2024-05-20T21:00:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.679+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841a8760 con 0x7f6d841037f0 2024-05-20T21:00:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.682+0000 7f6d82ffd700 1 -- 172.21.15.26:0/545018177 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6d7401c070 con 0x7f6d841037f0 2024-05-20T21:00:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.683+0000 7f6d82ffd700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6d6c0b2880 0x7f6d6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.683+0000 7f6d892f4700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6d6c0b2880 0x7f6d6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.683+0000 7f6d82ffd700 1 -- 172.21.15.26:0/545018177 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f6d740e17d0 con 0x7f6d841037f0 2024-05-20T21:00:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.683+0000 7f6d965efb80 1 Processor -- start 2024-05-20T21:00:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.683+0000 7f6d965efb80 1 -- start start 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2133690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d892f4700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6d6c0b2880 0x7f6d6c0b4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d84103100 tx=0x7f6d7800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec212dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec213a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d965efb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ec208cd90 con 0x557ec212e2a0 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d965efb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557ec207c930 con 0x557ec212d8e0 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d965efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ec2080970 con 0x557ec21332b0 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec213a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec213a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54656/0 (socket says 172.21.15.26:54656) 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/1222723111 learned_addr learned my addr 172.21.15.26:0/1222723111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d892f4700 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec212dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d89af5700 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/1222723111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 msgr2=0x557ec2133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/1222723111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 msgr2=0x557ec212dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec212dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/1222723111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec1fcc9d0 con 0x557ec212e2a0 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.684+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec213a0e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099c0 tx=0x7f6d7c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c8177dd1dd9fef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d6bfff700 1 -- 172.21.15.26:0/1222723111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c027070 con 0x557ec212e2a0 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d6bfff700 1 -- 172.21.15.26:0/1222723111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c00bc20 con 0x557ec212e2a0 2024-05-20T21:00:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d6bfff700 1 -- 172.21.15.26:0/1222723111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c003ae0 con 0x557ec212e2a0 2024-05-20T21:00:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 -- 172.21.15.26:0/1222723111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 msgr2=0x557ec213a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec213a0e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099c0 tx=0x7f6d7c005c80 comp rx=0 tx=0).stop 2024-05-20T21:00:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 -- 172.21.15.26:0/1222723111 shutdown_connections 2024-05-20T21:00:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec213a0e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1222723111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec212dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.685+0000 7f6d965efb80 1 -- 172.21.15.26:0/1222723111 >> 172.21.15.26:0/1222723111 conn(0x557ec205b180 msgr2=0x557ec21281f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 -- 172.21.15.26:0/1222723111 shutdown_connections 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 -- 172.21.15.26:0/1222723111 wait complete. 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 -- start start 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 Processor -- start 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec2137110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2137650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ec1f830a0 con 0x557ec212e2a0 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557ec201e8c0 con 0x557ec212d8e0 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.686+0000 7f6d965efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ec213a680 con 0x557ec21332b0 2024-05-20T21:00:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d965efb80 10 client.0 _ll_register_callbacks cb 0x557ec2017a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec2137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec2137110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54664/0 (socket says 172.21.15.26:54664) 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d89af5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 -- 172.21.15.26:0/3717847676 learned_addr learned my addr 172.21.15.26:0/3717847676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2137650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 -- 172.21.15.26:0/3717847676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 msgr2=0x557ec2137650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2137650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 -- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 msgr2=0x557ec2139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 -- 172.21.15.26:0/3717847676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec201b790 con 0x557ec212e2a0 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2137650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d89af5700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d892f4700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec2137110 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6d780036d0 tx=0x7f6d780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96c7b439c733fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d697fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d697fa700 1 -- 172.21.15.26:0/3717847676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78016070 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d7c00bc20 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.687+0000 7f6d697fa700 1 -- 172.21.15.26:0/3717847676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f6d78010840 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d697fa700 1 -- 172.21.15.26:0/3717847676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d78010a40 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 10 client.16641 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ec2144e80 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d697fa700 1 -- 172.21.15.26:0/3717847676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c00bc20 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d697fa700 1 -- 172.21.15.26:0/3717847676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f6d78027ba0 con 0x557ec212e2a0 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 10 client.16641 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 10 client.16641 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 20 client.16641 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec2147410 0x557ec2149830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 4 client.16641 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ec20f9ae0 con 0x557ec2147410 2024-05-20T21:00:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.688+0000 7f6d89af5700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec2147410 0x557ec2149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.689+0000 7f6d697fa700 10 client.16641 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.689+0000 7f6d89af5700 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec2147410 0x557ec2149830 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0cca927f5bb722b server_cookie=360406fb2cbd71e8 in_seq=0 out_seq=0 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d697fa700 1 -- 172.21.15.26:0/3717847676 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ec20f9ae0 con 0x557ec2147410 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d697fa700 10 client.16641 handle_command_reply: tid=0 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 client.16641 shutdown 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 msgr2=0x557ec2137110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec2137110 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6d780036d0 tx=0x7f6d780047c0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 shutdown_connections 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec2147410 0x557ec2149830 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec21332b0 0x557ec2137650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 --2- 172.21.15.26:0/3717847676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212e2a0 0x557ec2137110 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 >> 172.21.15.26:0/3717847676 conn(0x557ec205b180 msgr2=0x557ec2122ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:11.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d697fa700 0 client.16641 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:11.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 shutdown_connections 2024-05-20T21:00:11.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 1 -- 172.21.15.26:0/3717847676 wait complete. 2024-05-20T21:00:11.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.690+0000 7f6d965efb80 20 client.16641 trim_cache size 0 max 16384 2024-05-20T21:00:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 Processor -- start 2024-05-20T21:00:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 -- start start 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec214f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 0x557ec214f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841b2690 con 0x557ec21332b0 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841aff20 con 0x557ec212d8e0 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d965efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841add70 con 0x557ec212e2a0 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d8a2f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 0x557ec214f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d8a2f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 0x557ec214f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54672/0 (socket says 172.21.15.26:54672) 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/428714397 learned_addr learned my addr 172.21.15.26:0/428714397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d89af5700 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.711+0000 7f6d892f4700 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec214f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/428714397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 msgr2=0x557ec214f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec214f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/428714397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 msgr2=0x557ec2139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d8a2f6700 1 -- 172.21.15.26:0/428714397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec21249f0 con 0x557ec21332b0 2024-05-20T21:00:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 0x557ec214f670 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c024000 tx=0x7f6d7c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ed2a78ab4daa36 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d6b7fe700 1 -- 172.21.15.26:0/428714397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c031070 con 0x557ec21332b0 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d6b7fe700 1 -- 172.21.15.26:0/428714397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c003b80 con 0x557ec21332b0 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.712+0000 7f6d6b7fe700 1 -- 172.21.15.26:0/428714397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c027040 con 0x557ec21332b0 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 -- 172.21.15.26:0/428714397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 msgr2=0x557ec214f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 0x557ec214f670 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c024000 tx=0x7f6d7c003680 comp rx=0 tx=0).stop 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 -- 172.21.15.26:0/428714397 shutdown_connections 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec21332b0 0x557ec214f670 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec212d8e0 0x557ec2139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 --2- 172.21.15.26:0/428714397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec214f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 -- 172.21.15.26:0/428714397 >> 172.21.15.26:0/428714397 conn(0x557ec205b180 msgr2=0x557ec21388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 -- 172.21.15.26:0/428714397 shutdown_connections 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 -- 172.21.15.26:0/428714397 wait complete. 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 -- start start 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.713+0000 7f6d965efb80 1 Processor -- start 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 0x557ec21400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec213c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec21332b0 0x557ec213cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841ae3e0 con 0x557ec212d8e0 2024-05-20T21:00:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841af850 con 0x557ec21332b0 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841aeab0 con 0x557ec212e2a0 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 10 client.0 _ll_register_callbacks cb 0x557ec2017a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 0x557ec21400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 0x557ec21400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54678/0 (socket says 172.21.15.26:54678) 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 -- 172.21.15.26:0/1065924721 learned_addr learned my addr 172.21.15.26:0/1065924721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d8a2f6700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec21332b0 0x557ec213cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d892f4700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec213c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 -- 172.21.15.26:0/1065924721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 msgr2=0x557ec213c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec213c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 -- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec21332b0 msgr2=0x557ec213cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec21332b0 0x557ec213cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 -- 172.21.15.26:0/1065924721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec213d700 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d89af5700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 0x557ec21400d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba20 tx=0x7f6d740e1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d4bf891982ce98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d697fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d697fa700 1 -- 172.21.15.26:0/1065924721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740c33f0 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d697fa700 1 -- 172.21.15.26:0/1065924721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f6d74014440 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.714+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d7c003b80 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d697fa700 1 -- 172.21.15.26:0/1065924721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d740b1760 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 10 client.16647 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ec21b31c0 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d697fa700 1 -- 172.21.15.26:0/1065924721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c003b80 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d697fa700 1 -- 172.21.15.26:0/1065924721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f6d740a4700 con 0x557ec212d8e0 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 10 client.16647 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 10 client.16647 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 20 client.16647 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec21b3820 0x557ec21b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 4 client.16647 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "be68fc79", "release": true, "await": true}] 2024-05-20T21:00:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "be68fc79", "release": true, "await": true}) -- 0x557ec208ece0 con 0x557ec21b3820 2024-05-20T21:00:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.715+0000 7f6d892f4700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec21b3820 0x557ec21b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.716+0000 7f6d892f4700 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec21b3820 0x557ec21b5c40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33a0596bfa54bce server_cookie=d56b99f240e50301 in_seq=0 out_seq=0 2024-05-20T21:00:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.716+0000 7f6d697fa700 10 client.16647 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:11.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d697fa700 1 -- 172.21.15.26:0/1065924721 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557ec208ece0 con 0x557ec21b3820 2024-05-20T21:00:11.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d697fa700 10 client.16647 handle_command_reply: tid=0 2024-05-20T21:00:11.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 client.16647 shutdown 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 msgr2=0x557ec21400d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 0x557ec21400d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba20 tx=0x7f6d740e1040 comp rx=0 tx=0).stop 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 shutdown_connections 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ec21b3820 0x557ec21b5c40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ec212d8e0 0x557ec21400d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d697fa700 0 client.16647 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ec21332b0 0x557ec213cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 --2- 172.21.15.26:0/1065924721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ec212e2a0 0x557ec213c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 >> 172.21.15.26:0/1065924721 conn(0x557ec205b180 msgr2=0x557ec2131620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 shutdown_connections 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 1 -- 172.21.15.26:0/1065924721 wait complete. 2024-05-20T21:00:11.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.727+0000 7f6d965efb80 20 client.16647 trim_cache size 0 max 16384 2024-05-20T21:00:11.729 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:00:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 28, 2024-05-20T21:00:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:00:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 16, 2024-05-20T21:00:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:00:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "be68fc79": { 2024-05-20T21:00:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 16, 2024-05-20T21:00:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:00:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:00:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 143.1, 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:11.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:11.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:11.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:11.732 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:00:11.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6d6c0b2880 msgr2=0x7f6d6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6d6c0b2880 0x7f6d6c0b4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d84103100 tx=0x7f6d7800a380 comp rx=0 tx=0).stop 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 msgr2=0x7f6d84075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 0x7f6d84075840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba50 tx=0x7f6d7400bd60 comp rx=0 tx=0).stop 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 shutdown_connections 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6d6c0b2880 0x7f6d6c0b4d40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d841037f0 0x7f6d84075840 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d841088f0 0x7f6d840762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 --2- 172.21.15.26:0/545018177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d841041a0 0x7f6d84075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 >> 172.21.15.26:0/545018177 conn(0x7f6d8407a6a0 msgr2=0x7f6d841017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 shutdown_connections 2024-05-20T21:00:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:11.728+0000 7f6d8bd59700 1 -- 172.21.15.26:0/545018177 wait complete. 2024-05-20T21:00:12.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 67148 0.40 MB/sec execute 18 sec latency 21801.648 ms 2024-05-20T21:00:12.558 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'be68fc79', total seconds elapsed: 30.6 2024-05-20T21:00:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:13 smithi055 ceph-mon[35213]: pgmap v511: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T21:00:13.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:13 smithi026 ceph-mon[31990]: pgmap v511: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T21:00:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:13 smithi144 ceph-mon[35660]: pgmap v511: 97 pgs: 97 active+clean; 23 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T21:00:13.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 67628 0.61 MB/sec execute 19 sec latency 18.515 ms 2024-05-20T21:00:14.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 68034 0.60 MB/sec execute 20 sec latency 17.834 ms 2024-05-20T21:00:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:15 smithi055 ceph-mon[35213]: pgmap v512: 97 pgs: 97 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 135 KiB/s wr, 16 op/s 2024-05-20T21:00:15.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:15 smithi026 ceph-mon[31990]: pgmap v512: 97 pgs: 97 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 135 KiB/s wr, 16 op/s 2024-05-20T21:00:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:15 smithi144 ceph-mon[35660]: pgmap v512: 97 pgs: 97 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 135 KiB/s wr, 16 op/s 2024-05-20T21:00:15.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 68658 0.67 MB/sec execute 21 sec latency 18.582 ms 2024-05-20T21:00:16.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 69385 0.79 MB/sec execute 22 sec latency 17.928 ms 2024-05-20T21:00:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:17 smithi055 ceph-mon[35213]: pgmap v513: 97 pgs: 97 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 292 KiB/s wr, 16 op/s 2024-05-20T21:00:17.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:17 smithi026 ceph-mon[31990]: pgmap v513: 97 pgs: 97 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 292 KiB/s wr, 16 op/s 2024-05-20T21:00:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:17 smithi144 ceph-mon[35660]: pgmap v513: 97 pgs: 97 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 292 KiB/s wr, 16 op/s 2024-05-20T21:00:17.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 70008 0.87 MB/sec execute 23 sec latency 18.391 ms 2024-05-20T21:00:18.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 70748 1.10 MB/sec execute 24 sec latency 17.988 ms 2024-05-20T21:00:19.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:19 smithi055 ceph-mon[35213]: pgmap v514: 97 pgs: 97 active+clean; 26 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 511 KiB/s wr, 42 op/s 2024-05-20T21:00:19.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:19 smithi026 ceph-mon[31990]: pgmap v514: 97 pgs: 97 active+clean; 26 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 511 KiB/s wr, 42 op/s 2024-05-20T21:00:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:19 smithi144 ceph-mon[35660]: pgmap v514: 97 pgs: 97 active+clean; 26 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 511 KiB/s wr, 42 op/s 2024-05-20T21:00:19.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 71353 1.24 MB/sec execute 25 sec latency 15.347 ms 2024-05-20T21:00:20.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 71902 1.23 MB/sec execute 26 sec latency 20.649 ms 2024-05-20T21:00:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 -- 172.21.15.26:0/2909292416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84104670 msgr2=0x7f2b8410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2909292416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84104670 0x7f2b8410af00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009be0 tx=0x7f2b7c02a6c0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 -- 172.21.15.26:0/2909292416 shutdown_connections 2024-05-20T21:00:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2909292416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84104670 0x7f2b8410af00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2909292416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2909292416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b84102ac0 0x7f2b84102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 -- 172.21.15.26:0/2909292416 >> 172.21.15.26:0/2909292416 conn(0x7f2b840fe030 msgr2=0x7f2b84100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 -- 172.21.15.26:0/2909292416 shutdown_connections 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 -- 172.21.15.26:0/2909292416 wait complete. 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 Processor -- start 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.153+0000 7f2b8995f640 1 -- start start 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b8995f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84102ac0 0x7f2b84198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b8995f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b8995f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b84104670 0x7f2b841a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b8995f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410e4a0 con 0x7f2b84103cb0 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b8995f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410e320 con 0x7f2b84102ac0 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b8995f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410e620 con 0x7f2b84104670 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45912/0 (socket says 172.21.15.26:45912) 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 -- 172.21.15.26:0/2361999990 learned_addr learned my addr 172.21.15.26:0/2361999990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84102ac0 0x7f2b84198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b837fe640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b84104670 0x7f2b841a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 -- 172.21.15.26:0/2361999990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b84104670 msgr2=0x7f2b841a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b84104670 0x7f2b841a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 -- 172.21.15.26:0/2361999990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84102ac0 msgr2=0x7f2b84198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84102ac0 0x7f2b84198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 -- 172.21.15.26:0/2361999990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b841a0720 con 0x7f2b84103cb0 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84102ac0 0x7f2b84198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.154+0000 7f2b827fc640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84199090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e9c0 tx=0x7f2b7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.155+0000 7f2b8895d640 1 -- 172.21.15.26:0/2361999990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800cde0 con 0x7f2b84103cb0 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.155+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b841a0a10 con 0x7f2b84103cb0 2024-05-20T21:00:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.155+0000 7f2b8895d640 1 -- 172.21.15.26:0/2361999990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b780035f0 con 0x7f2b84103cb0 2024-05-20T21:00:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.155+0000 7f2b8895d640 1 -- 172.21.15.26:0/2361999990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78010690 con 0x7f2b84103cb0 2024-05-20T21:00:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.155+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841a0f60 con 0x7f2b84103cb0 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.157+0000 7f2b8c67a740 1 Processor -- start 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.157+0000 7f2b8c67a740 1 -- start start 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.157+0000 7f2b8895d640 1 -- 172.21.15.26:0/2361999990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2b78004000 con 0x7f2b84103cb0 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.157+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.157+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559fbe158c30 con 0x559fbe1e80b0 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559fbe1487d0 con 0x559fbe1ed220 2024-05-20T21:00:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559fbe14c810 con 0x559fbe1e76c0 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b827fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1ecbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45918/0 (socket says 172.21.15.26:45918) 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3464054959 learned_addr learned my addr 172.21.15.26:0/3464054959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b837fe640 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3464054959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 msgr2=0x559fbe1e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3464054959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 msgr2=0x559fbe1ef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1ef6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3464054959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fbde5d210 con 0x559fbe1e80b0 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1ecbb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2b740114f0 tx=0x7f2b740119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925de654dc047540 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b8895d640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2b6c0b28a0 0x7f2b6c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.158+0000 7f2b69ffb640 1 -- 172.21.15.26:0/3464054959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7401e070 con 0x559fbe1e80b0 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b69ffb640 1 -- 172.21.15.26:0/3464054959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b740035f0 con 0x559fbe1e80b0 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b69ffb640 1 -- 172.21.15.26:0/3464054959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740044c0 con 0x559fbe1e80b0 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b837fe640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2b6c0b28a0 0x7f2b6c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8895d640 1 -- 172.21.15.26:0/2361999990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2b780e25c0 con 0x7f2b84103cb0 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3464054959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 msgr2=0x559fbe1ecbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1ecbb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2b740114f0 tx=0x7f2b740119b0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b837fe640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2b6c0b28a0 0x7f2b6c0b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x559fbe1e8910 tx=0x7f2b7c030860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3464054959 shutdown_connections 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1ecbb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3464054959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3464054959 >> 172.21.15.26:0/3464054959 conn(0x559fbe1e2d30 msgr2=0x559fbe1e3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3464054959 shutdown_connections 2024-05-20T21:00:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.159+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3464054959 wait complete. 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 -- start start 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 Processor -- start 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e76c0 0x559fbe1f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1ed220 0x559fbe1f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559fbde2e790 con 0x559fbe1e80b0 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559fbe104890 con 0x559fbe1e76c0 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559fbe1f01d0 con 0x559fbe1ed220 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b8c67a740 10 client.0 _ll_register_callbacks cb 0x559fbde5ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45932/0 (socket says 172.21.15.26:45932) 2024-05-20T21:00:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b827fc640 1 -- 172.21.15.26:0/3904174895 learned_addr learned my addr 172.21.15.26:0/3904174895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.160+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e76c0 0x559fbe1f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b837fe640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1ed220 0x559fbe1f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3904174895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1ed220 msgr2=0x559fbe1f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1ed220 0x559fbe1f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3904174895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 msgr2=0x559fbe1f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b82ffd640 1 -- 172.21.15.26:0/3904174895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fbdf29620 con 0x559fbe1e76c0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e76c0 0x559fbe1f1810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x559fbe1e3c00 tx=0x7f2b74004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cb2676d96bcd8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3904174895 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74026020 con 0x559fbe1e76c0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3904174895 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2b74004440 con 0x559fbe1e76c0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b740035f0 con 0x559fbe1e76c0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3904174895 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b74031050 con 0x559fbe1e76c0 2024-05-20T21:00:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b8c67a740 10 client.26171 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.161+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559fbe1fbd70 con 0x559fbe1e76c0 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3904174895 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b740035f0 con 0x559fbe1e76c0 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3904174895 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f2b74021030 con 0x559fbe1e76c0 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b8c67a740 10 client.26171 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b8c67a740 10 client.26171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b8c67a740 20 client.26171 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe1fd2a0 0x559fbe1ff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b8c67a740 4 client.26171 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fbe15ab80 con 0x559fbe1fd2a0 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b827fc640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe1fd2a0 0x559fbe1ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b827fc640 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe1fd2a0 0x559fbe1ff690 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9935c7f3cee0113b server_cookie=74d7404b6c20f86b in_seq=0 out_seq=0 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.162+0000 7f2b62ffd640 10 client.26171 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3904174895 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559fbe15ab80 con 0x559fbe1fd2a0 2024-05-20T21:00:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b62ffd640 10 client.26171 handle_command_reply: tid=0 2024-05-20T21:00:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 client.26171 shutdown 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e76c0 msgr2=0x559fbe1f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e76c0 0x559fbe1f1810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x559fbe1e3c00 tx=0x7f2b74004250 comp rx=0 tx=0).stop 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 shutdown_connections 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe1fd2a0 0x559fbe1ff690 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1ed220 0x559fbe1f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e80b0 0x559fbe1f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b62ffd640 0 client.26171 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3904174895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e76c0 0x559fbe1f1810 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.163+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 >> 172.21.15.26:0/3904174895 conn(0x559fbe1e2d30 msgr2=0x559fbe1e1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.164+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 shutdown_connections 2024-05-20T21:00:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.164+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3904174895 wait complete. 2024-05-20T21:00:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.164+0000 7f2b8c67a740 20 client.26171 trim_cache size 0 max 16384 2024-05-20T21:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.182+0000 7f2b8c67a740 1 Processor -- start 2024-05-20T21:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.182+0000 7f2b8c67a740 1 -- start start 2024-05-20T21:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.182+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 0x559fbe1e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.182+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e80b0 0x559fbe1f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.182+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841ac780 con 0x559fbe1e76c0 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841aa050 con 0x559fbe1ed220 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841a7ea0 con 0x559fbe1e80b0 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 0x559fbe1e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 0x559fbe1e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45938/0 (socket says 172.21.15.26:45938) 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 -- 172.21.15.26:0/328560197 learned_addr learned my addr 172.21.15.26:0/328560197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b827fc640 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e80b0 0x559fbe1f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 -- 172.21.15.26:0/328560197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e80b0 msgr2=0x559fbe1f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e80b0 0x559fbe1f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 -- 172.21.15.26:0/328560197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 msgr2=0x559fbe1dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 -- 172.21.15.26:0/328560197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fbe1efa70 con 0x559fbe1e76c0 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b837fe640 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 0x559fbe1e64d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009710 tx=0x7f2b74009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06e362f6599c0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b697fa640 1 -- 172.21.15.26:0/328560197 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74026070 con 0x559fbe1e76c0 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b697fa640 1 -- 172.21.15.26:0/328560197 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b74016070 con 0x559fbe1e76c0 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.183+0000 7f2b697fa640 1 -- 172.21.15.26:0/328560197 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7401e040 con 0x559fbe1e76c0 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 -- 172.21.15.26:0/328560197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 msgr2=0x559fbe1e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 0x559fbe1e64d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009710 tx=0x7f2b74009f60 comp rx=0 tx=0).stop 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 -- 172.21.15.26:0/328560197 shutdown_connections 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1ed220 0x559fbe1dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e80b0 0x559fbe1f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/328560197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1e76c0 0x559fbe1e64d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 -- 172.21.15.26:0/328560197 >> 172.21.15.26:0/328560197 conn(0x559fbe1e2d30 msgr2=0x559fbe1ec960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 -- 172.21.15.26:0/328560197 shutdown_connections 2024-05-20T21:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 -- 172.21.15.26:0/328560197 wait complete. 2024-05-20T21:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.184+0000 7f2b8c67a740 1 -- start start 2024-05-20T21:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 Processor -- start 2024-05-20T21:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e80b0 0x559fbe1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 0x559fbe1f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841a8510 con 0x559fbe1ed220 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841a9980 con 0x559fbe1e80b0 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841a8be0 con 0x559fbe1e76c0 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b8c67a740 10 client.0 _ll_register_callbacks cb 0x7f2b7401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 0x559fbe1f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b827fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e80b0 0x559fbe1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 0x559fbe1f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45946/0 (socket says 172.21.15.26:45946) 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 -- 172.21.15.26:0/3777516657 learned_addr learned my addr 172.21.15.26:0/3777516657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 -- 172.21.15.26:0/3777516657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 msgr2=0x559fbe1f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 -- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e80b0 msgr2=0x559fbe1f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e80b0 0x559fbe1f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.185+0000 7f2b837fe640 1 -- 172.21.15.26:0/3777516657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fbe265ec0 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b837fe640 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 0x559fbe1f5e50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c033000 tx=0x7f2b7c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d879d5d6ee8194 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3777516657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c051070 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3777516657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2b7c004450 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b74016070 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3777516657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c05d050 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b8c67a740 10 client.16665 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559fbe2671e0 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3777516657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b74016070 con 0x559fbe1ed220 2024-05-20T21:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3777516657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f2b7c03c040 con 0x559fbe1ed220 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b8c67a740 10 client.16665 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b8c67a740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.186+0000 7f2b8c67a740 20 client.16665 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.187+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe267c10 0x559fbe26a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.187+0000 7f2b8c67a740 4 client.16665 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.187+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fbe154360 con 0x559fbe267c10 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.187+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe267c10 0x559fbe26a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.187+0000 7f2b62ffd640 10 client.16665 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.187+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe267c10 0x559fbe26a030 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7626775e56359e41 server_cookie=b97425a98adb6aea in_seq=0 out_seq=0 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b62ffd640 1 -- 172.21.15.26:0/3777516657 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559fbe154360 con 0x559fbe267c10 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b62ffd640 10 client.16665 handle_command_reply: tid=0 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 client.16665 shutdown 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 msgr2=0x559fbe1f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 0x559fbe1f5e50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c033000 tx=0x7f2b7c02a6c0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 shutdown_connections 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559fbe267c10 0x559fbe26a030 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fbe1ed220 0x559fbe1f5e50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fbe1e80b0 0x559fbe1f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 --2- 172.21.15.26:0/3777516657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fbe1e76c0 0x559fbe1f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 >> 172.21.15.26:0/3777516657 conn(0x559fbe1e2d30 msgr2=0x559fbe1dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b62ffd640 0 client.16665 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 shutdown_connections 2024-05-20T21:00:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 1 -- 172.21.15.26:0/3777516657 wait complete. 2024-05-20T21:00:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.256+0000 7f2b8c67a740 20 client.16665 trim_cache size 0 max 16384 2024-05-20T21:00:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:00:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:00:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:00:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2b6c0b28a0 msgr2=0x7f2b6c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2b6c0b28a0 0x7f2b6c0b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x559fbe1e8910 tx=0x7f2b7c030860 comp rx=0 tx=0).stop 2024-05-20T21:00:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 msgr2=0x7f2b84199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84199090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e9c0 tx=0x7f2b7800ee80 comp rx=0 tx=0).stop 2024-05-20T21:00:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 shutdown_connections 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2b6c0b28a0 0x7f2b6c0b4d60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b84104670 0x7f2b841a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84103cb0 0x7f2b84199090 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 --2- 172.21.15.26:0/2361999990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b84102ac0 0x7f2b84198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 >> 172.21.15.26:0/2361999990 conn(0x7f2b840fe030 msgr2=0x7f2b84107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 shutdown_connections 2024-05-20T21:00:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:21.257+0000 7f2b8995f640 1 -- 172.21.15.26:0/2361999990 wait complete. 2024-05-20T21:00:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:21 smithi055 ceph-mon[35213]: pgmap v515: 97 pgs: 97 active+clean; 26 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 724 KiB/s wr, 47 op/s 2024-05-20T21:00:21.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:21 smithi026 ceph-mon[31990]: pgmap v515: 97 pgs: 97 active+clean; 26 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 724 KiB/s wr, 47 op/s 2024-05-20T21:00:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:21 smithi144 ceph-mon[35660]: pgmap v515: 97 pgs: 97 active+clean; 26 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 724 KiB/s wr, 47 op/s 2024-05-20T21:00:21.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 72575 1.34 MB/sec execute 27 sec latency 29.892 ms 2024-05-20T21:00:22.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 73333 1.36 MB/sec execute 28 sec latency 18.183 ms 2024-05-20T21:00:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:23 smithi055 ceph-mon[35213]: pgmap v516: 97 pgs: 97 active+clean; 27 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-05-20T21:00:23.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:23 smithi026 ceph-mon[31990]: pgmap v516: 97 pgs: 97 active+clean; 27 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-05-20T21:00:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:23 smithi144 ceph-mon[35660]: pgmap v516: 97 pgs: 97 active+clean; 27 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-05-20T21:00:23.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 74181 1.53 MB/sec execute 29 sec latency 17.542 ms 2024-05-20T21:00:24.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 74752 1.67 MB/sec execute 30 sec latency 18.161 ms 2024-05-20T21:00:25.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:25 smithi055 ceph-mon[35213]: pgmap v517: 97 pgs: 97 active+clean; 24 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-05-20T21:00:25.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:25 smithi026 ceph-mon[31990]: pgmap v517: 97 pgs: 97 active+clean; 24 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-05-20T21:00:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:25 smithi144 ceph-mon[35660]: pgmap v517: 97 pgs: 97 active+clean; 24 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-05-20T21:00:25.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 75165 1.62 MB/sec execute 31 sec latency 12.476 ms 2024-05-20T21:00:25.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:00:26.056 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:26.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 75750 1.64 MB/sec execute 32 sec latency 40.818 ms 2024-05-20T21:00:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:27 smithi055 ceph-mon[35213]: pgmap v518: 97 pgs: 97 active+clean; 24 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:00:27.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:27 smithi026 ceph-mon[31990]: pgmap v518: 97 pgs: 97 active+clean; 24 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:00:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:27 smithi144 ceph-mon[35660]: pgmap v518: 97 pgs: 97 active+clean; 24 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:00:27.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 76467 1.69 MB/sec execute 33 sec latency 18.390 ms 2024-05-20T21:00:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 -- 172.21.15.26:0/3538598319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 msgr2=0x7f9d0c109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3538598319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c109980 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009990 tx=0x7f9d08009ca0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 -- 172.21.15.26:0/3538598319 shutdown_connections 2024-05-20T21:00:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3538598319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c109980 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3538598319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d0c06c470 0x7f9d0c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3538598319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06bab0 0x7f9d0c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.912+0000 7f9d13de9700 1 -- 172.21.15.26:0/3538598319 >> 172.21.15.26:0/3538598319 conn(0x7f9d0c0793b0 msgr2=0x7f9d0c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.913+0000 7f9d13de9700 1 -- 172.21.15.26:0/3538598319 shutdown_connections 2024-05-20T21:00:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.913+0000 7f9d13de9700 1 -- 172.21.15.26:0/3538598319 wait complete. 2024-05-20T21:00:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.913+0000 7f9d13de9700 1 Processor -- start 2024-05-20T21:00:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.913+0000 7f9d13de9700 1 -- start start 2024-05-20T21:00:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d13de9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d0c06bab0 0x7f9d0c102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d13de9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06c470 0x7f9d0c102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d13de9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d13de9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c113580 con 0x7f9d0c1095a0 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d13de9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c113400 con 0x7f9d0c06c470 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d13de9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c113700 con 0x7f9d0c06bab0 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45966/0 (socket says 172.21.15.26:45966) 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 -- 172.21.15.26:0/3607733890 learned_addr learned my addr 172.21.15.26:0/3607733890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d11384700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06c470 0x7f9d0c102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d11b85700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d0c06bab0 0x7f9d0c102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 -- 172.21.15.26:0/3607733890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d0c06bab0 msgr2=0x7f9d0c102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d0c06bab0 0x7f9d0c102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 -- 172.21.15.26:0/3607733890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06c470 msgr2=0x7f9d0c102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06c470 0x7f9d0c102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 -- 172.21.15.26:0/3607733890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0c1a8000 con 0x7f9d0c1095a0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d11384700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06c470 0x7f9d0c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.914+0000 7f9d12386700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c102f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9d04007ee0 tx=0x7f9d0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.915+0000 7f9d02ffd700 1 -- 172.21.15.26:0/3607733890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004bb0 con 0x7f9d0c1095a0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.915+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0c1a82f0 con 0x7f9d0c1095a0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.915+0000 7f9d02ffd700 1 -- 172.21.15.26:0/3607733890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d04004d50 con 0x7f9d0c1095a0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.915+0000 7f9d02ffd700 1 -- 172.21.15.26:0/3607733890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040058b0 con 0x7f9d0c1095a0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.915+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0c1a8880 con 0x7f9d0c1095a0 2024-05-20T21:00:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.917+0000 7f9d02ffd700 1 -- 172.21.15.26:0/3607733890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9d0401a020 con 0x7f9d0c1095a0 2024-05-20T21:00:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.918+0000 7f9d02ffd700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9cf80b2880 0x7f9cf80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.918+0000 7f9d11b85700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9cf80b2880 0x7f9cf80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.918+0000 7f9d02ffd700 1 -- 172.21.15.26:0/3607733890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9d040e1820 con 0x7f9d0c1095a0 2024-05-20T21:00:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 Processor -- start 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d11b85700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9cf80b2880 0x7f9cf80b4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b2e0 tx=0x7f9d08005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 -- start start 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc43788450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc4378c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc4378cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dc436dec90 con 0x55dc4378c770 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dc436ce830 con 0x55dc43788a20 2024-05-20T21:00:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc436d2870 con 0x55dc43788070 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d12386700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc4378cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d11384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc4378c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.919+0000 7f9d12386700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc4378cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45974/0 (socket says 172.21.15.26:45974) 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d11384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc4378c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33412/0 (socket says 172.21.15.26:33412) 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 -- 172.21.15.26:0/1965571388 learned_addr learned my addr 172.21.15.26:0/1965571388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d11b85700 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc43788450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 -- 172.21.15.26:0/1965571388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 msgr2=0x55dc43788450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc43788450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 -- 172.21.15.26:0/1965571388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 msgr2=0x55dc4378c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc4378c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 -- 172.21.15.26:0/1965571388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4361e8d0 con 0x55dc4378c770 2024-05-20T21:00:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d11384700 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc4378c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9d12386700 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc4378cbb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9d040a4e60 tx=0x7f9d040a8730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e854c1e507bbfba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9cf7fff700 1 -- 172.21.15.26:0/1965571388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040ac070 con 0x55dc4378c770 2024-05-20T21:00:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9cf7fff700 1 -- 172.21.15.26:0/1965571388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d040a94e0 con 0x55dc4378c770 2024-05-20T21:00:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.920+0000 7f9cf7fff700 1 -- 172.21.15.26:0/1965571388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040a92a0 con 0x55dc4378c770 2024-05-20T21:00:27.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/1965571388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 msgr2=0x55dc4378cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc4378cbb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9d040a4e60 tx=0x7f9d040a8730 comp rx=0 tx=0).stop 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/1965571388 shutdown_connections 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc4378cbb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc43788450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/1965571388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc4378c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/1965571388 >> 172.21.15.26:0/1965571388 conn(0x55dc436ad080 msgr2=0x55dc43779e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/1965571388 shutdown_connections 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.921+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/1965571388 wait complete. 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 -- start start 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 Processor -- start 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc43793c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43790f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc4378c770 0x55dc43791460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dc435d4fa0 con 0x55dc43788070 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dc436707c0 con 0x55dc43788a20 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc4377fa80 con 0x55dc4378c770 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d1e67fb80 10 client.0 _ll_register_callbacks cb 0x55dc43669920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d11b85700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc43793c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d11b85700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc43793c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45978/0 (socket says 172.21.15.26:45978) 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d11b85700 1 -- 172.21.15.26:0/2992404089 learned_addr learned my addr 172.21.15.26:0/2992404089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d11384700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43790f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.922+0000 7f9d12386700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc4378c770 0x55dc43791460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11384700 1 -- 172.21.15.26:0/2992404089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc4378c770 msgr2=0x55dc43791460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11384700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc4378c770 0x55dc43791460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11384700 1 -- 172.21.15.26:0/2992404089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 msgr2=0x55dc43793c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11384700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc43793c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11384700 1 -- 172.21.15.26:0/2992404089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4366d690 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11b85700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc43793c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d11384700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43790f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c079b00 tx=0x7f9cfc00f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5051997cf922ae1e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9cf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2992404089 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc004030 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2992404089 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9cfc0041d0 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d040a94e0 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2992404089 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc0043d0 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d1e67fb80 10 client.26189 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.923+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc437a7a40 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2992404089 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d040a94e0 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2992404089 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f9cfc025050 con 0x55dc43788a20 2024-05-20T21:00:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d1e67fb80 10 client.26189 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d1e67fb80 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d1e67fb80 20 client.26189 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc437a9f10 0x55dc437ac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d1e67fb80 4 client.26189 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc4374b9e0 con 0x55dc437a9f10 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d11b85700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc437a9f10 0x55dc437ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9cf57fa700 10 client.26189 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.924+0000 7f9d11b85700 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc437a9f10 0x55dc437ac300 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ad2a46deb24124 server_cookie=47ec3ccead688f90 in_seq=0 out_seq=0 2024-05-20T21:00:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2992404089 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc4374b9e0 con 0x55dc437a9f10 2024-05-20T21:00:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9cf57fa700 10 client.26189 handle_command_reply: tid=0 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 client.26189 shutdown 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 msgr2=0x55dc43790f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43790f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c079b00 tx=0x7f9cfc00f3c0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 shutdown_connections 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc437a9f10 0x55dc437ac300 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc43793c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc4378c770 0x55dc43791460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9cf57fa700 0 client.26189 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2992404089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43790f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.925+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 >> 172.21.15.26:0/2992404089 conn(0x55dc436ad080 msgr2=0x55dc43779510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.926+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 shutdown_connections 2024-05-20T21:00:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.926+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2992404089 wait complete. 2024-05-20T21:00:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.926+0000 7f9d1e67fb80 20 client.26189 trim_cache size 0 max 16384 2024-05-20T21:00:27.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 Processor -- start 2024-05-20T21:00:27.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 -- start start 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc437b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788a20 0x55dc437b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 0x55dc4378b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1ae410 con 0x55dc43788070 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1abca0 con 0x55dc4378c770 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c19da10 con 0x55dc43788a20 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d11b85700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc437b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d12386700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 0x55dc4378b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d12386700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 0x55dc4378b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33416/0 (socket says 172.21.15.26:33416) 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.948+0000 7f9d12386700 1 -- 172.21.15.26:0/3162169784 learned_addr learned my addr 172.21.15.26:0/3162169784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:27.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11384700 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788a20 0x55dc437b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11b85700 1 -- 172.21.15.26:0/3162169784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788a20 msgr2=0x55dc437b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11b85700 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788a20 0x55dc437b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11b85700 1 -- 172.21.15.26:0/3162169784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 msgr2=0x55dc4378b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11b85700 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 0x55dc4378b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11b85700 1 -- 172.21.15.26:0/3162169784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc43779c90 con 0x55dc43788070 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d12386700 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 0x55dc4378b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9d11b85700 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc437b1b80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55dc437acac0 tx=0x7f9d0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf01cb3f218c657 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9cf77fe700 1 -- 172.21.15.26:0/3162169784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08034430 con 0x55dc43788070 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9cf77fe700 1 -- 172.21.15.26:0/3162169784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d08034a60 con 0x55dc43788070 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.949+0000 7f9cf77fe700 1 -- 172.21.15.26:0/3162169784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08025c50 con 0x55dc43788070 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/3162169784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 msgr2=0x55dc437b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc437b1b80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55dc437acac0 tx=0x7f9d0801f000 comp rx=0 tx=0).stop 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/3162169784 shutdown_connections 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc43788070 0x55dc437b1b80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc4378c770 0x55dc4378b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/3162169784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788a20 0x55dc437b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/3162169784 >> 172.21.15.26:0/3162169784 conn(0x55dc436ad080 msgr2=0x55dc4377ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/3162169784 shutdown_connections 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.950+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/3162169784 wait complete. 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 -- start start 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 Processor -- start 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc437863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43786920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc43784920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1aa160 con 0x55dc4378c770 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1ab5d0 con 0x55dc43788a20 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1aa830 con 0x55dc43788070 2024-05-20T21:00:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.951+0000 7f9d1e67fb80 10 client.0 _ll_register_callbacks cb 0x55dc43669920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:27.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43786920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11b85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc437863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43786920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33424/0 (socket says 172.21.15.26:33424) 2024-05-20T21:00:27.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 -- 172.21.15.26:0/2593951214 learned_addr learned my addr 172.21.15.26:0/2593951214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:27.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d12386700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc43784920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 -- 172.21.15.26:0/2593951214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 msgr2=0x55dc437863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc437863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 -- 172.21.15.26:0/2593951214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 msgr2=0x55dc43784920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc43784920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 -- 172.21.15.26:0/2593951214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc43785020 con 0x55dc43788a20 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d11384700 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43786920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55dc43772680 tx=0x7f9cfc009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=424df5ef0f63aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:27.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9cf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2593951214 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01c460 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.952+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d08034a60 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2593951214 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9cfc00f930 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2593951214 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc003930 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 10 client.26195 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc43815620 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2593951214 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d08034a60 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2593951214 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f9cfc025040 con 0x55dc43788a20 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 10 client.26195 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 20 client.26195 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc43815db0 0x55dc438181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:27.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 4 client.26195 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.953+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc436e0be0 con 0x55dc43815db0 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9d11b85700 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc43815db0 0x55dc438181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9cf57fa700 10 client.26195 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9d11b85700 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc43815db0 0x55dc438181d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4caca49f822689da server_cookie=8f7f85333c07f87a in_seq=0 out_seq=0 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9cf57fa700 1 -- 172.21.15.26:0/2593951214 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc436e0be0 con 0x55dc43815db0 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9cf57fa700 10 client.26195 handle_command_reply: tid=0 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9d1e67fb80 1 client.26195 shutdown 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 msgr2=0x55dc43786920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.954+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43786920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55dc43772680 tx=0x7f9cfc009e90 comp rx=0 tx=0).stop 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 shutdown_connections 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dc43815db0 0x55dc438181d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4378c770 0x55dc43784920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9cf57fa700 0 client.26195 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc43788070 0x55dc437863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 --2- 172.21.15.26:0/2593951214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc43788a20 0x55dc43786920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 >> 172.21.15.26:0/2593951214 conn(0x55dc436ad080 msgr2=0x55dc43792750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 shutdown_connections 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 1 -- 172.21.15.26:0/2593951214 wait complete. 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.955+0000 7f9d1e67fb80 20 client.26195 trim_cache size 0 max 16384 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:00:27.960 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9cf80b2880 msgr2=0x7f9cf80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9cf80b2880 0x7f9cf80b4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b2e0 tx=0x7f9d08005fb0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 msgr2=0x7f9d0c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c102f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9d04007ee0 tx=0x7f9d0400d3b0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 shutdown_connections 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9cf80b2880 0x7f9cf80b4d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d0c1095a0 0x7f9d0c102f80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d0c06bab0 0x7f9d0c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 --2- 172.21.15.26:0/3607733890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d0c06c470 0x7f9d0c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:27.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 >> 172.21.15.26:0/3607733890 conn(0x7f9d0c0793b0 msgr2=0x7f9d0c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:27.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 shutdown_connections 2024-05-20T21:00:27.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:27.956+0000 7f9d13de9700 1 -- 172.21.15.26:0/3607733890 wait complete. 2024-05-20T21:00:28.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 77124 1.71 MB/sec execute 34 sec latency 18.352 ms 2024-05-20T21:00:28.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:00:28.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:00:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:29.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:29 smithi026 ceph-mon[31990]: pgmap v519: 97 pgs: 97 active+clean; 26 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-05-20T21:00:29.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:29 smithi055 ceph-mon[35213]: pgmap v519: 97 pgs: 97 active+clean; 26 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-05-20T21:00:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:29 smithi144 ceph-mon[35660]: pgmap v519: 97 pgs: 97 active+clean; 26 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-05-20T21:00:29.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 77872 1.85 MB/sec execute 35 sec latency 18.656 ms 2024-05-20T21:00:30.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 78457 1.93 MB/sec execute 36 sec latency 12.107 ms 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 -- 172.21.15.26:0/583928848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 msgr2=0x7f185c110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 --2- 172.21.15.26:0/583928848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c110490 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f185400b0d0 tx=0x7f185400b3e0 comp rx=0 tx=0).stop 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 -- 172.21.15.26:0/583928848 shutdown_connections 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 --2- 172.21.15.26:0/583928848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c110490 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 --2- 172.21.15.26:0/583928848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 --2- 172.21.15.26:0/583928848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c1041d0 0x7f185c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 -- 172.21.15.26:0/583928848 >> 172.21.15.26:0/583928848 conn(0x7f185c07a6b0 msgr2=0x7f185c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 -- 172.21.15.26:0/583928848 shutdown_connections 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.915+0000 7f1862a13700 1 -- 172.21.15.26:0/583928848 wait complete. 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 Processor -- start 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 -- start start 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c1041d0 0x7f185c1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f185c113590 con 0x7f185c108910 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f185c113410 con 0x7f185c103820 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.916+0000 7f1862a13700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f185c113710 con 0x7f185c1041d0 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f185bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:30.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43032/0 (socket says 172.21.15.26:43032) 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f185bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c1a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43436/0 (socket says 172.21.15.26:43436) 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 -- 172.21.15.26:0/1762663048 learned_addr learned my addr 172.21.15.26:0/1762663048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1853fff700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c1041d0 0x7f185c1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 -- 172.21.15.26:0/1762663048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c1041d0 msgr2=0x7f185c1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c1041d0 0x7f185c1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 -- 172.21.15.26:0/1762663048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 msgr2=0x7f185c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 -- 172.21.15.26:0/1762663048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19c270 con 0x7f185c108910 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f185bfff700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1860fb0700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c19bb10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1854009850 tx=0x7f1854007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185401f070 con 0x7f185c108910 2024-05-20T21:00:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1854004c50 con 0x7f185c108910 2024-05-20T21:00:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185401b690 con 0x7f185c108910 2024-05-20T21:00:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.917+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19c500 con 0x7f185c108910 2024-05-20T21:00:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.918+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c1a8780 con 0x7f185c108910 2024-05-20T21:00:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.919+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c06a230 con 0x7f185c108910 2024-05-20T21:00:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.920+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1854022070 con 0x7f185c108910 2024-05-20T21:00:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.921+0000 7f1859ffb700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18480b2900 0x7f18480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.921+0000 7f185bfff700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18480b2900 0x7f18480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.921+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f18540e3110 con 0x7f185c108910 2024-05-20T21:00:30.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.922+0000 7f185bfff700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18480b2900 0x7f18480b4dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f185c1107d0 tx=0x7f184c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:30.925+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18540aaee0 con 0x7f185c108910 2024-05-20T21:00:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.237+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f185c1a8950 con 0x7f185c108910 2024-05-20T21:00:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.238+0000 7f1859ffb700 1 -- 172.21.15.26:0/1762663048 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f1854009cb0 con 0x7f185c108910 2024-05-20T21:00:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:00:31.239 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:00:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18480b2900 msgr2=0x7f18480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18480b2900 0x7f18480b4dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f185c1107d0 tx=0x7f184c009260 comp rx=0 tx=0).stop 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 msgr2=0x7f185c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c19bb10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1854009850 tx=0x7f1854007a20 comp rx=0 tx=0).stop 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 shutdown_connections 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18480b2900 0x7f18480b4dc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f185c108910 0x7f185c19bb10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f185c103820 0x7f185c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 --2- 172.21.15.26:0/1762663048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c1041d0 0x7f185c1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.240+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 >> 172.21.15.26:0/1762663048 conn(0x7f185c07a6b0 msgr2=0x7f185c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.241+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 shutdown_connections 2024-05-20T21:00:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:31.241+0000 7f1862a13700 1 -- 172.21.15.26:0/1762663048 wait complete. 2024-05-20T21:00:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:00:31.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:31 smithi055 ceph-mon[35213]: pgmap v520: 97 pgs: 97 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-05-20T21:00:31.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:31 smithi026 ceph-mon[31990]: pgmap v520: 97 pgs: 97 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-05-20T21:00:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:31 smithi144 ceph-mon[35660]: pgmap v520: 97 pgs: 97 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-05-20T21:00:31.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 78913 1.89 MB/sec execute 37 sec latency 18.604 ms 2024-05-20T21:00:31.998 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:00:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:32.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:32 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1762663048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:32.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:32 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1762663048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:32 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1762663048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:32.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 79616 1.96 MB/sec execute 38 sec latency 14.153 ms 2024-05-20T21:00:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:33 smithi055 ceph-mon[35213]: pgmap v521: 97 pgs: 97 active+clean; 26 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-05-20T21:00:33.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:33 smithi026 ceph-mon[31990]: pgmap v521: 97 pgs: 97 active+clean; 26 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-05-20T21:00:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:33 smithi144 ceph-mon[35660]: pgmap v521: 97 pgs: 97 active+clean; 26 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-05-20T21:00:33.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 80310 1.93 MB/sec execute 39 sec latency 18.483 ms 2024-05-20T21:00:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 -- 172.21.15.26:0/531729602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d0103600 msgr2=0x7fd1d0103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/531729602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d0103600 0x7fd1d0103a80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40099c0 tx=0x7fd1c4009cd0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 -- 172.21.15.26:0/531729602 shutdown_connections 2024-05-20T21:00:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/531729602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d01103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/531729602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d01095e0 0x7fd1d01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/531729602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d0103600 0x7fd1d0103a80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.147+0000 7fd1d56b5700 1 -- 172.21.15.26:0/531729602 >> 172.21.15.26:0/531729602 conn(0x7fd1d007a700 msgr2=0x7fd1d007ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.148+0000 7fd1d56b5700 1 -- 172.21.15.26:0/531729602 shutdown_connections 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.148+0000 7fd1d56b5700 1 -- 172.21.15.26:0/531729602 wait complete. 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.148+0000 7fd1d56b5700 1 Processor -- start 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 -- start start 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d0103600 0x7fd1d019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d01095e0 0x7fd1d0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01135f0 con 0x7fd1d01095e0 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d0113470 con 0x7fd1d0103fc0 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1d56b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d0113770 con 0x7fd1d0103600 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43456/0 (socket says 172.21.15.26:43456) 2024-05-20T21:00:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1661516176 learned_addr learned my addr 172.21.15.26:0/1661516176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d0103600 0x7fd1d019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1661516176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d0103600 msgr2=0x7fd1d019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d0103600 0x7fd1d019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d01095e0 msgr2=0x7fd1d0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.149+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d01095e0 0x7fd1d0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1661516176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d019a070 con 0x7fd1d0103fc0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d019fc90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4006010 tx=0x7fd1c40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1b7fff700 1 -- 172.21.15.26:0/1661516176 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4026070 con 0x7fd1d0103fc0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d019a300 con 0x7fd1d0103fc0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1b7fff700 1 -- 172.21.15.26:0/1661516176 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c400f460 con 0x7fd1d0103fc0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1b7fff700 1 -- 172.21.15.26:0/1661516176 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4009e60 con 0x7fd1d0103fc0 2024-05-20T21:00:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.150+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d0075770 con 0x7fd1d0103fc0 2024-05-20T21:00:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.153+0000 7fd1b7fff700 1 -- 172.21.15.26:0/1661516176 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd1c4029070 con 0x7fd1d0103fc0 2024-05-20T21:00:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.154+0000 7fd1b7fff700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1b80b2900 0x7fd1b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.154+0000 7fd1b7fff700 1 -- 172.21.15.26:0/1661516176 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd1c40e2a50 con 0x7fd1d0103fc0 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.154+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1b80b2900 0x7fd1b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.154+0000 7fd1dff4bb80 1 Processor -- start 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.154+0000 7fd1dff4bb80 1 -- start start 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.154+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1b80b2900 0x7fd1b80b4dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1d019aa50 tx=0x7fd1c000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dc070 0x55ee656dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee65632c90 con 0x55ee656e0770 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee65622830 con 0x55ee656dc070 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee65626870 con 0x55ee656dca20 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656e0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dc070 0x55ee656dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656e0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43046/0 (socket says 172.21.15.26:43046) 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 -- 172.21.15.26:0/764224572 learned_addr learned my addr 172.21.15.26:0/764224572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 -- 172.21.15.26:0/764224572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 msgr2=0x55ee656e0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 -- 172.21.15.26:0/764224572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dc070 msgr2=0x55ee656dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dc070 0x55ee656dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 -- 172.21.15.26:0/764224572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee655728d0 con 0x55ee656e0770 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1cf7fe700 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656e0bb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800e400 tx=0x7fd1c800e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80be31428a319453 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.155+0000 7fd1b57fa700 1 -- 172.21.15.26:0/764224572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8004bb0 con 0x55ee656e0770 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1b57fa700 1 -- 172.21.15.26:0/764224572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c8004d50 con 0x55ee656e0770 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1b57fa700 1 -- 172.21.15.26:0/764224572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8004320 con 0x55ee656e0770 2024-05-20T21:00:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/764224572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 msgr2=0x55ee656e0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656e0bb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800e400 tx=0x7fd1c800e710 comp rx=0 tx=0).stop 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/764224572 shutdown_connections 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656e0bb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dc070 0x55ee656dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/764224572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/764224572 >> 172.21.15.26:0/764224572 conn(0x55ee65601080 msgr2=0x55ee656cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.156+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/764224572 shutdown_connections 2024-05-20T21:00:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.157+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/764224572 wait complete. 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.157+0000 7fd1dff4bb80 1 -- start start 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.157+0000 7fd1dff4bb80 1 Processor -- start 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.157+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.157+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656e7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43054/0 (socket says 172.21.15.26:43054) 2024-05-20T21:00:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656e0770 0x55ee656e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 -- 172.21.15.26:0/3844712965 learned_addr learned my addr 172.21.15.26:0/3844712965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee65528fa0 con 0x55ee656dc070 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee655c47c0 con 0x55ee656e0770 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee656e88d0 con 0x55ee656dca20 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1dff4bb80 10 client.0 _ll_register_callbacks cb 0x55ee655bd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1cf7fe700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656e0770 0x55ee656e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 -- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 msgr2=0x55ee656e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 -- 172.21.15.26:0/3844712965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656e0770 msgr2=0x55ee656e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656e0770 0x55ee656e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ceffd700 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee655c1690 con 0x55ee656dc070 2024-05-20T21:00:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.158+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656e7c20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0003e80 tx=0x7fd1c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc1806f34aa4902 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/3844712965 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011ce0 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/3844712965 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd1c0028470 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c8004d50 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/3844712965 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1c0028670 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1dff4bb80 10 client.16698 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee656f3090 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.159+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/3844712965 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c8004d50 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/3844712965 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fd1c001f6c0 con 0x55ee656dc070 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1dff4bb80 10 client.16698 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1dff4bb80 10 client.16698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1dff4bb80 20 client.16698 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee656f5620 0x55ee656f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1dff4bb80 4 client.16698 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee6569f9e0 con 0x55ee656f5620 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee656f5620 0x55ee656f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee656f5620 0x55ee656f7a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27b6667c512bbfe0 server_cookie=fc336daf38025810 in_seq=0 out_seq=0 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.160+0000 7fd1b2ff5700 10 client.16698 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.161+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/3844712965 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ee6569f9e0 con 0x55ee656f5620 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.161+0000 7fd1b2ff5700 10 client.16698 handle_command_reply: tid=0 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.161+0000 7fd1dff4bb80 1 client.16698 shutdown 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 msgr2=0x55ee656e7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656e7c20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0003e80 tx=0x7fd1c00047c0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 shutdown_connections 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee656f5620 0x55ee656f7a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656e7c20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656e0770 0x55ee656e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/3844712965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dca20 0x55ee656e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1b2ff5700 0 client.16698 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 >> 172.21.15.26:0/3844712965 conn(0x55ee65601080 msgr2=0x55ee656cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 shutdown_connections 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/3844712965 wait complete. 2024-05-20T21:00:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.162+0000 7fd1dff4bb80 20 client.16698 trim_cache size 0 max 16384 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.179+0000 7fd1dff4bb80 1 Processor -- start 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.179+0000 7fd1dff4bb80 1 -- start start 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dc070 0x55ee656e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01ae2c0 con 0x55ee656e0770 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01abb50 con 0x55ee656dca20 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01a99a0 con 0x55ee656dc070 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43064/0 (socket says 172.21.15.26:43064) 2024-05-20T21:00:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1cf7fe700 1 -- 172.21.15.26:0/1334454784 learned_addr learned my addr 172.21.15.26:0/1334454784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dc070 0x55ee656e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1334454784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dc070 msgr2=0x55ee656e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dc070 0x55ee656e7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1334454784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 msgr2=0x55ee656fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1334454784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee656cf8e0 con 0x55ee656dca20 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1cf7fe700 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.180+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656cdc90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40a9d40 tx=0x7fd1c4009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975ab14a28f2c5d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1b4ff9700 1 -- 172.21.15.26:0/1334454784 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40aa040 con 0x55ee656dca20 2024-05-20T21:00:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1b4ff9700 1 -- 172.21.15.26:0/1334454784 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c40ae3f0 con 0x55ee656dca20 2024-05-20T21:00:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1b4ff9700 1 -- 172.21.15.26:0/1334454784 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40ae1a0 con 0x55ee656dca20 2024-05-20T21:00:34.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1334454784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 msgr2=0x55ee656cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656cdc90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40a9d40 tx=0x7fd1c4009d30 comp rx=0 tx=0).stop 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1334454784 shutdown_connections 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656e0770 0x55ee656fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656dc070 0x55ee656e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.181+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1334454784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656cdc90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1334454784 >> 172.21.15.26:0/1334454784 conn(0x55ee65601080 msgr2=0x55ee656db710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1334454784 shutdown_connections 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1334454784 wait complete. 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- start start 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 Processor -- start 2024-05-20T21:00:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656e0770 0x55ee656eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01aa010 con 0x55ee656dc070 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01ab480 con 0x55ee656dca20 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.182+0000 7fd1dff4bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01aa6e0 con 0x55ee656e0770 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1dff4bb80 10 client.0 _ll_register_callbacks cb 0x55ee655bd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656edee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43068/0 (socket says 172.21.15.26:43068) 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ceffd700 1 -- 172.21.15.26:0/1802925639 learned_addr learned my addr 172.21.15.26:0/1802925639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1cf7fe700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656e0770 0x55ee656eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656e0770 msgr2=0x55ee656eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656e0770 0x55ee656eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1802925639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 msgr2=0x55ee656edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 -- 172.21.15.26:0/1802925639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee656ebd50 con 0x55ee656dca20 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1ce7fc700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656eb2d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40a9fd0 tx=0x7fd1c4003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51bd6c948f52bade server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/1802925639 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40c9060 con 0x55ee656dca20 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/1802925639 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fd1c40ae110 con 0x55ee656dca20 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.183+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c40ae3f0 con 0x55ee656dca20 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/1802925639 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1c400f610 con 0x55ee656dca20 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 10 client.26219 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee6575fec0 con 0x55ee656dca20 2024-05-20T21:00:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/1802925639 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c40ae3f0 con 0x55ee656dca20 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/1802925639 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fd1c40a59b0 con 0x55ee656dca20 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 10 client.26219 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 10 client.26219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 20 client.26219 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee65760440 0x55ee65762860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 4 client.26219 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee65634be0 con 0x55ee65760440 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.184+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee65760440 0x55ee65762860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1ceffd700 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee65760440 0x55ee65762860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=727bd791f71dc3e1 server_cookie=6f8c607193efaff9 in_seq=0 out_seq=0 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1b2ff5700 10 client.26219 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1b2ff5700 1 -- 172.21.15.26:0/1802925639 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee65634be0 con 0x55ee65760440 2024-05-20T21:00:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1b2ff5700 10 client.26219 handle_command_reply: tid=0 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 client.26219 shutdown 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 msgr2=0x55ee656eb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656eb2d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40a9fd0 tx=0x7fd1c4003ef0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 shutdown_connections 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ee65760440 0x55ee65762860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee656dc070 0x55ee656edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee656e0770 0x55ee656eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1b2ff5700 0 client.26219 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 --2- 172.21.15.26:0/1802925639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee656dca20 0x55ee656eb2d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 >> 172.21.15.26:0/1802925639 conn(0x55ee65601080 msgr2=0x55ee656df0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 shutdown_connections 2024-05-20T21:00:34.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 1 -- 172.21.15.26:0/1802925639 wait complete. 2024-05-20T21:00:34.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.185+0000 7fd1dff4bb80 20 client.26219 trim_cache size 0 max 16384 2024-05-20T21:00:34.191 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1b80b2900 msgr2=0x7fd1b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1b80b2900 0x7fd1b80b4dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1d019aa50 tx=0x7fd1c000b3c0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 msgr2=0x7fd1d019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d019fc90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4006010 tx=0x7fd1c40045e0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 shutdown_connections 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd1b80b2900 0x7fd1b80b4dc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1d01095e0 0x7fd1d0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d0103600 0x7fd1d019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 --2- 172.21.15.26:0/1661516176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1d0103fc0 0x7fd1d019fc90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 >> 172.21.15.26:0/1661516176 conn(0x7fd1d007a700 msgr2=0x7fd1d01023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 shutdown_connections 2024-05-20T21:00:34.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:34.186+0000 7fd1d56b5700 1 -- 172.21.15.26:0/1661516176 wait complete. 2024-05-20T21:00:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 -- 172.21.15.26:0/1723009319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60102ac0 msgr2=0x7f4b60102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 --2- 172.21.15.26:0/1723009319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60102ac0 0x7f4b60102ee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009900 tx=0x7f4b5002a410 comp rx=0 tx=0).stop 2024-05-20T21:00:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 -- 172.21.15.26:0/1723009319 shutdown_connections 2024-05-20T21:00:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 --2- 172.21.15.26:0/1723009319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60104670 0x7f4b6010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 --2- 172.21.15.26:0/1723009319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b60103cb0 0x7f4b60104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 --2- 172.21.15.26:0/1723009319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60102ac0 0x7f4b60102ee0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 -- 172.21.15.26:0/1723009319 >> 172.21.15.26:0/1723009319 conn(0x7f4b600fe010 msgr2=0x7f4b60100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 -- 172.21.15.26:0/1723009319 shutdown_connections 2024-05-20T21:00:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.358+0000 7f4b652d9640 1 -- 172.21.15.26:0/1723009319 wait complete. 2024-05-20T21:00:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 Processor -- start 2024-05-20T21:00:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 -- start start 2024-05-20T21:00:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b60102ac0 0x7f4b60198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 0x7f4b60199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60104670 0x7f4b601a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6010e470 con 0x7f4b60104670 2024-05-20T21:00:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6010e2f0 con 0x7f4b60103cb0 2024-05-20T21:00:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b652d9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6010e5f0 con 0x7f4b60102ac0 2024-05-20T21:00:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 0x7f4b60199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60104670 0x7f4b601a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 0x7f4b60199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43490/0 (socket says 172.21.15.26:43490) 2024-05-20T21:00:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/2875056543 learned_addr learned my addr 172.21.15.26:0/2875056543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.359+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b60102ac0 0x7f4b60198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/2875056543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b60102ac0 msgr2=0x7f4b60198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b60102ac0 0x7f4b60198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60104670 msgr2=0x7f4b601a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60104670 0x7f4b601a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/2875056543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b601a0710 con 0x7f4b60103cb0 2024-05-20T21:00:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60104670 0x7f4b601a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 0x7f4b60199030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400c920 tx=0x7f4b5400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b47fff640 1 -- 172.21.15.26:0/2875056543 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54007c70 con 0x7f4b60103cb0 2024-05-20T21:00:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b601a0a00 con 0x7f4b60103cb0 2024-05-20T21:00:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b47fff640 1 -- 172.21.15.26:0/2875056543 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b54007e10 con 0x7f4b60103cb0 2024-05-20T21:00:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b47fff640 1 -- 172.21.15.26:0/2875056543 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540186f0 con 0x7f4b60103cb0 2024-05-20T21:00:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.360+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b601a0fa0 con 0x7f4b60103cb0 2024-05-20T21:00:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.362+0000 7f4b47fff640 1 -- 172.21.15.26:0/2875056543 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4b5400f450 con 0x7f4b60103cb0 2024-05-20T21:00:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 Processor -- start 2024-05-20T21:00:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 -- start start 2024-05-20T21:00:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17bc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5560c1725c30 con 0x5560c17b50b0 2024-05-20T21:00:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5560c17157d0 con 0x5560c17ba220 2024-05-20T21:00:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.363+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560c1719810 con 0x5560c17b46c0 2024-05-20T21:00:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17bc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17b9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43080/0 (socket says 172.21.15.26:43080) 2024-05-20T21:00:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 -- 172.21.15.26:0/2830872870 learned_addr learned my addr 172.21.15.26:0/2830872870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b47fff640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4b480b28a0 0x7f4b480b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4b480b28a0 0x7f4b480b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 -- 172.21.15.26:0/2830872870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 msgr2=0x5560c17b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 -- 172.21.15.26:0/2830872870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 msgr2=0x5560c17bc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17bc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 -- 172.21.15.26:0/2830872870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c142a210 con 0x5560c17b50b0 2024-05-20T21:00:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b47fff640 1 -- 172.21.15.26:0/2875056543 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f4b540e23c0 con 0x7f4b60103cb0 2024-05-20T21:00:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17b9bb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4b50000c00 tx=0x7f4b50031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dc5f627486275 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4b480b28a0 0x7f4b480b4d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5560c17b5910 tx=0x7f4b58004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.364+0000 7f4b457fa640 1 -- 172.21.15.26:0/2830872870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50035030 con 0x5560c17b50b0 2024-05-20T21:00:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b457fa640 1 -- 172.21.15.26:0/2830872870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b50040070 con 0x5560c17b50b0 2024-05-20T21:00:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b457fa640 1 -- 172.21.15.26:0/2830872870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50003850 con 0x5560c17b50b0 2024-05-20T21:00:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2830872870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 msgr2=0x5560c17b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17b9bb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4b50000c00 tx=0x7f4b50031580 comp rx=0 tx=0).stop 2024-05-20T21:00:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2830872870 shutdown_connections 2024-05-20T21:00:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17b9bb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2830872870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2830872870 >> 172.21.15.26:0/2830872870 conn(0x5560c17afd30 msgr2=0x5560c17b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2830872870 shutdown_connections 2024-05-20T21:00:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.365+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2830872870 wait complete. 2024-05-20T21:00:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 -- start start 2024-05-20T21:00:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 Processor -- start 2024-05-20T21:00:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 0x5560c17be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17ba220 0x5560c17c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5560c13fb790 con 0x5560c17b46c0 2024-05-20T21:00:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5560c16d1890 con 0x5560c17b50b0 2024-05-20T21:00:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560c17bd170 con 0x5560c17ba220 2024-05-20T21:00:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b67ff0740 10 client.0 _ll_register_callbacks cb 0x5560c1429a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 0x5560c17be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 0x5560c17be780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43088/0 (socket says 172.21.15.26:43088) 2024-05-20T21:00:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 -- 172.21.15.26:0/1113078942 learned_addr learned my addr 172.21.15.26:0/1113078942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17ba220 0x5560c17c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 -- 172.21.15.26:0/1113078942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17ba220 msgr2=0x5560c17c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17ba220 0x5560c17c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 -- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 msgr2=0x5560c17becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5effd640 1 -- 172.21.15.26:0/1113078942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c14f6620 con 0x5560c17b46c0 2024-05-20T21:00:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.366+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b5effd640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 0x5560c17be780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4b500057c0 tx=0x7f4b50031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c0fedaec3717a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b42ff5640 1 -- 172.21.15.26:0/1113078942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50051060 con 0x5560c17b46c0 2024-05-20T21:00:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b42ff5640 1 -- 172.21.15.26:0/1113078942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f4b5003c640 con 0x5560c17b46c0 2024-05-20T21:00:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b50040070 con 0x5560c17b46c0 2024-05-20T21:00:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b42ff5640 1 -- 172.21.15.26:0/1113078942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b50035070 con 0x5560c17b46c0 2024-05-20T21:00:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b67ff0740 10 client.16716 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560c17c8ce0 con 0x5560c17b46c0 2024-05-20T21:00:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b42ff5640 1 -- 172.21.15.26:0/1113078942 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b50040070 con 0x5560c17b46c0 2024-05-20T21:00:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b42ff5640 1 -- 172.21.15.26:0/1113078942 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f4b50055030 con 0x5560c17b46c0 2024-05-20T21:00:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.367+0000 7f4b67ff0740 10 client.16716 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b67ff0740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b67ff0740 20 client.16716 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c17cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b67ff0740 4 client.16716 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c1727b80 con 0x5560c17ca030 2024-05-20T21:00:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c17cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c17cc420 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c807a46dbb5cedd2 server_cookie=66fb97300318f5c1 in_seq=0 out_seq=0 2024-05-20T21:00:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.368+0000 7f4b42ff5640 10 client.16716 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b42ff5640 1 -- 172.21.15.26:0/1113078942 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5560c1727b80 con 0x5560c17ca030 2024-05-20T21:00:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b42ff5640 10 client.16716 handle_command_reply: tid=0 2024-05-20T21:00:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 client.16716 shutdown 2024-05-20T21:00:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 msgr2=0x5560c17be780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 0x5560c17be780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4b500057c0 tx=0x7f4b50031a70 comp rx=0 tx=0).stop 2024-05-20T21:00:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 shutdown_connections 2024-05-20T21:00:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c17cc420 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b42ff5640 0 client.16716 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17ba220 0x5560c17c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1113078942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b46c0 0x5560c17be780 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 >> 172.21.15.26:0/1113078942 conn(0x5560c17afd30 msgr2=0x5560c17aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 shutdown_connections 2024-05-20T21:00:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.369+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1113078942 wait complete. 2024-05-20T21:00:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.370+0000 7f4b67ff0740 20 client.16716 trim_cache size 0 max 16384 2024-05-20T21:00:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.387+0000 7f4b67ff0740 1 Processor -- start 2024-05-20T21:00:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.387+0000 7f4b67ff0740 1 -- start start 2024-05-20T21:00:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ac700 con 0x5560c17b50b0 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a9fd0 con 0x5560c17ba220 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a7e20 con 0x5560c17b46c0 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43104/0 (socket says 172.21.15.26:43104) 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/1665475613 learned_addr learned my addr 172.21.15.26:0/1665475613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5effd640 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/1665475613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 msgr2=0x5560c17bdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17bdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/1665475613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 msgr2=0x5560c17be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 -- 172.21.15.26:0/1665475613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c17bd2d0 con 0x5560c17b50b0 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.388+0000 7f4b5e7fc640 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17c5cc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4b540a59a0 tx=0x7f4b540e4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547d5cac12aa2afb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b44ff9640 1 -- 172.21.15.26:0/1665475613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5408ae60 con 0x5560c17b50b0 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b44ff9640 1 -- 172.21.15.26:0/1665475613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b540a9430 con 0x5560c17b50b0 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b44ff9640 1 -- 172.21.15.26:0/1665475613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a5bb0 con 0x5560c17b50b0 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1665475613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 msgr2=0x5560c17c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17c5cc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4b540a59a0 tx=0x7f4b540e4040 comp rx=0 tx=0).stop 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1665475613 shutdown_connections 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17ba220 0x5560c17be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17b50b0 0x5560c17c5cc0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/1665475613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17bdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1665475613 >> 172.21.15.26:0/1665475613 conn(0x5560c17afd30 msgr2=0x5560c17b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1665475613 shutdown_connections 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.389+0000 7f4b67ff0740 1 -- 172.21.15.26:0/1665475613 wait complete. 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 -- start start 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 Processor -- start 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 0x5560c1832ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a8490 con 0x5560c17ba220 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a9900 con 0x5560c17b50b0 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a8b60 con 0x5560c17b46c0 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b67ff0740 10 client.0 _ll_register_callbacks cb 0x7f4b540a5ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 0x5560c1832ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 0x5560c1832ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43106/0 (socket says 172.21.15.26:43106) 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 -- 172.21.15.26:0/2121019552 learned_addr learned my addr 172.21.15.26:0/2121019552 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43526/0 (socket says 172.21.15.26:43526) 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 -- 172.21.15.26:0/2121019552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 msgr2=0x5560c17c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 -- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 msgr2=0x5560c17c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.390+0000 7f4b5f7fe640 1 -- 172.21.15.26:0/2121019552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c17c5f20 con 0x5560c17ba220 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b5f7fe640 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 0x5560c1832ae0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4b58010b90 tx=0x7f4b58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eebe8a6283fdb98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b42ff5640 1 -- 172.21.15.26:0/2121019552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58004b20 con 0x5560c17ba220 2024-05-20T21:00:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b42ff5640 1 -- 172.21.15.26:0/2121019552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f4b58004cc0 con 0x5560c17ba220 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b540a9430 con 0x5560c17ba220 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b42ff5640 1 -- 172.21.15.26:0/2121019552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b580228f0 con 0x5560c17ba220 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b67ff0740 10 client.16722 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560c1834070 con 0x5560c17ba220 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b42ff5640 1 -- 172.21.15.26:0/2121019552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b540a9430 con 0x5560c17ba220 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.391+0000 7f4b42ff5640 1 -- 172.21.15.26:0/2121019552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f4b5801f040 con 0x5560c17ba220 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b67ff0740 10 client.16722 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b67ff0740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b67ff0740 20 client.16722 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c1836b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b67ff0740 4 client.16722 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560c1721360 con 0x5560c17ca030 2024-05-20T21:00:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c1836b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b42ff5640 10 client.16722 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.392+0000 7f4b5effd640 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c1836b30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25433546a9b12366 server_cookie=67c30ff4cf8e6cda in_seq=0 out_seq=0 2024-05-20T21:00:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b42ff5640 1 -- 172.21.15.26:0/2121019552 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560c1721360 con 0x5560c17ca030 2024-05-20T21:00:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b42ff5640 10 client.16722 handle_command_reply: tid=0 2024-05-20T21:00:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 client.16722 shutdown 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 msgr2=0x5560c1832ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 0x5560c1832ae0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4b58010b90 tx=0x7f4b58004a40 comp rx=0 tx=0).stop 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 shutdown_connections 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560c17ca030 0x5560c1836b30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c17ba220 0x5560c1832ae0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560c17b50b0 0x5560c17c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b42ff5640 0 client.16722 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 --2- 172.21.15.26:0/2121019552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c17b46c0 0x5560c17c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 >> 172.21.15.26:0/2121019552 conn(0x5560c17afd30 msgr2=0x5560c17b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 shutdown_connections 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 1 -- 172.21.15.26:0/2121019552 wait complete. 2024-05-20T21:00:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.430+0000 7f4b67ff0740 20 client.16722 trim_cache size 0 max 16384 2024-05-20T21:00:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:00:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:00:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:00:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4b480b28a0 msgr2=0x7f4b480b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4b480b28a0 0x7f4b480b4d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5560c17b5910 tx=0x7f4b58004ab0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 msgr2=0x7f4b60199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 0x7f4b60199030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400c920 tx=0x7f4b5400cde0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 shutdown_connections 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4b480b28a0 0x7f4b480b4d60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b60104670 0x7f4b601a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b60103cb0 0x7f4b60199030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 --2- 172.21.15.26:0/2875056543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b60102ac0 0x7f4b60198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 >> 172.21.15.26:0/2875056543 conn(0x7f4b600fe010 msgr2=0x7f4b60106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 shutdown_connections 2024-05-20T21:00:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:34.431+0000 7f4b652d9640 1 -- 172.21.15.26:0/2875056543 wait complete. 2024-05-20T21:00:34.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 81026 2.05 MB/sec execute 40 sec latency 52.979 ms 2024-05-20T21:00:34.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:00:35.154 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:35 smithi055 ceph-mon[35213]: pgmap v522: 97 pgs: 97 active+clean; 27 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:00:35.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:35 smithi026 ceph-mon[31990]: pgmap v522: 97 pgs: 97 active+clean; 27 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:00:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:35 smithi144 ceph-mon[35660]: pgmap v522: 97 pgs: 97 active+clean; 27 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:00:35.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 81652 2.12 MB/sec execute 41 sec latency 12.757 ms 2024-05-20T21:00:36.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 82156 2.11 MB/sec execute 42 sec latency 15.307 ms 2024-05-20T21:00:37.268 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:37 smithi026 ceph-mon[31990]: pgmap v523: 97 pgs: 97 active+clean; 28 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-05-20T21:00:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 -- 172.21.15.26:0/3770201064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 msgr2=0x7f35a01088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 --2- 172.21.15.26:0/3770201064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 0x7f35a01088a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3590009990 tx=0x7f3590009ca0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 -- 172.21.15.26:0/3770201064 shutdown_connections 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 --2- 172.21.15.26:0/3770201064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 --2- 172.21.15.26:0/3770201064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 0x7f35a01088a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 --2- 172.21.15.26:0/3770201064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0102c40 0x7f35a01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 -- 172.21.15.26:0/3770201064 >> 172.21.15.26:0/3770201064 conn(0x7f35a0078ab0 msgr2=0x7f35a00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.268+0000 7f35a579c700 1 -- 172.21.15.26:0/3770201064 shutdown_connections 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.269+0000 7f35a579c700 1 -- 172.21.15.26:0/3770201064 wait complete. 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.269+0000 7f35a579c700 1 Processor -- start 2024-05-20T21:00:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 -- start start 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0102c40 0x7f35a019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 0x7f35a01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01124a0 con 0x7f35a0103600 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0112320 con 0x7f35a01084c0 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f35a579c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0112620 con 0x7f35a0102c40 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43120/0 (socket says 172.21.15.26:43120) 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f359e7fc700 1 -- 172.21.15.26:0/3610684093 learned_addr learned my addr 172.21.15.26:0/3610684093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f359effd700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0102c40 0x7f35a019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.270+0000 7f359f7fe700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 0x7f35a01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f359e7fc700 1 -- 172.21.15.26:0/3610684093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0102c40 msgr2=0x7f35a019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f359e7fc700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0102c40 0x7f35a019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f359e7fc700 1 -- 172.21.15.26:0/3610684093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 msgr2=0x7f35a01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f359e7fc700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 0x7f35a01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f359e7fc700 1 -- 172.21.15.26:0/3610684093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a01a0da0 con 0x7f35a0103600 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f359e7fc700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a019c2d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f359400e9c0 tx=0x7f359400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f3587fff700 1 -- 172.21.15.26:0/3610684093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400cbd0 con 0x7f35a0103600 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a01a1090 con 0x7f35a0103600 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a01a8910 con 0x7f35a0103600 2024-05-20T21:00:37.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.271+0000 7f3587fff700 1 -- 172.21.15.26:0/3610684093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f359400cd70 con 0x7f35a0103600 2024-05-20T21:00:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.272+0000 7f3587fff700 1 -- 172.21.15.26:0/3610684093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594005540 con 0x7f35a0103600 2024-05-20T21:00:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.275+0000 7f3587fff700 1 -- 172.21.15.26:0/3610684093 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3594021440 con 0x7f35a0103600 2024-05-20T21:00:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.275+0000 7f3587fff700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35880b2900 0x7f35880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.276+0000 7f359effd700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35880b2900 0x7f35880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.276+0000 7f3587fff700 1 -- 172.21.15.26:0/3610684093 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f35940e2420 con 0x7f35a0103600 2024-05-20T21:00:37.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.276+0000 7f359effd700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35880b2900 0x7f35880b4dc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f359000b2e0 tx=0x7f3590005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 Processor -- start 2024-05-20T21:00:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 -- start start 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed95940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55869ecf0c90 con 0x55869ed91860 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55869ece0830 con 0x55869ed95f60 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.277+0000 7f35b0032b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55869ece4870 con 0x55869ed92210 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed91c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43134/0 (socket says 172.21.15.26:43134) 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 -- 172.21.15.26:0/1380105842 learned_addr learned my addr 172.21.15.26:0/1380105842 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359f7fe700 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359e7fc700 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed95940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 -- 172.21.15.26:0/1380105842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 msgr2=0x55869ed95940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed95940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 -- 172.21.15.26:0/1380105842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 msgr2=0x55869ed963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 -- 172.21.15.26:0/1380105842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869ec308d0 con 0x55869ed91860 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359f7fe700 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.278+0000 7f359effd700 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed91c40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3590015000 tx=0x7f359001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f19702a81c29a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.279+0000 7f35857fa700 1 -- 172.21.15.26:0/1380105842 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359000b6b0 con 0x55869ed91860 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.279+0000 7f35857fa700 1 -- 172.21.15.26:0/1380105842 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f359000bce0 con 0x55869ed91860 2024-05-20T21:00:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.279+0000 7f35857fa700 1 -- 172.21.15.26:0/1380105842 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590025cf0 con 0x55869ed91860 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 -- 172.21.15.26:0/1380105842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 msgr2=0x55869ed91c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed91c40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3590015000 tx=0x7f359001f000 comp rx=0 tx=0).stop 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 -- 172.21.15.26:0/1380105842 shutdown_connections 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed91c40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 --2- 172.21.15.26:0/1380105842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed95940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 -- 172.21.15.26:0/1380105842 >> 172.21.15.26:0/1380105842 conn(0x55869ecbf080 msgr2=0x55869ed8be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 -- 172.21.15.26:0/1380105842 shutdown_connections 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 -- 172.21.15.26:0/1380105842 wait complete. 2024-05-20T21:00:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 -- start start 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.280+0000 7f35b0032b80 1 Processor -- start 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869ed9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed95f60 0x55869ed9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55869ebe6fa0 con 0x55869ed91860 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55869ec827c0 con 0x55869ed92210 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55869ed9e160 con 0x55869ed95f60 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f35b0032b80 10 client.0 _ll_register_callbacks cb 0x55869ec7b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869ed9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869ed9a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43552/0 (socket says 172.21.15.26:43552) 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 -- 172.21.15.26:0/2353640103 learned_addr learned my addr 172.21.15.26:0/2353640103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359f7fe700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed95f60 0x55869ed9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 -- 172.21.15.26:0/2353640103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed95f60 msgr2=0x55869ed9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed95f60 0x55869ed9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 -- 172.21.15.26:0/2353640103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 msgr2=0x55869ed9d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.281+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f359e7fc700 1 -- 172.21.15.26:0/2353640103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869ec7f690 con 0x55869ed92210 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f359f7fe700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed95f60 0x55869ed9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869ed9a740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f359400f010 tx=0x7f35940aaf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5096d85e7c544014 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f3582ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f3582ff5700 1 -- 172.21.15.26:0/2353640103 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35940a60d0 con 0x55869ed92210 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f3582ff5700 1 -- 172.21.15.26:0/2353640103 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f35940a6f30 con 0x55869ed92210 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f359000bce0 con 0x55869ed92210 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f3582ff5700 1 -- 172.21.15.26:0/2353640103 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35940cc2e0 con 0x55869ed92210 2024-05-20T21:00:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f35b0032b80 10 client.26249 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.282+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55869eda89d0 con 0x55869ed92210 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f3582ff5700 1 -- 172.21.15.26:0/2353640103 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f359000bce0 con 0x55869ed92210 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f3582ff5700 1 -- 172.21.15.26:0/2353640103 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f35940a62b0 con 0x55869ed92210 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f35b0032b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f35b0032b80 10 client.26249 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f35b0032b80 20 client.26249 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869edab010 0x55869edad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f35b0032b80 4 client.26249 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55869ed5d9e0 con 0x55869edab010 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.283+0000 7f359effd700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869edab010 0x55869edad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.284+0000 7f3582ff5700 10 client.26249 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.284+0000 7f359effd700 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869edab010 0x55869edad400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d086056adb8bd95 server_cookie=70a888bc4eb1b7cb in_seq=0 out_seq=0 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.284+0000 7f3582ff5700 1 -- 172.21.15.26:0/2353640103 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55869ed5d9e0 con 0x55869edab010 2024-05-20T21:00:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.284+0000 7f3582ff5700 10 client.26249 handle_command_reply: tid=0 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.284+0000 7f35b0032b80 1 client.26249 shutdown 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 msgr2=0x55869ed9a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869ed9a740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f359400f010 tx=0x7f35940aaf50 comp rx=0 tx=0).stop 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 shutdown_connections 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869edab010 0x55869edad400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed95f60 0x55869ed9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f3582ff5700 0 client.26249 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2353640103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869ed9a740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 >> 172.21.15.26:0/2353640103 conn(0x55869ecbf080 msgr2=0x55869ed8b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 shutdown_connections 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 1 -- 172.21.15.26:0/2353640103 wait complete. 2024-05-20T21:00:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.285+0000 7f35b0032b80 20 client.26249 trim_cache size 0 max 16384 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 Processor -- start 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 -- start start 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed9c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f358c00fde0 con 0x55869ed91860 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f358c00d670 con 0x55869ed95f60 2024-05-20T21:00:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f35b0032b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f358c00b4c0 con 0x55869ed92210 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f359effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f359effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43156/0 (socket says 172.21.15.26:43156) 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f359f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f359effd700 1 -- 172.21.15.26:0/2598736637 learned_addr learned my addr 172.21.15.26:0/2598736637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.303+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed9c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359f7fe700 1 -- 172.21.15.26:0/2598736637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 msgr2=0x55869ed9c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359f7fe700 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed9c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359f7fe700 1 -- 172.21.15.26:0/2598736637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 msgr2=0x55869ed9acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359f7fe700 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359f7fe700 1 -- 172.21.15.26:0/2598736637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869ed9b6e0 con 0x55869ed95f60 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359effd700 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f359f7fe700 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed938b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3598008770 tx=0x7f3598009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411eb9680bc31975 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f3584ff9700 1 -- 172.21.15.26:0/2598736637 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359801b070 con 0x55869ed95f60 2024-05-20T21:00:37.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f3584ff9700 1 -- 172.21.15.26:0/2598736637 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3598017ce0 con 0x55869ed95f60 2024-05-20T21:00:37.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.304+0000 7f3584ff9700 1 -- 172.21.15.26:0/2598736637 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598007c80 con 0x55869ed95f60 2024-05-20T21:00:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 -- 172.21.15.26:0/2598736637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 msgr2=0x55869ed938b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed938b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3598008770 tx=0x7f3598009920 comp rx=0 tx=0).stop 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 -- 172.21.15.26:0/2598736637 shutdown_connections 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed91860 0x55869ed9acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed95f60 0x55869ed938b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2598736637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed92210 0x55869ed9c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 -- 172.21.15.26:0/2598736637 >> 172.21.15.26:0/2598736637 conn(0x55869ecbf080 msgr2=0x55869ed9bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 -- 172.21.15.26:0/2598736637 shutdown_connections 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 -- 172.21.15.26:0/2598736637 wait complete. 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 -- start start 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.305+0000 7f35b0032b80 1 Processor -- start 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed91860 0x55869eda3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869eda3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed95f60 0x55869ed9ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f358c00bb30 con 0x55869ed95f60 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f358c00cfa0 con 0x55869ed92210 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f358c00c200 con 0x55869ed91860 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f35b0032b80 10 client.0 _ll_register_callbacks cb 0x7f35940a6890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869eda3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed95f60 0x55869ed9ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869eda3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43584/0 (socket says 172.21.15.26:43584) 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 -- 172.21.15.26:0/2887010210 learned_addr learned my addr 172.21.15.26:0/2887010210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359effd700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed91860 0x55869eda3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 -- 172.21.15.26:0/2887010210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed91860 msgr2=0x55869eda3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed91860 0x55869eda3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 -- 172.21.15.26:0/2887010210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed95f60 msgr2=0x55869ed9ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed95f60 0x55869ed9ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359e7fc700 1 -- 172.21.15.26:0/2887010210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869ed9f520 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.306+0000 7f359f7fe700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed95f60 0x55869ed9ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f359e7fc700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869eda3ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35940a67a0 tx=0x7f35940a64e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99bcbe1c85d9f2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f3582ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f3582ff5700 1 -- 172.21.15.26:0/2887010210 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35940a60d0 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f3582ff5700 1 -- 172.21.15.26:0/2887010210 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f35940a9930 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3598017ce0 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f3582ff5700 1 -- 172.21.15.26:0/2887010210 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35940b8050 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 10 client.26255 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55869ee15870 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f3582ff5700 1 -- 172.21.15.26:0/2887010210 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3598017ce0 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f3582ff5700 1 -- 172.21.15.26:0/2887010210 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f35940afe80 con 0x55869ed92210 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 10 client.26255 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 20 client.26255 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869ee15d60 0x55869ee18180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 4 client.26255 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.307+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55869ecf2be0 con 0x55869ee15d60 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f359effd700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869ee15d60 0x55869ee18180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f3582ff5700 10 client.26255 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f359effd700 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869ee15d60 0x55869ee18180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88547cde2454098e server_cookie=30654d8a53f6fe7a in_seq=0 out_seq=0 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f3582ff5700 1 -- 172.21.15.26:0/2887010210 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55869ecf2be0 con 0x55869ee15d60 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f3582ff5700 10 client.26255 handle_command_reply: tid=0 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 client.26255 shutdown 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 msgr2=0x55869eda3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869eda3ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35940a67a0 tx=0x7f35940a64e0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 shutdown_connections 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55869ee15d60 0x55869ee18180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f3582ff5700 0 client.26255 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869ed95f60 0x55869ed9ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55869ed91860 0x55869eda3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.308+0000 7f35b0032b80 1 --2- 172.21.15.26:0/2887010210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55869ed92210 0x55869eda3ca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 >> 172.21.15.26:0/2887010210 conn(0x55869ecbf080 msgr2=0x55869ed9c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 shutdown_connections 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35b0032b80 1 -- 172.21.15.26:0/2887010210 wait complete. 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35b0032b80 20 client.26255 trim_cache size 0 max 16384 2024-05-20T21:00:37.312 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:00:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35880b2900 msgr2=0x7f35880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35880b2900 0x7f35880b4dc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f359000b2e0 tx=0x7f3590005fb0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 msgr2=0x7f35a019c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a019c2d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f359400e9c0 tx=0x7f359400ecd0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 shutdown_connections 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a01084c0 0x7f35a01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0102c40 0x7f35a019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35880b2900 0x7f35880b4dc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 --2- 172.21.15.26:0/3610684093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35a0103600 0x7f35a019c2d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.309+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 >> 172.21.15.26:0/3610684093 conn(0x7f35a0078ab0 msgr2=0x7f35a0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.310+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 shutdown_connections 2024-05-20T21:00:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:37.310+0000 7f35a579c700 1 -- 172.21.15.26:0/3610684093 wait complete. 2024-05-20T21:00:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:37 smithi144 ceph-mon[35660]: pgmap v523: 97 pgs: 97 active+clean; 28 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-05-20T21:00:37.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 82732 2.10 MB/sec execute 43 sec latency 34.502 ms 2024-05-20T21:00:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:37 smithi055 ceph-mon[35213]: pgmap v523: 97 pgs: 97 active+clean; 28 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-05-20T21:00:38.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:00:38.442 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:38.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 83444 2.13 MB/sec execute 44 sec latency 17.068 ms 2024-05-20T21:00:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:39 smithi144 ceph-mon[35660]: pgmap v524: 97 pgs: 97 active+clean; 28 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T21:00:39.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 84069 2.13 MB/sec execute 45 sec latency 17.691 ms 2024-05-20T21:00:39.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:39 smithi055 ceph-mon[35213]: pgmap v524: 97 pgs: 97 active+clean; 28 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T21:00:39.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:39 smithi026 ceph-mon[31990]: pgmap v524: 97 pgs: 97 active+clean; 28 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T21:00:40.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 -- 172.21.15.26:0/794132621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102440 msgr2=0x7fb0641028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/794132621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102440 0x7fb0641028c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009990 tx=0x7fb04c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 -- 172.21.15.26:0/794132621 shutdown_connections 2024-05-20T21:00:40.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/794132621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102e00 0x7fb06410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/794132621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb064108520 0x7fb064108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/794132621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102440 0x7fb0641028c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 -- 172.21.15.26:0/794132621 >> 172.21.15.26:0/794132621 conn(0x7fb0640793f0 msgr2=0x7fb064079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 -- 172.21.15.26:0/794132621 shutdown_connections 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.259+0000 7fb06a4c0700 1 -- 172.21.15.26:0/794132621 wait complete. 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 Processor -- start 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 -- start start 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102440 0x7fb06419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 0x7fb06419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb064108520 0x7fb0641a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb064112480 con 0x7fb064102e00 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb064112300 con 0x7fb064102440 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb06a4c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb064112600 con 0x7fb064108520 2024-05-20T21:00:40.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 0x7fb06419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 0x7fb06419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49268/0 (socket says 172.21.15.26:49268) 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb05bfff700 1 -- 172.21.15.26:0/811355186 learned_addr learned my addr 172.21.15.26:0/811355186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb063fff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102440 0x7fb06419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb068a5d700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb064108520 0x7fb0641a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.260+0000 7fb05bfff700 1 -- 172.21.15.26:0/811355186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb064108520 msgr2=0x7fb0641a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb05bfff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb064108520 0x7fb0641a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb05bfff700 1 -- 172.21.15.26:0/811355186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102440 msgr2=0x7fb06419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb05bfff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102440 0x7fb06419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb05bfff700 1 -- 172.21.15.26:0/811355186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0641a5f40 con 0x7fb064102e00 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb063fff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102440 0x7fb06419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb05bfff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 0x7fb06419aec0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb04c006010 tx=0x7fb04c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb061ffb700 1 -- 172.21.15.26:0/811355186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c027070 con 0x7fb064102e00 2024-05-20T21:00:40.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0641a6110 con 0x7fb064102e00 2024-05-20T21:00:40.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0641989a0 con 0x7fb064102e00 2024-05-20T21:00:40.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb061ffb700 1 -- 172.21.15.26:0/811355186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb04c00fb90 con 0x7fb064102e00 2024-05-20T21:00:40.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.261+0000 7fb061ffb700 1 -- 172.21.15.26:0/811355186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0173f0 con 0x7fb064102e00 2024-05-20T21:00:40.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.264+0000 7fb061ffb700 1 -- 172.21.15.26:0/811355186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb04c02a070 con 0x7fb064102e00 2024-05-20T21:00:40.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.265+0000 7fb061ffb700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0500b2880 0x7fb0500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.265+0000 7fb063fff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0500b2880 0x7fb0500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.265+0000 7fb061ffb700 1 -- 172.21.15.26:0/811355186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fb04c0e3190 con 0x7fb064102e00 2024-05-20T21:00:40.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.265+0000 7fb074d56b80 1 Processor -- start 2024-05-20T21:00:40.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.265+0000 7fb063fff700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0500b2880 0x7fb0500b4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb05400bb90 tx=0x7fb054005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:40.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.265+0000 7fb074d56b80 1 -- start start 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110eb320 0x55e2110eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb074d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e211043d70 con 0x55e2110e6260 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb074d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e211033910 con 0x55e2110e58a0 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb074d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e211037950 con 0x55e2110eb320 2024-05-20T21:00:40.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49282/0 (socket says 172.21.15.26:49282) 2024-05-20T21:00:40.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 -- 172.21.15.26:0/3309738274 learned_addr learned my addr 172.21.15.26:0/3309738274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:40.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb063fff700 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110eb320 0x55e2110eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb05bfff700 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 -- 172.21.15.26:0/3309738274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110eb320 msgr2=0x55e2110eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110eb320 0x55e2110eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 -- 172.21.15.26:0/3309738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 msgr2=0x55e2110e5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110e5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.266+0000 7fb068a5d700 1 -- 172.21.15.26:0/3309738274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e210fdcfb0 con 0x55e2110e6260 2024-05-20T21:00:40.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.267+0000 7fb068a5d700 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f2080 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb05c011660 tx=0x7fb05c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a59e9da00b7114 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.267+0000 7fb05a7fc700 1 -- 172.21.15.26:0/3309738274 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c019030 con 0x55e2110e6260 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.267+0000 7fb05a7fc700 1 -- 172.21.15.26:0/3309738274 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb05c0092e0 con 0x55e2110e6260 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.267+0000 7fb05a7fc700 1 -- 172.21.15.26:0/3309738274 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004890 con 0x55e2110e6260 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 -- 172.21.15.26:0/3309738274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 msgr2=0x55e2110f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f2080 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb05c011660 tx=0x7fb05c009920 comp rx=0 tx=0).stop 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 -- 172.21.15.26:0/3309738274 shutdown_connections 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f2080 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110eb320 0x55e2110eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3309738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110e5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 -- 172.21.15.26:0/3309738274 >> 172.21.15.26:0/3309738274 conn(0x55e211011240 msgr2=0x55e2110e03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 -- 172.21.15.26:0/3309738274 shutdown_connections 2024-05-20T21:00:40.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.268+0000 7fb074d56b80 1 -- 172.21.15.26:0/3309738274 wait complete. 2024-05-20T21:00:40.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.269+0000 7fb074d56b80 1 -- start start 2024-05-20T21:00:40.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.269+0000 7fb074d56b80 1 Processor -- start 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 0x55e2110ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e210f39fa0 con 0x55e2110eb320 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e210f9bda0 con 0x55e2110e58a0 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2110f26e0 con 0x55e2110e6260 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb074d56b80 10 client.0 _ll_register_callbacks cb 0x55e210fce950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 0x55e2110ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb063fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 0x55e2110ef0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49712/0 (socket says 172.21.15.26:49712) 2024-05-20T21:00:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb068a5d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 -- 172.21.15.26:0/1899625116 learned_addr learned my addr 172.21.15.26:0/1899625116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 -- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 msgr2=0x55e2110f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 -- 172.21.15.26:0/1899625116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 msgr2=0x55e2110ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 -- 172.21.15.26:0/1899625116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e210f838d0 con 0x55e2110e6260 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb05bfff700 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 0x55e2110ef0d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00b2e0 tx=0x7fb04c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=858f4f2770f06c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.270+0000 7fb0437fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb0437fe700 1 -- 172.21.15.26:0/1899625116 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0aa2f0 con 0x55e2110e6260 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb0437fe700 1 -- 172.21.15.26:0/1899625116 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fb04c0b3330 con 0x55e2110e6260 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c0092e0 con 0x55e2110e6260 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb0437fe700 1 -- 172.21.15.26:0/1899625116 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb04c0aaee0 con 0x55e2110e6260 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb074d56b80 10 client.26164 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:40.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e210fed760 con 0x55e2110e6260 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb0437fe700 1 -- 172.21.15.26:0/1899625116 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb05c0092e0 con 0x55e2110e6260 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb0437fe700 1 -- 172.21.15.26:0/1899625116 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fb04c005110 con 0x55e2110e6260 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb074d56b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb074d56b80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.271+0000 7fb074d56b80 20 client.26164 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.272+0000 7fb074d56b80 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e2110ff2b0 0x55e2111016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.272+0000 7fb074d56b80 4 client.26164 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.272+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e211045cc0 con 0x55e2110ff2b0 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.272+0000 7fb063fff700 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e2110ff2b0 0x55e2111016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.272+0000 7fb063fff700 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e2110ff2b0 0x55e2111016d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afd4ecf6c8d53d1 server_cookie=83069a53c0e3a894 in_seq=0 out_seq=0 2024-05-20T21:00:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.272+0000 7fb0437fe700 10 client.26164 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.273+0000 7fb0437fe700 1 -- 172.21.15.26:0/1899625116 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e211045cc0 con 0x55e2110ff2b0 2024-05-20T21:00:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.273+0000 7fb0437fe700 10 client.26164 handle_command_reply: tid=0 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.273+0000 7fb074d56b80 1 client.26164 shutdown 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 msgr2=0x55e2110ef0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 0x55e2110ef0d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00b2e0 tx=0x7fb04c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 shutdown_connections 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e2110ff2b0 0x55e2111016d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e58a0 0x55e2110f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 --2- 172.21.15.26:0/1899625116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e6260 0x55e2110ef0d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 >> 172.21.15.26:0/1899625116 conn(0x55e211011240 msgr2=0x55e2110dfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb0437fe700 0 client.26164 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 shutdown_connections 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 1 -- 172.21.15.26:0/1899625116 wait complete. 2024-05-20T21:00:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.274+0000 7fb074d56b80 20 client.26164 trim_cache size 0 max 16384 2024-05-20T21:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 Processor -- start 2024-05-20T21:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 -- start start 2024-05-20T21:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110fb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 0x55e2110ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641b2700 con 0x55e2110eb320 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641aff90 con 0x55e2110e6260 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb074d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641adde0 con 0x55e2110e58a0 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 0x55e2110ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb068a5d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 0x55e2110ee270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55586/0 (socket says 172.21.15.26:55586) 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 -- 172.21.15.26:0/3346083192 learned_addr learned my addr 172.21.15.26:0/3346083192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb063fff700 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 -- 172.21.15.26:0/3346083192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 msgr2=0x55e2110fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 -- 172.21.15.26:0/3346083192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 msgr2=0x55e2110e0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110e0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.292+0000 7fb05bfff700 1 -- 172.21.15.26:0/3346083192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2110f1a10 con 0x55e2110e6260 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.293+0000 7fb05bfff700 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 0x55e2110ee270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0e0fa0 tx=0x7fb04c0e0c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da0f7e54f6475a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.293+0000 7fb059ffb700 1 -- 172.21.15.26:0/3346083192 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0b10b0 con 0x55e2110e6260 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.293+0000 7fb059ffb700 1 -- 172.21.15.26:0/3346083192 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb04c0aa7e0 con 0x55e2110e6260 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.293+0000 7fb059ffb700 1 -- 172.21.15.26:0/3346083192 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c005110 con 0x55e2110e6260 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 -- 172.21.15.26:0/3346083192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 msgr2=0x55e2110ee270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 0x55e2110ee270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0e0fa0 tx=0x7fb04c0e0c80 comp rx=0 tx=0).stop 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 -- 172.21.15.26:0/3346083192 shutdown_connections 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110eb320 0x55e2110e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 --2- 172.21.15.26:0/3346083192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110e6260 0x55e2110ee270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 -- 172.21.15.26:0/3346083192 >> 172.21.15.26:0/3346083192 conn(0x55e211011240 msgr2=0x55e2110e8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 -- 172.21.15.26:0/3346083192 shutdown_connections 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 -- 172.21.15.26:0/3346083192 wait complete. 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 -- start start 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.294+0000 7fb074d56b80 1 Processor -- start 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110f7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110eb320 0x55e211105f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641ae450 con 0x55e2110e6260 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641af8c0 con 0x55e2110eb320 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641aeb20 con 0x55e2110e58a0 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb074d56b80 10 client.0 _ll_register_callbacks cb 0x55e210fce950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49302/0 (socket says 172.21.15.26:49302) 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb05bfff700 1 -- 172.21.15.26:0/2299711643 learned_addr learned my addr 172.21.15.26:0/2299711643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110eb320 0x55e211105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb063fff700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 -- 172.21.15.26:0/2299711643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 msgr2=0x55e2110f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 -- 172.21.15.26:0/2299711643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 msgr2=0x55e2110f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 -- 172.21.15.26:0/2299711643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2111064b0 con 0x55e2110eb320 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb068a5d700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110eb320 0x55e211105f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0041f0 tx=0x7fb05c0040a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f1675f87ae1c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.295+0000 7fb0437fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb0437fe700 1 -- 172.21.15.26:0/2299711643 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c005490 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c0aa7e0 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb0437fe700 1 -- 172.21.15.26:0/2299711643 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fb05c005630 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb0437fe700 1 -- 172.21.15.26:0/2299711643 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb05c032050 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb074d56b80 10 client.26273 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e211169530 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb0437fe700 1 -- 172.21.15.26:0/2299711643 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb04c0aa7e0 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.296+0000 7fb0437fe700 1 -- 172.21.15.26:0/2299711643 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fb05c027020 con 0x55e2110eb320 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb074d56b80 10 client.26273 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb074d56b80 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb074d56b80 20 client.26273 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb074d56b80 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e211169940 0x55e21116bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb074d56b80 4 client.26273 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e21103f4a0 con 0x55e211169940 2024-05-20T21:00:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb063fff700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e211169940 0x55e21116bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb0437fe700 10 client.26273 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb063fff700 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e211169940 0x55e21116bd30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93894839e5a6f1a5 server_cookie=977d3fdbe1e47087 in_seq=0 out_seq=0 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.297+0000 7fb0437fe700 1 -- 172.21.15.26:0/2299711643 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e21103f4a0 con 0x55e211169940 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb0437fe700 10 client.26273 handle_command_reply: tid=0 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 client.26273 shutdown 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110eb320 msgr2=0x55e211105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110eb320 0x55e211105f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0041f0 tx=0x7fb05c0040a0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 shutdown_connections 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e211169940 0x55e21116bd30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e2110eb320 0x55e211105f70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2110e58a0 0x55e2110f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 --2- 172.21.15.26:0/2299711643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2110e6260 0x55e2110f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 >> 172.21.15.26:0/2299711643 conn(0x55e211011240 msgr2=0x55e2110e95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb0437fe700 0 client.26273 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 shutdown_connections 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 1 -- 172.21.15.26:0/2299711643 wait complete. 2024-05-20T21:00:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.298+0000 7fb074d56b80 20 client.26273 trim_cache size 0 max 16384 2024-05-20T21:00:40.303 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0500b2880 msgr2=0x7fb0500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0500b2880 0x7fb0500b4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb05400bb90 tx=0x7fb054005d80 comp rx=0 tx=0).stop 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 msgr2=0x7fb06419aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 0x7fb06419aec0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb04c006010 tx=0x7fb04c0057c0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 shutdown_connections 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb064108520 0x7fb0641a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb064102440 0x7fb06419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0500b2880 0x7fb0500b4d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 --2- 172.21.15.26:0/811355186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb064102e00 0x7fb06419aec0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 >> 172.21.15.26:0/811355186 conn(0x7fb0640793f0 msgr2=0x7fb064100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 shutdown_connections 2024-05-20T21:00:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:40.299+0000 7fb06a4c0700 1 -- 172.21.15.26:0/811355186 wait complete. 2024-05-20T21:00:40.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 84931 2.24 MB/sec execute 46 sec latency 12.691 ms 2024-05-20T21:00:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:41 smithi144 ceph-mon[35660]: pgmap v525: 97 pgs: 97 active+clean; 29 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:00:41.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 85468 2.28 MB/sec execute 47 sec latency 18.561 ms 2024-05-20T21:00:41.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:41 smithi055 ceph-mon[35213]: pgmap v525: 97 pgs: 97 active+clean; 29 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:00:41.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:41 smithi026 ceph-mon[31990]: pgmap v525: 97 pgs: 97 active+clean; 29 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:00:42.099 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:00:42.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:00:42.370 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:42.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 85947 2.25 MB/sec execute 48 sec latency 19.533 ms 2024-05-20T21:00:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:43 smithi144 ceph-mon[35660]: pgmap v526: 97 pgs: 97 active+clean; 29 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-05-20T21:00:43.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 86561 2.30 MB/sec execute 49 sec latency 87.448 ms 2024-05-20T21:00:43.664 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:43 smithi026 ceph-mon[31990]: pgmap v526: 97 pgs: 97 active+clean; 29 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-05-20T21:00:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:43 smithi055 ceph-mon[35213]: pgmap v526: 97 pgs: 97 active+clean; 29 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-05-20T21:00:44.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 87099 2.26 MB/sec execute 50 sec latency 109.228 ms 2024-05-20T21:00:45.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:45 smithi026 ceph-mon[31990]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-20T21:00:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:45 smithi144 ceph-mon[35660]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-20T21:00:45.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 87679 2.26 MB/sec execute 51 sec latency 109.099 ms 2024-05-20T21:00:45.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:45 smithi055 ceph-mon[35213]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-20T21:00:46.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 88491 2.35 MB/sec execute 52 sec latency 16.776 ms 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 -- 172.21.15.26:0/947303847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806c470 msgr2=0x7f71881103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 --2- 172.21.15.26:0/947303847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806c470 0x7f71881103c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f718000b0a0 tx=0x7f718000b3b0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 -- 172.21.15.26:0/947303847 shutdown_connections 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 --2- 172.21.15.26:0/947303847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806c470 0x7f71881103c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 --2- 172.21.15.26:0/947303847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71881095a0 0x7f7188109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 --2- 172.21.15.26:0/947303847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f718806bab0 0x7f718806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.745+0000 7f718da2a700 1 -- 172.21.15.26:0/947303847 >> 172.21.15.26:0/947303847 conn(0x7f71880793b0 msgr2=0x7f71880797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.746+0000 7f718da2a700 1 -- 172.21.15.26:0/947303847 shutdown_connections 2024-05-20T21:00:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.746+0000 7f718da2a700 1 -- 172.21.15.26:0/947303847 wait complete. 2024-05-20T21:00:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.746+0000 7f718da2a700 1 Processor -- start 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.746+0000 7f718da2a700 1 -- start start 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f718da2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 0x7f71881a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f718da2a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f718806c470 0x7f71881a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f718da2a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71881095a0 0x7f718819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f718da2a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7188113560 con 0x7f718806bab0 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f718da2a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f71881133e0 con 0x7f718806c470 2024-05-20T21:00:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f718da2a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71881136e0 con 0x7f71881095a0 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 0x7f71881a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f71867fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f718806c470 0x7f71881a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 0x7f71881a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49318/0 (socket says 172.21.15.26:49318) 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 -- 172.21.15.26:0/3712100637 learned_addr learned my addr 172.21.15.26:0/3712100637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f71877fe700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71881095a0 0x7f718819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 -- 172.21.15.26:0/3712100637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71881095a0 msgr2=0x7f718819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71881095a0 0x7f718819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 -- 172.21.15.26:0/3712100637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f718806c470 msgr2=0x7f71881a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f718806c470 0x7f71881a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f7186ffd700 1 -- 172.21.15.26:0/3712100637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718819c1c0 con 0x7f718806bab0 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.747+0000 7f71867fc700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f718806c470 0x7f71881a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.748+0000 7f7186ffd700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 0x7f71881a18a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f717800e9a0 tx=0x7f717800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.748+0000 7f718ca28700 1 -- 172.21.15.26:0/3712100637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717800ca10 con 0x7f718806bab0 2024-05-20T21:00:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.748+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718819c450 con 0x7f718806bab0 2024-05-20T21:00:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.748+0000 7f718ca28700 1 -- 172.21.15.26:0/3712100637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f717800cbb0 con 0x7f718806bab0 2024-05-20T21:00:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.748+0000 7f718ca28700 1 -- 172.21.15.26:0/3712100637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178018680 con 0x7f718806bab0 2024-05-20T21:00:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.748+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71881a8770 con 0x7f718806bab0 2024-05-20T21:00:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.751+0000 7f718ca28700 1 -- 172.21.15.26:0/3712100637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7178010430 con 0x7f718806bab0 2024-05-20T21:00:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.752+0000 7f718ca28700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71700b2880 0x7f71700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.752+0000 7f71867fc700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71700b2880 0x7f71700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.752+0000 7f718ca28700 1 -- 172.21.15.26:0/3712100637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f71780e2410 con 0x7f718806bab0 2024-05-20T21:00:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.752+0000 7f71867fc700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71700b2880 0x7f71700b4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7188104490 tx=0x7f717c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 Processor -- start 2024-05-20T21:00:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 -- start start 2024-05-20T21:00:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585727780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x562585721da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x56258572e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56258567fdf0 con 0x562585721920 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56258566f990 con 0x5625857222e0 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71982c0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625856739d0 con 0x5625857273a0 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x562585721da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71877fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x56258572e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x562585721da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49322/0 (socket says 172.21.15.26:49322) 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71867fc700 1 -- 172.21.15.26:0/537667698 learned_addr learned my addr 172.21.15.26:0/537667698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f7186ffd700 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585727780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71877fe700 1 -- 172.21.15.26:0/537667698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 msgr2=0x562585727780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71877fe700 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585727780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71877fe700 1 -- 172.21.15.26:0/537667698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 msgr2=0x562585721da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71877fe700 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x562585721da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.753+0000 7f71877fe700 1 -- 172.21.15.26:0/537667698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562585619030 con 0x5625857222e0 2024-05-20T21:00:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.754+0000 7f71877fe700 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x56258572e100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f718000b070 tx=0x7f7180009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64acbfa60003077f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.754+0000 7f716dffb700 1 -- 172.21.15.26:0/537667698 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180020070 con 0x5625857222e0 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.754+0000 7f716dffb700 1 -- 172.21.15.26:0/537667698 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f718000bcb0 con 0x5625857222e0 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.754+0000 7f716dffb700 1 -- 172.21.15.26:0/537667698 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180007c20 con 0x5625857222e0 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 -- 172.21.15.26:0/537667698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 msgr2=0x56258572e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x56258572e100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f718000b070 tx=0x7f7180009410 comp rx=0 tx=0).stop 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 -- 172.21.15.26:0/537667698 shutdown_connections 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x56258572e100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585727780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 --2- 172.21.15.26:0/537667698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x562585721da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 -- 172.21.15.26:0/537667698 >> 172.21.15.26:0/537667698 conn(0x56258564d2c0 msgr2=0x56258571c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 -- 172.21.15.26:0/537667698 shutdown_connections 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 -- 172.21.15.26:0/537667698 wait complete. 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 -- start start 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.755+0000 7f71982c0b80 1 Processor -- start 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562585721920 0x56258572dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x56258572b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x56258572b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562585576020 con 0x5625857222e0 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5625855d7e20 con 0x562585721920 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56258572e700 con 0x5625857273a0 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71982c0b80 10 client.0 _ll_register_callbacks cb 0x56258560a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x56258572b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f7186ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562585721920 0x56258572dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x56258572b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49338/0 (socket says 172.21.15.26:49338) 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 -- 172.21.15.26:0/1237923961 learned_addr learned my addr 172.21.15.26:0/1237923961 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71877fe700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x56258572b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 -- 172.21.15.26:0/1237923961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 msgr2=0x56258572b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x56258572b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 -- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562585721920 msgr2=0x56258572dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562585721920 0x56258572dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f71867fc700 1 -- 172.21.15.26:0/1237923961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625855bf950 con 0x5625857222e0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.756+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562585721920 0x56258572dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f71867fc700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x56258572b090 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f717c01dee0 tx=0x7f717c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acebc498a1e7415 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f7166ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f7166ffd700 1 -- 172.21.15.26:0/1237923961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c016070 con 0x5625857222e0 2024-05-20T21:00:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f7166ffd700 1 -- 172.21.15.26:0/1237923961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f717c004020 con 0x5625857222e0 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718000bcb0 con 0x5625857222e0 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f7166ffd700 1 -- 172.21.15.26:0/1237923961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f717c004260 con 0x5625857222e0 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f71982c0b80 10 client.16770 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625856297e0 con 0x5625857222e0 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.757+0000 7f7166ffd700 1 -- 172.21.15.26:0/1237923961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f718000bcb0 con 0x5625857222e0 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f7166ffd700 1 -- 172.21.15.26:0/1237923961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f717c024030 con 0x5625857222e0 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f71982c0b80 10 client.16770 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f71982c0b80 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f71982c0b80 20 client.16770 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x56258573d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f71982c0b80 4 client.16770 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562585681d40 con 0x56258573b280 2024-05-20T21:00:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x56258573d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x56258573d670 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c0cece5f90cfb3 server_cookie=fb511ff6a8d27653 in_seq=0 out_seq=0 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.758+0000 7f7166ffd700 10 client.16770 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.759+0000 7f7166ffd700 1 -- 172.21.15.26:0/1237923961 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562585681d40 con 0x56258573b280 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.759+0000 7f7166ffd700 10 client.16770 handle_command_reply: tid=0 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.759+0000 7f71982c0b80 1 client.16770 shutdown 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 msgr2=0x56258572b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x56258572b090 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f717c01dee0 tx=0x7f717c0047c0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 shutdown_connections 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x56258573d670 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x56258572b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f7166ffd700 0 client.16770 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562585721920 0x56258572dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1237923961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x56258572b090 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 >> 172.21.15.26:0/1237923961 conn(0x56258564d2c0 msgr2=0x56258571bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 shutdown_connections 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 1 -- 172.21.15.26:0/1237923961 wait complete. 2024-05-20T21:00:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.760+0000 7f71982c0b80 20 client.16770 trim_cache size 0 max 16384 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.784+0000 7f71982c0b80 1 Processor -- start 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.784+0000 7f71982c0b80 1 -- start start 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x56258571c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x562585742f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585743480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71982c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71881b25e0 con 0x562585721920 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71982c0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f71881afe70 con 0x5625857222e0 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71982c0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71881adcc0 con 0x5625857273a0 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x56258571c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x56258571c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49350/0 (socket says 172.21.15.26:49350) 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 -- 172.21.15.26:0/1918016665 learned_addr learned my addr 172.21.15.26:0/1918016665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71867fc700 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x562585742f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71877fe700 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585743480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 -- 172.21.15.26:0/1918016665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 msgr2=0x562585743480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585743480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 -- 172.21.15.26:0/1918016665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 msgr2=0x562585742f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x562585742f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f7186ffd700 1 -- 172.21.15.26:0/1918016665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258572ae60 con 0x562585721920 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.785+0000 7f71877fe700 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585743480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.786+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x56258571c2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f71780a9900 tx=0x7f71780a93f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f06fcc36b764c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.786+0000 7f716d7fa700 1 -- 172.21.15.26:0/1918016665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780aa070 con 0x562585721920 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.786+0000 7f716d7fa700 1 -- 172.21.15.26:0/1918016665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f717808f070 con 0x562585721920 2024-05-20T21:00:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.786+0000 7f716d7fa700 1 -- 172.21.15.26:0/1918016665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780b25a0 con 0x562585721920 2024-05-20T21:00:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 -- 172.21.15.26:0/1918016665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 msgr2=0x56258571c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x56258571c2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f71780a9900 tx=0x7f71780a93f0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 -- 172.21.15.26:0/1918016665 shutdown_connections 2024-05-20T21:00:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562585721920 0x56258571c2f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625857273a0 0x562585743480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1918016665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857222e0 0x562585742f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 -- 172.21.15.26:0/1918016665 >> 172.21.15.26:0/1918016665 conn(0x56258564d2c0 msgr2=0x562585725660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 -- 172.21.15.26:0/1918016665 shutdown_connections 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 -- 172.21.15.26:0/1918016665 wait complete. 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 -- start start 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.787+0000 7f71982c0b80 1 Processor -- start 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562585721920 0x562585733f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x562585730920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857273a0 0x562585730e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71881ae330 con 0x5625857222e0 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f71881af7a0 con 0x5625857273a0 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71881aea00 con 0x562585721920 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71982c0b80 10 client.0 _ll_register_callbacks cb 0x56258560a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x562585730920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x562585730920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49356/0 (socket says 172.21.15.26:49356) 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 -- 172.21.15.26:0/1999400646 learned_addr learned my addr 172.21.15.26:0/1999400646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562585721920 0x562585733f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71877fe700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857273a0 0x562585730e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 -- 172.21.15.26:0/1999400646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562585721920 msgr2=0x562585733f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562585721920 0x562585733f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 -- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857273a0 msgr2=0x562585730e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857273a0 0x562585730e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 -- 172.21.15.26:0/1999400646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562585731590 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71877fe700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857273a0 0x562585730e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f71867fc700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x562585730920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f717c005c20 tx=0x7f717c00ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1189fde32ecee042 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.788+0000 7f7166ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f7166ffd700 1 -- 172.21.15.26:0/1999400646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c03c060 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f7166ffd700 1 -- 172.21.15.26:0/1999400646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f717c010910 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f717808f070 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f7166ffd700 1 -- 172.21.15.26:0/1999400646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f717c033040 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 10 client.16776 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625857a6d70 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f7166ffd700 1 -- 172.21.15.26:0/1999400646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f717808f070 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f7166ffd700 1 -- 172.21.15.26:0/1999400646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f717c040030 con 0x5625857222e0 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 10 client.16776 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 20 client.16776 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x5625857a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 4 client.16776 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.789+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56258567b520 con 0x56258573b280 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.790+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x5625857a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.790+0000 7f7166ffd700 10 client.16776 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.790+0000 7f7186ffd700 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x5625857a94f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fe32928f5d46b6 server_cookie=94a6bc5d4a98af8d in_seq=0 out_seq=0 2024-05-20T21:00:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f7166ffd700 1 -- 172.21.15.26:0/1999400646 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56258567b520 con 0x56258573b280 2024-05-20T21:00:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f7166ffd700 10 client.16776 handle_command_reply: tid=0 2024-05-20T21:00:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 client.16776 shutdown 2024-05-20T21:00:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 msgr2=0x562585730920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x562585730920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f717c005c20 tx=0x7f717c00ff80 comp rx=0 tx=0).stop 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 shutdown_connections 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56258573b280 0x5625857a94f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5625857273a0 0x562585730e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562585721920 0x562585733f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f7166ffd700 0 client.16776 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 --2- 172.21.15.26:0/1999400646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625857222e0 0x562585730920 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 >> 172.21.15.26:0/1999400646 conn(0x56258564d2c0 msgr2=0x562585720910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 shutdown_connections 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 1 -- 172.21.15.26:0/1999400646 wait complete. 2024-05-20T21:00:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.791+0000 7f71982c0b80 20 client.16776 trim_cache size 0 max 16384 2024-05-20T21:00:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:00:46.796 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:00:46.796 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "03968ab5-9c9f-4c53-9c34-88644ab3d6b3", 2024-05-20T21:00:46.796 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:00:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71700b2880 msgr2=0x7f71700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71700b2880 0x7f71700b4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7188104490 tx=0x7f717c00a380 comp rx=0 tx=0).stop 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 msgr2=0x7f71881a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 0x7f71881a18a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f717800e9a0 tx=0x7f717800ecb0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 shutdown_connections 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f71700b2880 0x7f71700b4d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718806bab0 0x7f71881a18a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71881095a0 0x7f718819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 --2- 172.21.15.26:0/3712100637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f718806c470 0x7f71881a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.792+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 >> 172.21.15.26:0/3712100637 conn(0x7f71880793b0 msgr2=0x7f718810e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.793+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 shutdown_connections 2024-05-20T21:00:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:46.793+0000 7f718da2a700 1 -- 172.21.15.26:0/3712100637 wait complete. 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 -- 172.21.15.26:0/3798183684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 msgr2=0x7f3d2c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 --2- 172.21.15.26:0/3798183684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c101dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009900 tx=0x7f3d1c02a410 comp rx=0 tx=0).stop 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 -- 172.21.15.26:0/3798183684 shutdown_connections 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 --2- 172.21.15.26:0/3798183684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c103560 0x7f3d2c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 --2- 172.21.15.26:0/3798183684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 --2- 172.21.15.26:0/3798183684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c101dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 -- 172.21.15.26:0/3798183684 >> 172.21.15.26:0/3798183684 conn(0x7f3d2c0fcf20 msgr2=0x7f3d2c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 -- 172.21.15.26:0/3798183684 shutdown_connections 2024-05-20T21:00:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.531+0000 7f3d32264640 1 -- 172.21.15.26:0/3798183684 wait complete. 2024-05-20T21:00:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 Processor -- start 2024-05-20T21:00:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 -- start start 2024-05-20T21:00:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d2c103560 0x7f3d2c1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10d360 con 0x7f3d2c102ba0 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10d1e0 con 0x7f3d2c1019b0 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.532+0000 7f3d32264640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10d4e0 con 0x7f3d2c103560 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49382/0 (socket says 172.21.15.26:49382) 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55666/0 (socket says 172.21.15.26:55666) 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 -- 172.21.15.26:0/247924135 learned_addr learned my addr 172.21.15.26:0/247924135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d30801640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d2c103560 0x7f3d2c1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 -- 172.21.15.26:0/247924135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d2c103560 msgr2=0x7f3d2c1a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d2c103560 0x7f3d2c1a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 -- 172.21.15.26:0/247924135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 msgr2=0x7f3d2c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 -- 172.21.15.26:0/247924135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c1a07d0 con 0x7f3d2c102ba0 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c1990f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000e9f0 tx=0x7f3d2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d297fa640 1 -- 172.21.15.26:0/247924135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000cd80 con 0x7f3d2c102ba0 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c1a0ac0 con 0x7f3d2c102ba0 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d297fa640 1 -- 172.21.15.26:0/247924135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d200108c0 con 0x7f3d2c102ba0 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d297fa640 1 -- 172.21.15.26:0/247924135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200187c0 con 0x7f3d2c102ba0 2024-05-20T21:00:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.533+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c1a1060 con 0x7f3d2c102ba0 2024-05-20T21:00:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 Processor -- start 2024-05-20T21:00:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 -- start start 2024-05-20T21:00:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d297fa640 1 -- 172.21.15.26:0/247924135 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3d20010430 con 0x7f3d2c102ba0 2024-05-20T21:00:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 0x555dbbf85ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf8b220 0x555dbbf8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555dbbef6c30 con 0x555dbbf8b220 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555dbbee67d0 con 0x555dbbf856c0 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555dbbeea810 con 0x555dbbf860b0 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.536+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf8b220 0x555dbbf8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 0x555dbbf85ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 0x555dbbf85ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55668/0 (socket says 172.21.15.26:55668) 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 -- 172.21.15.26:0/787029729 learned_addr learned my addr 172.21.15.26:0/787029729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 -- 172.21.15.26:0/787029729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 msgr2=0x555dbbf8abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 -- 172.21.15.26:0/787029729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf8b220 msgr2=0x555dbbf8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf8b220 0x555dbbf8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 -- 172.21.15.26:0/787029729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dbbbfb210 con 0x555dbbf856c0 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf8b220 0x555dbbf8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d30801640 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 0x555dbbf85ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009fd0 tx=0x7f3d240118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dff01fe49bc818c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d297fa640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3d140b28a0 0x7f3d140b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d127fc640 1 -- 172.21.15.26:0/787029729 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24004d20 con 0x555dbbf856c0 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d127fc640 1 -- 172.21.15.26:0/787029729 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d24004050 con 0x555dbbf856c0 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d127fc640 1 -- 172.21.15.26:0/787029729 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24005260 con 0x555dbbf856c0 2024-05-20T21:00:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3d140b28a0 0x7f3d140b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.537+0000 7f3d297fa640 1 -- 172.21.15.26:0/247924135 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f3d200e2600 con 0x7f3d2c102ba0 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 -- 172.21.15.26:0/787029729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 msgr2=0x555dbbf85ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3d140b28a0 0x7f3d140b4d60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0057c0 tx=0x7f3d1c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 0x555dbbf85ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009fd0 tx=0x7f3d240118d0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 -- 172.21.15.26:0/787029729 shutdown_connections 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf8b220 0x555dbbf8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/787029729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf856c0 0x555dbbf85ae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 -- 172.21.15.26:0/787029729 >> 172.21.15.26:0/787029729 conn(0x555dbbf80d30 msgr2=0x555dbbf81130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 -- 172.21.15.26:0/787029729 shutdown_connections 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.538+0000 7f3d34f7e740 1 -- 172.21.15.26:0/787029729 wait complete. 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 -- start start 2024-05-20T21:00:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 Processor -- start 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555dbbbcc790 con 0x555dbbf856c0 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555dbbea2890 con 0x555dbbf8b220 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555dbbf8e1d0 con 0x555dbbf860b0 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d34f7e740 10 client.0 _ll_register_callbacks cb 0x555dbbbfaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf8f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49398/0 (socket says 172.21.15.26:49398) 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d2bfff640 1 -- 172.21.15.26:0/3059053432 learned_addr learned my addr 172.21.15.26:0/3059053432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.539+0000 7f3d30801640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d2bfff640 1 -- 172.21.15.26:0/3059053432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 msgr2=0x555dbbf8fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d2bfff640 1 -- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 msgr2=0x555dbbf96d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf96d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d2bfff640 1 -- 172.21.15.26:0/3059053432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dbbcc7620 con 0x555dbbf856c0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf8f7b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006010 tx=0x7f3d1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12db8669493d2bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/3059053432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c035040 con 0x555dbbf856c0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d24004050 con 0x555dbbf856c0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/3059053432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f3d1c004300 con 0x555dbbf856c0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/3059053432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c0035f0 con 0x555dbbf856c0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d34f7e740 10 client.16788 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555dbbf99d80 con 0x555dbbf856c0 2024-05-20T21:00:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.540+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/3059053432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d24004050 con 0x555dbbf856c0 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/3059053432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f3d1c044070 con 0x555dbbf856c0 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d34f7e740 10 client.16788 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d34f7e740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d34f7e740 20 client.16788 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbbf9b4c0 0x555dbbf9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d34f7e740 4 client.16788 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dbbef8b80 con 0x555dbbf9b4c0 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbbf9b4c0 0x555dbbf9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbbf9b4c0 0x555dbbf9d8b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ebb6a9e09f3502 server_cookie=4190369dbfa3abe0 in_seq=0 out_seq=0 2024-05-20T21:00:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.541+0000 7f3d0b7fe640 10 client.16788 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/3059053432 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555dbbef8b80 con 0x555dbbf9b4c0 2024-05-20T21:00:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d0b7fe640 10 client.16788 handle_command_reply: tid=0 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 client.16788 shutdown 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 msgr2=0x555dbbf8f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf8f7b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006010 tx=0x7f3d1c009260 comp rx=0 tx=0).stop 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 shutdown_connections 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbbf9b4c0 0x555dbbf9d8b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf8fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d0b7fe640 0 client.16788 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/3059053432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf8f7b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 >> 172.21.15.26:0/3059053432 conn(0x555dbbf80d30 msgr2=0x555dbbf7ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 shutdown_connections 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 1 -- 172.21.15.26:0/3059053432 wait complete. 2024-05-20T21:00:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.542+0000 7f3d34f7e740 20 client.16788 trim_cache size 0 max 16384 2024-05-20T21:00:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:47 smithi144 ceph-mon[35660]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:00:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:47 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:00:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:47 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:00:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:47 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:00:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:47 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:00:47.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 89042 2.39 MB/sec execute 53 sec latency 18.419 ms 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 Processor -- start 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 -- start start 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf860b0 0x555dbbf96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf8b220 0x555dbbfa21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1ac7d0 con 0x555dbbf856c0 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1aa0a0 con 0x555dbbf860b0 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1a7ef0 con 0x555dbbf8b220 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf843a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49414/0 (socket says 172.21.15.26:49414) 2024-05-20T21:00:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 -- 172.21.15.26:0/1640897281 learned_addr learned my addr 172.21.15.26:0/1640897281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf860b0 0x555dbbf96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d30801640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf8b220 0x555dbbfa21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 -- 172.21.15.26:0/1640897281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf8b220 msgr2=0x555dbbfa21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf8b220 0x555dbbfa21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 -- 172.21.15.26:0/1640897281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf860b0 msgr2=0x555dbbf96cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf860b0 0x555dbbf96cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 -- 172.21.15.26:0/1640897281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dbbf8a5e0 con 0x555dbbf856c0 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d30801640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf8b220 0x555dbbfa21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf860b0 0x555dbbf96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.559+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf843a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009230 tx=0x7f3d1c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef87f806d13d0edc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d11ffb640 1 -- 172.21.15.26:0/1640897281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c035040 con 0x555dbbf856c0 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d11ffb640 1 -- 172.21.15.26:0/1640897281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d1c03d590 con 0x555dbbf856c0 2024-05-20T21:00:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d11ffb640 1 -- 172.21.15.26:0/1640897281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c003c30 con 0x555dbbf856c0 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 -- 172.21.15.26:0/1640897281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 msgr2=0x555dbbf843a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf843a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009230 tx=0x7f3d1c032da0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 -- 172.21.15.26:0/1640897281 shutdown_connections 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf8b220 0x555dbbfa21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf860b0 0x555dbbf96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/1640897281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf843a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 -- 172.21.15.26:0/1640897281 >> 172.21.15.26:0/1640897281 conn(0x555dbbf80d30 msgr2=0x555dbbf83c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 -- 172.21.15.26:0/1640897281 shutdown_connections 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.560+0000 7f3d34f7e740 1 -- 172.21.15.26:0/1640897281 wait complete. 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 -- start start 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 Processor -- start 2024-05-20T21:00:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf94bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1a8560 con 0x555dbbf856c0 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1a99d0 con 0x555dbbf8b220 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1a8c30 con 0x555dbbf860b0 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d34f7e740 10 client.0 _ll_register_callbacks cb 0x7f3d1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf94bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf94bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49428/0 (socket says 172.21.15.26:49428) 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.561+0000 7f3d2bfff640 1 -- 172.21.15.26:0/2519887756 learned_addr learned my addr 172.21.15.26:0/2519887756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d30801640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2bfff640 1 -- 172.21.15.26:0/2519887756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 msgr2=0x555dbbf95110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf95110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2bfff640 1 -- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 msgr2=0x555dbbf960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2bfff640 1 -- 172.21.15.26:0/2519887756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dbbf966c0 con 0x555dbbf856c0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d2bfff640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf94bd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c03d760 tx=0x7f3d1c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3200cc30defc5e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/2519887756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c035040 con 0x555dbbf856c0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/2519887756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f3d1c003970 con 0x555dbbf856c0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1c03d590 con 0x555dbbf856c0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/2519887756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c003b90 con 0x555dbbf856c0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d34f7e740 10 client.16794 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555dbc0066f0 con 0x555dbbf856c0 2024-05-20T21:00:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.562+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/2519887756 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d1c03d590 con 0x555dbbf856c0 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/2519887756 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f3d1c044040 con 0x555dbbf856c0 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d34f7e740 10 client.16794 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d34f7e740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d34f7e740 20 client.16794 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbc007040 0x555dbc009460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d34f7e740 4 client.16794 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555dbbef2360 con 0x555dbc007040 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbc007040 0x555dbc009460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d2b7fe640 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbc007040 0x555dbc009460 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4cef442aaf0220 server_cookie=458034e0321f36a3 in_seq=0 out_seq=0 2024-05-20T21:00:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.563+0000 7f3d0b7fe640 10 client.16794 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:47.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:47 smithi026 ceph-mon[31990]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:00:47.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:47 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:00:47.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:47 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:00:47.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:47 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:00:47.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:47 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:00:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.612+0000 7f3d0b7fe640 1 -- 172.21.15.26:0/2519887756 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555dbbef2360 con 0x555dbc007040 2024-05-20T21:00:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.612+0000 7f3d0b7fe640 10 client.16794 handle_command_reply: tid=0 2024-05-20T21:00:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 client.16794 shutdown 2024-05-20T21:00:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 msgr2=0x555dbbf94bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf94bd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c03d760 tx=0x7f3d1c0317b0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 shutdown_connections 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555dbc007040 0x555dbc009460 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555dbbf8b220 0x555dbbf960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d0b7fe640 0 client.16794 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dbbf860b0 0x555dbbf95110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 --2- 172.21.15.26:0/2519887756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555dbbf856c0 0x555dbbf94bd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 >> 172.21.15.26:0/2519887756 conn(0x555dbbf80d30 msgr2=0x555dbbf7f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 shutdown_connections 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 1 -- 172.21.15.26:0/2519887756 wait complete. 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.613+0000 7f3d34f7e740 20 client.16794 trim_cache size 0 max 16384 2024-05-20T21:00:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:00:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:00:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:00:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3d140b28a0 msgr2=0x7f3d140b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3d140b28a0 0x7f3d140b4d60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0057c0 tx=0x7f3d1c031400 comp rx=0 tx=0).stop 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 msgr2=0x7f3d2c1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c1990f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000e9f0 tx=0x7f3d2000eeb0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 shutdown_connections 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3d140b28a0 0x7f3d140b4d60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d2c103560 0x7f3d2c1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d2c102ba0 0x7f3d2c1990f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 --2- 172.21.15.26:0/247924135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1019b0 0x7f3d2c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 >> 172.21.15.26:0/247924135 conn(0x7f3d2c0fcf20 msgr2=0x7f3d2c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 shutdown_connections 2024-05-20T21:00:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:00:47.614+0000 7f3d32264640 1 -- 172.21.15.26:0/247924135 wait complete. 2024-05-20T21:00:47.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:00:47.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:47 smithi055 ceph-mon[35213]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:00:47.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:47 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:00:47.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:47 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:00:47.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:47 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:00:47.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:47 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:00:47.904 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:48.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 89517 2.35 MB/sec execute 54 sec latency 20.677 ms 2024-05-20T21:00:49.366 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:49 smithi026 ceph-mon[31990]: pgmap v529: 97 pgs: 97 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-05-20T21:00:49.501 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:49 smithi055 ceph-mon[35213]: pgmap v529: 97 pgs: 97 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-05-20T21:00:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:49 smithi144 ceph-mon[35660]: pgmap v529: 97 pgs: 97 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-05-20T21:00:49.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 90198 2.40 MB/sec execute 55 sec latency 16.041 ms 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 -- 172.21.15.26:0/2346030569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 msgr2=0x7fc4781103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 --2- 172.21.15.26:0/2346030569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc4781103d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc47000b0a0 tx=0x7fc47000b3b0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 -- 172.21.15.26:0/2346030569 shutdown_connections 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 --2- 172.21.15.26:0/2346030569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc4781103d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 --2- 172.21.15.26:0/2346030569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4781095b0 0x7fc478109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 --2- 172.21.15.26:0/2346030569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc47806b220 0x7fc47806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 -- 172.21.15.26:0/2346030569 >> 172.21.15.26:0/2346030569 conn(0x7fc4780793f0 msgr2=0x7fc478079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.783+0000 7fc47effc700 1 -- 172.21.15.26:0/2346030569 shutdown_connections 2024-05-20T21:00:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 -- 172.21.15.26:0/2346030569 wait complete. 2024-05-20T21:00:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 Processor -- start 2024-05-20T21:00:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 -- start start 2024-05-20T21:00:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc47806b220 0x7fc47819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc47819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4781095b0 0x7fc478199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781135f0 con 0x7fc47806bbe0 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc478113470 con 0x7fc4781095b0 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.784+0000 7fc47effc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc478113770 con 0x7fc47806b220 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc47819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc47d599700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4781095b0 0x7fc478199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc47819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32858/0 (socket says 172.21.15.26:32858) 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 -- 172.21.15.26:0/3654643874 learned_addr learned my addr 172.21.15.26:0/3654643874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc47806b220 0x7fc47819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 -- 172.21.15.26:0/3654643874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc47806b220 msgr2=0x7fc47819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc47806b220 0x7fc47819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 -- 172.21.15.26:0/3654643874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4781095b0 msgr2=0x7fc478199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4781095b0 0x7fc478199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 -- 172.21.15.26:0/3654643874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47819a0b0 con 0x7fc47806bbe0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc477fff700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc47819fcd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc464007d40 tx=0x7fc46400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.785+0000 7fc475ffb700 1 -- 172.21.15.26:0/3654643874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464007e30 con 0x7fc47806bbe0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.786+0000 7fc475ffb700 1 -- 172.21.15.26:0/3654643874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4640044b0 con 0x7fc47806bbe0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.786+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47819a340 con 0x7fc47806bbe0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.786+0000 7fc475ffb700 1 -- 172.21.15.26:0/3654643874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464004a90 con 0x7fc47806bbe0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.786+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc478102440 con 0x7fc47806bbe0 2024-05-20T21:00:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.788+0000 7fc475ffb700 1 -- 172.21.15.26:0/3654643874 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc464020020 con 0x7fc47806bbe0 2024-05-20T21:00:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.789+0000 7fc475ffb700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc4680b2880 0x7fc4680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.789+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc4680b2880 0x7fc4680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.789+0000 7fc475ffb700 1 -- 172.21.15.26:0/3654643874 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fc4640e58d0 con 0x7fc47806bbe0 2024-05-20T21:00:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc4680b2880 0x7fc4680b4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc47819aa90 tx=0x7fc46c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 Processor -- start 2024-05-20T21:00:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 -- start start 2024-05-20T21:00:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1570700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b156ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1577080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5616b14c8d70 con 0x5616b156a8a0 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5616b14b8910 con 0x5616b1570320 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc489892b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616b14bc950 con 0x5616b156b260 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b156ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b156ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32868/0 (socket says 172.21.15.26:32868) 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc477fff700 1 -- 172.21.15.26:0/2549546126 learned_addr learned my addr 172.21.15.26:0/2549546126 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.790+0000 7fc47cd98700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1570700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc47d599700 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1577080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc477fff700 1 -- 172.21.15.26:0/2549546126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 msgr2=0x5616b1577080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc477fff700 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1577080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc477fff700 1 -- 172.21.15.26:0/2549546126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 msgr2=0x5616b1570700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc477fff700 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1570700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc477fff700 1 -- 172.21.15.26:0/2549546126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b1461fb0 con 0x5616b156a8a0 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc47cd98700 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1570700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc477fff700 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b156ad20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc4640a8f10 tx=0x7fc4640ac6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ec00d348a5e1a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc462ffd700 1 -- 172.21.15.26:0/2549546126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46401d030 con 0x5616b156a8a0 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc462ffd700 1 -- 172.21.15.26:0/2549546126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4640ace90 con 0x5616b156a8a0 2024-05-20T21:00:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.791+0000 7fc462ffd700 1 -- 172.21.15.26:0/2549546126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4640b5ac0 con 0x5616b156a8a0 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 -- 172.21.15.26:0/2549546126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 msgr2=0x5616b156ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b156ad20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc4640a8f10 tx=0x7fc4640ac6f0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 -- 172.21.15.26:0/2549546126 shutdown_connections 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1577080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1570700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 --2- 172.21.15.26:0/2549546126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b156ad20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 -- 172.21.15.26:0/2549546126 >> 172.21.15.26:0/2549546126 conn(0x5616b1496240 msgr2=0x5616b15653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 -- 172.21.15.26:0/2549546126 shutdown_connections 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.792+0000 7fc489892b80 1 -- 172.21.15.26:0/2549546126 wait complete. 2024-05-20T21:00:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 -- start start 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 Processor -- start 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156a8a0 0x5616b1576dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b15740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1574610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5616b13befa0 con 0x5616b156b260 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5616b1420da0 con 0x5616b1570320 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616b15776e0 con 0x5616b156a8a0 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc489892b80 10 client.0 _ll_register_callbacks cb 0x5616b1453950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b15740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc47cd98700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156a8a0 0x5616b1576dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b15740d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32884/0 (socket says 172.21.15.26:32884) 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc477fff700 1 -- 172.21.15.26:0/1402841440 learned_addr learned my addr 172.21.15.26:0/1402841440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.793+0000 7fc47d599700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1574610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc47cd98700 1 -- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 msgr2=0x5616b1574610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc47cd98700 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1574610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc47cd98700 1 -- 172.21.15.26:0/1402841440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 msgr2=0x5616b15740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc47cd98700 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b15740d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc47cd98700 1 -- 172.21.15.26:0/1402841440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b14088d0 con 0x5616b156a8a0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc477fff700 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b15740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc47cd98700 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156a8a0 0x5616b1576dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00e4b0 tx=0x7fc46c004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa9e1ded5b232a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc457fff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc457fff700 1 -- 172.21.15.26:0/1402841440 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c025070 con 0x5616b156a8a0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4640ace90 con 0x5616b156a8a0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc457fff700 1 -- 172.21.15.26:0/1402841440 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fc46c015da0 con 0x5616b156a8a0 2024-05-20T21:00:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc457fff700 1 -- 172.21.15.26:0/1402841440 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc46c01b040 con 0x5616b156a8a0 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc489892b80 10 client.26209 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.794+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616b1472760 con 0x5616b156a8a0 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc457fff700 1 -- 172.21.15.26:0/1402841440 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4640ace90 con 0x5616b156a8a0 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc457fff700 1 -- 172.21.15.26:0/1402841440 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fc46c0065d0 con 0x5616b156a8a0 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc489892b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc489892b80 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc489892b80 20 client.26209 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc489892b80 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b1584330 0x5616b1586720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc489892b80 4 client.26209 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b14cacc0 con 0x5616b1584330 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.795+0000 7fc477fff700 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b1584330 0x5616b1586720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc457fff700 10 client.26209 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc477fff700 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b1584330 0x5616b1586720 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235013a0af31756a server_cookie=4f468e5f05411c60 in_seq=0 out_seq=0 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc457fff700 1 -- 172.21.15.26:0/1402841440 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5616b14cacc0 con 0x5616b1584330 2024-05-20T21:00:49.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc457fff700 10 client.26209 handle_command_reply: tid=0 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc489892b80 1 client.26209 shutdown 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156a8a0 msgr2=0x5616b1576dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.796+0000 7fc489892b80 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156a8a0 0x5616b1576dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00e4b0 tx=0x7fc46c004860 comp rx=0 tx=0).stop 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 shutdown_connections 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b1584330 0x5616b1586720 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1574610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc457fff700 0 client.26209 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156a8a0 0x5616b1576dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 --2- 172.21.15.26:0/1402841440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b15740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 >> 172.21.15.26:0/1402841440 conn(0x5616b1496240 msgr2=0x5616b1564af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 shutdown_connections 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 1 -- 172.21.15.26:0/1402841440 wait complete. 2024-05-20T21:00:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.797+0000 7fc489892b80 20 client.26209 trim_cache size 0 max 16384 2024-05-20T21:00:49.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 Processor -- start 2024-05-20T21:00:49.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 -- start start 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b1580740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1565270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1576010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781ae360 con 0x5616b156a8a0 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781abbf0 con 0x5616b1570320 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc489892b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781a9a40 con 0x5616b156b260 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b1580740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b1580740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32900/0 (socket says 172.21.15.26:32900) 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 -- 172.21.15.26:0/4231631820 learned_addr learned my addr 172.21.15.26:0/4231631820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47d599700 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1576010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc477fff700 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1565270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 -- 172.21.15.26:0/4231631820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 msgr2=0x5616b1565270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1565270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 -- 172.21.15.26:0/4231631820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 msgr2=0x5616b1576010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1576010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47cd98700 1 -- 172.21.15.26:0/4231631820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b1586d80 con 0x5616b156a8a0 2024-05-20T21:00:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.821+0000 7fc47d599700 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1576010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.822+0000 7fc47cd98700 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b1580740 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00fef0 tx=0x7fc46c00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84f7eb974d75ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.822+0000 7fc4627fc700 1 -- 172.21.15.26:0/4231631820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c03a020 con 0x5616b156a8a0 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.822+0000 7fc4627fc700 1 -- 172.21.15.26:0/4231631820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc46c013a00 con 0x5616b156a8a0 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.822+0000 7fc4627fc700 1 -- 172.21.15.26:0/4231631820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c025040 con 0x5616b156a8a0 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 -- 172.21.15.26:0/4231631820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 msgr2=0x5616b1580740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b1580740 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00fef0 tx=0x7fc46c00ff20 comp rx=0 tx=0).stop 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 -- 172.21.15.26:0/4231631820 shutdown_connections 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156a8a0 0x5616b1580740 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b1570320 0x5616b1576010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 --2- 172.21.15.26:0/4231631820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b156b260 0x5616b1565270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 -- 172.21.15.26:0/4231631820 >> 172.21.15.26:0/4231631820 conn(0x5616b1496240 msgr2=0x5616b1575900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 -- 172.21.15.26:0/4231631820 shutdown_connections 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 -- 172.21.15.26:0/4231631820 wait complete. 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 -- start start 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.823+0000 7fc489892b80 1 Processor -- start 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 0x5616b157c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b157ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b1570320 0x5616b15778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781aa0b0 con 0x5616b156b260 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781ab520 con 0x5616b156a8a0 2024-05-20T21:00:49.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781aa780 con 0x5616b1570320 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc489892b80 10 client.0 _ll_register_callbacks cb 0x5616b1453950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b157ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 0x5616b157c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc477fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b157ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32912/0 (socket says 172.21.15.26:32912) 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc477fff700 1 -- 172.21.15.26:0/3687255784 learned_addr learned my addr 172.21.15.26:0/3687255784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 0x5616b157c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:48286/0 (socket says 172.21.15.26:48286) 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47d599700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b1570320 0x5616b15778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 -- 172.21.15.26:0/3687255784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b1570320 msgr2=0x5616b15778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b1570320 0x5616b15778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 -- 172.21.15.26:0/3687255784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 msgr2=0x5616b157ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:00:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b157ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.824+0000 7fc47cd98700 1 -- 172.21.15.26:0/3687255784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b157d550 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc47cd98700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 0x5616b157c940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006970 tx=0x7fc46c013770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e25b112a0b5e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc457fff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc457fff700 1 -- 172.21.15.26:0/3687255784 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c045020 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc457fff700 1 -- 172.21.15.26:0/3687255784 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fc46c014730 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46c013a00 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc457fff700 1 -- 172.21.15.26:0/3687255784 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc46c03a030 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc489892b80 10 client.26327 fetch_fsmap learned FSMap version 28 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616b15ef530 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc457fff700 1 -- 172.21.15.26:0/3687255784 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc46c013a00 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.825+0000 7fc457fff700 1 -- 172.21.15.26:0/3687255784 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fc46c049030 con 0x5616b156a8a0 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 10 client.26327 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:00:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 20 client.26327 populate_metadata read hostname 'smithi026' 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b15efad0 0x5616b15f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 4 client.26327 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616b14c44a0 con 0x5616b15efad0 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc477fff700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b15efad0 0x5616b15f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc477fff700 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b15efad0 0x5616b15f1ec0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd92e031953872b server_cookie=6068ea8c8e5b5d40 in_seq=0 out_seq=0 2024-05-20T21:00:49.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc457fff700 10 client.26327 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc457fff700 1 -- 172.21.15.26:0/3687255784 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5616b14c44a0 con 0x5616b15efad0 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc457fff700 10 client.26327 handle_command_reply: tid=0 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.826+0000 7fc489892b80 1 client.26327 shutdown 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 msgr2=0x5616b157c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 0x5616b157c940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006970 tx=0x7fc46c013770 comp rx=0 tx=0).stop 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 shutdown_connections 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616b15efad0 0x5616b15f1ec0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616b1570320 0x5616b15778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616b156a8a0 0x5616b157c940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 --2- 172.21.15.26:0/3687255784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616b156b260 0x5616b157ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc457fff700 0 client.26327 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 >> 172.21.15.26:0/3687255784 conn(0x5616b1496240 msgr2=0x5616b1576190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 shutdown_connections 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 1 -- 172.21.15.26:0/3687255784 wait complete. 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.827+0000 7fc489892b80 20 client.26327 trim_cache size 0 max 16384 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:00:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "03968ab5-9c9f-4c53-9c34-88644ab3d6b3": { 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "03968ab5-9c9f-4c53-9c34-88644ab3d6b3", 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:00:49.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc4680b2880 msgr2=0x7fc4680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc4680b2880 0x7fc4680b4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc47819aa90 tx=0x7fc46c009190 comp rx=0 tx=0).stop 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 msgr2=0x7fc47819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc47819fcd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc464007d40 tx=0x7fc46400b540 comp rx=0 tx=0).stop 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 shutdown_connections 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4781095b0 0x7fc478199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc47806b220 0x7fc47819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc4680b2880 0x7fc4680b4d40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 --2- 172.21.15.26:0/3654643874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc47806bbe0 0x7fc47819fcd0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 >> 172.21.15.26:0/3654643874 conn(0x7fc4780793f0 msgr2=0x7fc4781012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 shutdown_connections 2024-05-20T21:00:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:49.828+0000 7fc47effc700 1 -- 172.21.15.26:0/3654643874 wait complete. 2024-05-20T21:00:50.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 90869 2.37 MB/sec execute 56 sec latency 30.462 ms 2024-05-20T21:00:50.664 INFO:tasks.cephfs.filesystem:scrub status for tag:03968ab5-9c9f-4c53-9c34-88644ab3d6b3 - {'path': '/', 'tag': '03968ab5-9c9f-4c53-9c34-88644ab3d6b3', 'options': 'recursive,force'} 2024-05-20T21:00:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:51 smithi144 ceph-mon[35660]: pgmap v530: 97 pgs: 97 active+clean; 27 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-05-20T21:00:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:51 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:00:51.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 91604 2.42 MB/sec execute 57 sec latency 20.556 ms 2024-05-20T21:00:51.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:51 smithi055 ceph-mon[35213]: pgmap v530: 97 pgs: 97 active+clean; 27 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-05-20T21:00:51.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:51 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:00:51.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:51 smithi026 ceph-mon[31990]: pgmap v530: 97 pgs: 97 active+clean; 27 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-05-20T21:00:51.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:51 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:00:52.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 92277 2.49 MB/sec execute 58 sec latency 10.849 ms 2024-05-20T21:00:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:53 smithi144 ceph-mon[35660]: pgmap v531: 97 pgs: 97 active+clean; 27 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:00:53.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 92781 2.47 MB/sec execute 59 sec latency 12.330 ms 2024-05-20T21:00:53.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:53 smithi055 ceph-mon[35213]: pgmap v531: 97 pgs: 97 active+clean; 27 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:00:53.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:53 smithi026 ceph-mon[31990]: pgmap v531: 97 pgs: 97 active+clean; 27 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:00:54.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 93407 2.47 MB/sec execute 60 sec latency 26.611 ms 2024-05-20T21:00:55.460 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-20T21:00:55.460 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.3 2024-05-20T21:00:55.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:00:55.485 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:55 smithi026 ceph-mon[31990]: pgmap v532: 97 pgs: 97 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-05-20T21:00:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:55 smithi144 ceph-mon[35660]: pgmap v532: 97 pgs: 97 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-05-20T21:00:55.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 94140 2.48 MB/sec execute 61 sec latency 18.085 ms 2024-05-20T21:00:55.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:55 smithi055 ceph-mon[35213]: pgmap v532: 97 pgs: 97 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-05-20T21:00:55.745 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:56.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 94769 2.48 MB/sec execute 62 sec latency 18.036 ms 2024-05-20T21:00:57.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:57 smithi026 ceph-mon[31990]: pgmap v533: 97 pgs: 97 active+clean; 28 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-05-20T21:00:57.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:00:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:57 smithi144 ceph-mon[35660]: pgmap v533: 97 pgs: 97 active+clean; 28 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-05-20T21:00:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:00:57.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 95579 2.54 MB/sec execute 63 sec latency 48.334 ms 2024-05-20T21:00:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 -- 172.21.15.26:0/2761015775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90901095b0 msgr2=0x7f9090109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 --2- 172.21.15.26:0/2761015775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90901095b0 0x7f9090109990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f908c009990 tx=0x7f908c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 -- 172.21.15.26:0/2761015775 shutdown_connections 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 --2- 172.21.15.26:0/2761015775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90901095b0 0x7f9090109990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 --2- 172.21.15.26:0/2761015775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006bbe0 0x7f90901103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 --2- 172.21.15.26:0/2761015775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909006b220 0x7f909006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 -- 172.21.15.26:0/2761015775 >> 172.21.15.26:0/2761015775 conn(0x7f90900793f0 msgr2=0x7f9090079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 -- 172.21.15.26:0/2761015775 shutdown_connections 2024-05-20T21:00:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.638+0000 7f9097d29700 1 -- 172.21.15.26:0/2761015775 wait complete. 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 Processor -- start 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 -- start start 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006b220 0x7f9090102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 0x7f9090102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90901095b0 0x7f9090102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9090113590 con 0x7f909006bbe0 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9090113410 con 0x7f909006b220 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9097d29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9090113710 con 0x7f90901095b0 2024-05-20T21:00:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f90952c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 0x7f9090102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f90952c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 0x7f9090102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32936/0 (socket says 172.21.15.26:32936) 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f90952c4700 1 -- 172.21.15.26:0/3647267222 learned_addr learned my addr 172.21.15.26:0/3647267222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.639+0000 7f9095ac5700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006b220 0x7f9090102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90962c6700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90901095b0 0x7f9090102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90952c4700 1 -- 172.21.15.26:0/3647267222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90901095b0 msgr2=0x7f9090102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90952c4700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90901095b0 0x7f9090102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90952c4700 1 -- 172.21.15.26:0/3647267222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006b220 msgr2=0x7f9090102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90952c4700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006b220 0x7f9090102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90952c4700 1 -- 172.21.15.26:0/3647267222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90901a8070 con 0x7f909006bbe0 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f9095ac5700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006b220 0x7f9090102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f90952c4700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 0x7f9090102a50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f908000d980 tx=0x7f908000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080017070 con 0x7f909006bbe0 2024-05-20T21:00:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9080005020 con 0x7f909006bbe0 2024-05-20T21:00:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908000f980 con 0x7f909006bbe0 2024-05-20T21:00:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90901a8360 con 0x7f909006bbe0 2024-05-20T21:00:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.640+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90901a8850 con 0x7f909006bbe0 2024-05-20T21:00:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.642+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909010ba90 con 0x7f909006bbe0 2024-05-20T21:00:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.643+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9080004a30 con 0x7f909006bbe0 2024-05-20T21:00:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.644+0000 7f9086ffd700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f907c0b2900 0x7f907c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:00:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.644+0000 7f9095ac5700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f907c0b2900 0x7f907c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:00:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.644+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f90800e2070 con 0x7f909006bbe0 2024-05-20T21:00:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.644+0000 7f9095ac5700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f907c0b2900 0x7f907c0b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f908c00b2e0 tx=0x7f908c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:00:57.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.649+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9080020020 con 0x7f909006bbe0 2024-05-20T21:00:57.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:57 smithi055 ceph-mon[35213]: pgmap v533: 97 pgs: 97 active+clean; 28 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-05-20T21:00:57.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:00:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.957+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f909010b470 con 0x7f909006bbe0 2024-05-20T21:00:57.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.958+0000 7f9086ffd700 1 -- 172.21.15.26:0/3647267222 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f90800a9d30 con 0x7f909006bbe0 2024-05-20T21:00:57.958 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:00:57.958 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f907c0b2900 msgr2=0x7f907c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f907c0b2900 0x7f907c0b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f908c00b2e0 tx=0x7f908c01a040 comp rx=0 tx=0).stop 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 msgr2=0x7f9090102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 0x7f9090102a50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f908000d980 tx=0x7f908000dc90 comp rx=0 tx=0).stop 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 shutdown_connections 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90901095b0 0x7f9090102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f909006b220 0x7f9090102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f907c0b2900 0x7f907c0b4dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 --2- 172.21.15.26:0/3647267222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f909006bbe0 0x7f9090102a50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:00:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 >> 172.21.15.26:0/3647267222 conn(0x7f90900793f0 msgr2=0x7f9090101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:00:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 shutdown_connections 2024-05-20T21:00:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:00:57.961+0000 7f9097d29700 1 -- 172.21.15.26:0/3647267222 wait complete. 2024-05-20T21:00:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:00:58.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: Unable to set osd_memory_target on smithi026 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:00:58.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:58 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3647267222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:58.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 96128 2.57 MB/sec execute 64 sec latency 17.685 ms 2024-05-20T21:00:58.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: Unable to set osd_memory_target on smithi026 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:00:58.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:58 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3647267222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: Unable to set osd_memory_target on smithi026 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:00:58.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:58 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3647267222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:00:58.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0' 2024-05-20T21:00:58.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0 2024-05-20T21:00:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:00:59.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:00:59 smithi026 ceph-mon[31990]: pgmap v534: 97 pgs: 97 active+clean; 28 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-20T21:00:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:00:59 smithi144 ceph-mon[35660]: pgmap v534: 97 pgs: 97 active+clean; 28 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-20T21:00:59.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 96609 2.54 MB/sec execute 65 sec latency 26.954 ms 2024-05-20T21:00:59.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:00:59 smithi055 ceph-mon[35213]: pgmap v534: 97 pgs: 97 active+clean; 28 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-20T21:01:00.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97286 2.58 MB/sec execute 66 sec latency 17.595 ms 2024-05-20T21:01:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 -- 172.21.15.26:0/2363656910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 msgr2=0x7f41c4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/2363656910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4105920 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009900 tx=0x7f41c002a410 comp rx=0 tx=0).stop 2024-05-20T21:01:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 -- 172.21.15.26:0/2363656910 shutdown_connections 2024-05-20T21:01:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/2363656910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 0x7f41c410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/2363656910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 0x7f41c4108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/2363656910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4105920 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 -- 172.21.15.26:0/2363656910 >> 172.21.15.26:0/2363656910 conn(0x7f41c40fcf40 msgr2=0x7f41c40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 -- 172.21.15.26:0/2363656910 shutdown_connections 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 -- 172.21.15.26:0/2363656910 wait complete. 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 Processor -- start 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.711+0000 7f41cc1f0640 1 -- start start 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41cc1f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41cc1f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 0x7f41c41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41cc1f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 0x7f41c41a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41cc1f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410e490 con 0x7f41c4103530 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41cc1f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410e310 con 0x7f41c4108790 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41cc1f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410e610 con 0x7f41c4105e60 2024-05-20T21:01:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45334/0 (socket says 172.21.15.26:45334) 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 -- 172.21.15.26:0/3296480925 learned_addr learned my addr 172.21.15.26:0/3296480925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 0x7f41c41a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c978b640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 0x7f41c41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 -- 172.21.15.26:0/3296480925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 msgr2=0x7f41c41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 0x7f41c41990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 -- 172.21.15.26:0/3296480925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 msgr2=0x7f41c41a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 0x7f41c41a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c9f8c640 1 -- 172.21.15.26:0/3296480925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c41a0780 con 0x7f41c4103530 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41c978b640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 0x7f41c41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.712+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 0x7f41c41a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.713+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4198b60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009a20 tx=0x7f41c00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.713+0000 7f41baffd640 1 -- 172.21.15.26:0/3296480925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c003f030 con 0x7f41c4103530 2024-05-20T21:01:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.713+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c4101320 con 0x7f41c4103530 2024-05-20T21:01:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.713+0000 7f41baffd640 1 -- 172.21.15.26:0/3296480925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41c0003a80 con 0x7f41c4103530 2024-05-20T21:01:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.713+0000 7f41baffd640 1 -- 172.21.15.26:0/3296480925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0035070 con 0x7f41c4103530 2024-05-20T21:01:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.713+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c4101810 con 0x7f41c4103530 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.716+0000 7f41cefe5740 1 Processor -- start 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 -- start start 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 0x56167fdebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdec450 0x56167fdf0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe09b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56167fd61c30 con 0x56167fdf15c0 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56167fd517d0 con 0x56167fdec450 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41cefe5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56167fd55810 con 0x56167fdeba60 2024-05-20T21:01:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41ca78d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 0x56167fdebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41c978b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe09b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41ca78d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 0x56167fdebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:40710/0 (socket says 172.21.15.26:40710) 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41ca78d640 1 -- 172.21.15.26:0/3378407955 learned_addr learned my addr 172.21.15.26:0/3378407955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdec450 0x56167fdf0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41ca78d640 1 -- 172.21.15.26:0/3378407955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdec450 msgr2=0x56167fdf0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.717+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdec450 0x56167fdf0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41ca78d640 1 -- 172.21.15.26:0/3378407955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 msgr2=0x56167fe09b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe09b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41ca78d640 1 -- 172.21.15.26:0/3378407955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167fa66210 con 0x56167fdeba60 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 0x56167fdebe80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0337a0 tx=0x7f41bc033c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713b7190264d4004 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41affff640 1 -- 172.21.15.26:0/3378407955 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc003c30 con 0x56167fdeba60 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41affff640 1 -- 172.21.15.26:0/3378407955 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41bc031e40 con 0x56167fdeba60 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.718+0000 7f41affff640 1 -- 172.21.15.26:0/3378407955 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc004850 con 0x56167fdeba60 2024-05-20T21:01:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41baffd640 1 -- 172.21.15.26:0/3296480925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f41c0042030 con 0x7f41c4103530 2024-05-20T21:01:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 -- 172.21.15.26:0/3378407955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 msgr2=0x56167fdebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 0x56167fdebe80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0337a0 tx=0x7f41bc033c60 comp rx=0 tx=0).stop 2024-05-20T21:01:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 -- 172.21.15.26:0/3378407955 shutdown_connections 2024-05-20T21:01:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe09b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdec450 0x56167fdf0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 --2- 172.21.15.26:0/3378407955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdeba60 0x56167fdebe80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.719+0000 7f41cefe5740 1 -- 172.21.15.26:0/3378407955 >> 172.21.15.26:0/3378407955 conn(0x56167fdf4540 msgr2=0x56167fdf4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.720+0000 7f41cefe5740 1 -- 172.21.15.26:0/3378407955 shutdown_connections 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.720+0000 7f41cefe5740 1 -- 172.21.15.26:0/3378407955 wait complete. 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.720+0000 7f41cefe5740 1 -- start start 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.720+0000 7f41cefe5740 1 Processor -- start 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41baffd640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f41b00b2840 0x7f41b00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdeba60 0x56167fdfa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdfad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdf15c0 0x56167fe01d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56167fa37790 con 0x56167fdeba60 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56167fd0d890 con 0x56167fdf15c0 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56167fdf8c90 con 0x56167fdec450 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41cefe5740 10 client.0 _ll_register_callbacks cb 0x56167fa65a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdfad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdfad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:40720/0 (socket says 172.21.15.26:40720) 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 -- 172.21.15.26:0/21385686 learned_addr learned my addr 172.21.15.26:0/21385686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f41b00b2840 0x7f41b00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41ca78d640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdf15c0 0x56167fe01d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdeba60 0x56167fdfa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41baffd640 1 -- 172.21.15.26:0/3296480925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f41c01028f0 con 0x7f41c4103530 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 -- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdf15c0 msgr2=0x56167fe01d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdf15c0 0x56167fe01d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 -- 172.21.15.26:0/21385686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdeba60 msgr2=0x56167fdfa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.721+0000 7f41c978b640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdeba60 0x56167fdfa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41c978b640 1 -- 172.21.15.26:0/21385686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167fb32620 con 0x56167fdec450 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdeba60 0x56167fdfa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41c978b640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdfad20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41b400bad0 tx=0x7f41b400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87dbed7f501cfa9c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:01:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41ca78d640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f41b00b2840 0x7f41b00b4d00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0335b0 tx=0x7f41bc000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41ad7fa640 1 -- 172.21.15.26:0/21385686 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b401b640 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41bc031e40 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41ad7fa640 1 -- 172.21.15.26:0/21385686 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f41b401b7e0 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41ad7fa640 1 -- 172.21.15.26:0/21385686 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41b4003e40 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41cefe5740 10 client.26233 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.722+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56167fe04c50 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41ad7fa640 1 -- 172.21.15.26:0/21385686 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41bc031e40 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41ad7fa640 1 -- 172.21.15.26:0/21385686 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f41b4024050 con 0x56167fdec450 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41cefe5740 10 client.26233 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41cefe5740 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41cefe5740 20 client.26233 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41cefe5740 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe06280 0x56167fe08690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41cefe5740 4 client.26233 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56167fd63b80 con 0x56167fe06280 2024-05-20T21:01:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe06280 0x56167fe08690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41ad7fa640 10 client.26233 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.723+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe06280 0x56167fe08690 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b554e17259e989ca server_cookie=4b98ef501cac7144 in_seq=0 out_seq=0 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41ad7fa640 1 -- 172.21.15.26:0/21385686 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56167fd63b80 con 0x56167fe06280 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41ad7fa640 10 client.26233 handle_command_reply: tid=0 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 client.26233 shutdown 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 msgr2=0x56167fdfad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdfad20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41b400bad0 tx=0x7f41b400bf90 comp rx=0 tx=0).stop 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 shutdown_connections 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe06280 0x56167fe08690 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdf15c0 0x56167fe01d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdfad20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 --2- 172.21.15.26:0/21385686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdeba60 0x56167fdfa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 >> 172.21.15.26:0/21385686 conn(0x56167fdf4540 msgr2=0x56167fdeafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41ad7fa640 0 client.26233 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 shutdown_connections 2024-05-20T21:01:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 1 -- 172.21.15.26:0/21385686 wait complete. 2024-05-20T21:01:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.725+0000 7f41cefe5740 20 client.26233 trim_cache size 0 max 16384 2024-05-20T21:01:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 Processor -- start 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 -- start start 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdf9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fe01d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fdf7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41aa650 con 0x56167fdf15c0 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41a7f20 con 0x56167fdeba60 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41cefe5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41a5d70 con 0x56167fdec450 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdf9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdf9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51822/0 (socket says 172.21.15.26:51822) 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 -- 172.21.15.26:0/1345956603 learned_addr learned my addr 172.21.15.26:0/1345956603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41ca78d640 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fdf7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c978b640 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fe01d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 -- 172.21.15.26:0/1345956603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 msgr2=0x56167fe01d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fe01d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 -- 172.21.15.26:0/1345956603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 msgr2=0x56167fdf7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fdf7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.743+0000 7f41c9f8c640 1 -- 172.21.15.26:0/1345956603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167fde76f0 con 0x56167fdeba60 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.744+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdf9060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41c00c9b50 tx=0x7f41c002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ca54cfb3e34f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.744+0000 7f41af7fe640 1 -- 172.21.15.26:0/1345956603 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0049040 con 0x56167fdeba60 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.744+0000 7f41af7fe640 1 -- 172.21.15.26:0/1345956603 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41c00cc070 con 0x56167fdeba60 2024-05-20T21:01:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.744+0000 7f41af7fe640 1 -- 172.21.15.26:0/1345956603 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c00ce200 con 0x56167fdeba60 2024-05-20T21:01:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 -- 172.21.15.26:0/1345956603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 msgr2=0x56167fdf9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdf9060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41c00c9b50 tx=0x7f41c002a410 comp rx=0 tx=0).stop 2024-05-20T21:01:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 -- 172.21.15.26:0/1345956603 shutdown_connections 2024-05-20T21:01:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fdf7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fe01d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 --2- 172.21.15.26:0/1345956603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdf9060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 -- 172.21.15.26:0/1345956603 >> 172.21.15.26:0/1345956603 conn(0x56167fdf4540 msgr2=0x56167fdedbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 -- 172.21.15.26:0/1345956603 shutdown_connections 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 -- 172.21.15.26:0/1345956603 wait complete. 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 -- start start 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.745+0000 7f41cefe5740 1 Processor -- start 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe00600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41a63e0 con 0x56167fdf15c0 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41a7850 con 0x56167fdeba60 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41a6ab0 con 0x56167fdec450 2024-05-20T21:01:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41cefe5740 10 client.0 _ll_register_callbacks cb 0x7f41b4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c978b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c978b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:40740/0 (socket says 172.21.15.26:40740) 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41ca78d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe00600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c978b640 1 -- 172.21.15.26:0/197295596 learned_addr learned my addr 172.21.15.26:0/197295596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c978b640 1 -- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 msgr2=0x56167fdff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c978b640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.746+0000 7f41c978b640 1 -- 172.21.15.26:0/197295596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 msgr2=0x56167fe00600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41c978b640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe00600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41c978b640 1 -- 172.21.15.26:0/197295596 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167fe00bf0 con 0x56167fdec450 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41ca78d640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41c978b640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdff690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x56167fdfbcc0 tx=0x7f41b4011650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976bae26df40b323 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:01:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41ad7fa640 1 -- 172.21.15.26:0/197295596 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4005240 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41ad7fa640 1 -- 172.21.15.26:0/197295596 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f41b40053e0 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c00cc070 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41ad7fa640 1 -- 172.21.15.26:0/197295596 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41b40039c0 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41cefe5740 10 client.26239 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.747+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56167fe01fa0 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41ad7fa640 1 -- 172.21.15.26:0/197295596 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41c00cc070 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41ad7fa640 1 -- 172.21.15.26:0/197295596 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f41b4038050 con 0x56167fdec450 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41cefe5740 10 client.26239 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41cefe5740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41cefe5740 20 client.26239 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41cefe5740 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe714e0 0x56167fe73900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41cefe5740 4 client.26239 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56167fd5d360 con 0x56167fe714e0 2024-05-20T21:01:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.748+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe714e0 0x56167fe73900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.749+0000 7f41c9f8c640 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe714e0 0x56167fe73900 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9844cd1338d6a9df server_cookie=222b22baf33e1fc1 in_seq=0 out_seq=0 2024-05-20T21:01:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.749+0000 7f41ad7fa640 10 client.26239 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41ad7fa640 1 -- 172.21.15.26:0/197295596 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56167fd5d360 con 0x56167fe714e0 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41ad7fa640 10 client.26239 handle_command_reply: tid=0 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 client.26239 shutdown 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 msgr2=0x56167fdff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdff690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x56167fdfbcc0 tx=0x7f41b4011650 comp rx=0 tx=0).stop 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 shutdown_connections 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56167fe714e0 0x56167fe73900 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56167fdf15c0 0x56167fe00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56167fdec450 0x56167fdff690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41ad7fa640 0 client.26239 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 --2- 172.21.15.26:0/197295596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56167fdeba60 0x56167fdff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.806+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 >> 172.21.15.26:0/197295596 conn(0x56167fdf4540 msgr2=0x56167fdf7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 shutdown_connections 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cefe5740 1 -- 172.21.15.26:0/197295596 wait complete. 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cefe5740 20 client.26239 trim_cache size 0 max 16384 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f41b00b2840 msgr2=0x7f41b00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f41b00b2840 0x7f41b00b4d00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0335b0 tx=0x7f41bc000f40 comp rx=0 tx=0).stop 2024-05-20T21:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 msgr2=0x7f41c4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4198b60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009a20 tx=0x7f41c00040c0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 shutdown_connections 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f41b00b2840 0x7f41b00b4d00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4108790 0x7f41c41a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41c4105e60 0x7f41c41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 --2- 172.21.15.26:0/3296480925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c4103530 0x7f41c4198b60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 >> 172.21.15.26:0/3296480925 conn(0x7f41c40fcf40 msgr2=0x7f41c4103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 shutdown_connections 2024-05-20T21:01:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:00.807+0000 7f41cc1f0640 1 -- 172.21.15.26:0/3296480925 wait complete. 2024-05-20T21:01:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 -- 172.21.15.26:0/902832140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 msgr2=0x7fa4d8102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 --2- 172.21.15.26:0/902832140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d8102950 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009990 tx=0x7fa4c8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 -- 172.21.15.26:0/902832140 shutdown_connections 2024-05-20T21:01:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 --2- 172.21.15.26:0/902832140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4d8102e90 0x7fa4d810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 --2- 172.21.15.26:0/902832140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d81084d0 0x7fa4d81088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 --2- 172.21.15.26:0/902832140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d8102950 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 -- 172.21.15.26:0/902832140 >> 172.21.15.26:0/902832140 conn(0x7fa4d8078490 msgr2=0x7fa4d80788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 -- 172.21.15.26:0/902832140 shutdown_connections 2024-05-20T21:01:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.011+0000 7fa4dff02700 1 -- 172.21.15.26:0/902832140 wait complete. 2024-05-20T21:01:01.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 Processor -- start 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 -- start start 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d81040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4d8102e90 0x7fa4d81045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d81084d0 0x7fa4d8104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8112510 con 0x7fa4d81024d0 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8112390 con 0x7fa4d81084d0 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dff02700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8112690 con 0x7fa4d8102e90 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d81040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d81040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45370/0 (socket says 172.21.15.26:45370) 2024-05-20T21:01:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1119957183 learned_addr learned my addr 172.21.15.26:0/1119957183 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4de49f700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d81084d0 0x7fa4d8104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4d8102e90 0x7fa4d81045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1119957183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4d8102e90 msgr2=0x7fa4d81045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4d8102e90 0x7fa4d81045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1119957183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d81084d0 msgr2=0x7fa4d8104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d81084d0 0x7fa4d8104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.012+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1119957183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d81a8010 con 0x7fa4d81024d0 2024-05-20T21:01:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.013+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d81040a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400e9a0 tx=0x7fa4d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.013+0000 7fa4ceffd700 1 -- 172.21.15.26:0/1119957183 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400ca10 con 0x7fa4d81024d0 2024-05-20T21:01:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.013+0000 7fa4ceffd700 1 -- 172.21.15.26:0/1119957183 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4d400cbb0 con 0x7fa4d81024d0 2024-05-20T21:01:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.013+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d81a8300 con 0x7fa4d81024d0 2024-05-20T21:01:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.013+0000 7fa4ceffd700 1 -- 172.21.15.26:0/1119957183 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4018680 con 0x7fa4d81024d0 2024-05-20T21:01:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.013+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d81a8810 con 0x7fa4d81024d0 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.016+0000 7fa4ceffd700 1 -- 172.21.15.26:0/1119957183 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa4d4010430 con 0x7fa4d81024d0 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.017+0000 7fa4ceffd700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4c40b2930 0x7fa4c40b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.017+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4c40b2930 0x7fa4c40b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.017+0000 7fa4ceffd700 1 -- 172.21.15.26:0/1119957183 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa4d40e23b0 con 0x7fa4d81024d0 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.017+0000 7fa4ea798b80 1 Processor -- start 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 -- start start 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4c40b2930 0x7fa4c40b4df0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8105b30 tx=0x7fa4c8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 0x56365b7c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 0x56365b7c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56365b722f00 con 0x56365b7c5640 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56365b712aa0 con 0x56365b7c97b0 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56365b716ae0 con 0x56365b7c4c80 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4de49f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4dd49d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 0x56365b7c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 0x56365b7c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4de49f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45386/0 (socket says 172.21.15.26:45386) 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4dd49d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 0x56365b7c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:40774/0 (socket says 172.21.15.26:40774) 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 0x56365b7c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51852/0 (socket says 172.21.15.26:51852) 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4de49f700 1 -- 172.21.15.26:0/852416135 learned_addr learned my addr 172.21.15.26:0/852416135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/852416135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 msgr2=0x56365b7c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 0x56365b7c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/852416135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 msgr2=0x56365b7d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/852416135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365b6bc140 con 0x56365b7c97b0 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4de49f700 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.018+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 0x56365b7c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.019+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 0x56365b7c9b90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40e01c0 tx=0x7fa4d40a9500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0832cb2a5ce0bf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.019+0000 7fa4c3fff700 1 -- 172.21.15.26:0/852416135 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40a9970 con 0x56365b7c97b0 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.019+0000 7fa4c3fff700 1 -- 172.21.15.26:0/852416135 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4d4010e50 con 0x56365b7c97b0 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.019+0000 7fa4c3fff700 1 -- 172.21.15.26:0/852416135 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40cc2b0 con 0x56365b7c97b0 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 -- 172.21.15.26:0/852416135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 msgr2=0x56365b7c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 0x56365b7c9b90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40e01c0 tx=0x7fa4d40a9500 comp rx=0 tx=0).stop 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 -- 172.21.15.26:0/852416135 shutdown_connections 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c97b0 0x56365b7c9b90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/852416135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c4c80 0x56365b7c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 -- 172.21.15.26:0/852416135 >> 172.21.15.26:0/852416135 conn(0x56365b6f03d0 msgr2=0x56365b7bf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 -- 172.21.15.26:0/852416135 shutdown_connections 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 -- 172.21.15.26:0/852416135 wait complete. 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.020+0000 7fa4ea798b80 1 -- start start 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 Processor -- start 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b7cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56365b619130 con 0x56365b7c97b0 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56365b67af30 con 0x56365b7c4c80 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56365b7d1d10 con 0x56365b7c5640 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ea798b80 10 client.0 _ll_register_callbacks cb 0x56365b6adae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4de49f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b7cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4de49f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b7cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45390/0 (socket says 172.21.15.26:45390) 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4de49f700 1 -- 172.21.15.26:0/1712162970 learned_addr learned my addr 172.21.15.26:0/1712162970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.021+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1712162970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 msgr2=0x56365b7ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1712162970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 msgr2=0x56365b7cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b7cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1712162970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365b662a60 con 0x56365b7c4c80 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4de49f700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b7cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7d14a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40adee0 tx=0x7fa4d40aa110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8932c26aa3faaf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1712162970 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40c4020 con 0x56365b7c4c80 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1712162970 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa4d40b2550 con 0x56365b7c4c80 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d4010e50 con 0x56365b7c4c80 2024-05-20T21:01:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1712162970 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4d40add20 con 0x56365b7c4c80 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ea798b80 10 client.26360 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.022+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56365b6cc8f0 con 0x56365b7c4c80 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1712162970 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4d4010e50 con 0x56365b7c4c80 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1712162970 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fa4d40a8c60 con 0x56365b7c4c80 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4ea798b80 10 client.26360 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4ea798b80 10 client.26360 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4ea798b80 20 client.26360 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b7e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4ea798b80 4 client.26360 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365b724e50 con 0x56365b7de8a0 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b7e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b7e0c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ac4eaf9a13dc52 server_cookie=15ee50a4ca2816d8 in_seq=0 out_seq=0 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.023+0000 7fa4c17fa700 10 client.26360 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.024+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1712162970 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56365b724e50 con 0x56365b7de8a0 2024-05-20T21:01:01.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.024+0000 7fa4c17fa700 10 client.26360 handle_command_reply: tid=0 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.024+0000 7fa4ea798b80 1 client.26360 shutdown 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.024+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 msgr2=0x56365b7d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.024+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7d14a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40adee0 tx=0x7fa4d40aa110 comp rx=0 tx=0).stop 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 shutdown_connections 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b7e0c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4c17fa700 0 client.26360 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b7cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7d14a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1712162970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 >> 172.21.15.26:0/1712162970 conn(0x56365b6f03d0 msgr2=0x56365b7c3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 shutdown_connections 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1712162970 wait complete. 2024-05-20T21:01:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.025+0000 7fa4ea798b80 20 client.26360 trim_cache size 0 max 16384 2024-05-20T21:01:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 Processor -- start 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 -- start start 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c97b0 0x56365b7e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81b2830 con 0x56365b7c5640 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81b00c0 con 0x56365b7c4c80 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81adf10 con 0x56365b7c97b0 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4dd49d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.048+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51878/0 (socket says 172.21.15.26:51878) 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/40330820 learned_addr learned my addr 172.21.15.26:0/40330820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4de49f700 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c97b0 0x56365b7e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/40330820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c97b0 msgr2=0x56365b7e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c97b0 0x56365b7e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/40330820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 msgr2=0x56365b7caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/40330820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365b7ccd10 con 0x56365b7c4c80 2024-05-20T21:01:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4de49f700 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c97b0 0x56365b7e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7b9f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40e01c0 tx=0x7fa4d40aa070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0160c932ed19d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4c37fe700 1 -- 172.21.15.26:0/40330820 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40c4040 con 0x56365b7c4c80 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4c37fe700 1 -- 172.21.15.26:0/40330820 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4d4010e50 con 0x56365b7c4c80 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.049+0000 7fa4c37fe700 1 -- 172.21.15.26:0/40330820 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40a8bd0 con 0x56365b7c4c80 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 -- 172.21.15.26:0/40330820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 msgr2=0x56365b7b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7b9f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40e01c0 tx=0x7fa4d40aa070 comp rx=0 tx=0).stop 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 -- 172.21.15.26:0/40330820 shutdown_connections 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c97b0 0x56365b7e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7b9f00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/40330820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c5640 0x56365b7caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 -- 172.21.15.26:0/40330820 >> 172.21.15.26:0/40330820 conn(0x56365b6f03d0 msgr2=0x56365b7cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 -- 172.21.15.26:0/40330820 shutdown_connections 2024-05-20T21:01:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.050+0000 7fa4ea798b80 1 -- 172.21.15.26:0/40330820 wait complete. 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 -- start start 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 Processor -- start 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7e2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b8480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81ae580 con 0x56365b7c97b0 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81af9f0 con 0x56365b7c4c80 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81aec50 con 0x56365b7c5640 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ea798b80 10 client.0 _ll_register_callbacks cb 0x56365b6adae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4de49f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b8480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ddc9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7e21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51886/0 (socket says 172.21.15.26:51886) 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1517965912 learned_addr learned my addr 172.21.15.26:0/1517965912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.051+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1517965912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 msgr2=0x56365b7e2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7e2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1517965912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 msgr2=0x56365b8480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b8480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ddc9e700 1 -- 172.21.15.26:0/1517965912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365b8486f0 con 0x56365b7c4c80 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ddc9e700 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7e21e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40aa1a0 tx=0x7fa4d4009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e0aacb805796bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1517965912 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4087070 con 0x56365b7c4c80 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1517965912 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa4d4010720 con 0x56365b7c4c80 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d4010e50 con 0x56365b7c4c80 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1517965912 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4d40c4040 con 0x56365b7c4c80 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ea798b80 10 client.26366 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56365b8486f0 con 0x56365b7c4c80 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.052+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1517965912 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4d40a5610 con 0x56365b7c4c80 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1517965912 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fa4d408a070 con 0x56365b7c4c80 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4ea798b80 10 client.26366 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4ea798b80 10 client.26366 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4ea798b80 20 client.26366 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b84bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4ea798b80 4 client.26366 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}] 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}) -- 0x7fa4d4010e50 con 0x56365b7de8a0 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b84bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4dd49d700 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b84bce0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37c0c3ebc23410b server_cookie=fc124312b59df131 in_seq=0 out_seq=0 2024-05-20T21:01:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.053+0000 7fa4c17fa700 10 client.26366 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:01.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4c17fa700 1 -- 172.21.15.26:0/1517965912 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fa4d4010e50 con 0x56365b7de8a0 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4c17fa700 10 client.26366 handle_command_reply: tid=0 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4ea798b80 1 client.26366 shutdown 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 msgr2=0x56365b7e21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7e21e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40aa1a0 tx=0x7fa4d4009790 comp rx=0 tx=0).stop 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 shutdown_connections 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.164+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56365b7de8a0 0x56365b84bce0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56365b7c97b0 0x56365b8480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56365b7c4c80 0x56365b7e21e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 1 --2- 172.21.15.26:0/1517965912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56365b7c5640 0x56365b7e2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 >> 172.21.15.26:0/1517965912 conn(0x56365b6f03d0 msgr2=0x56365b7cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4c17fa700 0 client.26366 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 shutdown_connections 2024-05-20T21:01:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 1 -- 172.21.15.26:0/1517965912 wait complete. 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4ea798b80 20 client.26366 trim_cache size 0 max 16384 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 28, 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 18, 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout: "c3a433f9": { 2024-05-20T21:01:01.166 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 18, 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.3, 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:01:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:01.168 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4c40b2930 msgr2=0x7fa4c40b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4dff02700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4c40b2930 0x7fa4c40b4df0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8105b30 tx=0x7fa4c8005fb0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 msgr2=0x7fa4d81040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.165+0000 7fa4dff02700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d81040a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400e9a0 tx=0x7fa4d400ecb0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 shutdown_connections 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4c40b2930 0x7fa4c40b4df0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d81024d0 0x7fa4d81040a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d81084d0 0x7fa4d8104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 --2- 172.21.15.26:0/1119957183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4d8102e90 0x7fa4d81045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 >> 172.21.15.26:0/1119957183 conn(0x7fa4d8078490 msgr2=0x7fa4d806bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 shutdown_connections 2024-05-20T21:01:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:01.166+0000 7fa4dff02700 1 -- 172.21.15.26:0/1119957183 wait complete. 2024-05-20T21:01:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:01 smithi026 ceph-mon[31990]: pgmap v535: 97 pgs: 97 active+clean; 29 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-05-20T21:01:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:01 smithi144 ceph-mon[35660]: pgmap v535: 97 pgs: 97 active+clean; 29 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-05-20T21:01:01.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.54 MB/sec execute 67 sec latency 342.243 ms 2024-05-20T21:01:01.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:01 smithi055 ceph-mon[35213]: pgmap v535: 97 pgs: 97 active+clean; 29 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-05-20T21:01:02.033 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c3a433f9', quiesce took 6.6 seconds. Will release after: 14.700000000000001 2024-05-20T21:01:02.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.50 MB/sec execute 68 sec latency 1342.359 ms 2024-05-20T21:01:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:03 smithi144 ceph-mon[35660]: pgmap v536: 97 pgs: 97 active+clean; 29 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-20T21:01:03.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.47 MB/sec execute 69 sec latency 2342.401 ms 2024-05-20T21:01:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:03 smithi055 ceph-mon[35213]: pgmap v536: 97 pgs: 97 active+clean; 29 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-20T21:01:03.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:03 smithi026 ceph-mon[31990]: pgmap v536: 97 pgs: 97 active+clean; 29 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-20T21:01:04.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.43 MB/sec execute 70 sec latency 3342.496 ms 2024-05-20T21:01:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:05 smithi144 ceph-mon[35660]: pgmap v537: 97 pgs: 97 active+clean; 25 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:01:05.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.40 MB/sec execute 71 sec latency 4342.611 ms 2024-05-20T21:01:05.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:05 smithi055 ceph-mon[35213]: pgmap v537: 97 pgs: 97 active+clean; 25 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:01:05.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:05 smithi026 ceph-mon[31990]: pgmap v537: 97 pgs: 97 active+clean; 25 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:01:06.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.37 MB/sec execute 72 sec latency 5342.705 ms 2024-05-20T21:01:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:07 smithi144 ceph-mon[35660]: pgmap v538: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-20T21:01:07.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.33 MB/sec execute 73 sec latency 6342.835 ms 2024-05-20T21:01:07.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:07 smithi055 ceph-mon[35213]: pgmap v538: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-20T21:01:07.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:07 smithi026 ceph-mon[31990]: pgmap v538: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-20T21:01:08.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.30 MB/sec execute 74 sec latency 7342.953 ms 2024-05-20T21:01:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:09 smithi144 ceph-mon[35660]: pgmap v539: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 918 KiB/s wr, 91 op/s 2024-05-20T21:01:09.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.27 MB/sec execute 75 sec latency 8343.057 ms 2024-05-20T21:01:09.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:09 smithi055 ceph-mon[35213]: pgmap v539: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 918 KiB/s wr, 91 op/s 2024-05-20T21:01:09.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:09 smithi026 ceph-mon[31990]: pgmap v539: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 918 KiB/s wr, 91 op/s 2024-05-20T21:01:10.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.24 MB/sec execute 76 sec latency 9343.163 ms 2024-05-20T21:01:11.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.21 MB/sec execute 77 sec latency 10343.271 ms 2024-05-20T21:01:11.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:11 smithi055 ceph-mon[35213]: pgmap v540: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 848 KiB/s wr, 86 op/s 2024-05-20T21:01:11.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:11 smithi026 ceph-mon[31990]: pgmap v540: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 848 KiB/s wr, 86 op/s 2024-05-20T21:01:11.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:11 smithi144 ceph-mon[35660]: pgmap v540: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 848 KiB/s wr, 86 op/s 2024-05-20T21:01:12.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.18 MB/sec execute 78 sec latency 11343.388 ms 2024-05-20T21:01:13.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.16 MB/sec execute 79 sec latency 12343.499 ms 2024-05-20T21:01:13.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:13 smithi055 ceph-mon[35213]: pgmap v541: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 480 KiB/s wr, 61 op/s 2024-05-20T21:01:13.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:13 smithi026 ceph-mon[31990]: pgmap v541: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 480 KiB/s wr, 61 op/s 2024-05-20T21:01:13.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:13 smithi144 ceph-mon[35660]: pgmap v541: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 480 KiB/s wr, 61 op/s 2024-05-20T21:01:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 -- 172.21.15.26:0/1379627872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 msgr2=0x7f551410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 --2- 172.21.15.26:0/1379627872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f551410ac00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f550c009900 tx=0x7f550c02a3f0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 -- 172.21.15.26:0/1379627872 shutdown_connections 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 --2- 172.21.15.26:0/1379627872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f551410ac00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 --2- 172.21.15.26:0/1379627872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 0x7f55141082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 --2- 172.21.15.26:0/1379627872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55141035b0 0x7f55141059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 -- 172.21.15.26:0/1379627872 >> 172.21.15.26:0/1379627872 conn(0x7f55140fcf20 msgr2=0x7f55140ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 -- 172.21.15.26:0/1379627872 shutdown_connections 2024-05-20T21:01:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.909+0000 7f5518f19640 1 -- 172.21.15.26:0/1379627872 wait complete. 2024-05-20T21:01:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 Processor -- start 2024-05-20T21:01:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 -- start start 2024-05-20T21:01:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55141035b0 0x7f5514198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 0x7f5514199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f55141a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f551410e500 con 0x7f5514108810 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f551410e380 con 0x7f55141035b0 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5518f19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f551410e680 con 0x7f5514105ee0 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5512d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f55141a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5512d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f55141a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43586/0 (socket says 172.21.15.26:43586) 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f5512d9d640 1 -- 172.21.15.26:0/2573063641 learned_addr learned my addr 172.21.15.26:0/2573063641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.910+0000 7f551259c640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55141035b0 0x7f5514198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 0x7f5514199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5512d9d640 1 -- 172.21.15.26:0/2573063641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 msgr2=0x7f5514199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5512d9d640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 0x7f5514199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5512d9d640 1 -- 172.21.15.26:0/2573063641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55141035b0 msgr2=0x7f5514198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5512d9d640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55141035b0 0x7f5514198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5512d9d640 1 -- 172.21.15.26:0/2573063641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55141a0760 con 0x7f5514108810 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 0x7f5514199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5512d9d640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f55141a00f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f550c005940 tx=0x7f550c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f55037fe640 1 -- 172.21.15.26:0/2573063641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c03f030 con 0x7f5514108810 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55141a09f0 con 0x7f5514108810 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f55037fe640 1 -- 172.21.15.26:0/2573063641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f550c003c90 con 0x7f5514108810 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f55037fe640 1 -- 172.21.15.26:0/2573063641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c035070 con 0x7f5514108810 2024-05-20T21:01:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.911+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55141a0f60 con 0x7f5514108810 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.914+0000 7f55037fe640 1 -- 172.21.15.26:0/2573063641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f550c042030 con 0x7f5514108810 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.914+0000 7f551bc39740 1 Processor -- start 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 -- start start 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9bad0 0x55e12fa9bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9c4c0 0x55e12faa0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12faa3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e12fa08c30 con 0x55e12faa1630 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e12f9f87d0 con 0x55e12fa9c4c0 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551bc39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e12f9fc810 con 0x55e12fa9bad0 2024-05-20T21:01:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f55037fe640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b28a0 0x7f54fc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f5511d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12faa3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f5511d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12faa3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43590/0 (socket says 172.21.15.26:43590) 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f5511d9b640 1 -- 172.21.15.26:0/765171910 learned_addr learned my addr 172.21.15.26:0/765171910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f5512d9d640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b28a0 0x7f54fc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f551259c640 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9c4c0 0x55e12faa0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f5511d9b640 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9bad0 0x55e12fa9bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.915+0000 7f55037fe640 1 -- 172.21.15.26:0/2573063641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f550c102a40 con 0x7f5514108810 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5511d9b640 1 -- 172.21.15.26:0/765171910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9bad0 msgr2=0x55e12fa9bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5511d9b640 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9bad0 0x55e12fa9bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5511d9b640 1 -- 172.21.15.26:0/765171910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9c4c0 msgr2=0x55e12faa0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5511d9b640 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9c4c0 0x55e12faa0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5511d9b640 1 -- 172.21.15.26:0/765171910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12f70d210 con 0x55e12faa1630 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5512d9d640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b28a0 0x7f54fc0b4d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f550c0c7210 tx=0x7f550c08fa30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5511d9b640 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12faa3ac0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5508016520 tx=0x7f550800a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfd1965db294a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5500ff9640 1 -- 172.21.15.26:0/765171910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508018430 con 0x55e12faa1630 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5500ff9640 1 -- 172.21.15.26:0/765171910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5508018a60 con 0x55e12faa1630 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.916+0000 7f5500ff9640 1 -- 172.21.15.26:0/765171910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508017690 con 0x55e12faa1630 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 -- 172.21.15.26:0/765171910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 msgr2=0x55e12faa3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12faa3ac0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5508016520 tx=0x7f550800a310 comp rx=0 tx=0).stop 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 -- 172.21.15.26:0/765171910 shutdown_connections 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12faa3ac0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9c4c0 0x55e12faa0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 --2- 172.21.15.26:0/765171910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9bad0 0x55e12fa9bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 -- 172.21.15.26:0/765171910 >> 172.21.15.26:0/765171910 conn(0x55e12fa92d30 msgr2=0x55e12fa93130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 -- 172.21.15.26:0/765171910 shutdown_connections 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 -- 172.21.15.26:0/765171910 wait complete. 2024-05-20T21:01:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 -- start start 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.917+0000 7f551bc39740 1 Processor -- start 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 0x55e12faa60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12faa1630 0x55e12faad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e12f6de790 con 0x55e12fa9c4c0 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e12f9b4890 con 0x55e12fa9bad0 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e12faa4580 con 0x55e12faa1630 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 10 client.0 _ll_register_callbacks cb 0x55e12f70ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 0x55e12faa60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 0x55e12faa60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43592/0 (socket says 172.21.15.26:43592) 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 -- 172.21.15.26:0/2697366456 learned_addr learned my addr 172.21.15.26:0/2697366456 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12faa1630 0x55e12faad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 -- 172.21.15.26:0/2697366456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12faa1630 msgr2=0x55e12faad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12faa1630 0x55e12faad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 -- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 msgr2=0x55e12faa5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 -- 172.21.15.26:0/2697366456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12f7d9620 con 0x55e12fa9c4c0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551259c640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 0x55e12faa60d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5504010b50 tx=0x7f550400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f40af2b025801ea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f54f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2697366456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550400e5c0 con 0x55e12fa9c4c0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2697366456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f550400e760 con 0x55e12fa9c4c0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.918+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5508018a60 con 0x55e12fa9c4c0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2697366456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f550400e960 con 0x55e12fa9c4c0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 10 client.16866 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e12fab01c0 con 0x55e12fa9c4c0 2024-05-20T21:01:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2697366456 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5508018a60 con 0x55e12fa9c4c0 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2697366456 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f5504022020 con 0x55e12fa9c4c0 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 10 client.16866 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 20 client.16866 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fab1900 0x55e12fab3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 4 client.16866 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e12fa0ab80 con 0x55e12fab1900 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.919+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fab1900 0x55e12fab3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.920+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fab1900 0x55e12fab3d50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e037ba79584a5e server_cookie=d58a3e9177fdf616 in_seq=0 out_seq=0 2024-05-20T21:01:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.920+0000 7f54f9ffb640 10 client.16866 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2697366456 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e12fa0ab80 con 0x55e12fab1900 2024-05-20T21:01:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f54f9ffb640 10 client.16866 handle_command_reply: tid=0 2024-05-20T21:01:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 client.16866 shutdown 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 msgr2=0x55e12faa60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 0x55e12faa60d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5504010b50 tx=0x7f550400f7f0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 shutdown_connections 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fab1900 0x55e12fab3d50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12faa1630 0x55e12faad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f54f9ffb640 0 client.16866 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9c4c0 0x55e12faa60d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 --2- 172.21.15.26:0/2697366456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 >> 172.21.15.26:0/2697366456 conn(0x55e12fa92d30 msgr2=0x55e12fa91fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 shutdown_connections 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 1 -- 172.21.15.26:0/2697366456 wait complete. 2024-05-20T21:01:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.921+0000 7f551bc39740 20 client.16866 trim_cache size 0 max 16384 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.936+0000 7f551bc39740 1 Processor -- start 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.936+0000 7f551bc39740 1 -- start start 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f551bc39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 0x55e12fa927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f551bc39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f551bc39740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12faa1630 0x55e12fa8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f551bc39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55141aa5b0 con 0x55e12fa9bad0 2024-05-20T21:01:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f551bc39740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f55141a7e80 con 0x55e12faa1630 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f551bc39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55141a5cd0 con 0x55e12fa9c4c0 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 0x55e12fa927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 0x55e12fa927d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43594/0 (socket says 172.21.15.26:43594) 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 -- 172.21.15.26:0/1138859940 learned_addr learned my addr 172.21.15.26:0/1138859940 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 -- 172.21.15.26:0/1138859940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 msgr2=0x55e12faad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 -- 172.21.15.26:0/1138859940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12faa1630 msgr2=0x55e12fa8dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12faa1630 0x55e12fa8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 -- 172.21.15.26:0/1138859940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12fab43d0 con 0x55e12fa9bad0 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f5511d9b640 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 0x55e12fa927d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5508006aa0 tx=0x7f550800aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb2774cdca648cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f54fbfff640 1 -- 172.21.15.26:0/1138859940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550802f020 con 0x55e12fa9bad0 2024-05-20T21:01:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f54fbfff640 1 -- 172.21.15.26:0/1138859940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5508017570 con 0x55e12fa9bad0 2024-05-20T21:01:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.937+0000 7f54fbfff640 1 -- 172.21.15.26:0/1138859940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508025670 con 0x55e12fa9bad0 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 -- 172.21.15.26:0/1138859940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 msgr2=0x55e12fa927d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 0x55e12fa927d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5508006aa0 tx=0x7f550800aa70 comp rx=0 tx=0).stop 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 -- 172.21.15.26:0/1138859940 shutdown_connections 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12faa1630 0x55e12fa8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 --2- 172.21.15.26:0/1138859940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12fa9bad0 0x55e12fa927d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 -- 172.21.15.26:0/1138859940 >> 172.21.15.26:0/1138859940 conn(0x55e12fa92d30 msgr2=0x55e12fa93110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 -- 172.21.15.26:0/1138859940 shutdown_connections 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 -- 172.21.15.26:0/1138859940 wait complete. 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 -- start start 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.938+0000 7f551bc39740 1 Processor -- start 2024-05-20T21:01:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faaa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12fb1a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55141a6340 con 0x55e12faa1630 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f55141a77b0 con 0x55e12fa9bad0 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55141a6a10 con 0x55e12fa9c4c0 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 10 client.0 _ll_register_callbacks cb 0x7f5504010760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12fb1a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12fb1a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43610/0 (socket says 172.21.15.26:43610) 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 -- 172.21.15.26:0/2406662588 learned_addr learned my addr 172.21.15.26:0/2406662588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551259c640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faaa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 -- 172.21.15.26:0/2406662588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 msgr2=0x55e12faaa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faaa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 -- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 msgr2=0x55e12faa9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 -- 172.21.15.26:0/2406662588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12fb1aa60 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551259c640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faaa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f5511d9b640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12fb1a520 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f550800a200 tx=0x7f5508018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6f3584d48a40f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f54f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2406662588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550803c060 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2406662588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f5508017920 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.939+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5508017570 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2406662588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f550802f030 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 10 client.16872 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e12fb1bbf0 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2406662588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5508017570 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2406662588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f5508034040 con 0x55e12faa1630 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 10 client.16872 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 20 client.16872 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fb1c300 0x55e12fb1e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 4 client.16872 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e12fa04360 con 0x55e12fb1c300 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.940+0000 7f551259c640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fb1c300 0x55e12fb1e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.941+0000 7f54f9ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.941+0000 7f551259c640 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fb1c300 0x55e12fb1e720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa034fb2f6c17554 server_cookie=f32bc67eb13de7a2 in_seq=0 out_seq=0 2024-05-20T21:01:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f54f9ffb640 1 -- 172.21.15.26:0/2406662588 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e12fa04360 con 0x55e12fb1c300 2024-05-20T21:01:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f54f9ffb640 10 client.16872 handle_command_reply: tid=0 2024-05-20T21:01:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 client.16872 shutdown 2024-05-20T21:01:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 msgr2=0x55e12fb1a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12fb1a520 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f550800a200 tx=0x7f5508018f90 comp rx=0 tx=0).stop 2024-05-20T21:01:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 shutdown_connections 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e12fb1c300 0x55e12fb1e720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e12faa1630 0x55e12fb1a520 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e12fa9c4c0 0x55e12faaa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 --2- 172.21.15.26:0/2406662588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e12fa9bad0 0x55e12faa9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f54f9ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 >> 172.21.15.26:0/2406662588 conn(0x55e12fa92d30 msgr2=0x55e12faa0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 shutdown_connections 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 1 -- 172.21.15.26:0/2406662588 wait complete. 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.966+0000 7f551bc39740 20 client.16872 trim_cache size 0 max 16384 2024-05-20T21:01:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b28a0 msgr2=0x7f54fc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b28a0 0x7f54fc0b4d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f550c0c7210 tx=0x7f550c08fa30 comp rx=0 tx=0).stop 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 msgr2=0x7f55141a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f55141a00f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f550c005940 tx=0x7f550c0042f0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 shutdown_connections 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f54fc0b28a0 0x7f54fc0b4d60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514108810 0x7f55141a00f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5514105ee0 0x7f5514199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 --2- 172.21.15.26:0/2573063641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55141035b0 0x7f5514198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 >> 172.21.15.26:0/2573063641 conn(0x7f55140fcf20 msgr2=0x7f5514103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 shutdown_connections 2024-05-20T21:01:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:13.967+0000 7f5518f19640 1 -- 172.21.15.26:0/2573063641 wait complete. 2024-05-20T21:01:14.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.13 MB/sec execute 80 sec latency 13343.648 ms 2024-05-20T21:01:15.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.10 MB/sec execute 81 sec latency 14343.762 ms 2024-05-20T21:01:15.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:15 smithi055 ceph-mon[35213]: pgmap v542: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 234 KiB/s wr, 55 op/s 2024-05-20T21:01:15.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:15 smithi026 ceph-mon[31990]: pgmap v542: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 234 KiB/s wr, 55 op/s 2024-05-20T21:01:15.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:15 smithi144 ceph-mon[35660]: pgmap v542: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 234 KiB/s wr, 55 op/s 2024-05-20T21:01:16.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.08 MB/sec execute 82 sec latency 15343.868 ms 2024-05-20T21:01:16.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:01:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:17 smithi026 ceph-mon[31990]: pgmap v543: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-05-20T21:01:17.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.05 MB/sec execute 83 sec latency 16343.930 ms 2024-05-20T21:01:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:17 smithi055 ceph-mon[35213]: pgmap v543: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-05-20T21:01:17.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:17 smithi144 ceph-mon[35660]: pgmap v543: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-05-20T21:01:18.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.03 MB/sec execute 84 sec latency 17344.050 ms 2024-05-20T21:01:19.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:19 smithi026 ceph-mon[31990]: pgmap v544: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-20T21:01:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 -- 172.21.15.26:0/2558879876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c1035d0 msgr2=0x7f527c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 --2- 172.21.15.26:0/2558879876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c1035d0 0x7f527c103a50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f526c0099c0 tx=0x7f526c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 -- 172.21.15.26:0/2558879876 shutdown_connections 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 --2- 172.21.15.26:0/2558879876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f527c103f90 0x7f527c110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 --2- 172.21.15.26:0/2558879876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527c109670 0x7f527c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 --2- 172.21.15.26:0/2558879876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c1035d0 0x7f527c103a50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.395+0000 7f528340a700 1 -- 172.21.15.26:0/2558879876 >> 172.21.15.26:0/2558879876 conn(0x7f527c07a730 msgr2=0x7f527c07ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.396+0000 7f528340a700 1 -- 172.21.15.26:0/2558879876 shutdown_connections 2024-05-20T21:01:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.396+0000 7f528340a700 1 -- 172.21.15.26:0/2558879876 wait complete. 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.396+0000 7f528340a700 1 Processor -- start 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.396+0000 7f528340a700 1 -- start start 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f528340a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f527c1035d0 0x7f527c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f528340a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 0x7f527c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f528340a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527c109670 0x7f527c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f528340a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f527c113680 con 0x7f527c103f90 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f528340a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f527c113500 con 0x7f527c1035d0 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f528340a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f527c113800 con 0x7f527c109670 2024-05-20T21:01:19.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 0x7f527c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 0x7f527c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43618/0 (socket says 172.21.15.26:43618) 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52811a6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f527c1035d0 0x7f527c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 -- 172.21.15.26:0/1877178635 learned_addr learned my addr 172.21.15.26:0/1877178635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52819a7700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527c109670 0x7f527c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 -- 172.21.15.26:0/1877178635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527c109670 msgr2=0x7f527c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527c109670 0x7f527c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 -- 172.21.15.26:0/1877178635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f527c1035d0 msgr2=0x7f527c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f527c1035d0 0x7f527c19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 -- 172.21.15.26:0/1877178635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c1a0d80 con 0x7f527c103f90 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.397+0000 7f52809a5700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 0x7f527c19c2b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f526c006010 tx=0x7f526c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.398+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c026070 con 0x7f527c103f90 2024-05-20T21:01:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.398+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c1a1010 con 0x7f527c103f90 2024-05-20T21:01:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.398+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f526c00f460 con 0x7f527c103f90 2024-05-20T21:01:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.398+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c009e60 con 0x7f527c103f90 2024-05-20T21:01:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.398+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c1a88f0 con 0x7f527c103f90 2024-05-20T21:01:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.399+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527c06a230 con 0x7f527c103f90 2024-05-20T21:01:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.400+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f526c029070 con 0x7f527c103f90 2024-05-20T21:01:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.401+0000 7f52727fc700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52680b2880 0x7f52680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.401+0000 7f52811a6700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52680b2880 0x7f52680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.401+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f526c0e2ad0 con 0x7f527c103f90 2024-05-20T21:01:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.401+0000 7f52811a6700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52680b2880 0x7f52680b4d40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5278009be0 tx=0x7f5278009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:19.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.405+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f526c013070 con 0x7f527c103f90 2024-05-20T21:01:19.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 2.00 MB/sec execute 85 sec latency 18344.116 ms 2024-05-20T21:01:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.680+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f527c1a8ac0 con 0x7f527c103f90 2024-05-20T21:01:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.681+0000 7f52727fc700 1 -- 172.21.15.26:0/1877178635 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f526c0aa820 con 0x7f527c103f90 2024-05-20T21:01:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:01:19.681 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:01:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52680b2880 msgr2=0x7f52680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52680b2880 0x7f52680b4d40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5278009be0 tx=0x7f5278009260 comp rx=0 tx=0).stop 2024-05-20T21:01:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 msgr2=0x7f527c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 0x7f527c19c2b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f526c006010 tx=0x7f526c0045e0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 shutdown_connections 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527c109670 0x7f527c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f527c1035d0 0x7f527c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52680b2880 0x7f52680b4d40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 --2- 172.21.15.26:0/1877178635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527c103f90 0x7f527c19c2b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 >> 172.21.15.26:0/1877178635 conn(0x7f527c07a730 msgr2=0x7f527c1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 shutdown_connections 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:19.683+0000 7f528340a700 1 -- 172.21.15.26:0/1877178635 wait complete. 2024-05-20T21:01:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:01:19.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:19 smithi055 ceph-mon[35213]: pgmap v544: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-20T21:01:19.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:19 smithi144 ceph-mon[35660]: pgmap v544: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-20T21:01:20.510 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id c3a433f9 --release --await' 2024-05-20T21:01:20.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id c3a433f9 --release --await 2024-05-20T21:01:20.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 1.98 MB/sec execute 86 sec latency 19344.160 ms 2024-05-20T21:01:20.662 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:20 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1877178635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:20.665 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:01:20.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:20 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1877178635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:20.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:20 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1877178635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:21 smithi026 ceph-mon[31990]: pgmap v545: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-20T21:01:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:21.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 1.96 MB/sec execute 87 sec latency 20344.194 ms 2024-05-20T21:01:21.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:21 smithi055 ceph-mon[35213]: pgmap v545: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-20T21:01:21.701 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:21 smithi144 ceph-mon[35660]: pgmap v545: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-20T21:01:22.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 1.94 MB/sec execute 88 sec latency 21344.327 ms 2024-05-20T21:01:22.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:22 smithi055 ceph-mon[35213]: pgmap v546: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-05-20T21:01:22.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:22 smithi026 ceph-mon[31990]: pgmap v546: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-05-20T21:01:22.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:22 smithi144 ceph-mon[35660]: pgmap v546: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-05-20T21:01:23.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97580 1.91 MB/sec execute 89 sec latency 22344.438 ms 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.038+0000 7f8471c65700 1 -- 172.21.15.26:0/1624191024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 msgr2=0x7f846c06c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.038+0000 7f8471c65700 1 --2- 172.21.15.26:0/1624191024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c06c2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f845c0099c0 tx=0x7f845c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 -- 172.21.15.26:0/1624191024 shutdown_connections 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 --2- 172.21.15.26:0/1624191024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c06c2f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 --2- 172.21.15.26:0/1624191024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846c108860 0x7f846c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 --2- 172.21.15.26:0/1624191024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f846c06c8c0 0x7f846c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 -- 172.21.15.26:0/1624191024 >> 172.21.15.26:0/1624191024 conn(0x7f846c078490 msgr2=0x7f846c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 -- 172.21.15.26:0/1624191024 shutdown_connections 2024-05-20T21:01:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 -- 172.21.15.26:0/1624191024 wait complete. 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 Processor -- start 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.039+0000 7f8471c65700 1 -- start start 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8471c65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8471c65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f846c06c8c0 0x7f846c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8471c65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846c108860 0x7f846c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8471c65700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113600 con 0x7f846c06bf10 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8471c65700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113480 con 0x7f846c108860 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8471c65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113780 con 0x7f846c06c8c0 2024-05-20T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49572/0 (socket says 172.21.15.26:49572) 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 -- 172.21.15.26:0/618089875 learned_addr learned my addr 172.21.15.26:0/618089875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846bfff700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846c108860 0x7f846c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f8462ffd700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f846c06c8c0 0x7f846c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 -- 172.21.15.26:0/618089875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f846c06c8c0 msgr2=0x7f846c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f846c06c8c0 0x7f846c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 -- 172.21.15.26:0/618089875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846c108860 msgr2=0x7f846c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846c108860 0x7f846c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 -- 172.21.15.26:0/618089875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c19a140 con 0x7f846c06bf10 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.040+0000 7f846b7fe700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c19f7d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f845c000c00 tx=0x7f845c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.041+0000 7f84697fa700 1 -- 172.21.15.26:0/618089875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c00f8a0 con 0x7f846c06bf10 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.041+0000 7f84697fa700 1 -- 172.21.15.26:0/618089875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f845c004830 con 0x7f846c06bf10 2024-05-20T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.041+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c19a3d0 con 0x7f846c06bf10 2024-05-20T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.041+0000 7f84697fa700 1 -- 172.21.15.26:0/618089875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c005410 con 0x7f846c06bf10 2024-05-20T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.041+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a66a0 con 0x7f846c06bf10 2024-05-20T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.044+0000 7f84697fa700 1 -- 172.21.15.26:0/618089875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f845c029060 con 0x7f846c06bf10 2024-05-20T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f84697fa700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84580b2880 0x7f84580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f8462ffd700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84580b2880 0x7f84580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f84697fa700 1 -- 172.21.15.26:0/618089875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f845c0e2da0 con 0x7f846c06bf10 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 Processor -- start 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 -- start start 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf048ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf048f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf0484bd90 con 0x55cf048ec8e0 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cf0483b930 con 0x55cf048f22b0 2024-05-20T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf0483f970 con 0x55cf048ed2a0 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.045+0000 7f8462ffd700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84580b2880 0x7f84580b4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8454005950 tx=0x7f84540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf048ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf048ecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49582/0 (socket says 172.21.15.26:49582) 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8462ffd700 1 -- 172.21.15.26:0/626818293 learned_addr learned my addr 172.21.15.26:0/626818293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846bfff700 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf048f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 -- 172.21.15.26:0/626818293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 msgr2=0x55cf048f90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf048f90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 -- 172.21.15.26:0/626818293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 msgr2=0x55cf048ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf048ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 -- 172.21.15.26:0/626818293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf0478b9d0 con 0x55cf048f22b0 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8462ffd700 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf048ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f846b7fe700 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048f2690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f845c0e07e0 tx=0x7f845c0a9bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ed77c144ed0c04 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8461ffb700 1 -- 172.21.15.26:0/626818293 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0ae740 con 0x55cf048f22b0 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8461ffb700 1 -- 172.21.15.26:0/626818293 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f845c0b3160 con 0x55cf048f22b0 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.046+0000 7f8461ffb700 1 -- 172.21.15.26:0/626818293 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0aa850 con 0x55cf048f22b0 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 -- 172.21.15.26:0/626818293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 msgr2=0x55cf048f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048f2690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f845c0e07e0 tx=0x7f845c0a9bf0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 -- 172.21.15.26:0/626818293 shutdown_connections 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf048f90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048f2690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/626818293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf048ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 -- 172.21.15.26:0/626818293 >> 172.21.15.26:0/626818293 conn(0x55cf0481a180 msgr2=0x55cf048e71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 -- 172.21.15.26:0/626818293 shutdown_connections 2024-05-20T21:01:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.047+0000 7f847c4fbb80 1 -- 172.21.15.26:0/626818293 wait complete. 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 -- start start 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 Processor -- start 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf048fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048ed2a0 0x55cf048fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 0x55cf048f6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf047420a0 con 0x55cf048f22b0 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cf047dd8c0 con 0x55cf048ed2a0 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf0483b930 con 0x55cf048ec8e0 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f847c4fbb80 10 client.0 _ll_register_callbacks cb 0x55cf047d6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f846bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 0x55cf048f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f846bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 0x55cf048f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49586/0 (socket says 172.21.15.26:49586) 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f846bfff700 1 -- 172.21.15.26:0/1468776859 learned_addr learned my addr 172.21.15.26:0/1468776859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f8462ffd700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048ed2a0 0x55cf048fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.048+0000 7f846b7fe700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf048fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f846bfff700 1 -- 172.21.15.26:0/1468776859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 msgr2=0x55cf048fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f846bfff700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf048fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f846bfff700 1 -- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048ed2a0 msgr2=0x55cf048fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f846bfff700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048ed2a0 0x55cf048fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f846bfff700 1 -- 172.21.15.26:0/1468776859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf047da790 con 0x55cf048f22b0 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f846bfff700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 0x55cf048f6020 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8464003fd0 tx=0x7f8464004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca213bd46f3c68c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f844affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f844affd700 1 -- 172.21.15.26:0/1468776859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846401a070 con 0x55cf048f22b0 2024-05-20T21:01:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f844affd700 1 -- 172.21.15.26:0/1468776859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8464004db0 con 0x55cf048f22b0 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845c0b3160 con 0x55cf048f22b0 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f844affd700 1 -- 172.21.15.26:0/1468776859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8464025050 con 0x55cf048f22b0 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f847c4fbb80 10 client.16890 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf04907e70 con 0x55cf048f22b0 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.049+0000 7f844affd700 1 -- 172.21.15.26:0/1468776859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f845c0b3160 con 0x55cf048f22b0 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f844affd700 1 -- 172.21.15.26:0/1468776859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f84640164c0 con 0x55cf048f22b0 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f847c4fbb80 10 client.16890 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f847c4fbb80 10 client.16890 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f847c4fbb80 20 client.16890 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf0490a450 0x55cf0490c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f847c4fbb80 4 client.16890 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf048b8ae0 con 0x55cf0490a450 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.050+0000 7f846b7fe700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf0490a450 0x55cf0490c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f846b7fe700 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf0490a450 0x55cf0490c870 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1a758c27ce58ad server_cookie=e995bbb6245a2658 in_seq=0 out_seq=0 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f844affd700 10 client.16890 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f844affd700 1 -- 172.21.15.26:0/1468776859 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cf048b8ae0 con 0x55cf0490a450 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f844affd700 10 client.16890 handle_command_reply: tid=0 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f847c4fbb80 1 client.16890 shutdown 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 msgr2=0x55cf048f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.051+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 0x55cf048f6020 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8464003fd0 tx=0x7f8464004040 comp rx=0 tx=0).stop 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 shutdown_connections 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf0490a450 0x55cf0490c870 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048f22b0 0x55cf048f6020 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf048fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/1468776859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048ed2a0 0x55cf048fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 >> 172.21.15.26:0/1468776859 conn(0x55cf0481a180 msgr2=0x55cf048e1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f844affd700 0 client.16890 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 shutdown_connections 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 1 -- 172.21.15.26:0/1468776859 wait complete. 2024-05-20T21:01:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.052+0000 7f847c4fbb80 20 client.16890 trim_cache size 0 max 16384 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.072+0000 7f847c4fbb80 1 Processor -- start 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.072+0000 7f847c4fbb80 1 -- start start 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf04912120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 0x55cf04912660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1ae250 con 0x55cf048ed2a0 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1abae0 con 0x55cf048f22b0 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1a9930 con 0x55cf048ec8e0 2024-05-20T21:01:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 0x55cf04912660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 0x55cf04912660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49590/0 (socket says 172.21.15.26:49590) 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 -- 172.21.15.26:0/465318536 learned_addr learned my addr 172.21.15.26:0/465318536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f846bfff700 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f846b7fe700 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf04912120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 -- 172.21.15.26:0/465318536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 msgr2=0x55cf04912120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf04912120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 -- 172.21.15.26:0/465318536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 msgr2=0x55cf048efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 -- 172.21.15.26:0/465318536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf0490ced0 con 0x55cf048ed2a0 2024-05-20T21:01:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f846bfff700 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.073+0000 7f8462ffd700 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 0x55cf04912660 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f84540115d0 tx=0x7f8454012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9a384c6f663a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f84617fa700 1 -- 172.21.15.26:0/465318536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454011cf0 con 0x55cf048ed2a0 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f84617fa700 1 -- 172.21.15.26:0/465318536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f845401f8c0 con 0x55cf048ed2a0 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f84617fa700 1 -- 172.21.15.26:0/465318536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845401e820 con 0x55cf048ed2a0 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 -- 172.21.15.26:0/465318536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 msgr2=0x55cf04912660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 0x55cf04912660 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f84540115d0 tx=0x7f8454012000 comp rx=0 tx=0).stop 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 -- 172.21.15.26:0/465318536 shutdown_connections 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf048efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ec8e0 0x55cf04912120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/465318536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ed2a0 0x55cf04912660 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 -- 172.21.15.26:0/465318536 >> 172.21.15.26:0/465318536 conn(0x55cf0481a180 msgr2=0x55cf048ec160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 -- 172.21.15.26:0/465318536 shutdown_connections 2024-05-20T21:01:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.074+0000 7f847c4fbb80 1 -- 172.21.15.26:0/465318536 wait complete. 2024-05-20T21:01:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 -- start start 2024-05-20T21:01:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 Processor -- start 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf04902130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf04902670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf04900670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1a9fa0 con 0x55cf048ec8e0 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1ab410 con 0x55cf048f22b0 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1aa670 con 0x55cf048ed2a0 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f847c4fbb80 10 client.0 _ll_register_callbacks cb 0x55cf047d6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.075+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf04902130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf04902130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49604/0 (socket says 172.21.15.26:49604) 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 -- 172.21.15.26:0/3734746066 learned_addr learned my addr 172.21.15.26:0/3734746066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846bfff700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf04900670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f8462ffd700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf04902670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 -- 172.21.15.26:0/3734746066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 msgr2=0x55cf04902670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf04902670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 -- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 msgr2=0x55cf04900670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf04900670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 -- 172.21.15.26:0/3734746066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf04900d40 con 0x55cf048ec8e0 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f8462ffd700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf04902670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f846b7fe700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf04902130 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f845c0aa090 tx=0x7f845c0e0b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dd269b1aaf376e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f844affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f844affd700 1 -- 172.21.15.26:0/3734746066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c06d070 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f844affd700 1 -- 172.21.15.26:0/3734746066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f845c0055b0 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845401f8c0 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f844affd700 1 -- 172.21.15.26:0/3734746066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f845c0b3160 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f847c4fbb80 10 client.16896 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.076+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf04975b90 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f844affd700 1 -- 172.21.15.26:0/3734746066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f845401f8c0 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f844affd700 1 -- 172.21.15.26:0/3734746066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f845c0b2cd0 con 0x55cf048ec8e0 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f847c4fbb80 10 client.16896 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f847c4fbb80 10 client.16896 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f847c4fbb80 20 client.16896 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf04976190 0x55cf049785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f847c4fbb80 4 client.16896 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c3a433f9", "release": true, "await": true}] 2024-05-20T21:01:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c3a433f9", "release": true, "await": true}) -- 0x55cf048f1d60 con 0x55cf04976190 2024-05-20T21:01:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.077+0000 7f8462ffd700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf04976190 0x55cf049785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.078+0000 7f844affd700 10 client.16896 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.078+0000 7f8462ffd700 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf04976190 0x55cf049785b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9d9f9dae41d2ee server_cookie=4d96183abbca4697 in_seq=0 out_seq=0 2024-05-20T21:01:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f844affd700 1 -- 172.21.15.26:0/3734746066 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf048f1d60 con 0x55cf04976190 2024-05-20T21:01:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f844affd700 10 client.16896 handle_command_reply: tid=0 2024-05-20T21:01:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 client.16896 shutdown 2024-05-20T21:01:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 msgr2=0x55cf04902130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf04902130 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f845c0aa090 tx=0x7f845c0e0b00 comp rx=0 tx=0).stop 2024-05-20T21:01:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 shutdown_connections 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cf04976190 0x55cf049785b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf048ec8e0 0x55cf04902130 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf048f22b0 0x55cf04900670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 --2- 172.21.15.26:0/3734746066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf048ed2a0 0x55cf04902670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 >> 172.21.15.26:0/3734746066 conn(0x55cf0481a180 msgr2=0x55cf048efd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f844affd700 0 client.16896 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 shutdown_connections 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.086+0000 7f847c4fbb80 1 -- 172.21.15.26:0/3734746066 wait complete. 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f847c4fbb80 20 client.16896 trim_cache size 0 max 16384 2024-05-20T21:01:24.088 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 28, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 20, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "c3a433f9": { 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 20, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:01:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.3, 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:24.090 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84580b2880 msgr2=0x7f84580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84580b2880 0x7f84580b4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8454005950 tx=0x7f84540058e0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 msgr2=0x7f846c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c19f7d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f845c000c00 tx=0x7f845c0047b0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 shutdown_connections 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84580b2880 0x7f84580b4d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c06bf10 0x7f846c19f7d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846c108860 0x7f846c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 --2- 172.21.15.26:0/618089875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f846c06c8c0 0x7f846c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 >> 172.21.15.26:0/618089875 conn(0x7f846c078490 msgr2=0x7f846c1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 shutdown_connections 2024-05-20T21:01:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.087+0000 7f8471c65700 1 -- 172.21.15.26:0/618089875 wait complete. 2024-05-20T21:01:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/3361243951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac108510 msgr2=0x7f1fac1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/3361243951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac108510 0x7f1fac1088f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009990 tx=0x7f1fa8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/3361243951 shutdown_connections 2024-05-20T21:01:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/3361243951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac108510 0x7f1fac1088f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/3361243951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac102df0 0x7f1fac10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/3361243951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 0x7f1fac1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/3361243951 >> 172.21.15.26:0/3361243951 conn(0x7f1fac0793b0 msgr2=0x7f1fac0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/3361243951 shutdown_connections 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.254+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/3361243951 wait complete. 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 Processor -- start 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 -- start start 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 0x7f1fac19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 0x7f1fac19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac108510 0x7f1fac197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1113c0 con 0x7f1fac102df0 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac111240 con 0x7f1fac108510 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.255+0000 7f1fb3c4c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac111540 con 0x7f1fac102430 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 0x7f1fac19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 0x7f1fac19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49612/0 (socket says 172.21.15.26:49612) 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb11e7700 1 -- 172.21.15.26:0/1535369888 learned_addr learned my addr 172.21.15.26:0/1535369888 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac108510 0x7f1fac197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 0x7f1fac19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 -- 172.21.15.26:0/1535369888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 msgr2=0x7f1fac19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 0x7f1fac19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 -- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 msgr2=0x7f1fac19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 0x7f1fac19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 -- 172.21.15.26:0/1535369888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fac197fe0 con 0x7f1fac108510 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 0x7f1fac19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 0x7f1fac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac108510 0x7f1fac197700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e390 tx=0x7f1fa400e6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fa2ffd700 1 -- 172.21.15.26:0/1535369888 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400ee40 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fac198270 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fa2ffd700 1 -- 172.21.15.26:0/1535369888 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa400b560 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fa2ffd700 1 -- 172.21.15.26:0/1535369888 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4009d40 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.256+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fac1a43e0 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.259+0000 7f1fa2ffd700 1 -- 172.21.15.26:0/1535369888 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1fa401c020 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.260+0000 7f1fa2ffd700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f980b2880 0x7f1f980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.260+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f980b2880 0x7f1f980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.260+0000 7f1fa2ffd700 1 -- 172.21.15.26:0/1535369888 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1fa40e3df0 con 0x7f1fac108510 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.260+0000 7f1fbe4e2b80 1 Processor -- start 2024-05-20T21:01:24.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 -- start start 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f980b2880 0x7f1f980b4d40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800b2e0 tx=0x7f1fa801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d568909c90 con 0x55d5689b3370 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d5688f9830 con 0x55d5689af620 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d5688fd870 con 0x55d5689aec70 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40316/0 (socket says 172.21.15.26:40316) 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fb11e7700 1 -- 172.21.15.26:0/1411765153 learned_addr learned my addr 172.21.15.26:0/1411765153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.261+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1fb11e7700 1 -- 172.21.15.26:0/1411765153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 msgr2=0x55d5689af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1fb11e7700 1 -- 172.21.15.26:0/1411765153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 msgr2=0x55d5689b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1fb11e7700 1 -- 172.21.15.26:0/1411765153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5688498d0 con 0x55d5689af620 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b2d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c010580 tx=0x7f1f9c010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b2be832f284edeb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1f97fff700 1 -- 172.21.15.26:0/1411765153 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c01c680 con 0x55d5689af620 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1f97fff700 1 -- 172.21.15.26:0/1411765153 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9c01ccb0 con 0x55d5689af620 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.262+0000 7f1f97fff700 1 -- 172.21.15.26:0/1411765153 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011c40 con 0x55d5689af620 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/1411765153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 msgr2=0x55d5689b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b2d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c010580 tx=0x7f1f9c010890 comp rx=0 tx=0).stop 2024-05-20T21:01:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/1411765153 shutdown_connections 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/1411765153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b2d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/1411765153 >> 172.21.15.26:0/1411765153 conn(0x55d5688d8080 msgr2=0x55d5689a4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/1411765153 shutdown_connections 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.263+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/1411765153 wait complete. 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 -- start start 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 Processor -- start 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d5687fffa0 con 0x55d5689b3370 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d56889b7c0 con 0x55d5689af620 2024-05-20T21:01:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d5689bb530 con 0x55d5689aec70 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fbe4e2b80 10 client.0 _ll_register_callbacks cb 0x55d568894920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fb21e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fb21e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49624/0 (socket says 172.21.15.26:49624) 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.264+0000 7f1fb21e9700 1 -- 172.21.15.26:0/3833418811 learned_addr learned my addr 172.21.15.26:0/3833418811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 -- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 msgr2=0x55d5689b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 -- 172.21.15.26:0/3833418811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 msgr2=0x55d5689b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 -- 172.21.15.26:0/3833418811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d568898690 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689ba770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800b650 tx=0x7f1fa8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851603903c320bae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1f957fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1f957fa700 1 -- 172.21.15.26:0/3833418811 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8023810 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9c01ccb0 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.265+0000 7f1f957fa700 1 -- 172.21.15.26:0/3833418811 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1fa80239b0 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1f957fa700 1 -- 172.21.15.26:0/3833418811 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fa8023bb0 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1fbe4e2b80 10 client.26305 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5689c5c20 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1f957fa700 1 -- 172.21.15.26:0/3833418811 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9c01ccb0 con 0x55d5689aec70 2024-05-20T21:01:24.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1f957fa700 1 -- 172.21.15.26:0/3833418811 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f1fa8022070 con 0x55d5689aec70 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1fbe4e2b80 10 client.26305 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.266+0000 7f1fbe4e2b80 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1fbe4e2b80 20 client.26305 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5689c82b0 0x55d5689ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1fbe4e2b80 4 client.26305 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5689769e0 con 0x55d5689c82b0 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5689c82b0 0x55d5689ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5689c82b0 0x55d5689ca6d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c350ceaad25d68 server_cookie=9a6298cd94b3bdce in_seq=0 out_seq=0 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.267+0000 7f1f957fa700 10 client.26305 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1f957fa700 1 -- 172.21.15.26:0/3833418811 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d5689769e0 con 0x55d5689c82b0 2024-05-20T21:01:24.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1f957fa700 10 client.26305 handle_command_reply: tid=0 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 client.26305 shutdown 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 msgr2=0x55d5689ba770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689ba770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800b650 tx=0x7f1fa8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 shutdown_connections 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d5689c82b0 0x55d5689ca6d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689ba770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3833418811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1f957fa700 0 client.26305 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 >> 172.21.15.26:0/3833418811 conn(0x55d5688d8080 msgr2=0x55d5689b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.268+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 shutdown_connections 2024-05-20T21:01:24.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.269+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3833418811 wait complete. 2024-05-20T21:01:24.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.269+0000 7f1fbe4e2b80 20 client.26305 trim_cache size 0 max 16384 2024-05-20T21:01:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.294+0000 7f1fbe4e2b80 1 Processor -- start 2024-05-20T21:01:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.294+0000 7f1fbe4e2b80 1 -- start start 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689a5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1ae1f0 con 0x55d5689b3370 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1aba80 con 0x55d5689af620 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1a98d0 con 0x55d5689aec70 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49630/0 (socket says 172.21.15.26:49630) 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 -- 172.21.15.26:0/3756622379 learned_addr learned my addr 172.21.15.26:0/3756622379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 -- 172.21.15.26:0/3756622379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 msgr2=0x55d5689a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 -- 172.21.15.26:0/3756622379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 msgr2=0x55d5689cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 -- 172.21.15.26:0/3756622379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5689b1a10 con 0x55d5689b3370 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689d04e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40a8620 tx=0x7f1fa4093050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b227ebe9ed45b393 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.295+0000 7f1f977fe700 1 -- 172.21.15.26:0/3756622379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40aaec0 con 0x55d5689b3370 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1f977fe700 1 -- 172.21.15.26:0/3756622379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa40c5070 con 0x55d5689b3370 2024-05-20T21:01:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1f977fe700 1 -- 172.21.15.26:0/3756622379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40b3fe0 con 0x55d5689b3370 2024-05-20T21:01:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3756622379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 msgr2=0x55d5689d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689d04e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40a8620 tx=0x7f1fa4093050 comp rx=0 tx=0).stop 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3756622379 shutdown_connections 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689b3370 0x55d5689d04e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689aec70 0x55d5689a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/3756622379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689af620 0x55d5689cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.296+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3756622379 >> 172.21.15.26:0/3756622379 conn(0x55d5688d8080 msgr2=0x55d5689a9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.297+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3756622379 shutdown_connections 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.297+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/3756622379 wait complete. 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.297+0000 7f1fbe4e2b80 1 -- start start 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.297+0000 7f1fbe4e2b80 1 Processor -- start 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 0x55d5689c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 0x55d5689c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689b3370 0x55d5689bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1a9f40 con 0x55d5689af620 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1ab3b0 con 0x55d5689aec70 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1aa610 con 0x55d5689b3370 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fbe4e2b80 10 client.0 _ll_register_callbacks cb 0x55d568894920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 0x55d5689c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb19e8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 0x55d5689c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 0x55d5689c1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49636/0 (socket says 172.21.15.26:49636) 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb19e8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 0x55d5689c0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40342/0 (socket says 172.21.15.26:40342) 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 -- 172.21.15.26:0/2060901555 learned_addr learned my addr 172.21.15.26:0/2060901555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689b3370 0x55d5689bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 -- 172.21.15.26:0/2060901555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689b3370 msgr2=0x55d5689bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689b3370 0x55d5689bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 -- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 msgr2=0x55d5689c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 0x55d5689c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 -- 172.21.15.26:0/2060901555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5689bf910 con 0x55d5689af620 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 0x55d5689c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb21e9700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689b3370 0x55d5689bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1fb11e7700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 0x55d5689c1010 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b750 tx=0x7f1f9c00bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638f5de1247cf468 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.298+0000 7f1f957fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1f957fa700 1 -- 172.21.15.26:0/2060901555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c005590 con 0x55d5689af620 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1f957fa700 1 -- 172.21.15.26:0/2060901555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1f9c0046f0 con 0x55d5689af620 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa40c5070 con 0x55d5689af620 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1f957fa700 1 -- 172.21.15.26:0/2060901555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f9c0048f0 con 0x55d5689af620 2024-05-20T21:01:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 10 client.16914 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d568a33a60 con 0x55d5689af620 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1f957fa700 1 -- 172.21.15.26:0/2060901555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9c011600 con 0x55d5689af620 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1f957fa700 1 -- 172.21.15.26:0/2060901555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f1f9c01c480 con 0x55d5689af620 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 10 client.16914 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 20 client.16914 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.299+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d568a33fb0 0x55d568a363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.300+0000 7f1fbe4e2b80 4 client.16914 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:01:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.300+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1fa40c5070 con 0x55d568a33fb0 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.300+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d568a33fb0 0x55d568a363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.301+0000 7f1fb19e8700 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d568a33fb0 0x55d568a363d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c422a03a6a1a6cd server_cookie=6e4e7134069c0e6c in_seq=0 out_seq=0 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.301+0000 7f1f957fa700 10 client.16914 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.301+0000 7f1f957fa700 1 -- 172.21.15.26:0/2060901555 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f1fa40c5070 con 0x55d568a33fb0 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.301+0000 7f1f957fa700 10 client.16914 handle_command_reply: tid=0 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.301+0000 7f1fbe4e2b80 1 client.16914 shutdown 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 msgr2=0x55d5689c1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 0x55d5689c1010 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b750 tx=0x7f1f9c00bc90 comp rx=0 tx=0).stop 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 shutdown_connections 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d568a33fb0 0x55d568a363d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5689b3370 0x55d5689bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5689aec70 0x55d5689c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 --2- 172.21.15.26:0/2060901555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5689af620 0x55d5689c1010 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 >> 172.21.15.26:0/2060901555 conn(0x55d5688d8080 msgr2=0x55d5689b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1f957fa700 0 client.16914 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 shutdown_connections 2024-05-20T21:01:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 1 -- 172.21.15.26:0/2060901555 wait complete. 2024-05-20T21:01:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.302+0000 7f1fbe4e2b80 20 client.16914 trim_cache size 0 max 16384 2024-05-20T21:01:24.309 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:24.309 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:01:24.309 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:01:24.309 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f980b2880 msgr2=0x7f1f980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f980b2880 0x7f1f980b4d40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800b2e0 tx=0x7f1fa801a040 comp rx=0 tx=0).stop 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac108510 msgr2=0x7f1fac197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac108510 0x7f1fac197700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e390 tx=0x7f1fa400e6a0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 shutdown_connections 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fac108510 0x7f1fac197700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fac102430 0x7f1fac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1f980b2880 0x7f1f980b4d40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 --2- 172.21.15.26:0/1535369888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fac102df0 0x7f1fac19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 >> 172.21.15.26:0/1535369888 conn(0x7f1fac0793b0 msgr2=0x7f1fac1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 shutdown_connections 2024-05-20T21:01:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:24.303+0000 7f1fb3c4c700 1 -- 172.21.15.26:0/1535369888 wait complete. 2024-05-20T21:01:24.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97879 1.90 MB/sec execute 90 sec latency 22876.751 ms 2024-05-20T21:01:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:24 smithi026 ceph-mon[31990]: pgmap v547: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-05-20T21:01:25.123 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c3a433f9', total seconds elapsed: 29.7 2024-05-20T21:01:25.123 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.3 2024-05-20T21:01:25.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:01:25.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:24 smithi055 ceph-mon[35213]: pgmap v547: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-05-20T21:01:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:24 smithi144 ceph-mon[35660]: pgmap v547: 97 pgs: 97 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-05-20T21:01:25.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 98438 1.90 MB/sec execute 91 sec latency 18.651 ms 2024-05-20T21:01:25.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:01:25.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:01:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:26.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 99186 1.95 MB/sec execute 92 sec latency 15.268 ms 2024-05-20T21:01:26.664 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:27.045 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:26 smithi026 ceph-mon[31990]: pgmap v548: 97 pgs: 97 active+clean; 25 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 118 KiB/s wr, 15 op/s 2024-05-20T21:01:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c1ffff640 1 -- 172.21.15.26:0/1431136529 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18037070 con 0x7f8c28101c00 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1431136529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c28101c00 msgr2=0x7f8c28102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1431136529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c28101c00 0x7f8c28102080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009860 tx=0x7f8c1802a340 comp rx=0 tx=0).stop 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1431136529 shutdown_connections 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1431136529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c281025c0 0x7f8c28109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1431136529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c28101c00 0x7f8c28102080 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1431136529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28102fb0 0x7f8c281033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.070+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1431136529 >> 172.21.15.26:0/1431136529 conn(0x7f8c280fc6d0 msgr2=0x7f8c280feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1431136529 shutdown_connections 2024-05-20T21:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1431136529 wait complete. 2024-05-20T21:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 Processor -- start 2024-05-20T21:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 -- start start 2024-05-20T21:01:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28101c00 0x7f8c280745c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c281025c0 0x7f8c28100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c28102fb0 0x7f8c28101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806c800 con 0x7f8c28101c00 2024-05-20T21:01:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806c680 con 0x7f8c281025c0 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.071+0000 7f8c2f67e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806c980 con 0x7f8c28102fb0 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28101c00 0x7f8c280745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c281025c0 0x7f8c28100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28101c00 0x7f8c280745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49640/0 (socket says 172.21.15.26:49640) 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2d41a640 1 -- 172.21.15.26:0/1983120069 learned_addr learned my addr 172.21.15.26:0/1983120069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c28102fb0 0x7f8c28101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1983120069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c28102fb0 msgr2=0x7f8c28101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c28102fb0 0x7f8c28101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28101c00 msgr2=0x7f8c280745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28101c00 0x7f8c280745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1983120069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c281ab1e0 con 0x7f8c281025c0 2024-05-20T21:01:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c281025c0 0x7f8c28100d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c1802a930 tx=0x7f8c18004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c1e7fc640 1 -- 172.21.15.26:0/1983120069 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1803f070 con 0x7f8c281025c0 2024-05-20T21:01:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c1e7fc640 1 -- 172.21.15.26:0/1983120069 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c18033e00 con 0x7f8c281025c0 2024-05-20T21:01:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c1e7fc640 1 -- 172.21.15.26:0/1983120069 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18037070 con 0x7f8c281025c0 2024-05-20T21:01:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.072+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c281ab470 con 0x7f8c281025c0 2024-05-20T21:01:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.073+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c281ab920 con 0x7f8c281025c0 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.075+0000 7f8c1e7fc640 1 -- 172.21.15.26:0/1983120069 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8c18042070 con 0x7f8c281025c0 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.075+0000 7f8c3239b740 1 Processor -- start 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.075+0000 7f8c3239b740 1 -- start start 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e71ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e720b0 0x561392e76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e796b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c3239b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561392de2c30 con 0x561392e716c0 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c3239b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561392dd27d0 con 0x561392e720b0 2024-05-20T21:01:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c3239b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561392dd6810 con 0x561392e77220 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e71ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e720b0 0x561392e76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e71ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49642/0 (socket says 172.21.15.26:49642) 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/887816193 learned_addr learned my addr 172.21.15.26:0/887816193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/887816193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 msgr2=0x561392e796b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e796b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/887816193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e720b0 msgr2=0x561392e76bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e720b0 0x561392e76bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/887816193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561392ae7210 con 0x561392e716c0 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e796b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e720b0 0x561392e76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c1e7fc640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8c140b2920 0x7f8c140b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8c140b2920 0x7f8c140b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.076+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e71ae0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009ee0 tx=0x7f8c20010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9106cdce67cfe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c1e7fc640 1 -- 172.21.15.26:0/1983120069 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8c181022c0 con 0x7f8c281025c0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c137fe640 1 -- 172.21.15.26:0/887816193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20016070 con 0x561392e716c0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c137fe640 1 -- 172.21.15.26:0/887816193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c20004d20 con 0x561392e716c0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c137fe640 1 -- 172.21.15.26:0/887816193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20012650 con 0x561392e716c0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c3239b740 1 -- 172.21.15.26:0/887816193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 msgr2=0x561392e71ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8c140b2920 0x7f8c140b4de0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x561392e724b0 tx=0x7f8c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.077+0000 7f8c3239b740 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e71ae0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009ee0 tx=0x7f8c20010740 comp rx=0 tx=0).stop 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 -- 172.21.15.26:0/887816193 shutdown_connections 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e796b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e720b0 0x561392e76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 --2- 172.21.15.26:0/887816193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e71ae0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 -- 172.21.15.26:0/887816193 >> 172.21.15.26:0/887816193 conn(0x561392e6cd30 msgr2=0x561392e6d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 -- 172.21.15.26:0/887816193 shutdown_connections 2024-05-20T21:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 -- 172.21.15.26:0/887816193 wait complete. 2024-05-20T21:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.078+0000 7f8c3239b740 1 -- start start 2024-05-20T21:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 Processor -- start 2024-05-20T21:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e7bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e77220 0x561392e82de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561392ab8790 con 0x561392e77220 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e7b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40376/0 (socket says 172.21.15.26:40376) 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c2d41a640 1 -- 172.21.15.26:0/3747420362 learned_addr learned my addr 172.21.15.26:0/3747420362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561392d8e890 con 0x561392e716c0 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561392e7a220 con 0x561392e720b0 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e77220 0x561392e82de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.079+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e7bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c3239b740 10 client.0 _ll_register_callbacks cb 0x561392ae6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/3747420362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 msgr2=0x561392e7bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e7bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 msgr2=0x561392e7b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e7b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/3747420362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561392bb3620 con 0x561392e77220 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e77220 0x561392e82de0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x561392e72910 tx=0x7f8c20004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66de8cbd9e92bd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c10ff9640 1 -- 172.21.15.26:0/3747420362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20028060 con 0x561392e77220 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c10ff9640 1 -- 172.21.15.26:0/3747420362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8c200050e0 con 0x561392e77220 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c20004d20 con 0x561392e77220 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c10ff9640 1 -- 172.21.15.26:0/3747420362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c20016070 con 0x561392e77220 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c3239b740 10 client.16926 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.080+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561392e85d80 con 0x561392e77220 2024-05-20T21:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c10ff9640 1 -- 172.21.15.26:0/3747420362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c20004d20 con 0x561392e77220 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c10ff9640 1 -- 172.21.15.26:0/3747420362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8c2002c030 con 0x561392e77220 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c3239b740 10 client.16926 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c3239b740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c3239b740 20 client.16926 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c3239b740 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392e89570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c3239b740 4 client.16926 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561392de4b80 con 0x561392e87180 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392e89570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392e89570 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49efd25a97f87019 server_cookie=b6f677788263fb32 in_seq=0 out_seq=0 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.081+0000 7f8c10ff9640 10 client.16926 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c10ff9640 1 -- 172.21.15.26:0/3747420362 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561392de4b80 con 0x561392e87180 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c10ff9640 10 client.16926 handle_command_reply: tid=0 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 client.16926 shutdown 2024-05-20T21:01:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e77220 msgr2=0x561392e82de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e77220 0x561392e82de0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x561392e72910 tx=0x7f8c20004c70 comp rx=0 tx=0).stop 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 shutdown_connections 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392e89570 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e77220 0x561392e82de0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c10ff9640 0 client.16926 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e7bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 --2- 172.21.15.26:0/3747420362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e7b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.082+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 >> 172.21.15.26:0/3747420362 conn(0x561392e6cd30 msgr2=0x561392e6bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.083+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 shutdown_connections 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.083+0000 7f8c3239b740 1 -- 172.21.15.26:0/3747420362 wait complete. 2024-05-20T21:01:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.083+0000 7f8c3239b740 20 client.16926 trim_cache size 0 max 16384 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 Processor -- start 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 -- start start 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e6b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 0x561392e82da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e6bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b71b0 con 0x561392e720b0 2024-05-20T21:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b4a80 con 0x561392e716c0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c3239b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b28d0 con 0x561392e77220 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 0x561392e82da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 0x561392e82da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49650/0 (socket says 172.21.15.26:49650) 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1589139617 learned_addr learned my addr 172.21.15.26:0/1589139617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e6b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e6bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1589139617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 msgr2=0x561392e6bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e6bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1589139617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 msgr2=0x561392e6b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.101+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e6b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.102+0000 7f8c2cc19640 1 -- 172.21.15.26:0/1589139617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561392e6cb60 con 0x561392e720b0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.102+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 0x561392e82da0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8c180c5900 tx=0x7f8c180d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55952b6a92576aab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.102+0000 7f8c12ffd640 1 -- 172.21.15.26:0/1589139617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18049040 con 0x561392e720b0 2024-05-20T21:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.102+0000 7f8c12ffd640 1 -- 172.21.15.26:0/1589139617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c18030e60 con 0x561392e720b0 2024-05-20T21:01:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.102+0000 7f8c12ffd640 1 -- 172.21.15.26:0/1589139617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180c9f80 con 0x561392e720b0 2024-05-20T21:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 -- 172.21.15.26:0/1589139617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 msgr2=0x561392e82da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 0x561392e82da0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8c180c5900 tx=0x7f8c180d2040 comp rx=0 tx=0).stop 2024-05-20T21:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 -- 172.21.15.26:0/1589139617 shutdown_connections 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e77220 0x561392e6bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e720b0 0x561392e82da0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1589139617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e716c0 0x561392e6b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 -- 172.21.15.26:0/1589139617 >> 172.21.15.26:0/1589139617 conn(0x561392e6cd30 msgr2=0x561392e7b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 -- 172.21.15.26:0/1589139617 shutdown_connections 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 -- 172.21.15.26:0/1589139617 wait complete. 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 -- start start 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.103+0000 7f8c3239b740 1 Processor -- start 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e77220 0x561392e81880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b2f40 con 0x561392e716c0 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b43b0 con 0x561392e77220 2024-05-20T21:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b3610 con 0x561392e720b0 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c3239b740 10 client.0 _ll_register_callbacks cb 0x7f8c24009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2d41a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e80330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49660/0 (socket says 172.21.15.26:49660) 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2d41a640 1 -- 172.21.15.26:0/1632352745 learned_addr learned my addr 172.21.15.26:0/1632352745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2cc19640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e77220 0x561392e81880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/1632352745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 msgr2=0x561392e80870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e80870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/1632352745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 msgr2=0x561392e80330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e80330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 -- 172.21.15.26:0/1632352745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561392e81e70 con 0x561392e77220 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c2dc1b640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e77220 0x561392e81880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000a010 tx=0x7f8c20009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d6cefdb203bbc52 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.104+0000 7f8c10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c10ff9640 1 -- 172.21.15.26:0/1632352745 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20028030 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c10ff9640 1 -- 172.21.15.26:0/1632352745 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8c20009a20 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c18030e60 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c10ff9640 1 -- 172.21.15.26:0/1632352745 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c20004ce0 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 10 client.26438 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561392ef1c10 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c10ff9640 1 -- 172.21.15.26:0/1632352745 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c18030e60 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c10ff9640 1 -- 172.21.15.26:0/1632352745 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8c2003f050 con 0x561392e77220 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 10 client.26438 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 20 client.26438 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392ef43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 4 client.26438 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.105+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561392dde360 con 0x561392e87180 2024-05-20T21:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.106+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392ef43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.106+0000 7f8c10ff9640 10 client.26438 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.106+0000 7f8c2d41a640 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392ef43b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81fc68bc3c90cb38 server_cookie=5371d3945e2d829e in_seq=0 out_seq=0 2024-05-20T21:01:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c10ff9640 1 -- 172.21.15.26:0/1632352745 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561392dde360 con 0x561392e87180 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c10ff9640 10 client.26438 handle_command_reply: tid=0 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 client.26438 shutdown 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e77220 msgr2=0x561392e81880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e77220 0x561392e81880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000a010 tx=0x7f8c20009dc0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 shutdown_connections 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561392e87180 0x561392ef43b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561392e77220 0x561392e81880 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561392e720b0 0x561392e80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c10ff9640 0 client.26438 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 --2- 172.21.15.26:0/1632352745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561392e716c0 0x561392e80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 >> 172.21.15.26:0/1632352745 conn(0x561392e6cd30 msgr2=0x561392e6b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 shutdown_connections 2024-05-20T21:01:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 1 -- 172.21.15.26:0/1632352745 wait complete. 2024-05-20T21:01:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.140+0000 7f8c3239b740 20 client.26438 trim_cache size 0 max 16384 2024-05-20T21:01:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:01:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:01:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8c140b2920 msgr2=0x7f8c140b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8c140b2920 0x7f8c140b4de0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x561392e724b0 tx=0x7f8c24009260 comp rx=0 tx=0).stop 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c281025c0 msgr2=0x7f8c28100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c281025c0 0x7f8c28100d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c1802a930 tx=0x7f8c18004270 comp rx=0 tx=0).stop 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 shutdown_connections 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8c140b2920 0x7f8c140b4de0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c28102fb0 0x7f8c28101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c281025c0 0x7f8c28100d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 --2- 172.21.15.26:0/1983120069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c28101c00 0x7f8c280745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 >> 172.21.15.26:0/1983120069 conn(0x7f8c280fc6d0 msgr2=0x7f8c28105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 shutdown_connections 2024-05-20T21:01:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:27.141+0000 7f8c2f67e640 1 -- 172.21.15.26:0/1983120069 wait complete. 2024-05-20T21:01:27.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:26 smithi055 ceph-mon[35213]: pgmap v548: 97 pgs: 97 active+clean; 25 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 118 KiB/s wr, 15 op/s 2024-05-20T21:01:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:26 smithi144 ceph-mon[35660]: pgmap v548: 97 pgs: 97 active+clean; 25 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 118 KiB/s wr, 15 op/s 2024-05-20T21:01:27.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 99782 1.98 MB/sec execute 93 sec latency 14.674 ms 2024-05-20T21:01:28.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 100306 1.97 MB/sec execute 94 sec latency 20.174 ms 2024-05-20T21:01:28.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:28 smithi026 ceph-mon[31990]: pgmap v549: 97 pgs: 97 active+clean; 25 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 139 KiB/s wr, 19 op/s 2024-05-20T21:01:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1935812063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc406bbe0 msgr2=0x7f2cc41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1935812063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc406bbe0 0x7f2cc41103d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b0a0 tx=0x7f2cbc00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1935812063 shutdown_connections 2024-05-20T21:01:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1935812063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc406bbe0 0x7f2cc41103d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1935812063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc41095b0 0x7f2cc4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1935812063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc406b220 0x7f2cc406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1935812063 >> 172.21.15.26:0/1935812063 conn(0x7f2cc40793f0 msgr2=0x7f2cc4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1935812063 shutdown_connections 2024-05-20T21:01:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.116+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1935812063 wait complete. 2024-05-20T21:01:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.117+0000 7f2cc91ea700 1 Processor -- start 2024-05-20T21:01:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.117+0000 7f2cc91ea700 1 -- start start 2024-05-20T21:01:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.117+0000 7f2cc91ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc406b220 0x7f2cc41a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.117+0000 7f2cc91ea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc406bbe0 0x7f2cc41a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc91ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 0x7f2cc419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc91ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41135a0 con 0x7f2cc41095b0 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc91ea700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4113420 con 0x7f2cc406bbe0 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc91ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4113720 con 0x7f2cc406b220 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 0x7f2cc419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 0x7f2cc419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49680/0 (socket says 172.21.15.26:49680) 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 -- 172.21.15.26:0/1625771090 learned_addr learned my addr 172.21.15.26:0/1625771090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc2d9d700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc406b220 0x7f2cc41a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc259c700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc406bbe0 0x7f2cc41a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 -- 172.21.15.26:0/1625771090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc406b220 msgr2=0x7f2cc41a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc406b220 0x7f2cc41a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 -- 172.21.15.26:0/1625771090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc406bbe0 msgr2=0x7f2cc41a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc406bbe0 0x7f2cc41a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.118+0000 7f2cc359e700 1 -- 172.21.15.26:0/1625771090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc419c400 con 0x7f2cc41095b0 2024-05-20T21:01:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.119+0000 7f2cc359e700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 0x7f2cc419bb20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b780 tx=0x7f2cbc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.119+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc021070 con 0x7f2cc41095b0 2024-05-20T21:01:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.119+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc419c690 con 0x7f2cc41095b0 2024-05-20T21:01:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.119+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cbc007b80 con 0x7f2cc41095b0 2024-05-20T21:01:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.119+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0052a0 con 0x7f2cc41095b0 2024-05-20T21:01:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.119+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc41a89d0 con 0x7f2cc41095b0 2024-05-20T21:01:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.120+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc4104c70 con 0x7f2cc41095b0 2024-05-20T21:01:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.122+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2cbc024070 con 0x7f2cc41095b0 2024-05-20T21:01:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.122+0000 7f2cabfff700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2cac0b2880 0x7f2cac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.123+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2cbc00f020 con 0x7f2cc41095b0 2024-05-20T21:01:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.123+0000 7f2cc2d9d700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2cac0b2880 0x7f2cac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.123+0000 7f2cc2d9d700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2cac0b2880 0x7f2cac0b4d40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009be0 tx=0x7f2cb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.128+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cbc02b020 con 0x7f2cc41095b0 2024-05-20T21:01:29.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:28 smithi055 ceph-mon[35213]: pgmap v549: 97 pgs: 97 active+clean; 25 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 139 KiB/s wr, 19 op/s 2024-05-20T21:01:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:28 smithi144 ceph-mon[35660]: pgmap v549: 97 pgs: 97 active+clean; 25 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 139 KiB/s wr, 19 op/s 2024-05-20T21:01:29.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.446+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cc41a8ba0 con 0x7f2cc41095b0 2024-05-20T21:01:29.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.447+0000 7f2cabfff700 1 -- 172.21.15.26:0/1625771090 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f2cbc0ab4b0 con 0x7f2cc41095b0 2024-05-20T21:01:29.447 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:01:29.447 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2cac0b2880 msgr2=0x7f2cac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2cac0b2880 0x7f2cac0b4d40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009be0 tx=0x7f2cb4009260 comp rx=0 tx=0).stop 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 msgr2=0x7f2cc419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 0x7f2cc419bb20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b780 tx=0x7f2cbc004d00 comp rx=0 tx=0).stop 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 shutdown_connections 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2cac0b2880 0x7f2cac0b4d40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc41095b0 0x7f2cc419bb20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc406b220 0x7f2cc41a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 --2- 172.21.15.26:0/1625771090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc406bbe0 0x7f2cc41a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 >> 172.21.15.26:0/1625771090 conn(0x7f2cc40793f0 msgr2=0x7f2cc410e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.449+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 shutdown_connections 2024-05-20T21:01:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.450+0000 7f2cc91ea700 1 -- 172.21.15.26:0/1625771090 wait complete. 2024-05-20T21:01:29.451 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:01:29.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 100973 1.99 MB/sec execute 95 sec latency 47.433 ms 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 -- 172.21.15.26:0/3655521392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441041a0 msgr2=0x7fdc441081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 --2- 172.21.15.26:0/3655521392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441041a0 0x7fdc441081c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fdc380099c0 tx=0x7fdc38009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 -- 172.21.15.26:0/3655521392 shutdown_connections 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 --2- 172.21.15.26:0/3655521392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc441088f0 0x7fdc44110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 --2- 172.21.15.26:0/3655521392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc441037f0 0x7fdc44103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 --2- 172.21.15.26:0/3655521392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441041a0 0x7fdc441081c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 -- 172.21.15.26:0/3655521392 >> 172.21.15.26:0/3655521392 conn(0x7fdc4407a6a0 msgr2=0x7fdc4407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 -- 172.21.15.26:0/3655521392 shutdown_connections 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 -- 172.21.15.26:0/3655521392 wait complete. 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.627+0000 7fdc4c635700 1 Processor -- start 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 -- start start 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 0x7fdc4419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc441041a0 0x7fdc4419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc441088f0 0x7fdc44199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441135d0 con 0x7fdc441037f0 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44113450 con 0x7fdc441088f0 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4c635700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44113750 con 0x7fdc441041a0 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 0x7fdc4419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 0x7fdc4419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49698/0 (socket says 172.21.15.26:49698) 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 -- 172.21.15.26:0/4040142198 learned_addr learned my addr 172.21.15.26:0/4040142198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4abd2700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc441088f0 0x7fdc44199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc49bd0700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc441041a0 0x7fdc4419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 -- 172.21.15.26:0/4040142198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc441041a0 msgr2=0x7fdc4419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc441041a0 0x7fdc4419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 -- 172.21.15.26:0/4040142198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc441088f0 msgr2=0x7fdc44199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc441088f0 0x7fdc44199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.628+0000 7fdc4a3d1700 1 -- 172.21.15.26:0/4040142198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4419a090 con 0x7fdc441037f0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.629+0000 7fdc4a3d1700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 0x7fdc4419f770 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000d980 tx=0x7fdc4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.629+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40017070 con 0x7fdc441037f0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.629+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc40005020 con 0x7fdc441037f0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.629+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4419a380 con 0x7fdc441037f0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.629+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40009ab0 con 0x7fdc441037f0 2024-05-20T21:01:29.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.629+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc441a6640 con 0x7fdc441037f0 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.630+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4406a230 con 0x7fdc441037f0 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.632+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdc400051d0 con 0x7fdc441037f0 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.633+0000 7fdc377fe700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdc300b2880 0x7fdc300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.633+0000 7fdc49bd0700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdc300b2880 0x7fdc300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.633+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fdc400e2580 con 0x7fdc441037f0 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.633+0000 7fdc49bd0700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdc300b2880 0x7fdc300b4d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc44108430 tx=0x7fdc38005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.637+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc40020020 con 0x7fdc441037f0 2024-05-20T21:01:29.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.940+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc4402e560 con 0x7fdc441037f0 2024-05-20T21:01:29.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.941+0000 7fdc377fe700 1 -- 172.21.15.26:0/4040142198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7fdc400aa240 con 0x7fdc441037f0 2024-05-20T21:01:29.941 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:01:29.941 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:01:29.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdc300b2880 msgr2=0x7fdc300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdc300b2880 0x7fdc300b4d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc44108430 tx=0x7fdc38005fb0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 msgr2=0x7fdc4419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:29.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 0x7fdc4419f770 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000d980 tx=0x7fdc4000dc90 comp rx=0 tx=0).stop 2024-05-20T21:01:29.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 shutdown_connections 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdc300b2880 0x7fdc300b4d40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc441037f0 0x7fdc4419f770 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc441088f0 0x7fdc44199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 --2- 172.21.15.26:0/4040142198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc441041a0 0x7fdc4419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 >> 172.21.15.26:0/4040142198 conn(0x7fdc4407a6a0 msgr2=0x7fdc44101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 shutdown_connections 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:29.943+0000 7fdc4c635700 1 -- 172.21.15.26:0/4040142198 wait complete. 2024-05-20T21:01:29.944 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:01:30.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:29 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1625771090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:30.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:29 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1625771090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:30.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:29 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1625771090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:30.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 101697 1.99 MB/sec execute 96 sec latency 18.390 ms 2024-05-20T21:01:30.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:01:31.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:30 smithi026 ceph-mon[31990]: pgmap v550: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 345 KiB/s wr, 39 op/s 2024-05-20T21:01:31.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:30 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/4040142198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:31.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:30 smithi055 ceph-mon[35213]: pgmap v550: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 345 KiB/s wr, 39 op/s 2024-05-20T21:01:31.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:30 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/4040142198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:31.227 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0' 2024-05-20T21:01:31.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0 2024-05-20T21:01:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:30 smithi144 ceph-mon[35660]: pgmap v550: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 345 KiB/s wr, 39 op/s 2024-05-20T21:01:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:30 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/4040142198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:31.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 102524 2.03 MB/sec execute 97 sec latency 20.034 ms 2024-05-20T21:01:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:32.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 103043 2.06 MB/sec execute 98 sec latency 21.950 ms 2024-05-20T21:01:33.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:32 smithi055 ceph-mon[35213]: pgmap v551: 97 pgs: 97 active+clean; 27 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 730 KiB/s wr, 51 op/s 2024-05-20T21:01:33.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:32 smithi026 ceph-mon[31990]: pgmap v551: 97 pgs: 97 active+clean; 27 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 730 KiB/s wr, 51 op/s 2024-05-20T21:01:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:32 smithi144 ceph-mon[35660]: pgmap v551: 97 pgs: 97 active+clean; 27 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 730 KiB/s wr, 51 op/s 2024-05-20T21:01:33.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 103506 2.05 MB/sec execute 99 sec latency 15.995 ms 2024-05-20T21:01:34.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 -- 172.21.15.26:0/94836051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 msgr2=0x7f041c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 --2- 172.21.15.26:0/94836051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c1081c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f040c0099c0 tx=0x7f040c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 -- 172.21.15.26:0/94836051 shutdown_connections 2024-05-20T21:01:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 --2- 172.21.15.26:0/94836051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 0x7f041c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 --2- 172.21.15.26:0/94836051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c06bed0 0x7f041c06c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 --2- 172.21.15.26:0/94836051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c1081c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 -- 172.21.15.26:0/94836051 >> 172.21.15.26:0/94836051 conn(0x7f041c079440 msgr2=0x7f041c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 -- 172.21.15.26:0/94836051 shutdown_connections 2024-05-20T21:01:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.212+0000 7f0424265700 1 -- 172.21.15.26:0/94836051 wait complete. 2024-05-20T21:01:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 Processor -- start 2024-05-20T21:01:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 -- start start 2024-05-20T21:01:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c06bed0 0x7f041c1024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 0x7f041c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113580 con 0x7f041c06c880 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113400 con 0x7f041c1088f0 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.213+0000 7f0424265700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113700 con 0x7f041c06bed0 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0422802700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 0x7f041c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54870/0 (socket says 172.21.15.26:54870) 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 -- 172.21.15.26:0/629395453 learned_addr learned my addr 172.21.15.26:0/629395453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0422001700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c06bed0 0x7f041c1024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 -- 172.21.15.26:0/629395453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c06bed0 msgr2=0x7f041c1024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c06bed0 0x7f041c1024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 -- 172.21.15.26:0/629395453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 msgr2=0x7f041c102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 0x7f041c102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 -- 172.21.15.26:0/629395453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a7fe0 con 0x7f041c06c880 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0422802700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 0x7f041c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0421800700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c102a30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f040c006010 tx=0x7f040c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.214+0000 7f0412ffd700 1 -- 172.21.15.26:0/629395453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c026070 con 0x7f041c06c880 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.215+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a8270 con 0x7f041c06c880 2024-05-20T21:01:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.215+0000 7f0412ffd700 1 -- 172.21.15.26:0/629395453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c004630 con 0x7f041c06c880 2024-05-20T21:01:34.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.215+0000 7f0412ffd700 1 -- 172.21.15.26:0/629395453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00faa0 con 0x7f041c06c880 2024-05-20T21:01:34.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.215+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a8720 con 0x7f041c06c880 2024-05-20T21:01:34.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.218+0000 7f0412ffd700 1 -- 172.21.15.26:0/629395453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f040c029070 con 0x7f041c06c880 2024-05-20T21:01:34.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 Processor -- start 2024-05-20T21:01:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 -- start start 2024-05-20T21:01:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f0412ffd700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b2900 0x7f04080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 0x55f9c171b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1715d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c1722080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f9c1673d70 con 0x55f9c17158a0 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f9c1663910 con 0x55f9c1716260 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f042eafbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9c1667950 con 0x55f9c171b320 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.219+0000 7f0412ffd700 1 -- 172.21.15.26:0/629395453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f040c0e28e0 con 0x7f041c06c880 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 0x55f9c171b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0421800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1715d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 0x55f9c171b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:35264/0 (socket says 172.21.15.26:35264) 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 -- 172.21.15.26:0/2671354944 learned_addr learned my addr 172.21.15.26:0/2671354944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422802700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b2900 0x7f04080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 -- 172.21.15.26:0/2671354944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 msgr2=0x55f9c1722080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c1722080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 -- 172.21.15.26:0/2671354944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 msgr2=0x55f9c1715d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1715d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 -- 172.21.15.26:0/2671354944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c160cfb0 con 0x55f9c171b320 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0421800700 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1715d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422802700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b2900 0x7f04080b4dc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0414003ed0 tx=0x7f041400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.220+0000 7f0422001700 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 0x55f9c171b700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0418009db0 tx=0x7f0418011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9590865b308a0de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.221+0000 7f0407fff700 1 -- 172.21.15.26:0/2671354944 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041801e070 con 0x55f9c171b320 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.221+0000 7f0407fff700 1 -- 172.21.15.26:0/2671354944 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0418009e40 con 0x55f9c171b320 2024-05-20T21:01:34.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.221+0000 7f0407fff700 1 -- 172.21.15.26:0/2671354944 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041800e970 con 0x55f9c171b320 2024-05-20T21:01:34.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.221+0000 7f042eafbb80 1 -- 172.21.15.26:0/2671354944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 msgr2=0x55f9c171b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.221+0000 7f042eafbb80 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 0x55f9c171b700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0418009db0 tx=0x7f0418011720 comp rx=0 tx=0).stop 2024-05-20T21:01:34.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 -- 172.21.15.26:0/2671354944 shutdown_connections 2024-05-20T21:01:34.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c1722080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c171b320 0x55f9c171b700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 --2- 172.21.15.26:0/2671354944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1715d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 -- 172.21.15.26:0/2671354944 >> 172.21.15.26:0/2671354944 conn(0x55f9c1641240 msgr2=0x55f9c17103e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 -- 172.21.15.26:0/2671354944 shutdown_connections 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 -- 172.21.15.26:0/2671354944 wait complete. 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 -- start start 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.222+0000 7f042eafbb80 1 Processor -- start 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c17158a0 0x55f9c1721d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c171f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f9c1569fa0 con 0x55f9c171b320 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f9c15cbda0 con 0x55f9c1716260 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9c17226e0 con 0x55f9c17158a0 2024-05-20T21:01:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f042eafbb80 10 client.0 _ll_register_callbacks cb 0x55f9c15fe950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c171f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c171f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46076/0 (socket says 172.21.15.26:46076) 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 -- 172.21.15.26:0/3489194957 learned_addr learned my addr 172.21.15.26:0/3489194957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0422001700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c17158a0 0x55f9c1721d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0422802700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 -- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c17158a0 msgr2=0x55f9c1721d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c17158a0 0x55f9c1721d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 -- 172.21.15.26:0/3489194957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 msgr2=0x55f9c171f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0421800700 1 -- 172.21.15.26:0/3489194957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c15b38d0 con 0x55f9c1716260 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.223+0000 7f0422001700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c17158a0 0x55f9c1721d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f0421800700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c171f030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f040c009fa0 tx=0x7f040c003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85c121f07e677f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f04057fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:01:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f04057fa700 1 -- 172.21.15.26:0/3489194957 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0a9a40 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9c1675cc0 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f04057fa700 1 -- 172.21.15.26:0/3489194957 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f040c0aa3a0 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f04057fa700 1 -- 172.21.15.26:0/3489194957 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f040c0aa630 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f042eafbb80 10 client.26459 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9c161d760 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f04057fa700 1 -- 172.21.15.26:0/3489194957 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f9c1675cc0 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.224+0000 7f04057fa700 1 -- 172.21.15.26:0/3489194957 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f040c0c0050 con 0x55f9c1716260 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f042eafbb80 10 client.26459 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f042eafbb80 10 client.26459 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f042eafbb80 20 client.26459 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c172f210 0x55f9c1731690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f042eafbb80 4 client.26459 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c166f4a0 con 0x55f9c172f210 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f0422001700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c172f210 0x55f9c1731690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f04057fa700 10 client.26459 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.225+0000 7f0422001700 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c172f210 0x55f9c1731690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=164f8006a17eadea server_cookie=26010bfe05711bb4 in_seq=0 out_seq=0 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f04057fa700 1 -- 172.21.15.26:0/3489194957 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f9c166f4a0 con 0x55f9c172f210 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f04057fa700 10 client.26459 handle_command_reply: tid=0 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 client.26459 shutdown 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 msgr2=0x55f9c171f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c171f030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f040c009fa0 tx=0x7f040c003e00 comp rx=0 tx=0).stop 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 shutdown_connections 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c172f210 0x55f9c1731690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c17158a0 0x55f9c1721d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3489194957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c1716260 0x55f9c171f030 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 >> 172.21.15.26:0/3489194957 conn(0x55f9c1641240 msgr2=0x55f9c170faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.226+0000 7f04057fa700 0 client.26459 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.227+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 shutdown_connections 2024-05-20T21:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.227+0000 7f042eafbb80 1 -- 172.21.15.26:0/3489194957 wait complete. 2024-05-20T21:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.227+0000 7f042eafbb80 20 client.26459 trim_cache size 0 max 16384 2024-05-20T21:01:34.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.244+0000 7f042eafbb80 1 Processor -- start 2024-05-20T21:01:34.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.244+0000 7f042eafbb80 1 -- start start 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 0x55f9c172b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c171f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f042eafbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b26a0 con 0x55f9c171b320 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f042eafbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aff30 con 0x55f9c17158a0 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f042eafbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1add80 con 0x55f9c1716260 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422001700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 0x55f9c172b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422001700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 0x55f9c172b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46088/0 (socket says 172.21.15.26:46088) 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422001700 1 -- 172.21.15.26:0/1374723956 learned_addr learned my addr 172.21.15.26:0/1374723956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0421800700 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c171f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 -- 172.21.15.26:0/1374723956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 msgr2=0x55f9c171f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c171f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 -- 172.21.15.26:0/1374723956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 msgr2=0x55f9c172b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 0x55f9c172b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 -- 172.21.15.26:0/1374723956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c1722530 con 0x55f9c171b320 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422001700 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 0x55f9c172b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.245+0000 7f0422802700 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171a870 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0414009fd0 tx=0x7f0414014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c9e81ffc28e81c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.246+0000 7f04077fe700 1 -- 172.21.15.26:0/1374723956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414027030 con 0x55f9c171b320 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.246+0000 7f04077fe700 1 -- 172.21.15.26:0/1374723956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f041401b070 con 0x55f9c171b320 2024-05-20T21:01:34.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.246+0000 7f04077fe700 1 -- 172.21.15.26:0/1374723956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414008510 con 0x55f9c171b320 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.246+0000 7f042eafbb80 1 -- 172.21.15.26:0/1374723956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 msgr2=0x55f9c171a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.246+0000 7f042eafbb80 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171a870 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0414009fd0 tx=0x7f0414014000 comp rx=0 tx=0).stop 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 -- 172.21.15.26:0/1374723956 shutdown_connections 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c171b320 0x55f9c171a870 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c17158a0 0x55f9c172b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 --2- 172.21.15.26:0/1374723956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c171f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 -- 172.21.15.26:0/1374723956 >> 172.21.15.26:0/1374723956 conn(0x55f9c1641240 msgr2=0x55f9c171a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 -- 172.21.15.26:0/1374723956 shutdown_connections 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 -- 172.21.15.26:0/1374723956 wait complete. 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 -- start start 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.247+0000 7f042eafbb80 1 Processor -- start 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1727930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c1727e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c171b320 0x55f9c1735f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ae3f0 con 0x55f9c17158a0 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1af860 con 0x55f9c171b320 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aeac0 con 0x55f9c1716260 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1727930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f042eafbb80 10 client.0 _ll_register_callbacks cb 0x55f9c15fe950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1727930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54920/0 (socket says 172.21.15.26:54920) 2024-05-20T21:01:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 -- 172.21.15.26:0/3083345674 learned_addr learned my addr 172.21.15.26:0/3083345674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422802700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c171b320 0x55f9c1735f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0421800700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c1727e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 -- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 msgr2=0x55f9c1727e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c1727e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 -- 172.21.15.26:0/3083345674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c171b320 msgr2=0x55f9c1735f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c171b320 0x55f9c1735f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 -- 172.21.15.26:0/3083345674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c1737af0 con 0x55f9c17158a0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0421800700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c1727e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422802700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c171b320 0x55f9c1735f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f0422001700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1727930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f041801c040 tx=0x7f0418005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53608b6b01416a0d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.248+0000 7f04057fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f04057fa700 1 -- 172.21.15.26:0/3083345674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041801e040 con 0x55f9c17158a0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f04057fa700 1 -- 172.21.15.26:0/3083345674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f0418005240 con 0x55f9c17158a0 2024-05-20T21:01:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f041401b070 con 0x55f9c17158a0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f04057fa700 1 -- 172.21.15.26:0/3083345674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f041803c050 con 0x55f9c17158a0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 10 client.16962 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9c1799460 con 0x55f9c17158a0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f04057fa700 1 -- 172.21.15.26:0/3083345674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f041401b070 con 0x55f9c17158a0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f04057fa700 1 -- 172.21.15.26:0/3083345674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f0418003680 con 0x55f9c17158a0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 10 client.16962 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 10 client.16962 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 20 client.16962 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c1799af0 0x55f9c179bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 4 client.16962 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.249+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9c14e6250 con 0x55f9c1799af0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f0421800700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c1799af0 0x55f9c179bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f04057fa700 10 client.16962 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f0421800700 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c1799af0 0x55f9c179bf10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a235a440d6912fb8 server_cookie=a829b1baf202cabd in_seq=0 out_seq=0 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f04057fa700 1 -- 172.21.15.26:0/3083345674 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9c14e6250 con 0x55f9c1799af0 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f04057fa700 10 client.16962 handle_command_reply: tid=0 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f042eafbb80 1 client.16962 shutdown 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 msgr2=0x55f9c1727930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.250+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1727930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f041801c040 tx=0x7f0418005fa0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 shutdown_connections 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55f9c1799af0 0x55f9c179bf10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c17158a0 0x55f9c1727930 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f04057fa700 0 client.16962 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9c171b320 0x55f9c1735f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 --2- 172.21.15.26:0/3083345674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9c1716260 0x55f9c1727e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 >> 172.21.15.26:0/3083345674 conn(0x55f9c1641240 msgr2=0x55f9c1718d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 shutdown_connections 2024-05-20T21:01:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 1 -- 172.21.15.26:0/3083345674 wait complete. 2024-05-20T21:01:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.251+0000 7f042eafbb80 20 client.16962 trim_cache size 0 max 16384 2024-05-20T21:01:34.256 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b2900 msgr2=0x7f04080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b2900 0x7f04080b4dc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0414003ed0 tx=0x7f041400b040 comp rx=0 tx=0).stop 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 msgr2=0x7f041c102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c102a30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f040c006010 tx=0x7f040c004270 comp rx=0 tx=0).stop 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 shutdown_connections 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f041c1088f0 0x7f041c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c06bed0 0x7f041c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04080b2900 0x7f04080b4dc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 --2- 172.21.15.26:0/629395453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041c06c880 0x7f041c102a30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 >> 172.21.15.26:0/629395453 conn(0x7f041c079440 msgr2=0x7f041c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 shutdown_connections 2024-05-20T21:01:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.252+0000 7f0424265700 1 -- 172.21.15.26:0/629395453 wait complete. 2024-05-20T21:01:34.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104094 2.05 MB/sec execute 100 sec latency 22.214 ms 2024-05-20T21:01:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 -- 172.21.15.26:0/2096386108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b006c4d0 msgr2=0x7f61b01103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 --2- 172.21.15.26:0/2096386108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b006c4d0 0x7f61b01103f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b0a0 tx=0x7f61a800b3b0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 -- 172.21.15.26:0/2096386108 shutdown_connections 2024-05-20T21:01:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 --2- 172.21.15.26:0/2096386108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b006c4d0 0x7f61b01103f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 --2- 172.21.15.26:0/2096386108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b01095d0 0x7f61b01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 --2- 172.21.15.26:0/2096386108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006bb10 0x7f61b006bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 -- 172.21.15.26:0/2096386108 >> 172.21.15.26:0/2096386108 conn(0x7f61b0079480 msgr2=0x7f61b0079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 -- 172.21.15.26:0/2096386108 shutdown_connections 2024-05-20T21:01:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.768+0000 7f61b63bc700 1 -- 172.21.15.26:0/2096386108 wait complete. 2024-05-20T21:01:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.769+0000 7f61b63bc700 1 Processor -- start 2024-05-20T21:01:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.769+0000 7f61b63bc700 1 -- start start 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.769+0000 7f61b63bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b006bb10 0x7f61b01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.769+0000 7f61b63bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 0x7f61b01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.769+0000 7f61b63bc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b01095d0 0x7f61b019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.769+0000 7f61b63bc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01135b0 con 0x7f61b01095d0 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b63bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0113430 con 0x7f61b006c4d0 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b63bc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0113730 con 0x7f61b006bb10 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 0x7f61b01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b01095d0 0x7f61b019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 0x7f61b01a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46116/0 (socket says 172.21.15.26:46116) 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61af7fe700 1 -- 172.21.15.26:0/529251407 learned_addr learned my addr 172.21.15.26:0/529251407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61affff700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b006bb10 0x7f61b01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 -- 172.21.15.26:0/529251407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b006bb10 msgr2=0x7f61b01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b006bb10 0x7f61b01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 -- 172.21.15.26:0/529251407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 msgr2=0x7f61b01a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 0x7f61b01a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 -- 172.21.15.26:0/529251407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b019c1a0 con 0x7f61b01095d0 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61af7fe700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 0x7f61b01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.770+0000 7f61b4959700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b01095d0 0x7f61b019ba40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b0a0 tx=0x7f61a8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.771+0000 7f61ad7fa700 1 -- 172.21.15.26:0/529251407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8021070 con 0x7f61b01095d0 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.771+0000 7f61ad7fa700 1 -- 172.21.15.26:0/529251407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61a8007b80 con 0x7f61b01095d0 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.771+0000 7f61ad7fa700 1 -- 172.21.15.26:0/529251407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8005040 con 0x7f61b01095d0 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.771+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b019c430 con 0x7f61b01095d0 2024-05-20T21:01:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.771+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01a8750 con 0x7f61b01095d0 2024-05-20T21:01:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.774+0000 7f61ad7fa700 1 -- 172.21.15.26:0/529251407 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f61a8024070 con 0x7f61b01095d0 2024-05-20T21:01:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.774+0000 7f61ad7fa700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f61980b2930 0x7f61980b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61affff700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f61980b2930 0x7f61980b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61ad7fa700 1 -- 172.21.15.26:0/529251407 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f61a800f020 con 0x7f61b01095d0 2024-05-20T21:01:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61c0c52b80 1 Processor -- start 2024-05-20T21:01:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61affff700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f61980b2930 0x7f61980b4df0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f61a0005950 tx=0x7f61a000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61c0c52b80 1 -- start start 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4980720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.775+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b497adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4987170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d7b48d9e20 con 0x55d7b497b330 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d7b48c99c0 con 0x55d7b4980340 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7b48cda00 con 0x55d7b497a970 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4987170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4987170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54948/0 (socket says 172.21.15.26:54948) 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 -- 172.21.15.26:0/2037521824 learned_addr learned my addr 172.21.15.26:0/2037521824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61affff700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4980720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61af7fe700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b497adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 -- 172.21.15.26:0/2037521824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 msgr2=0x55d7b497adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b497adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 -- 172.21.15.26:0/2037521824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 msgr2=0x55d7b4980720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4980720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 -- 172.21.15.26:0/2037521824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b4819a60 con 0x55d7b497b330 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61af7fe700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b497adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61affff700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4980720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61b4959700 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4987170 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f61a80aac30 tx=0x7f61a800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28070fe52f5a95b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61967fc700 1 -- 172.21.15.26:0/2037521824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80ab3a0 con 0x55d7b497b330 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.776+0000 7f61967fc700 1 -- 172.21.15.26:0/2037521824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61a80aaca0 con 0x55d7b497b330 2024-05-20T21:01:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61967fc700 1 -- 172.21.15.26:0/2037521824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80aaf00 con 0x55d7b497b330 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2037521824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 msgr2=0x55d7b4987170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4987170 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f61a80aac30 tx=0x7f61a800b3b0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2037521824 shutdown_connections 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4987170 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4980720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2037521824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b497adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2037521824 >> 172.21.15.26:0/2037521824 conn(0x55d7b48a8210 msgr2=0x55d7b4975280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2037521824 shutdown_connections 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.777+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2037521824 wait complete. 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 -- start start 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 Processor -- start 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b4986f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4984230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4984770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d7b47d0130 con 0x55d7b497b330 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d7b486b950 con 0x55d7b4980340 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7b4980ee0 con 0x55d7b497a970 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.778+0000 7f61c0c52b80 10 client.0 _ll_register_callbacks cb 0x55d7b4864ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4984230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4984230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54952/0 (socket says 172.21.15.26:54952) 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61af7fe700 1 -- 172.21.15.26:0/1555760151 learned_addr learned my addr 172.21.15.26:0/1555760151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b4986f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61b4959700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4984770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 -- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 msgr2=0x55d7b4984770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4984770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 -- 172.21.15.26:0/1555760151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 msgr2=0x55d7b4984230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4984230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 -- 172.21.15.26:0/1555760151 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b4868820 con 0x55d7b497a970 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61af7fe700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4984230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f61affff700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b4986f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61a001da30 tx=0x7f61a00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b4fa8eb0880df3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.779+0000 7f618f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f618f7fe700 1 -- 172.21.15.26:0/1555760151 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a002a060 con 0x55d7b497a970 2024-05-20T21:01:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a80aaca0 con 0x55d7b497a970 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f618f7fe700 1 -- 172.21.15.26:0/1555760151 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f61a0010840 con 0x55d7b497a970 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f618f7fe700 1 -- 172.21.15.26:0/1555760151 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61a0016070 con 0x55d7b497a970 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 10 client.26359 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7b4991cd0 con 0x55d7b497a970 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f618f7fe700 1 -- 172.21.15.26:0/1555760151 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61a80aaca0 con 0x55d7b497a970 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f618f7fe700 1 -- 172.21.15.26:0/1555760151 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f61a002e030 con 0x55d7b497a970 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 10 client.26359 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 10 client.26359 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 20 client.26359 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4994310 0x55d7b4996740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 4 client.26359 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.780+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7b4946b70 con 0x55d7b4994310 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.781+0000 7f61af7fe700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4994310 0x55d7b4996740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.781+0000 7f618f7fe700 10 client.26359 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.781+0000 7f61af7fe700 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4994310 0x55d7b4996740 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c1b690746e8684 server_cookie=e54814a7a96b6c3d in_seq=0 out_seq=0 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f618f7fe700 1 -- 172.21.15.26:0/1555760151 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d7b4946b70 con 0x55d7b4994310 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f618f7fe700 10 client.26359 handle_command_reply: tid=0 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 client.26359 shutdown 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 msgr2=0x55d7b4986f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b4986f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61a001da30 tx=0x7f61a00047c0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 shutdown_connections 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4994310 0x55d7b4996740 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b4980340 0x55d7b4984770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497a970 0x55d7b4986f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/1555760151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497b330 0x55d7b4984230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f618f7fe700 0 client.26359 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:34.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 >> 172.21.15.26:0/1555760151 conn(0x55d7b48a8210 msgr2=0x55d7b496fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 shutdown_connections 2024-05-20T21:01:34.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 1 -- 172.21.15.26:0/1555760151 wait complete. 2024-05-20T21:01:34.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.782+0000 7f61c0c52b80 20 client.26359 trim_cache size 0 max 16384 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 Processor -- start 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 -- start start 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497a970 0x55d7b4990590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497b330 0x55d7b4986a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 0x55d7b497e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01b25c0 con 0x55d7b4980340 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01afe50 con 0x55d7b497a970 2024-05-20T21:01:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01adca0 con 0x55d7b497b330 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61b4959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 0x55d7b497e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497b330 0x55d7b4986a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61b4959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 0x55d7b497e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54958/0 (socket says 172.21.15.26:54958) 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61b4959700 1 -- 172.21.15.26:0/2429870785 learned_addr learned my addr 172.21.15.26:0/2429870785 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.806+0000 7f61affff700 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497a970 0x55d7b4990590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61b4959700 1 -- 172.21.15.26:0/2429870785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497b330 msgr2=0x55d7b4986a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61b4959700 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497b330 0x55d7b4986a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61b4959700 1 -- 172.21.15.26:0/2429870785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497a970 msgr2=0x55d7b4990590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61b4959700 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497a970 0x55d7b4990590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61b4959700 1 -- 172.21.15.26:0/2429870785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b49827a0 con 0x55d7b4980340 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61affff700 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497a970 0x55d7b4990590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f61b4959700 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 0x55d7b497e530 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f61a80a6d20 tx=0x7f61a80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fdf6d448b87b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f6195ffb700 1 -- 172.21.15.26:0/2429870785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80b3a70 con 0x55d7b4980340 2024-05-20T21:01:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f6195ffb700 1 -- 172.21.15.26:0/2429870785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61a80ab3a0 con 0x55d7b4980340 2024-05-20T21:01:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.807+0000 7f6195ffb700 1 -- 172.21.15.26:0/2429870785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80ab160 con 0x55d7b4980340 2024-05-20T21:01:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2429870785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 msgr2=0x55d7b497e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 0x55d7b497e530 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f61a80a6d20 tx=0x7f61a80099e0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2429870785 shutdown_connections 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b4980340 0x55d7b497e530 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497a970 0x55d7b4990590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/2429870785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b497b330 0x55d7b4986a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2429870785 >> 172.21.15.26:0/2429870785 conn(0x55d7b48a8210 msgr2=0x55d7b497de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2429870785 shutdown_connections 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 -- 172.21.15.26:0/2429870785 wait complete. 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.808+0000 7f61c0c52b80 1 -- start start 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 Processor -- start 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497a970 0x55d7b4989750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497b330 0x55d7b4989c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b4980340 0x55d7b498a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01ae310 con 0x55d7b497a970 2024-05-20T21:01:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01af780 con 0x55d7b497b330 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01ae9e0 con 0x55d7b4980340 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61c0c52b80 10 client.0 _ll_register_callbacks cb 0x55d7b4864ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497b330 0x55d7b4989c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61affff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497a970 0x55d7b4989750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497b330 0x55d7b4989c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46166/0 (socket says 172.21.15.26:46166) 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61af7fe700 1 -- 172.21.15.26:0/124862173 learned_addr learned my addr 172.21.15.26:0/124862173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61b4959700 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b4980340 0x55d7b498a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61affff700 1 -- 172.21.15.26:0/124862173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b4980340 msgr2=0x55d7b498a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61affff700 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b4980340 0x55d7b498a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61affff700 1 -- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497b330 msgr2=0x55d7b4989c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.809+0000 7f61affff700 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497b330 0x55d7b4989c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f61affff700 1 -- 172.21.15.26:0/124862173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b49ff060 con 0x55d7b497a970 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f61affff700 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497a970 0x55d7b4989750 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f61a001bc50 tx=0x7f61a001b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5aa8defa9f25372 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f618f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f618f7fe700 1 -- 172.21.15.26:0/124862173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0010840 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f618f7fe700 1 -- 172.21.15.26:0/124862173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f61a00109e0 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a80ab3a0 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f618f7fe700 1 -- 172.21.15.26:0/124862173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61a001dd40 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f61c0c52b80 10 client.16980 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7b4a00300 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.810+0000 7f618f7fe700 1 -- 172.21.15.26:0/124862173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61a80ab3a0 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f618f7fe700 1 -- 172.21.15.26:0/124862173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f61a0040020 con 0x55d7b497a970 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61c0c52b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61c0c52b80 10 client.16980 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61c0c52b80 20 client.16980 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4a00ab0 0x55d7b4a02ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:34.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61c0c52b80 4 client.16980 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}] 2024-05-20T21:01:34.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}) -- 0x55d7b48dbd70 con 0x55d7b4a00ab0 2024-05-20T21:01:34.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61af7fe700 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4a00ab0 0x55d7b4a02ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:34.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f618f7fe700 10 client.16980 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:34.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.811+0000 7f61af7fe700 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4a00ab0 0x55d7b4a02ed0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13948ccaacff385 server_cookie=9ce7d4ce7ceb9bae in_seq=0 out_seq=0 2024-05-20T21:01:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.961+0000 7f618f7fe700 1 -- 172.21.15.26:0/124862173 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d7b48dbd70 con 0x55d7b4a00ab0 2024-05-20T21:01:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.961+0000 7f618f7fe700 10 client.16980 handle_command_reply: tid=0 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.961+0000 7f61c0c52b80 1 client.16980 shutdown 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497a970 msgr2=0x55d7b4989750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497a970 0x55d7b4989750 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f61a001bc50 tx=0x7f61a001b670 comp rx=0 tx=0).stop 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 shutdown_connections 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d7b4a00ab0 0x55d7b4a02ed0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b497a970 0x55d7b4989750 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7b4980340 0x55d7b498a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f618f7fe700 0 client.16980 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 --2- 172.21.15.26:0/124862173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7b497b330 0x55d7b4989c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 >> 172.21.15.26:0/124862173 conn(0x55d7b48a8210 msgr2=0x55d7b497e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 shutdown_connections 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 1 -- 172.21.15.26:0/124862173 wait complete. 2024-05-20T21:01:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.962+0000 7f61c0c52b80 20 client.16980 trim_cache size 0 max 16384 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 28, 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 22, 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "84e775fb": { 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 22, 2024-05-20T21:01:34.964 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.3, 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:34.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:34.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:34.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:34.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:34.966 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f61980b2930 msgr2=0x7f61980b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f61980b2930 0x7f61980b4df0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f61a0005950 tx=0x7f61a000a300 comp rx=0 tx=0).stop 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b01095d0 msgr2=0x7f61b019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b01095d0 0x7f61b019ba40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b0a0 tx=0x7f61a8004d00 comp rx=0 tx=0).stop 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 shutdown_connections 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f61980b2930 0x7f61980b4df0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61b01095d0 0x7f61b019ba40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b006bb10 0x7f61b01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 --2- 172.21.15.26:0/529251407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b006c4d0 0x7f61b01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 >> 172.21.15.26:0/529251407 conn(0x7f61b0079480 msgr2=0x7f61b010e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 shutdown_connections 2024-05-20T21:01:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:34.963+0000 7f61b63bc700 1 -- 172.21.15.26:0/529251407 wait complete. 2024-05-20T21:01:35.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:34 smithi055 ceph-mon[35213]: pgmap v552: 97 pgs: 97 active+clean; 29 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-05-20T21:01:35.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:34 smithi026 ceph-mon[31990]: pgmap v552: 97 pgs: 97 active+clean; 29 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-05-20T21:01:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:34 smithi144 ceph-mon[35660]: pgmap v552: 97 pgs: 97 active+clean; 29 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-05-20T21:01:35.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 2.06 MB/sec execute 101 sec latency 590.720 ms 2024-05-20T21:01:35.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:01:36.176 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '84e775fb', quiesce took 11.1 seconds. Will release after: 10.200000000000001 2024-05-20T21:01:36.255 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:36.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 2.04 MB/sec execute 102 sec latency 1590.766 ms 2024-05-20T21:01:37.116 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:36 smithi026 ceph-mon[31990]: pgmap v553: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 89 op/s 2024-05-20T21:01:37.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:36 smithi055 ceph-mon[35213]: pgmap v553: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 89 op/s 2024-05-20T21:01:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:36 smithi144 ceph-mon[35660]: pgmap v553: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 89 op/s 2024-05-20T21:01:37.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 2.02 MB/sec execute 103 sec latency 2590.803 ms 2024-05-20T21:01:38.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 -- 172.21.15.26:0/3520636804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102440 msgr2=0x7f3af41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/3520636804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102440 0x7f3af41028c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400b0a0 tx=0x7f3ae400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 -- 172.21.15.26:0/3520636804 shutdown_connections 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/3520636804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3af4102e00 0x7f3af410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/3520636804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af4108520 0x7f3af4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/3520636804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102440 0x7f3af41028c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.149+0000 7f3afb1eb700 1 -- 172.21.15.26:0/3520636804 >> 172.21.15.26:0/3520636804 conn(0x7f3af40793f0 msgr2=0x7f3af4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.150+0000 7f3afb1eb700 1 -- 172.21.15.26:0/3520636804 shutdown_connections 2024-05-20T21:01:38.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.150+0000 7f3afb1eb700 1 -- 172.21.15.26:0/3520636804 wait complete. 2024-05-20T21:01:38.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.150+0000 7f3afb1eb700 1 Processor -- start 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.150+0000 7f3afb1eb700 1 -- start start 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.150+0000 7f3afb1eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af4102440 0x7f3af419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3afb1eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 0x7f3af419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3afb1eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3af4108520 0x7f3af41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3afb1eb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4112500 con 0x7f3af4102e00 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3afb1eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4112380 con 0x7f3af4108520 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3afb1eb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4112680 con 0x7f3af4102440 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 0x7f3af419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 0x7f3af419c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54986/0 (socket says 172.21.15.26:54986) 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 -- 172.21.15.26:0/2232604613 learned_addr learned my addr 172.21.15.26:0/2232604613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:38.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af9788700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3af4108520 0x7f3af41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af4102440 0x7f3af419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 -- 172.21.15.26:0/2232604613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af4102440 msgr2=0x7f3af419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af4102440 0x7f3af419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 -- 172.21.15.26:0/2232604613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3af4108520 msgr2=0x7f3af41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3af4108520 0x7f3af41a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 -- 172.21.15.26:0/2232604613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af41a0de0 con 0x7f3af4102e00 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 0x7f3af419c360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40062a0 tx=0x7f3ae4004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.151+0000 7f3af1ffb700 1 -- 172.21.15.26:0/2232604613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4021070 con 0x7f3af4102e00 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.152+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41a1070 con 0x7f3af4102e00 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.152+0000 7f3af1ffb700 1 -- 172.21.15.26:0/2232604613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ae4007c40 con 0x7f3af4102e00 2024-05-20T21:01:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.152+0000 7f3af1ffb700 1 -- 172.21.15.26:0/2232604613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4005040 con 0x7f3af4102e00 2024-05-20T21:01:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.152+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a8950 con 0x7f3af4102e00 2024-05-20T21:01:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.154+0000 7f3af1ffb700 1 -- 172.21.15.26:0/2232604613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3ae4024070 con 0x7f3af4102e00 2024-05-20T21:01:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.155+0000 7f3af1ffb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ae00b2880 0x7f3ae00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.155+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ae00b2880 0x7f3ae00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.155+0000 7f3af1ffb700 1 -- 172.21.15.26:0/2232604613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f3ae400f020 con 0x7f3af4102e00 2024-05-20T21:01:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 Processor -- start 2024-05-20T21:01:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ae00b2880 0x7f3ae00b4d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009a10 tx=0x7f3adc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 -- start start 2024-05-20T21:01:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6376700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d63708a0 0x5631d6370d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d6371260 0x5631d637d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5631d62ced70 con 0x5631d6376320 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5631d62be910 con 0x5631d6371260 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5631d62c2950 con 0x5631d63708a0 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6376700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6376700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54998/0 (socket says 172.21.15.26:54998) 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 -- 172.21.15.26:0/1368933125 learned_addr learned my addr 172.21.15.26:0/1368933125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af9788700 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d6371260 0x5631d637d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af3fff700 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d63708a0 0x5631d6370d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 -- 172.21.15.26:0/1368933125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d63708a0 msgr2=0x5631d6370d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d63708a0 0x5631d6370d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 -- 172.21.15.26:0/1368933125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d6371260 msgr2=0x5631d637d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d6371260 0x5631d637d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.156+0000 7f3af8f87700 1 -- 172.21.15.26:0/1368933125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d6267fb0 con 0x5631d6376320 2024-05-20T21:01:38.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.157+0000 7f3af3fff700 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d63708a0 0x5631d6370d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:38.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.157+0000 7f3af8f87700 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6376700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00ea00 tx=0x7f3adc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c742e030d222ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:38.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.157+0000 7f3aeaffd700 1 -- 172.21.15.26:0/1368933125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc01b040 con 0x5631d6376320 2024-05-20T21:01:38.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.157+0000 7f3aeaffd700 1 -- 172.21.15.26:0/1368933125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3adc024070 con 0x5631d6376320 2024-05-20T21:01:38.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.157+0000 7f3aeaffd700 1 -- 172.21.15.26:0/1368933125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0136b0 con 0x5631d6376320 2024-05-20T21:01:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1368933125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 msgr2=0x5631d6376700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6376700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00ea00 tx=0x7f3adc011000 comp rx=0 tx=0).stop 2024-05-20T21:01:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1368933125 shutdown_connections 2024-05-20T21:01:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6376700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d6371260 0x5631d637d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1368933125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d63708a0 0x5631d6370d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1368933125 >> 172.21.15.26:0/1368933125 conn(0x5631d629c240 msgr2=0x5631d636b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1368933125 shutdown_connections 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1368933125 wait complete. 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 -- start start 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.158+0000 7f3b05a81b80 1 Processor -- start 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d637cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d637a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d637a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5631d61c4fa0 con 0x5631d6376320 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5631d6226da0 con 0x5631d63708a0 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5631d637d6e0 con 0x5631d6371260 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 10 client.0 _ll_register_callbacks cb 0x5631d6259950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d637a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d637a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55014/0 (socket says 172.21.15.26:55014) 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 -- 172.21.15.26:0/2894417366 learned_addr learned my addr 172.21.15.26:0/2894417366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d637cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d637a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 -- 172.21.15.26:0/2894417366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 msgr2=0x5631d637a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d637a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 -- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 msgr2=0x5631d637cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d637cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 -- 172.21.15.26:0/2894417366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d620e8d0 con 0x5631d6376320 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d637a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d637cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3af9788700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d637a570 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3aec003a70 tx=0x7f3aec003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55773eaab9c90ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3ad3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2894417366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00bdb0 con 0x5631d6376320 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2894417366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f3aec004830 con 0x5631d6376320 2024-05-20T21:01:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.159+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3adc024070 con 0x5631d6376320 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2894417366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3aec004a30 con 0x5631d6376320 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 10 client.16992 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631d6278760 con 0x5631d6376320 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2894417366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3adc024070 con 0x5631d6376320 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2894417366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f3aec024050 con 0x5631d6376320 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 10 client.16992 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 20 client.16992 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d638a1f0 0x5631d638c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 4 client.16992 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.160+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d62d0cc0 con 0x5631d638a1f0 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.161+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d638a1f0 0x5631d638c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.161+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d638a1f0 0x5631d638c5e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cbe3613a4aa4a1b server_cookie=794261940520b568 in_seq=0 out_seq=0 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.161+0000 7f3ad3fff700 10 client.16992 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:01:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2894417366 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5631d62d0cc0 con 0x5631d638a1f0 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3ad3fff700 10 client.16992 handle_command_reply: tid=0 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 client.16992 shutdown 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 msgr2=0x5631d637a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d637a570 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3aec003a70 tx=0x7f3aec003e30 comp rx=0 tx=0).stop 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 shutdown_connections 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d638a1f0 0x5631d638c5e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d637a570 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d637cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3ad3fff700 0 client.16992 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2894417366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d637a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 >> 172.21.15.26:0/2894417366 conn(0x5631d629c240 msgr2=0x5631d636aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 shutdown_connections 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2894417366 wait complete. 2024-05-20T21:01:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.162+0000 7f3b05a81b80 20 client.16992 trim_cache size 0 max 16384 2024-05-20T21:01:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 Processor -- start 2024-05-20T21:01:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 -- start start 2024-05-20T21:01:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6375820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 0x5631d636b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6376320 0x5631d6391ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41b26c0 con 0x5631d6371260 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41aff50 con 0x5631d63708a0 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41adda0 con 0x5631d6376320 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 0x5631d636b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 0x5631d636b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55016/0 (socket says 172.21.15.26:55016) 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 -- 172.21.15.26:0/1819511773 learned_addr learned my addr 172.21.15.26:0/1819511773 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af8f87700 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6375820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 -- 172.21.15.26:0/1819511773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6376320 msgr2=0x5631d6391ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6376320 0x5631d6391ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 -- 172.21.15.26:0/1819511773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 msgr2=0x5631d6375820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6375820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 -- 172.21.15.26:0/1819511773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d63785a0 con 0x5631d6371260 2024-05-20T21:01:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.178+0000 7f3af3fff700 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 0x5631d636b270 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40aabf0 tx=0x7f3ae40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0b33a7e2aea22d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3aea7fc700 1 -- 172.21.15.26:0/1819511773 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40aa770 con 0x5631d6371260 2024-05-20T21:01:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3aea7fc700 1 -- 172.21.15.26:0/1819511773 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ae40ab360 con 0x5631d6371260 2024-05-20T21:01:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3aea7fc700 1 -- 172.21.15.26:0/1819511773 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40a72b0 con 0x5631d6371260 2024-05-20T21:01:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1819511773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 msgr2=0x5631d636b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 0x5631d636b270 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40aabf0 tx=0x7f3ae40099e0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1819511773 shutdown_connections 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6376320 0x5631d6391ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6375820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/1819511773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6371260 0x5631d636b270 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1819511773 >> 172.21.15.26:0/1819511773 conn(0x5631d629c240 msgr2=0x5631d6375110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1819511773 shutdown_connections 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.179+0000 7f3b05a81b80 1 -- 172.21.15.26:0/1819511773 wait complete. 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 -- start start 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 Processor -- start 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6383030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d63916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6391c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41ae410 con 0x5631d6376320 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41af880 con 0x5631d63708a0 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41aeae0 con 0x5631d6371260 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3b05a81b80 10 client.0 _ll_register_callbacks cb 0x5631d6259950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3af9788700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6391c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3af9788700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6391c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55026/0 (socket says 172.21.15.26:55026) 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3af9788700 1 -- 172.21.15.26:0/2498946447 learned_addr learned my addr 172.21.15.26:0/2498946447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6383030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.180+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d63916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af9788700 1 -- 172.21.15.26:0/2498946447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 msgr2=0x5631d63916e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af9788700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d63916e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af9788700 1 -- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 msgr2=0x5631d6383030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af9788700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6383030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af9788700 1 -- 172.21.15.26:0/2498946447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d6392430 con 0x5631d6376320 2024-05-20T21:01:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6383030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af3fff700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d63916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3af9788700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6391c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3aec003790 tx=0x7f3aec00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e068a5abeb0a5e65 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3ad3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2498946447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004860 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae40ab360 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2498946447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f3aec011a50 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2498946447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3aec011c50 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3b05a81b80 10 client.16998 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631d63f4d70 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.181+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2498946447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ae40ab360 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2498946447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f3aec036020 con 0x5631d6376320 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3b05a81b80 10 client.16998 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3b05a81b80 10 client.16998 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3b05a81b80 20 client.16998 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d63f5420 0x5631d63f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3b05a81b80 4 client.16998 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631d62ca4a0 con 0x5631d63f5420 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d63f5420 0x5631d63f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3af8f87700 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d63f5420 0x5631d63f7810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a70e43ad96c41133 server_cookie=6fe101037a90d084 in_seq=0 out_seq=0 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.182+0000 7f3ad3fff700 10 client.16998 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3ad3fff700 1 -- 172.21.15.26:0/2498946447 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631d62ca4a0 con 0x5631d63f5420 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3ad3fff700 10 client.16998 handle_command_reply: tid=0 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 client.16998 shutdown 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 msgr2=0x5631d6391c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6391c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3aec003790 tx=0x7f3aec00b950 comp rx=0 tx=0).stop 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 shutdown_connections 2024-05-20T21:01:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5631d63f5420 0x5631d63f7810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631d6376320 0x5631d6391c20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5631d63708a0 0x5631d6383030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3ad3fff700 0 client.16998 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 --2- 172.21.15.26:0/2498946447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631d6371260 0x5631d63916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 >> 172.21.15.26:0/2498946447 conn(0x5631d629c240 msgr2=0x5631d6373ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 shutdown_connections 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 1 -- 172.21.15.26:0/2498946447 wait complete. 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.183+0000 7f3b05a81b80 20 client.16998 trim_cache size 0 max 16384 2024-05-20T21:01:38.187 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ae00b2880 msgr2=0x7f3ae00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ae00b2880 0x7f3ae00b4d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009a10 tx=0x7f3adc009190 comp rx=0 tx=0).stop 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 msgr2=0x7f3af419c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 0x7f3af419c360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40062a0 tx=0x7f3ae4004dd0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 shutdown_connections 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3af4108520 0x7f3af41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af4102440 0x7f3af419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ae00b2880 0x7f3ae00b4d40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 --2- 172.21.15.26:0/2232604613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af4102e00 0x7f3af419c360 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 >> 172.21.15.26:0/2232604613 conn(0x7f3af40793f0 msgr2=0x7f3af406b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 shutdown_connections 2024-05-20T21:01:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:38.184+0000 7f3afb1eb700 1 -- 172.21.15.26:0/2232604613 wait complete. 2024-05-20T21:01:38.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 2.00 MB/sec execute 104 sec latency 3590.833 ms 2024-05-20T21:01:38.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:38 smithi026 ceph-mon[31990]: pgmap v554: 97 pgs: 97 active+clean; 27 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-05-20T21:01:39.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:01:39.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:38 smithi055 ceph-mon[35213]: pgmap v554: 97 pgs: 97 active+clean; 27 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-05-20T21:01:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:38 smithi144 ceph-mon[35660]: pgmap v554: 97 pgs: 97 active+clean; 27 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-05-20T21:01:39.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.98 MB/sec execute 105 sec latency 4590.860 ms 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 -- 172.21.15.26:0/3314434316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8102ba0 msgr2=0x7f5fe8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 --2- 172.21.15.26:0/3314434316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8102ba0 0x7f5fe8103020 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc02a410 comp rx=0 tx=0).stop 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 -- 172.21.15.26:0/3314434316 shutdown_connections 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 --2- 172.21.15.26:0/3314434316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8103560 0x7f5fe8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 --2- 172.21.15.26:0/3314434316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8102ba0 0x7f5fe8103020 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 --2- 172.21.15.26:0/3314434316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe81019b0 0x7f5fe8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 -- 172.21.15.26:0/3314434316 >> 172.21.15.26:0/3314434316 conn(0x7f5fe80fcf40 msgr2=0x7f5fe80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 -- 172.21.15.26:0/3314434316 shutdown_connections 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.236+0000 7f5fee971640 1 -- 172.21.15.26:0/3314434316 wait complete. 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 Processor -- start 2024-05-20T21:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 -- start start 2024-05-20T21:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 0x7f5fe8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8102ba0 0x7f5fe81990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe8103560 0x7f5fe81a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe810d370 con 0x7f5fe81019b0 2024-05-20T21:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe810d1f0 con 0x7f5fe8102ba0 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fee971640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe810d4f0 con 0x7f5fe8103560 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fe7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 0x7f5fe8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fe7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 0x7f5fe8198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58794/0 (socket says 172.21.15.26:58794) 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fe7fff640 1 -- 172.21.15.26:0/1559515202 learned_addr learned my addr 172.21.15.26:0/1559515202 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe8103560 0x7f5fe81a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.237+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8102ba0 0x7f5fe81990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe7fff640 1 -- 172.21.15.26:0/1559515202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe8103560 msgr2=0x7f5fe81a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe8103560 0x7f5fe81a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe7fff640 1 -- 172.21.15.26:0/1559515202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8102ba0 msgr2=0x7f5fe81990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8102ba0 0x7f5fe81990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe7fff640 1 -- 172.21.15.26:0/1559515202 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe81a0790 con 0x7f5fe81019b0 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8102ba0 0x7f5fe81990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 0x7f5fe8198b70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800e9c0 tx=0x7f5fd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe57fa640 1 -- 172.21.15.26:0/1559515202 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800cd80 con 0x7f5fe81019b0 2024-05-20T21:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe81a0a80 con 0x7f5fe81019b0 2024-05-20T21:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe57fa640 1 -- 172.21.15.26:0/1559515202 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd80044b0 con 0x7f5fe81019b0 2024-05-20T21:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fe57fa640 1 -- 172.21.15.26:0/1559515202 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8010430 con 0x7f5fe81019b0 2024-05-20T21:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.238+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81a1020 con 0x7f5fe81019b0 2024-05-20T21:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.240+0000 7f5fe57fa640 1 -- 172.21.15.26:0/1559515202 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5fd8003f00 con 0x7f5fe81019b0 2024-05-20T21:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 Processor -- start 2024-05-20T21:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 -- start start 2024-05-20T21:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 0x5578b3c64ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c650b0 0x5578b3c69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c6c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5578b3bd5c30 con 0x5578b3c646c0 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5578b3bc57d0 con 0x5578b3c650b0 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5ff1763740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5578b3bc9810 con 0x5578b3c6a220 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5fecf0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 0x5578b3c64ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5fecf0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 0x5578b3c64ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58806/0 (socket says 172.21.15.26:58806) 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5fecf0e640 1 -- 172.21.15.26:0/3093633807 learned_addr learned my addr 172.21.15.26:0/3093633807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c6c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.241+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c650b0 0x5578b3c69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 -- 172.21.15.26:0/3093633807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 msgr2=0x5578b3c6c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c6c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 -- 172.21.15.26:0/3093633807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c650b0 msgr2=0x5578b3c69bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c650b0 0x5578b3c69bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fe57fa640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5fd00b28a0 0x7f5fd00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 -- 172.21.15.26:0/3093633807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b38da210 con 0x5578b3c646c0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fe57fa640 1 -- 172.21.15.26:0/1559515202 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f5fd80e2710 con 0x7f5fe81019b0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5fd00b28a0 0x7f5fd00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 0x5578b3c64ae0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00039f0 tx=0x7f5fe0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363e11b7f1030220 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fce7fc640 1 -- 172.21.15.26:0/3093633807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0022070 con 0x5578b3c646c0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fce7fc640 1 -- 172.21.15.26:0/3093633807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fe0015070 con 0x5578b3c646c0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.242+0000 7f5fce7fc640 1 -- 172.21.15.26:0/3093633807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00108f0 con 0x5578b3c646c0 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 -- 172.21.15.26:0/3093633807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 msgr2=0x5578b3c64ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 0x5578b3c64ae0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00039f0 tx=0x7f5fe0003eb0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5fd00b28a0 0x7f5fd00b4d60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5fe001f000 tx=0x7f5fe0020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 -- 172.21.15.26:0/3093633807 shutdown_connections 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c6c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c650b0 0x5578b3c69bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3093633807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c646c0 0x5578b3c64ae0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 -- 172.21.15.26:0/3093633807 >> 172.21.15.26:0/3093633807 conn(0x5578b3c5fd30 msgr2=0x5578b3c60130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 -- 172.21.15.26:0/3093633807 shutdown_connections 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 -- 172.21.15.26:0/3093633807 wait complete. 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.243+0000 7f5ff1763740 1 -- start start 2024-05-20T21:01:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 Processor -- start 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c646c0 0x5578b3c6e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c6ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c75cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5578b38ab790 con 0x5578b3c650b0 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5578b3b81890 con 0x5578b3c646c0 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5578b3c6d1d0 con 0x5578b3c6a220 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5ff1763740 10 client.0 _ll_register_callbacks cb 0x5578b38d9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c6ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c6ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58812/0 (socket says 172.21.15.26:58812) 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 -- 172.21.15.26:0/434747293 learned_addr learned my addr 172.21.15.26:0/434747293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c646c0 0x5578b3c6e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c75cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 -- 172.21.15.26:0/434747293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 msgr2=0x5578b3c75cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c75cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 -- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c646c0 msgr2=0x5578b3c6e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c646c0 0x5578b3c6e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 -- 172.21.15.26:0/434747293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b39a6620 con 0x5578b3c650b0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c646c0 0x5578b3c6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c6ec90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f0738890a0e40b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.244+0000 7f5fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5fc77fe640 1 -- 172.21.15.26:0/434747293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc035040 con 0x5578b3c650b0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5fc77fe640 1 -- 172.21.15.26:0/434747293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f5fdc004450 con 0x5578b3c650b0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe0015070 con 0x5578b3c650b0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5fc77fe640 1 -- 172.21.15.26:0/434747293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc0035f0 con 0x5578b3c650b0 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 10 client.17010 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578b3c78d20 con 0x5578b3c650b0 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5fc77fe640 1 -- 172.21.15.26:0/434747293 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fe0015070 con 0x5578b3c650b0 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5fc77fe640 1 -- 172.21.15.26:0/434747293 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f5fdc03a040 con 0x5578b3c650b0 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 10 client.17010 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 20 client.17010 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3c7a460 0x5578b3c7c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 4 client.17010 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.245+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b3bd7b80 con 0x5578b3c7a460 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.246+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3c7a460 0x5578b3c7c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.246+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3c7a460 0x5578b3c7c8d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278a9c9bdc35834e server_cookie=5c2d84f170316775 in_seq=0 out_seq=0 2024-05-20T21:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.246+0000 7f5fc77fe640 10 client.17010 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5fc77fe640 1 -- 172.21.15.26:0/434747293 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5578b3bd7b80 con 0x5578b3c7a460 2024-05-20T21:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5fc77fe640 10 client.17010 handle_command_reply: tid=0 2024-05-20T21:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 client.17010 shutdown 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 msgr2=0x5578b3c6ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c6ec90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc031ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 shutdown_connections 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3c7a460 0x5578b3c7c8d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c6a220 0x5578b3c75cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c6ec90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5fc77fe640 0 client.17010 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 --2- 172.21.15.26:0/434747293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c646c0 0x5578b3c6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 >> 172.21.15.26:0/434747293 conn(0x5578b3c5fd30 msgr2=0x5578b3c5efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 shutdown_connections 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 1 -- 172.21.15.26:0/434747293 wait complete. 2024-05-20T21:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.247+0000 7f5ff1763740 20 client.17010 trim_cache size 0 max 16384 2024-05-20T21:01:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 Processor -- start 2024-05-20T21:01:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 -- start start 2024-05-20T21:01:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3c5ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c62a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3c62f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81ac800 con 0x5578b3c650b0 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81aa0d0 con 0x5578b3c6a220 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5ff1763740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81a7f20 con 0x5578b3c646c0 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c62a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c62a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58824/0 (socket says 172.21.15.26:58824) 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.262+0000 7f5fe77fe640 1 -- 172.21.15.26:0/3532778524 learned_addr learned my addr 172.21.15.26:0/3532778524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fe77fe640 1 -- 172.21.15.26:0/3532778524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 msgr2=0x5578b3c5ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3c5ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fe77fe640 1 -- 172.21.15.26:0/3532778524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 msgr2=0x5578b3c62f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3c62f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3c62f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fe77fe640 1 -- 172.21.15.26:0/3532778524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b3c5fb60 con 0x5578b3c650b0 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3c62f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c62a10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005ab0 tx=0x7f5fdc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44acb7423bb50c1e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fcdffb640 1 -- 172.21.15.26:0/3532778524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc035070 con 0x5578b3c650b0 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fcdffb640 1 -- 172.21.15.26:0/3532778524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fdc053020 con 0x5578b3c650b0 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.263+0000 7f5fcdffb640 1 -- 172.21.15.26:0/3532778524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc000e60 con 0x5578b3c650b0 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- 172.21.15.26:0/3532778524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 msgr2=0x5578b3c62a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c62a10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005ab0 tx=0x7f5fdc02a410 comp rx=0 tx=0).stop 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- 172.21.15.26:0/3532778524 shutdown_connections 2024-05-20T21:01:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3c62f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3c62a10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- 172.21.15.26:0/3532778524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3c5ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- 172.21.15.26:0/3532778524 >> 172.21.15.26:0/3532778524 conn(0x5578b3c5fd30 msgr2=0x5578b3c698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- 172.21.15.26:0/3532778524 shutdown_connections 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- 172.21.15.26:0/3532778524 wait complete. 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- start start 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 Processor -- start 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3ce0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3ce0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3ce7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81a8590 con 0x5578b3c650b0 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81a9a00 con 0x5578b3c6a220 2024-05-20T21:01:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81a8c60 con 0x5578b3c646c0 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.264+0000 7f5ff1763740 10 client.0 _ll_register_callbacks cb 0x7f5fdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3ce0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3ce0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58826/0 (socket says 172.21.15.26:58826) 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 -- 172.21.15.26:0/2823212376 learned_addr learned my addr 172.21.15.26:0/2823212376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fecf0e640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3ce7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 -- 172.21.15.26:0/2823212376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 msgr2=0x5578b3ce0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3ce0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3ce0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 -- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 msgr2=0x5578b3ce7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3ce7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 -- 172.21.15.26:0/2823212376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b3ce8270 con 0x5578b3c650b0 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3ce0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fe77fe640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3ce0c40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005a80 tx=0x7f5fdc0044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a08aacc760aff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fc77fe640 1 -- 172.21.15.26:0/2823212376 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc05c020 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fc77fe640 1 -- 172.21.15.26:0/2823212376 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f5fdc0039a0 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fdc053020 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5fc77fe640 1 -- 172.21.15.26:0/2823212376 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc067050 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5ff1763740 10 client.17016 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.265+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578b3ce94f0 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5fc77fe640 1 -- 172.21.15.26:0/2823212376 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fdc053020 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5fc77fe640 1 -- 172.21.15.26:0/2823212376 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f5fdc003bc0 con 0x5578b3c650b0 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5ff1763740 10 client.17016 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5ff1763740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5ff1763740 20 client.17016 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5ff1763740 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3ce9eb0 0x5578b3cec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5ff1763740 4 client.17016 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:01:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578b3bd1360 con 0x5578b3ce9eb0 2024-05-20T21:01:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3ce9eb0 0x5578b3cec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5fe7fff640 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3ce9eb0 0x5578b3cec2d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677819ac5ddc2218 server_cookie=6adf105909d9fa98 in_seq=0 out_seq=0 2024-05-20T21:01:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.266+0000 7f5fc77fe640 10 client.17016 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.293+0000 7f5fc77fe640 1 -- 172.21.15.26:0/2823212376 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5578b3bd1360 con 0x5578b3ce9eb0 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.293+0000 7f5fc77fe640 10 client.17016 handle_command_reply: tid=0 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.293+0000 7f5ff1763740 1 client.17016 shutdown 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 msgr2=0x5578b3ce0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3ce0c40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005a80 tx=0x7f5fdc0044e0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 shutdown_connections 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5578b3ce9eb0 0x5578b3cec2d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5578b3c6a220 0x5578b3ce7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b3c650b0 0x5578b3ce0c40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5fc77fe640 0 client.17016 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 --2- 172.21.15.26:0/2823212376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b3c646c0 0x5578b3ce0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 >> 172.21.15.26:0/2823212376 conn(0x5578b3c5fd30 msgr2=0x5578b3c69490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 shutdown_connections 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 1 -- 172.21.15.26:0/2823212376 wait complete. 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5ff1763740 20 client.17016 trim_cache size 0 max 16384 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:01:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:01:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5fd00b28a0 msgr2=0x7f5fd00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5fee971640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5fd00b28a0 0x7f5fd00b4d60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5fe001f000 tx=0x7f5fe0020000 comp rx=0 tx=0).stop 2024-05-20T21:01:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 msgr2=0x7f5fe8198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.294+0000 7f5fee971640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 0x7f5fe8198b70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800e9c0 tx=0x7f5fd800ee80 comp rx=0 tx=0).stop 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 shutdown_connections 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5fd00b28a0 0x7f5fd00b4d60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe8103560 0x7f5fe81a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fe8102ba0 0x7f5fe81990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 --2- 172.21.15.26:0/1559515202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe81019b0 0x7f5fe8198b70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 >> 172.21.15.26:0/1559515202 conn(0x7f5fe80fcf40 msgr2=0x7f5fe80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 shutdown_connections 2024-05-20T21:01:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:40.295+0000 7f5fee971640 1 -- 172.21.15.26:0/1559515202 wait complete. 2024-05-20T21:01:40.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.97 MB/sec execute 106 sec latency 5590.888 ms 2024-05-20T21:01:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 -- 172.21.15.26:0/3703756615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81041a0 msgr2=0x7f56c81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 --2- 172.21.15.26:0/3703756615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81041a0 0x7f56c81081c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f56b80099c0 tx=0x7f56b8009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 -- 172.21.15.26:0/3703756615 shutdown_connections 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 --2- 172.21.15.26:0/3703756615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56c81088f0 0x7f56c8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 --2- 172.21.15.26:0/3703756615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 0x7f56c8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 --2- 172.21.15.26:0/3703756615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81041a0 0x7f56c81081c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 -- 172.21.15.26:0/3703756615 >> 172.21.15.26:0/3703756615 conn(0x7f56c807a6a0 msgr2=0x7f56c807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 -- 172.21.15.26:0/3703756615 shutdown_connections 2024-05-20T21:01:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.068+0000 7f56cfb69700 1 -- 172.21.15.26:0/3703756615 wait complete. 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 Processor -- start 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 -- start start 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 0x7f56c819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56c81041a0 0x7f56c819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 0x7f56c8199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8113580 con 0x7f56c81088f0 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8113400 con 0x7f56c81041a0 2024-05-20T21:01:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cfb69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8113700 con 0x7f56c81037f0 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56ce106700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 0x7f56c8199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56ce106700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 0x7f56c8199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58838/0 (socket says 172.21.15.26:58838) 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56ce106700 1 -- 172.21.15.26:0/275269433 learned_addr learned my addr 172.21.15.26:0/275269433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.069+0000 7f56cd104700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56c81041a0 0x7f56c819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56cd905700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 0x7f56c819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56ce106700 1 -- 172.21.15.26:0/275269433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 msgr2=0x7f56c819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56ce106700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 0x7f56c819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56ce106700 1 -- 172.21.15.26:0/275269433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56c81041a0 msgr2=0x7f56c819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56ce106700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56c81041a0 0x7f56c819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56ce106700 1 -- 172.21.15.26:0/275269433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c819a080 con 0x7f56c81088f0 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56cd905700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 0x7f56c819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56ce106700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 0x7f56c8199920 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e940 tx=0x7f56c000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56beffd700 1 -- 172.21.15.26:0/275269433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c000cbd0 con 0x7f56c81088f0 2024-05-20T21:01:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56beffd700 1 -- 172.21.15.26:0/275269433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56c000ee90 con 0x7f56c81088f0 2024-05-20T21:01:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56beffd700 1 -- 172.21.15.26:0/275269433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0005520 con 0x7f56c81088f0 2024-05-20T21:01:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c819a370 con 0x7f56c81088f0 2024-05-20T21:01:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.070+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c81a65e0 con 0x7f56c81088f0 2024-05-20T21:01:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.073+0000 7f56beffd700 1 -- 172.21.15.26:0/275269433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f56c0004750 con 0x7f56c81088f0 2024-05-20T21:01:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.074+0000 7f56beffd700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b40b2880 0x7f56b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.074+0000 7f56cd905700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b40b2880 0x7f56b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:40 smithi026 ceph-mon[31990]: pgmap v555: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.074+0000 7f56beffd700 1 -- 172.21.15.26:0/275269433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f56c00e2530 con 0x7f56c81088f0 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.074+0000 7f56da3ffb80 1 Processor -- start 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 -- start start 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd905700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b40b2880 0x7f56b40b4d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f56c81032a0 tx=0x7f56c400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c9f320 0x564cf3c9f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 0x564cf3c99d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564cf3bf7d70 con 0x564cf3c998a0 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564cf3be7910 con 0x564cf3c9f320 2024-05-20T21:01:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564cf3beb950 con 0x564cf3c9a260 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 0x564cf3c99d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 0x564cf3c99d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58854/0 (socket says 172.21.15.26:58854) 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 -- 172.21.15.26:0/2822316044 learned_addr learned my addr 172.21.15.26:0/2822316044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56ce106700 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd905700 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c9f320 0x564cf3c9f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 -- 172.21.15.26:0/2822316044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 msgr2=0x564cf3ca6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 -- 172.21.15.26:0/2822316044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c9f320 msgr2=0x564cf3c9f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c9f320 0x564cf3c9f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56cd104700 1 -- 172.21.15.26:0/2822316044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf3b90fb0 con 0x564cf3c998a0 2024-05-20T21:01:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.075+0000 7f56ce106700 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.076+0000 7f56cd104700 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 0x564cf3c99d20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f56b80099c0 tx=0x7f56b80036c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918db16694bf1bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.076+0000 7f56b3fff700 1 -- 172.21.15.26:0/2822316044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8028070 con 0x564cf3c998a0 2024-05-20T21:01:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.076+0000 7f56b3fff700 1 -- 172.21.15.26:0/2822316044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b80037c0 con 0x564cf3c998a0 2024-05-20T21:01:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.076+0000 7f56b3fff700 1 -- 172.21.15.26:0/2822316044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b800b990 con 0x564cf3c998a0 2024-05-20T21:01:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2822316044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 msgr2=0x564cf3c99d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 0x564cf3c99d20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f56b80099c0 tx=0x7f56b80036c0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2822316044 shutdown_connections 2024-05-20T21:01:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c9f320 0x564cf3c9f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2822316044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c998a0 0x564cf3c99d20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2822316044 >> 172.21.15.26:0/2822316044 conn(0x564cf3bc5240 msgr2=0x564cf3c943e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2822316044 shutdown_connections 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2822316044 wait complete. 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 -- start start 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.077+0000 7f56da3ffb80 1 Processor -- start 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3ca5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3ca3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564cf3aedfa0 con 0x564cf3c9f320 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564cf3b4fda0 con 0x564cf3c998a0 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564cf3ca6680 con 0x564cf3c9a260 2024-05-20T21:01:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56da3ffb80 10 client.0 _ll_register_callbacks cb 0x564cf3b82950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3ca3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3ca3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58864/0 (socket says 172.21.15.26:58864) 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 -- 172.21.15.26:0/1445570930 learned_addr learned my addr 172.21.15.26:0/1445570930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56cd905700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3ca5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56cd104700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 -- 172.21.15.26:0/1445570930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 msgr2=0x564cf3ca2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 -- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 msgr2=0x564cf3ca5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3ca5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 -- 172.21.15.26:0/1445570930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf3b378d0 con 0x564cf3c9f320 2024-05-20T21:01:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56ce106700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3ca3510 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009fd0 tx=0x7f56c0016630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d5f9fe0eca0521 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.078+0000 7f56b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56b17fa700 1 -- 172.21.15.26:0/1445570930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00b2290 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56b17fa700 1 -- 172.21.15.26:0/1445570930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f56c00056e0 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b80037c0 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56b17fa700 1 -- 172.21.15.26:0/1445570930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56c00a9600 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 10 client.17028 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564cf3ba1760 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56b17fa700 1 -- 172.21.15.26:0/1445570930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b80037c0 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56b17fa700 1 -- 172.21.15.26:0/1445570930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f56c00b24c0 con 0x564cf3c9f320 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 10 client.17028 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 20 client.17028 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3cb31d0 0x564cf3cb5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 4 client.17028 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.079+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cf3bf9cc0 con 0x564cf3cb31d0 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.080+0000 7f56cd905700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3cb31d0 0x564cf3cb5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.080+0000 7f56cd905700 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3cb31d0 0x564cf3cb5600 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6acbb8471ebd12 server_cookie=483b7f0cd3376fcd in_seq=0 out_seq=0 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.080+0000 7f56b17fa700 10 client.17028 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56b17fa700 1 -- 172.21.15.26:0/1445570930 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564cf3bf9cc0 con 0x564cf3cb31d0 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56b17fa700 10 client.17028 handle_command_reply: tid=0 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 client.17028 shutdown 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 msgr2=0x564cf3ca3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3ca3510 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009fd0 tx=0x7f56c0016630 comp rx=0 tx=0).stop 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 shutdown_connections 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3cb31d0 0x564cf3cb5600 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3ca3510 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3ca5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56b17fa700 0 client.17028 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/1445570930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3ca2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 >> 172.21.15.26:0/1445570930 conn(0x564cf3bc5240 msgr2=0x564cf3c93af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 shutdown_connections 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 1 -- 172.21.15.26:0/1445570930 wait complete. 2024-05-20T21:01:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.081+0000 7f56da3ffb80 20 client.17028 trim_cache size 0 max 16384 2024-05-20T21:01:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 Processor -- start 2024-05-20T21:01:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 -- start start 2024-05-20T21:01:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 0x564cf3c94270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9f320 0x564cf3ca4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81ae210 con 0x564cf3c9a260 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81abaa0 con 0x564cf3c998a0 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81a98f0 con 0x564cf3c9f320 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.097+0000 7f56cd104700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 0x564cf3c94270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 0x564cf3c94270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58872/0 (socket says 172.21.15.26:58872) 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 -- 172.21.15.26:0/110827030 learned_addr learned my addr 172.21.15.26:0/110827030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd905700 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56ce106700 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9f320 0x564cf3ca4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 -- 172.21.15.26:0/110827030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9f320 msgr2=0x564cf3ca4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9f320 0x564cf3ca4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 -- 172.21.15.26:0/110827030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 msgr2=0x564cf3cb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cb55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 -- 172.21.15.26:0/110827030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf3ca1540 con 0x564cf3c9a260 2024-05-20T21:01:41.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56ce106700 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9f320 0x564cf3ca4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:41.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56cd104700 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 0x564cf3c94270 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f56b80099c0 tx=0x7f56b8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8021a908433c4349 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:41.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56b37fe700 1 -- 172.21.15.26:0/110827030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031020 con 0x564cf3c9a260 2024-05-20T21:01:41.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56b37fe700 1 -- 172.21.15.26:0/110827030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b800b790 con 0x564cf3c9a260 2024-05-20T21:01:41.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.098+0000 7f56b37fe700 1 -- 172.21.15.26:0/110827030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8028040 con 0x564cf3c9a260 2024-05-20T21:01:41.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 -- 172.21.15.26:0/110827030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 msgr2=0x564cf3c94270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 0x564cf3c94270 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f56b80099c0 tx=0x7f56b8009710 comp rx=0 tx=0).stop 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 -- 172.21.15.26:0/110827030 shutdown_connections 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9f320 0x564cf3ca4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/110827030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9a260 0x564cf3c94270 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 -- 172.21.15.26:0/110827030 >> 172.21.15.26:0/110827030 conn(0x564cf3bc5240 msgr2=0x564cf3ca4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 -- 172.21.15.26:0/110827030 shutdown_connections 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 -- 172.21.15.26:0/110827030 wait complete. 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.099+0000 7f56da3ffb80 1 -- start start 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 Processor -- start 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cabfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3cba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3cbaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81a9f60 con 0x564cf3c9f320 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81ab3d0 con 0x564cf3c998a0 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81aa630 con 0x564cf3c9a260 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56da3ffb80 10 client.0 _ll_register_callbacks cb 0x564cf3b82950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3cbaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3cbaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58880/0 (socket says 172.21.15.26:58880) 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 -- 172.21.15.26:0/2299801154 learned_addr learned my addr 172.21.15.26:0/2299801154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56cd905700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cabfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56cd104700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3cba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 -- 172.21.15.26:0/2299801154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 msgr2=0x564cf3cba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3cba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 -- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 msgr2=0x564cf3cabfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cabfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.100+0000 7f56ce106700 1 -- 172.21.15.26:0/2299801154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf3cbb490 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56ce106700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3cbaca0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f56c00a97a0 tx=0x7f56c0004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ed91129fe9b112 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56b17fa700 1 -- 172.21.15.26:0/2299801154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00b2290 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56b17fa700 1 -- 172.21.15.26:0/2299801154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f56c00a60e0 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b800b790 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56b17fa700 1 -- 172.21.15.26:0/2299801154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56c008e050 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 10 client.17034 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564cf3d1ddf0 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56b17fa700 1 -- 172.21.15.26:0/2299801154 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b800b790 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56b17fa700 1 -- 172.21.15.26:0/2299801154 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f56c0087060 con 0x564cf3c9f320 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 10 client.17034 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 20 client.17034 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:41.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3d1e4a0 0x564cf3d208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 4 client.17034 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.101+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cf3bf34a0 con 0x564cf3d1e4a0 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.102+0000 7f56cd905700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3d1e4a0 0x564cf3d208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.102+0000 7f56cd905700 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3d1e4a0 0x564cf3d208c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63cf52cbf14e0198 server_cookie=adbe372a8f397131 in_seq=0 out_seq=0 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.102+0000 7f56b17fa700 10 client.17034 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.102+0000 7f56b17fa700 1 -- 172.21.15.26:0/2299801154 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cf3bf34a0 con 0x564cf3d1e4a0 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.102+0000 7f56b17fa700 10 client.17034 handle_command_reply: tid=0 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.102+0000 7f56da3ffb80 1 client.17034 shutdown 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 msgr2=0x564cf3cbaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3cbaca0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f56c00a97a0 tx=0x7f56c0004f20 comp rx=0 tx=0).stop 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 shutdown_connections 2024-05-20T21:01:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564cf3d1e4a0 0x564cf3d208c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564cf3c9f320 0x564cf3cbaca0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cf3c998a0 0x564cf3cabfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 --2- 172.21.15.26:0/2299801154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564cf3c9a260 0x564cf3cba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56b17fa700 0 client.17034 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 >> 172.21.15.26:0/2299801154 conn(0x564cf3bc5240 msgr2=0x564cf3ca08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 shutdown_connections 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 1 -- 172.21.15.26:0/2299801154 wait complete. 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56da3ffb80 20 client.17034 trim_cache size 0 max 16384 2024-05-20T21:01:41.107 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:01:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b40b2880 msgr2=0x7f56b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56cfb69700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b40b2880 0x7f56b40b4d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f56c81032a0 tx=0x7f56c400b040 comp rx=0 tx=0).stop 2024-05-20T21:01:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 msgr2=0x7f56c8199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.103+0000 7f56cfb69700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 0x7f56c8199920 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e940 tx=0x7f56c000ec50 comp rx=0 tx=0).stop 2024-05-20T21:01:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 shutdown_connections 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56b40b2880 0x7f56b40b4d40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c81088f0 0x7f56c8199920 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c81037f0 0x7f56c819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 --2- 172.21.15.26:0/275269433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56c81041a0 0x7f56c819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 >> 172.21.15.26:0/275269433 conn(0x7f56c807a6a0 msgr2=0x7f56c8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 shutdown_connections 2024-05-20T21:01:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:41.104+0000 7f56cfb69700 1 -- 172.21.15.26:0/275269433 wait complete. 2024-05-20T21:01:41.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:40 smithi055 ceph-mon[35213]: pgmap v555: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-05-20T21:01:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:40 smithi144 ceph-mon[35660]: pgmap v555: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-05-20T21:01:41.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.95 MB/sec execute 107 sec latency 6590.934 ms 2024-05-20T21:01:42.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.93 MB/sec execute 108 sec latency 7590.984 ms 2024-05-20T21:01:42.895 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:01:42.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:01:43.162 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:43.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:42 smithi055 ceph-mon[35213]: pgmap v556: 97 pgs: 97 active+clean; 27 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-20T21:01:43.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:42 smithi026 ceph-mon[31990]: pgmap v556: 97 pgs: 97 active+clean; 27 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-20T21:01:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:42 smithi144 ceph-mon[35660]: pgmap v556: 97 pgs: 97 active+clean; 27 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-20T21:01:43.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.91 MB/sec execute 109 sec latency 8591.062 ms 2024-05-20T21:01:44.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.89 MB/sec execute 110 sec latency 9591.150 ms 2024-05-20T21:01:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 -- 172.21.15.26:0/1330945756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 msgr2=0x7f05bc109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 --2- 172.21.15.26:0/1330945756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc109960 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0099c0 tx=0x7f05ac009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 -- 172.21.15.26:0/1330945756 shutdown_connections 2024-05-20T21:01:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 --2- 172.21.15.26:0/1330945756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc109960 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 --2- 172.21.15.26:0/1330945756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06c470 0x7f05bc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 --2- 172.21.15.26:0/1330945756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05bc06bab0 0x7f05bc06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 -- 172.21.15.26:0/1330945756 >> 172.21.15.26:0/1330945756 conn(0x7f05bc079280 msgr2=0x7f05bc079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 -- 172.21.15.26:0/1330945756 shutdown_connections 2024-05-20T21:01:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.153+0000 7f05bb59e700 1 -- 172.21.15.26:0/1330945756 wait complete. 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 Processor -- start 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 -- start start 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06bab0 0x7f05bc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05bc06c470 0x7f05bc19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc113580 con 0x7f05bc109580 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc113400 con 0x7f05bc06c470 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bb59e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc113700 con 0x7f05bc06bab0 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58902/0 (socket says 172.21.15.26:58902) 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 -- 172.21.15.26:0/2085785016 learned_addr learned my addr 172.21.15.26:0/2085785016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05ba59c700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06bab0 0x7f05bc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05bc06c470 0x7f05bc19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 -- 172.21.15.26:0/2085785016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06bab0 msgr2=0x7f05bc19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06bab0 0x7f05bc19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 -- 172.21.15.26:0/2085785016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05bc06c470 msgr2=0x7f05bc19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05bc06c470 0x7f05bc19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.154+0000 7f05bad9d700 1 -- 172.21.15.26:0/2085785016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc19a020 con 0x7f05bc109580 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.155+0000 7f05bad9d700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc199840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f05b400cca0 tx=0x7f05b400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.155+0000 7f05ab7fe700 1 -- 172.21.15.26:0/2085785016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4016020 con 0x7f05bc109580 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.155+0000 7f05ab7fe700 1 -- 172.21.15.26:0/2085785016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05b400ad60 con 0x7f05bc109580 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.155+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc19a310 con 0x7f05bc109580 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.155+0000 7f05ab7fe700 1 -- 172.21.15.26:0/2085785016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40115e0 con 0x7f05bc109580 2024-05-20T21:01:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.155+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc1a64d0 con 0x7f05bc109580 2024-05-20T21:01:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.157+0000 7f05ab7fe700 1 -- 172.21.15.26:0/2085785016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f05b4019020 con 0x7f05bc109580 2024-05-20T21:01:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.158+0000 7f05ab7fe700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f05a40b2880 0x7f05a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.158+0000 7f05ba59c700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f05a40b2880 0x7f05a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.158+0000 7f05ab7fe700 1 -- 172.21.15.26:0/2085785016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f05b40e2290 con 0x7f05bc109580 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 Processor -- start 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05ba59c700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f05a40b2880 0x7f05a40b4d40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 -- start start 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed0e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed09b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed15da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ec68d10 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ec588b0 con 0x55bb5ed09700 2024-05-20T21:01:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ec5c8f0 con 0x55bb5ed0de60 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed15da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed15da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58906/0 (socket says 172.21.15.26:58906) 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05bad9d700 1 -- 172.21.15.26:0/3523230295 learned_addr learned my addr 172.21.15.26:0/3523230295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed0e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.159+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed09b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05bad9d700 1 -- 172.21.15.26:0/3523230295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 msgr2=0x55bb5ed0e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05bad9d700 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed0e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05bad9d700 1 -- 172.21.15.26:0/3523230295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 msgr2=0x55bb5ed09b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05bad9d700 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed09b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05bad9d700 1 -- 172.21.15.26:0/3523230295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5eba8950 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05bad9d700 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed15da0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f05b4005c30 tx=0x7f05b400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd1ac7509dc6b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05a8ff9700 1 -- 172.21.15.26:0/3523230295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40a9ef0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05a8ff9700 1 -- 172.21.15.26:0/3523230295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05b40a97f0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.160+0000 7f05a8ff9700 1 -- 172.21.15.26:0/3523230295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40b21c0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3523230295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 msgr2=0x55bb5ed15da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed15da0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f05b4005c30 tx=0x7f05b400cea0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3523230295 shutdown_connections 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed15da0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed0e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3523230295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed09b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3523230295 >> 172.21.15.26:0/3523230295 conn(0x55bb5ec37100 msgr2=0x55bb5ed03ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:45.164 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:44 smithi026 ceph-mon[31990]: pgmap v557: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 798 KiB/s wr, 68 op/s 2024-05-20T21:01:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3523230295 shutdown_connections 2024-05-20T21:01:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3523230295 wait complete. 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.161+0000 7f05cb69ab80 1 -- start start 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 Processor -- start 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed15ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed12dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb5eb5f020 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ebfa840 con 0x55bb5ed09700 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ed16430 con 0x55bb5ed0de60 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05cb69ab80 10 client.0 _ll_register_callbacks cb 0x55bb5ebf39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05b9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed12dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05b9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed12dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58910/0 (socket says 172.21.15.26:58910) 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05b9d9b700 1 -- 172.21.15.26:0/2573449228 learned_addr learned my addr 172.21.15.26:0/2573449228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05ba59c700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed15ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.162+0000 7f05bad9d700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05b9d9b700 1 -- 172.21.15.26:0/2573449228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 msgr2=0x55bb5ed13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05b9d9b700 1 -- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 msgr2=0x55bb5ed15ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed15ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05b9d9b700 1 -- 172.21.15.26:0/2573449228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5ebf7710 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed12dd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f05b000de80 tx=0x7f05b000f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1cc8d1991698ce server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05a1ffb700 1 -- 172.21.15.26:0/2573449228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b000fe30 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05a1ffb700 1 -- 172.21.15.26:0/2573449228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f05b0015410 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b40a97f0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05a1ffb700 1 -- 172.21.15.26:0/2573449228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05b0015610 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05cb69ab80 10 client.17046 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb5ed250a0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.163+0000 7f05a1ffb700 1 -- 172.21.15.26:0/2573449228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05b001a070 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05a1ffb700 1 -- 172.21.15.26:0/2573449228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f05b0013850 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05cb69ab80 10 client.17046 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05cb69ab80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05cb69ab80 20 client.17046 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed276e0 0x55bb5ed29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05cb69ab80 4 client.17046 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f05b40a97f0 con 0x55bb5ed276e0 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.164+0000 7f05ba59c700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed276e0 0x55bb5ed29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.165+0000 7f05ba59c700 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed276e0 0x55bb5ed29ad0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a05042b16d4e8f server_cookie=38d57e5dad615ec5 in_seq=0 out_seq=0 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.165+0000 7f05a1ffb700 10 client.17046 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.165+0000 7f05a1ffb700 1 -- 172.21.15.26:0/2573449228 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f05b40a97f0 con 0x55bb5ed276e0 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.165+0000 7f05a1ffb700 10 client.17046 handle_command_reply: tid=0 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.165+0000 7f05cb69ab80 1 client.17046 shutdown 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 msgr2=0x55bb5ed12dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed12dd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f05b000de80 tx=0x7f05b000f4c0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 shutdown_connections 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed276e0 0x55bb5ed29ad0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0de60 0x55bb5ed13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed09700 0x55bb5ed15ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/2573449228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed12dd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05a1ffb700 0 client.17046 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 >> 172.21.15.26:0/2573449228 conn(0x55bb5ec37100 msgr2=0x55bb5ed0f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 shutdown_connections 2024-05-20T21:01:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 1 -- 172.21.15.26:0/2573449228 wait complete. 2024-05-20T21:01:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.166+0000 7f05cb69ab80 20 client.17046 trim_cache size 0 max 16384 2024-05-20T21:01:45.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 Processor -- start 2024-05-20T21:01:45.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 -- start start 2024-05-20T21:01:45.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 0x55bb5ed0d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed11d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed2f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1ae160 con 0x55bb5ed09700 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1ab9f0 con 0x55bb5ed0de60 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1a9840 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05ba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 0x55bb5ed0d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05ba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 0x55bb5ed0d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58924/0 (socket says 172.21.15.26:58924) 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.187+0000 7f05ba59c700 1 -- 172.21.15.26:0/3787819663 learned_addr learned my addr 172.21.15.26:0/3787819663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05bad9d700 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed2f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed11d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05ba59c700 1 -- 172.21.15.26:0/3787819663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0a0c0 msgr2=0x55bb5ed11d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed11d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05ba59c700 1 -- 172.21.15.26:0/3787819663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 msgr2=0x55bb5ed2f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed2f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05ba59c700 1 -- 172.21.15.26:0/3787819663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5ed1f4d0 con 0x55bb5ed09700 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 0x55bb5ed0d610 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00bfd0 tx=0x7f05ac01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7e562780a0801 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05a3fff700 1 -- 172.21.15.26:0/3787819663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac004890 con 0x55bb5ed09700 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05a3fff700 1 -- 172.21.15.26:0/3787819663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05ac00f460 con 0x55bb5ed09700 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.188+0000 7f05a3fff700 1 -- 172.21.15.26:0/3787819663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac025680 con 0x55bb5ed09700 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3787819663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 msgr2=0x55bb5ed0d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 0x55bb5ed0d610 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00bfd0 tx=0x7f05ac01e000 comp rx=0 tx=0).stop 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3787819663 shutdown_connections 2024-05-20T21:01:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed09700 0x55bb5ed0d610 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed2f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3787819663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed11d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3787819663 >> 172.21.15.26:0/3787819663 conn(0x55bb5ec37100 msgr2=0x55bb5ed0cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3787819663 shutdown_connections 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3787819663 wait complete. 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 -- start start 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.189+0000 7f05cb69ab80 1 Processor -- start 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed09700 0x55bb5ed0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed1b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed1bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1a9eb0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1ab320 con 0x55bb5ed0de60 2024-05-20T21:01:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1aa580 con 0x55bb5ed09700 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05cb69ab80 10 client.0 _ll_register_callbacks cb 0x55bb5ebf39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed1b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed1b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58938/0 (socket says 172.21.15.26:58938) 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 -- 172.21.15.26:0/3459555817 learned_addr learned my addr 172.21.15.26:0/3459555817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed09700 0x55bb5ed0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 -- 172.21.15.26:0/3459555817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed09700 msgr2=0x55bb5ed0c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed09700 0x55bb5ed0c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05bad9d700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed1bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 -- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 msgr2=0x55bb5ed1bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed1bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 -- 172.21.15.26:0/3459555817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5ed0ab30 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05bad9d700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed1bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05b9d9b700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed1b6c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f05b0000c00 tx=0x7f05b00153f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c888bad1eff5a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.190+0000 7f05a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05a1ffb700 1 -- 172.21.15.26:0/3459555817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b002d020 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05a1ffb700 1 -- 172.21.15.26:0/3459555817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f05b0013810 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ac00f460 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05a1ffb700 1 -- 172.21.15.26:0/3459555817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05b000f810 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 10 client.17052 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb5ed935a0 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05a1ffb700 1 -- 172.21.15.26:0/3459555817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05ac00f460 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05a1ffb700 1 -- 172.21.15.26:0/3459555817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f05b0028580 con 0x55bb5ed0a0c0 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 10 client.17052 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 20 client.17052 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:45.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed93d00 0x55bb5ed96120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:45.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 4 client.17052 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:01:45.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb5ecd5a60 con 0x55bb5ed93d00 2024-05-20T21:01:45.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.191+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed93d00 0x55bb5ed96120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:45.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.192+0000 7f05ba59c700 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed93d00 0x55bb5ed96120 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd15cae9cf2534c9 server_cookie=44369393a54c6fb4 in_seq=0 out_seq=0 2024-05-20T21:01:45.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.192+0000 7f05a1ffb700 10 client.17052 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:45.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05a1ffb700 1 -- 172.21.15.26:0/3459555817 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb5ecd5a60 con 0x55bb5ed93d00 2024-05-20T21:01:45.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05a1ffb700 10 client.17052 handle_command_reply: tid=0 2024-05-20T21:01:45.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 client.17052 shutdown 2024-05-20T21:01:45.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 msgr2=0x55bb5ed1b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed1b6c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f05b0000c00 tx=0x7f05b00153f0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 shutdown_connections 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bb5ed93d00 0x55bb5ed96120 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bb5ed0de60 0x55bb5ed1bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb5ed09700 0x55bb5ed0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 --2- 172.21.15.26:0/3459555817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb5ed0a0c0 0x55bb5ed1b6c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 >> 172.21.15.26:0/3459555817 conn(0x55bb5ec37100 msgr2=0x55bb5ed02d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05a1ffb700 0 client.17052 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.193+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 shutdown_connections 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05cb69ab80 1 -- 172.21.15.26:0/3459555817 wait complete. 2024-05-20T21:01:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05cb69ab80 20 client.17052 trim_cache size 0 max 16384 2024-05-20T21:01:45.198 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:45.198 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:01:45.198 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "e6ca1e3d-2535-4cfe-93bc-a8dc3254c9fa", 2024-05-20T21:01:45.198 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:01:45.198 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f05a40b2880 msgr2=0x7f05a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f05a40b2880 0x7f05a40b4d40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac01a040 comp rx=0 tx=0).stop 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 msgr2=0x7f05bc199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc199840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f05b400cca0 tx=0x7f05b400a340 comp rx=0 tx=0).stop 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 shutdown_connections 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f05a40b2880 0x7f05a40b4d40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05bc109580 0x7f05bc199840 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06bab0 0x7f05bc19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 --2- 172.21.15.26:0/2085785016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05bc06c470 0x7f05bc19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 >> 172.21.15.26:0/2085785016 conn(0x7f05bc079280 msgr2=0x7f05bc101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 shutdown_connections 2024-05-20T21:01:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:45.194+0000 7f05bb59e700 1 -- 172.21.15.26:0/2085785016 wait complete. 2024-05-20T21:01:45.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:44 smithi144 ceph-mon[35660]: pgmap v557: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 798 KiB/s wr, 68 op/s 2024-05-20T21:01:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:44 smithi055 ceph-mon[35213]: pgmap v557: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 798 KiB/s wr, 68 op/s 2024-05-20T21:01:45.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.88 MB/sec execute 111 sec latency 10591.250 ms 2024-05-20T21:01:45.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:01:46.244 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:46.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:01:46.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.86 MB/sec execute 112 sec latency 11591.292 ms 2024-05-20T21:01:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:46 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:01:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:46 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:01:47.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:46 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:01:47.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:46 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:01:47.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:46 smithi144 ceph-mon[35660]: pgmap v558: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 431 KiB/s wr, 40 op/s 2024-05-20T21:01:47.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:46 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:01:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:46 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:01:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:46 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:01:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:46 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:01:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:46 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:01:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:46 smithi055 ceph-mon[35213]: pgmap v558: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 431 KiB/s wr, 40 op/s 2024-05-20T21:01:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:46 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:01:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:46 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:01:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:46 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:01:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:46 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:01:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:46 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:01:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:46 smithi026 ceph-mon[31990]: pgmap v558: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 431 KiB/s wr, 40 op/s 2024-05-20T21:01:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:46 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:01:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:47.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.84 MB/sec execute 113 sec latency 12591.326 ms 2024-05-20T21:01:48.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.83 MB/sec execute 114 sec latency 13591.421 ms 2024-05-20T21:01:49.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:48 smithi055 ceph-mon[35213]: pgmap v559: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-05-20T21:01:49.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:48 smithi144 ceph-mon[35660]: pgmap v559: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-05-20T21:01:49.357 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:48 smithi026 ceph-mon[31990]: pgmap v559: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-05-20T21:01:49.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.81 MB/sec execute 115 sec latency 14591.531 ms 2024-05-20T21:01:49.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.473+0000 7f1c220d0700 1 -- 172.21.15.26:0/3474437468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101f00 msgr2=0x7f1c1c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:49.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.473+0000 7f1c220d0700 1 --2- 172.21.15.26:0/3474437468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101f00 0x7f1c1c102380 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009990 tx=0x7f1c10009ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 -- 172.21.15.26:0/3474437468 shutdown_connections 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 --2- 172.21.15.26:0/3474437468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 --2- 172.21.15.26:0/3474437468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c108620 0x7f1c1c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 --2- 172.21.15.26:0/3474437468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101f00 0x7f1c1c102380 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 -- 172.21.15.26:0/3474437468 >> 172.21.15.26:0/3474437468 conn(0x7f1c1c077780 msgr2=0x7f1c1c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 -- 172.21.15.26:0/3474437468 shutdown_connections 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 -- 172.21.15.26:0/3474437468 wait complete. 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.474+0000 7f1c220d0700 1 Processor -- start 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 -- start start 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c101f00 0x7f1c1c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c108620 0x7f1c1c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c113660 con 0x7f1c1c1028c0 2024-05-20T21:01:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1134e0 con 0x7f1c1c108620 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c220d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1137e0 con 0x7f1c1c101f00 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58960/0 (socket says 172.21.15.26:58960) 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 -- 172.21.15.26:0/2406374870 learned_addr learned my addr 172.21.15.26:0/2406374870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c108620 0x7f1c1c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c101f00 0x7f1c1c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 -- 172.21.15.26:0/2406374870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c101f00 msgr2=0x7f1c1c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c101f00 0x7f1c1c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 -- 172.21.15.26:0/2406374870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c108620 msgr2=0x7f1c1c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c108620 0x7f1c1c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.475+0000 7f1c1affd700 1 -- 172.21.15.26:0/2406374870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c19a140 con 0x7f1c1c1028c0 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.476+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c19fd10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c00 tx=0x7f1c100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:49.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.476+0000 7f1c18ff9700 1 -- 172.21.15.26:0/2406374870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10027070 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.476+0000 7f1c18ff9700 1 -- 172.21.15.26:0/2406374870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c1000fb80 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.476+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c19a3d0 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.476+0000 7f1c18ff9700 1 -- 172.21.15.26:0/2406374870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10022d00 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.476+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a66a0 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.479+0000 7f1c18ff9700 1 -- 172.21.15.26:0/2406374870 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1c1002a070 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.479+0000 7f1c18ff9700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c040b2900 0x7f1c040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.479+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c040b2900 0x7f1c040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c18ff9700 1 -- 172.21.15.26:0/2406374870 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1c100e3030 con 0x7f1c1c1028c0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 Processor -- start 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 -- start start 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c040b2900 0x7f1c040b4dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006fd0 tx=0x7f1c0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cd210 0x5628589d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589d0f60 0x5628589d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56285892bc90 con 0x5628589cc860 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56285891b830 con 0x5628589cd210 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56285891f870 con 0x5628589d0f60 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58972/0 (socket says 172.21.15.26:58972) 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/1849844631 learned_addr learned my addr 172.21.15.26:0/1849844631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589d0f60 0x5628589d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1affd700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cd210 0x5628589d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/1849844631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589d0f60 msgr2=0x5628589d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589d0f60 0x5628589d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.480+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/1849844631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cd210 msgr2=0x5628589d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cd210 0x5628589d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/1849844631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285886b8d0 con 0x5628589cc860 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589d0f60 0x5628589d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c1affd700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cd210 0x5628589d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589ccc40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00d6a0 tx=0x7f1c0c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aa41dc8eeeab50 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c01ffb700 1 -- 172.21.15.26:0/1849844631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c024070 con 0x5628589cc860 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c01ffb700 1 -- 172.21.15.26:0/1849844631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c0c014ad0 con 0x5628589cc860 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.481+0000 7f1c01ffb700 1 -- 172.21.15.26:0/1849844631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c01f9a0 con 0x5628589cc860 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 -- 172.21.15.26:0/1849844631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 msgr2=0x5628589ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589ccc40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00d6a0 tx=0x7f1c0c010000 comp rx=0 tx=0).stop 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 -- 172.21.15.26:0/1849844631 shutdown_connections 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589ccc40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589d0f60 0x5628589d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/1849844631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cd210 0x5628589d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 -- 172.21.15.26:0/1849844631 >> 172.21.15.26:0/1849844631 conn(0x5628588fa080 msgr2=0x5628589c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 -- 172.21.15.26:0/1849844631 shutdown_connections 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 -- 172.21.15.26:0/1849844631 wait complete. 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 -- start start 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.482+0000 7f1c2c966b80 1 Processor -- start 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cc860 0x5628589d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 0x5628589d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562858821fa0 con 0x5628589cd210 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5628588bd7c0 con 0x5628589d0f60 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628589d9120 con 0x5628589cc860 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 10 client.0 _ll_register_callbacks cb 0x5628588b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 0x5628589d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 0x5628589d5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58982/0 (socket says 172.21.15.26:58982) 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 -- 172.21.15.26:0/2471090933 learned_addr learned my addr 172.21.15.26:0/2471090933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 -- 172.21.15.26:0/2471090933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cc860 msgr2=0x5628589d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cc860 0x5628589d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cc860 0x5628589d8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 -- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 msgr2=0x5628589d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 -- 172.21.15.26:0/2471090933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628588ba690 con 0x5628589cd210 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cc860 0x5628589d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 0x5628589d5720 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005cd0 tx=0x7f1c100227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e139b134205e8e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1bfaffd700 1 -- 172.21.15.26:0/2471090933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c100aea30 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c014ad0 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.483+0000 7f1bfaffd700 1 -- 172.21.15.26:0/2471090933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1c100920f0 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1bfaffd700 1 -- 172.21.15.26:0/2471090933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c100b3450 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 10 client.17064 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628589e3810 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1bfaffd700 1 -- 172.21.15.26:0/2471090933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c0c014ad0 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1bfaffd700 1 -- 172.21.15.26:0/2471090933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f1c100a98e0 con 0x5628589cd210 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 10 client.17064 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 20 client.17064 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x5628589e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 4 client.17064 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628589989e0 con 0x5628589e5ce0 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.484+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x5628589e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.485+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x5628589e80d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410870da1e43ee server_cookie=40d4dc8baa646bec in_seq=0 out_seq=0 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.485+0000 7f1bfaffd700 10 client.17064 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.485+0000 7f1bfaffd700 1 -- 172.21.15.26:0/2471090933 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5628589989e0 con 0x5628589e5ce0 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.485+0000 7f1bfaffd700 10 client.17064 handle_command_reply: tid=0 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.485+0000 7f1c2c966b80 1 client.17064 shutdown 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 msgr2=0x5628589d5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 0x5628589d5720 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005cd0 tx=0x7f1c100227e0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 shutdown_connections 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x5628589e80d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1bfaffd700 0 client.17064 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cc860 0x5628589d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/2471090933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cd210 0x5628589d5720 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 >> 172.21.15.26:0/2471090933 conn(0x5628588fa080 msgr2=0x5628589c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 shutdown_connections 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 1 -- 172.21.15.26:0/2471090933 wait complete. 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.486+0000 7f1c2c966b80 20 client.17064 trim_cache size 0 max 16384 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 Processor -- start 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 -- start start 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589edf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1b0520 con 0x5628589cc860 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1addb0 con 0x5628589d0f60 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1abc00 con 0x5628589cd210 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58994/0 (socket says 172.21.15.26:58994) 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/3393601800 learned_addr learned my addr 172.21.15.26:0/3393601800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1affd700 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/3393601800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 msgr2=0x5628589ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/3393601800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 msgr2=0x5628589edf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589edf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 -- 172.21.15.26:0/3393601800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628589d3cb0 con 0x5628589cc860 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.507+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589c6c90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c01f550 tx=0x7f1c0c01f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d137f23c1b44703 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c017fa700 1 -- 172.21.15.26:0/3393601800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c03a070 con 0x5628589cc860 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c017fa700 1 -- 172.21.15.26:0/3393601800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c0c01f5f0 con 0x5628589cc860 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c017fa700 1 -- 172.21.15.26:0/3393601800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c031030 con 0x5628589cc860 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 -- 172.21.15.26:0/3393601800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 msgr2=0x5628589c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589c6c90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c01f550 tx=0x7f1c0c01f580 comp rx=0 tx=0).stop 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 -- 172.21.15.26:0/3393601800 shutdown_connections 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589cc860 0x5628589c6c90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589d0f60 0x5628589edf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/3393601800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 -- 172.21.15.26:0/3393601800 >> 172.21.15.26:0/3393601800 conn(0x5628588fa080 msgr2=0x5628589cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 -- 172.21.15.26:0/3393601800 shutdown_connections 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.508+0000 7f1c2c966b80 1 -- 172.21.15.26:0/3393601800 wait complete. 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 -- start start 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 Processor -- start 2024-05-20T21:01:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cc860 0x5628589dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589dc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 0x5628589dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ac270 con 0x5628589d0f60 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ad6e0 con 0x5628589cc860 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ac940 con 0x5628589cd210 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c2c966b80 10 client.0 _ll_register_callbacks cb 0x5628588b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 0x5628589dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 0x5628589dcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59002/0 (socket says 172.21.15.26:59002) 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.509+0000 7f1c1bfff700 1 -- 172.21.15.26:0/336220525 learned_addr learned my addr 172.21.15.26:0/336220525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1affd700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589dc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cc860 0x5628589dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1bfff700 1 -- 172.21.15.26:0/336220525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 msgr2=0x5628589dc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589dc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1bfff700 1 -- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cc860 msgr2=0x5628589dc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cc860 0x5628589dc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1bfff700 1 -- 172.21.15.26:0/336220525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562858a4fad0 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cc860 0x5628589dc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c1bfff700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 0x5628589dcd60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1c14004070 tx=0x7f1c14004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfe31597ebcd9da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1bfaffd700 1 -- 172.21.15.26:0/336220525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1401b070 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c01f5f0 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1bfaffd700 1 -- 172.21.15.26:0/336220525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1c140079d0 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1bfaffd700 1 -- 172.21.15.26:0/336220525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c14027050 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c2c966b80 10 client.17070 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562858a50e70 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.510+0000 7f1bfaffd700 1 -- 172.21.15.26:0/336220525 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c0c01f5f0 con 0x5628589d0f60 2024-05-20T21:01:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1bfaffd700 1 -- 172.21.15.26:0/336220525 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f1c14017c90 con 0x5628589d0f60 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c2c966b80 10 client.17070 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c2c966b80 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c2c966b80 20 client.17070 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x562858a53570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c2c966b80 4 client.17070 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56285892dbe0 con 0x5628589e5ce0 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x562858a53570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1bfaffd700 10 client.17070 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.511+0000 7f1c1b7fe700 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x562858a53570 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6f0bcd1f23f2c6 server_cookie=22c1333a691023b6 in_seq=0 out_seq=0 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1bfaffd700 1 -- 172.21.15.26:0/336220525 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56285892dbe0 con 0x5628589e5ce0 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1bfaffd700 10 client.17070 handle_command_reply: tid=0 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 client.17070 shutdown 2024-05-20T21:01:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 msgr2=0x5628589dcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 0x5628589dcd60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1c14004070 tx=0x7f1c14004430 comp rx=0 tx=0).stop 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 shutdown_connections 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5628589e5ce0 0x562858a53570 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628589d0f60 0x5628589dcd60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5628589cc860 0x5628589dc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 --2- 172.21.15.26:0/336220525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628589cd210 0x5628589dc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1bfaffd700 0 client.17070 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 >> 172.21.15.26:0/336220525 conn(0x5628588fa080 msgr2=0x5628589cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 shutdown_connections 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 1 -- 172.21.15.26:0/336220525 wait complete. 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.512+0000 7f1c2c966b80 20 client.17070 trim_cache size 0 max 16384 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c040b2900 msgr2=0x7f1c040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c040b2900 0x7f1c040b4dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006fd0 tx=0x7f1c0c008040 comp rx=0 tx=0).stop 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 msgr2=0x7f1c1c19fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c19fd10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c00 tx=0x7f1c100047b0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 shutdown_connections 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c108620 0x7f1c1c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c101f00 0x7f1c1c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c040b2900 0x7f1c040b4dc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 --2- 172.21.15.26:0/2406374870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1028c0 0x7f1c1c19fd10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 >> 172.21.15.26:0/2406374870 conn(0x7f1c1c077780 msgr2=0x7f1c1c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 shutdown_connections 2024-05-20T21:01:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:49.513+0000 7f1c220d0700 1 -- 172.21.15.26:0/2406374870 wait complete. 2024-05-20T21:01:50.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.80 MB/sec execute 116 sec latency 15591.929 ms 2024-05-20T21:01:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:50 smithi026 ceph-mon[31990]: pgmap v560: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 125 KiB/s wr, 35 op/s 2024-05-20T21:01:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:50 smithi144 ceph-mon[35660]: pgmap v560: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 125 KiB/s wr, 35 op/s 2024-05-20T21:01:51.428 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:01:51.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:01:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:50 smithi055 ceph-mon[35213]: pgmap v560: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 125 KiB/s wr, 35 op/s 2024-05-20T21:01:51.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.78 MB/sec execute 117 sec latency 16591.982 ms 2024-05-20T21:01:51.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 -- 172.21.15.26:0/3911903473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406c6d0 msgr2=0x7f4564110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:51.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 --2- 172.21.15.26:0/3911903473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406c6d0 0x7f4564110400 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b0d0 tx=0x7f455c00b3e0 comp rx=0 tx=0).stop 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 -- 172.21.15.26:0/3911903473 shutdown_connections 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 --2- 172.21.15.26:0/3911903473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406c6d0 0x7f4564110400 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 --2- 172.21.15.26:0/3911903473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4564109630 0x7f4564109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 --2- 172.21.15.26:0/3911903473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f456406bd10 0x7f456406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 -- 172.21.15.26:0/3911903473 >> 172.21.15.26:0/3911903473 conn(0x7f45640794e0 msgr2=0x7f45640798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 -- 172.21.15.26:0/3911903473 shutdown_connections 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.963+0000 7f456a1db700 1 -- 172.21.15.26:0/3911903473 wait complete. 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 Processor -- start 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 -- start start 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 0x7f456419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f456406c6d0 0x7f456419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4564109630 0x7f45641998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4564113640 con 0x7f456406bd10 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45641134c0 con 0x7f4564109630 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f456a1db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45641137c0 con 0x7f456406c6d0 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f45637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 0x7f456419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f45637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 0x7f456419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36470/0 (socket says 172.21.15.26:36470) 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.964+0000 7f45637fe700 1 -- 172.21.15.26:0/3914427215 learned_addr learned my addr 172.21.15.26:0/3914427215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f455affd700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f456406c6d0 0x7f456419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f4563fff700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4564109630 0x7f45641998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45637fe700 1 -- 172.21.15.26:0/3914427215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f456406c6d0 msgr2=0x7f456419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45637fe700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f456406c6d0 0x7f456419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45637fe700 1 -- 172.21.15.26:0/3914427215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4564109630 msgr2=0x7f45641998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45637fe700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4564109630 0x7f45641998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:51.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45637fe700 1 -- 172.21.15.26:0/3914427215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456419a040 con 0x7f456406bd10 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f4563fff700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4564109630 0x7f45641998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45637fe700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 0x7f456419f6d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f455400d950 tx=0x7f455400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554016070 con 0x7f456406bd10 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4554009dd0 con 0x7f456406bd10 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456419a330 con 0x7f456406bd10 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.965+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455400f3c0 con 0x7f456406bd10 2024-05-20T21:01:51.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.966+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a65a0 con 0x7f456406bd10 2024-05-20T21:01:51.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.967+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456406a230 con 0x7f456406bd10 2024-05-20T21:01:51.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.968+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4554019070 con 0x7f456406bd10 2024-05-20T21:01:51.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.969+0000 7f45617fa700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45500b2880 0x7f45500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:51.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.970+0000 7f455affd700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45500b2880 0x7f45500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:51.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.970+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f45540e1960 con 0x7f456406bd10 2024-05-20T21:01:51.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.970+0000 7f455affd700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45500b2880 0x7f45500b4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4564079ac0 tx=0x7f454c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:51.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:51.974+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45540a9620 con 0x7f456406bd10 2024-05-20T21:01:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.257+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f456402e560 con 0x7f456406bd10 2024-05-20T21:01:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.258+0000 7f45617fa700 1 -- 172.21.15.26:0/3914427215 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f45540a8f20 con 0x7f456406bd10 2024-05-20T21:01:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:01:52.257 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:01:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45500b2880 msgr2=0x7f45500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45500b2880 0x7f45500b4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4564079ac0 tx=0x7f454c00b410 comp rx=0 tx=0).stop 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 msgr2=0x7f456419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 0x7f456419f6d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f455400d950 tx=0x7f455400dc60 comp rx=0 tx=0).stop 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 shutdown_connections 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f45500b2880 0x7f45500b4d40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456406bd10 0x7f456419f6d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4564109630 0x7f45641998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 --2- 172.21.15.26:0/3914427215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f456406c6d0 0x7f456419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.260+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 >> 172.21.15.26:0/3914427215 conn(0x7f45640794e0 msgr2=0x7f45641010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.261+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 shutdown_connections 2024-05-20T21:01:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:52.261+0000 7f456a1db700 1 -- 172.21.15.26:0/3914427215 wait complete. 2024-05-20T21:01:52.261 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:01:52.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.77 MB/sec execute 118 sec latency 17592.018 ms 2024-05-20T21:01:53.229 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:52 smithi026 ceph-mon[31990]: pgmap v561: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-05-20T21:01:53.229 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3914427215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:53.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:52 smithi144 ceph-mon[35660]: pgmap v561: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-05-20T21:01:53.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:52 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3914427215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 -- 172.21.15.26:0/322613170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0104660 msgr2=0x7f15b010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 --2- 172.21.15.26:0/322613170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0104660 0x7f15b010aef0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009ca0 tx=0x7f15a802a8c0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 -- 172.21.15.26:0/322613170 shutdown_connections 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 --2- 172.21.15.26:0/322613170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0104660 0x7f15b010aef0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 --2- 172.21.15.26:0/322613170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15b0103ca0 0x7f15b0104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 --2- 172.21.15.26:0/322613170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0102ab0 0x7f15b0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 -- 172.21.15.26:0/322613170 >> 172.21.15.26:0/322613170 conn(0x7f15b00fe040 msgr2=0x7f15b0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 -- 172.21.15.26:0/322613170 shutdown_connections 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.387+0000 7f15b6409640 1 -- 172.21.15.26:0/322613170 wait complete. 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 Processor -- start 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 -- start start 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15b0102ab0 0x7f15b0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 0x7f15b0198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0104660 0x7f15b019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15b010e4c0 con 0x7f15b0103ca0 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15b010e340 con 0x7f15b0102ab0 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b6409640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15b010e640 con 0x7f15b0104660 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 0x7f15b0198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 0x7f15b0198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36482/0 (socket says 172.21.15.26:36482) 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 -- 172.21.15.26:0/3235411127 learned_addr learned my addr 172.21.15.26:0/3235411127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15affff640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15b0102ab0 0x7f15b0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b49a6640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0104660 0x7f15b019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 -- 172.21.15.26:0/3235411127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0104660 msgr2=0x7f15b019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0104660 0x7f15b019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 -- 172.21.15.26:0/3235411127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15b0102ab0 msgr2=0x7f15b0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15b0102ab0 0x7f15b0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15a7fff640 1 -- 172.21.15.26:0/3235411127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b01a0660 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.388+0000 7f15b49a6640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0104660 0x7f15b019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.389+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 0x7f15b0198f80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f159800c920 tx=0x7f159800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.389+0000 7f15adffb640 1 -- 172.21.15.26:0/3235411127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598007c70 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.389+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01a0950 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.389+0000 7f15adffb640 1 -- 172.21.15.26:0/3235411127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1598007e10 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.389+0000 7f15adffb640 1 -- 172.21.15.26:0/3235411127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980186f0 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.389+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b01a0ea0 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.391+0000 7f15b9203740 1 Processor -- start 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.391+0000 7f15b9203740 1 -- start start 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.391+0000 7f15adffb640 1 -- 172.21.15.26:0/3235411127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f159800f450 con 0x7f15b0103ca0 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.391+0000 7f15b9203740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 0x560961228ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b9203740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612290b0 0x56096122dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b9203740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x5609612306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b9203740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560961199c30 con 0x5609612286c0 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b9203740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5609611897d0 con 0x5609612290b0 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b9203740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56096118d810 con 0x56096122e220 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 0x560961228ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 0x560961228ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36486/0 (socket says 172.21.15.26:36486) 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 -- 172.21.15.26:0/3909569477 learned_addr learned my addr 172.21.15.26:0/3909569477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x5609612306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 -- 172.21.15.26:0/3909569477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 msgr2=0x5609612306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15affff640 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612290b0 0x56096122dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x5609612306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 -- 172.21.15.26:0/3909569477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612290b0 msgr2=0x56096122dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612290b0 0x56096122dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 -- 172.21.15.26:0/3909569477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560960e9e210 con 0x5609612286c0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x5609612306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15b49a6640 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 0x560961228ae0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f15a8035000 tx=0x7f15a8031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114ea947415cd893 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15adffb640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f159c0b28a0 0x7f159c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15a67fc640 1 -- 172.21.15.26:0/3909569477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8031c80 con 0x5609612286c0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15a67fc640 1 -- 172.21.15.26:0/3909569477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a8036040 con 0x5609612286c0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15a67fc640 1 -- 172.21.15.26:0/3909569477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8004720 con 0x5609612286c0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.392+0000 7f15affff640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f159c0b28a0 0x7f159c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15adffb640 1 -- 172.21.15.26:0/3235411127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f15980e2400 con 0x7f15b0103ca0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 -- 172.21.15.26:0/3909569477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 msgr2=0x560961228ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15affff640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f159c0b28a0 0x7f159c0b4d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5609612294b0 tx=0x7f15a000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 0x560961228ae0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f15a8035000 tx=0x7f15a8031580 comp rx=0 tx=0).stop 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 -- 172.21.15.26:0/3909569477 shutdown_connections 2024-05-20T21:01:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x5609612306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612290b0 0x56096122dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 --2- 172.21.15.26:0/3909569477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612286c0 0x560961228ae0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 -- 172.21.15.26:0/3909569477 >> 172.21.15.26:0/3909569477 conn(0x560961223d30 msgr2=0x560961224130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 -- 172.21.15.26:0/3909569477 shutdown_connections 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 -- 172.21.15.26:0/3909569477 wait complete. 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.393+0000 7f15b9203740 1 -- start start 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 Processor -- start 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612286c0 0x560961232870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961232db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56096122e220 0x560961239d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560960e6f790 con 0x5609612290b0 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560961145890 con 0x56096122e220 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560961231170 con 0x5609612286c0 2024-05-20T21:01:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b9203740 10 client.0 _ll_register_callbacks cb 0x560960e9da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961232db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961232db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36500/0 (socket says 172.21.15.26:36500) 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 -- 172.21.15.26:0/3819988192 learned_addr learned my addr 172.21.15.26:0/3819988192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15b49a6640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56096122e220 0x560961239d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15affff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612286c0 0x560961232870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 -- 172.21.15.26:0/3819988192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612286c0 msgr2=0x560961232870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612286c0 0x560961232870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 -- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56096122e220 msgr2=0x560961239d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56096122e220 0x560961239d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 -- 172.21.15.26:0/3819988192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560960f6a620 con 0x5609612290b0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15affff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612286c0 0x560961232870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.394+0000 7f15a7fff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961232db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f15980a9950 tx=0x7f15980a94d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978381c2976c70ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f158f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f158f7fe640 1 -- 172.21.15.26:0/3819988192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980add70 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f158f7fe640 1 -- 172.21.15.26:0/3819988192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f15980a5c50 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a8036040 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f158f7fe640 1 -- 172.21.15.26:0/3819988192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15980b2820 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 10 client.17088 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56096123cd40 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f158f7fe640 1 -- 172.21.15.26:0/3819988192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a8036040 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f158f7fe640 1 -- 172.21.15.26:0/3819988192 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f159806f2a0 con 0x5609612290b0 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 10 client.17088 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 20 client.17088 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56096123e410 0x560961240890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 4 client.17088 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.395+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56096119bb80 con 0x56096123e410 2024-05-20T21:01:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.396+0000 7f15affff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56096123e410 0x560961240890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.396+0000 7f15affff640 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56096123e410 0x560961240890 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a9351ed919966d server_cookie=1d9fd6945b026477 in_seq=0 out_seq=0 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.396+0000 7f158f7fe640 10 client.17088 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f158f7fe640 1 -- 172.21.15.26:0/3819988192 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56096119bb80 con 0x56096123e410 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f158f7fe640 10 client.17088 handle_command_reply: tid=0 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 client.17088 shutdown 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 msgr2=0x560961232db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961232db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f15980a9950 tx=0x7f15980a94d0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 shutdown_connections 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56096123e410 0x560961240890 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56096122e220 0x560961239d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961232db0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f158f7fe640 0 client.17088 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 --2- 172.21.15.26:0/3819988192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612286c0 0x560961232870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 >> 172.21.15.26:0/3819988192 conn(0x560961223d30 msgr2=0x560961222fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 shutdown_connections 2024-05-20T21:01:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 1 -- 172.21.15.26:0/3819988192 wait complete. 2024-05-20T21:01:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.397+0000 7f15b9203740 20 client.17088 trim_cache size 0 max 16384 2024-05-20T21:01:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 Processor -- start 2024-05-20T21:01:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 -- start start 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x56096122a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612290b0 0x56096122a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 0x560961239d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01ac6b0 con 0x56096122e220 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01a9f80 con 0x5609612286c0 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b9203740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01a7dd0 con 0x5609612290b0 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 0x560961239d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 0x560961239d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36506/0 (socket says 172.21.15.26:36506) 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 -- 172.21.15.26:0/1569465880 learned_addr learned my addr 172.21.15.26:0/1569465880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 -- 172.21.15.26:0/1569465880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612290b0 msgr2=0x56096122a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612290b0 0x56096122a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 -- 172.21.15.26:0/1569465880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 msgr2=0x56096122a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.413+0000 7f15b49a6640 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x56096122a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b49a6640 1 -- 172.21.15.26:0/1569465880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560961228aa0 con 0x56096122e220 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b49a6640 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 0x560961239d40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f15a80315f0 tx=0x7f15a8031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c4414ac425ad9d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15a5ffb640 1 -- 172.21.15.26:0/1569465880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a804a040 con 0x56096122e220 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15a5ffb640 1 -- 172.21.15.26:0/1569465880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a8036040 con 0x56096122e220 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15a5ffb640 1 -- 172.21.15.26:0/1569465880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8004840 con 0x56096122e220 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 -- 172.21.15.26:0/1569465880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 msgr2=0x560961239d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 0x560961239d40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f15a80315f0 tx=0x7f15a8031cc0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 -- 172.21.15.26:0/1569465880 shutdown_connections 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56096122e220 0x560961239d40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612290b0 0x56096122a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 --2- 172.21.15.26:0/1569465880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x56096122a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.414+0000 7f15b9203740 1 -- 172.21.15.26:0/1569465880 >> 172.21.15.26:0/1569465880 conn(0x560961223d30 msgr2=0x560961224110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 -- 172.21.15.26:0/1569465880 shutdown_connections 2024-05-20T21:01:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 -- 172.21.15.26:0/1569465880 wait complete. 2024-05-20T21:01:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 -- start start 2024-05-20T21:01:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 Processor -- start 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x560961236ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961237420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x560961238430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01a8440 con 0x5609612290b0 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01a98b0 con 0x5609612286c0 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01a8b10 con 0x56096122e220 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15b9203740 10 client.0 _ll_register_callbacks cb 0x7f159800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961237420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961237420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36520/0 (socket says 172.21.15.26:36520) 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15a7fff640 1 -- 172.21.15.26:0/1331123778 learned_addr learned my addr 172.21.15.26:0/1331123778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.415+0000 7f15affff640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x560961236ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15b49a6640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x560961238430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15a7fff640 1 -- 172.21.15.26:0/1331123778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 msgr2=0x560961238430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15a7fff640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x560961238430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15a7fff640 1 -- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 msgr2=0x560961236ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15a7fff640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x560961236ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15a7fff640 1 -- 172.21.15.26:0/1331123778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560961238a20 con 0x5609612290b0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15b49a6640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x560961238430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15a7fff640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961237420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f15980a6c30 tx=0x7f15980a9200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef10e02f3d9ca1f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f158f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f158f7fe640 1 -- 172.21.15.26:0/1331123778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980a9310 con 0x5609612290b0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f158f7fe640 1 -- 172.21.15.26:0/1331123778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f15980a55e0 con 0x5609612290b0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a8036040 con 0x5609612290b0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f158f7fe640 1 -- 172.21.15.26:0/1331123778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15980b2390 con 0x5609612290b0 2024-05-20T21:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15b9203740 10 client.17094 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560961239da0 con 0x5609612290b0 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f158f7fe640 1 -- 172.21.15.26:0/1331123778 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a8036040 con 0x5609612290b0 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.416+0000 7f158f7fe640 1 -- 172.21.15.26:0/1331123778 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f15980a8690 con 0x5609612290b0 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15b9203740 10 client.17094 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15b9203740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15b9203740 20 client.17094 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15b9203740 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5609612a9540 0x5609612ab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15b9203740 4 client.17094 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:01:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560961195360 con 0x5609612a9540 2024-05-20T21:01:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15affff640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5609612a9540 0x5609612ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f15affff640 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5609612a9540 0x5609612ab930 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441cc3a3e86e8439 server_cookie=5b8e63bfd4357a03 in_seq=0 out_seq=0 2024-05-20T21:01:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.417+0000 7f158f7fe640 10 client.17094 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f158f7fe640 1 -- 172.21.15.26:0/1331123778 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560961195360 con 0x5609612a9540 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f158f7fe640 10 client.17094 handle_command_reply: tid=0 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f15b9203740 1 client.17094 shutdown 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 msgr2=0x560961237420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f15b9203740 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961237420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f15980a6c30 tx=0x7f15980a9200 comp rx=0 tx=0).stop 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 shutdown_connections 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f15b9203740 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5609612a9540 0x5609612ab930 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.432+0000 7f15b9203740 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56096122e220 0x560961238430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b9203740 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609612290b0 0x560961237420 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b9203740 1 --2- 172.21.15.26:0/1331123778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5609612286c0 0x560961236ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f158f7fe640 0 client.17094 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 >> 172.21.15.26:0/1331123778 conn(0x560961223d30 msgr2=0x56096122a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 shutdown_connections 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b9203740 1 -- 172.21.15.26:0/1331123778 wait complete. 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b9203740 20 client.17094 trim_cache size 0 max 16384 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:01:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:01:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f159c0b28a0 msgr2=0x7f159c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f159c0b28a0 0x7f159c0b4d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5609612294b0 tx=0x7f15a000a2d0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 msgr2=0x7f15b0198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 0x7f15b0198f80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f159800c920 tx=0x7f159800cde0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 shutdown_connections 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f159c0b28a0 0x7f159c0b4d60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0104660 0x7f15b019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b0103ca0 0x7f15b0198f80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 --2- 172.21.15.26:0/3235411127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15b0102ab0 0x7f15b0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 >> 172.21.15.26:0/3235411127 conn(0x7f15b00fe040 msgr2=0x7f15b0106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 shutdown_connections 2024-05-20T21:01:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:01:53.433+0000 7f15b6409640 1 -- 172.21.15.26:0/3235411127 wait complete. 2024-05-20T21:01:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:52 smithi055 ceph-mon[35213]: pgmap v561: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-05-20T21:01:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:52 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3914427215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:53.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.75 MB/sec execute 119 sec latency 18592.069 ms 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 -- 172.21.15.26:0/1286185416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8301025d0 msgr2=0x7fe8301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 --2- 172.21.15.26:0/1286185416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8301025d0 0x7fe8301029b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0099c0 tx=0x7fe82c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 -- 172.21.15.26:0/1286185416 shutdown_connections 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 --2- 172.21.15.26:0/1286185416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8301025d0 0x7fe8301029b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 --2- 172.21.15.26:0/1286185416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8301076b0 0x7fe83010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 --2- 172.21.15.26:0/1286185416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe830102f80 0x7fe830106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 -- 172.21.15.26:0/1286185416 >> 172.21.15.26:0/1286185416 conn(0x7fe8300fde60 msgr2=0x7fe830100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 -- 172.21.15.26:0/1286185416 shutdown_connections 2024-05-20T21:01:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.053+0000 7fe8365a1700 1 -- 172.21.15.26:0/1286185416 wait complete. 2024-05-20T21:01:54.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.054+0000 7fe8365a1700 1 Processor -- start 2024-05-20T21:01:54.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.054+0000 7fe8365a1700 1 -- start start 2024-05-20T21:01:54.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.054+0000 7fe8365a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8301025d0 0x7fe83019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:54.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8365a1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 0x7fe83019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:54.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8365a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8301076b0 0x7fe830199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:54.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8365a1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe830112410 con 0x7fe830102f80 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8365a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe830112290 con 0x7fe8301076b0 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8365a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe830112590 con 0x7fe8301025d0 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 0x7fe83019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 0x7fe83019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36548/0 (socket says 172.21.15.26:36548) 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 -- 172.21.15.26:0/221576566 learned_addr learned my addr 172.21.15.26:0/221576566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe83559f700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8301025d0 0x7fe83019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe835da0700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8301076b0 0x7fe830199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 -- 172.21.15.26:0/221576566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8301025d0 msgr2=0x7fe83019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8301025d0 0x7fe83019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 -- 172.21.15.26:0/221576566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8301076b0 msgr2=0x7fe830199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8301076b0 0x7fe830199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 -- 172.21.15.26:0/221576566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe830199fe0 con 0x7fe830102f80 2024-05-20T21:01:54.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe83559f700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8301025d0 0x7fe83019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:54.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe834d9e700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 0x7fe83019fc00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe82000d950 tx=0x7fe82000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:54.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820017070 con 0x7fe830102f80 2024-05-20T21:01:54.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe82000de30 con 0x7fe830102f80 2024-05-20T21:01:54.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83019a2d0 con 0x7fe830102f80 2024-05-20T21:01:54.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.055+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820005310 con 0x7fe830102f80 2024-05-20T21:01:54.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.056+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8301a6590 con 0x7fe830102f80 2024-05-20T21:01:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.057+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83006a230 con 0x7fe830102f80 2024-05-20T21:01:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.058+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe820004750 con 0x7fe830102f80 2024-05-20T21:01:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.059+0000 7fe8267fc700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe81c0b2880 0x7fe81c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.059+0000 7fe83559f700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe81c0b2880 0x7fe81c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.059+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fe8200e1be0 con 0x7fe830102f80 2024-05-20T21:01:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.059+0000 7fe83559f700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe81c0b2880 0x7fe81c0b4d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009990 tx=0x7fe82c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:54.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.063+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8200a9840 con 0x7fe830102f80 2024-05-20T21:01:54.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.331+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe83002e560 con 0x7fe830102f80 2024-05-20T21:01:54.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.332+0000 7fe8267fc700 1 -- 172.21.15.26:0/221576566 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7fe8200a9140 con 0x7fe830102f80 2024-05-20T21:01:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:01:54.331 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:01:54.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe81c0b2880 msgr2=0x7fe81c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe81c0b2880 0x7fe81c0b4d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009990 tx=0x7fe82c01a040 comp rx=0 tx=0).stop 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 msgr2=0x7fe83019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 0x7fe83019fc00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe82000d950 tx=0x7fe82000dd10 comp rx=0 tx=0).stop 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 shutdown_connections 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8301076b0 0x7fe830199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8301025d0 0x7fe83019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe81c0b2880 0x7fe81c0b4d40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 --2- 172.21.15.26:0/221576566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe830102f80 0x7fe83019fc00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 >> 172.21.15.26:0/221576566 conn(0x7fe8300fde60 msgr2=0x7fe830100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 shutdown_connections 2024-05-20T21:01:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:54.334+0000 7fe8365a1700 1 -- 172.21.15.26:0/221576566 wait complete. 2024-05-20T21:01:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:01:54.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.74 MB/sec execute 120 sec latency 19592.126 ms 2024-05-20T21:01:54.629 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 84e775fb --release --await' 2024-05-20T21:01:54.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 84e775fb --release --await 2024-05-20T21:01:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:54 smithi144 ceph-mon[35660]: pgmap v562: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 22 op/s 2024-05-20T21:01:55.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:54 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/221576566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:55.314 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:54 smithi026 ceph-mon[31990]: pgmap v562: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 22 op/s 2024-05-20T21:01:55.314 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:54 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/221576566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:55.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:01:55.368 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:54 smithi055 ceph-mon[35213]: pgmap v562: 97 pgs: 97 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 22 op/s 2024-05-20T21:01:55.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:54 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/221576566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:01:55.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.72 MB/sec execute 121 sec latency 20592.187 ms 2024-05-20T21:01:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:01:56.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.71 MB/sec execute 122 sec latency 21592.274 ms 2024-05-20T21:01:57.173 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:56 smithi026 ceph-mon[31990]: pgmap v563: 97 pgs: 97 active+clean; 23 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-05-20T21:01:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:56 smithi144 ceph-mon[35660]: pgmap v563: 97 pgs: 97 active+clean; 23 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-05-20T21:01:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:56 smithi055 ceph-mon[35213]: pgmap v563: 97 pgs: 97 active+clean; 23 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-05-20T21:01:57.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104313 1.69 MB/sec execute 123 sec latency 22592.307 ms 2024-05-20T21:01:57.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.824+0000 7f15cbff2700 1 -- 172.21.15.26:0/1395022870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4102430 msgr2=0x7f15c41028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.824+0000 7f15cbff2700 1 --2- 172.21.15.26:0/1395022870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4102430 0x7f15c41028b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009990 tx=0x7f15b4009ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 -- 172.21.15.26:0/1395022870 shutdown_connections 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 --2- 172.21.15.26:0/1395022870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102df0 0x7f15c410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 --2- 172.21.15.26:0/1395022870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c4108510 0x7f15c41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 --2- 172.21.15.26:0/1395022870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4102430 0x7f15c41028b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 -- 172.21.15.26:0/1395022870 >> 172.21.15.26:0/1395022870 conn(0x7f15c40793b0 msgr2=0x7f15c40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 -- 172.21.15.26:0/1395022870 shutdown_connections 2024-05-20T21:01:57.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 -- 172.21.15.26:0/1395022870 wait complete. 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 Processor -- start 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.825+0000 7f15cbff2700 1 -- start start 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15cbff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 0x7f15c4104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15cbff2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c4102df0 0x7f15c41045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15cbff2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4108510 0x7f15c4104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15cbff2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4112490 con 0x7f15c4108510 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15cbff2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4112310 con 0x7f15c4102430 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15cbff2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4112610 con 0x7f15c4102df0 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15ca58f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4108510 0x7f15c4104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 0x7f15c4104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 0x7f15c4104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46988/0 (socket says 172.21.15.26:46988) 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 -- 172.21.15.26:0/2101085046 learned_addr learned my addr 172.21.15.26:0/2101085046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c958d700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c4102df0 0x7f15c41045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 -- 172.21.15.26:0/2101085046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c4102df0 msgr2=0x7f15c41045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c4102df0 0x7f15c41045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 -- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4108510 msgr2=0x7f15c4104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4108510 0x7f15c4104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 -- 172.21.15.26:0/2101085046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c41a8090 con 0x7f15c4102430 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 0x7f15c4104070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9c0 tx=0x7f15c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.826+0000 7f15baffd700 1 -- 172.21.15.26:0/2101085046 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c000cbd0 con 0x7f15c4102430 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.827+0000 7f15baffd700 1 -- 172.21.15.26:0/2101085046 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15c000ee50 con 0x7f15c4102430 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.827+0000 7f15baffd700 1 -- 172.21.15.26:0/2101085046 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00054e0 con 0x7f15c4102430 2024-05-20T21:01:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.827+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c41a8380 con 0x7f15c4102430 2024-05-20T21:01:57.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.827+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c41a8890 con 0x7f15c4102430 2024-05-20T21:01:57.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.829+0000 7f15baffd700 1 -- 172.21.15.26:0/2101085046 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f15c0004750 con 0x7f15c4102430 2024-05-20T21:01:57.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.830+0000 7f15baffd700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b00b2900 0x7f15b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.830+0000 7f15c958d700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b00b2900 0x7f15b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15baffd700 1 -- 172.21.15.26:0/2101085046 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f15c00e2440 con 0x7f15c4102430 2024-05-20T21:01:57.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 Processor -- start 2024-05-20T21:01:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 -- start start 2024-05-20T21:01:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15c958d700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b00b2900 0x7f15b00b4dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15c4105ab0 tx=0x7f15b4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273776690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d273770d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27377d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d2736cfd90 con 0x55d2737708e0 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d2736bf930 con 0x55d2737762b0 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15d6888b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d2736c3970 con 0x55d2737712a0 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.831+0000 7f15c958d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d273770d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273776690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c958d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d273770d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36576/0 (socket says 172.21.15.26:36576) 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c958d700 1 -- 172.21.15.26:0/4235693037 learned_addr learned my addr 172.21.15.26:0/4235693037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15ca58f700 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27377d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 -- 172.21.15.26:0/4235693037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 msgr2=0x55d27377d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27377d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 -- 172.21.15.26:0/4235693037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 msgr2=0x55d273770d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d273770d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 -- 172.21.15.26:0/4235693037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27360f9d0 con 0x55d2737762b0 2024-05-20T21:01:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c958d700 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d273770d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273776690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15c00a6c10 tx=0x7f15c00a9210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13d3868ef927a49 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15affff700 1 -- 172.21.15.26:0/4235693037 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00b2150 con 0x55d2737762b0 2024-05-20T21:01:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15affff700 1 -- 172.21.15.26:0/4235693037 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15c00aa0a0 con 0x55d2737762b0 2024-05-20T21:01:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.832+0000 7f15affff700 1 -- 172.21.15.26:0/4235693037 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00cc2e0 con 0x55d2737762b0 2024-05-20T21:01:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 -- 172.21.15.26:0/4235693037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 msgr2=0x55d273776690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273776690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15c00a6c10 tx=0x7f15c00a9210 comp rx=0 tx=0).stop 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 -- 172.21.15.26:0/4235693037 shutdown_connections 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27377d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273776690 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 --2- 172.21.15.26:0/4235693037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d273770d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 -- 172.21.15.26:0/4235693037 >> 172.21.15.26:0/4235693037 conn(0x55d27369e180 msgr2=0x55d27376b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 -- 172.21.15.26:0/4235693037 shutdown_connections 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.833+0000 7f15d6888b80 1 -- 172.21.15.26:0/4235693037 wait complete. 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 -- start start 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 Processor -- start 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d27377cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d2735c60a0 con 0x55d2737712a0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d2736618c0 con 0x55d2737762b0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d27377d620 con 0x55d2737708e0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15d6888b80 10 client.0 _ll_register_callbacks cb 0x55d27365aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d27377cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d27377cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:51022/0 (socket says 172.21.15.26:51022) 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15c9d8e700 1 -- 172.21.15.26:0/3977050905 learned_addr learned my addr 172.21.15.26:0/3977050905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15ca58f700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.834+0000 7f15c958d700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ca58f700 1 -- 172.21.15.26:0/3977050905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 msgr2=0x55d27377cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ca58f700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d27377cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ca58f700 1 -- 172.21.15.26:0/3977050905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 msgr2=0x55d27377a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ca58f700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ca58f700 1 -- 172.21.15.26:0/3977050905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27365e790 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d27377cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ca58f700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377a650 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15bc014e00 tx=0x7f15bc012580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e6f5de62965633 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ad7fa700 1 -- 172.21.15.26:0/3977050905 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc012b10 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ad7fa700 1 -- 172.21.15.26:0/3977050905 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f15bc012cb0 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c00aa0a0 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ad7fa700 1 -- 172.21.15.26:0/3977050905 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15bc007500 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15d6888b80 10 client.26582 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d273787dd0 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.835+0000 7f15ad7fa700 1 -- 172.21.15.26:0/3977050905 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15c00aa0a0 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15ad7fa700 1 -- 172.21.15.26:0/3977050905 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f15bc027050 con 0x55d2737762b0 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15d6888b80 10 client.26582 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15d6888b80 10 client.26582 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15d6888b80 20 client.26582 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d27378a360 0x55d27378c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15d6888b80 4 client.26582 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27373cae0 con 0x55d27378a360 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d27378a360 0x55d27378c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15ad7fa700 10 client.26582 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.836+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d27378a360 0x55d27378c750 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db9808fa46a50c6 server_cookie=1b951378420ea847 in_seq=0 out_seq=0 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15ad7fa700 1 -- 172.21.15.26:0/3977050905 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d27373cae0 con 0x55d27378a360 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15ad7fa700 10 client.26582 handle_command_reply: tid=0 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 client.26582 shutdown 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 msgr2=0x55d27377a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377a650 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15bc014e00 tx=0x7f15bc012580 comp rx=0 tx=0).stop 2024-05-20T21:01:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 shutdown_connections 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d27378a360 0x55d27378c750 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377a650 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d27377cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15ad7fa700 0 client.26582 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3977050905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 >> 172.21.15.26:0/3977050905 conn(0x55d27369e180 msgr2=0x55d273765ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 shutdown_connections 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.837+0000 7f15d6888b80 1 -- 172.21.15.26:0/3977050905 wait complete. 2024-05-20T21:01:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.838+0000 7f15d6888b80 20 client.26582 trim_cache size 0 max 16384 2024-05-20T21:01:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 Processor -- start 2024-05-20T21:01:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 -- start start 2024-05-20T21:01:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d27376f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27376fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273770350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41b2800 con 0x55d2737708e0 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41b0090 con 0x55d2737762b0 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.858+0000 7f15d6888b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41adee0 con 0x55d2737712a0 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d27376f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15ca58f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273770350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27376fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15ca58f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273770350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:47016/0 (socket says 172.21.15.26:47016) 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c9d8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d27376f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36602/0 (socket says 172.21.15.26:36602) 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15ca58f700 1 -- 172.21.15.26:0/3035964003 learned_addr learned my addr 172.21.15.26:0/3035964003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 -- 172.21.15.26:0/3035964003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 msgr2=0x55d273770350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273770350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 -- 172.21.15.26:0/3035964003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 msgr2=0x55d27376f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d27376f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 -- 172.21.15.26:0/3035964003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27377b010 con 0x55d2737712a0 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15ca58f700 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273770350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d27376f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15c958d700 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27376fe10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000c00 tx=0x7f15b4022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6317818fac198fde server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15af7fe700 1 -- 172.21.15.26:0/3035964003 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b400bc90 con 0x55d2737712a0 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15af7fe700 1 -- 172.21.15.26:0/3035964003 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b400be30 con 0x55d2737712a0 2024-05-20T21:01:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.859+0000 7f15af7fe700 1 -- 172.21.15.26:0/3035964003 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4032670 con 0x55d2737712a0 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 -- 172.21.15.26:0/3035964003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 msgr2=0x55d27376fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27376fe10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000c00 tx=0x7f15b4022000 comp rx=0 tx=0).stop 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 -- 172.21.15.26:0/3035964003 shutdown_connections 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737708e0 0x55d27376f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d273770350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 --2- 172.21.15.26:0/3035964003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737712a0 0x55d27376fe10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 -- 172.21.15.26:0/3035964003 >> 172.21.15.26:0/3035964003 conn(0x55d27369e180 msgr2=0x55d273775270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 -- 172.21.15.26:0/3035964003 shutdown_connections 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.860+0000 7f15d6888b80 1 -- 172.21.15.26:0/3035964003 wait complete. 2024-05-20T21:01:57.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 -- start start 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 Processor -- start 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d273783110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41ae550 con 0x55d2737712a0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41af9c0 con 0x55d2737762b0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41aec20 con 0x55d2737708e0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15d6888b80 10 client.0 _ll_register_callbacks cb 0x55d27365aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15c958d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15c958d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36604/0 (socket says 172.21.15.26:36604) 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15c958d700 1 -- 172.21.15.26:0/2990417918 learned_addr learned my addr 172.21.15.26:0/2990417918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.861+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d273783110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15ca58f700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15c958d700 1 -- 172.21.15.26:0/2990417918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 msgr2=0x55d273783110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15c958d700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d273783110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15c958d700 1 -- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 msgr2=0x55d27377ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15c958d700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15c958d700 1 -- 172.21.15.26:0/2990417918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2737f4e80 con 0x55d2737712a0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15c958d700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377f880 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005010 tx=0x7f15b400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d200daa5276a10 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15ad7fa700 1 -- 172.21.15.26:0/2990417918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b400fd10 con 0x55d2737712a0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15ad7fa700 1 -- 172.21.15.26:0/2990417918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f15b4032410 con 0x55d2737712a0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b400be30 con 0x55d2737712a0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15ad7fa700 1 -- 172.21.15.26:0/2990417918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15b4032610 con 0x55d2737712a0 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15d6888b80 10 client.17118 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.862+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2737f6080 con 0x55d2737712a0 2024-05-20T21:01:57.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15ad7fa700 1 -- 172.21.15.26:0/2990417918 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b400be30 con 0x55d2737712a0 2024-05-20T21:01:57.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15ad7fa700 1 -- 172.21.15.26:0/2990417918 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f15b4043030 con 0x55d2737712a0 2024-05-20T21:01:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15d6888b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15d6888b80 10 client.17118 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:01:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15d6888b80 20 client.17118 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15d6888b80 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d2737f66a0 0x55d2737f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15d6888b80 4 client.17118 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "84e775fb", "release": true, "await": true}] 2024-05-20T21:01:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "84e775fb", "release": true, "await": true}) -- 0x55d2736d1ce0 con 0x55d2737f66a0 2024-05-20T21:01:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d2737f66a0 0x55d2737f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15c9d8e700 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d2737f66a0 0x55d2737f8ac0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331b68974c727a04 server_cookie=6ac93089d2001a06 in_seq=0 out_seq=0 2024-05-20T21:01:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.863+0000 7f15ad7fa700 10 client.17118 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.871+0000 7f15ad7fa700 1 -- 172.21.15.26:0/2990417918 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d2736d1ce0 con 0x55d2737f66a0 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.871+0000 7f15ad7fa700 10 client.17118 handle_command_reply: tid=0 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 client.17118 shutdown 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 msgr2=0x55d27377f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377f880 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005010 tx=0x7f15b400f710 comp rx=0 tx=0).stop 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 shutdown_connections 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d2737f66a0 0x55d2737f8ac0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2737762b0 0x55d27377ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2737708e0 0x55d273783110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 --2- 172.21.15.26:0/2990417918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2737712a0 0x55d27377f880 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15ad7fa700 0 client.17118 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 >> 172.21.15.26:0/2990417918 conn(0x55d27369e180 msgr2=0x55d273765410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 shutdown_connections 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 1 -- 172.21.15.26:0/2990417918 wait complete. 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15d6888b80 20 client.17118 trim_cache size 0 max 16384 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 28, 2024-05-20T21:01:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 24, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "84e775fb": { 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 24, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.3, 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:01:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:01:57.875 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b00b2900 msgr2=0x7f15b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.872+0000 7f15cbff2700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b00b2900 0x7f15b00b4dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15c4105ab0 tx=0x7f15b4005fb0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 msgr2=0x7f15c4104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 0x7f15c4104070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9c0 tx=0x7f15c000ecd0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 shutdown_connections 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b00b2900 0x7f15b00b4dc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c4108510 0x7f15c4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c4102430 0x7f15c4104070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 --2- 172.21.15.26:0/2101085046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c4102df0 0x7f15c41045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 >> 172.21.15.26:0/2101085046 conn(0x7f15c40793b0 msgr2=0x7f15c406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 shutdown_connections 2024-05-20T21:01:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:57.873+0000 7f15cbff2700 1 -- 172.21.15.26:0/2101085046 wait complete. 2024-05-20T21:01:58.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:57 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:01:58.431 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:57 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:01:58.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:57 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:01:58.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104757 1.68 MB/sec execute 124 sec latency 22907.332 ms 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: pgmap v564: 97 pgs: 97 active+clean; 23 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi144 to 176.1M 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: Unable to set osd_memory_target on smithi144 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: Unable to set osd_memory_target on smithi055 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:01:59.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:01:58 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:01:59.168 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '84e775fb', total seconds elapsed: 34.0 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: pgmap v564: 97 pgs: 97 active+clean; 23 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi144 to 176.1M 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: Unable to set osd_memory_target on smithi144 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.303 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: Unable to set osd_memory_target on smithi055 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:01:59.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:01:58 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:01:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: pgmap v564: 97 pgs: 97 active+clean; 23 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi144 to 176.1M 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: Unable to set osd_memory_target on smithi144 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:01:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-05-20T21:01:59.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: Unable to set osd_memory_target on smithi055 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-20T21:01:59.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:01:59.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:01:59.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:01:58 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:01:59.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 105369 1.68 MB/sec execute 125 sec latency 66.672 ms 2024-05-20T21:01:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 -- 172.21.15.26:0/3723488235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 msgr2=0x7f35dc1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 --2- 172.21.15.26:0/3723488235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 0x7f35dc1088f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009990 tx=0x7f35d8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 -- 172.21.15.26:0/3723488235 shutdown_connections 2024-05-20T21:01:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 --2- 172.21.15.26:0/3723488235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 0x7f35dc1088f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 --2- 172.21.15.26:0/3723488235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102df0 0x7f35dc10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 --2- 172.21.15.26:0/3723488235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102430 0x7f35dc1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 -- 172.21.15.26:0/3723488235 >> 172.21.15.26:0/3723488235 conn(0x7f35dc0793b0 msgr2=0x7f35dc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 -- 172.21.15.26:0/3723488235 shutdown_connections 2024-05-20T21:01:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.660+0000 7f35e4679700 1 -- 172.21.15.26:0/3723488235 wait complete. 2024-05-20T21:01:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 Processor -- start 2024-05-20T21:01:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 -- start start 2024-05-20T21:01:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102430 0x7f35dc1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 0x7f35dc1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 0x7f35dc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1124e0 con 0x7f35dc108510 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc112360 con 0x7f35dc102430 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e4679700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc112660 con 0x7f35dc102df0 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 0x7f35dc1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e2415700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102430 0x7f35dc1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 0x7f35dc1a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:51066/0 (socket says 172.21.15.26:51066) 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e1c14700 1 -- 172.21.15.26:0/2496472866 learned_addr learned my addr 172.21.15.26:0/2496472866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e2415700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102430 0x7f35dc1a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:47054/0 (socket says 172.21.15.26:47054) 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.661+0000 7f35e2c16700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 0x7f35dc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e1c14700 1 -- 172.21.15.26:0/2496472866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102430 msgr2=0x7f35dc1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e1c14700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102430 0x7f35dc1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e1c14700 1 -- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 msgr2=0x7f35dc19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e1c14700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 0x7f35dc19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e1c14700 1 -- 172.21.15.26:0/2496472866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc19c440 con 0x7f35dc102df0 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e1c14700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 0x7f35dc1a1f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c9b0 tx=0x7f35cc00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35d37fe700 1 -- 172.21.15.26:0/2496472866 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc007b00 con 0x7f35dc102df0 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35d37fe700 1 -- 172.21.15.26:0/2496472866 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35cc00ce50 con 0x7f35dc102df0 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35d37fe700 1 -- 172.21.15.26:0/2496472866 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc018660 con 0x7f35dc102df0 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc19c730 con 0x7f35dc102df0 2024-05-20T21:01:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.662+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1a89c0 con 0x7f35dc102df0 2024-05-20T21:01:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.665+0000 7f35d37fe700 1 -- 172.21.15.26:0/2496472866 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f35cc00f450 con 0x7f35dc102df0 2024-05-20T21:01:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.666+0000 7f35d37fe700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35c80b2900 0x7f35c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.666+0000 7f35eef0fb80 1 Processor -- start 2024-05-20T21:01:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.666+0000 7f35e2415700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35c80b2900 0x7f35c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.666+0000 7f35d37fe700 1 -- 172.21.15.26:0/2496472866 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f35cc0e2130 con 0x7f35dc102df0 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.666+0000 7f35eef0fb80 1 -- start start 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e2415700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35c80b2900 0x7f35c80b4dc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b2e0 tx=0x7f35d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3dfa320 0x562ba3dfa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3df4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e01080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562ba3d52d70 con 0x562ba3df48a0 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562ba3d42910 con 0x562ba3df5260 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ba3d46950 con 0x562ba3dfa320 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3df4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3df4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40322/0 (socket says 172.21.15.26:40322) 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 -- 172.21.15.26:0/53911792 learned_addr learned my addr 172.21.15.26:0/53911792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e2c16700 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e01080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e2415700 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3dfa320 0x562ba3dfa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 -- 172.21.15.26:0/53911792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3dfa320 msgr2=0x562ba3dfa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3dfa320 0x562ba3dfa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 -- 172.21.15.26:0/53911792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 msgr2=0x562ba3e01080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e01080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 -- 172.21.15.26:0/53911792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3cebfb0 con 0x562ba3df48a0 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e2415700 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3dfa320 0x562ba3dfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.667+0000 7f35e1c14700 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3df4d20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0a5770 tx=0x7f35cc0cc150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a22c4bfb8bc307f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.668+0000 7f35d0ff9700 1 -- 172.21.15.26:0/53911792 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0a9df0 con 0x562ba3df48a0 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.668+0000 7f35d0ff9700 1 -- 172.21.15.26:0/53911792 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35cc0a96f0 con 0x562ba3df48a0 2024-05-20T21:01:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.668+0000 7f35d0ff9700 1 -- 172.21.15.26:0/53911792 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0910b0 con 0x562ba3df48a0 2024-05-20T21:01:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.668+0000 7f35eef0fb80 1 -- 172.21.15.26:0/53911792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 msgr2=0x562ba3df4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.668+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3df4d20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0a5770 tx=0x7f35cc0cc150 comp rx=0 tx=0).stop 2024-05-20T21:01:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 -- 172.21.15.26:0/53911792 shutdown_connections 2024-05-20T21:01:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e01080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3dfa320 0x562ba3dfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/53911792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3df4d20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 -- 172.21.15.26:0/53911792 >> 172.21.15.26:0/53911792 conn(0x562ba3d20240 msgr2=0x562ba3def3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 -- 172.21.15.26:0/53911792 shutdown_connections 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 -- 172.21.15.26:0/53911792 wait complete. 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 -- start start 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.669+0000 7f35eef0fb80 1 Processor -- start 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3e00cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df5260 0x562ba3dfdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3dfe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562ba3c48fa0 con 0x562ba3df48a0 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562ba3caada0 con 0x562ba3dfa320 2024-05-20T21:01:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ba3e016e0 con 0x562ba3df5260 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35eef0fb80 10 client.0 _ll_register_callbacks cb 0x562ba3cdd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3e00cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3e00cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40332/0 (socket says 172.21.15.26:40332) 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 -- 172.21.15.26:0/876642328 learned_addr learned my addr 172.21.15.26:0/876642328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e1c14700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df5260 0x562ba3dfdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2c16700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3dfe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 -- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df5260 msgr2=0x562ba3dfdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df5260 0x562ba3dfdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 -- 172.21.15.26:0/876642328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 msgr2=0x562ba3dfe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3dfe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 -- 172.21.15.26:0/876642328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba3c928d0 con 0x562ba3df48a0 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35e2415700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3e00cd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f35d8015000 tx=0x7f35d8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6064ab35e0b78915 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.670+0000 7f35c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35c5ffb700 1 -- 172.21.15.26:0/876642328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d800bb40 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35c5ffb700 1 -- 172.21.15.26:0/876642328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f35d800bce0 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35cc0a96f0 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35c5ffb700 1 -- 172.21.15.26:0/876642328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35d800f460 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 10 client.17130 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ba3cfc760 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35c5ffb700 1 -- 172.21.15.26:0/876642328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35cc0a96f0 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35c5ffb700 1 -- 172.21.15.26:0/876642328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f35d80264c0 con 0x562ba3df48a0 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 10 client.17130 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 10 client.17130 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 20 client.17130 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e0e220 0x562ba3e10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 4 client.17130 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:01:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.671+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ba3d54cc0 con 0x562ba3e0e220 2024-05-20T21:01:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.672+0000 7f35e1c14700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e0e220 0x562ba3e10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.672+0000 7f35e1c14700 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e0e220 0x562ba3e10650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b49862902e8207e server_cookie=bd5d92e29f396e0d in_seq=0 out_seq=0 2024-05-20T21:01:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.672+0000 7f35c5ffb700 10 client.17130 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35c5ffb700 1 -- 172.21.15.26:0/876642328 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562ba3d54cc0 con 0x562ba3e0e220 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35c5ffb700 10 client.17130 handle_command_reply: tid=0 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 client.17130 shutdown 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 msgr2=0x562ba3e00cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3e00cd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f35d8015000 tx=0x7f35d8009310 comp rx=0 tx=0).stop 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 shutdown_connections 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e0e220 0x562ba3e10650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df48a0 0x562ba3e00cd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3dfe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35c5ffb700 0 client.17130 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:59.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/876642328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df5260 0x562ba3dfdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 >> 172.21.15.26:0/876642328 conn(0x562ba3d20240 msgr2=0x562ba3deeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:59.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 shutdown_connections 2024-05-20T21:01:59.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 1 -- 172.21.15.26:0/876642328 wait complete. 2024-05-20T21:01:59.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.673+0000 7f35eef0fb80 20 client.17130 trim_cache size 0 max 16384 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.691+0000 7f35eef0fb80 1 Processor -- start 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.691+0000 7f35eef0fb80 1 -- start start 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.691+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3df9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.691+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df5260 0x562ba3dfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.691+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3def270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b26c0 con 0x562ba3df5260 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1aff50 con 0x562ba3dfa320 2024-05-20T21:01:59.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1adda0 con 0x562ba3df48a0 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df5260 0x562ba3dfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3df9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2c16700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3def270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df5260 0x562ba3dfcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40340/0 (socket says 172.21.15.26:40340) 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e1c14700 1 -- 172.21.15.26:0/1338519128 learned_addr learned my addr 172.21.15.26:0/1338519128 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 -- 172.21.15.26:0/1338519128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 msgr2=0x562ba3def270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3def270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 -- 172.21.15.26:0/1338519128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df5260 msgr2=0x562ba3dfcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df5260 0x562ba3dfcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 -- 172.21.15.26:0/1338519128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3dfc540 con 0x562ba3df48a0 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2c16700 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3def270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:01:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.692+0000 7f35e2415700 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3df9990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35d80098e0 tx=0x7f35d8004fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ed7b3c1eb70110 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35c7fff700 1 -- 172.21.15.26:0/1338519128 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8026430 con 0x562ba3df48a0 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35c7fff700 1 -- 172.21.15.26:0/1338519128 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d803f020 con 0x562ba3df48a0 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35c7fff700 1 -- 172.21.15.26:0/1338519128 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d800f870 con 0x562ba3df48a0 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 -- 172.21.15.26:0/1338519128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 msgr2=0x562ba3df9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3df9990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35d80098e0 tx=0x7f35d8004fd0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 -- 172.21.15.26:0/1338519128 shutdown_connections 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3dfa320 0x562ba3def270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3df9990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/1338519128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3df5260 0x562ba3dfcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 -- 172.21.15.26:0/1338519128 >> 172.21.15.26:0/1338519128 conn(0x562ba3d20240 msgr2=0x562ba3df3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.693+0000 7f35eef0fb80 1 -- 172.21.15.26:0/1338519128 shutdown_connections 2024-05-20T21:01:59.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 -- 172.21.15.26:0/1338519128 wait complete. 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 -- start start 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 Processor -- start 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3e06de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3dfa320 0x562ba3e15a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1ae410 con 0x562ba3dfa320 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1af880 con 0x562ba3df5260 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1aeae0 con 0x562ba3df48a0 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35eef0fb80 10 client.0 _ll_register_callbacks cb 0x562ba3cdd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35e2c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3dfa320 0x562ba3e15a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35e1c14700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35e2c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3dfa320 0x562ba3e15a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40342/0 (socket says 172.21.15.26:40342) 2024-05-20T21:01:59.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35e2c16700 1 -- 172.21.15.26:0/855085933 learned_addr learned my addr 172.21.15.26:0/855085933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.694+0000 7f35e2415700 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3e06de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35e1c14700 1 -- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 msgr2=0x562ba3e06de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35e1c14700 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3e06de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35e1c14700 1 -- 172.21.15.26:0/855085933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3dfa320 msgr2=0x562ba3e15a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35e1c14700 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3dfa320 0x562ba3e15a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35e1c14700 1 -- 172.21.15.26:0/855085933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba3e15fe0 con 0x562ba3df5260 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35e1c14700 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e15510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0a9680 tx=0x7f35cc091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cd79226064c015 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35c5ffb700 1 -- 172.21.15.26:0/855085933 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0b2550 con 0x562ba3df5260 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d803f020 con 0x562ba3df5260 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35c5ffb700 1 -- 172.21.15.26:0/855085933 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f35cc083e00 con 0x562ba3df5260 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35c5ffb700 1 -- 172.21.15.26:0/855085933 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35cc0adaa0 con 0x562ba3df5260 2024-05-20T21:01:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35eef0fb80 10 client.26606 fetch_fsmap learned FSMap version 28 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.695+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ba3e05740 con 0x562ba3df5260 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35c5ffb700 1 -- 172.21.15.26:0/855085933 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d803f020 con 0x562ba3df5260 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35c5ffb700 1 -- 172.21.15.26:0/855085933 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f35cc0910b0 con 0x562ba3df5260 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35eef0fb80 10 client.26606 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35eef0fb80 10 client.26606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35eef0fb80 20 client.26606 populate_metadata read hostname 'smithi026' 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e78eb0 0x562ba3e7b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35eef0fb80 4 client.26606 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:01:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ba3d4e4a0 con 0x562ba3e78eb0 2024-05-20T21:01:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.696+0000 7f35e2415700 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e78eb0 0x562ba3e7b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:01:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35e2415700 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e78eb0 0x562ba3e7b2a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=939bc0fddeca2660 server_cookie=553ba379a7e5cba5 in_seq=0 out_seq=0 2024-05-20T21:01:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35c5ffb700 10 client.26606 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35c5ffb700 1 -- 172.21.15.26:0/855085933 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ba3d4e4a0 con 0x562ba3e78eb0 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35c5ffb700 10 client.26606 handle_command_reply: tid=0 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 client.26606 shutdown 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 msgr2=0x562ba3e15510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e15510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0a9680 tx=0x7f35cc091040 comp rx=0 tx=0).stop 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 shutdown_connections 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x562ba3e78eb0 0x562ba3e7b2a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ba3dfa320 0x562ba3e15a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ba3df48a0 0x562ba3e06de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 --2- 172.21.15.26:0/855085933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ba3df5260 0x562ba3e15510 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35c5ffb700 0 client.26606 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 >> 172.21.15.26:0/855085933 conn(0x562ba3d20240 msgr2=0x562ba3e00090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 shutdown_connections 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 1 -- 172.21.15.26:0/855085933 wait complete. 2024-05-20T21:01:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.697+0000 7f35eef0fb80 20 client.26606 trim_cache size 0 max 16384 2024-05-20T21:01:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35c80b2900 msgr2=0x7f35c80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35c80b2900 0x7f35c80b4dc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b2e0 tx=0x7f35d8005fb0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 msgr2=0x7f35dc1a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 0x7f35dc1a1f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c9b0 tx=0x7f35cc00ccc0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 shutdown_connections 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f35c80b2900 0x7f35c80b4dc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35dc108510 0x7f35dc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35dc102430 0x7f35dc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 --2- 172.21.15.26:0/2496472866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc102df0 0x7f35dc1a1f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 >> 172.21.15.26:0/2496472866 conn(0x7f35dc0793b0 msgr2=0x7f35dc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:01:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 shutdown_connections 2024-05-20T21:01:59.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:01:59.698+0000 7f35e4679700 1 -- 172.21.15.26:0/2496472866 wait complete. 2024-05-20T21:02:00.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:02:00.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 106218 1.73 MB/sec execute 126 sec latency 15.944 ms 2024-05-20T21:02:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:01.162 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:00 smithi026 ceph-mon[31990]: pgmap v565: 97 pgs: 97 active+clean; 23 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-05-20T21:02:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:00 smithi144 ceph-mon[35660]: pgmap v565: 97 pgs: 97 active+clean; 23 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-05-20T21:02:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:00 smithi055 ceph-mon[35213]: pgmap v565: 97 pgs: 97 active+clean; 23 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-05-20T21:02:01.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 106769 1.75 MB/sec execute 127 sec latency 18.556 ms 2024-05-20T21:02:02.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 107241 1.74 MB/sec execute 128 sec latency 62.920 ms 2024-05-20T21:02:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 -- 172.21.15.26:0/782542878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc102df0 msgr2=0x7f78bc10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 --2- 172.21.15.26:0/782542878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc102df0 0x7f78bc10f2e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b0a0 tx=0x7f78b400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 -- 172.21.15.26:0/782542878 shutdown_connections 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 --2- 172.21.15.26:0/782542878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc102df0 0x7f78bc10f2e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 --2- 172.21.15.26:0/782542878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc108510 0x7f78bc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 --2- 172.21.15.26:0/782542878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 -- 172.21.15.26:0/782542878 >> 172.21.15.26:0/782542878 conn(0x7f78bc0793b0 msgr2=0x7f78bc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 -- 172.21.15.26:0/782542878 shutdown_connections 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 -- 172.21.15.26:0/782542878 wait complete. 2024-05-20T21:02:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.626+0000 7f78c1a87700 1 Processor -- start 2024-05-20T21:02:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 -- start start 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 0x7f78bc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc108510 0x7f78bc1a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112490 con 0x7f78bc108510 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112310 con 0x7f78bc102df0 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78c1a87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112610 con 0x7f78bc102430 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78baffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 0x7f78bc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:47792/0 (socket says 172.21.15.26:47792) 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78baffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 0x7f78bc19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33772/0 (socket says 172.21.15.26:33772) 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78bb7fe700 1 -- 172.21.15.26:0/4142467281 learned_addr learned my addr 172.21.15.26:0/4142467281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.627+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc108510 0x7f78bc1a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78bb7fe700 1 -- 172.21.15.26:0/4142467281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 msgr2=0x7f78bc19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 0x7f78bc19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78bb7fe700 1 -- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc108510 msgr2=0x7f78bc1a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc108510 0x7f78bc1a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78bb7fe700 1 -- 172.21.15.26:0/4142467281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc1a5f10 con 0x7f78bc102430 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78baffd700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 0x7f78bc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc19a950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9a0 tx=0x7f78ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78b8ff9700 1 -- 172.21.15.26:0/4142467281 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00ca10 con 0x7f78bc102430 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc1a60e0 con 0x7f78bc102430 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78b8ff9700 1 -- 172.21.15.26:0/4142467281 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78ac00cbb0 con 0x7f78bc102430 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78b8ff9700 1 -- 172.21.15.26:0/4142467281 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac018680 con 0x7f78bc102430 2024-05-20T21:02:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.628+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a6640 con 0x7f78bc102430 2024-05-20T21:02:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.632+0000 7f78b8ff9700 1 -- 172.21.15.26:0/4142467281 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f78ac010430 con 0x7f78bc102430 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.633+0000 7f78cc31db80 1 Processor -- start 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.633+0000 7f78b8ff9700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78a40b2980 0x7f78a40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.633+0000 7f78cc31db80 1 -- start start 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78b8ff9700 1 -- 172.21.15.26:0/4142467281 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f78ac0e2350 con 0x7f78bc102430 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78bbfff700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78a40b2980 0x7f78a40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037e860 0x55cf6037ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf60382940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf603833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78cc31db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf602ddc90 con 0x55cf6037e860 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78cc31db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cf602cd830 con 0x55cf60382f60 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78cc31db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf602d1870 con 0x55cf6037f210 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf603833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf603833a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33778/0 (socket says 172.21.15.26:33778) 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78bbfff700 1 -- 172.21.15.26:0/243716055 learned_addr learned my addr 172.21.15.26:0/243716055 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037e860 0x55cf6037ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.634+0000 7f78baffd700 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf60382940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bbfff700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78a40b2980 0x7f78a40b4e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f78b4015000 tx=0x7f78b400bb90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bb7fe700 1 -- 172.21.15.26:0/243716055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 msgr2=0x55cf60382940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf60382940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bb7fe700 1 -- 172.21.15.26:0/243716055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 msgr2=0x55cf603833a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf603833a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bb7fe700 1 -- 172.21.15.26:0/243716055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6021d8d0 con 0x55cf6037e860 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78baffd700 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf60382940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037e860 0x55cf6037ec40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a6b20 tx=0x7f78ac011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6832fa30b4c4d6a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78a1ffb700 1 -- 172.21.15.26:0/243716055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0a93c0 con 0x55cf6037e860 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78a1ffb700 1 -- 172.21.15.26:0/243716055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78ac0a9fb0 con 0x55cf6037e860 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.635+0000 7f78a1ffb700 1 -- 172.21.15.26:0/243716055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0b2280 con 0x55cf6037e860 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 -- 172.21.15.26:0/243716055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037e860 msgr2=0x55cf6037ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037e860 0x55cf6037ec40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a6b20 tx=0x7f78ac011ee0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 -- 172.21.15.26:0/243716055 shutdown_connections 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037e860 0x55cf6037ec40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf603833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 --2- 172.21.15.26:0/243716055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf60382940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 -- 172.21.15.26:0/243716055 >> 172.21.15.26:0/243716055 conn(0x55cf602ac080 msgr2=0x55cf60378e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 -- 172.21.15.26:0/243716055 shutdown_connections 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.636+0000 7f78cc31db80 1 -- 172.21.15.26:0/243716055 wait complete. 2024-05-20T21:02:02.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.637+0000 7f78cc31db80 1 -- start start 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.637+0000 7f78cc31db80 1 Processor -- start 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf6038a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf603877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf60382f60 0x55cf60387ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf601d3fa0 con 0x55cf60382f60 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cf6026f7c0 con 0x55cf6037e860 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf6038f530 con 0x55cf6037f210 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78cc31db80 10 client.0 _ll_register_callbacks cb 0x55cf60268920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf6038a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf60382f60 0x55cf60387ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf6038a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33780/0 (socket says 172.21.15.26:33780) 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf60382f60 0x55cf60387ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40380/0 (socket says 172.21.15.26:40380) 2024-05-20T21:02:02.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78bb7fe700 1 -- 172.21.15.26:0/3616154009 learned_addr learned my addr 172.21.15.26:0/3616154009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78baffd700 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf603877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78baffd700 1 -- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 msgr2=0x55cf6038a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78baffd700 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf6038a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78baffd700 1 -- 172.21.15.26:0/3616154009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf60382f60 msgr2=0x55cf60387ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78baffd700 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf60382f60 0x55cf60387ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.638+0000 7f78baffd700 1 -- 172.21.15.26:0/3616154009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6026c690 con 0x55cf6037f210 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f78baffd700 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf603877a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78bc06c600 tx=0x7f78b0011cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76fc587d866ccba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f789affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f789affd700 1 -- 172.21.15.26:0/3616154009 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00109b0 con 0x55cf6037f210 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f789affd700 1 -- 172.21.15.26:0/3616154009 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f78b0010b50 con 0x55cf6037f210 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78ac0a9fb0 con 0x55cf6037f210 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f789affd700 1 -- 172.21.15.26:0/3616154009 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78b0010d50 con 0x55cf6037f210 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f78cc31db80 10 client.26494 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:02.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.639+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf60399d40 con 0x55cf6037f210 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f789affd700 1 -- 172.21.15.26:0/3616154009 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78ac0a9fb0 con 0x55cf6037f210 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f789affd700 1 -- 172.21.15.26:0/3616154009 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f78b0003bb0 con 0x55cf6037f210 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78cc31db80 10 client.26494 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78cc31db80 10 client.26494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78cc31db80 20 client.26494 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78cc31db80 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf6039c3d0 0x55cf6039e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78cc31db80 4 client.26494 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6034a9e0 con 0x55cf6039c3d0 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.640+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf6039c3d0 0x55cf6039e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.641+0000 7f789affd700 10 client.26494 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:02.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.641+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf6039c3d0 0x55cf6039e7c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18e5a10fb3cc19d1 server_cookie=523d319c1614860c in_seq=0 out_seq=0 2024-05-20T21:02:02.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.641+0000 7f789affd700 1 -- 172.21.15.26:0/3616154009 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cf6034a9e0 con 0x55cf6039c3d0 2024-05-20T21:02:02.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.641+0000 7f789affd700 10 client.26494 handle_command_reply: tid=0 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 client.26494 shutdown 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 msgr2=0x55cf603877a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf603877a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78bc06c600 tx=0x7f78b0011cb0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 shutdown_connections 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf6039c3d0 0x55cf6039e7c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf60382f60 0x55cf60387ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf6038a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f789affd700 0 client.26494 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 --2- 172.21.15.26:0/3616154009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037f210 0x55cf603877a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 >> 172.21.15.26:0/3616154009 conn(0x55cf602ac080 msgr2=0x55cf60378510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 shutdown_connections 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 1 -- 172.21.15.26:0/3616154009 wait complete. 2024-05-20T21:02:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.642+0000 7f78cc31db80 20 client.26494 trim_cache size 0 max 16384 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.659+0000 7f78cc31db80 1 Processor -- start 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.659+0000 7f78cc31db80 1 -- start start 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf603896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf60378c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf60382f60 0x55cf603791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78cc31db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1ae3a0 con 0x55cf6037f210 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78cc31db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1abc30 con 0x55cf6037e860 2024-05-20T21:02:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78cc31db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1a9a80 con 0x55cf60382f60 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf60378c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf60378c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40390/0 (socket says 172.21.15.26:40390) 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 -- 172.21.15.26:0/69714859 learned_addr learned my addr 172.21.15.26:0/69714859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf603896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78bbfff700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf60382f60 0x55cf603791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 -- 172.21.15.26:0/69714859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf60382f60 msgr2=0x55cf603791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf60382f60 0x55cf603791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 -- 172.21.15.26:0/69714859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 msgr2=0x55cf603896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf603896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78baffd700 1 -- 172.21.15.26:0/69714859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf60385d10 con 0x55cf6037f210 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf603896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.660+0000 7f78bbfff700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf60382f60 0x55cf603791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.661+0000 7f78baffd700 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf60378c90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f78b001a540 tx=0x7f78b0010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0eb9c41b5513f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.661+0000 7f78a17fa700 1 -- 172.21.15.26:0/69714859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0010a70 con 0x55cf6037f210 2024-05-20T21:02:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.661+0000 7f78a17fa700 1 -- 172.21.15.26:0/69714859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78b0005580 con 0x55cf6037f210 2024-05-20T21:02:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.661+0000 7f78a17fa700 1 -- 172.21.15.26:0/69714859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b001a570 con 0x55cf6037f210 2024-05-20T21:02:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 -- 172.21.15.26:0/69714859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 msgr2=0x55cf60378c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf60378c90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f78b001a540 tx=0x7f78b0010430 comp rx=0 tx=0).stop 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 -- 172.21.15.26:0/69714859 shutdown_connections 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf60382f60 0x55cf603791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf6037e860 0x55cf603896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 --2- 172.21.15.26:0/69714859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf60378c90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 -- 172.21.15.26:0/69714859 >> 172.21.15.26:0/69714859 conn(0x55cf602ac080 msgr2=0x55cf60388fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 -- 172.21.15.26:0/69714859 shutdown_connections 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 -- 172.21.15.26:0/69714859 wait complete. 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 -- start start 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.662+0000 7f78cc31db80 1 Processor -- start 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037e860 0x55cf603ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf603ff810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf60391d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf603ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf603ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40400/0 (socket says 172.21.15.26:40400) 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037e860 0x55cf603ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037e860 0x55cf603ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:47824/0 (socket says 172.21.15.26:47824) 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf60391d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78baffd700 1 -- 172.21.15.26:0/2235753318 learned_addr learned my addr 172.21.15.26:0/2235753318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1aa0f0 con 0x55cf6037f210 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1ab560 con 0x55cf60382f60 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1aa7c0 con 0x55cf6037e860 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 10 client.0 _ll_register_callbacks cb 0x55cf60268920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bbfff700 1 -- 172.21.15.26:0/2235753318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037e860 msgr2=0x55cf603ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bbfff700 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037e860 0x55cf603ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bbfff700 1 -- 172.21.15.26:0/2235753318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 msgr2=0x55cf603ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bbfff700 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf603ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78bbfff700 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf60392440 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78baffd700 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf603ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.663+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b0005580 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f78bbfff700 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf60391d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55cf6037f150 tx=0x7f78b4023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12945a6dc928c282 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f789affd700 10 client.26624 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f789affd700 1 -- 172.21.15.26:0/2235753318 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4039050 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f789affd700 1 -- 172.21.15.26:0/2235753318 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f78b4007cd0 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f789affd700 1 -- 172.21.15.26:0/2235753318 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78b40050c0 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f78cc31db80 10 client.26624 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.664+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55cf604069c0 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f789affd700 1 -- 172.21.15.26:0/2235753318 <== mon.2 v2:172.21.15.55:3300/0 4 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f78b4011070 con 0x55cf60382f60 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 10 client.26624 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 10 client.26624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 20 client.26624 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf60406fb0 0x55cf604093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 4 client.26624 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf602dfbe0 con 0x55cf60406fb0 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf60406fb0 0x55cf604093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf60406fb0 0x55cf604093d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2970ba992a12e908 server_cookie=c94b1ea9206ab6d1 in_seq=0 out_seq=0 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f789affd700 10 client.26624 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f789affd700 1 -- 172.21.15.26:0/2235753318 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf602dfbe0 con 0x55cf60406fb0 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f789affd700 10 client.26624 handle_command_reply: tid=0 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.665+0000 7f78cc31db80 1 client.26624 shutdown 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 msgr2=0x55cf60391d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf60391d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55cf6037f150 tx=0x7f78b4023040 comp rx=0 tx=0).stop 2024-05-20T21:02:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 shutdown_connections 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55cf60406fb0 0x55cf604093d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cf60382f60 0x55cf60391d70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf6037e860 0x55cf603ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 --2- 172.21.15.26:0/2235753318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf6037f210 0x55cf603ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 >> 172.21.15.26:0/2235753318 conn(0x55cf602ac080 msgr2=0x55cf6037df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f789affd700 0 client.26624 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 shutdown_connections 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 1 -- 172.21.15.26:0/2235753318 wait complete. 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.666+0000 7f78cc31db80 20 client.26624 trim_cache size 0 max 16384 2024-05-20T21:02:02.671 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78a40b2980 msgr2=0x7f78a40b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78a40b2980 0x7f78a40b4e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f78b4015000 tx=0x7f78b400bb90 comp rx=0 tx=0).stop 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 msgr2=0x7f78bc19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc19a950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9a0 tx=0x7f78ac00ecb0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 shutdown_connections 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f78a40b2980 0x7f78a40b4e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc108510 0x7f78bc1a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78bc102430 0x7f78bc19a950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 --2- 172.21.15.26:0/4142467281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78bc102df0 0x7f78bc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:02.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 >> 172.21.15.26:0/4142467281 conn(0x7f78bc0793b0 msgr2=0x7f78bc100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:02.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 shutdown_connections 2024-05-20T21:02:02.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:02.667+0000 7f78c1a87700 1 -- 172.21.15.26:0/4142467281 wait complete. 2024-05-20T21:02:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:02 smithi144 ceph-mon[35660]: pgmap v566: 97 pgs: 97 active+clean; 24 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 198 KiB/s wr, 16 op/s 2024-05-20T21:02:03.363 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:02 smithi026 ceph-mon[31990]: pgmap v566: 97 pgs: 97 active+clean; 24 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 198 KiB/s wr, 16 op/s 2024-05-20T21:02:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:02 smithi055 ceph-mon[35213]: pgmap v566: 97 pgs: 97 active+clean; 24 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 198 KiB/s wr, 16 op/s 2024-05-20T21:02:03.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:02:03.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 107939 1.76 MB/sec execute 129 sec latency 15.956 ms 2024-05-20T21:02:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:04.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 108599 1.75 MB/sec execute 130 sec latency 18.336 ms 2024-05-20T21:02:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:04 smithi144 ceph-mon[35660]: pgmap v567: 97 pgs: 97 active+clean; 27 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 582 KiB/s wr, 41 op/s 2024-05-20T21:02:05.329 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:04 smithi026 ceph-mon[31990]: pgmap v567: 97 pgs: 97 active+clean; 27 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 582 KiB/s wr, 41 op/s 2024-05-20T21:02:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:04 smithi055 ceph-mon[35213]: pgmap v567: 97 pgs: 97 active+clean; 27 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 582 KiB/s wr, 41 op/s 2024-05-20T21:02:05.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 109372 1.79 MB/sec execute 131 sec latency 19.618 ms 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 -- 172.21.15.26:0/3835914921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c60103fa0 msgr2=0x7f1c60110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 --2- 172.21.15.26:0/3835914921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c60103fa0 0x7f1c60110450 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b0d0 tx=0x7f1c5800b3e0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 -- 172.21.15.26:0/3835914921 shutdown_connections 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 --2- 172.21.15.26:0/3835914921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c60103fa0 0x7f1c60110450 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 --2- 172.21.15.26:0/3835914921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60109680 0x7f1c60109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 --2- 172.21.15.26:0/3835914921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c601035e0 0x7f1c60103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.708+0000 7f1c65519700 1 -- 172.21.15.26:0/3835914921 >> 172.21.15.26:0/3835914921 conn(0x7f1c6007a6f0 msgr2=0x7f1c6007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.709+0000 7f1c65519700 1 -- 172.21.15.26:0/3835914921 shutdown_connections 2024-05-20T21:02:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.709+0000 7f1c65519700 1 -- 172.21.15.26:0/3835914921 wait complete. 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.709+0000 7f1c65519700 1 Processor -- start 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.709+0000 7f1c65519700 1 -- start start 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c65519700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 0x7f1c60075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c65519700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60103fa0 0x7f1c60075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c65519700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60109680 0x7f1c600762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c65519700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60113650 con 0x7f1c601035e0 2024-05-20T21:02:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c65519700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601134d0 con 0x7f1c60103fa0 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c65519700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601137d0 con 0x7f1c60109680 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 0x7f1c60075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 0x7f1c60075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40426/0 (socket says 172.21.15.26:40426) 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 -- 172.21.15.26:0/1460114253 learned_addr learned my addr 172.21.15.26:0/1460114253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60103fa0 0x7f1c60075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 -- 172.21.15.26:0/1460114253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60109680 msgr2=0x7f1c600762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60109680 0x7f1c600762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 -- 172.21.15.26:0/1460114253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60103fa0 msgr2=0x7f1c60075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60103fa0 0x7f1c60075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 -- 172.21.15.26:0/1460114253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c601a8000 con 0x7f1c601035e0 2024-05-20T21:02:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.710+0000 7f1c5effd700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 0x7f1c60075840 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000d950 tx=0x7f1c5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.711+0000 7f1c47fff700 1 -- 172.21.15.26:0/1460114253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50016070 con 0x7f1c601035e0 2024-05-20T21:02:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.711+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c601a82f0 con 0x7f1c601035e0 2024-05-20T21:02:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.711+0000 7f1c47fff700 1 -- 172.21.15.26:0/1460114253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c50009dd0 con 0x7f1c601035e0 2024-05-20T21:02:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.711+0000 7f1c47fff700 1 -- 172.21.15.26:0/1460114253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000f3c0 con 0x7f1c601035e0 2024-05-20T21:02:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.711+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c601a8880 con 0x7f1c601035e0 2024-05-20T21:02:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.713+0000 7f1c47fff700 1 -- 172.21.15.26:0/1460114253 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1c50019070 con 0x7f1c601035e0 2024-05-20T21:02:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.714+0000 7f1c47fff700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c480b2880 0x7f1c480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.714+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c480b2880 0x7f1c480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.714+0000 7f1c47fff700 1 -- 172.21.15.26:0/1460114253 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1c500e1e50 con 0x7f1c601035e0 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c480b2880 0x7f1c480b4d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1c6007add0 tx=0x7f1c5400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 Processor -- start 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 -- start start 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b6c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562102ac6c90 con 0x562102b67860 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562102ab6830 con 0x562102b68210 2024-05-20T21:02:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562102aba870 con 0x562102b6bf60 2024-05-20T21:02:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c5e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c5e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b6b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33814/0 (socket says 172.21.15.26:33814) 2024-05-20T21:02:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.715+0000 7f1c5e7fc700 1 -- 172.21.15.26:0/2900674387 learned_addr learned my addr 172.21.15.26:0/2900674387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5f7fe700 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b6c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5e7fc700 1 -- 172.21.15.26:0/2900674387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 msgr2=0x562102b6c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b6c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5e7fc700 1 -- 172.21.15.26:0/2900674387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 msgr2=0x562102b67c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b67c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5e7fc700 1 -- 172.21.15.26:0/2900674387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562102a068d0 con 0x562102b68210 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5f7fe700 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b6c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b6b940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c54003aa0 tx=0x7f1c54013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e06ac2c7809e0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c457fa700 1 -- 172.21.15.26:0/2900674387 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c540118d0 con 0x562102b68210 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c457fa700 1 -- 172.21.15.26:0/2900674387 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c5401e420 con 0x562102b68210 2024-05-20T21:02:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.716+0000 7f1c457fa700 1 -- 172.21.15.26:0/2900674387 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5401c610 con 0x562102b68210 2024-05-20T21:02:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2900674387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 msgr2=0x562102b6b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b6b940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c54003aa0 tx=0x7f1c54013000 comp rx=0 tx=0).stop 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2900674387 shutdown_connections 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b67c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b6c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2900674387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b6b940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2900674387 >> 172.21.15.26:0/2900674387 conn(0x562102a95080 msgr2=0x562102b61e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.717+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2900674387 shutdown_connections 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.718+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2900674387 wait complete. 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.718+0000 7f1c6fdafb80 1 -- start start 2024-05-20T21:02:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.718+0000 7f1c6fdafb80 1 Processor -- start 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.718+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.718+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.718+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b70c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5621029bcfa0 con 0x562102b67860 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562102a587c0 con 0x562102b68210 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562102b74120 con 0x562102b6bf60 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40444/0 (socket says 172.21.15.26:40444) 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 10 client.0 _ll_register_callbacks cb 0x562102a51920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 -- 172.21.15.26:0/3709075612 learned_addr learned my addr 172.21.15.26:0/3709075612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:05.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5f7fe700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b70c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 -- 172.21.15.26:0/3709075612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 msgr2=0x562102b70c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b70c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 -- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 msgr2=0x562102b706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 -- 172.21.15.26:0/3709075612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562102a55690 con 0x562102b67860 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c5effd700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b733e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b2f0 tx=0x7f1c500a9c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530cd7f9143c34c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3709075612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50020070 con 0x562102b67860 2024-05-20T21:02:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3709075612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1c500b1de0 con 0x562102b67860 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5401e420 con 0x562102b67860 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3709075612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c500b2270 con 0x562102b67860 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 10 client.17166 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.719+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562102b7e850 con 0x562102b67860 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3709075612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c5401e420 con 0x562102b67860 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3709075612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f1c500a3b30 con 0x562102b67860 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c6fdafb80 10 client.17166 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c6fdafb80 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c6fdafb80 20 client.17166 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102b80e30 0x562102b83260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c6fdafb80 4 client.17166 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562102b339e0 con 0x562102b80e30 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.720+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102b80e30 0x562102b83260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.721+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102b80e30 0x562102b83260 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f89a9720f81fb32 server_cookie=ff8f16f7982163f7 in_seq=0 out_seq=0 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.721+0000 7f1c42ff5700 10 client.17166 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.721+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3709075612 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562102b339e0 con 0x562102b80e30 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.721+0000 7f1c42ff5700 10 client.17166 handle_command_reply: tid=0 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.721+0000 7f1c6fdafb80 1 client.17166 shutdown 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 msgr2=0x562102b733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b733e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b2f0 tx=0x7f1c500a9c40 comp rx=0 tx=0).stop 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 shutdown_connections 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102b80e30 0x562102b83260 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b67860 0x562102b733e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b70c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c42ff5700 0 client.17166 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3709075612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b68210 0x562102b706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 >> 172.21.15.26:0/3709075612 conn(0x562102a95080 msgr2=0x562102b6d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 shutdown_connections 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3709075612 wait complete. 2024-05-20T21:02:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.722+0000 7f1c6fdafb80 20 client.17166 trim_cache size 0 max 16384 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 Processor -- start 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 -- start start 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b88b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601b28c0 con 0x562102b68210 2024-05-20T21:02:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601b0150 con 0x562102b67860 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601adfa0 con 0x562102b6bf60 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c5e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.740+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40458/0 (socket says 172.21.15.26:40458) 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5e7fc700 1 -- 172.21.15.26:0/2205512739 learned_addr learned my addr 172.21.15.26:0/2205512739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5f7fe700 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b88b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5effd700 1 -- 172.21.15.26:0/2205512739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 msgr2=0x562102b88b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5effd700 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b88b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5effd700 1 -- 172.21.15.26:0/2205512739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 msgr2=0x562102b72620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5effd700 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b72620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5effd700 1 -- 172.21.15.26:0/2205512739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562102b63520 con 0x562102b67860 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b72620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c5effd700 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b610e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c500a5490 tx=0x7f1c50091160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2805bbe380ce669d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:05.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c44ff9700 1 -- 172.21.15.26:0/2205512739 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50020070 con 0x562102b67860 2024-05-20T21:02:05.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c44ff9700 1 -- 172.21.15.26:0/2205512739 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c500a3b30 con 0x562102b67860 2024-05-20T21:02:05.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.741+0000 7f1c44ff9700 1 -- 172.21.15.26:0/2205512739 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500b2020 con 0x562102b67860 2024-05-20T21:02:05.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2205512739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 msgr2=0x562102b610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b610e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c500a5490 tx=0x7f1c50091160 comp rx=0 tx=0).stop 2024-05-20T21:02:05.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2205512739 shutdown_connections 2024-05-20T21:02:05.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102b88b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b610e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/2205512739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b72620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2205512739 >> 172.21.15.26:0/2205512739 conn(0x562102a95080 msgr2=0x562102b71f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2205512739 shutdown_connections 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.742+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/2205512739 wait complete. 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 -- start start 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 Processor -- start 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b76960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102be9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601ae610 con 0x562102b68210 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601afa80 con 0x562102b67860 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601aece0 con 0x562102b6bf60 2024-05-20T21:02:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c6fdafb80 10 client.0 _ll_register_callbacks cb 0x562102a51920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b76960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b76960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33844/0 (socket says 172.21.15.26:33844) 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c5effd700 1 -- 172.21.15.26:0/3152737868 learned_addr learned my addr 172.21.15.26:0/3152737868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.743+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5f7fe700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102be9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5effd700 1 -- 172.21.15.26:0/3152737868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 msgr2=0x562102be9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5effd700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102be9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5effd700 1 -- 172.21.15.26:0/3152737868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 msgr2=0x562102b76ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5effd700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b76ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5effd700 1 -- 172.21.15.26:0/3152737868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562102be9f90 con 0x562102b67860 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5f7fe700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102be9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c5effd700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b76960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c500c0040 tx=0x7f1c500a88a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d499c544a4687ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3152737868 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500c7070 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3152737868 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f1c50009b50 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c500a3b30 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3152737868 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c50020070 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c6fdafb80 10 client.26642 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.744+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562102beb250 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3152737868 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c500a3b30 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3152737868 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f1c500ca020 con 0x562102b67860 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c6fdafb80 10 client.26642 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c6fdafb80 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c6fdafb80 20 client.26642 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102beb6c0 0x562102bedae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c6fdafb80 4 client.26642 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:02:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562102ac8be0 con 0x562102beb6c0 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102beb6c0 0x562102bedae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c5e7fc700 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102beb6c0 0x562102bedae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849f3d72527b6c88 server_cookie=c40d57c6ffc98e1d in_seq=0 out_seq=0 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.745+0000 7f1c42ff5700 10 client.26642 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c42ff5700 1 -- 172.21.15.26:0/3152737868 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562102ac8be0 con 0x562102beb6c0 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c42ff5700 10 client.26642 handle_command_reply: tid=0 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 client.26642 shutdown 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 msgr2=0x562102b76960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b76960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c500c0040 tx=0x7f1c500a88a0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 shutdown_connections 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562102beb6c0 0x562102bedae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562102b6bf60 0x562102be9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562102b67860 0x562102b76960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c42ff5700 0 client.26642 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 --2- 172.21.15.26:0/3152737868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562102b68210 0x562102b76ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 >> 172.21.15.26:0/3152737868 conn(0x562102a95080 msgr2=0x562102b6f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 shutdown_connections 2024-05-20T21:02:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 1 -- 172.21.15.26:0/3152737868 wait complete. 2024-05-20T21:02:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.746+0000 7f1c6fdafb80 20 client.26642 trim_cache size 0 max 16384 2024-05-20T21:02:05.751 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c480b2880 msgr2=0x7f1c480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c480b2880 0x7f1c480b4d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1c6007add0 tx=0x7f1c5400b3c0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 msgr2=0x7f1c60075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 0x7f1c60075840 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000d950 tx=0x7f1c5000dc60 comp rx=0 tx=0).stop 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 shutdown_connections 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1c480b2880 0x7f1c480b4d40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c601035e0 0x7f1c60075840 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60109680 0x7f1c600762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 --2- 172.21.15.26:0/1460114253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60103fa0 0x7f1c60075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:05.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 >> 172.21.15.26:0/1460114253 conn(0x7f1c6007a6f0 msgr2=0x7f1c601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:05.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 shutdown_connections 2024-05-20T21:02:05.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:05.747+0000 7f1c65519700 1 -- 172.21.15.26:0/1460114253 wait complete. 2024-05-20T21:02:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 -- 172.21.15.26:0/3842584288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074105e60 msgr2=0x7ff074108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 --2- 172.21.15.26:0/3842584288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074105e60 0x7ff074108250 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009900 tx=0x7ff05c02a410 comp rx=0 tx=0).stop 2024-05-20T21:02:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 -- 172.21.15.26:0/3842584288 shutdown_connections 2024-05-20T21:02:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 --2- 172.21.15.26:0/3842584288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074108790 0x7ff07410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 --2- 172.21.15.26:0/3842584288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074105e60 0x7ff074108250 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 --2- 172.21.15.26:0/3842584288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff074103530 0x7ff074105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 -- 172.21.15.26:0/3842584288 >> 172.21.15.26:0/3842584288 conn(0x7ff0740fcf20 msgr2=0x7ff0740ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 -- 172.21.15.26:0/3842584288 shutdown_connections 2024-05-20T21:02:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.538+0000 7ff078a15640 1 -- 172.21.15.26:0/3842584288 wait complete. 2024-05-20T21:02:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 Processor -- start 2024-05-20T21:02:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 -- start start 2024-05-20T21:02:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 0x7ff074102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff074105e60 0x7ff0741014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074108790 0x7ff074101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff07410e4a0 con 0x7ff074108790 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff07410e320 con 0x7ff074103530 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff078a15640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff07410e620 con 0x7ff074105e60 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 0x7ff074102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff072d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074108790 0x7ff074101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 0x7ff074102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33870/0 (socket says 172.21.15.26:33870) 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 -- 172.21.15.26:0/3568543414 learned_addr learned my addr 172.21.15.26:0/3568543414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff06bfff640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff074105e60 0x7ff0741014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 -- 172.21.15.26:0/3568543414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff074105e60 msgr2=0x7ff0741014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff074105e60 0x7ff0741014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 -- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074108790 msgr2=0x7ff074101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.539+0000 7ff07259c640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074108790 0x7ff074101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff07259c640 1 -- 172.21.15.26:0/3568543414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0741022b0 con 0x7ff074103530 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff07259c640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 0x7ff074102ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff06400d770 tx=0x7ff06400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff06b7fe640 1 -- 172.21.15.26:0/3568543414 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064017070 con 0x7ff074103530 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0741a2cc0 con 0x7ff074103530 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff06b7fe640 1 -- 172.21.15.26:0/3568543414 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0640054b0 con 0x7ff074103530 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff06b7fe640 1 -- 172.21.15.26:0/3568543414 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064004c60 con 0x7ff074103530 2024-05-20T21:02:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.540+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0741a3190 con 0x7ff074103530 2024-05-20T21:02:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff06b7fe640 1 -- 172.21.15.26:0/3568543414 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff064004e00 con 0x7ff074103530 2024-05-20T21:02:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 Processor -- start 2024-05-20T21:02:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 -- start start 2024-05-20T21:02:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add09ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0a0b0 0x5619add0ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 0x5619add116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5619adc7ac30 con 0x5619add0f220 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5619adc6a7d0 con 0x5619add0a0b0 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.543+0000 7ff07b806740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5619adc6e810 con 0x5619add096c0 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 0x5619add116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff07259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0a0b0 0x5619add0ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 0x5619add116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40514/0 (socket says 172.21.15.26:40514) 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff06bfff640 1 -- 172.21.15.26:0/118346733 learned_addr learned my addr 172.21.15.26:0/118346733 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add09ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff06b7fe640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0600b28a0 0x7ff0600b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 -- 172.21.15.26:0/118346733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0a0b0 msgr2=0x5619add0ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0a0b0 0x5619add0ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 -- 172.21.15.26:0/118346733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 msgr2=0x5619add116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 0x5619add116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 -- 172.21.15.26:0/118346733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ad97f210 con 0x5619add096c0 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff06bfff640 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 0x5619add116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0600b28a0 0x7ff0600b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff06b7fe640 1 -- 172.21.15.26:0/3568543414 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7ff0640e1c10 con 0x7ff074103530 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff072d9d640 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add09ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009fd0 tx=0x7ff06c011920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed2ec68f7f3cb46 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.544+0000 7ff068ff9640 1 -- 172.21.15.26:0/118346733 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c017070 con 0x5619add096c0 2024-05-20T21:02:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff072d9d640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0600b28a0 0x7ff0600b4d60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff06c022000 tx=0x7ff06c023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff068ff9640 1 -- 172.21.15.26:0/118346733 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff06c00fc90 con 0x5619add096c0 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff068ff9640 1 -- 172.21.15.26:0/118346733 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00e950 con 0x5619add096c0 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 -- 172.21.15.26:0/118346733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 msgr2=0x5619add09ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add09ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009fd0 tx=0x7ff06c011920 comp rx=0 tx=0).stop 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 -- 172.21.15.26:0/118346733 shutdown_connections 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0f220 0x5619add116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0a0b0 0x5619add0ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 --2- 172.21.15.26:0/118346733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add09ae0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.545+0000 7ff07b806740 1 -- 172.21.15.26:0/118346733 >> 172.21.15.26:0/118346733 conn(0x5619add04d30 msgr2=0x5619add05130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 -- 172.21.15.26:0/118346733 shutdown_connections 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 -- 172.21.15.26:0/118346733 wait complete. 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 -- start start 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 Processor -- start 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add13810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add13d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5619ad950790 con 0x5619add0a0b0 2024-05-20T21:02:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5619adc26890 con 0x5619add0f220 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5619add121d0 con 0x5619add096c0 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07b806740 10 client.0 _ll_register_callbacks cb 0x5619ad97ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add13d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add13d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40522/0 (socket says 172.21.15.26:40522) 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff072d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff06bfff640 1 -- 172.21.15.26:0/3213334302 learned_addr learned my addr 172.21.15.26:0/3213334302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.546+0000 7ff07259c640 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add13810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff06bfff640 1 -- 172.21.15.26:0/3213334302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 msgr2=0x5619add13810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff06bfff640 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add13810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff06bfff640 1 -- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 msgr2=0x5619add1ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff06bfff640 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add1ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff06bfff640 1 -- 172.21.15.26:0/3213334302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ada4b620 con 0x5619add0a0b0 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff06bfff640 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add13d50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009230 tx=0x7ff05c031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51ac8d66e86880d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff059ffb640 1 -- 172.21.15.26:0/3213334302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035040 con 0x5619add0a0b0 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff059ffb640 1 -- 172.21.15.26:0/3213334302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7ff05c004240 con 0x5619add0a0b0 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c00fc90 con 0x5619add0a0b0 2024-05-20T21:02:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff059ffb640 1 -- 172.21.15.26:0/3213334302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff05c004440 con 0x5619add0a0b0 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff07b806740 10 client.17184 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.547+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619add1dde0 con 0x5619add0a0b0 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff059ffb640 1 -- 172.21.15.26:0/3213334302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff06c00fc90 con 0x5619add0a0b0 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff059ffb640 1 -- 172.21.15.26:0/3213334302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7ff05c03c5d0 con 0x5619add0a0b0 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07b806740 10 client.17184 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07b806740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07b806740 20 client.17184 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07b806740 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add1f540 0x5619add21950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07b806740 4 client.17184 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619adc7cb80 con 0x5619add1f540 2024-05-20T21:02:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07259c640 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add1f540 0x5619add21950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff07259c640 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add1f540 0x5619add21950 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2523febba7c23e18 server_cookie=7d43d19469db5b29 in_seq=0 out_seq=0 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.548+0000 7ff059ffb640 10 client.17184 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.549+0000 7ff059ffb640 1 -- 172.21.15.26:0/3213334302 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5619adc7cb80 con 0x5619add1f540 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff059ffb640 10 client.17184 handle_command_reply: tid=0 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 client.17184 shutdown 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 msgr2=0x5619add13d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add13d50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009230 tx=0x7ff05c031870 comp rx=0 tx=0).stop 2024-05-20T21:02:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 shutdown_connections 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add1f540 0x5619add21950 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add1ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add13d50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 --2- 172.21.15.26:0/3213334302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add13810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 >> 172.21.15.26:0/3213334302 conn(0x5619add04d30 msgr2=0x5619add03fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff059ffb640 0 client.17184 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 shutdown_connections 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 1 -- 172.21.15.26:0/3213334302 wait complete. 2024-05-20T21:02:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.550+0000 7ff07b806740 20 client.17184 trim_cache size 0 max 16384 2024-05-20T21:02:06.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 110006 1.82 MB/sec execute 132 sec latency 17.879 ms 2024-05-20T21:02:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 Processor -- start 2024-05-20T21:02:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 -- start start 2024-05-20T21:02:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add08390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add1ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add27260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741ae9e0 con 0x5619add0a0b0 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741ac2b0 con 0x5619add0f220 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07b806740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741aa100 con 0x5619add096c0 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add1ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add1ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40532/0 (socket says 172.21.15.26:40532) 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff06bfff640 1 -- 172.21.15.26:0/1107668494 learned_addr learned my addr 172.21.15.26:0/1107668494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff072d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add27260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.567+0000 7ff07259c640 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add08390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff06bfff640 1 -- 172.21.15.26:0/1107668494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 msgr2=0x5619add08390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff06bfff640 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add08390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff06bfff640 1 -- 172.21.15.26:0/1107668494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 msgr2=0x5619add27260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff06bfff640 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add27260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff06bfff640 1 -- 172.21.15.26:0/1107668494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619add0e640 con 0x5619add0a0b0 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff06bfff640 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add1ad50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02a930 tx=0x7ff05c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3caf16cd2473c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff05bfff640 1 -- 172.21.15.26:0/1107668494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c000e60 con 0x5619add0a0b0 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff05bfff640 1 -- 172.21.15.26:0/1107668494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff05c03cc00 con 0x5619add0a0b0 2024-05-20T21:02:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.568+0000 7ff05bfff640 1 -- 172.21.15.26:0/1107668494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035040 con 0x5619add0a0b0 2024-05-20T21:02:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 -- 172.21.15.26:0/1107668494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 msgr2=0x5619add1ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add1ad50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02a930 tx=0x7ff05c02a410 comp rx=0 tx=0).stop 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 -- 172.21.15.26:0/1107668494 shutdown_connections 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add27260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add1ad50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 --2- 172.21.15.26:0/1107668494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add08390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 -- 172.21.15.26:0/1107668494 >> 172.21.15.26:0/1107668494 conn(0x5619add04d30 msgr2=0x5619add07c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 -- 172.21.15.26:0/1107668494 shutdown_connections 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 -- 172.21.15.26:0/1107668494 wait complete. 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 -- start start 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.569+0000 7ff07b806740 1 Processor -- start 2024-05-20T21:02:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add18d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741aa770 con 0x5619add0a0b0 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741abbe0 con 0x5619add0f220 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741aae40 con 0x5619add096c0 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 10 client.0 _ll_register_callbacks cb 0x7ff05c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add18d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff072d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add18d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40534/0 (socket says 172.21.15.26:40534) 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 -- 172.21.15.26:0/2461152018 learned_addr learned my addr 172.21.15.26:0/2461152018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07259c640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 -- 172.21.15.26:0/2461152018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 msgr2=0x5619add187e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add187e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 -- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 msgr2=0x5619add19cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add19cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 -- 172.21.15.26:0/2461152018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619add1a2b0 con 0x5619add0a0b0 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07259c640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff072d9d640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff06bfff640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add18d20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0096c0 tx=0x7ff05c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ff5e2167008e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff059ffb640 1 -- 172.21.15.26:0/2461152018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035040 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff059ffb640 1 -- 172.21.15.26:0/2461152018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7ff05c03c640 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.570+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c03cc00 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff059ffb640 1 -- 172.21.15.26:0/2461152018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff05c0467f0 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 10 client.17190 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619add8a410 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff059ffb640 1 -- 172.21.15.26:0/2461152018 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff05c03cc00 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff059ffb640 1 -- 172.21.15.26:0/2461152018 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7ff05c05b990 con 0x5619add0a0b0 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 10 client.17190 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 20 client.17190 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add8ad90 0x5619add8d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 4 client.17190 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:02:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619add13600 con 0x5619add8ad90 2024-05-20T21:02:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.571+0000 7ff07259c640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add8ad90 0x5619add8d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.572+0000 7ff07259c640 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add8ad90 0x5619add8d1b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f795e6f146a77cc server_cookie=93e70e8e7d3f83f2 in_seq=0 out_seq=0 2024-05-20T21:02:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.572+0000 7ff059ffb640 10 client.17190 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff059ffb640 1 -- 172.21.15.26:0/2461152018 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5619add13600 con 0x5619add8ad90 2024-05-20T21:02:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff059ffb640 10 client.17190 handle_command_reply: tid=0 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 client.17190 shutdown 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 msgr2=0x5619add18d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add18d20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0096c0 tx=0x7ff05c005b50 comp rx=0 tx=0).stop 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 shutdown_connections 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5619add8ad90 0x5619add8d1b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5619add0f220 0x5619add19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619add0a0b0 0x5619add18d20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 --2- 172.21.15.26:0/2461152018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619add096c0 0x5619add187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff059ffb640 0 client.17190 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 >> 172.21.15.26:0/2461152018 conn(0x5619add04d30 msgr2=0x5619add03720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 shutdown_connections 2024-05-20T21:02:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 1 -- 172.21.15.26:0/2461152018 wait complete. 2024-05-20T21:02:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.617+0000 7ff07b806740 20 client.17190 trim_cache size 0 max 16384 2024-05-20T21:02:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:02:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:02:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:02:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0600b28a0 msgr2=0x7ff0600b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0600b28a0 0x7ff0600b4d60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff06c022000 tx=0x7ff06c023000 comp rx=0 tx=0).stop 2024-05-20T21:02:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 msgr2=0x7ff074102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 0x7ff074102ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff06400d770 tx=0x7ff06400dc30 comp rx=0 tx=0).stop 2024-05-20T21:02:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 shutdown_connections 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0600b28a0 0x7ff0600b4d60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff074108790 0x7ff074101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff074105e60 0x7ff0741014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 --2- 172.21.15.26:0/3568543414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff074103530 0x7ff074102ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 >> 172.21.15.26:0/3568543414 conn(0x7ff0740fcf20 msgr2=0x7ff074105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 shutdown_connections 2024-05-20T21:02:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:06.618+0000 7ff078a15640 1 -- 172.21.15.26:0/3568543414 wait complete. 2024-05-20T21:02:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:06 smithi144 ceph-mon[35660]: pgmap v568: 97 pgs: 97 active+clean; 27 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 845 KiB/s wr, 45 op/s 2024-05-20T21:02:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:06 smithi055 ceph-mon[35213]: pgmap v568: 97 pgs: 97 active+clean; 27 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 845 KiB/s wr, 45 op/s 2024-05-20T21:02:07.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:06 smithi026 ceph-mon[31990]: pgmap v568: 97 pgs: 97 active+clean; 27 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 845 KiB/s wr, 45 op/s 2024-05-20T21:02:07.520 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:02:07.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:02:07.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 110468 1.82 MB/sec execute 133 sec latency 38.349 ms 2024-05-20T21:02:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:08.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 111064 1.82 MB/sec execute 134 sec latency 18.600 ms 2024-05-20T21:02:09.216 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:08 smithi026 ceph-mon[31990]: pgmap v569: 97 pgs: 97 active+clean; 26 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 988 KiB/s wr, 59 op/s 2024-05-20T21:02:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:08 smithi144 ceph-mon[35660]: pgmap v569: 97 pgs: 97 active+clean; 26 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 988 KiB/s wr, 59 op/s 2024-05-20T21:02:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:08 smithi055 ceph-mon[35213]: pgmap v569: 97 pgs: 97 active+clean; 26 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 988 KiB/s wr, 59 op/s 2024-05-20T21:02:09.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 111695 1.83 MB/sec execute 135 sec latency 35.076 ms 2024-05-20T21:02:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.658+0000 7fa4aa7fc700 1 -- 172.21.15.26:0/1563675490 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000e3e0 con 0x7fa4ac111d20 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.658+0000 7fa4b2171700 1 -- 172.21.15.26:0/1563675490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 msgr2=0x7fa4ac10a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.658+0000 7fa4b2171700 1 --2- 172.21.15.26:0/1563675490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac10a540 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00098f0 tx=0x7fa4a0009c00 comp rx=0 tx=0).stop 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 -- 172.21.15.26:0/1563675490 shutdown_connections 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 --2- 172.21.15.26:0/1563675490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac11f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 --2- 172.21.15.26:0/1563675490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac10a540 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 --2- 172.21.15.26:0/1563675490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ac10aa80 0x7fa4ac10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 -- 172.21.15.26:0/1563675490 >> 172.21.15.26:0/1563675490 conn(0x7fa4ac06e6b0 msgr2=0x7fa4ac06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 -- 172.21.15.26:0/1563675490 shutdown_connections 2024-05-20T21:02:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.659+0000 7fa4b2171700 1 -- 172.21.15.26:0/1563675490 wait complete. 2024-05-20T21:02:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.660+0000 7fa4b2171700 1 Processor -- start 2024-05-20T21:02:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 -- start start 2024-05-20T21:02:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ac10aa80 0x7fa4ac1ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac1aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac1a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1224e0 con 0x7fa4ac10aa80 2024-05-20T21:02:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac122360 con 0x7fa4ac11cd20 2024-05-20T21:02:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.661+0000 7fa4b2171700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac122660 con 0x7fa4ac111d20 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac1a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4aaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac1aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac1a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33930/0 (socket says 172.21.15.26:33930) 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 -- 172.21.15.26:0/2449611417 learned_addr learned my addr 172.21.15.26:0/2449611417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ac10aa80 0x7fa4ac1ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 -- 172.21.15.26:0/2449611417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 msgr2=0x7fa4ac1aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac1aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 -- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ac10aa80 msgr2=0x7fa4ac1ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ac10aa80 0x7fa4ac1ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 -- 172.21.15.26:0/2449611417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac1a9020 con 0x7fa4ac11cd20 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac1aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.662+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac1a88c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400ef90 tx=0x7fa4a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.663+0000 7fa4a8ff9700 1 -- 172.21.15.26:0/2449611417 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40092e0 con 0x7fa4ac11cd20 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.663+0000 7fa4a8ff9700 1 -- 172.21.15.26:0/2449611417 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a4004bb0 con 0x7fa4ac11cd20 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.663+0000 7fa4a8ff9700 1 -- 172.21.15.26:0/2449611417 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4004020 con 0x7fa4ac11cd20 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.663+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac1a92b0 con 0x7fa4ac11cd20 2024-05-20T21:02:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.663+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac1b55d0 con 0x7fa4ac11cd20 2024-05-20T21:02:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.666+0000 7fa4a8ff9700 1 -- 172.21.15.26:0/2449611417 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa4a401b030 con 0x7fa4ac11cd20 2024-05-20T21:02:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.667+0000 7fa4a8ff9700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4940b2900 0x7fa4940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.667+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4940b2900 0x7fa4940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.667+0000 7fa4a8ff9700 1 -- 172.21.15.26:0/2449611417 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa4a40e1bf0 con 0x7fa4ac11cd20 2024-05-20T21:02:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.667+0000 7fa4bca07b80 1 Processor -- start 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 -- start start 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4940b2900 0x7fa4940b4dc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0000c00 tx=0x7fa4a0005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c068e0 0x558f64c06cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c0a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 0x558f64c0b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558f64b65d10 con 0x558f64c0afe0 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558f64b558b0 con 0x558f64c07290 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558f64b598f0 con 0x558f64c068e0 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c068e0 0x558f64c06cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 0x558f64c0b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c068e0 0x558f64c06cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:54150/0 (socket says 172.21.15.26:54150) 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 0x558f64c0b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43218/0 (socket says 172.21.15.26:43218) 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4ab7fe700 1 -- 172.21.15.26:0/2571610786 learned_addr learned my addr 172.21.15.26:0/2571610786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c0a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 -- 172.21.15.26:0/2571610786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c068e0 msgr2=0x558f64c06cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c068e0 0x558f64c06cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 -- 172.21.15.26:0/2571610786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 msgr2=0x558f64c0a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c0a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.668+0000 7fa4abfff700 1 -- 172.21.15.26:0/2571610786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f64aa5950 con 0x558f64c0afe0 2024-05-20T21:02:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.669+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 0x558f64c0b420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40df6e0 tx=0x7fa4a40a9170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ea315800f30817 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.669+0000 7fa491ffb700 1 -- 172.21.15.26:0/2571610786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40a8a20 con 0x558f64c0afe0 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.669+0000 7fa491ffb700 1 -- 172.21.15.26:0/2571610786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a40a8dc0 con 0x558f64c0afe0 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.669+0000 7fa491ffb700 1 -- 172.21.15.26:0/2571610786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40a8510 con 0x558f64c0afe0 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2571610786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 msgr2=0x558f64c0b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 0x558f64c0b420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40df6e0 tx=0x7fa4a40a9170 comp rx=0 tx=0).stop 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2571610786 shutdown_connections 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c0afe0 0x558f64c0b420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c068e0 0x558f64c06cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2571610786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c0a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2571610786 >> 172.21.15.26:0/2571610786 conn(0x558f64b34100 msgr2=0x558f64c00f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2571610786 shutdown_connections 2024-05-20T21:02:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2571610786 wait complete. 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 -- start start 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.670+0000 7fa4bca07b80 1 Processor -- start 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c12460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c0fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558f64a5c020 con 0x558f64c068e0 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558f64af7840 con 0x558f64c0afe0 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558f64c131a0 con 0x558f64c07290 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4bca07b80 10 client.0 _ll_register_callbacks cb 0x558f64af09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c12460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c12460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43220/0 (socket says 172.21.15.26:43220) 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4ab7fe700 1 -- 172.21.15.26:0/390139401 learned_addr learned my addr 172.21.15.26:0/390139401 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4abfff700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c0fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4ab7fe700 1 -- 172.21.15.26:0/390139401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 msgr2=0x558f64c0f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.671+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c0f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4ab7fe700 1 -- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 msgr2=0x558f64c0fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c0fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4ab7fe700 1 -- 172.21.15.26:0/390139401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f64af4710 con 0x558f64c068e0 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4abfff700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c0fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c12460 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00098c0 tx=0x7fa4a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef33afc29c8e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa48affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa48affd700 1 -- 172.21.15.26:0/390139401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0005630 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa48affd700 1 -- 172.21.15.26:0/390139401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa4a001e030 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a40a8dc0 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa48affd700 1 -- 172.21.15.26:0/390139401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4a0003680 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4bca07b80 10 client.17202 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.672+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f64c1d960 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa48affd700 1 -- 172.21.15.26:0/390139401 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a40a8dc0 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa48affd700 1 -- 172.21.15.26:0/390139401 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fa4a000b970 con 0x558f64c068e0 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4bca07b80 10 client.17202 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4bca07b80 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4bca07b80 20 client.17202 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c20070 0x558f64c224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4bca07b80 4 client.17202 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f64bd2a60 con 0x558f64c20070 2024-05-20T21:02:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.673+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c20070 0x558f64c224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.674+0000 7fa48affd700 10 client.17202 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.674+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c20070 0x558f64c224d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b22293cb311c5c server_cookie=6633db3f94308fef in_seq=0 out_seq=0 2024-05-20T21:02:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.674+0000 7fa48affd700 1 -- 172.21.15.26:0/390139401 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558f64bd2a60 con 0x558f64c20070 2024-05-20T21:02:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.674+0000 7fa48affd700 10 client.17202 handle_command_reply: tid=0 2024-05-20T21:02:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.674+0000 7fa4bca07b80 1 client.17202 shutdown 2024-05-20T21:02:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 msgr2=0x558f64c12460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c12460 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00098c0 tx=0x7fa4a0009260 comp rx=0 tx=0).stop 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 shutdown_connections 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c20070 0x558f64c224d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c12460 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c0fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/390139401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa48affd700 0 client.17202 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 >> 172.21.15.26:0/390139401 conn(0x558f64b34100 msgr2=0x558f64c00590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 shutdown_connections 2024-05-20T21:02:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 1 -- 172.21.15.26:0/390139401 wait complete. 2024-05-20T21:02:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.675+0000 7fa4bca07b80 20 client.17202 trim_cache size 0 max 16384 2024-05-20T21:02:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.698+0000 7fa4bca07b80 1 Processor -- start 2024-05-20T21:02:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.698+0000 7fa4bca07b80 1 -- start start 2024-05-20T21:02:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c1c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c0afe0 0x558f64c00d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa49800fde0 con 0x558f64c068e0 2024-05-20T21:02:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa49800d670 con 0x558f64c07290 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa49800b4c0 con 0x558f64c0afe0 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c1c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c11820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41736/0 (socket says 172.21.15.26:41736) 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 -- 172.21.15.26:0/4232074123 learned_addr learned my addr 172.21.15.26:0/4232074123 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4abfff700 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c0afe0 0x558f64c00d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 -- 172.21.15.26:0/4232074123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c0afe0 msgr2=0x558f64c00d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c0afe0 0x558f64c00d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 -- 172.21.15.26:0/4232074123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 msgr2=0x558f64c1c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c1c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 -- 172.21.15.26:0/4232074123 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f64c0a550 con 0x558f64c07290 2024-05-20T21:02:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4abfff700 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c0afe0 0x558f64c00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c1c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.699+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c11820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x558f64bf9790 tx=0x7fa49c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e78b416f3496f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4917fa700 1 -- 172.21.15.26:0/4232074123 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00f7a0 con 0x558f64c07290 2024-05-20T21:02:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4917fa700 1 -- 172.21.15.26:0/4232074123 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa49c00fdd0 con 0x558f64c07290 2024-05-20T21:02:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4917fa700 1 -- 172.21.15.26:0/4232074123 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c01ec80 con 0x558f64c07290 2024-05-20T21:02:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 -- 172.21.15.26:0/4232074123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 msgr2=0x558f64c11820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c11820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x558f64bf9790 tx=0x7fa49c009260 comp rx=0 tx=0).stop 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 -- 172.21.15.26:0/4232074123 shutdown_connections 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c1c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c0afe0 0x558f64c00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/4232074123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c07290 0x558f64c11820 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.700+0000 7fa4bca07b80 1 -- 172.21.15.26:0/4232074123 >> 172.21.15.26:0/4232074123 conn(0x558f64b34100 msgr2=0x558f64c05fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 -- 172.21.15.26:0/4232074123 shutdown_connections 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 -- 172.21.15.26:0/4232074123 wait complete. 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 -- start start 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 Processor -- start 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c16dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.701+0000 7fa4bca07b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c17310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa49800bb30 con 0x558f64c068e0 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa49800cfa0 con 0x558f64c0afe0 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4bca07b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa49800c200 con 0x558f64c07290 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4bca07b80 10 client.0 _ll_register_callbacks cb 0x558f64af09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c17310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4ab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c18a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43248/0 (socket says 172.21.15.26:43248) 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4ab7fe700 1 -- 172.21.15.26:0/2042298048 learned_addr learned my addr 172.21.15.26:0/2042298048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4aaffd700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c16dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 -- 172.21.15.26:0/2042298048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 msgr2=0x558f64c16dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c16dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 -- 172.21.15.26:0/2042298048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 msgr2=0x558f64c18a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c18a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 -- 172.21.15.26:0/2042298048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f64c17a30 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c18a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4abfff700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c17310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40df690 tx=0x7fa4a4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5090df936c8e6a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa48affd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa48affd700 1 -- 172.21.15.26:0/2042298048 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40a8a20 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa48affd700 1 -- 172.21.15.26:0/2042298048 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa4a40a4e30 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.702+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c00fdd0 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa48affd700 1 -- 172.21.15.26:0/2042298048 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4a40ad5f0 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 10 client.26678 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f64c8bc20 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa48affd700 1 -- 172.21.15.26:0/2042298048 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa49c00fdd0 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa48affd700 1 -- 172.21.15.26:0/2042298048 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fa4a40a7f80 con 0x558f64c0afe0 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 10 client.26678 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 20 client.26678 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c8bf60 0x558f64c8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 4 client.26678 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558f64b67c60 con 0x558f64c8bf60 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.703+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c8bf60 0x558f64c8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.704+0000 7fa48affd700 10 client.26678 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.704+0000 7fa4ab7fe700 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c8bf60 0x558f64c8e350 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5517949729db9687 server_cookie=c2acbf842bf34055 in_seq=0 out_seq=0 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa48affd700 1 -- 172.21.15.26:0/2042298048 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558f64b67c60 con 0x558f64c8bf60 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa48affd700 10 client.26678 handle_command_reply: tid=0 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 client.26678 shutdown 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 msgr2=0x558f64c17310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c17310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40df690 tx=0x7fa4a4004b40 comp rx=0 tx=0).stop 2024-05-20T21:02:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 shutdown_connections 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558f64c8bf60 0x558f64c8e350 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f64c068e0 0x558f64c18a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f64c0afe0 0x558f64c17310 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 --2- 172.21.15.26:0/2042298048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f64c07290 0x558f64c16dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 >> 172.21.15.26:0/2042298048 conn(0x558f64b34100 msgr2=0x558f64c11820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa48affd700 0 client.26678 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 shutdown_connections 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 1 -- 172.21.15.26:0/2042298048 wait complete. 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.705+0000 7fa4bca07b80 20 client.26678 trim_cache size 0 max 16384 2024-05-20T21:02:09.743 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:02:09.744 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:02:09.744 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "7c367106-f69c-4d17-8e77-2da5d66b95f5", 2024-05-20T21:02:09.744 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:02:09.744 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:02:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4940b2900 msgr2=0x7fa4940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4940b2900 0x7fa4940b4dc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0000c00 tx=0x7fa4a0005b40 comp rx=0 tx=0).stop 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 msgr2=0x7fa4ac1a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac1a88c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400ef90 tx=0x7fa4a400c3a0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 shutdown_connections 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa4940b2900 0x7fa4940b4dc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ac10aa80 0x7fa4ac1ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4ac11cd20 0x7fa4ac1a88c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 --2- 172.21.15.26:0/2449611417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ac111d20 0x7fa4ac1aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 >> 172.21.15.26:0/2449611417 conn(0x7fa4ac06e6b0 msgr2=0x7fa4ac11ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 shutdown_connections 2024-05-20T21:02:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:09.706+0000 7fa4b2171700 1 -- 172.21.15.26:0/2449611417 wait complete. 2024-05-20T21:02:10.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:02:10.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 112392 1.83 MB/sec execute 136 sec latency 18.699 ms 2024-05-20T21:02:10.829 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:11.255 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:11 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:02:11.255 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:11 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:11.255 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:11 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:02:11.255 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:11 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:11.255 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:11 smithi026 ceph-mon[31990]: pgmap v570: 97 pgs: 97 active+clean; 28 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-05-20T21:02:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:11 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:02:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:11 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:11 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:02:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:11 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:11 smithi144 ceph-mon[35660]: pgmap v570: 97 pgs: 97 active+clean; 28 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-05-20T21:02:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:11 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:02:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:11 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:11 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:02:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:11 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:11 smithi055 ceph-mon[35213]: pgmap v570: 97 pgs: 97 active+clean; 28 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-05-20T21:02:11.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 113184 1.86 MB/sec execute 137 sec latency 19.411 ms 2024-05-20T21:02:12.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:12 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:02:12.337 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:12 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:02:12.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:12 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:02:12.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 113628 1.88 MB/sec execute 138 sec latency 83.478 ms 2024-05-20T21:02:12.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 -- 172.21.15.26:0/1027761078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d881095b0 msgr2=0x7f8d88109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/1027761078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d881095b0 0x7f8d88109990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009990 tx=0x7f8d84009ca0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 -- 172.21.15.26:0/1027761078 shutdown_connections 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/1027761078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d881095b0 0x7f8d88109990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/1027761078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806bbe0 0x7f8d881103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/1027761078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d8806b220 0x7f8d8806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 -- 172.21.15.26:0/1027761078 >> 172.21.15.26:0/1027761078 conn(0x7f8d880793f0 msgr2=0x7f8d88079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 -- 172.21.15.26:0/1027761078 shutdown_connections 2024-05-20T21:02:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.769+0000 7f8d8f88a700 1 -- 172.21.15.26:0/1027761078 wait complete. 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 Processor -- start 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 -- start start 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 0x7f8d88102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d8806bbe0 0x7f8d88102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d881095b0 0x7f8d88102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113590 con 0x7f8d8806b220 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113410 con 0x7f8d881095b0 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8f88a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113710 con 0x7f8d8806bbe0 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8d626700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 0x7f8d88102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8d626700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 0x7f8d88102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43260/0 (socket says 172.21.15.26:43260) 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.770+0000 7f8d8d626700 1 -- 172.21.15.26:0/2257638811 learned_addr learned my addr 172.21.15.26:0/2257638811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8de27700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d881095b0 0x7f8d88102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d8806bbe0 0x7f8d88102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8d626700 1 -- 172.21.15.26:0/2257638811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d8806bbe0 msgr2=0x7f8d88102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8d626700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d8806bbe0 0x7f8d88102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8d626700 1 -- 172.21.15.26:0/2257638811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d881095b0 msgr2=0x7f8d88102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8d626700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d881095b0 0x7f8d88102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8d626700 1 -- 172.21.15.26:0/2257638811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d881a8070 con 0x7f8d8806b220 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8d626700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 0x7f8d88102510 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8d84006010 tx=0x7f8d840057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d7e7fc700 1 -- 172.21.15.26:0/2257638811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84027070 con 0x7f8d8806b220 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d7e7fc700 1 -- 172.21.15.26:0/2257638811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d8400fb90 con 0x7f8d8806b220 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d881a8300 con 0x7f8d8806b220 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d7e7fc700 1 -- 172.21.15.26:0/2257638811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840173f0 con 0x7f8d8806b220 2024-05-20T21:02:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.771+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d881a8750 con 0x7f8d8806b220 2024-05-20T21:02:12.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.774+0000 7f8d7e7fc700 1 -- 172.21.15.26:0/2257638811 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8d8402a070 con 0x7f8d8806b220 2024-05-20T21:02:12.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.775+0000 7f8d7e7fc700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d740b2880 0x7f8d740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.775+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d740b2880 0x7f8d740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.775+0000 7f8d7e7fc700 1 -- 172.21.15.26:0/2257638811 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8d840e3190 con 0x7f8d8806b220 2024-05-20T21:02:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.775+0000 7f8d9a120b80 1 Processor -- start 2024-05-20T21:02:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 -- start start 2024-05-20T21:02:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d740b2880 0x7f8d740b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d88079b30 tx=0x7f8d78005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffba79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb99680 0x55dcffb99b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcffaf8c90 con 0x55dcffb99680 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dcffae8830 con 0x55dcffb9a040 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dcffaec870 con 0x55dcffba7600 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8ce25700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb99680 0x55dcffb99b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41774/0 (socket says 172.21.15.26:41774) 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 -- 172.21.15.26:0/823676876 learned_addr learned my addr 172.21.15.26:0/823676876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8d626700 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffba79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 -- 172.21.15.26:0/823676876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 msgr2=0x55dcffba79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffba79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 -- 172.21.15.26:0/823676876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb99680 msgr2=0x55dcffb99b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb99680 0x55dcffb99b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.776+0000 7f8d8de27700 1 -- 172.21.15.26:0/823676876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcffa388d0 con 0x55dcffb9a040 2024-05-20T21:02:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.777+0000 7f8d8de27700 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbae410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011680 tx=0x7f8d80009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d575849556d43f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.777+0000 7f8d737fe700 1 -- 172.21.15.26:0/823676876 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80019030 con 0x55dcffb9a040 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.777+0000 7f8d737fe700 1 -- 172.21.15.26:0/823676876 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d800092e0 con 0x55dcffb9a040 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.777+0000 7f8d737fe700 1 -- 172.21.15.26:0/823676876 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80007500 con 0x55dcffb9a040 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.777+0000 7f8d9a120b80 1 -- 172.21.15.26:0/823676876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 msgr2=0x55dcffbae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.777+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbae410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011680 tx=0x7f8d80009920 comp rx=0 tx=0).stop 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 -- 172.21.15.26:0/823676876 shutdown_connections 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbae410 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffba79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/823676876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb99680 0x55dcffb99b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 -- 172.21.15.26:0/823676876 >> 172.21.15.26:0/823676876 conn(0x55dcffac7080 msgr2=0x55dcffb93e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 -- 172.21.15.26:0/823676876 shutdown_connections 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 -- 172.21.15.26:0/823676876 wait complete. 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 -- start start 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.778+0000 7f8d9a120b80 1 Processor -- start 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffbab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcff9eefa0 con 0x55dcffb9a040 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dcffa8a7c0 con 0x55dcffb99680 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dcffbaea70 con 0x55dcffba7600 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d9a120b80 10 client.0 _ll_register_callbacks cb 0x55dcffa83920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8ce25700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffbab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41782/0 (socket says 172.21.15.26:41782) 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 -- 172.21.15.26:0/3213820568 learned_addr learned my addr 172.21.15.26:0/3213820568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffbab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43278/0 (socket says 172.21.15.26:43278) 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8de27700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 -- 172.21.15.26:0/3213820568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 msgr2=0x55dcffbab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 -- 172.21.15.26:0/3213820568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 msgr2=0x55dcffbab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffbab3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 -- 172.21.15.26:0/3213820568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcffa87690 con 0x55dcffb99680 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d8d626700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbae0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d840aa770 tx=0x7f8d840a9f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c55bf4091d382c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.779+0000 7f8d70ff9700 1 -- 172.21.15.26:0/3213820568 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840aaee0 con 0x55dcffb99680 2024-05-20T21:02:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d70ff9700 1 -- 172.21.15.26:0/3213820568 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8d840b3330 con 0x55dcffb99680 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d800092e0 con 0x55dcffb99680 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d70ff9700 1 -- 172.21.15.26:0/3213820568 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d840b10b0 con 0x55dcffb99680 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 10 client.26690 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dcffbb9140 con 0x55dcffb99680 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d70ff9700 1 -- 172.21.15.26:0/3213820568 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d800092e0 con 0x55dcffb99680 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d70ff9700 1 -- 172.21.15.26:0/3213820568 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8d840a9a40 con 0x55dcffb99680 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 10 client.26690 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 20 client.26690 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffbbb6b0 0x55dcffbbdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 4 client.26690 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.780+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcffb659e0 con 0x55dcffbbb6b0 2024-05-20T21:02:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.781+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffbbb6b0 0x55dcffbbdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.781+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffbbb6b0 0x55dcffbbdb00 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6487c3ab0860bfcb server_cookie=f326509d6457b193 in_seq=0 out_seq=0 2024-05-20T21:02:12.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.781+0000 7f8d70ff9700 10 client.26690 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d70ff9700 1 -- 172.21.15.26:0/3213820568 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dcffb659e0 con 0x55dcffbbb6b0 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d70ff9700 10 client.26690 handle_command_reply: tid=0 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 client.26690 shutdown 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 msgr2=0x55dcffbae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbae0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d840aa770 tx=0x7f8d840a9f50 comp rx=0 tx=0).stop 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 shutdown_connections 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffbbb6b0 0x55dcffbbdb00 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbae0b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3213820568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffbab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 >> 172.21.15.26:0/3213820568 conn(0x55dcffac7080 msgr2=0x55dcffb93510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:12.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d70ff9700 0 client.26690 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:12.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 shutdown_connections 2024-05-20T21:02:12.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3213820568 wait complete. 2024-05-20T21:02:12.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.782+0000 7f8d9a120b80 20 client.26690 trim_cache size 0 max 16384 2024-05-20T21:02:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.805+0000 7f8d9a120b80 1 Processor -- start 2024-05-20T21:02:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 -- start start 2024-05-20T21:02:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbaa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffb93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881ae2c0 con 0x55dcffb9a040 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881abb50 con 0x55dcffb99680 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8819da70 con 0x55dcffba7600 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8d626700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbaa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8ce25700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffb93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8d626700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbaa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41792/0 (socket says 172.21.15.26:41792) 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8d626700 1 -- 172.21.15.26:0/3913569641 learned_addr learned my addr 172.21.15.26:0/3913569641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8de27700 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8ce25700 1 -- 172.21.15.26:0/3913569641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 msgr2=0x55dcffbad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8ce25700 1 -- 172.21.15.26:0/3913569641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 msgr2=0x55dcffbaa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbaa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8ce25700 1 -- 172.21.15.26:0/3913569641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcffb959a0 con 0x55dcffb9a040 2024-05-20T21:02:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.806+0000 7f8d8d626700 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbaa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.807+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffb93c90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x55dcffbac980 tx=0x7f8d78014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc6b388df5055f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.807+0000 7f8d72ffd700 1 -- 172.21.15.26:0/3913569641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78010c70 con 0x55dcffb9a040 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.807+0000 7f8d72ffd700 1 -- 172.21.15.26:0/3913569641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d78010e10 con 0x55dcffb9a040 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.807+0000 7f8d72ffd700 1 -- 172.21.15.26:0/3913569641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78029670 con 0x55dcffb9a040 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3913569641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 msgr2=0x55dcffb93c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffb93c90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x55dcffbac980 tx=0x7f8d78014000 comp rx=0 tx=0).stop 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3913569641 shutdown_connections 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffba7600 0x55dcffbad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb99680 0x55dcffbaa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/3913569641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffb9a040 0x55dcffb93c90 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3913569641 >> 172.21.15.26:0/3913569641 conn(0x55dcffac7080 msgr2=0x55dcffbacbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3913569641 shutdown_connections 2024-05-20T21:02:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 -- 172.21.15.26:0/3913569641 wait complete. 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 -- start start 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.808+0000 7f8d9a120b80 1 Processor -- start 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffb99680 0x55dcffbb3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbb4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 0x55dcffbb0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881aa010 con 0x55dcffba7600 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881ab480 con 0x55dcffb9a040 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881aa6e0 con 0x55dcffb99680 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d9a120b80 10 client.0 _ll_register_callbacks cb 0x55dcffa83920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 0x55dcffbb0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 0x55dcffbb0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43296/0 (socket says 172.21.15.26:43296) 2024-05-20T21:02:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 -- 172.21.15.26:0/2595979124 learned_addr learned my addr 172.21.15.26:0/2595979124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8ce25700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbb4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8d626700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffb99680 0x55dcffbb3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 -- 172.21.15.26:0/2595979124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffb99680 msgr2=0x55dcffbb3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffb99680 0x55dcffbb3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 -- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 msgr2=0x55dcffbb4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbb4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 -- 172.21.15.26:0/2595979124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcffbb4890 con 0x55dcffba7600 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d8de27700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 0x55dcffbb0c90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011650 tx=0x7f8d800040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7174d11792c847c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.809+0000 7f8d70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d70ff9700 1 -- 172.21.15.26:0/2595979124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80004510 con 0x55dcffba7600 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d70ff9700 1 -- 172.21.15.26:0/2595979124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8d800046b0 con 0x55dcffba7600 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d78010e10 con 0x55dcffba7600 2024-05-20T21:02:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d70ff9700 1 -- 172.21.15.26:0/2595979124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d800048b0 con 0x55dcffba7600 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 10 client.17226 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dcffc268d0 con 0x55dcffba7600 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d70ff9700 1 -- 172.21.15.26:0/2595979124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d78010e10 con 0x55dcffba7600 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d70ff9700 1 -- 172.21.15.26:0/2595979124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8d8002f050 con 0x55dcffba7600 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 10 client.17226 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 20 client.17226 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffc26d10 0x55dcffc29130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.810+0000 7f8d9a120b80 4 client.17226 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dcffafabe0 con 0x55dcffc26d10 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d8d626700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffc26d10 0x55dcffc29130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d8d626700 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffc26d10 0x55dcffc29130 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20286e447c1d4ff0 server_cookie=55e683c2ca103e14 in_seq=0 out_seq=0 2024-05-20T21:02:12.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d70ff9700 10 client.17226 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:12.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d70ff9700 1 -- 172.21.15.26:0/2595979124 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dcffafabe0 con 0x55dcffc26d10 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d70ff9700 10 client.17226 handle_command_reply: tid=0 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.811+0000 7f8d9a120b80 1 client.17226 shutdown 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 msgr2=0x55dcffbb0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 0x55dcffbb0c90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011650 tx=0x7f8d800040d0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 shutdown_connections 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcffc26d10 0x55dcffc29130 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcffba7600 0x55dcffbb0c90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcffb99680 0x55dcffbb3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 --2- 172.21.15.26:0/2595979124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcffb9a040 0x55dcffbb4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 >> 172.21.15.26:0/2595979124 conn(0x55dcffac7080 msgr2=0x55dcffb98f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d70ff9700 0 client.17226 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 shutdown_connections 2024-05-20T21:02:12.817 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:02:12.817 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:02:12.817 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:02:12.817 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 1 -- 172.21.15.26:0/2595979124 wait complete. 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.812+0000 7f8d9a120b80 20 client.17226 trim_cache size 0 max 16384 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d740b2880 msgr2=0x7f8d740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d740b2880 0x7f8d740b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d88079b30 tx=0x7f8d78005eb0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 msgr2=0x7f8d88102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 0x7f8d88102510 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8d84006010 tx=0x7f8d840057c0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 shutdown_connections 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d740b2880 0x7f8d740b4d40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d8806b220 0x7f8d88102510 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d881095b0 0x7f8d88102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 --2- 172.21.15.26:0/2257638811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d8806bbe0 0x7f8d88102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:12.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 >> 172.21.15.26:0/2257638811 conn(0x7f8d880793f0 msgr2=0x7f8d88101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:12.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 shutdown_connections 2024-05-20T21:02:12.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:12.813+0000 7f8d8f88a700 1 -- 172.21.15.26:0/2257638811 wait complete. 2024-05-20T21:02:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:13 smithi144 ceph-mon[35660]: pgmap v571: 97 pgs: 97 active+clean; 27 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-05-20T21:02:13.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:13 smithi026 ceph-mon[31990]: pgmap v571: 97 pgs: 97 active+clean; 27 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-05-20T21:02:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:13 smithi055 ceph-mon[35213]: pgmap v571: 97 pgs: 97 active+clean; 27 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-05-20T21:02:13.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 114051 1.87 MB/sec execute 139 sec latency 30.603 ms 2024-05-20T21:02:13.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:02:13.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:02:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:14.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 114665 1.88 MB/sec execute 140 sec latency 25.394 ms 2024-05-20T21:02:15.142 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:15 smithi026 ceph-mon[31990]: pgmap v572: 97 pgs: 97 active+clean; 29 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-05-20T21:02:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:15 smithi144 ceph-mon[35660]: pgmap v572: 97 pgs: 97 active+clean; 29 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-05-20T21:02:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:15 smithi055 ceph-mon[35213]: pgmap v572: 97 pgs: 97 active+clean; 29 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-05-20T21:02:15.560 INFO:tasks.workunit.client.0.smithi026.stdout: 1 115390 1.89 MB/sec execute 141 sec latency 18.437 ms 2024-05-20T21:02:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 -- 172.21.15.26:0/496751432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64108520 msgr2=0x7f2d64108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 --2- 172.21.15.26:0/496751432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64108520 0x7f2d64108900 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009990 tx=0x7f2d54009ca0 comp rx=0 tx=0).stop 2024-05-20T21:02:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 -- 172.21.15.26:0/496751432 shutdown_connections 2024-05-20T21:02:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 --2- 172.21.15.26:0/496751432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64108520 0x7f2d64108900 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 --2- 172.21.15.26:0/496751432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102e00 0x7f2d6410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 --2- 172.21.15.26:0/496751432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d64102440 0x7f2d641028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 -- 172.21.15.26:0/496751432 >> 172.21.15.26:0/496751432 conn(0x7f2d640793f0 msgr2=0x7f2d64079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 -- 172.21.15.26:0/496751432 shutdown_connections 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 -- 172.21.15.26:0/496751432 wait complete. 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 Processor -- start 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.753+0000 7f2d693bd700 1 -- start start 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d693bd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102440 0x7f2d6419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d693bd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 0x7f2d6419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d693bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d64108520 0x7f2d641999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d693bd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641124d0 con 0x7f2d64102e00 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d693bd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64112350 con 0x7f2d64102440 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d693bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64112650 con 0x7f2d64108520 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d627fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 0x7f2d6419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102440 0x7f2d6419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d627fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 0x7f2d6419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43308/0 (socket says 172.21.15.26:43308) 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d627fc700 1 -- 172.21.15.26:0/2826074520 learned_addr learned my addr 172.21.15.26:0/2826074520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d637fe700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d64108520 0x7f2d641999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 -- 172.21.15.26:0/2826074520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d64108520 msgr2=0x7f2d641999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d64108520 0x7f2d641999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 -- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 msgr2=0x7f2d6419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 0x7f2d6419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 -- 172.21.15.26:0/2826074520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6419a100 con 0x7f2d64102440 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d627fc700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 0x7f2d6419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.754+0000 7f2d62ffd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102440 0x7f2d6419f7e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098e0 tx=0x7f2d540047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.755+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54026070 con 0x7f2d64102440 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.755+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d54021410 con 0x7f2d64102440 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.755+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6419a390 con 0x7f2d64102440 2024-05-20T21:02:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.755+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54017650 con 0x7f2d64102440 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.755+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641a66b0 con 0x7f2d64102440 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.756+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d64103b60 con 0x7f2d64102440 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.757+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2d54029070 con 0x7f2d64102440 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.758+0000 7f2d4bfff700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d4c0b2880 0x7f2d4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.758+0000 7f2d627fc700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d4c0b2880 0x7f2d4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.758+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2d540e3810 con 0x7f2d64102440 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.758+0000 7f2d627fc700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d4c0b2880 0x7f2d4c0b4d40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2d64079a60 tx=0x7f2d5800b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:15.761+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d54030070 con 0x7f2d64102440 2024-05-20T21:02:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.099+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d641a6880 con 0x7f2d64102440 2024-05-20T21:02:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.100+0000 7f2d4bfff700 1 -- 172.21.15.26:0/2826074520 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f2d54013070 con 0x7f2d64102440 2024-05-20T21:02:16.100 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:02:16.100 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:02:16.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d4c0b2880 msgr2=0x7f2d4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d4c0b2880 0x7f2d4c0b4d40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2d64079a60 tx=0x7f2d5800b500 comp rx=0 tx=0).stop 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102440 msgr2=0x7f2d6419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102440 0x7f2d6419f7e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098e0 tx=0x7f2d540047b0 comp rx=0 tx=0).stop 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 shutdown_connections 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d64108520 0x7f2d641999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d64102440 0x7f2d6419f7e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d4c0b2880 0x7f2d4c0b4d40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 --2- 172.21.15.26:0/2826074520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d64102e00 0x7f2d6419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.102+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 >> 172.21.15.26:0/2826074520 conn(0x7f2d640793f0 msgr2=0x7f2d6406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.103+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 shutdown_connections 2024-05-20T21:02:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:16.103+0000 7f2d693bd700 1 -- 172.21.15.26:0/2826074520 wait complete. 2024-05-20T21:02:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:02:16.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 116041 1.89 MB/sec execute 142 sec latency 18.483 ms 2024-05-20T21:02:16.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:02:17.182 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:17.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:17 smithi026 ceph-mon[31990]: pgmap v573: 97 pgs: 97 active+clean; 29 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T21:02:17.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:17 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2826074520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:02:17.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:17 smithi144 ceph-mon[35660]: pgmap v573: 97 pgs: 97 active+clean; 29 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T21:02:17.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:17 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2826074520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:02:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:17 smithi055 ceph-mon[35213]: pgmap v573: 97 pgs: 97 active+clean; 29 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-05-20T21:02:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:17 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2826074520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:02:17.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 116860 1.92 MB/sec execute 143 sec latency 11.875 ms 2024-05-20T21:02:18.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 117416 1.94 MB/sec execute 144 sec latency 18.283 ms 2024-05-20T21:02:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/1978954383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 msgr2=0x7f9b041099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/1978954383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b041099b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009990 tx=0x7f9b00009ca0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/1978954383 shutdown_connections 2024-05-20T21:02:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/1978954383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b041099b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/1978954383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b0406c5d0 0x7f9b041103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/1978954383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406bc10 0x7f9b0406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/1978954383 >> 172.21.15.26:0/1978954383 conn(0x7f9b040793f0 msgr2=0x7f9b04079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.056 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:19 smithi026 ceph-mon[31990]: pgmap v574: 97 pgs: 97 active+clean; 30 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:02:19.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/1978954383 shutdown_connections 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.055+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/1978954383 wait complete. 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 Processor -- start 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 -- start start 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b0406bc10 0x7f9b041028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406c5d0 0x7f9b04102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b04103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04113590 con 0x7f9b041095d0 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04113410 con 0x7f9b0406bc10 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b0b6f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04113710 con 0x7f9b0406c5d0 2024-05-20T21:02:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.056+0000 7f9b09c8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b04103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b04103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43322/0 (socket says 172.21.15.26:43322) 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 -- 172.21.15.26:0/3430634496 learned_addr learned my addr 172.21.15.26:0/3430634496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406c5d0 0x7f9b04102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b0948d700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b0406bc10 0x7f9b041028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 -- 172.21.15.26:0/3430634496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406c5d0 msgr2=0x7f9b04102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406c5d0 0x7f9b04102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 -- 172.21.15.26:0/3430634496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b0406bc10 msgr2=0x7f9b041028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b0406bc10 0x7f9b041028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 -- 172.21.15.26:0/3430634496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b041a80b0 con 0x7f9b041095d0 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b04103370 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00ef30 tx=0x7f9afc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.057+0000 7f9afa7fc700 1 -- 172.21.15.26:0/3430634496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0092e0 con 0x7f9b041095d0 2024-05-20T21:02:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.058+0000 7f9afa7fc700 1 -- 172.21.15.26:0/3430634496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9afc004bb0 con 0x7f9b041095d0 2024-05-20T21:02:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.058+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b041a8340 con 0x7f9b041095d0 2024-05-20T21:02:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.058+0000 7f9afa7fc700 1 -- 172.21.15.26:0/3430634496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc004020 con 0x7f9b041095d0 2024-05-20T21:02:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.058+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041a8830 con 0x7f9b041095d0 2024-05-20T21:02:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.061+0000 7f9afa7fc700 1 -- 172.21.15.26:0/3430634496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9afc01b030 con 0x7f9b041095d0 2024-05-20T21:02:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.061+0000 7f9afa7fc700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9af00b2900 0x7f9af00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9afa7fc700 1 -- 172.21.15.26:0/3430634496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9afc0e1ce0 con 0x7f9b041095d0 2024-05-20T21:02:19.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b0948d700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9af00b2900 0x7f9af00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 Processor -- start 2024-05-20T21:02:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 -- start start 2024-05-20T21:02:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b0948d700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9af00b2900 0x7f9af00b4dc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b2e0 tx=0x7f9b0001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d06496c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d063f5c90 con 0x556d06496860 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556d063e5830 con 0x556d0649af60 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.062+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d063e9870 con 0x556d06497210 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d06496c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d06496c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43334/0 (socket says 172.21.15.26:43334) 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b08c8c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 -- 172.21.15.26:0/1406047966 learned_addr learned my addr 172.21.15.26:0/1406047966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 -- 172.21.15.26:0/1406047966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 msgr2=0x556d0649a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 -- 172.21.15.26:0/1406047966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 msgr2=0x556d0649b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 -- 172.21.15.26:0/1406047966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d063358d0 con 0x556d06496860 2024-05-20T21:02:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d06496c40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000bda0 tx=0x7f9b0001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f9a1157f9d357 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.063+0000 7f9aef7fe700 1 -- 172.21.15.26:0/1406047966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000fb50 con 0x556d06496860 2024-05-20T21:02:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9aef7fe700 1 -- 172.21.15.26:0/1406047966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b00032070 con 0x556d06496860 2024-05-20T21:02:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9aef7fe700 1 -- 172.21.15.26:0/1406047966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0002d3f0 con 0x556d06496860 2024-05-20T21:02:19.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1406047966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 msgr2=0x556d06496c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d06496c40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000bda0 tx=0x7f9b0001e000 comp rx=0 tx=0).stop 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1406047966 shutdown_connections 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d06496c40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1406047966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1406047966 >> 172.21.15.26:0/1406047966 conn(0x556d063c4080 msgr2=0x556d06490e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1406047966 shutdown_connections 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.064+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1406047966 wait complete. 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 -- start start 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 Processor -- start 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d064a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d062ebfa0 con 0x556d06496860 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556d063877c0 con 0x556d0649af60 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d064a30c0 con 0x556d06497210 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.065+0000 7f9b15f87b80 10 client.0 _ll_register_callbacks cb 0x556d06380920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d064a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d064a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43348/0 (socket says 172.21.15.26:43348) 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 -- 172.21.15.26:0/1714754936 learned_addr learned my addr 172.21.15.26:0/1714754936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 -- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 msgr2=0x556d0649f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 -- 172.21.15.26:0/1714754936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 msgr2=0x556d0649fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 -- 172.21.15.26:0/1714754936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d06384690 con 0x556d06496860 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d064a2370 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9b00023a80 tx=0x7f9b00005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969ba8c4b05dfdfa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9aecff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9aecff9700 1 -- 172.21.15.26:0/1714754936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00036420 con 0x556d06496860 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.066+0000 7f9aecff9700 1 -- 172.21.15.26:0/1714754936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9b000365c0 con 0x556d06496860 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00032070 con 0x556d06496860 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9aecff9700 1 -- 172.21.15.26:0/1714754936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9b00047050 con 0x556d06496860 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 10 client.17244 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d064ad780 con 0x556d06496860 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9aecff9700 1 -- 172.21.15.26:0/1714754936 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b00032070 con 0x556d06496860 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9aecff9700 1 -- 172.21.15.26:0/1714754936 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f9b0002d620 con 0x556d06496860 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 10 client.17244 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 20 client.17244 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d064afc50 0x556d064b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 4 client.17244 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.067+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d064629e0 con 0x556d064afc50 2024-05-20T21:02:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.068+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d064afc50 0x556d064b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.068+0000 7f9aecff9700 10 client.17244 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:02:19.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.068+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d064afc50 0x556d064b2040 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7118eb5b71a59734 server_cookie=ff1416a1836706f7 in_seq=0 out_seq=0 2024-05-20T21:02:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9aecff9700 1 -- 172.21.15.26:0/1714754936 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556d064629e0 con 0x556d064afc50 2024-05-20T21:02:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9aecff9700 10 client.17244 handle_command_reply: tid=0 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 client.17244 shutdown 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 msgr2=0x556d064a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d064a2370 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9b00023a80 tx=0x7f9b00005ea0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 shutdown_connections 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d064afc50 0x556d064b2040 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06496860 0x556d064a2370 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d0649fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1714754936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06497210 0x556d0649f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 >> 172.21.15.26:0/1714754936 conn(0x556d063c4080 msgr2=0x556d06490510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9aecff9700 0 client.17244 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:02:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 shutdown_connections 2024-05-20T21:02:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.069+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1714754936 wait complete. 2024-05-20T21:02:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.070+0000 7f9b15f87b80 20 client.17244 trim_cache size 0 max 16384 2024-05-20T21:02:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 Processor -- start 2024-05-20T21:02:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 -- start start 2024-05-20T21:02:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d06496860 0x556d06495ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d06490c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d0649af60 0x556d064911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041b2780 con 0x556d06497210 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041b0010 con 0x556d06496860 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041ade60 con 0x556d0649af60 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b08c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d06490c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b08c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d06490c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43360/0 (socket says 172.21.15.26:43360) 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b0948d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d06496860 0x556d06495ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.087+0000 7f9b08c8c700 1 -- 172.21.15.26:0/1921992749 learned_addr learned my addr 172.21.15.26:0/1921992749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d0649af60 0x556d064911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b0948d700 1 -- 172.21.15.26:0/1921992749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d0649af60 msgr2=0x556d064911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d0649af60 0x556d064911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b0948d700 1 -- 172.21.15.26:0/1921992749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 msgr2=0x556d06490c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d06490c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b0948d700 1 -- 172.21.15.26:0/1921992749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0649dc20 con 0x556d06496860 2024-05-20T21:02:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d0649af60 0x556d064911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:02:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d06490c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9b0948d700 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d06496860 0x556d06495ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002dac0 tx=0x7f9b00005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb03b742f5dd0f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9aeeffd700 1 -- 172.21.15.26:0/1921992749 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000236e0 con 0x556d06496860 2024-05-20T21:02:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9aeeffd700 1 -- 172.21.15.26:0/1921992749 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b00032040 con 0x556d06496860 2024-05-20T21:02:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.088+0000 7f9aeeffd700 1 -- 172.21.15.26:0/1921992749 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000359e0 con 0x556d06496860 2024-05-20T21:02:19.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1921992749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d06496860 msgr2=0x556d06495ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d06496860 0x556d06495ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002dac0 tx=0x7f9b00005f10 comp rx=0 tx=0).stop 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1921992749 shutdown_connections 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d0649af60 0x556d064911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d06496860 0x556d06495ce0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/1921992749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d06490c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1921992749 >> 172.21.15.26:0/1921992749 conn(0x556d063c4080 msgr2=0x556d064a1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1921992749 shutdown_connections 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 -- 172.21.15.26:0/1921992749 wait complete. 2024-05-20T21:02:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.089+0000 7f9b15f87b80 1 -- start start 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 Processor -- start 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06496860 0x556d064b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d064b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d06511530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041ae4d0 con 0x556d06497210 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041af940 con 0x556d0649af60 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041aeba0 con 0x556d06496860 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b15f87b80 10 client.0 _ll_register_callbacks cb 0x556d06380920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b08c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d064b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b09c8e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d06511530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b08c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d064b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43370/0 (socket says 172.21.15.26:43370) 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b08c8c700 1 -- 172.21.15.26:0/752716974 learned_addr learned my addr 172.21.15.26:0/752716974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.090+0000 7f9b0948d700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06496860 0x556d064b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b08c8c700 1 -- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06496860 msgr2=0x556d064b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06496860 0x556d064b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b08c8c700 1 -- 172.21.15.26:0/752716974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 msgr2=0x556d06511530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d06511530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b08c8c700 1 -- 172.21.15.26:0/752716974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d06511c10 con 0x556d06497210 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b09c8e700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d06511530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b0948d700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06496860 0x556d064b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b08c8c700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d064b4ed0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005950 tx=0x7f9af401f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c978cd390ed06d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9aecff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9aecff9700 1 -- 172.21.15.26:0/752716974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004020 con 0x556d06497210 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00032040 con 0x556d06497210 2024-05-20T21:02:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9aecff9700 1 -- 172.21.15.26:0/752716974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9af40041c0 con 0x556d06497210 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9aecff9700 1 -- 172.21.15.26:0/752716974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9af40043c0 con 0x556d06497210 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b15f87b80 10 client.17250 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.091+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d0651a150 con 0x556d06497210 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9aecff9700 1 -- 172.21.15.26:0/752716974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b00032040 con 0x556d06497210 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9aecff9700 1 -- 172.21.15.26:0/752716974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f9af40053c0 con 0x556d06497210 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b15f87b80 10 client.17250 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b15f87b80 10 client.17250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:02:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b15f87b80 20 client.17250 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d0651a8c0 0x556d0651cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b15f87b80 4 client.17250 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d063f7be0 con 0x556d0651a8c0 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.092+0000 7f9b0948d700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d0651a8c0 0x556d0651cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9aecff9700 10 client.17250 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b0948d700 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d0651a8c0 0x556d0651cce0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b65a3479def06181 server_cookie=c95d7fb30d62e1c8 in_seq=0 out_seq=0 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9aecff9700 1 -- 172.21.15.26:0/752716974 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d063f7be0 con 0x556d0651a8c0 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9aecff9700 10 client.17250 handle_command_reply: tid=0 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 client.17250 shutdown 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 msgr2=0x556d064b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d064b4ed0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005950 tx=0x7f9af401f790 comp rx=0 tx=0).stop 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 shutdown_connections 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x556d0651a8c0 0x556d0651cce0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556d0649af60 0x556d06511530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d06496860 0x556d064b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9aecff9700 0 client.17250 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 --2- 172.21.15.26:0/752716974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d06497210 0x556d064b4ed0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 >> 172.21.15.26:0/752716974 conn(0x556d063c4080 msgr2=0x556d064a0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 shutdown_connections 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 1 -- 172.21.15.26:0/752716974 wait complete. 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.093+0000 7f9b15f87b80 20 client.17250 trim_cache size 0 max 16384 2024-05-20T21:02:19.100 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:02:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9af00b2900 msgr2=0x7f9af00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9af00b2900 0x7f9af00b4dc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b2e0 tx=0x7f9b0001a040 comp rx=0 tx=0).stop 2024-05-20T21:02:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 msgr2=0x7f9b04103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b04103370 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00ef30 tx=0x7f9afc00c3a0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 shutdown_connections 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9af00b2900 0x7f9af00b4dc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b041095d0 0x7f9b04103370 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b0406bc10 0x7f9b041028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 --2- 172.21.15.26:0/3430634496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406c5d0 0x7f9b04102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 >> 172.21.15.26:0/3430634496 conn(0x7f9b040793f0 msgr2=0x7f9b041016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 shutdown_connections 2024-05-20T21:02:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:19.094+0000 7f9b0b6f1700 1 -- 172.21.15.26:0/3430634496 wait complete. 2024-05-20T21:02:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:19 smithi144 ceph-mon[35660]: pgmap v574: 97 pgs: 97 active+clean; 30 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:02:19.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:19 smithi055 ceph-mon[35213]: pgmap v574: 97 pgs: 97 active+clean; 30 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:02:19.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 117912 1.93 MB/sec execute 145 sec latency 26.961 ms 2024-05-20T21:02:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 -- 172.21.15.26:0/1036275763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c103590 msgr2=0x7f8a6c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 --2- 172.21.15.26:0/1036275763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c103590 0x7f8a6c105980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c02a410 comp rx=0 tx=0).stop 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 -- 172.21.15.26:0/1036275763 shutdown_connections 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 --2- 172.21.15.26:0/1036275763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c1087f0 0x7f8a6c10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 --2- 172.21.15.26:0/1036275763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a6c105ec0 0x7f8a6c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 --2- 172.21.15.26:0/1036275763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c103590 0x7f8a6c105980 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 -- 172.21.15.26:0/1036275763 >> 172.21.15.26:0/1036275763 conn(0x7f8a6c0fcf20 msgr2=0x7f8a6c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 -- 172.21.15.26:0/1036275763 shutdown_connections 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.723+0000 7f8a720a0640 1 -- 172.21.15.26:0/1036275763 wait complete. 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 Processor -- start 2024-05-20T21:02:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 -- start start 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 0x7f8a6c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a6c105ec0 0x7f8a6c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c1087f0 0x7f8a6c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10e4b0 con 0x7f8a6c1087f0 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10e330 con 0x7f8a6c103590 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a720a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10e630 con 0x7f8a6c105ec0 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 0x7f8a6c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c1087f0 0x7f8a6c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 0x7f8a6c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57728/0 (socket says 172.21.15.26:57728) 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a62ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a6c105ec0 0x7f8a6c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 -- 172.21.15.26:0/538775648 learned_addr learned my addr 172.21.15.26:0/538775648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 -- 172.21.15.26:0/538775648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a6c105ec0 msgr2=0x7f8a6c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a6c105ec0 0x7f8a6c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 -- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c1087f0 msgr2=0x7f8a6c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.724+0000 7f8a6b7fe640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c1087f0 0x7f8a6c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a6b7fe640 1 -- 172.21.15.26:0/538775648 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c1a0800 con 0x7f8a6c103590 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a6b7fe640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 0x7f8a6c198be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009a20 tx=0x7f8a5c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a697fa640 1 -- 172.21.15.26:0/538775648 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c03f030 con 0x7f8a6c103590 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a697fa640 1 -- 172.21.15.26:0/538775648 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a5c003a80 con 0x7f8a6c103590 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c1a0a90 con 0x7f8a6c103590 2024-05-20T21:02:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a697fa640 1 -- 172.21.15.26:0/538775648 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c035070 con 0x7f8a6c103590 2024-05-20T21:02:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.725+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1a0fd0 con 0x7f8a6c103590 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.728+0000 7f8a697fa640 1 -- 172.21.15.26:0/538775648 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8a5c042030 con 0x7f8a6c103590 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.728+0000 7f8a74dc2740 1 Processor -- start 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 -- start start 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc965ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc96abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc96d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b5cc8d6c30 con 0x55b5cc96b220 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b5cc8c67d0 con 0x55b5cc9656c0 2024-05-20T21:02:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b5cc8ca810 con 0x55b5cc9660b0 2024-05-20T21:02:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc965ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc96d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc965ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57736/0 (socket says 172.21.15.26:57736) 2024-05-20T21:02:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 -- 172.21.15.26:0/4017015409 learned_addr learned my addr 172.21.15.26:0/4017015409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6b7fe640 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc96abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 -- 172.21.15.26:0/4017015409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 msgr2=0x55b5cc96abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc96abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 -- 172.21.15.26:0/4017015409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 msgr2=0x55b5cc96d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc96d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.729+0000 7f8a6bfff640 1 -- 172.21.15.26:0/4017015409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5cc5db210 con 0x55b5cc9656c0 2024-05-20T21:02:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc965ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009fd0 tx=0x7f8a640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b781972e5be98aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a697fa640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8a580b2920 0x7f8a580b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a61ffb640 1 -- 172.21.15.26:0/4017015409 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64004d20 con 0x55b5cc9656c0 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a6b7fe640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8a580b2920 0x7f8a580b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a61ffb640 1 -- 172.21.15.26:0/4017015409 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a64004050 con 0x55b5cc9656c0 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a61ffb640 1 -- 172.21.15.26:0/4017015409 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64005220 con 0x55b5cc9656c0 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.730+0000 7f8a697fa640 1 -- 172.21.15.26:0/538775648 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8a5c102790 con 0x7f8a6c103590 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a6b7fe640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8a580b2920 0x7f8a580b4de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c1005a0 tx=0x7f8a5c104040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 -- 172.21.15.26:0/4017015409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 msgr2=0x55b5cc965ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc965ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009fd0 tx=0x7f8a640118d0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 -- 172.21.15.26:0/4017015409 shutdown_connections 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc96d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc96abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/4017015409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc965ae0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 -- 172.21.15.26:0/4017015409 >> 172.21.15.26:0/4017015409 conn(0x55b5cc960d30 msgr2=0x55b5cc961130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 -- 172.21.15.26:0/4017015409 shutdown_connections 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 -- 172.21.15.26:0/4017015409 wait complete. 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.731+0000 7f8a74dc2740 1 -- start start 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 Processor -- start 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc973b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9660b0 0x55b5cc9740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc96b220 0x55b5cc97b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b5cc5ac790 con 0x55b5cc9660b0 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b5cc882890 con 0x55b5cc9656c0 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b5cc9725e0 con 0x55b5cc96b220 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a74dc2740 10 client.0 _ll_register_callbacks cb 0x55b5cc5daa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc973b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9660b0 0x55b5cc9740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc973b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57744/0 (socket says 172.21.15.26:57744) 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a6bfff640 1 -- 172.21.15.26:0/2070122907 learned_addr learned my addr 172.21.15.26:0/2070122907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc96b220 0x55b5cc97b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2070122907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc96b220 msgr2=0x55b5cc97b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc96b220 0x55b5cc97b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 msgr2=0x55b5cc973b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc973b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2070122907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5cc6a7620 con 0x55b5cc9660b0 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc973b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9660b0 0x55b5cc9740d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c19a0f0 tx=0x7f8a5400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faba9359f3597956 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.732+0000 7f8a4affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a4affd640 1 -- 172.21.15.26:0/2070122907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54003d90 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a4affd640 1 -- 172.21.15.26:0/2070122907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8a54005020 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a64004050 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a4affd640 1 -- 172.21.15.26:0/2070122907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a54005220 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 10 client.17262 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5cc97e1c0 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a4affd640 1 -- 172.21.15.26:0/2070122907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a64004050 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a4affd640 1 -- 172.21.15.26:0/2070122907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8a54024050 con 0x55b5cc9660b0 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 10 client.17262 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 20 client.17262 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc97f840 0x55b5cc981ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 4 client.17262 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5cc8d8b80 con 0x55b5cc97f840 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.733+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc97f840 0x55b5cc981ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.734+0000 7f8a4affd640 10 client.17262 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.734+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc97f840 0x55b5cc981ca0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68eb4e00bd25552 server_cookie=83a9345ca4b6cad5 in_seq=0 out_seq=0 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a4affd640 1 -- 172.21.15.26:0/2070122907 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b5cc8d8b80 con 0x55b5cc97f840 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a4affd640 10 client.17262 handle_command_reply: tid=0 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 client.17262 shutdown 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9660b0 msgr2=0x55b5cc9740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9660b0 0x55b5cc9740d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c19a0f0 tx=0x7f8a5400bd80 comp rx=0 tx=0).stop 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 shutdown_connections 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc97f840 0x55b5cc981ca0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc96b220 0x55b5cc97b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9660b0 0x55b5cc9740d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a4affd640 0 client.17262 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2070122907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9656c0 0x55b5cc973b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 >> 172.21.15.26:0/2070122907 conn(0x55b5cc960d30 msgr2=0x55b5cc95ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 shutdown_connections 2024-05-20T21:02:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2070122907 wait complete. 2024-05-20T21:02:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.735+0000 7f8a74dc2740 20 client.17262 trim_cache size 0 max 16384 2024-05-20T21:02:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 Processor -- start 2024-05-20T21:02:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 -- start start 2024-05-20T21:02:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 0x55b5cc972980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc97b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc96b220 0x55b5cc96a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1aa600 con 0x55b5cc9656c0 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a7ed0 con 0x55b5cc96b220 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a5d20 con 0x55b5cc9660b0 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 0x55b5cc972980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 0x55b5cc972980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39900/0 (socket says 172.21.15.26:39900) 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 -- 172.21.15.26:0/3373698077 learned_addr learned my addr 172.21.15.26:0/3373698077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc96b220 0x55b5cc96a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc97b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 -- 172.21.15.26:0/3373698077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 msgr2=0x55b5cc97b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc97b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 -- 172.21.15.26:0/3373698077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc96b220 msgr2=0x55b5cc96a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc96b220 0x55b5cc96a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.752+0000 7f8a6bfff640 1 -- 172.21.15.26:0/3373698077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5cc966d80 con 0x55b5cc9656c0 2024-05-20T21:02:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.753+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 0x55b5cc972980 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8a64005c10 tx=0x7f8a64009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9319e182c3e39b4d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.753+0000 7f8a617fa640 1 -- 172.21.15.26:0/3373698077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640040d0 con 0x55b5cc9656c0 2024-05-20T21:02:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.753+0000 7f8a617fa640 1 -- 172.21.15.26:0/3373698077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a6400f4e0 con 0x55b5cc9656c0 2024-05-20T21:02:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.753+0000 7f8a617fa640 1 -- 172.21.15.26:0/3373698077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64028450 con 0x55b5cc9656c0 2024-05-20T21:02:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 -- 172.21.15.26:0/3373698077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 msgr2=0x55b5cc972980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 0x55b5cc972980 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8a64005c10 tx=0x7f8a64009500 comp rx=0 tx=0).stop 2024-05-20T21:02:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 -- 172.21.15.26:0/3373698077 shutdown_connections 2024-05-20T21:02:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc96b220 0x55b5cc96a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9660b0 0x55b5cc97b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/3373698077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc9656c0 0x55b5cc972980 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 -- 172.21.15.26:0/3373698077 >> 172.21.15.26:0/3373698077 conn(0x55b5cc960d30 msgr2=0x55b5cc961110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 -- 172.21.15.26:0/3373698077 shutdown_connections 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 -- 172.21.15.26:0/3373698077 wait complete. 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 -- start start 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 Processor -- start 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.754+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9656c0 0x55b5cc977d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 0x55b5cc978280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc9e8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a6390 con 0x55b5cc96b220 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a7800 con 0x55b5cc9660b0 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a6a60 con 0x55b5cc9656c0 2024-05-20T21:02:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 10 client.0 _ll_register_callbacks cb 0x7f8a540042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 0x55b5cc978280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc9e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 0x55b5cc978280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57762/0 (socket says 172.21.15.26:57762) 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2418644667 learned_addr learned my addr 172.21.15.26:0/2418644667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9656c0 0x55b5cc977d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2418644667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9656c0 msgr2=0x55b5cc977d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9656c0 0x55b5cc977d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2418644667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 msgr2=0x55b5cc9e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc9e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 -- 172.21.15.26:0/2418644667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5cc97b350 con 0x55b5cc9660b0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a4affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a62ffd640 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 0x55b5cc978280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55b5cc975070 tx=0x7f8a5401bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2db50450d7bdb4f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a4affd640 1 -- 172.21.15.26:0/2418644667 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54033060 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.755+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a6400f4e0 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a4affd640 1 -- 172.21.15.26:0/2418644667 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f8a54003d90 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a4affd640 1 -- 172.21.15.26:0/2418644667 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a540053c0 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 10 client.26738 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5cc9e99f0 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a4affd640 1 -- 172.21.15.26:0/2418644667 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a6400f4e0 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a4affd640 1 -- 172.21.15.26:0/2418644667 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8a54037030 con 0x55b5cc9660b0 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 10 client.26738 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 10 client.26738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 20 client.26738 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc9ea3c0 0x55b5cc9ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 4 client.26738 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:02:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5cc8d2360 con 0x55b5cc9ea3c0 2024-05-20T21:02:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.756+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc9ea3c0 0x55b5cc9ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.757+0000 7f8a6bfff640 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc9ea3c0 0x55b5cc9ec7e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c614a647d6a7cf server_cookie=9db9ba57220d1120 in_seq=0 out_seq=0 2024-05-20T21:02:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.757+0000 7f8a4affd640 10 client.26738 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a4affd640 1 -- 172.21.15.26:0/2418644667 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b5cc8d2360 con 0x55b5cc9ea3c0 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a4affd640 10 client.26738 handle_command_reply: tid=0 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 client.26738 shutdown 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 msgr2=0x55b5cc978280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 0x55b5cc978280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55b5cc975070 tx=0x7f8a5401bd50 comp rx=0 tx=0).stop 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 shutdown_connections 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b5cc9ea3c0 0x55b5cc9ec7e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5cc96b220 0x55b5cc9e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b5cc9660b0 0x55b5cc978280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a4affd640 0 client.26738 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 --2- 172.21.15.26:0/2418644667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5cc9656c0 0x55b5cc977d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 >> 172.21.15.26:0/2418644667 conn(0x55b5cc960d30 msgr2=0x55b5cc972980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 shutdown_connections 2024-05-20T21:02:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.810+0000 7f8a74dc2740 1 -- 172.21.15.26:0/2418644667 wait complete. 2024-05-20T21:02:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a74dc2740 20 client.26738 trim_cache size 0 max 16384 2024-05-20T21:02:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:02:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:02:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8a580b2920 msgr2=0x7f8a580b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8a580b2920 0x7f8a580b4de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c1005a0 tx=0x7f8a5c104040 comp rx=0 tx=0).stop 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 msgr2=0x7f8a6c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 0x7f8a6c198be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009a20 tx=0x7f8a5c0040c0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 shutdown_connections 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8a580b2920 0x7f8a580b4de0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a6c1087f0 0x7f8a6c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a6c105ec0 0x7f8a6c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 --2- 172.21.15.26:0/538775648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c103590 0x7f8a6c198be0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.811+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 >> 172.21.15.26:0/538775648 conn(0x7f8a6c0fcf20 msgr2=0x7f8a6c104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.812+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 shutdown_connections 2024-05-20T21:02:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:19.812+0000 7f8a720a0640 1 -- 172.21.15.26:0/538775648 wait complete. 2024-05-20T21:02:19.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:02:20.251 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:20.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 118562 1.95 MB/sec execute 146 sec latency 41.759 ms 2024-05-20T21:02:21.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:21 smithi144 ceph-mon[35660]: pgmap v575: 97 pgs: 97 active+clean; 30 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T21:02:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:21 smithi055 ceph-mon[35213]: pgmap v575: 97 pgs: 97 active+clean; 30 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T21:02:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:21 smithi026 ceph-mon[31990]: pgmap v575: 97 pgs: 97 active+clean; 30 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T21:02:21.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119151 1.94 MB/sec execute 147 sec latency 66.462 ms 2024-05-20T21:02:22.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119670 1.94 MB/sec execute 148 sec latency 83.526 ms 2024-05-20T21:02:23.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:23 smithi026 ceph-mon[31990]: pgmap v576: 97 pgs: 97 active+clean; 31 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-05-20T21:02:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:23 smithi144 ceph-mon[35660]: pgmap v576: 97 pgs: 97 active+clean; 31 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-05-20T21:02:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:23 smithi055 ceph-mon[35213]: pgmap v576: 97 pgs: 97 active+clean; 31 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-05-20T21:02:23.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120400 1.97 MB/sec execute 149 sec latency 88.353 ms 2024-05-20T21:02:24.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120927 1.99 MB/sec execute 150 sec latency 43.586 ms 2024-05-20T21:02:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 -- 172.21.15.26:0/3835925285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 msgr2=0x7f17dc06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3835925285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc06b6a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009990 tx=0x7f17d0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 -- 172.21.15.26:0/3835925285 shutdown_connections 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3835925285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17dc06bbe0 0x7f17dc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3835925285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17dc1095b0 0x7f17dc109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3835925285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc06b6a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 -- 172.21.15.26:0/3835925285 >> 172.21.15.26:0/3835925285 conn(0x7f17dc0793f0 msgr2=0x7f17dc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 -- 172.21.15.26:0/3835925285 shutdown_connections 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.934+0000 7f17e1e30700 1 -- 172.21.15.26:0/3835925285 wait complete. 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 Processor -- start 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 -- start start 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17dc06bbe0 0x7f17dc19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17dc1095b0 0x7f17dc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1135d0 con 0x7f17dc06b220 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc113450 con 0x7f17dc06bbe0 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17e1e30700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc113750 con 0x7f17dc1095b0 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39918/0 (socket says 172.21.15.26:39918) 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17db7fe700 1 -- 172.21.15.26:0/3482539214 learned_addr learned my addr 172.21.15.26:0/3482539214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17dbfff700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17dc1095b0 0x7f17dc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.935+0000 7f17daffd700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17dc06bbe0 0x7f17dc19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17db7fe700 1 -- 172.21.15.26:0/3482539214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17dc1095b0 msgr2=0x7f17dc199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17dc1095b0 0x7f17dc199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17db7fe700 1 -- 172.21.15.26:0/3482539214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17dc06bbe0 msgr2=0x7f17dc19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17dc06bbe0 0x7f17dc19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17db7fe700 1 -- 172.21.15.26:0/3482539214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc19a0f0 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc19f7d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d950 tx=0x7f17cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17d8ff9700 1 -- 172.21.15.26:0/3482539214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc016070 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc19a3e0 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17d8ff9700 1 -- 172.21.15.26:0/3482539214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17cc00fd20 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17d8ff9700 1 -- 172.21.15.26:0/3482539214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00e3b0 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.936+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a66a0 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.939+0000 7f17d8ff9700 1 -- 172.21.15.26:0/3482539214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f17cc019070 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.939+0000 7f17d8ff9700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f17c40b2880 0x7f17c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.940+0000 7f17daffd700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f17c40b2880 0x7f17c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.940+0000 7f17d8ff9700 1 -- 172.21.15.26:0/3482539214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f17cc0e1d00 con 0x7f17dc06b220 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.940+0000 7f17daffd700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f17c40b2880 0x7f17c40b4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f17dc100a40 tx=0x7f17d000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.940+0000 7f17ec6c6b80 1 Processor -- start 2024-05-20T21:02:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.940+0000 7f17ec6c6b80 1 -- start start 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc919bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc919f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558bc90fac90 con 0x558bc919b860 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558bc90ea830 con 0x558bc919ff60 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bc90ee870 con 0x558bc919c210 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc919bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc919bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39924/0 (socket says 172.21.15.26:39924) 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 -- 172.21.15.26:0/4176999252 learned_addr learned my addr 172.21.15.26:0/4176999252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17daffd700 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc919f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17dbfff700 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 -- 172.21.15.26:0/4176999252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 msgr2=0x558bc919f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc919f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 -- 172.21.15.26:0/4176999252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 msgr2=0x558bc91a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 -- 172.21.15.26:0/4176999252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc903a8d0 con 0x558bc919b860 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17dbfff700 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17db7fe700 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc919bc40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00ff80 tx=0x7f17cc0a91c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55769068e02cd79d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17c1ffb700 1 -- 172.21.15.26:0/4176999252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc020040 con 0x558bc919b860 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17c1ffb700 1 -- 172.21.15.26:0/4176999252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17cc0ad670 con 0x558bc919b860 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.941+0000 7f17c1ffb700 1 -- 172.21.15.26:0/4176999252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0b1c90 con 0x558bc919b860 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/4176999252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 msgr2=0x558bc919bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc919bc40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00ff80 tx=0x7f17cc0a91c0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/4176999252 shutdown_connections 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc919bc40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/4176999252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc919f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/4176999252 >> 172.21.15.26:0/4176999252 conn(0x558bc90c9080 msgr2=0x558bc9195e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/4176999252 shutdown_connections 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.942+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/4176999252 wait complete. 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 -- start start 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 Processor -- start 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc91a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558bc8ff0fa0 con 0x558bc919b860 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558bc908c7c0 con 0x558bc919ff60 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bc91a8060 con 0x558bc919c210 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.943+0000 7f17ec6c6b80 10 client.0 _ll_register_callbacks cb 0x558bc9085920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc91a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc91a7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39940/0 (socket says 172.21.15.26:39940) 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17db7fe700 1 -- 172.21.15.26:0/1226278045 learned_addr learned my addr 172.21.15.26:0/1226278045 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17daffd700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 -- 172.21.15.26:0/1226278045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 msgr2=0x558bc91a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 -- 172.21.15.26:0/1226278045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 msgr2=0x558bc91a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc91a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 -- 172.21.15.26:0/1226278045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc9089690 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17daffd700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17db7fe700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc91a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17dbfff700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a4c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17d4008e60 tx=0x7f17d4010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b4e156ff99f746 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17baffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17baffd700 1 -- 172.21.15.26:0/1226278045 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4010bc0 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17baffd700 1 -- 172.21.15.26:0/1226278045 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f17d4010d60 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.944+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc0ad670 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17baffd700 1 -- 172.21.15.26:0/1226278045 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17d4016470 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 10 client.26744 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558bc91b27b0 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17baffd700 1 -- 172.21.15.26:0/1226278045 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17cc0ad670 con 0x558bc919ff60 2024-05-20T21:02:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17baffd700 1 -- 172.21.15.26:0/1226278045 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f17d40206f0 con 0x558bc919ff60 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 10 client.26744 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 10 client.26744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 20 client.26744 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc91b7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 4 client.26744 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bc91679e0 con 0x558bc91b4e40 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.945+0000 7f17db7fe700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc91b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.946+0000 7f17db7fe700 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc91b7230 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4d474ccdd311ee0 server_cookie=9fc519b48b237cc0 in_seq=0 out_seq=0 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.946+0000 7f17baffd700 10 client.26744 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17baffd700 1 -- 172.21.15.26:0/1226278045 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558bc91679e0 con 0x558bc91b4e40 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17baffd700 10 client.26744 handle_command_reply: tid=0 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 client.26744 shutdown 2024-05-20T21:02:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 msgr2=0x558bc91a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a4c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17d4008e60 tx=0x7f17d4010430 comp rx=0 tx=0).stop 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 shutdown_connections 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc91b7230 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17baffd700 0 client.26744 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919b860 0x558bc91a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919ff60 0x558bc91a4c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/1226278045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 >> 172.21.15.26:0/1226278045 conn(0x558bc90c9080 msgr2=0x558bc9195510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 shutdown_connections 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/1226278045 wait complete. 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.947+0000 7f17ec6c6b80 20 client.26744 trim_cache size 0 max 16384 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 Processor -- start 2024-05-20T21:02:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 -- start start 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0680 con 0x558bc919ff60 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1adf10 con 0x558bc919b860 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.965+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1abd60 con 0x558bc919c210 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17daffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91bcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39956/0 (socket says 172.21.15.26:39956) 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 -- 172.21.15.26:0/3778043332 learned_addr learned my addr 172.21.15.26:0/3778043332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57802/0 (socket says 172.21.15.26:57802) 2024-05-20T21:02:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 -- 172.21.15.26:0/3778043332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 msgr2=0x558bc91a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 -- 172.21.15.26:0/3778043332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 msgr2=0x558bc91b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 -- 172.21.15.26:0/3778043332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc91a44f0 con 0x558bc919ff60 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17dbfff700 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91bcb20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007580 tx=0x7f17d4010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c73224183f52a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17c17fa700 1 -- 172.21.15.26:0/3778043332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4010e40 con 0x558bc919ff60 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17c17fa700 1 -- 172.21.15.26:0/3778043332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17d401b070 con 0x558bc919ff60 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.966+0000 7f17c17fa700 1 -- 172.21.15.26:0/3778043332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4016610 con 0x558bc919ff60 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3778043332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 msgr2=0x558bc91bcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91bcb20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007580 tx=0x7f17d4010c50 comp rx=0 tx=0).stop 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3778043332 shutdown_connections 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91bcb20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3778043332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3778043332 >> 172.21.15.26:0/3778043332 conn(0x558bc90c9080 msgr2=0x558bc91a5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3778043332 shutdown_connections 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.967+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3778043332 wait complete. 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 -- start start 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 Processor -- start 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ac3d0 con 0x558bc919ff60 2024-05-20T21:02:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ad840 con 0x558bc919b860 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1acaa0 con 0x558bc919c210 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17ec6c6b80 10 client.0 _ll_register_callbacks cb 0x558bc9085920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57804/0 (socket says 172.21.15.26:57804) 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39972/0 (socket says 172.21.15.26:39972) 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17db7fe700 1 -- 172.21.15.26:0/3503504131 learned_addr learned my addr 172.21.15.26:0/3503504131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.968+0000 7f17daffd700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17db7fe700 1 -- 172.21.15.26:0/3503504131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 msgr2=0x558bc91aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17db7fe700 1 -- 172.21.15.26:0/3503504131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 msgr2=0x558bc91ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17db7fe700 1 -- 172.21.15.26:0/3503504131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc91aba60 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17dbfff700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17daffd700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17db7fe700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91ad970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0a8fc0 tx=0x7f17cc0a8f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352295870e8407d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17baffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17baffd700 1 -- 172.21.15.26:0/3503504131 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc020040 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17baffd700 1 -- 172.21.15.26:0/3503504131 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f17cc0a9520 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d401b070 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17baffd700 1 -- 172.21.15.26:0/3503504131 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17cc0ca350 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17ec6c6b80 10 client.26750 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558bc921f9d0 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.969+0000 7f17baffd700 1 -- 172.21.15.26:0/3503504131 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17d401b070 con 0x558bc919b860 2024-05-20T21:02:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17baffd700 1 -- 172.21.15.26:0/3503504131 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f17cc0ad120 con 0x558bc919b860 2024-05-20T21:02:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17ec6c6b80 10 client.26750 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17ec6c6b80 10 client.26750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:02:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17ec6c6b80 20 client.26750 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:25.074 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc9221bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17ec6c6b80 4 client.26750 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bc90fcbe0 con 0x558bc91b4e40 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17daffd700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc9221bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17baffd700 10 client.26750 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17daffd700 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc9221bb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de83ced344f3785f server_cookie=e6f75a3356abeaca in_seq=0 out_seq=0 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17baffd700 1 -- 172.21.15.26:0/3503504131 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bc90fcbe0 con 0x558bc91b4e40 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.970+0000 7f17baffd700 10 client.26750 handle_command_reply: tid=0 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 client.26750 shutdown 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 msgr2=0x558bc91ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91ad970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0a8fc0 tx=0x7f17cc0a8f00 comp rx=0 tx=0).stop 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 shutdown_connections 2024-05-20T21:02:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x558bc91b4e40 0x558bc9221bb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558bc919ff60 0x558bc91ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17baffd700 0 client.26750 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558bc919b860 0x558bc91ad970 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 --2- 172.21.15.26:0/3503504131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bc919c210 0x558bc91aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 >> 172.21.15.26:0/3503504131 conn(0x558bc90c9080 msgr2=0x558bc919af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 shutdown_connections 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 1 -- 172.21.15.26:0/3503504131 wait complete. 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17ec6c6b80 20 client.26750 trim_cache size 0 max 16384 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f17c40b2880 msgr2=0x7f17c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f17c40b2880 0x7f17c40b4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f17dc100a40 tx=0x7f17d000b540 comp rx=0 tx=0).stop 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.971+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 msgr2=0x7f17dc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc19f7d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d950 tx=0x7f17cc00dc60 comp rx=0 tx=0).stop 2024-05-20T21:02:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 shutdown_connections 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f17c40b2880 0x7f17c40b4d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17dc06b220 0x7f17dc19f7d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17dc1095b0 0x7f17dc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 --2- 172.21.15.26:0/3482539214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17dc06bbe0 0x7f17dc19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 >> 172.21.15.26:0/3482539214 conn(0x7f17dc0793f0 msgr2=0x7f17dc101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 shutdown_connections 2024-05-20T21:02:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:24.972+0000 7f17e1e30700 1 -- 172.21.15.26:0/3482539214 wait complete. 2024-05-20T21:02:25.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:25 smithi026 ceph-mon[31990]: pgmap v577: 97 pgs: 97 active+clean; 27 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-05-20T21:02:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:25 smithi144 ceph-mon[35660]: pgmap v577: 97 pgs: 97 active+clean; 27 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-05-20T21:02:25.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:25 smithi055 ceph-mon[35213]: pgmap v577: 97 pgs: 97 active+clean; 27 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-05-20T21:02:25.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 121389 1.98 MB/sec execute 151 sec latency 17.863 ms 2024-05-20T21:02:25.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:02:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:26.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 122085 2.00 MB/sec execute 152 sec latency 19.035 ms 2024-05-20T21:02:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:27 smithi144 ceph-mon[35660]: pgmap v578: 97 pgs: 97 active+clean; 26 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:02:27.429 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:27 smithi026 ceph-mon[31990]: pgmap v578: 97 pgs: 97 active+clean; 26 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:02:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:27 smithi055 ceph-mon[35213]: pgmap v578: 97 pgs: 97 active+clean; 26 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:02:27.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 122788 1.99 MB/sec execute 153 sec latency 13.689 ms 2024-05-20T21:02:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 -- 172.21.15.26:0/3880637053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 msgr2=0x7fa9281028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 --2- 172.21.15.26:0/3880637053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa9281028c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098f0 tx=0x7fa918009c00 comp rx=0 tx=0).stop 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 -- 172.21.15.26:0/3880637053 shutdown_connections 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 --2- 172.21.15.26:0/3880637053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 --2- 172.21.15.26:0/3880637053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108520 0x7fa928108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 --2- 172.21.15.26:0/3880637053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa9281028c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 -- 172.21.15.26:0/3880637053 >> 172.21.15.26:0/3880637053 conn(0x7fa9280793f0 msgr2=0x7fa928079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 -- 172.21.15.26:0/3880637053 shutdown_connections 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.009+0000 7fa9302a5700 1 -- 172.21.15.26:0/3880637053 wait complete. 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 Processor -- start 2024-05-20T21:02:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 -- start start 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa92819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108520 0x7fa9281a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa928112480 con 0x7fa928102e00 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa928112300 con 0x7fa928102440 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa9302a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa928112600 con 0x7fa928108520 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa92d840700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa92d840700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92819aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40004/0 (socket says 172.21.15.26:40004) 2024-05-20T21:02:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa92d840700 1 -- 172.21.15.26:0/1523009712 learned_addr learned my addr 172.21.15.26:0/1523009712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa92e041700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa92819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.010+0000 7fa92e842700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108520 0x7fa9281a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92d840700 1 -- 172.21.15.26:0/1523009712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108520 msgr2=0x7fa9281a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92d840700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108520 0x7fa9281a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92d840700 1 -- 172.21.15.26:0/1523009712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 msgr2=0x7fa92819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92d840700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa92819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92d840700 1 -- 172.21.15.26:0/1523009712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281a6070 con 0x7fa928102e00 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92e041700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa92819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa92d840700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92819aff0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa91800fa00 tx=0x7fa91800fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa91f7fe700 1 -- 172.21.15.26:0/1523009712 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918027070 con 0x7fa928102e00 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa91f7fe700 1 -- 172.21.15.26:0/1523009712 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa918005040 con 0x7fa928102e00 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9281a6240 con 0x7fa928102e00 2024-05-20T21:02:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa91f7fe700 1 -- 172.21.15.26:0/1523009712 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918004930 con 0x7fa928102e00 2024-05-20T21:02:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.011+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281a66f0 con 0x7fa928102e00 2024-05-20T21:02:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.014+0000 7fa91f7fe700 1 -- 172.21.15.26:0/1523009712 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa91802a070 con 0x7fa928102e00 2024-05-20T21:02:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.015+0000 7fa91f7fe700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9140b2900 0x7fa9140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.015+0000 7fa93ab3bb80 1 Processor -- start 2024-05-20T21:02:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.015+0000 7fa92e041700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9140b2900 0x7fa9140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.015+0000 7fa91f7fe700 1 -- 172.21.15.26:0/1523009712 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa9180e2d70 con 0x7fa928102e00 2024-05-20T21:02:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.015+0000 7fa93ab3bb80 1 -- start start 2024-05-20T21:02:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f050a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f050e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f050ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d3f0463d70 con 0x55d3f0509e90 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d3f0453910 con 0x55d3f050a840 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3f0457950 con 0x55d3f050e8e0 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92e041700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9140b2900 0x7fa9140b4dc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa924005950 tx=0x7fa92400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f050e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f050e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57834/0 (socket says 172.21.15.26:57834) 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 -- 172.21.15.26:0/3681463766 learned_addr learned my addr 172.21.15.26:0/3681463766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92e041700 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f050a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92e842700 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f050ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 -- 172.21.15.26:0/3681463766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 msgr2=0x55d3f050ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f050ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 -- 172.21.15.26:0/3681463766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 msgr2=0x55d3f050a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f050a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 -- 172.21.15.26:0/3681463766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f03fcfb0 con 0x55d3f050a840 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.016+0000 7fa92d840700 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f050e2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9180a6440 tx=0x7fa918005390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cfbd83dc0749b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa91cd70700 1 -- 172.21.15.26:0/3681463766 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180ae770 con 0x55d3f050a840 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa91cd70700 1 -- 172.21.15.26:0/3681463766 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9180b3190 con 0x55d3f050a840 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa91cd70700 1 -- 172.21.15.26:0/3681463766 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180aa8b0 con 0x55d3f050a840 2024-05-20T21:02:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa92e041700 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f050a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/3681463766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 msgr2=0x55d3f050e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f050e2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9180a6440 tx=0x7fa918005390 comp rx=0 tx=0).stop 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/3681463766 shutdown_connections 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f050a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f050ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/3681463766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f050e2c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.017+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/3681463766 >> 172.21.15.26:0/3681463766 conn(0x55d3f0431240 msgr2=0x55d3f05003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/3681463766 shutdown_connections 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/3681463766 wait complete. 2024-05-20T21:02:28.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 -- start start 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 Processor -- start 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f0515de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0513620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d3f0359fa0 con 0x55d3f0509e90 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d3f03bbda0 con 0x55d3f050a840 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3f0516850 con 0x55d3f050e8e0 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.018+0000 7fa93ab3bb80 10 client.0 _ll_register_callbacks cb 0x55d3f03ee950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92e041700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f0515de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05130e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57848/0 (socket says 172.21.15.26:57848) 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92e041700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f0515de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40030/0 (socket says 172.21.15.26:40030) 2024-05-20T21:02:28.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 -- 172.21.15.26:0/2972935523 learned_addr learned my addr 172.21.15.26:0/2972935523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92e842700 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0513620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 -- 172.21.15.26:0/2972935523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 msgr2=0x55d3f0513620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0513620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 -- 172.21.15.26:0/2972935523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 msgr2=0x55d3f0515de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f0515de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 -- 172.21.15.26:0/2972935523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f03a38d0 con 0x55d3f050a840 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa92d840700 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05130e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa9180e0860 tx=0x7fa9180aa980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b5ce778d362639 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa911ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa911ffb700 1 -- 172.21.15.26:0/2972935523 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180c3020 con 0x55d3f050a840 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa911ffb700 1 -- 172.21.15.26:0/2972935523 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa91800b790 con 0x55d3f050a840 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9180b3190 con 0x55d3f050a840 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa911ffb700 1 -- 172.21.15.26:0/2972935523 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa91800b990 con 0x55d3f050a840 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa93ab3bb80 10 client.26762 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.019+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d3f040d760 con 0x55d3f050a840 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa911ffb700 1 -- 172.21.15.26:0/2972935523 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9180b3190 con 0x55d3f050a840 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa911ffb700 1 -- 172.21.15.26:0/2972935523 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fa9180a5d30 con 0x55d3f050a840 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa93ab3bb80 10 client.26762 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa93ab3bb80 10 client.26762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa93ab3bb80 20 client.26762 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f05232a0 0x55d3f0525690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa93ab3bb80 4 client.26762 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f0465cc0 con 0x55d3f05232a0 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.020+0000 7fa92e041700 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f05232a0 0x55d3f0525690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.021+0000 7fa92e041700 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f05232a0 0x55d3f0525690 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba156f60c233a11b server_cookie=89debb69544b4050 in_seq=0 out_seq=0 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.021+0000 7fa911ffb700 10 client.26762 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.021+0000 7fa911ffb700 1 -- 172.21.15.26:0/2972935523 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d3f0465cc0 con 0x55d3f05232a0 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.021+0000 7fa911ffb700 10 client.26762 handle_command_reply: tid=0 2024-05-20T21:02:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.021+0000 7fa93ab3bb80 1 client.26762 shutdown 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 msgr2=0x55d3f05130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05130e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa9180e0860 tx=0x7fa9180aa980 comp rx=0 tx=0).stop 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 shutdown_connections 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f05232a0 0x55d3f0525690 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f0515de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0513620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/2972935523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05130e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa911ffb700 0 client.26762 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 >> 172.21.15.26:0/2972935523 conn(0x55d3f0431240 msgr2=0x55d3f050fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 shutdown_connections 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/2972935523 wait complete. 2024-05-20T21:02:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.022+0000 7fa93ab3bb80 20 client.26762 trim_cache size 0 max 16384 2024-05-20T21:02:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 Processor -- start 2024-05-20T21:02:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 -- start start 2024-05-20T21:02:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f051f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0511f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ae2b0 con 0x55d3f0509e90 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281abb40 con 0x55d3f050a840 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281a99b0 con 0x55d3f050e8e0 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92e041700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f051f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92e041700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f051f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40036/0 (socket says 172.21.15.26:40036) 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92e041700 1 -- 172.21.15.26:0/1120490668 learned_addr learned my addr 172.21.15.26:0/1120490668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92d840700 1 -- 172.21.15.26:0/1120490668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 msgr2=0x55d3f0511f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92d840700 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0511f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92d840700 1 -- 172.21.15.26:0/1120490668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 msgr2=0x55d3f051f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92d840700 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f051f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.041+0000 7fa92d840700 1 -- 172.21.15.26:0/1120490668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f0515920 con 0x55d3f050a840 2024-05-20T21:02:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.042+0000 7fa92e041700 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f051f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.042+0000 7fa92d840700 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05119d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa9180aa350 tx=0x7fa918009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a16fe0eaebd358 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.042+0000 7fa913fff700 1 -- 172.21.15.26:0/1120490668 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180c3070 con 0x55d3f050a840 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.042+0000 7fa913fff700 1 -- 172.21.15.26:0/1120490668 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9180b2f10 con 0x55d3f050a840 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.042+0000 7fa913fff700 1 -- 172.21.15.26:0/1120490668 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180a4ae0 con 0x55d3f050a840 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1120490668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 msgr2=0x55d3f05119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05119d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa9180aa350 tx=0x7fa918009c00 comp rx=0 tx=0).stop 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1120490668 shutdown_connections 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f0509e90 0x55d3f051f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f050e8e0 0x55d3f0511f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1120490668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f05119d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1120490668 >> 172.21.15.26:0/1120490668 conn(0x55d3f0431240 msgr2=0x55d3f0500380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1120490668 shutdown_connections 2024-05-20T21:02:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1120490668 wait complete. 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 -- start start 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 Processor -- start 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f0509e90 0x55d3f051bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f0517250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.043+0000 7fa93ab3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f050e8e0 0x55d3f0517790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281aa000 con 0x55d3f050e8e0 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ab470 con 0x55d3f050a840 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa93ab3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281aa6d0 con 0x55d3f0509e90 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa93ab3bb80 10 client.0 _ll_register_callbacks cb 0x55d3f03ee950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f0517250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92e842700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f050e8e0 0x55d3f0517790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f0517250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57854/0 (socket says 172.21.15.26:57854) 2024-05-20T21:02:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 -- 172.21.15.26:0/1440836131 learned_addr learned my addr 172.21.15.26:0/1440836131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92e041700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f0509e90 0x55d3f051bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 -- 172.21.15.26:0/1440836131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f0509e90 msgr2=0x55d3f051bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f0509e90 0x55d3f051bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 -- 172.21.15.26:0/1440836131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f050e8e0 msgr2=0x55d3f0517790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f050e8e0 0x55d3f0517790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 -- 172.21.15.26:0/1440836131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f0517d00 con 0x55d3f050a840 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92e842700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f050e8e0 0x55d3f0517790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa92d840700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f0517250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9180a6960 tx=0x7fa9180a5fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc9959b8fd525c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa911ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa911ffb700 1 -- 172.21.15.26:0/1440836131 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180c3070 con 0x55d3f050a840 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa911ffb700 1 -- 172.21.15.26:0/1440836131 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fa91800b790 con 0x55d3f050a840 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.044+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9180b2f10 con 0x55d3f050a840 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa911ffb700 1 -- 172.21.15.26:0/1440836131 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9180a9620 con 0x55d3f050a840 2024-05-20T21:02:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 10 client.26768 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d3f0517d00 con 0x55d3f050a840 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa911ffb700 1 -- 172.21.15.26:0/1440836131 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9180b2f10 con 0x55d3f050a840 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa911ffb700 1 -- 172.21.15.26:0/1440836131 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fa918084ad0 con 0x55d3f050a840 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 10 client.26768 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 10 client.26768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 20 client.26768 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f058dc50 0x55d3f0590040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 4 client.26768 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3f045f4a0 con 0x55d3f058dc50 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.045+0000 7fa92e041700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f058dc50 0x55d3f0590040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.046+0000 7fa92e041700 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f058dc50 0x55d3f0590040 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b7b051a8557091 server_cookie=16762826f4b3e8f7 in_seq=0 out_seq=0 2024-05-20T21:02:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.046+0000 7fa911ffb700 10 client.26768 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:28.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa911ffb700 1 -- 172.21.15.26:0/1440836131 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3f045f4a0 con 0x55d3f058dc50 2024-05-20T21:02:28.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa911ffb700 10 client.26768 handle_command_reply: tid=0 2024-05-20T21:02:28.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 client.26768 shutdown 2024-05-20T21:02:28.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 msgr2=0x55d3f0517250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f0517250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9180a6960 tx=0x7fa9180a5fe0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 shutdown_connections 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d3f058dc50 0x55d3f0590040 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3f050e8e0 0x55d3f0517790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3f0509e90 0x55d3f051bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 --2- 172.21.15.26:0/1440836131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3f050a840 0x55d3f0517250 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa911ffb700 0 client.26768 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 >> 172.21.15.26:0/1440836131 conn(0x55d3f0431240 msgr2=0x55d3f0511990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 shutdown_connections 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 1 -- 172.21.15.26:0/1440836131 wait complete. 2024-05-20T21:02:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.047+0000 7fa93ab3bb80 20 client.26768 trim_cache size 0 max 16384 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9140b2900 msgr2=0x7fa9140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9140b2900 0x7fa9140b4dc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa924005950 tx=0x7fa92400b410 comp rx=0 tx=0).stop 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 msgr2=0x7fa92819aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92819aff0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa91800fa00 tx=0x7fa91800fa30 comp rx=0 tx=0).stop 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 shutdown_connections 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108520 0x7fa9281a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa928102440 0x7fa92819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa9140b2900 0x7fa9140b4dc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 --2- 172.21.15.26:0/1523009712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa928102e00 0x7fa92819aff0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 >> 172.21.15.26:0/1523009712 conn(0x7fa9280793f0 msgr2=0x7fa9281006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 shutdown_connections 2024-05-20T21:02:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:28.048+0000 7fa9302a5700 1 -- 172.21.15.26:0/1523009712 wait complete. 2024-05-20T21:02:28.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 123559 2.02 MB/sec execute 154 sec latency 15.467 ms 2024-05-20T21:02:29.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:29 smithi026 ceph-mon[31990]: pgmap v579: 97 pgs: 97 active+clean; 28 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T21:02:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:29 smithi144 ceph-mon[35660]: pgmap v579: 97 pgs: 97 active+clean; 28 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T21:02:29.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:29 smithi055 ceph-mon[35213]: pgmap v579: 97 pgs: 97 active+clean; 28 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-05-20T21:02:29.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 124177 2.04 MB/sec execute 155 sec latency 17.254 ms 2024-05-20T21:02:29.949 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:02:29.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:02:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:30.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 124655 2.04 MB/sec execute 156 sec latency 16.591 ms 2024-05-20T21:02:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:31 smithi144 ceph-mon[35660]: pgmap v580: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-05-20T21:02:31.372 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:31 smithi026 ceph-mon[31990]: pgmap v580: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-05-20T21:02:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:31 smithi055 ceph-mon[35213]: pgmap v580: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-05-20T21:02:31.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 125242 2.04 MB/sec execute 157 sec latency 19.087 ms 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 -- 172.21.15.26:0/2794729315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 msgr2=0x7fc3dc06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/2794729315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc06bf30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009990 tx=0x7fc3d0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 -- 172.21.15.26:0/2794729315 shutdown_connections 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/2794729315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc06c470 0x7fc3dc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/2794729315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc1095a0 0x7fc3dc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/2794729315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc06bf30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 -- 172.21.15.26:0/2794729315 >> 172.21.15.26:0/2794729315 conn(0x7fc3dc0793b0 msgr2=0x7fc3dc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 -- 172.21.15.26:0/2794729315 shutdown_connections 2024-05-20T21:02:32.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.042+0000 7fc3e09a9700 1 -- 172.21.15.26:0/2794729315 wait complete. 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.043+0000 7fc3e09a9700 1 Processor -- start 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.043+0000 7fc3e09a9700 1 -- start start 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3e09a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3e09a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc06c470 0x7fc3dc19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3e09a9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc1095a0 0x7fc3dc1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3e09a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc113560 con 0x7fc3dc06bab0 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3e09a9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1133e0 con 0x7fc3dc1095a0 2024-05-20T21:02:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3e09a9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1136e0 con 0x7fc3dc06c470 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49178/0 (socket says 172.21.15.26:49178) 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 -- 172.21.15.26:0/1997170528 learned_addr learned my addr 172.21.15.26:0/1997170528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc1095a0 0x7fc3dc1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc06c470 0x7fc3dc19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 -- 172.21.15.26:0/1997170528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc06c470 msgr2=0x7fc3dc19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc06c470 0x7fc3dc19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 -- 172.21.15.26:0/1997170528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc1095a0 msgr2=0x7fc3dc1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc1095a0 0x7fc3dc1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 -- 172.21.15.26:0/1997170528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc1a0d70 con 0x7fc3dc06bab0 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc1095a0 0x7fc3dc1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc06c470 0x7fc3dc19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:32.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.044+0000 7fc3da59c700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc19bdb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00d980 tx=0x7fc3cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.045+0000 7fc3cb7fe700 1 -- 172.21.15.26:0/1997170528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc016070 con 0x7fc3dc06bab0 2024-05-20T21:02:32.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.045+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc1a1060 con 0x7fc3dc06bab0 2024-05-20T21:02:32.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.045+0000 7fc3cb7fe700 1 -- 172.21.15.26:0/1997170528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3cc004830 con 0x7fc3dc06bab0 2024-05-20T21:02:32.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.045+0000 7fc3cb7fe700 1 -- 172.21.15.26:0/1997170528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc005440 con 0x7fc3dc06bab0 2024-05-20T21:02:32.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.045+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a88e0 con 0x7fc3dc06bab0 2024-05-20T21:02:32.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.047+0000 7fc3cb7fe700 1 -- 172.21.15.26:0/1997170528 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc3cc0055e0 con 0x7fc3dc06bab0 2024-05-20T21:02:32.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.048+0000 7fc3cb7fe700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3c40b2880 0x7fc3c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3c40b2880 0x7fc3c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3cb7fe700 1 -- 172.21.15.26:0/1997170528 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fc3cc0e1fb0 con 0x7fc3dc06bab0 2024-05-20T21:02:32.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3eb23fb80 1 Processor -- start 2024-05-20T21:02:32.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3c40b2880 0x7fc3c40b4d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0006010 tx=0x7fc3d001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3eb23fb80 1 -- start start 2024-05-20T21:02:32.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e68e0 0x55fb212e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e7290 0x55fb212ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.049+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fb21245d10 con 0x55fb212eafe0 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fb212358b0 con 0x55fb212e7290 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fb212398f0 con 0x55fb212e68e0 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e7290 0x55fb212ea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212eb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49190/0 (socket says 172.21.15.26:49190) 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3dad9d700 1 -- 172.21.15.26:0/1056827892 learned_addr learned my addr 172.21.15.26:0/1056827892 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3da59c700 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e68e0 0x55fb212e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/1056827892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e68e0 msgr2=0x55fb212e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e68e0 0x55fb212e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/1056827892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 msgr2=0x55fb212eb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212eb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/1056827892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb21185950 con 0x55fb212e7290 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e7290 0x55fb212ea9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b650 tx=0x7fc3d001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af27062f9974d915 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3c8ff9700 1 -- 172.21.15.26:0/1056827892 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000fb20 con 0x55fb212e7290 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.050+0000 7fc3c8ff9700 1 -- 172.21.15.26:0/1056827892 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d0032070 con 0x55fb212e7290 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3c8ff9700 1 -- 172.21.15.26:0/1056827892 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0023d50 con 0x55fb212e7290 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/1056827892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e7290 msgr2=0x55fb212ea9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e7290 0x55fb212ea9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b650 tx=0x7fc3d001e000 comp rx=0 tx=0).stop 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/1056827892 shutdown_connections 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e68e0 0x55fb212e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/1056827892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e7290 0x55fb212ea9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/1056827892 >> 172.21.15.26:0/1056827892 conn(0x55fb21214100 msgr2=0x55fb212e0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/1056827892 shutdown_connections 2024-05-20T21:02:32.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.051+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/1056827892 wait complete. 2024-05-20T21:02:32.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 -- start start 2024-05-20T21:02:32.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 Processor -- start 2024-05-20T21:02:32.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212e7290 0x55fb212ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212eafe0 0x55fb212efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fb2113c020 con 0x55fb212e7290 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fb211d7840 con 0x55fb212e68e0 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fb212f3140 con 0x55fb212eafe0 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3eb23fb80 10 client.0 _ll_register_callbacks cb 0x55fb211d09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3d9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212e7290 0x55fb212ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3d9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212e7290 0x55fb212ef7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49200/0 (socket says 172.21.15.26:49200) 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/2707726201 learned_addr learned my addr 172.21.15.26:0/2707726201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.052+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212eafe0 0x55fb212efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3da59c700 1 -- 172.21.15.26:0/2707726201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212eafe0 msgr2=0x55fb212efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3da59c700 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212eafe0 0x55fb212efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3da59c700 1 -- 172.21.15.26:0/2707726201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212e7290 msgr2=0x55fb212ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3da59c700 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212e7290 0x55fb212ef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3da59c700 1 -- 172.21.15.26:0/2707726201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb211d4710 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3da59c700 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212f2470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0dfaa0 tx=0x7fc3cc004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e6b695302158e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2707726201 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0c3060 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2707726201 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fc3cc0a97d0 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d0032070 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2707726201 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3cc0c3e60 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3eb23fb80 10 client.26780 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb212fd8f0 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.053+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2707726201 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d0032070 con 0x55fb212e68e0 2024-05-20T21:02:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2707726201 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fc3cc0b1f40 con 0x55fb212e68e0 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3eb23fb80 10 client.26780 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3eb23fb80 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3eb23fb80 20 client.26780 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb212ffe80 0x55fb21302270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3eb23fb80 4 client.26780 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb212b2a60 con 0x55fb212ffe80 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.054+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb212ffe80 0x55fb21302270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.055+0000 7fc3c1ffb700 10 client.26780 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.055+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb212ffe80 0x55fb21302270 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e9184070cac0cb server_cookie=983b9f4e83c39fe3 in_seq=0 out_seq=0 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2707726201 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fb212b2a60 con 0x55fb212ffe80 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3c1ffb700 10 client.26780 handle_command_reply: tid=0 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 client.26780 shutdown 2024-05-20T21:02:32.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 msgr2=0x55fb212f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212f2470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0dfaa0 tx=0x7fc3cc004ef0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 shutdown_connections 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb212ffe80 0x55fb21302270 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212eafe0 0x55fb212efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212f2470 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3c1ffb700 0 client.26780 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2707726201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212e7290 0x55fb212ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 >> 172.21.15.26:0/2707726201 conn(0x55fb21214100 msgr2=0x55fb212e0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 shutdown_connections 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2707726201 wait complete. 2024-05-20T21:02:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.056+0000 7fc3eb23fb80 20 client.26780 trim_cache size 0 max 16384 2024-05-20T21:02:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 Processor -- start 2024-05-20T21:02:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 -- start start 2024-05-20T21:02:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb212efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1b2720 con 0x55fb212eafe0 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1affb0 con 0x55fb212e68e0 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.079+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1ade00 con 0x55fb212e7290 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212e0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49210/0 (socket says 172.21.15.26:49210) 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 -- 172.21.15.26:0/549159533 learned_addr learned my addr 172.21.15.26:0/549159533 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3da59c700 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb212efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 -- 172.21.15.26:0/549159533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 msgr2=0x55fb212efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb212efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 -- 172.21.15.26:0/549159533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 msgr2=0x55fb212ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 -- 172.21.15.26:0/549159533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb213028d0 con 0x55fb212eafe0 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3da59c700 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212e0d10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400e9a0 tx=0x7fc3d4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be24202b72fb0a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.080+0000 7fc3c3fff700 1 -- 172.21.15.26:0/549159533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4004db0 con 0x55fb212eafe0 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3c3fff700 1 -- 172.21.15.26:0/549159533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d4017d70 con 0x55fb212eafe0 2024-05-20T21:02:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3c3fff700 1 -- 172.21.15.26:0/549159533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d401fd30 con 0x55fb212eafe0 2024-05-20T21:02:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/549159533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 msgr2=0x55fb212e0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212e0d10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400e9a0 tx=0x7fc3d4009920 comp rx=0 tx=0).stop 2024-05-20T21:02:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/549159533 shutdown_connections 2024-05-20T21:02:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212e0d10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb212ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/549159533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb212efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/549159533 >> 172.21.15.26:0/549159533 conn(0x55fb21214100 msgr2=0x55fb212ef070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/549159533 shutdown_connections 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.081+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/549159533 wait complete. 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 -- start start 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 Processor -- start 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb21364210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb21364750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1ae470 con 0x55fb212eafe0 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1af8e0 con 0x55fb212e68e0 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1aeb40 con 0x55fb212e7290 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3eb23fb80 10 client.0 _ll_register_callbacks cb 0x55fb211d09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb21364210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49212/0 (socket says 172.21.15.26:49212) 2024-05-20T21:02:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3dad9d700 1 -- 172.21.15.26:0/2390865843 learned_addr learned my addr 172.21.15.26:0/2390865843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3da59c700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb21364210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51396/0 (socket says 172.21.15.26:51396) 2024-05-20T21:02:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.082+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb21364750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 msgr2=0x55fb21364210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb21364210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/2390865843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 msgr2=0x55fb212f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3d9d9b700 1 -- 172.21.15.26:0/2390865843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb212f8750 con 0x55fb212e7290 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3dad9d700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3d9d9b700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb21364750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55fb21302b00 tx=0x7fc3d0025040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9cb518e1f8812d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.083+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2390865843 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00059d0 con 0x55fb212e7290 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d4017d70 con 0x55fb212e7290 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2390865843 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fc3d000fdb0 con 0x55fb212e7290 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2390865843 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3d003d2d0 con 0x55fb212e7290 2024-05-20T21:02:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 10 client.26650 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb21375380 con 0x55fb212e7290 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2390865843 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d4017d70 con 0x55fb212e7290 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2390865843 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fc3d00235a0 con 0x55fb212e7290 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 10 client.26650 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 10 client.26650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 20 client.26650 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb213755b0 0x55fb213779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 4 client.26650 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb21247c60 con 0x55fb213755b0 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.084+0000 7fc3da59c700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb213755b0 0x55fb213779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.085+0000 7fc3da59c700 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb213755b0 0x55fb213779a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96310c54837f3387 server_cookie=4f7535a7f25ae68 in_seq=0 out_seq=0 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.085+0000 7fc3c1ffb700 10 client.26650 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3c1ffb700 1 -- 172.21.15.26:0/2390865843 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb21247c60 con 0x55fb213755b0 2024-05-20T21:02:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3c1ffb700 10 client.26650 handle_command_reply: tid=0 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 client.26650 shutdown 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 msgr2=0x55fb21364750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb21364750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55fb21302b00 tx=0x7fc3d0025040 comp rx=0 tx=0).stop 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 shutdown_connections 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb213755b0 0x55fb213779a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb212eafe0 0x55fb212f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb212e68e0 0x55fb21364210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 --2- 172.21.15.26:0/2390865843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb212e7290 0x55fb21364750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3c1ffb700 0 client.26650 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 >> 172.21.15.26:0/2390865843 conn(0x55fb21214100 msgr2=0x55fb212ef900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 shutdown_connections 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 1 -- 172.21.15.26:0/2390865843 wait complete. 2024-05-20T21:02:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.086+0000 7fc3eb23fb80 20 client.26650 trim_cache size 0 max 16384 2024-05-20T21:02:32.091 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:02:32.091 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:02:32.091 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "22a781bf-6a8c-473e-bdcd-6890dff5ac04", 2024-05-20T21:02:32.091 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:02:32.091 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3c40b2880 msgr2=0x7fc3c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3c40b2880 0x7fc3c40b4d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0006010 tx=0x7fc3d001a040 comp rx=0 tx=0).stop 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 msgr2=0x7fc3dc19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc19bdb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00d980 tx=0x7fc3cc00dc90 comp rx=0 tx=0).stop 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 shutdown_connections 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc3c40b2880 0x7fc3c40b4d40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3dc06bab0 0x7fc3dc19bdb0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3dc1095a0 0x7fc3dc1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 --2- 172.21.15.26:0/1997170528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3dc06c470 0x7fc3dc19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 >> 172.21.15.26:0/1997170528 conn(0x7fc3dc0793b0 msgr2=0x7fc3dc10e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 shutdown_connections 2024-05-20T21:02:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:32.087+0000 7fc3e09a9700 1 -- 172.21.15.26:0/1997170528 wait complete. 2024-05-20T21:02:32.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 125961 2.05 MB/sec execute 158 sec latency 19.319 ms 2024-05-20T21:02:32.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:02:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 -- 172.21.15.26:0/3443657460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 msgr2=0x7f603010abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 --2- 172.21.15.26:0/3443657460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 0x7f603010abe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6028009ca0 tx=0x7f602802a8c0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 -- 172.21.15.26:0/3443657460 shutdown_connections 2024-05-20T21:02:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 --2- 172.21.15.26:0/3443657460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 0x7f603010abe0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 --2- 172.21.15.26:0/3443657460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 --2- 172.21.15.26:0/3443657460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030103530 0x7f6030105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 -- 172.21.15.26:0/3443657460 >> 172.21.15.26:0/3443657460 conn(0x7f60300fcf20 msgr2=0x7f60300ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 -- 172.21.15.26:0/3443657460 shutdown_connections 2024-05-20T21:02:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.919+0000 7f6034dac640 1 -- 172.21.15.26:0/3443657460 wait complete. 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 Processor -- start 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 -- start start 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030103530 0x7f6030198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 0x7f60301a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f603010e500 con 0x7f6030108790 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f603010e380 con 0x7f6030103530 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f6034dac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f603010e680 con 0x7f6030105e60 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f602e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030103530 0x7f6030198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 0x7f60301a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:33486/0 (socket says 172.21.15.26:33486) 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.920+0000 7f602dd9b640 1 -- 172.21.15.26:0/1176364005 learned_addr learned my addr 172.21.15.26:0/1176364005 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f602dd9b640 1 -- 172.21.15.26:0/1176364005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030103530 msgr2=0x7f6030198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f602dd9b640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030103530 0x7f6030198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f602dd9b640 1 -- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 msgr2=0x7f60301a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f602dd9b640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 0x7f60301a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f602dd9b640 1 -- 172.21.15.26:0/1176364005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301a0770 con 0x7f6030105e60 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f602dd9b640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030199090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9f0 tx=0x7f602400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f601f7fe640 1 -- 172.21.15.26:0/1176364005 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400ccc0 con 0x7f6030105e60 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f601f7fe640 1 -- 172.21.15.26:0/1176364005 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60240054b0 con 0x7f6030105e60 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6030101380 con 0x7f6030105e60 2024-05-20T21:02:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f601f7fe640 1 -- 172.21.15.26:0/1176364005 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024010430 con 0x7f6030105e60 2024-05-20T21:02:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.921+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6030101880 con 0x7f6030105e60 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.924+0000 7f6037b9b740 1 Processor -- start 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.924+0000 7f6037b9b740 1 -- start start 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f601f7fe640 1 -- 172.21.15.26:0/1176364005 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f60240046c0 con 0x7f6030105e60 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146f99ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9a0b0 0x55d146f9ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9f220 0x55d146fa16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f6037b9b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d146f0ac30 con 0x55d146f9a0b0 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f6037b9b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d146efa7d0 con 0x55d146f9f220 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f6037b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d146efe810 con 0x55d146f996c0 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9f220 0x55d146fa16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9a0b0 0x55d146f9ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9f220 0x55d146fa16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51428/0 (socket says 172.21.15.26:51428) 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602dd9b640 1 -- 172.21.15.26:0/3887220549 learned_addr learned my addr 172.21.15.26:0/3887220549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602ed9d640 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146f99ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602ed9d640 1 -- 172.21.15.26:0/3887220549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9f220 msgr2=0x55d146fa16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602ed9d640 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9f220 0x55d146fa16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602ed9d640 1 -- 172.21.15.26:0/3887220549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9a0b0 msgr2=0x55d146f9ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602ed9d640 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9a0b0 0x55d146f9ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.925+0000 7f602ed9d640 1 -- 172.21.15.26:0/3887220549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d146c0f210 con 0x55d146f996c0 2024-05-20T21:02:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f602ed9d640 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146f99ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6028031040 tx=0x7f6028005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e150fe9913b59c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f601f7fe640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60180b28a0 0x7f60180b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f601cff9640 1 -- 172.21.15.26:0/3887220549 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280316d0 con 0x55d146f996c0 2024-05-20T21:02:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f601cff9640 1 -- 172.21.15.26:0/3887220549 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6028036030 con 0x55d146f996c0 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f601cff9640 1 -- 172.21.15.26:0/3887220549 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028004330 con 0x55d146f996c0 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f602e59c640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60180b28a0 0x7f60180b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.926+0000 7f601f7fe640 1 -- 172.21.15.26:0/1176364005 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f60240e25a0 con 0x7f6030105e60 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 -- 172.21.15.26:0/3887220549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 msgr2=0x55d146f99ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f602e59c640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60180b28a0 0x7f60180b4d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55d146f9a4b0 tx=0x7f6020012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146f99ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6028031040 tx=0x7f6028005ef0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 -- 172.21.15.26:0/3887220549 shutdown_connections 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9f220 0x55d146fa16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9a0b0 0x55d146f9ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 --2- 172.21.15.26:0/3887220549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146f99ae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 -- 172.21.15.26:0/3887220549 >> 172.21.15.26:0/3887220549 conn(0x55d146f94d30 msgr2=0x55d146f95130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 -- 172.21.15.26:0/3887220549 shutdown_connections 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 -- 172.21.15.26:0/3887220549 wait complete. 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.927+0000 7f6037b9b740 1 -- start start 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 Processor -- start 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146fa3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146fa3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146faabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d146be0790 con 0x55d146f9f220 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d146eb6890 con 0x55d146f996c0 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d146fa1ff0 con 0x55d146f9a0b0 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f6037b9b740 10 client.0 _ll_register_callbacks cb 0x55d146c0ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146faabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146fa3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146fa3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146fa3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51436/0 (socket says 172.21.15.26:51436) 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 -- 172.21.15.26:0/2321562315 learned_addr learned my addr 172.21.15.26:0/2321562315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146faabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49270/0 (socket says 172.21.15.26:49270) 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 -- 172.21.15.26:0/2321562315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 msgr2=0x55d146fa3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146fa3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 -- 172.21.15.26:0/2321562315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 msgr2=0x55d146faabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146faabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602e59c640 1 -- 172.21.15.26:0/2321562315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d146cdb620 con 0x55d146f996c0 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602dd9b640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146fa3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.928+0000 7f602ed9d640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146faabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f602e59c640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146fa3630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60200108e0 tx=0x7f6020004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105962757738e8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6015ffb640 1 -- 172.21.15.26:0/2321562315 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602001d840 con 0x55d146f996c0 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6015ffb640 1 -- 172.21.15.26:0/2321562315 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f602001d9e0 con 0x55d146f996c0 2024-05-20T21:02:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6028036030 con 0x55d146f996c0 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6015ffb640 1 -- 172.21.15.26:0/2321562315 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f602001dbe0 con 0x55d146f996c0 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6037b9b740 10 client.26798 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d146fadc00 con 0x55d146f996c0 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.929+0000 7f6015ffb640 1 -- 172.21.15.26:0/2321562315 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6028036030 con 0x55d146f996c0 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6015ffb640 1 -- 172.21.15.26:0/2321562315 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f6020027cd0 con 0x55d146f996c0 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6037b9b740 10 client.26798 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6037b9b740 10 client.26798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6037b9b740 20 client.26798 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d146faf2e0 0x55d146fb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6037b9b740 4 client.26798 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d146f0cb80 con 0x55d146faf2e0 2024-05-20T21:02:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f602dd9b640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d146faf2e0 0x55d146fb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f6015ffb640 10 client.26798 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.930+0000 7f602dd9b640 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d146faf2e0 0x55d146fb1710 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed7b28a8ab98e3 server_cookie=5b746af282626601 in_seq=0 out_seq=0 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6015ffb640 1 -- 172.21.15.26:0/2321562315 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d146f0cb80 con 0x55d146faf2e0 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6015ffb640 10 client.26798 handle_command_reply: tid=0 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 client.26798 shutdown 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 msgr2=0x55d146fa3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146fa3630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60200108e0 tx=0x7f6020004730 comp rx=0 tx=0).stop 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 shutdown_connections 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d146faf2e0 0x55d146fb1710 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146faabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146fa3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6015ffb640 0 client.26798 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2321562315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146fa3630 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.931+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 >> 172.21.15.26:0/2321562315 conn(0x55d146f94d30 msgr2=0x55d146f93fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.932+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 shutdown_connections 2024-05-20T21:02:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.932+0000 7f6037b9b740 1 -- 172.21.15.26:0/2321562315 wait complete. 2024-05-20T21:02:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.932+0000 7f6037b9b740 20 client.26798 trim_cache size 0 max 16384 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 Processor -- start 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 -- start start 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146f8fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146faab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146f93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60301aa510 con 0x55d146f9f220 2024-05-20T21:02:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f60301a7de0 con 0x55d146f996c0 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.951+0000 7f6037b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60301a5c30 con 0x55d146f9a0b0 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146f93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146f93720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49284/0 (socket says 172.21.15.26:49284) 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602ed9d640 1 -- 172.21.15.26:0/2049039472 learned_addr learned my addr 172.21.15.26:0/2049039472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146f8fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602dd9b640 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146faab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 -- 172.21.15.26:0/2049039472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 msgr2=0x55d146faab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146faab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 -- 172.21.15.26:0/2049039472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 msgr2=0x55d146f93720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146f93720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 -- 172.21.15.26:0/2049039472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d146f9e640 con 0x55d146f996c0 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602ed9d640 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146f93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f602e59c640 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146f8fe40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6020009e00 tx=0x7f6020005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de05d5d52a1f8472 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f6017fff640 1 -- 172.21.15.26:0/2049039472 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602003e020 con 0x55d146f996c0 2024-05-20T21:02:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f6017fff640 1 -- 172.21.15.26:0/2049039472 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f602002c070 con 0x55d146f996c0 2024-05-20T21:02:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.952+0000 7f6017fff640 1 -- 172.21.15.26:0/2049039472 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020004a00 con 0x55d146f996c0 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 -- 172.21.15.26:0/2049039472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 msgr2=0x55d146f8fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146f8fe40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6020009e00 tx=0x7f6020005f10 comp rx=0 tx=0).stop 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 -- 172.21.15.26:0/2049039472 shutdown_connections 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146f93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f9a0b0 0x55d146faab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 --2- 172.21.15.26:0/2049039472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f996c0 0x55d146f8fe40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 -- 172.21.15.26:0/2049039472 >> 172.21.15.26:0/2049039472 conn(0x55d146f94d30 msgr2=0x55d146f95110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 -- 172.21.15.26:0/2049039472 shutdown_connections 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.953+0000 7f6037b9b740 1 -- 172.21.15.26:0/2049039472 wait complete. 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 -- start start 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 Processor -- start 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146fa8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9a0b0 0x55d146fa8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146fa9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60301a62a0 con 0x55d146f9f220 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f60301a7710 con 0x55d146f9a0b0 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60301a6970 con 0x55d146f996c0 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f6037b9b740 10 client.0 _ll_register_callbacks cb 0x7f6020010c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146fa9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9a0b0 0x55d146fa8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146fa9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49288/0 (socket says 172.21.15.26:49288) 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f602ed9d640 1 -- 172.21.15.26:0/1878720806 learned_addr learned my addr 172.21.15.26:0/1878720806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.954+0000 7f602e59c640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146fa8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602ed9d640 1 -- 172.21.15.26:0/1878720806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 msgr2=0x55d146fa8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602ed9d640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146fa8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602ed9d640 1 -- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9a0b0 msgr2=0x55d146fa8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602ed9d640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9a0b0 0x55d146fa8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602ed9d640 1 -- 172.21.15.26:0/1878720806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d146fa9c80 con 0x55d146f9f220 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602e59c640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146fa8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602dd9b640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9a0b0 0x55d146fa8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f602ed9d640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146fa9690 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6028006010 tx=0x7f60280317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f260248dc28305cd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6015ffb640 1 -- 172.21.15.26:0/1878720806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602804a040 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6015ffb640 1 -- 172.21.15.26:0/1878720806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f6028003d80 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602002c070 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6015ffb640 1 -- 172.21.15.26:0/1878720806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6028004040 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6037b9b740 10 client.17340 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.955+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d147019ef0 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6015ffb640 1 -- 172.21.15.26:0/1878720806 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f602002c070 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6015ffb640 1 -- 172.21.15.26:0/1878720806 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f6028043070 con 0x55d146f9f220 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6037b9b740 10 client.17340 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6037b9b740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6037b9b740 20 client.17340 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6037b9b740 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d14701a550 0x55d14701c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6037b9b740 4 client.17340 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:02:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.956+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d146f06360 con 0x55d14701a550 2024-05-20T21:02:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.957+0000 7f602e59c640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d14701a550 0x55d14701c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.957+0000 7f602e59c640 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d14701a550 0x55d14701c970 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86674e3ab6e6e1d1 server_cookie=8518f6253f2e69c3 in_seq=0 out_seq=0 2024-05-20T21:02:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.957+0000 7f6015ffb640 10 client.17340 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.991+0000 7f6015ffb640 1 -- 172.21.15.26:0/1878720806 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d146f06360 con 0x55d14701a550 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.991+0000 7f6015ffb640 10 client.17340 handle_command_reply: tid=0 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.991+0000 7f6037b9b740 1 client.17340 shutdown 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.991+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 msgr2=0x55d146fa9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.991+0000 7f6037b9b740 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146fa9690 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6028006010 tx=0x7f60280317e0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 shutdown_connections 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d14701a550 0x55d14701c970 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d146f9f220 0x55d146fa9690 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d146f9a0b0 0x55d146fa8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 --2- 172.21.15.26:0/1878720806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d146f996c0 0x55d146fa8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 >> 172.21.15.26:0/1878720806 conn(0x55d146f94d30 msgr2=0x55d146f8fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6015ffb640 0 client.17340 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 shutdown_connections 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 1 -- 172.21.15.26:0/1878720806 wait complete. 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6037b9b740 20 client.17340 trim_cache size 0 max 16384 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:02:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:02:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60180b28a0 msgr2=0x7f60180b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6034dac640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60180b28a0 0x7f60180b4d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55d146f9a4b0 tx=0x7f6020012040 comp rx=0 tx=0).stop 2024-05-20T21:02:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.992+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 msgr2=0x7f6030199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030199090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9f0 tx=0x7f602400eeb0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 shutdown_connections 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60180b28a0 0x7f60180b4d60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6030108790 0x7f60301a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6030105e60 0x7f6030199090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 --2- 172.21.15.26:0/1176364005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030103530 0x7f6030198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 >> 172.21.15.26:0/1176364005 conn(0x7f60300fcf20 msgr2=0x7f6030103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 shutdown_connections 2024-05-20T21:02:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:32.993+0000 7f6034dac640 1 -- 172.21.15.26:0/1176364005 wait complete. 2024-05-20T21:02:33.082 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:02:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:33 smithi144 ceph-mon[35660]: pgmap v581: 97 pgs: 97 active+clean; 30 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:02:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:33 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:02:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:33 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:33 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:02:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:33 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:33.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:33 smithi026 ceph-mon[31990]: pgmap v581: 97 pgs: 97 active+clean; 30 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:02:33.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:33 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:02:33.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:33 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:33.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:33 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:02:33.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:33 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:33 smithi055 ceph-mon[35213]: pgmap v581: 97 pgs: 97 active+clean; 30 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:02:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:33 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:02:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:33 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:33 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:02:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:33 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:02:33.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 126576 2.04 MB/sec execute 159 sec latency 29.263 ms 2024-05-20T21:02:34.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 127472 2.08 MB/sec execute 160 sec latency 15.661 ms 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 -- 172.21.15.26:0/3034630235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf81095b0 msgr2=0x7fbbf8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 --2- 172.21.15.26:0/3034630235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf81095b0 0x7fbbf8109990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009990 tx=0x7fbbe8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 -- 172.21.15.26:0/3034630235 shutdown_connections 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 --2- 172.21.15.26:0/3034630235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf81095b0 0x7fbbf8109990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 --2- 172.21.15.26:0/3034630235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806bbe0 0x7fbbf81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 --2- 172.21.15.26:0/3034630235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf806b220 0x7fbbf806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 -- 172.21.15.26:0/3034630235 >> 172.21.15.26:0/3034630235 conn(0x7fbbf80793f0 msgr2=0x7fbbf8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 -- 172.21.15.26:0/3034630235 shutdown_connections 2024-05-20T21:02:34.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.980+0000 7fbbfd516700 1 -- 172.21.15.26:0/3034630235 wait complete. 2024-05-20T21:02:34.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.981+0000 7fbbfd516700 1 Processor -- start 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 -- start start 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 0x7fbbf819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf806bbe0 0x7fbbf819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf81095b0 0x7fbbf8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8113590 con 0x7fbbf806bbe0 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8113410 con 0x7fbbf81095b0 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbfd516700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8113710 con 0x7fbbf806b220 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 0x7fbbf819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 0x7fbbf819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:33528/0 (socket says 172.21.15.26:33528) 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/1989283834 learned_addr learned my addr 172.21.15.26:0/1989283834 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:34.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf806bbe0 0x7fbbf819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.982+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf81095b0 0x7fbbf8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/1989283834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf81095b0 msgr2=0x7fbbf8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf81095b0 0x7fbbf8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf806bbe0 msgr2=0x7fbbf819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf806bbe0 0x7fbbf819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/1989283834 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf819a090 con 0x7fbbf806b220 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf81095b0 0x7fbbf8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 0x7fbbf819f770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8006010 tx=0x7fbbe800bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbdffff700 1 -- 172.21.15.26:0/1989283834 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8027070 con 0x7fbbf806b220 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbdffff700 1 -- 172.21.15.26:0/1989283834 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbe8022470 con 0x7fbbf806b220 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf819a320 con 0x7fbbf806b220 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbdffff700 1 -- 172.21.15.26:0/1989283834 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8017650 con 0x7fbbf806b220 2024-05-20T21:02:34.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.983+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf8102440 con 0x7fbbf806b220 2024-05-20T21:02:34.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.987+0000 7fbbdffff700 1 -- 172.21.15.26:0/1989283834 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbbe8009d10 con 0x7fbbf806b220 2024-05-20T21:02:34.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbbdffff700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbbe00b2900 0x7fbbe00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbbe00b2900 0x7fbbe00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbbdffff700 1 -- 172.21.15.26:0/1989283834 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fbbe80e2cc0 con 0x7fbbf806b220 2024-05-20T21:02:34.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 Processor -- start 2024-05-20T21:02:34.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 -- start start 2024-05-20T21:02:34.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9a20 0x55f2994bd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f29940fc90 con 0x55f2994b9a20 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f2993ff830 con 0x55f2994bd770 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f299403870 con 0x55f2994b9070 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.988+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbbe00b2900 0x7fbbe00b4dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8079ad0 tx=0x7fbbec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9a20 0x55f2994bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994b9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:33538/0 (socket says 172.21.15.26:33538) 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/3564903302 learned_addr learned my addr 172.21.15.26:0/3564903302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/3564903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 msgr2=0x55f2994bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994bdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/3564903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9a20 msgr2=0x55f2994bd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:34.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9a20 0x55f2994bd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/3564903302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f29934f8d0 con 0x55f2994b9070 2024-05-20T21:02:34.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994b9450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80a6390 tx=0x7fbbe80a9ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3a9f16ec5ae0a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:34.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbdd7fa700 1 -- 172.21.15.26:0/3564903302 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80ae6c0 con 0x55f2994b9070 2024-05-20T21:02:34.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbdd7fa700 1 -- 172.21.15.26:0/3564903302 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbe80b30e0 con 0x55f2994b9070 2024-05-20T21:02:34.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.989+0000 7fbbdd7fa700 1 -- 172.21.15.26:0/3564903302 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80b2c50 con 0x55f2994b9070 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 -- 172.21.15.26:0/3564903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 msgr2=0x55f2994b9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994b9450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80a6390 tx=0x7fbbe80a9ea0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 -- 172.21.15.26:0/3564903302 shutdown_connections 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994bdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994b9450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/3564903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9a20 0x55f2994bd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 -- 172.21.15.26:0/3564903302 >> 172.21.15.26:0/3564903302 conn(0x55f2993de080 msgr2=0x55f2994aae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 -- 172.21.15.26:0/3564903302 shutdown_connections 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.990+0000 7fbc07dacb80 1 -- 172.21.15.26:0/3564903302 wait complete. 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 -- start start 2024-05-20T21:02:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 Processor -- start 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 0x55f2994c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9a20 0x55f2994c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f299305fa0 con 0x55f2994b9070 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f2993a17c0 con 0x55f2994bd770 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f2994c56f0 con 0x55f2994b9a20 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbc07dacb80 10 client.0 _ll_register_callbacks cb 0x55f29939a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 0x55f2994c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 0x55f2994c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49344/0 (socket says 172.21.15.26:49344) 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbbf6ffd700 1 -- 172.21.15.26:0/2221752170 learned_addr learned my addr 172.21.15.26:0/2221752170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9a20 0x55f2994c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.991+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf67fc700 1 -- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 msgr2=0x55f2994c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf67fc700 1 -- 172.21.15.26:0/2221752170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 msgr2=0x55f2994c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 0x55f2994c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf67fc700 1 -- 172.21.15.26:0/2221752170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29939e690 con 0x55f2994b9a20 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 0x55f2994c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9a20 0x55f2994c1f50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009b00 tx=0x7fbbec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f584eb3e0067de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbdaff5700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbdaff5700 1 -- 172.21.15.26:0/2221752170 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec004a30 con 0x55f2994b9a20 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbdaff5700 1 -- 172.21.15.26:0/2221752170 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fbbec004bd0 con 0x55f2994b9a20 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe80b30e0 con 0x55f2994b9a20 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbbdaff5700 1 -- 172.21.15.26:0/2221752170 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbec004dd0 con 0x55f2994b9a20 2024-05-20T21:02:34.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbc07dacb80 10 client.26677 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.992+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2994cfde0 con 0x55f2994b9a20 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbbdaff5700 1 -- 172.21.15.26:0/2221752170 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbe80b30e0 con 0x55f2994b9a20 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbbdaff5700 1 -- 172.21.15.26:0/2221752170 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fbbec030050 con 0x55f2994b9a20 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbc07dacb80 10 client.26677 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbc07dacb80 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbc07dacb80 20 client.26677 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f2994d23c0 0x55f2994d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbc07dacb80 4 client.26677 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f29947c9e0 con 0x55f2994d23c0 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.993+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f2994d23c0 0x55f2994d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.994+0000 7fbbdaff5700 10 client.26677 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.994+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f2994d23c0 0x55f2994d47b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e512f676ed508b server_cookie=6311145be05c62cf in_seq=0 out_seq=0 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.994+0000 7fbbdaff5700 1 -- 172.21.15.26:0/2221752170 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f29947c9e0 con 0x55f2994d23c0 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.994+0000 7fbbdaff5700 10 client.26677 handle_command_reply: tid=0 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.994+0000 7fbc07dacb80 1 client.26677 shutdown 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9a20 msgr2=0x55f2994c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9a20 0x55f2994c1f50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009b00 tx=0x7fbbec0047c0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 shutdown_connections 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f2994d23c0 0x55f2994d47b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994b9070 0x55f2994c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbbdaff5700 0 client.26677 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994bd770 0x55f2994c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/2221752170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9a20 0x55f2994c1f50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 >> 172.21.15.26:0/2221752170 conn(0x55f2993de080 msgr2=0x55f2994aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 shutdown_connections 2024-05-20T21:02:35.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 1 -- 172.21.15.26:0/2221752170 wait complete. 2024-05-20T21:02:35.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:34.995+0000 7fbc07dacb80 20 client.26677 trim_cache size 0 max 16384 2024-05-20T21:02:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 Processor -- start 2024-05-20T21:02:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 -- start start 2024-05-20T21:02:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994bc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81ae240 con 0x55f2994bd770 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81abad0 con 0x55f2994b9a20 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81a9920 con 0x55f2994b9070 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbbf67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbbf77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbbf67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994aac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51496/0 (socket says 172.21.15.26:51496) 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbbf77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49360/0 (socket says 172.21.15.26:49360) 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbbf67fc700 1 -- 172.21.15.26:0/1539170909 learned_addr learned my addr 172.21.15.26:0/1539170909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.019+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994bc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf77fe700 1 -- 172.21.15.26:0/1539170909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 msgr2=0x55f2994bc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994bc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf77fe700 1 -- 172.21.15.26:0/1539170909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 msgr2=0x55f2994aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf77fe700 1 -- 172.21.15.26:0/1539170909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2994abf70 con 0x55f2994bd770 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994ab1d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000aad0 tx=0x7fbbf00078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463fcd10f4624892 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbdcff9700 1 -- 172.21.15.26:0/1539170909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0004db0 con 0x55f2994bd770 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbdcff9700 1 -- 172.21.15.26:0/1539170909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbf0015c20 con 0x55f2994bd770 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.020+0000 7fbbdcff9700 1 -- 172.21.15.26:0/1539170909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf001fab0 con 0x55f2994bd770 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1539170909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 msgr2=0x55f2994ab1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994ab1d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000aad0 tx=0x7fbbf00078d0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1539170909 shutdown_connections 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994ab1d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994bc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1539170909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1539170909 >> 172.21.15.26:0/1539170909 conn(0x55f2993de080 msgr2=0x55f2994bc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1539170909 shutdown_connections 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1539170909 wait complete. 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.021+0000 7fbc07dacb80 1 -- start start 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 Processor -- start 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81a9f90 con 0x55f2994bd770 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81ab400 con 0x55f2994b9a20 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81aa660 con 0x55f2994b9070 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbc07dacb80 10 client.0 _ll_register_callbacks cb 0x55f29939a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51508/0 (socket says 172.21.15.26:51508) 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 -- 172.21.15.26:0/1643759007 learned_addr learned my addr 172.21.15.26:0/1643759007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf77fe700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 -- 172.21.15.26:0/1643759007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 msgr2=0x55f2994c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 -- 172.21.15.26:0/1643759007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 msgr2=0x55f2994c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.022+0000 7fbbf67fc700 1 -- 172.21.15.26:0/1643759007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29953bf00 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbf67fc700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994c8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009b00 tx=0x7fbbec004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1f99a70ac53da0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbdaff5700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbdaff5700 1 -- 172.21.15.26:0/1643759007 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec01eba0 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf0015c20 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbdaff5700 1 -- 172.21.15.26:0/1643759007 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fbbec01ed40 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbdaff5700 1 -- 172.21.15.26:0/1643759007 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbec004f70 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 10 client.26822 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f29953d180 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbdaff5700 1 -- 172.21.15.26:0/1643759007 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbf0015c20 con 0x55f2994b9a20 2024-05-20T21:02:35.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbbdaff5700 1 -- 172.21.15.26:0/1643759007 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fbbec044050 con 0x55f2994b9a20 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 10 client.26822 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 10 client.26822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 20 client.26822 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f29953d870 0x55f29953fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.023+0000 7fbc07dacb80 4 client.26822 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f299411be0 con 0x55f29953d870 2024-05-20T21:02:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f29953d870 0x55f29953fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:35.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbbdaff5700 10 client.26822 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:35.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbbf6ffd700 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f29953d870 0x55f29953fc90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d22f722b0936d31 server_cookie=4d366324a675abd9 in_seq=0 out_seq=0 2024-05-20T21:02:35.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbbdaff5700 1 -- 172.21.15.26:0/1643759007 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f299411be0 con 0x55f29953d870 2024-05-20T21:02:35.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbbdaff5700 10 client.26822 handle_command_reply: tid=0 2024-05-20T21:02:35.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 client.26822 shutdown 2024-05-20T21:02:35.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 msgr2=0x55f2994c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994c8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009b00 tx=0x7fbbec004a60 comp rx=0 tx=0).stop 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 shutdown_connections 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f29953d870 0x55f29953fc90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2994bd770 0x55f2994c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.024+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f2994b9070 0x55f2994c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbdaff5700 0 client.26822 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbc07dacb80 1 --2- 172.21.15.26:0/1643759007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f2994b9a20 0x55f2994c8b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 >> 172.21.15.26:0/1643759007 conn(0x55f2993de080 msgr2=0x55f2994bc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 shutdown_connections 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbc07dacb80 1 -- 172.21.15.26:0/1643759007 wait complete. 2024-05-20T21:02:35.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbc07dacb80 20 client.26822 trim_cache size 0 max 16384 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: "22a781bf-6a8c-473e-bdcd-6890dff5ac04": { 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "22a781bf-6a8c-473e-bdcd-6890dff5ac04", 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:02:35.031 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:02:35.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbbe00b2900 msgr2=0x7fbbe00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:35.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbbe00b2900 0x7fbbe00b4dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8079ad0 tx=0x7fbbec009260 comp rx=0 tx=0).stop 2024-05-20T21:02:35.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 msgr2=0x7fbbf819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:35.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 0x7fbbf819f770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8006010 tx=0x7fbbe800bfc0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 shutdown_connections 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf81095b0 0x7fbbf8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806b220 0x7fbbf819f770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbbe00b2900 0x7fbbe00b4dc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 --2- 172.21.15.26:0/1989283834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbf806bbe0 0x7fbbf819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.025+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 >> 172.21.15.26:0/1989283834 conn(0x7fbbf80793f0 msgr2=0x7fbbf8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.026+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 shutdown_connections 2024-05-20T21:02:35.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:02:35.026+0000 7fbbfd516700 1 -- 172.21.15.26:0/1989283834 wait complete. 2024-05-20T21:02:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:35 smithi144 ceph-mon[35660]: pgmap v582: 97 pgs: 97 active+clean; 26 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-20T21:02:35.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:35 smithi055 ceph-mon[35213]: pgmap v582: 97 pgs: 97 active+clean; 26 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-20T21:02:35.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:35 smithi026 ceph-mon[31990]: pgmap v582: 97 pgs: 97 active+clean; 26 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-20T21:02:35.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 127982 2.09 MB/sec execute 161 sec latency 18.508 ms 2024-05-20T21:02:36.052 INFO:tasks.cephfs.filesystem:scrub status for tag:22a781bf-6a8c-473e-bdcd-6890dff5ac04 - {'path': '/', 'tag': '22a781bf-6a8c-473e-bdcd-6890dff5ac04', 'options': 'recursive,force'} 2024-05-20T21:02:36.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 128449 2.08 MB/sec execute 162 sec latency 19.402 ms 2024-05-20T21:02:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:37 smithi144 ceph-mon[35660]: pgmap v583: 97 pgs: 97 active+clean; 26 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.6 MiB/s wr, 97 op/s 2024-05-20T21:02:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:37 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:02:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:37 smithi055 ceph-mon[35213]: pgmap v583: 97 pgs: 97 active+clean; 26 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.6 MiB/s wr, 97 op/s 2024-05-20T21:02:37.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:37 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:02:37.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:37 smithi026 ceph-mon[31990]: pgmap v583: 97 pgs: 97 active+clean; 26 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.6 MiB/s wr, 97 op/s 2024-05-20T21:02:37.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:37 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:02:37.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 129162 2.10 MB/sec execute 163 sec latency 13.546 ms 2024-05-20T21:02:38.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 129905 2.09 MB/sec execute 164 sec latency 11.065 ms 2024-05-20T21:02:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:39 smithi144 ceph-mon[35660]: pgmap v584: 97 pgs: 97 active+clean; 25 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:02:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:39 smithi055 ceph-mon[35213]: pgmap v584: 97 pgs: 97 active+clean; 25 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:02:39.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:39 smithi026 ceph-mon[31990]: pgmap v584: 97 pgs: 97 active+clean; 25 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:02:39.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 130692 2.12 MB/sec execute 165 sec latency 17.756 ms 2024-05-20T21:02:40.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 131307 2.14 MB/sec execute 166 sec latency 17.561 ms 2024-05-20T21:02:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:41 smithi144 ceph-mon[35660]: pgmap v585: 97 pgs: 97 active+clean; 28 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-05-20T21:02:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:41 smithi055 ceph-mon[35213]: pgmap v585: 97 pgs: 97 active+clean; 28 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-05-20T21:02:41.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:41 smithi026 ceph-mon[31990]: pgmap v585: 97 pgs: 97 active+clean; 28 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-05-20T21:02:41.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 131775 2.13 MB/sec execute 167 sec latency 16.956 ms 2024-05-20T21:02:42.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 132418 2.13 MB/sec execute 168 sec latency 17.348 ms 2024-05-20T21:02:43.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:43 smithi144 ceph-mon[35660]: pgmap v586: 97 pgs: 97 active+clean; 28 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:02:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:43 smithi055 ceph-mon[35213]: pgmap v586: 97 pgs: 97 active+clean; 28 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:02:43.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:43 smithi026 ceph-mon[31990]: pgmap v586: 97 pgs: 97 active+clean; 28 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:02:43.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 133136 2.14 MB/sec execute 169 sec latency 17.990 ms 2024-05-20T21:02:44.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 133826 2.14 MB/sec execute 170 sec latency 17.919 ms 2024-05-20T21:02:45.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:45 smithi144 ceph-mon[35660]: pgmap v587: 97 pgs: 97 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-05-20T21:02:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:45 smithi055 ceph-mon[35213]: pgmap v587: 97 pgs: 97 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-05-20T21:02:45.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:45 smithi026 ceph-mon[31990]: pgmap v587: 97 pgs: 97 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-05-20T21:02:45.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 134589 2.17 MB/sec execute 171 sec latency 17.355 ms 2024-05-20T21:02:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/1201631551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0103560 msgr2=0x7fb8b0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/1201631551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0103560 0x7fb8b0109df0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009ca0 tx=0x7fb8a802a8c0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/1201631551 shutdown_connections 2024-05-20T21:02:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/1201631551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0103560 0x7fb8b0109df0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/1201631551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8b0102ba0 0x7fb8b0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/1201631551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b01019b0 0x7fb8b0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/1201631551 >> 172.21.15.26:0/1201631551 conn(0x7fb8b00fcf20 msgr2=0x7fb8b00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/1201631551 shutdown_connections 2024-05-20T21:02:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/1201631551 wait complete. 2024-05-20T21:02:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 Processor -- start 2024-05-20T21:02:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.098+0000 7fb8b6ab4640 1 -- start start 2024-05-20T21:02:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8b01019b0 0x7fb8b006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 0x7fb8b019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0103560 0x7fb8b019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b010d3e0 con 0x7fb8b0102ba0 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b010d260 con 0x7fb8b01019b0 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b010d560 con 0x7fb8b0103560 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 0x7fb8b019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 0x7fb8b019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43502/0 (socket says 172.21.15.26:43502) 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 -- 172.21.15.26:0/3472409558 learned_addr learned my addr 172.21.15.26:0/3472409558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b4850640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8b01019b0 0x7fb8b006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b5051640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0103560 0x7fb8b019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 -- 172.21.15.26:0/3472409558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0103560 msgr2=0x7fb8b019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0103560 0x7fb8b019fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 -- 172.21.15.26:0/3472409558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8b01019b0 msgr2=0x7fb8b006ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8b01019b0 0x7fb8b006ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 -- 172.21.15.26:0/3472409558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b01a07d0 con 0x7fb8b0102ba0 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8affff640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 0x7fb8b019d8a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb89800e9f0 tx=0x7fb89800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8adffb640 1 -- 172.21.15.26:0/3472409558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89800cd80 con 0x7fb8b0102ba0 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b01a0a60 con 0x7fb8b0102ba0 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.099+0000 7fb8adffb640 1 -- 172.21.15.26:0/3472409558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8980108c0 con 0x7fb8b0102ba0 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.100+0000 7fb8adffb640 1 -- 172.21.15.26:0/3472409558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8980187c0 con 0x7fb8b0102ba0 2024-05-20T21:02:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.100+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b01a1000 con 0x7fb8b0102ba0 2024-05-20T21:02:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.102+0000 7fb8b97cf740 1 Processor -- start 2024-05-20T21:02:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.102+0000 7fb8b97cf740 1 -- start start 2024-05-20T21:02:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.102+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb4def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb52fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb55ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd2fabac30 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bd2faaa7d0 con 0x55bd2fb53630 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd2faae810 con 0x55bd2fb4dad0 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb52fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb52fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43510/0 (socket says 172.21.15.26:43510) 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 -- 172.21.15.26:0/1601980347 learned_addr learned my addr 172.21.15.26:0/1601980347 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8adffb640 1 -- 172.21.15.26:0/3472409558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb898010430 con 0x7fb8b0102ba0 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b5051640 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb4def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 -- 172.21.15.26:0/1601980347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 msgr2=0x55bd2fb4def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb4def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 -- 172.21.15.26:0/1601980347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 msgr2=0x55bd2fb55ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:02:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb55ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 -- 172.21.15.26:0/1601980347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2f7bf210 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8b4850640 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb52fc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bcb0 tx=0x7fb8a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc051cdab741da7b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8a6ffd640 1 -- 172.21.15.26:0/1601980347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0005020 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8a6ffd640 1 -- 172.21.15.26:0/1601980347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00047a0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.103+0000 7fb8a6ffd640 1 -- 172.21.15.26:0/1601980347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00116a0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8adffb640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb89c0b28a0 0x7fb89c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b5051640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb89c0b28a0 0x7fb89c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8adffb640 1 -- 172.21.15.26:0/3472409558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fb8980e26b0 con 0x7fb8b0102ba0 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1601980347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 msgr2=0x55bd2fb52fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb52fc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bcb0 tx=0x7fb8a00107a0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1601980347 shutdown_connections 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb55ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb52fc0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1601980347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb4def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1601980347 >> 172.21.15.26:0/1601980347 conn(0x55bd2fb44d30 msgr2=0x55bd2fb45130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1601980347 shutdown_connections 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b5051640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb89c0b28a0 0x7fb89c0b4d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8006010 tx=0x7fb8a8031980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.104+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1601980347 wait complete. 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 -- start start 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 Processor -- start 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb57cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb53630 0x55bd2fb5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd2f790790 con 0x55bd2fb53630 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bd2fa66890 con 0x55bd2fb4dad0 2024-05-20T21:02:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd2fb565e0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b4850640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb57cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b5051640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb53630 0x55bd2fb5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b97cf740 10 client.0 _ll_register_callbacks cb 0x55bd2f7bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b4850640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb57cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40014/0 (socket says 172.21.15.26:40014) 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8b4850640 1 -- 172.21.15.26:0/2100733288 learned_addr learned my addr 172.21.15.26:0/2100733288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.105+0000 7fb8affff640 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b4850640 1 -- 172.21.15.26:0/2100733288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4e4c0 msgr2=0x55bd2fb58200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b4850640 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb58200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b4850640 1 -- 172.21.15.26:0/2100733288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb53630 msgr2=0x55bd2fb5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b4850640 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb53630 0x55bd2fb5f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b4850640 1 -- 172.21.15.26:0/2100733288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd2f88b620 con 0x55bd2fb4dad0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b4850640 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb57cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bc80 tx=0x7fb8a000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c02359a3a13f9bb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb88ffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb88ffff640 1 -- 172.21.15.26:0/2100733288 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0005000 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a00047a0 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb88ffff640 1 -- 172.21.15.26:0/2100733288 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fb8a00051a0 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb88ffff640 1 -- 172.21.15.26:0/2100733288 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8a00053a0 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b97cf740 10 client.26831 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd2fb62200 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.106+0000 7fb88ffff640 1 -- 172.21.15.26:0/2100733288 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00047a0 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb88ffff640 1 -- 172.21.15.26:0/2100733288 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fb8a001b4c0 con 0x55bd2fb4dad0 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8b97cf740 10 client.26831 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8b97cf740 10 client.26831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8b97cf740 20 client.26831 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fb63600 0x55bd2fb659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8b97cf740 4 client.26831 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd2fabcb80 con 0x55bd2fb63600 2024-05-20T21:02:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8affff640 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fb63600 0x55bd2fb659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb88ffff640 10 client.26831 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.107+0000 7fb8affff640 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fb63600 0x55bd2fb659f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5426782a02d1e76 server_cookie=489500033a14daea in_seq=0 out_seq=0 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb88ffff640 1 -- 172.21.15.26:0/2100733288 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bd2fabcb80 con 0x55bd2fb63600 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb88ffff640 10 client.26831 handle_command_reply: tid=0 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 client.26831 shutdown 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 msgr2=0x55bd2fb57cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb57cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bc80 tx=0x7fb8a000bfd0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 shutdown_connections 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fb63600 0x55bd2fb659f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb53630 0x55bd2fb5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb58200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/2100733288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb57cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb88ffff640 0 client.26831 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 >> 172.21.15.26:0/2100733288 conn(0x55bd2fb44d30 msgr2=0x55bd2fb43fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 shutdown_connections 2024-05-20T21:02:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 1 -- 172.21.15.26:0/2100733288 wait complete. 2024-05-20T21:02:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.108+0000 7fb8b97cf740 20 client.26831 trim_cache size 0 max 16384 2024-05-20T21:02:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.127+0000 7fb8b97cf740 1 Processor -- start 2024-05-20T21:02:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.127+0000 7fb8b97cf740 1 -- start start 2024-05-20T21:02:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb43e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb53630 0x55bd2fb6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01aa630 con 0x55bd2fb4dad0 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01a7f00 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01a5d50 con 0x55bd2fb53630 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb43e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb43e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43536/0 (socket says 172.21.15.26:43536) 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 -- 172.21.15.26:0/1365443706 learned_addr learned my addr 172.21.15.26:0/1365443706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8affff640 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 -- 172.21.15.26:0/1365443706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb53630 msgr2=0x55bd2fb6a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb53630 0x55bd2fb6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 -- 172.21.15.26:0/1365443706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4e4c0 msgr2=0x55bd2fb5f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 -- 172.21.15.26:0/1365443706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2fb65d60 con 0x55bd2fb4dad0 2024-05-20T21:02:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8b4850640 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb43e30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0011010 tx=0x7fb8a00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3f45e789126c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8a67fc640 1 -- 172.21.15.26:0/1365443706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a003e070 con 0x55bd2fb4dad0 2024-05-20T21:02:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8a67fc640 1 -- 172.21.15.26:0/1365443706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00050f0 con 0x55bd2fb4dad0 2024-05-20T21:02:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.128+0000 7fb8a67fc640 1 -- 172.21.15.26:0/1365443706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0039460 con 0x55bd2fb4dad0 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1365443706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 msgr2=0x55bd2fb43e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb43e30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0011010 tx=0x7fb8a00056d0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1365443706 shutdown_connections 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb53630 0x55bd2fb6a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/1365443706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb43e30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1365443706 >> 172.21.15.26:0/1365443706 conn(0x55bd2fb44d30 msgr2=0x55bd2fb43720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1365443706 shutdown_connections 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 -- 172.21.15.26:0/1365443706 wait complete. 2024-05-20T21:02:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.129+0000 7fb8b97cf740 1 -- start start 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 Processor -- start 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb5dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01a63c0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01a7830 con 0x55bd2fb53630 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01a6a90 con 0x55bd2fb4dad0 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b97cf740 10 client.0 _ll_register_callbacks cb 0x7fb8a000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43546/0 (socket says 172.21.15.26:43546) 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 -- 172.21.15.26:0/4080432780 learned_addr learned my addr 172.21.15.26:0/4080432780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b5051640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb5dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8b4850640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 -- 172.21.15.26:0/4080432780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 msgr2=0x55bd2fb5c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb5c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 -- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 msgr2=0x55bd2fb5dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb5dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.130+0000 7fb8affff640 1 -- 172.21.15.26:0/4080432780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd2fb5e0a0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8affff640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5caa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb8980e01a0 tx=0x7fb89801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdf415016f77f3a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb88ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb88ffff640 1 -- 172.21.15.26:0/4080432780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8980c4350 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a00050f0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb88ffff640 1 -- 172.21.15.26:0/4080432780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7fb898010d50 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb88ffff640 1 -- 172.21.15.26:0/4080432780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8980aa370 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 10 client.17376 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd2fb5f480 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb88ffff640 1 -- 172.21.15.26:0/4080432780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00050f0 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb88ffff640 1 -- 172.21.15.26:0/4080432780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fb8980b2640 con 0x55bd2fb4e4c0 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 10 client.17376 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 20 client.17376 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fbce820 0x55bd2fbd0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 4 client.17376 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.131+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd2fab6360 con 0x55bd2fbce820 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.132+0000 7fb8b4850640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fbce820 0x55bd2fbd0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.132+0000 7fb8b4850640 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fbce820 0x55bd2fbd0c40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16dd64c7e691d2eb server_cookie=da4dee9efba5c760 in_seq=0 out_seq=0 2024-05-20T21:02:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.132+0000 7fb88ffff640 10 client.17376 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb88ffff640 1 -- 172.21.15.26:0/4080432780 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd2fab6360 con 0x55bd2fbce820 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb88ffff640 10 client.17376 handle_command_reply: tid=0 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 client.17376 shutdown 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 msgr2=0x55bd2fb5caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5caa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb8980e01a0 tx=0x7fb89801f000 comp rx=0 tx=0).stop 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 shutdown_connections 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55bd2fbce820 0x55bd2fbd0c40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd2fb53630 0x55bd2fb5dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd2fb4e4c0 0x55bd2fb5caa0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 --2- 172.21.15.26:0/4080432780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd2fb4dad0 0x55bd2fb5c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 >> 172.21.15.26:0/4080432780 conn(0x55bd2fb44d30 msgr2=0x55bd2fb43fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb88ffff640 0 client.17376 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 shutdown_connections 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 1 -- 172.21.15.26:0/4080432780 wait complete. 2024-05-20T21:02:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.169+0000 7fb8b97cf740 20 client.17376 trim_cache size 0 max 16384 2024-05-20T21:02:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:02:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:02:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb89c0b28a0 msgr2=0x7fb89c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb89c0b28a0 0x7fb89c0b4d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8006010 tx=0x7fb8a8031980 comp rx=0 tx=0).stop 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 msgr2=0x7fb8b019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 0x7fb8b019d8a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb89800e9f0 tx=0x7fb89800eeb0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 shutdown_connections 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb89c0b28a0 0x7fb89c0b4d60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0103560 0x7fb8b019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b0102ba0 0x7fb8b019d8a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 --2- 172.21.15.26:0/3472409558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8b01019b0 0x7fb8b006ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 >> 172.21.15.26:0/3472409558 conn(0x7fb8b00fcf20 msgr2=0x7fb8b0105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 shutdown_connections 2024-05-20T21:02:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:46.170+0000 7fb8b6ab4640 1 -- 172.21.15.26:0/3472409558 wait complete. 2024-05-20T21:02:46.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 135137 2.18 MB/sec execute 172 sec latency 32.866 ms 2024-05-20T21:02:47.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:47 smithi144 ceph-mon[35660]: pgmap v588: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2024-05-20T21:02:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:47 smithi055 ceph-mon[35213]: pgmap v588: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2024-05-20T21:02:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:47 smithi026 ceph-mon[31990]: pgmap v588: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2024-05-20T21:02:47.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 135644 2.17 MB/sec execute 173 sec latency 28.999 ms 2024-05-20T21:02:48.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 136303 2.19 MB/sec execute 174 sec latency 15.967 ms 2024-05-20T21:02:49.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:49 smithi144 ceph-mon[35660]: pgmap v589: 97 pgs: 97 active+clean; 30 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:02:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:49 smithi055 ceph-mon[35213]: pgmap v589: 97 pgs: 97 active+clean; 30 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:02:49.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:49 smithi026 ceph-mon[31990]: pgmap v589: 97 pgs: 97 active+clean; 30 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:02:49.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 137027 2.18 MB/sec execute 175 sec latency 10.550 ms 2024-05-20T21:02:50.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 137806 2.20 MB/sec execute 176 sec latency 19.021 ms 2024-05-20T21:02:51.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:51 smithi144 ceph-mon[35660]: pgmap v590: 97 pgs: 97 active+clean; 26 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-20T21:02:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:51 smithi055 ceph-mon[35213]: pgmap v590: 97 pgs: 97 active+clean; 26 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-20T21:02:51.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:51 smithi026 ceph-mon[31990]: pgmap v590: 97 pgs: 97 active+clean; 26 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-20T21:02:51.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 138387 2.22 MB/sec execute 177 sec latency 18.373 ms 2024-05-20T21:02:52.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 138890 2.22 MB/sec execute 178 sec latency 10.537 ms 2024-05-20T21:02:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:53 smithi055 ceph-mon[35213]: pgmap v591: 97 pgs: 97 active+clean; 25 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:02:53.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:53 smithi026 ceph-mon[31990]: pgmap v591: 97 pgs: 97 active+clean; 25 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:02:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:53 smithi144 ceph-mon[35660]: pgmap v591: 97 pgs: 97 active+clean; 25 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:02:53.561 INFO:tasks.workunit.client.0.smithi026.stdout: 1 139542 2.22 MB/sec execute 179 sec latency 20.013 ms 2024-05-20T21:02:54.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 140276 2.22 MB/sec execute 180 sec latency 13.659 ms 2024-05-20T21:02:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:55 smithi055 ceph-mon[35213]: pgmap v592: 97 pgs: 97 active+clean; 26 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-05-20T21:02:55.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:55 smithi026 ceph-mon[31990]: pgmap v592: 97 pgs: 97 active+clean; 26 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-05-20T21:02:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:55 smithi144 ceph-mon[35660]: pgmap v592: 97 pgs: 97 active+clean; 26 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-05-20T21:02:55.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 140947 2.22 MB/sec execute 181 sec latency 19.251 ms 2024-05-20T21:02:56.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 141685 2.25 MB/sec execute 182 sec latency 17.743 ms 2024-05-20T21:02:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:57 smithi055 ceph-mon[35213]: pgmap v593: 97 pgs: 97 active+clean; 27 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-05-20T21:02:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:57 smithi026 ceph-mon[31990]: pgmap v593: 97 pgs: 97 active+clean; 27 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-05-20T21:02:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:57 smithi144 ceph-mon[35660]: pgmap v593: 97 pgs: 97 active+clean; 27 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-05-20T21:02:57.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 142224 2.26 MB/sec execute 183 sec latency 17.994 ms 2024-05-20T21:02:58.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 142700 2.25 MB/sec execute 184 sec latency 30.724 ms 2024-05-20T21:02:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 -- 172.21.15.26:0/1633570895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 msgr2=0x7f91a010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 --2- 172.21.15.26:0/1633570895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a010ac40 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9198009ca0 tx=0x7f919802a8c0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 -- 172.21.15.26:0/1633570895 shutdown_connections 2024-05-20T21:02:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 --2- 172.21.15.26:0/1633570895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a010ac40 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 --2- 172.21.15.26:0/1633570895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a0105f20 0x7f91a0108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 --2- 172.21.15.26:0/1633570895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91a01035f0 0x7f91a01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 -- 172.21.15.26:0/1633570895 >> 172.21.15.26:0/1633570895 conn(0x7f91a00fcf20 msgr2=0x7f91a00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 -- 172.21.15.26:0/1633570895 shutdown_connections 2024-05-20T21:02:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.273+0000 7f91a49f0640 1 -- 172.21.15.26:0/1633570895 wait complete. 2024-05-20T21:02:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 Processor -- start 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 -- start start 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 0x7f91a0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91a0105f20 0x7f91a0199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a01a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91a010e510 con 0x7f91a0108850 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91a010e390 con 0x7f91a01035f0 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.274+0000 7f91a49f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91a010e690 con 0x7f91a0105f20 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a01a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 0x7f91a0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a01a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56008/0 (socket says 172.21.15.26:56008) 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91a0105f20 0x7f91a0199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919ed9d640 1 -- 172.21.15.26:0/2005090776 learned_addr learned my addr 172.21.15.26:0/2005090776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 0x7f91a0198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35958/0 (socket says 172.21.15.26:35958) 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 -- 172.21.15.26:0/2005090776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91a0105f20 msgr2=0x7f91a0199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91a0105f20 0x7f91a0199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 -- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 msgr2=0x7f91a01a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a01a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 -- 172.21.15.26:0/2005090776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a01a0770 con 0x7f91a01035f0 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f919e59c640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 0x7f91a0198b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9f0 tx=0x7f919000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f918f7fe640 1 -- 172.21.15.26:0/2005090776 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000ccc0 con 0x7f91a01035f0 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f918f7fe640 1 -- 172.21.15.26:0/2005090776 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9190005020 con 0x7f91a01035f0 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a01a0a60 con 0x7f91a01035f0 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f918f7fe640 1 -- 172.21.15.26:0/2005090776 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000ce60 con 0x7f91a01035f0 2024-05-20T21:02:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.275+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a01a0f60 con 0x7f91a01035f0 2024-05-20T21:02:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.278+0000 7f918f7fe640 1 -- 172.21.15.26:0/2005090776 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9190004a40 con 0x7f91a01035f0 2024-05-20T21:02:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.278+0000 7f91a77e3740 1 Processor -- start 2024-05-20T21:02:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 -- start start 2024-05-20T21:02:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba26c0 0x558e31ba2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba30b0 0x558e31ba7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31baa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558e31b13c30 con 0x558e31ba8220 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558e31b037d0 con 0x558e31ba30b0 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f91a77e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558e31b07810 con 0x558e31ba26c0 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31baa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba30b0 0x558e31ba7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba26c0 0x558e31ba2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31baa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56014/0 (socket says 172.21.15.26:56014) 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919dd9b640 1 -- 172.21.15.26:0/2721607139 learned_addr learned my addr 172.21.15.26:0/2721607139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f918f7fe640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b28a0 0x7f91880b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919ed9d640 1 -- 172.21.15.26:0/2721607139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba30b0 msgr2=0x558e31ba7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919ed9d640 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba30b0 0x558e31ba7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919ed9d640 1 -- 172.21.15.26:0/2721607139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 msgr2=0x558e31baa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919ed9d640 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31baa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.279+0000 7f919ed9d640 1 -- 172.21.15.26:0/2721607139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e31818210 con 0x558e31ba26c0 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.280+0000 7f918f7fe640 1 -- 172.21.15.26:0/2005090776 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f91900e21d0 con 0x7f91a01035f0 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.280+0000 7f919ed9d640 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba26c0 0x558e31ba2ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9198006010 tx=0x7f9198009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9dc7c520d79abd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.280+0000 7f919ed9d640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b28a0 0x7f91880b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.280+0000 7f918cff9640 1 -- 172.21.15.26:0/2721607139 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198003b30 con 0x558e31ba26c0 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.280+0000 7f918cff9640 1 -- 172.21.15.26:0/2721607139 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198036030 con 0x558e31ba26c0 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.280+0000 7f918cff9640 1 -- 172.21.15.26:0/2721607139 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91980305c0 con 0x558e31ba26c0 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 -- 172.21.15.26:0/2721607139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba26c0 msgr2=0x558e31ba2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f919ed9d640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b28a0 0x7f91880b4d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f919804a000 tx=0x7f919804b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba26c0 0x558e31ba2ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9198006010 tx=0x7f9198009a20 comp rx=0 tx=0).stop 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 -- 172.21.15.26:0/2721607139 shutdown_connections 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31baa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba30b0 0x558e31ba7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2721607139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba26c0 0x558e31ba2ae0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 -- 172.21.15.26:0/2721607139 >> 172.21.15.26:0/2721607139 conn(0x558e31b9dd30 msgr2=0x558e31b9e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 -- 172.21.15.26:0/2721607139 shutdown_connections 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.281+0000 7f91a77e3740 1 -- 172.21.15.26:0/2721607139 wait complete. 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 -- start start 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 Processor -- start 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558e317e9790 con 0x558e31ba8220 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558e31abf890 con 0x558e31ba26c0 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558e31bab050 con 0x558e31ba30b0 2024-05-20T21:02:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f91a77e3740 10 client.0 _ll_register_callbacks cb 0x558e31817a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f919ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35964/0 (socket says 172.21.15.26:35964) 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.282+0000 7f919e59c640 1 -- 172.21.15.26:0/2807891381 learned_addr learned my addr 172.21.15.26:0/2807891381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919e59c640 1 -- 172.21.15.26:0/2807891381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 msgr2=0x558e31bb0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919e59c640 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919e59c640 1 -- 172.21.15.26:0/2807891381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 msgr2=0x558e31bb7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919e59c640 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919e59c640 1 -- 172.21.15.26:0/2807891381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e318e4620 con 0x558e31ba26c0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919dd9b640 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f919e59c640 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb0a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9c0 tx=0x7f91900a9620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413c35342aa2468b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f9185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:02:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f9185ffb640 1 -- 172.21.15.26:0/2807891381 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919001c030 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9198036030 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f9185ffb640 1 -- 172.21.15.26:0/2807891381 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f91900054f0 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f9185ffb640 1 -- 172.21.15.26:0/2807891381 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91900e5e60 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f91a77e3740 10 client.26849 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.283+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e31bbafe0 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f9185ffb640 1 -- 172.21.15.26:0/2807891381 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198036030 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f9185ffb640 1 -- 172.21.15.26:0/2807891381 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f91900b2160 con 0x558e31ba26c0 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f91a77e3740 10 client.26849 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f91a77e3740 10 client.26849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f91a77e3740 20 client.26849 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31bbc6c0 0x558e31bbeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f91a77e3740 4 client.26849 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:02:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e31b15b80 con 0x558e31bbc6c0 2024-05-20T21:02:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f919dd9b640 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31bbc6c0 0x558e31bbeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f919dd9b640 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31bbc6c0 0x558e31bbeaf0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea87d22b4bd28422 server_cookie=24952e3569c52324 in_seq=0 out_seq=0 2024-05-20T21:02:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.284+0000 7f9185ffb640 10 client.26849 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f9185ffb640 1 -- 172.21.15.26:0/2807891381 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558e31b15b80 con 0x558e31bbc6c0 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f9185ffb640 10 client.26849 handle_command_reply: tid=0 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 client.26849 shutdown 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 msgr2=0x558e31bb0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb0a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9c0 tx=0x7f91900a9620 comp rx=0 tx=0).stop 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 shutdown_connections 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31bbc6c0 0x558e31bbeaf0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f9185ffb640 0 client.26849 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2807891381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb0a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 >> 172.21.15.26:0/2807891381 conn(0x558e31b9dd30 msgr2=0x558e31b9cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 shutdown_connections 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 1 -- 172.21.15.26:0/2807891381 wait complete. 2024-05-20T21:02:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.287+0000 7f91a77e3740 20 client.26849 trim_cache size 0 max 16384 2024-05-20T21:02:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.304+0000 7f91a77e3740 1 Processor -- start 2024-05-20T21:02:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.304+0000 7f91a77e3740 1 -- start start 2024-05-20T21:02:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.304+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 0x558e31baba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.304+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba8220 0x558e31babf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f91a77e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01aa4e0 con 0x558e31ba26c0 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f91a77e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01a7db0 con 0x558e31ba8220 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f91a77e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01a5c00 con 0x558e31ba30b0 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 0x558e31baba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 0x558e31baba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56040/0 (socket says 172.21.15.26:56040) 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 -- 172.21.15.26:0/2877466061 learned_addr learned my addr 172.21.15.26:0/2877466061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919ed9d640 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba8220 0x558e31babf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919dd9b640 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 -- 172.21.15.26:0/2877466061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 msgr2=0x558e31bb7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 -- 172.21.15.26:0/2877466061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba8220 msgr2=0x558e31babf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba8220 0x558e31babf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 -- 172.21.15.26:0/2877466061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e31bbf170 con 0x558e31ba26c0 2024-05-20T21:02:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f919e59c640 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 0x558e31baba00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f91900a5810 tx=0x7f9190005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e638233b4821d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.305+0000 7f9187fff640 1 -- 172.21.15.26:0/2877466061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919001c070 con 0x558e31ba26c0 2024-05-20T21:02:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f9187fff640 1 -- 172.21.15.26:0/2877466061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91900e2020 con 0x558e31ba26c0 2024-05-20T21:02:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f9187fff640 1 -- 172.21.15.26:0/2877466061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900a9790 con 0x558e31ba26c0 2024-05-20T21:02:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 -- 172.21.15.26:0/2877466061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 msgr2=0x558e31baba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 0x558e31baba00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f91900a5810 tx=0x7f9190005460 comp rx=0 tx=0).stop 2024-05-20T21:02:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 -- 172.21.15.26:0/2877466061 shutdown_connections 2024-05-20T21:02:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba8220 0x558e31babf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 --2- 172.21.15.26:0/2877466061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba26c0 0x558e31baba00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 -- 172.21.15.26:0/2877466061 >> 172.21.15.26:0/2877466061 conn(0x558e31b9dd30 msgr2=0x558e31b98e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 -- 172.21.15.26:0/2877466061 shutdown_connections 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.306+0000 7f91a77e3740 1 -- 172.21.15.26:0/2877466061 wait complete. 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 -- start start 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 Processor -- start 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01a6270 con 0x558e31ba8220 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01a76e0 con 0x558e31ba26c0 2024-05-20T21:02:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01a6940 con 0x558e31ba30b0 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f91a77e3740 10 client.0 _ll_register_callbacks cb 0x7f91900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f919e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35992/0 (socket says 172.21.15.26:35992) 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f919e59c640 1 -- 172.21.15.26:0/1178332067 learned_addr learned my addr 172.21.15.26:0/1178332067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f919dd9b640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.307+0000 7f919ed9d640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919dd9b640 1 -- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 msgr2=0x558e31bb5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919dd9b640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919dd9b640 1 -- 172.21.15.26:0/1178332067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 msgr2=0x558e31bb6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919dd9b640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919dd9b640 1 -- 172.21.15.26:0/1178332067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e31bb7280 con 0x558e31ba30b0 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919e59c640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:02:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f919dd9b640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb5cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x558e31bbf340 tx=0x7f9194012ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff7d3fd571e28c6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f9185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f9185ffb640 1 -- 172.21.15.26:0/1178332067 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91940035f0 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91900e2020 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f9185ffb640 1 -- 172.21.15.26:0/1178332067 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f9194003790 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f9185ffb640 1 -- 172.21.15.26:0/1178332067 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9194003990 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.308+0000 7f91a77e3740 10 client.26713 fetch_fsmap learned FSMap version 28 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e31c274e0 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f9185ffb640 1 -- 172.21.15.26:0/1178332067 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91900e2020 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f9185ffb640 1 -- 172.21.15.26:0/1178332067 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f9194031050 con 0x558e31ba30b0 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 10 client.26713 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 10 client.26713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 20 client.26713 populate_metadata read hostname 'smithi026' 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31c27e10 0x558e31c2a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:02:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 4 client.26713 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:02:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e31b0f360 con 0x558e31c27e10 2024-05-20T21:02:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.309+0000 7f919e59c640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31c27e10 0x558e31c2a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:02:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.310+0000 7f919e59c640 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31c27e10 0x558e31c2a230 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b5882fcb2b5ccd server_cookie=21bed51c68379401 in_seq=0 out_seq=0 2024-05-20T21:02:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.310+0000 7f9185ffb640 10 client.26713 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f9185ffb640 1 -- 172.21.15.26:0/1178332067 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558e31b0f360 con 0x558e31c27e10 2024-05-20T21:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f9185ffb640 10 client.26713 handle_command_reply: tid=0 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 client.26713 shutdown 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 msgr2=0x558e31bb5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb5cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x558e31bbf340 tx=0x7f9194012ae0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 shutdown_connections 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558e31c27e10 0x558e31c2a230 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e31ba8220 0x558e31bb6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e31ba30b0 0x558e31bb5cd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f9185ffb640 0 client.26713 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 --2- 172.21.15.26:0/1178332067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e31ba26c0 0x558e31bb5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 >> 172.21.15.26:0/1178332067 conn(0x558e31b9dd30 msgr2=0x558e31baba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 shutdown_connections 2024-05-20T21:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 1 -- 172.21.15.26:0/1178332067 wait complete. 2024-05-20T21:02:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.349+0000 7f91a77e3740 20 client.26713 trim_cache size 0 max 16384 2024-05-20T21:02:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:02:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:02:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b28a0 msgr2=0x7f91880b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b28a0 0x7f91880b4d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f919804a000 tx=0x7f919804b000 comp rx=0 tx=0).stop 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 msgr2=0x7f91a0198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 0x7f91a0198b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9f0 tx=0x7f919000eeb0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 shutdown_connections 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b28a0 0x7f91880b4d60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a0108850 0x7f91a01a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91a0105f20 0x7f91a0199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 --2- 172.21.15.26:0/2005090776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91a01035f0 0x7f91a0198b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:02:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 >> 172.21.15.26:0/2005090776 conn(0x7f91a00fcf20 msgr2=0x7f91a0104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:02:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 shutdown_connections 2024-05-20T21:02:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:02:59.350+0000 7f91a49f0640 1 -- 172.21.15.26:0/2005090776 wait complete. 2024-05-20T21:02:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:59 smithi055 ceph-mon[35213]: pgmap v594: 97 pgs: 97 active+clean; 28 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T21:02:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:02:59 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:02:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:59 smithi026 ceph-mon[31990]: pgmap v594: 97 pgs: 97 active+clean; 28 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T21:02:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:02:59 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:02:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:59 smithi144 ceph-mon[35660]: pgmap v594: 97 pgs: 97 active+clean; 28 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T21:02:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:02:59 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:02:59.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 143315 2.26 MB/sec execute 185 sec latency 34.089 ms 2024-05-20T21:03:00.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:03:00.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:03:00.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:03:00.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:03:00.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:03:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:03:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:03:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:03:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:03:00.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 144003 2.25 MB/sec execute 186 sec latency 17.634 ms 2024-05-20T21:03:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:01 smithi055 ceph-mon[35213]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-05-20T21:03:01.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:01 smithi026 ceph-mon[31990]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-05-20T21:03:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:01 smithi144 ceph-mon[35660]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-05-20T21:03:01.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 144821 2.28 MB/sec execute 187 sec latency 11.677 ms 2024-05-20T21:03:02.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 145490 2.29 MB/sec execute 188 sec latency 11.662 ms 2024-05-20T21:03:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:03 smithi055 ceph-mon[35213]: pgmap v596: 97 pgs: 97 active+clean; 29 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-05-20T21:03:03.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:03 smithi026 ceph-mon[31990]: pgmap v596: 97 pgs: 97 active+clean; 29 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-05-20T21:03:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:03 smithi144 ceph-mon[35660]: pgmap v596: 97 pgs: 97 active+clean; 29 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-05-20T21:03:03.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 145969 2.29 MB/sec execute 189 sec latency 17.609 ms 2024-05-20T21:03:04.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 146531 2.29 MB/sec execute 190 sec latency 29.716 ms 2024-05-20T21:03:05.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:05 smithi055 ceph-mon[35213]: pgmap v597: 97 pgs: 97 active+clean; 25 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-20T21:03:05.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:05 smithi026 ceph-mon[31990]: pgmap v597: 97 pgs: 97 active+clean; 25 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-20T21:03:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:05 smithi144 ceph-mon[35660]: pgmap v597: 97 pgs: 97 active+clean; 25 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-20T21:03:05.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 147302 2.29 MB/sec execute 191 sec latency 12.607 ms 2024-05-20T21:03:06.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:03:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:06.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 147928 2.29 MB/sec execute 192 sec latency 18.132 ms 2024-05-20T21:03:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:07 smithi055 ceph-mon[35213]: pgmap v598: 97 pgs: 97 active+clean; 26 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-05-20T21:03:07.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:07 smithi026 ceph-mon[31990]: pgmap v598: 97 pgs: 97 active+clean; 26 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-05-20T21:03:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:07 smithi144 ceph-mon[35660]: pgmap v598: 97 pgs: 97 active+clean; 26 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-05-20T21:03:07.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 148763 2.32 MB/sec execute 193 sec latency 18.263 ms 2024-05-20T21:03:08.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 -- 172.21.15.26:0/298119127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 msgr2=0x7ff4f0106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/298119127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0106810 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400aad0 tx=0x7ff4e400ade0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 -- 172.21.15.26:0/298119127 shutdown_connections 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/298119127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 0x7ff4f010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/298119127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f0101e60 0x7ff4f0102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/298119127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0106810 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 -- 172.21.15.26:0/298119127 >> 172.21.15.26:0/298119127 conn(0x7ff4f0078ab0 msgr2=0x7ff4f00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 -- 172.21.15.26:0/298119127 shutdown_connections 2024-05-20T21:03:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.211+0000 7ff4f5cba700 1 -- 172.21.15.26:0/298119127 wait complete. 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.212+0000 7ff4f5cba700 1 Processor -- start 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.212+0000 7ff4f5cba700 1 -- start start 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.212+0000 7ff4f5cba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f0101e60 0x7ff4f010a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4f5cba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0108b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4f5cba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 0x7ff4f01091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4f5cba700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01113a0 con 0x7ff4f0102810 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4f5cba700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0111220 con 0x7ff4f0106ef0 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4f5cba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0111520 con 0x7ff4f0101e60 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0108b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0108b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54764/0 (socket says 172.21.15.26:54764) 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 0x7ff4f01091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4eeffd700 1 -- 172.21.15.26:0/3217201919 learned_addr learned my addr 172.21.15.26:0/3217201919 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f0101e60 0x7ff4f010a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 -- 172.21.15.26:0/3217201919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f0101e60 msgr2=0x7ff4f010a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f0101e60 0x7ff4f010a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 -- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 msgr2=0x7ff4f0108b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0108b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 -- 172.21.15.26:0/3217201919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01ac470 con 0x7ff4f0106ef0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.213+0000 7ff4effff700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 0x7ff4f01091c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800efc0 tx=0x7ff4e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.214+0000 7ff4ecff9700 1 -- 172.21.15.26:0/3217201919 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80092e0 con 0x7ff4f0106ef0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.214+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f01ac6a0 con 0x7ff4f0106ef0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.214+0000 7ff4ecff9700 1 -- 172.21.15.26:0/3217201919 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e8004bb0 con 0x7ff4f0106ef0 2024-05-20T21:03:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.214+0000 7ff4ecff9700 1 -- 172.21.15.26:0/3217201919 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004020 con 0x7ff4f0106ef0 2024-05-20T21:03:08.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.214+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01acb60 con 0x7ff4f0106ef0 2024-05-20T21:03:08.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.217+0000 7ff4ecff9700 1 -- 172.21.15.26:0/3217201919 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff4e801b030 con 0x7ff4f0106ef0 2024-05-20T21:03:08.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.217+0000 7ff4ecff9700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4d80b2880 0x7ff4d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4d80b2880 0x7ff4d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4ecff9700 1 -- 172.21.15.26:0/3217201919 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7ff4e80e1ce0 con 0x7ff4f0106ef0 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 Processor -- start 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 -- start start 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4d80b2880 0x7ff4d80b4d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000ac50 tx=0x7ff4e000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 0x55a73566dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566e210 0x55a735671940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a7356723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7355ccc90 con 0x55a73566d860 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a7355bc830 con 0x55a73566e210 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff500550b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7355c0870 con 0x55a735671f60 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 0x55a73566dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 0x55a73566dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54774/0 (socket says 172.21.15.26:54774) 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/4012031186 learned_addr learned my addr 172.21.15.26:0/4012031186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.218+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566e210 0x55a735671940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4effff700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a7356723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4eeffd700 1 -- 172.21.15.26:0/4012031186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 msgr2=0x55a7356723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a7356723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4eeffd700 1 -- 172.21.15.26:0/4012031186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 msgr2=0x55a73566dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 0x55a73566dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4eeffd700 1 -- 172.21.15.26:0/4012031186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73550c8d0 con 0x55a73566e210 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4effff700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a7356723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 0x55a73566dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566e210 0x55a735671940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4006010 tx=0x7ff4e400a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e29a1a2a5bf931f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4d5ffb700 1 -- 172.21.15.26:0/4012031186 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4003780 con 0x55a73566e210 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4d5ffb700 1 -- 172.21.15.26:0/4012031186 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e4026070 con 0x55a73566e210 2024-05-20T21:03:08.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.219+0000 7ff4d5ffb700 1 -- 172.21.15.26:0/4012031186 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4005320 con 0x55a73566e210 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 -- 172.21.15.26:0/4012031186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566e210 msgr2=0x55a735671940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566e210 0x55a735671940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4006010 tx=0x7ff4e400a750 comp rx=0 tx=0).stop 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 -- 172.21.15.26:0/4012031186 shutdown_connections 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566d860 0x55a73566dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a7356723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 --2- 172.21.15.26:0/4012031186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566e210 0x55a735671940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 -- 172.21.15.26:0/4012031186 >> 172.21.15.26:0/4012031186 conn(0x55a73559b080 msgr2=0x55a735667e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 -- 172.21.15.26:0/4012031186 shutdown_connections 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.220+0000 7ff500550b80 1 -- 172.21.15.26:0/4012031186 wait complete. 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 -- start start 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 Processor -- start 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a7356793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566e210 0x55a7356766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a735671f60 0x55a735676c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7354c2fa0 con 0x55a735671f60 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a73555e7c0 con 0x55a73566d860 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a73567a0c0 con 0x55a73566e210 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff500550b80 10 client.0 _ll_register_callbacks cb 0x55a735557920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff4effff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a735671f60 0x55a735676c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a7356793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff4effff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a735671f60 0x55a735676c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54782/0 (socket says 172.21.15.26:54782) 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.221+0000 7ff4effff700 1 -- 172.21.15.26:0/1952510831 learned_addr learned my addr 172.21.15.26:0/1952510831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566e210 0x55a7356766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/1952510831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566e210 msgr2=0x55a7356766c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566e210 0x55a7356766c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/1952510831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a735671f60 msgr2=0x55a735676c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a735671f60 0x55a735676c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/1952510831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73555b690 con 0x55a73566d860 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566e210 0x55a7356766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a7356793c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0003900 tx=0x7ff4e00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71091eac5756df8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ceffd700 1 -- 172.21.15.26:0/1952510831 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0016070 con 0x55a73566d860 2024-05-20T21:03:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e4026070 con 0x55a73566d860 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ceffd700 1 -- 172.21.15.26:0/1952510831 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7ff4e0010a80 con 0x55a73566d860 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff4ceffd700 1 -- 172.21.15.26:0/1952510831 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4e0010c80 con 0x55a73566d860 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff500550b80 10 client.26867 fetch_fsmap learned FSMap version 28 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.222+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a735684880 con 0x55a73566d860 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff4ceffd700 1 -- 172.21.15.26:0/1952510831 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e4026070 con 0x55a73566d860 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff4ceffd700 1 -- 172.21.15.26:0/1952510831 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7ff4e0030050 con 0x55a73566d860 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff500550b80 10 client.26867 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff500550b80 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff500550b80 20 client.26867 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff500550b80 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a735686f10 0x55a735689300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff500550b80 4 client.26867 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7356399e0 con 0x55a735686f10 2024-05-20T21:03:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.223+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a735686f10 0x55a735689300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a735686f10 0x55a735689300 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a5d4acfbc45bf1 server_cookie=1dc7705f2a27ba4e in_seq=0 out_seq=0 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff4ceffd700 10 client.26867 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff4ceffd700 1 -- 172.21.15.26:0/1952510831 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a7356399e0 con 0x55a735686f10 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff4ceffd700 10 client.26867 handle_command_reply: tid=0 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff500550b80 1 client.26867 shutdown 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 msgr2=0x55a7356793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.224+0000 7ff500550b80 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a7356793c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0003900 tx=0x7ff4e00047c0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 shutdown_connections 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a735686f10 0x55a735689300 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a735671f60 0x55a735676c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a7356793c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 --2- 172.21.15.26:0/1952510831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566e210 0x55a7356766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff4ceffd700 0 client.26867 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:08.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 >> 172.21.15.26:0/1952510831 conn(0x55a73559b080 msgr2=0x55a735673350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:08.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 shutdown_connections 2024-05-20T21:03:08.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 1 -- 172.21.15.26:0/1952510831 wait complete. 2024-05-20T21:03:08.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.225+0000 7ff500550b80 20 client.26867 trim_cache size 0 max 16384 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 Processor -- start 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 -- start start 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566d860 0x55a735678600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a735667c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a735671f60 0x55a7356681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01b6a50 con 0x55a73566e210 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01b42e0 con 0x55a735671f60 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff500550b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01b2130 con 0x55a73566d860 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a735667c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a735667c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54792/0 (socket says 172.21.15.26:54792) 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.248+0000 7ff4eeffd700 1 -- 172.21.15.26:0/553642363 learned_addr learned my addr 172.21.15.26:0/553642363 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566d860 0x55a735678600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4effff700 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a735671f60 0x55a7356681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4eeffd700 1 -- 172.21.15.26:0/553642363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566d860 msgr2=0x55a735678600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566d860 0x55a735678600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4eeffd700 1 -- 172.21.15.26:0/553642363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a735671f60 msgr2=0x55a7356681d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a735671f60 0x55a7356681d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4eeffd700 1 -- 172.21.15.26:0/553642363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a735674c30 con 0x55a73566e210 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566d860 0x55a735678600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a735667c90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400a860 tx=0x7ff4e40059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4207ef1325e12dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4d57fa700 1 -- 172.21.15.26:0/553642363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4003d30 con 0x55a73566e210 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4d57fa700 1 -- 172.21.15.26:0/553642363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e4026040 con 0x55a73566e210 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.249+0000 7ff4d57fa700 1 -- 172.21.15.26:0/553642363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40333b0 con 0x55a73566e210 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 -- 172.21.15.26:0/553642363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 msgr2=0x55a735667c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a735667c90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400a860 tx=0x7ff4e40059e0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 -- 172.21.15.26:0/553642363 shutdown_connections 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a735671f60 0x55a7356681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a73566d860 0x55a735678600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 --2- 172.21.15.26:0/553642363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a735667c90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 -- 172.21.15.26:0/553642363 >> 172.21.15.26:0/553642363 conn(0x55a73559b080 msgr2=0x55a735677ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 -- 172.21.15.26:0/553642363 shutdown_connections 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 -- 172.21.15.26:0/553642363 wait complete. 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.250+0000 7ff500550b80 1 -- start start 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 Processor -- start 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a73567f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a73567f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a73567c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01b27a0 con 0x55a73566e210 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01b3c10 con 0x55a73566d860 2024-05-20T21:03:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01b2e70 con 0x55a735671f60 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff500550b80 10 client.0 _ll_register_callbacks cb 0x55a735557920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a73567f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a73567f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a73567f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54794/0 (socket says 172.21.15.26:54794) 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a73567f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39584/0 (socket says 172.21.15.26:39584) 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4eeffd700 1 -- 172.21.15.26:0/686828034 learned_addr learned my addr 172.21.15.26:0/686828034 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4effff700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a73567c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/686828034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 msgr2=0x55a73567c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a73567c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/686828034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 msgr2=0x55a73567f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a73567f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4ef7fe700 1 -- 172.21.15.26:0/686828034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73567c860 con 0x55a73566d860 2024-05-20T21:03:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a73567f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.251+0000 7ff4effff700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a73567c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ef7fe700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a73567f350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000fa30 tx=0x7ff4e001d740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b0b0fae7a6af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ceffd700 1 -- 172.21.15.26:0/686828034 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e001dd20 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ceffd700 1 -- 172.21.15.26:0/686828034 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7ff4e001e410 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e4026040 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ceffd700 1 -- 172.21.15.26:0/686828034 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4e000ae40 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 10 client.26873 fetch_fsmap learned FSMap version 28 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7356f2220 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ceffd700 1 -- 172.21.15.26:0/686828034 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e4026040 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff4ceffd700 1 -- 172.21.15.26:0/686828034 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7ff4e0042020 con 0x55a73566d860 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 10 client.26873 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:03:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 10 client.26873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 20 client.26873 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.252+0000 7ff500550b80 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a7356f2900 0x55a7356f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff500550b80 4 client.26873 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7355cebe0 con 0x55a7356f2900 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a7356f2900 0x55a7356f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff4ceffd700 10 client.26873 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff4eeffd700 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a7356f2900 0x55a7356f4d20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98bfb584414fae07 server_cookie=e8af10fa584e9713 in_seq=0 out_seq=0 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff4ceffd700 1 -- 172.21.15.26:0/686828034 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a7355cebe0 con 0x55a7356f2900 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff4ceffd700 10 client.26873 handle_command_reply: tid=0 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff500550b80 1 client.26873 shutdown 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 msgr2=0x55a73567f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.253+0000 7ff500550b80 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a73567f350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000fa30 tx=0x7ff4e001d740 comp rx=0 tx=0).stop 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 shutdown_connections 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a7356f2900 0x55a7356f4d20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a735671f60 0x55a73567c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a73566d860 0x55a73567f350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 --2- 172.21.15.26:0/686828034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73566e210 0x55a73567f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 >> 172.21.15.26:0/686828034 conn(0x55a73559b080 msgr2=0x55a735678780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff4ceffd700 0 client.26873 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 shutdown_connections 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 1 -- 172.21.15.26:0/686828034 wait complete. 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff500550b80 20 client.26873 trim_cache size 0 max 16384 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:08.259 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4d80b2880 msgr2=0x7ff4d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4d80b2880 0x7ff4d80b4d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000ac50 tx=0x7ff4e000a380 comp rx=0 tx=0).stop 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 msgr2=0x7ff4f01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.254+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 0x7ff4f01091c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800efc0 tx=0x7ff4e800c3a0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 shutdown_connections 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f0106ef0 0x7ff4f01091c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f0101e60 0x7ff4f010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4d80b2880 0x7ff4d80b4d40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 --2- 172.21.15.26:0/3217201919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f0102810 0x7ff4f0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 >> 172.21.15.26:0/3217201919 conn(0x7ff4f0078ab0 msgr2=0x7ff4f00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 shutdown_connections 2024-05-20T21:03:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:08.255+0000 7ff4f5cba700 1 -- 172.21.15.26:0/3217201919 wait complete. 2024-05-20T21:03:08.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 149273 2.33 MB/sec execute 194 sec latency 17.809 ms 2024-05-20T21:03:09.080 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:03:09.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:03:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:09 smithi055 ceph-mon[35213]: pgmap v599: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-05-20T21:03:09.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:09 smithi026 ceph-mon[31990]: pgmap v599: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-05-20T21:03:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:09 smithi144 ceph-mon[35660]: pgmap v599: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-05-20T21:03:09.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 149711 2.32 MB/sec execute 195 sec latency 20.196 ms 2024-05-20T21:03:10.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 150425 2.33 MB/sec execute 196 sec latency 17.712 ms 2024-05-20T21:03:11.165 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:11 smithi026 ceph-mon[31990]: pgmap v600: 97 pgs: 97 active+clean; 26 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.281+0000 7f7bb63f1700 1 -- 172.21.15.26:0/2351536367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 msgr2=0x7f7bb01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.281+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/2351536367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb01028c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009990 tx=0x7f7ba4009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 -- 172.21.15.26:0/2351536367 shutdown_connections 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/2351536367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb0102e00 0x7f7bb010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/2351536367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 0x7f7bb0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/2351536367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb01028c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 -- 172.21.15.26:0/2351536367 >> 172.21.15.26:0/2351536367 conn(0x7f7bb00793f0 msgr2=0x7f7bb0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 -- 172.21.15.26:0/2351536367 shutdown_connections 2024-05-20T21:03:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 -- 172.21.15.26:0/2351536367 wait complete. 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 Processor -- start 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.282+0000 7f7bb63f1700 1 -- start start 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb63f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb63f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb0102e00 0x7f7bb019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb63f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 0x7f7bb0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb63f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb01124a0 con 0x7f7bb0102440 2024-05-20T21:03:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb63f1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0112320 con 0x7f7bb0102e00 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb63f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0112620 con 0x7f7bb0108520 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58456/0 (socket says 172.21.15.26:58456) 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 -- 172.21.15.26:0/1273815403 learned_addr learned my addr 172.21.15.26:0/1273815403 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baf7fe700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb0102e00 0x7f7bb019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb498e700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 0x7f7bb0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 -- 172.21.15.26:0/1273815403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 msgr2=0x7f7bb0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 0x7f7bb0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 -- 172.21.15.26:0/1273815403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb0102e00 msgr2=0x7f7bb019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb0102e00 0x7f7bb019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7baffff700 1 -- 172.21.15.26:0/1273815403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb019a0b0 con 0x7f7bb0102440 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.283+0000 7f7bb498e700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 0x7f7bb0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.284+0000 7f7baffff700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb019f790 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000e9c0 tx=0x7f7ba000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:11.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.284+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000cbd0 con 0x7f7bb0102440 2024-05-20T21:03:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.284+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ba000cd70 con 0x7f7bb0102440 2024-05-20T21:03:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.284+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb019a3a0 con 0x7f7bb0102440 2024-05-20T21:03:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.284+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0005540 con 0x7f7bb0102440 2024-05-20T21:03:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.284+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb0103fb0 con 0x7f7bb0102440 2024-05-20T21:03:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.285+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb006a230 con 0x7f7bb0102440 2024-05-20T21:03:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.287+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7ba0004750 con 0x7f7bb0102440 2024-05-20T21:03:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.287+0000 7f7bad7fa700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b980b2880 0x7f7b980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.288+0000 7f7baf7fe700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b980b2880 0x7f7b980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:11.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.288+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f7ba00e2700 con 0x7f7bb0102440 2024-05-20T21:03:11.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.288+0000 7f7baf7fe700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b980b2880 0x7f7b980b4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0079bf0 tx=0x7f7ba4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.292+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ba0054df0 con 0x7f7bb0102440 2024-05-20T21:03:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:11 smithi055 ceph-mon[35213]: pgmap v600: 97 pgs: 97 active+clean; 26 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-20T21:03:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:11 smithi144 ceph-mon[35660]: pgmap v600: 97 pgs: 97 active+clean; 26 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-20T21:03:11.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 151064 2.32 MB/sec execute 197 sec latency 19.029 ms 2024-05-20T21:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.591+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bb019ae80 con 0x7f7bb0102440 2024-05-20T21:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.592+0000 7f7bad7fa700 1 -- 172.21.15.26:0/1273815403 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11868 (secure 0 0 0) 0x7f7ba00aa3c0 con 0x7f7bb0102440 2024-05-20T21:03:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:03:11.592 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T20:59:43.103982+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1050250189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1050250189},{"type":"v1","addr":"172.21.15.144:6837","nonce":1050250189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:03:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b980b2880 msgr2=0x7f7b980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b980b2880 0x7f7b980b4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0079bf0 tx=0x7f7ba4005fb0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 msgr2=0x7f7bb019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb019f790 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000e9c0 tx=0x7f7ba000ecd0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 shutdown_connections 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b980b2880 0x7f7b980b4d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb0102440 0x7f7bb019f790 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bb0108520 0x7f7bb0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 --2- 172.21.15.26:0/1273815403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb0102e00 0x7f7bb019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 >> 172.21.15.26:0/1273815403 conn(0x7f7bb00793f0 msgr2=0x7f7bb006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.594+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 shutdown_connections 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:11.595+0000 7f7bb63f1700 1 -- 172.21.15.26:0/1273815403 wait complete. 2024-05-20T21:03:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-05-20T21:03:12.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:12 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1273815403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:12.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:12 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1273815403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 -- 172.21.15.26:0/3403017130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 msgr2=0x7f2d6c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 --2- 172.21.15.26:0/3403017130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c10abe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d6402a410 comp rx=0 tx=0).stop 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 -- 172.21.15.26:0/3403017130 shutdown_connections 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 --2- 172.21.15.26:0/3403017130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c10abe0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 --2- 172.21.15.26:0/3403017130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c105e60 0x7f2d6c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 --2- 172.21.15.26:0/3403017130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c103530 0x7f2d6c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 -- 172.21.15.26:0/3403017130 >> 172.21.15.26:0/3403017130 conn(0x7f2d6c0fcf20 msgr2=0x7f2d6c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 -- 172.21.15.26:0/3403017130 shutdown_connections 2024-05-20T21:03:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.453+0000 7f2d738a0640 1 -- 172.21.15.26:0/3403017130 wait complete. 2024-05-20T21:03:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 Processor -- start 2024-05-20T21:03:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 -- start start 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c103530 0x7f2d6c102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c105e60 0x7f2d6c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c10e490 con 0x7f2d6c108790 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c10e310 con 0x7f2d6c105e60 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d738a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c10e610 con 0x7f2d6c103530 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58482/0 (socket says 172.21.15.26:58482) 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 -- 172.21.15.26:0/1416544691 learned_addr learned my addr 172.21.15.26:0/1416544691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d7163c640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c103530 0x7f2d6c102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c105e60 0x7f2d6c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 -- 172.21.15.26:0/1416544691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c103530 msgr2=0x7f2d6c102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c103530 0x7f2d6c102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 -- 172.21.15.26:0/1416544691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c105e60 msgr2=0x7f2d6c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c105e60 0x7f2d6c1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.454+0000 7f2d71e3d640 1 -- 172.21.15.26:0/1416544691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c1022b0 con 0x7f2d6c108790 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.455+0000 7f2d71e3d640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c101a20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2d64004770 tx=0x7f2d640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.455+0000 7f2d627fc640 1 -- 172.21.15.26:0/1416544691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6403f030 con 0x7f2d6c108790 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.455+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c1a2bc0 con 0x7f2d6c108790 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.455+0000 7f2d627fc640 1 -- 172.21.15.26:0/1416544691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d64032e10 con 0x7f2d6c108790 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.455+0000 7f2d627fc640 1 -- 172.21.15.26:0/1416544691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64035070 con 0x7f2d6c108790 2024-05-20T21:03:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.455+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c1a3110 con 0x7f2d6c108790 2024-05-20T21:03:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.457+0000 7f2d627fc640 1 -- 172.21.15.26:0/1416544691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2d64042030 con 0x7f2d6c108790 2024-05-20T21:03:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 Processor -- start 2024-05-20T21:03:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 -- start start 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c9220 0x5569708cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556970834c30 con 0x5569708c40b0 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5569708247d0 con 0x5569708c9220 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d765c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556970828810 con 0x5569708c36c0 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d627fc640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d580b28a0 0x7f2d580b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58496/0 (socket says 172.21.15.26:58496) 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.458+0000 7f2d7163c640 1 -- 172.21.15.26:0/852977145 learned_addr learned my addr 172.21.15.26:0/852977145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c9220 0x5569708cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d71e3d640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d580b28a0 0x7f2d580b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d627fc640 1 -- 172.21.15.26:0/1416544691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2d64102730 con 0x7f2d6c108790 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 -- 172.21.15.26:0/852977145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 msgr2=0x5569708c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 -- 172.21.15.26:0/852977145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 msgr2=0x5569708c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 -- 172.21.15.26:0/852977145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556970539210 con 0x5569708c9220 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d7163c640 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d71e3d640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d580b28a0 0x7f2d580b4d60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2d641001d0 tx=0x7f2d640be670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c9220 0x5569708cb6b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009fd0 tx=0x7f2d5c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1340c333b4d58e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d577fe640 1 -- 172.21.15.26:0/852977145 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c004050 con 0x5569708c9220 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d577fe640 1 -- 172.21.15.26:0/852977145 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5c005020 con 0x5569708c9220 2024-05-20T21:03:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.459+0000 7f2d577fe640 1 -- 172.21.15.26:0/852977145 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c004a90 con 0x5569708c9220 2024-05-20T21:03:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 -- 172.21.15.26:0/852977145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c9220 msgr2=0x5569708cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c9220 0x5569708cb6b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009fd0 tx=0x7f2d5c0096c0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 -- 172.21.15.26:0/852977145 shutdown_connections 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c9220 0x5569708cb6b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 --2- 172.21.15.26:0/852977145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 -- 172.21.15.26:0/852977145 >> 172.21.15.26:0/852977145 conn(0x5569708bed30 msgr2=0x5569708bf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 -- 172.21.15.26:0/852977145 shutdown_connections 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 -- 172.21.15.26:0/852977145 wait complete. 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.460+0000 7f2d765c0740 1 -- start start 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 Processor -- start 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c9220 0x5569708d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55697050a790 con 0x5569708c9220 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5569707e0890 con 0x5569708c40b0 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5569708cc1d0 con 0x5569708c36c0 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d765c0740 10 client.0 _ll_register_callbacks cb 0x556970538a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d70e3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c9220 0x5569708d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33080/0 (socket says 172.21.15.26:33080) 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d7163c640 1 -- 172.21.15.26:0/4116020402 learned_addr learned my addr 172.21.15.26:0/4116020402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.461+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d7163c640 1 -- 172.21.15.26:0/4116020402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 msgr2=0x5569708cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d7163c640 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d7163c640 1 -- 172.21.15.26:0/4116020402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c9220 msgr2=0x5569708d4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d7163c640 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c9220 0x5569708d4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d7163c640 1 -- 172.21.15.26:0/4116020402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556970605620 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d7163c640 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708cdd30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011d10 tx=0x7f2d6800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421daf484d4ae11 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d54ff9640 1 -- 172.21.15.26:0/4116020402 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6801f060 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5c005020 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d54ff9640 1 -- 172.21.15.26:0/4116020402 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2d68017070 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d54ff9640 1 -- 172.21.15.26:0/4116020402 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d68003970 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d765c0740 10 client.26891 fetch_fsmap learned FSMap version 28 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569708d7e10 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.462+0000 7f2d54ff9640 1 -- 172.21.15.26:0/4116020402 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5c005020 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d54ff9640 1 -- 172.21.15.26:0/4116020402 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f2d68023030 con 0x5569708c40b0 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d765c0740 10 client.26891 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d765c0740 10 client.26891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d765c0740 20 client.26891 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d765c0740 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x5569708db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d765c0740 4 client.26891 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556970836b80 con 0x5569708d94f0 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x5569708db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x5569708db8e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c79e675bc2b6f62 server_cookie=246c3d45eaa5b44 in_seq=0 out_seq=0 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.463+0000 7f2d54ff9640 10 client.26891 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d54ff9640 1 -- 172.21.15.26:0/4116020402 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556970836b80 con 0x5569708d94f0 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d54ff9640 10 client.26891 handle_command_reply: tid=0 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 client.26891 shutdown 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 msgr2=0x5569708cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708cdd30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011d10 tx=0x7f2d6800f790 comp rx=0 tx=0).stop 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 shutdown_connections 2024-05-20T21:03:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x5569708db8e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c9220 0x5569708d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.464+0000 7f2d765c0740 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708cdd30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.465+0000 7f2d54ff9640 0 client.26891 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.465+0000 7f2d765c0740 1 --2- 172.21.15.26:0/4116020402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c36c0 0x5569708cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.465+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 >> 172.21.15.26:0/4116020402 conn(0x5569708bed30 msgr2=0x5569708bdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.465+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 shutdown_connections 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.465+0000 7f2d765c0740 1 -- 172.21.15.26:0/4116020402 wait complete. 2024-05-20T21:03:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.465+0000 7f2d765c0740 20 client.26891 trim_cache size 0 max 16384 2024-05-20T21:03:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 Processor -- start 2024-05-20T21:03:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 -- start start 2024-05-20T21:03:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c36c0 0x5569708c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ae9e0 con 0x5569708c40b0 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ac2b0 con 0x5569708c36c0 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.483+0000 7f2d765c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1aa100 con 0x5569708c9220 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708d4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58526/0 (socket says 172.21.15.26:58526) 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 -- 172.21.15.26:0/936443917 learned_addr learned my addr 172.21.15.26:0/936443917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c36c0 0x5569708c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 -- 172.21.15.26:0/936443917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 msgr2=0x5569708c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 -- 172.21.15.26:0/936443917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c36c0 msgr2=0x5569708c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c36c0 0x5569708c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 -- 172.21.15.26:0/936443917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569708cc8d0 con 0x5569708c40b0 2024-05-20T21:03:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c36c0 0x5569708c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d7163c640 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708d4d30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2d68000f80 tx=0x7f2d68009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8d51f21bc98a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d56ffd640 1 -- 172.21.15.26:0/936443917 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6801f030 con 0x5569708c40b0 2024-05-20T21:03:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d56ffd640 1 -- 172.21.15.26:0/936443917 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d680049c0 con 0x5569708c40b0 2024-05-20T21:03:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.484+0000 7f2d56ffd640 1 -- 172.21.15.26:0/936443917 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68004560 con 0x5569708c40b0 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 -- 172.21.15.26:0/936443917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 msgr2=0x5569708d4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708d4d30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2d68000f80 tx=0x7f2d68009570 comp rx=0 tx=0).stop 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 -- 172.21.15.26:0/936443917 shutdown_connections 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c40b0 0x5569708d4d30 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 --2- 172.21.15.26:0/936443917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c36c0 0x5569708c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 -- 172.21.15.26:0/936443917 >> 172.21.15.26:0/936443917 conn(0x5569708bed30 msgr2=0x5569708c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 -- 172.21.15.26:0/936443917 shutdown_connections 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.485+0000 7f2d765c0740 1 -- 172.21.15.26:0/936443917 wait complete. 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 -- start start 2024-05-20T21:03:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 Processor -- start 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 0x5569708d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1aa770 con 0x5569708c36c0 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1abbe0 con 0x5569708c40b0 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1aae40 con 0x5569708c9220 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d70e3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 0x5569708d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d765c0740 10 client.0 _ll_register_callbacks cb 0x7f2d68009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.486+0000 7f2d7163c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 0x5569708d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58540/0 (socket says 172.21.15.26:58540) 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 -- 172.21.15.26:0/142401225 learned_addr learned my addr 172.21.15.26:0/142401225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 -- 172.21.15.26:0/142401225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 msgr2=0x5569708d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 -- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 msgr2=0x5569708d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 -- 172.21.15.26:0/142401225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569708d4350 con 0x5569708c36c0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d7163c640 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d70e3b640 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 0x5569708d2860 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0115c0 tx=0x7f2d5c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614c768f40c71e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d54ff9640 1 -- 172.21.15.26:0/142401225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c011250 con 0x5569708c36c0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d54ff9640 1 -- 172.21.15.26:0/142401225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8327+0+0 (secure 0 0 0) 0x7f2d5c0040c0 con 0x5569708c36c0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d680049c0 con 0x5569708c36c0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d54ff9640 1 -- 172.21.15.26:0/142401225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d5c0042c0 con 0x5569708c36c0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d765c0740 10 client.17436 fetch_fsmap learned FSMap version 28 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.487+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569709443a0 con 0x5569708c36c0 2024-05-20T21:03:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d54ff9640 1 -- 172.21.15.26:0/142401225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d680049c0 con 0x5569708c36c0 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d54ff9640 1 -- 172.21.15.26:0/142401225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f2d5c00f820 con 0x5569708c36c0 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d765c0740 10 client.17436 fetch_fsmap finished waiting for FSMap version 28 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d765c0740 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d765c0740 20 client.17436 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d765c0740 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x556970946b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d765c0740 4 client.17436 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556970830360 con 0x5569708d94f0 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.488+0000 7f2d7163c640 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x556970946b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.489+0000 7f2d54ff9640 10 client.17436 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.489+0000 7f2d7163c640 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x556970946b60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb375c6df273f78 server_cookie=e82e9c37776d4ed7 in_seq=0 out_seq=0 2024-05-20T21:03:12.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.529+0000 7f2d54ff9640 1 -- 172.21.15.26:0/142401225 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556970830360 con 0x5569708d94f0 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.529+0000 7f2d54ff9640 10 client.17436 handle_command_reply: tid=0 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.529+0000 7f2d765c0740 1 client.17436 shutdown 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.529+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 msgr2=0x5569708d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.529+0000 7f2d765c0740 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 0x5569708d2860 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0115c0 tx=0x7f2d5c005850 comp rx=0 tx=0).stop 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 shutdown_connections 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5569708d94f0 0x556970946b60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569708c9220 0x5569708d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5569708c40b0 0x5569708d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d54ff9640 0 client.17436 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 --2- 172.21.15.26:0/142401225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569708c36c0 0x5569708d2860 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 >> 172.21.15.26:0/142401225 conn(0x5569708bed30 msgr2=0x5569708c1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 shutdown_connections 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 1 -- 172.21.15.26:0/142401225 wait complete. 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.530+0000 7f2d765c0740 20 client.17436 trim_cache size 0 max 16384 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:03:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d580b28a0 msgr2=0x7f2d580b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d580b28a0 0x7f2d580b4d60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2d641001d0 tx=0x7f2d640be670 comp rx=0 tx=0).stop 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 msgr2=0x7f2d6c101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c101a20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2d64004770 tx=0x7f2d640042f0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 shutdown_connections 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2d580b28a0 0x7f2d580b4d60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c108790 0x7f2d6c101a20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c105e60 0x7f2d6c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 --2- 172.21.15.26:0/1416544691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c103530 0x7f2d6c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 >> 172.21.15.26:0/1416544691 conn(0x7f2d6c0fcf20 msgr2=0x7f2d6c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 shutdown_connections 2024-05-20T21:03:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:12.531+0000 7f2d738a0640 1 -- 172.21.15.26:0/1416544691 wait complete. 2024-05-20T21:03:12.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:12 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1273815403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:12.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 151719 2.34 MB/sec execute 198 sec latency 38.262 ms 2024-05-20T21:03:12.774 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:13.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:13 smithi026 ceph-mon[31990]: pgmap v601: 97 pgs: 97 active+clean; 27 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-05-20T21:03:13.407 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:13 smithi144 ceph-mon[35660]: pgmap v601: 97 pgs: 97 active+clean; 27 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-05-20T21:03:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:13 smithi055 ceph-mon[35213]: pgmap v601: 97 pgs: 97 active+clean; 27 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-05-20T21:03:13.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 152372 2.34 MB/sec execute 199 sec latency 15.735 ms 2024-05-20T21:03:13.800 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:03:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: ignoring --setuser ceph since I am not root 2024-05-20T21:03:13.800 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:03:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: ignoring --setgroup ceph since I am not root 2024-05-20T21:03:13.800 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:03:13 smithi144 ceph-mds[65957]: -- 172.21.15.144:0/3102069728 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56500691fa40 con 0x5650068f2800 2024-05-20T21:03:13.801 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:03:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: starting mds.c at 2024-05-20T21:03:14.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:14 smithi026 ceph-mon[31990]: daemon mds.c restarted 2024-05-20T21:03:14.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:14 smithi055 ceph-mon[35213]: daemon mds.c restarted 2024-05-20T21:03:14.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:14 smithi144 ceph-mon[35660]: daemon mds.c restarted 2024-05-20T21:03:14.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 152959 2.35 MB/sec execute 200 sec latency 68.566 ms 2024-05-20T21:03:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 -- 172.21.15.26:0/2223935896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c108920 msgr2=0x7f745c1104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 --2- 172.21.15.26:0/2223935896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c108920 0x7f745c1104a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f745400b0d0 tx=0x7f745400b3e0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 -- 172.21.15.26:0/2223935896 shutdown_connections 2024-05-20T21:03:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 --2- 172.21.15.26:0/2223935896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c108920 0x7f745c1104a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 --2- 172.21.15.26:0/2223935896 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 --2- 172.21.15.26:0/2223935896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c104200 0x7f745c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 -- 172.21.15.26:0/2223935896 >> 172.21.15.26:0/2223935896 conn(0x7f745c07a5d0 msgr2=0x7f745c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 -- 172.21.15.26:0/2223935896 shutdown_connections 2024-05-20T21:03:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.585+0000 7f7463932700 1 -- 172.21.15.26:0/2223935896 wait complete. 2024-05-20T21:03:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 Processor -- start 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 -- start start 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c104200 0x7f745c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c108920 0x7f745c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f745c113600 con 0x7f745c104200 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f745c113480 con 0x7f745c103850 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7463932700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f745c113780 con 0x7f745c108920 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f7460ecd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c104200 0x7f745c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.586+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33120/0 (socket says 172.21.15.26:33120) 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 -- 172.21.15.26:0/1922804394 learned_addr learned my addr 172.21.15.26:0/1922804394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f7461ecf700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c108920 0x7f745c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 -- 172.21.15.26:0/1922804394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c108920 msgr2=0x7f745c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c108920 0x7f745c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 -- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c104200 msgr2=0x7f745c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c104200 0x7f745c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 -- 172.21.15.26:0/1922804394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c1a7fb0 con 0x7f745c103850 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f7460ecd700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c104200 0x7f745c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74616ce700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c075840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f745800eb30 tx=0x7f745800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74527fc700 1 -- 172.21.15.26:0/1922804394 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745800cbd0 con 0x7f745c103850 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c1a82a0 con 0x7f745c103850 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74527fc700 1 -- 172.21.15.26:0/1922804394 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74580044b0 con 0x7f745c103850 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f74527fc700 1 -- 172.21.15.26:0/1922804394 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458003dc0 con 0x7f745c103850 2024-05-20T21:03:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.587+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c1a8830 con 0x7f745c103850 2024-05-20T21:03:14.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.590+0000 7f74527fc700 1 -- 172.21.15.26:0/1922804394 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7458003f60 con 0x7f745c103850 2024-05-20T21:03:14.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.591+0000 7f74527fc700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f74480b2880 0x7f74480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.591+0000 7f746e1c8b80 1 Processor -- start 2024-05-20T21:03:14.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 -- start start 2024-05-20T21:03:14.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74527fc700 1 -- 172.21.15.26:0/1922804394 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f74580e2540 con 0x7f745c103850 2024-05-20T21:03:14.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f7460ecd700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f74480b2880 0x7f74480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a187c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a188210 0x55c80a18b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a18bf60 0x55c80a18c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c80a0e6c90 con 0x55c80a18bf60 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c80a0d6830 con 0x55c80a187860 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c80a0da870 con 0x55c80a188210 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a187c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f7461ecf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a18bf60 0x55c80a18c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a187c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33122/0 (socket says 172.21.15.26:33122) 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 -- 172.21.15.26:0/2819749081 learned_addr learned my addr 172.21.15.26:0/2819749081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f7460ecd700 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a188210 0x55c80a18b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 -- 172.21.15.26:0/2819749081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a188210 msgr2=0x55c80a18b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a188210 0x55c80a18b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 -- 172.21.15.26:0/2819749081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a18bf60 msgr2=0x55c80a18c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a18bf60 0x55c80a18c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f74616ce700 1 -- 172.21.15.26:0/2819749081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80a0268d0 con 0x55c80a187860 2024-05-20T21:03:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.592+0000 7f7460ecd700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f74480b2880 0x7f74480b4d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f745c103150 tx=0x7f744c010760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.593+0000 7f74616ce700 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a187c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f745800eb00 tx=0x7f74580095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744192a72904ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:14.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.593+0000 7f74477fe700 1 -- 172.21.15.26:0/2819749081 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580c3070 con 0x55c80a187860 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.593+0000 7f74477fe700 1 -- 172.21.15.26:0/2819749081 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74580b22b0 con 0x55c80a187860 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.593+0000 7f74477fe700 1 -- 172.21.15.26:0/2819749081 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580a9b00 con 0x55c80a187860 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2819749081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 msgr2=0x55c80a187c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a187c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f745800eb00 tx=0x7f74580095f0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2819749081 shutdown_connections 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a18bf60 0x55c80a18c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a187c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2819749081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a188210 0x55c80a18b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2819749081 >> 172.21.15.26:0/2819749081 conn(0x55c80a0b5080 msgr2=0x55c80a181e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2819749081 shutdown_connections 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2819749081 wait complete. 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 -- start start 2024-05-20T21:03:14.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.594+0000 7f746e1c8b80 1 Processor -- start 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 0x55c80a193440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a188210 0x55c80a190740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a190c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c809fdcfa0 con 0x55c80a187860 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c80a0787c0 con 0x55c80a188210 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c80a194120 con 0x55c80a18bf60 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f746e1c8b80 10 client.0 _ll_register_callbacks cb 0x55c80a071920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 0x55c80a193440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 0x55c80a193440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58574/0 (socket says 172.21.15.26:58574) 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 -- 172.21.15.26:0/600606799 learned_addr learned my addr 172.21.15.26:0/600606799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f7460ecd700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a188210 0x55c80a190740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f7461ecf700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a190c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 -- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 msgr2=0x55c80a190c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a190c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 -- 172.21.15.26:0/600606799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a188210 msgr2=0x55c80a190740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a188210 0x55c80a190740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f74616ce700 1 -- 172.21.15.26:0/600606799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80a075690 con 0x55c80a187860 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.595+0000 7f7461ecf700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a190c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f74616ce700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 0x55c80a193440 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7458004480 tx=0x7f74580a5310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d93cf230ff206b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f7444ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f7444ff9700 1 -- 172.21.15.26:0/600606799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745806d070 con 0x55c80a187860 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f7444ff9700 1 -- 172.21.15.26:0/600606799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f74580a9f70 con 0x55c80a187860 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74580b22b0 con 0x55c80a187860 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f7444ff9700 1 -- 172.21.15.26:0/600606799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74580c3070 con 0x55c80a187860 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 10 client.17454 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c80a19e8b0 con 0x55c80a187860 2024-05-20T21:03:14.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f7444ff9700 1 -- 172.21.15.26:0/600606799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74580a9610 con 0x55c80a187860 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f7444ff9700 1 -- 172.21.15.26:0/600606799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f74580b24d0 con 0x55c80a187860 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 10 client.17454 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 10 client.17454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 20 client.17454 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.596+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a1a0d80 0x55c80a1a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.597+0000 7f746e1c8b80 4 client.17454 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.597+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f74580b22b0 con 0x55c80a1a0d80 2024-05-20T21:03:14.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.597+0000 7f7460ecd700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a1a0d80 0x55c80a1a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.597+0000 7f7460ecd700 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a1a0d80 0x55c80a1a3170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c97737163dc2af1 server_cookie=a6bff6556e8dad73 in_seq=0 out_seq=0 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.597+0000 7f7444ff9700 10 client.17454 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f7444ff9700 1 -- 172.21.15.26:0/600606799 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f74580b22b0 con 0x55c80a1a0d80 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f7444ff9700 10 client.17454 handle_command_reply: tid=0 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 client.17454 shutdown 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 msgr2=0x55c80a193440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 0x55c80a193440 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7458004480 tx=0x7f74580a5310 comp rx=0 tx=0).stop 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 shutdown_connections 2024-05-20T21:03:14.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a1a0d80 0x55c80a1a3170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a187860 0x55c80a193440 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a190c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/600606799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a188210 0x55c80a190740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f7444ff9700 0 client.17454 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 >> 172.21.15.26:0/600606799 conn(0x55c80a0b5080 msgr2=0x55c80a181510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 shutdown_connections 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 1 -- 172.21.15.26:0/600606799 wait complete. 2024-05-20T21:03:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.598+0000 7f746e1c8b80 20 client.17454 trim_cache size 0 max 16384 2024-05-20T21:03:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 Processor -- start 2024-05-20T21:03:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 -- start start 2024-05-20T21:03:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a18f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a181c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b2830 con 0x55c80a188210 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b00c0 con 0x55c80a187860 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1adf10 con 0x55c80a18bf60 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f7460ecd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a181c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a18f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a18f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33140/0 (socket says 172.21.15.26:33140) 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f7460ecd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a181c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58584/0 (socket says 172.21.15.26:58584) 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.617+0000 7f74616ce700 1 -- 172.21.15.26:0/2919959111 learned_addr learned my addr 172.21.15.26:0/2919959111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f7461ecf700 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f74616ce700 1 -- 172.21.15.26:0/2919959111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 msgr2=0x55c80a1821d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f74616ce700 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1821d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f74616ce700 1 -- 172.21.15.26:0/2919959111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 msgr2=0x55c80a181c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f74616ce700 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a181c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f74616ce700 1 -- 172.21.15.26:0/2919959111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80a18ecb0 con 0x55c80a187860 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f7460ecd700 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a181c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f74616ce700 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a18f610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74580a6d70 tx=0x7f7458009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9193834890d46d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f7446ffd700 1 -- 172.21.15.26:0/2919959111 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745806d040 con 0x55c80a187860 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f7446ffd700 1 -- 172.21.15.26:0/2919959111 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74580b22b0 con 0x55c80a187860 2024-05-20T21:03:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.618+0000 7f7446ffd700 1 -- 172.21.15.26:0/2919959111 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580b2960 con 0x55c80a187860 2024-05-20T21:03:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2919959111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 msgr2=0x55c80a18f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a18f610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74580a6d70 tx=0x7f7458009880 comp rx=0 tx=0).stop 2024-05-20T21:03:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2919959111 shutdown_connections 2024-05-20T21:03:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a18f610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/2919959111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a181c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2919959111 >> 172.21.15.26:0/2919959111 conn(0x55c80a0b5080 msgr2=0x55c80a18ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2919959111 shutdown_connections 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 -- 172.21.15.26:0/2919959111 wait complete. 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.619+0000 7f746e1c8b80 1 -- start start 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 Processor -- start 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a203c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a2041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1ae580 con 0x55c80a188210 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1af9f0 con 0x55c80a187860 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aec50 con 0x55c80a18bf60 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f746e1c8b80 10 client.0 _ll_register_callbacks cb 0x55c80a071920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:14.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f7460ecd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a2041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a203c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f7460ecd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a2041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58592/0 (socket says 172.21.15.26:58592) 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f74616ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a203c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33154/0 (socket says 172.21.15.26:33154) 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f7460ecd700 1 -- 172.21.15.26:0/3380655821 learned_addr learned my addr 172.21.15.26:0/3380655821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.620+0000 7f7461ecf700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f74616ce700 1 -- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 msgr2=0x55c80a1a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f74616ce700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f74616ce700 1 -- 172.21.15.26:0/3380655821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 msgr2=0x55c80a2041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f74616ce700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a2041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f74616ce700 1 -- 172.21.15.26:0/3380655821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80a1a61d0 con 0x55c80a187860 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f7460ecd700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a2041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:14.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f74616ce700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a203c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f74580a6d70 tx=0x7f74580a9c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc98ae47e55d63a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f7444ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f7444ff9700 1 -- 172.21.15.26:0/3380655821 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745806d040 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f7444ff9700 1 -- 172.21.15.26:0/3380655821 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f745806f4d0 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74580b22b0 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f7444ff9700 1 -- 172.21.15.26:0/3380655821 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74580a9610 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f746e1c8b80 10 client.26921 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.621+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c80a1a61d0 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f7444ff9700 1 -- 172.21.15.26:0/3380655821 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74580b22b0 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f7444ff9700 1 -- 172.21.15.26:0/3380655821 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f74580a5470 con 0x55c80a187860 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f746e1c8b80 10 client.26921 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f746e1c8b80 10 client.26921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f746e1c8b80 20 client.26921 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a20b700 0x55c80a20daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f746e1c8b80 4 client.26921 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c80a1539e0 con 0x55c80a20b700 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.622+0000 7f7460ecd700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a20b700 0x55c80a20daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f7444ff9700 10 client.26921 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f7460ecd700 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a20b700 0x55c80a20daf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92b594875380a7b3 server_cookie=7a35b43ccec3163c in_seq=0 out_seq=0 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f7444ff9700 1 -- 172.21.15.26:0/3380655821 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c80a1539e0 con 0x55c80a20b700 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f7444ff9700 10 client.26921 handle_command_reply: tid=0 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 client.26921 shutdown 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 msgr2=0x55c80a203c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a203c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f74580a6d70 tx=0x7f74580a9c30 comp rx=0 tx=0).stop 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 shutdown_connections 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55c80a20b700 0x55c80a20daf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c80a18bf60 0x55c80a1a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c80a187860 0x55c80a203c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 --2- 172.21.15.26:0/3380655821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c80a188210 0x55c80a2041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 >> 172.21.15.26:0/3380655821 conn(0x55c80a0b5080 msgr2=0x55c80a18f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f7444ff9700 0 client.26921 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 shutdown_connections 2024-05-20T21:03:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.623+0000 7f746e1c8b80 1 -- 172.21.15.26:0/3380655821 wait complete. 2024-05-20T21:03:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f746e1c8b80 20 client.26921 trim_cache size 0 max 16384 2024-05-20T21:03:14.629 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f74480b2880 msgr2=0x7f74480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f74480b2880 0x7f74480b4d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f745c103150 tx=0x7f744c010760 comp rx=0 tx=0).stop 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 msgr2=0x7f745c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c075840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f745800eb30 tx=0x7f745800ee40 comp rx=0 tx=0).stop 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 shutdown_connections 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c108920 0x7f745c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f745c103850 0x7f745c075840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f74480b2880 0x7f74480b4d40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 --2- 172.21.15.26:0/1922804394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c104200 0x7f745c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.624+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 >> 172.21.15.26:0/1922804394 conn(0x7f745c07a5d0 msgr2=0x7f745c1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.625+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 shutdown_connections 2024-05-20T21:03:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:14.625+0000 7f7463932700 1 -- 172.21.15.26:0/1922804394 wait complete. 2024-05-20T21:03:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:15 smithi026 ceph-mon[31990]: pgmap v602: 97 pgs: 97 active+clean; 27 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T21:03:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:15 smithi026 ceph-mon[31990]: osdmap e84: 12 total, 12 up, 12 in 2024-05-20T21:03:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:15 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.144:6836/3257965779,v1:172.21.15.144:6837/3257965779] up:boot 2024-05-20T21:03:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:15 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T21:03:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T21:03:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:15 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:03:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:15 smithi055 ceph-mon[35213]: pgmap v602: 97 pgs: 97 active+clean; 27 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T21:03:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:15 smithi055 ceph-mon[35213]: osdmap e84: 12 total, 12 up, 12 in 2024-05-20T21:03:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:15 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.144:6836/3257965779,v1:172.21.15.144:6837/3257965779] up:boot 2024-05-20T21:03:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:15 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T21:03:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T21:03:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:15 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:03:15.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:03:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:15 smithi144 ceph-mon[35660]: pgmap v602: 97 pgs: 97 active+clean; 27 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T21:03:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:15 smithi144 ceph-mon[35660]: osdmap e84: 12 total, 12 up, 12 in 2024-05-20T21:03:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:15 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.144:6836/3257965779,v1:172.21.15.144:6837/3257965779] up:boot 2024-05-20T21:03:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:15 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T21:03:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T21:03:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:15 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:03:15.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 153519 2.35 MB/sec execute 201 sec latency 26.875 ms 2024-05-20T21:03:15.843 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:16.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 154218 2.36 MB/sec execute 202 sec latency 13.556 ms 2024-05-20T21:03:17.293 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:17 smithi026 ceph-mon[31990]: pgmap v604: 97 pgs: 97 active+clean; 24 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-05-20T21:03:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:17 smithi055 ceph-mon[35213]: pgmap v604: 97 pgs: 97 active+clean; 24 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.536+0000 7f255ced7700 1 -- 172.21.15.26:0/2556458424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 msgr2=0x7f25581077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.536+0000 7f255ced7700 1 --2- 172.21.15.26:0/2556458424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f25581077d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2548009990 tx=0x7f2548009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 -- 172.21.15.26:0/2556458424 shutdown_connections 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 --2- 172.21.15.26:0/2556458424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f25581077d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 --2- 172.21.15.26:0/2556458424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2558102e60 0x7f255810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 --2- 172.21.15.26:0/2556458424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25581024a0 0x7f2558102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 -- 172.21.15.26:0/2556458424 >> 172.21.15.26:0/2556458424 conn(0x7f25580784e0 msgr2=0x7f25580788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 -- 172.21.15.26:0/2556458424 shutdown_connections 2024-05-20T21:03:17.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 -- 172.21.15.26:0/2556458424 wait complete. 2024-05-20T21:03:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.537+0000 7f255ced7700 1 Processor -- start 2024-05-20T21:03:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 -- start start 2024-05-20T21:03:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25581024a0 0x7f255819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 0x7f255819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f2558199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25581124d0 con 0x7f25581073f0 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2558112350 con 0x7f2558102e60 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255ced7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2558112650 con 0x7f25581024a0 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2556d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f2558199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 0x7f255819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2556d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f2558199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58612/0 (socket says 172.21.15.26:58612) 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2556d9d700 1 -- 172.21.15.26:0/3888334906 learned_addr learned my addr 172.21.15.26:0/3888334906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 0x7f255819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33168/0 (socket says 172.21.15.26:33168) 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f255659c700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25581024a0 0x7f255819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 -- 172.21.15.26:0/3888334906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25581024a0 msgr2=0x7f255819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25581024a0 0x7f255819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 -- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 msgr2=0x7f2558199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f2558199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2555d9b700 1 -- 172.21.15.26:0/3888334906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255819a0a0 con 0x7f2558102e60 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.538+0000 7f2556d9d700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f2558199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.539+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 0x7f255819fcc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f254c009c50 tx=0x7f254c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.539+0000 7f25477fe700 1 -- 172.21.15.26:0/3888334906 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c01e070 con 0x7f2558102e60 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.539+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255819a390 con 0x7f2558102e60 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.539+0000 7f25477fe700 1 -- 172.21.15.26:0/3888334906 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f254c019410 con 0x7f2558102e60 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.539+0000 7f25477fe700 1 -- 172.21.15.26:0/3888334906 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c018610 con 0x7f2558102e60 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.539+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2558103fd0 con 0x7f2558102e60 2024-05-20T21:03:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.541+0000 7f25477fe700 1 -- 172.21.15.26:0/3888334906 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f254c021070 con 0x7f2558102e60 2024-05-20T21:03:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.542+0000 7f25477fe700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25400b2880 0x7f25400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.542+0000 7f255659c700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25400b2880 0x7f25400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.542+0000 7f25477fe700 1 -- 172.21.15.26:0/3888334906 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f254c0e57c0 con 0x7f2558102e60 2024-05-20T21:03:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f256776db80 1 Processor -- start 2024-05-20T21:03:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f256776db80 1 -- start start 2024-05-20T21:03:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f255659c700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25400b2880 0x7f25400b4d40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f254800b6f0 tx=0x7f254801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f256776db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f256776db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38d4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f256776db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38d4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f2556d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f2556d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58618/0 (socket says 172.21.15.26:58618) 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38d4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33178/0 (socket says 172.21.15.26:33178) 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.543+0000 7f2556d9d700 1 -- 172.21.15.26:0/244418513 learned_addr learned my addr 172.21.15.26:0/244418513 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555db3832d70 con 0x555db38d5260 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555db3822910 con 0x555db38d48a0 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555db3826950 con 0x555db38da320 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f255659c700 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2555d9b700 1 -- 172.21.15.26:0/244418513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 msgr2=0x555db38da700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2555d9b700 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38da700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2555d9b700 1 -- 172.21.15.26:0/244418513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 msgr2=0x555db38e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2555d9b700 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2555d9b700 1 -- 172.21.15.26:0/244418513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db37cbfb0 con 0x555db38d48a0 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f255659c700 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38da700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2556d9d700 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2555d9b700 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38d4d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f254c0accf0 tx=0x7f254c0ac940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7cd46c78c91457b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.544+0000 7f2544ff9700 1 -- 172.21.15.26:0/244418513 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c0b1110 con 0x555db38d48a0 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.545+0000 7f2544ff9700 1 -- 172.21.15.26:0/244418513 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f254c0b5bc0 con 0x555db38d48a0 2024-05-20T21:03:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.545+0000 7f2544ff9700 1 -- 172.21.15.26:0/244418513 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c0b5990 con 0x555db38d48a0 2024-05-20T21:03:17.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 msgr2=0x555db38d4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38d4d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f254c0accf0 tx=0x7f254c0ac940 comp rx=0 tx=0).stop 2024-05-20T21:03:17.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 shutdown_connections 2024-05-20T21:03:17.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38da700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 --2- 172.21.15.26:0/244418513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38d4d20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 >> 172.21.15.26:0/244418513 conn(0x555db3800240 msgr2=0x555db38cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 shutdown_connections 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 -- 172.21.15.26:0/244418513 wait complete. 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 -- start start 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.546+0000 7f256776db80 1 Processor -- start 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555db3728fa0 con 0x555db38d5260 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555db378ada0 con 0x555db38d48a0 2024-05-20T21:03:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555db38e16e0 con 0x555db38da320 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f256776db80 10 client.0 _ll_register_callbacks cb 0x555db37bd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38e0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33192/0 (socket says 172.21.15.26:33192) 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 -- 172.21.15.26:0/3623495140 learned_addr learned my addr 172.21.15.26:0/3623495140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f2556d9d700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 -- 172.21.15.26:0/3623495140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 msgr2=0x555db38de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 -- 172.21.15.26:0/3623495140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 msgr2=0x555db38de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f255659c700 1 -- 172.21.15.26:0/3623495140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db37728d0 con 0x555db38d48a0 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.547+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f255659c700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38e0d60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c00 tx=0x7f2548009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83dea8f68ed54d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f253dffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f253dffb700 1 -- 172.21.15.26:0/3623495140 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548003e70 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f253dffb700 1 -- 172.21.15.26:0/3623495140 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f254800f460 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f254c0b5bc0 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f253dffb700 1 -- 172.21.15.26:0/3623495140 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f254800f660 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 10 client.26933 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555db37dc760 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f253dffb700 1 -- 172.21.15.26:0/3623495140 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f254c0b5bc0 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f253dffb700 1 -- 172.21.15.26:0/3623495140 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f2548024780 con 0x555db38d48a0 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 10 client.26933 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 10 client.26933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 20 client.26933 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db38ee260 0x555db38f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 4 client.26933 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.548+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db3834cc0 con 0x555db38ee260 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.549+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db38ee260 0x555db38f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.549+0000 7f253dffb700 10 client.26933 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.549+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db38ee260 0x555db38f06a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=981d134fd94becb3 server_cookie=1531e116a699eb65 in_seq=0 out_seq=0 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f253dffb700 1 -- 172.21.15.26:0/3623495140 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555db3834cc0 con 0x555db38ee260 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f253dffb700 10 client.26933 handle_command_reply: tid=0 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 client.26933 shutdown 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 msgr2=0x555db38e0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38e0d60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c00 tx=0x7f2548009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 shutdown_connections 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db38ee260 0x555db38f06a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38da320 0x555db38de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38d48a0 0x555db38e0d60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 --2- 172.21.15.26:0/3623495140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f253dffb700 0 client.26933 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:17.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 >> 172.21.15.26:0/3623495140 conn(0x555db3800240 msgr2=0x555db38ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:17.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 shutdown_connections 2024-05-20T21:03:17.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 1 -- 172.21.15.26:0/3623495140 wait complete. 2024-05-20T21:03:17.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.550+0000 7f256776db80 20 client.26933 trim_cache size 0 max 16384 2024-05-20T21:03:17.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:17 smithi144 ceph-mon[35660]: pgmap v604: 97 pgs: 97 active+clean; 24 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-05-20T21:03:17.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 154902 2.35 MB/sec execute 203 sec latency 18.483 ms 2024-05-20T21:03:17.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.567+0000 7f256776db80 1 Processor -- start 2024-05-20T21:03:17.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.567+0000 7f256776db80 1 -- start start 2024-05-20T21:03:17.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f256776db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 0x555db38ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f256776db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d5260 0x555db38cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f256776db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f256776db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25581ae3a0 con 0x555db38d48a0 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f256776db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25581abc30 con 0x555db38da320 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f256776db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25581a9a80 con 0x555db38d5260 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 0x555db38ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 0x555db38ea740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58636/0 (socket says 172.21.15.26:58636) 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 -- 172.21.15.26:0/259876659 learned_addr learned my addr 172.21.15.26:0/259876659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f2556d9d700 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f2555d9b700 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d5260 0x555db38cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 -- 172.21.15.26:0/259876659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d5260 msgr2=0x555db38cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d5260 0x555db38cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 -- 172.21.15.26:0/259876659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 msgr2=0x555db38dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38dffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 -- 172.21.15.26:0/259876659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db38dc5d0 con 0x555db38d48a0 2024-05-20T21:03:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f255659c700 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 0x555db38ea740 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2548024bc0 tx=0x7f2548005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc94ac15b70c210d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.568+0000 7f253ffff700 1 -- 172.21.15.26:0/259876659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548003e70 con 0x555db38d48a0 2024-05-20T21:03:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f253ffff700 1 -- 172.21.15.26:0/259876659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2548035070 con 0x555db38d48a0 2024-05-20T21:03:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f253ffff700 1 -- 172.21.15.26:0/259876659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254800fe50 con 0x555db38d48a0 2024-05-20T21:03:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 -- 172.21.15.26:0/259876659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 msgr2=0x555db38ea740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 0x555db38ea740 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2548024bc0 tx=0x7f2548005cb0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 -- 172.21.15.26:0/259876659 shutdown_connections 2024-05-20T21:03:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d48a0 0x555db38ea740 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 --2- 172.21.15.26:0/259876659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d5260 0x555db38cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 -- 172.21.15.26:0/259876659 >> 172.21.15.26:0/259876659 conn(0x555db3800240 msgr2=0x555db38df890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.569+0000 7f256776db80 1 -- 172.21.15.26:0/259876659 shutdown_connections 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 -- 172.21.15.26:0/259876659 wait complete. 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 -- start start 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 Processor -- start 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d48a0 0x555db38e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25581aa0f0 con 0x555db38d5260 2024-05-20T21:03:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25581ab560 con 0x555db38da320 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25581aa7c0 con 0x555db38d48a0 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f256776db80 10 client.0 _ll_register_callbacks cb 0x555db37bd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58650/0 (socket says 172.21.15.26:58650) 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.570+0000 7f2555d9b700 1 -- 172.21.15.26:0/3474198768 learned_addr learned my addr 172.21.15.26:0/3474198768 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2556d9d700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f255659c700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d48a0 0x555db38e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2555d9b700 1 -- 172.21.15.26:0/3474198768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d48a0 msgr2=0x555db38e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d48a0 0x555db38e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2555d9b700 1 -- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 msgr2=0x555db38f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2555d9b700 1 -- 172.21.15.26:0/3474198768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db38e4c70 con 0x555db38d5260 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f2555d9b700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38f5950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f254c0a8de0 tx=0x7f254c0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707c5537238d8286 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f253dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f253dffb700 1 -- 172.21.15.26:0/3474198768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c0b5bc0 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f253dffb700 1 -- 172.21.15.26:0/3474198768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f254c0c7310 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2548035070 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f253dffb700 1 -- 172.21.15.26:0/3474198768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f254c0b5730 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f256776db80 10 client.17478 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.571+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555db3959010 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f253dffb700 1 -- 172.21.15.26:0/3474198768 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2548035070 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f253dffb700 1 -- 172.21.15.26:0/3474198768 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f254c0a86d0 con 0x555db38d5260 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f256776db80 10 client.17478 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f256776db80 10 client.17478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f256776db80 20 client.17478 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f256776db80 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db3959330 0x555db395b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f256776db80 4 client.17478 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555db382e4a0 con 0x555db3959330 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f255659c700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db3959330 0x555db395b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f255659c700 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db3959330 0x555db395b720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a1f7ffc8b9a871e server_cookie=91f8133aa38edfae in_seq=0 out_seq=0 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.572+0000 7f253dffb700 10 client.17478 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f253dffb700 1 -- 172.21.15.26:0/3474198768 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555db382e4a0 con 0x555db3959330 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f253dffb700 10 client.17478 handle_command_reply: tid=0 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 client.17478 shutdown 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 msgr2=0x555db38f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38f5950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f254c0a8de0 tx=0x7f254c0108b0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 shutdown_connections 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x555db3959330 0x555db395b720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555db38da320 0x555db38f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555db38d48a0 0x555db38e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f253dffb700 0 client.17478 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 --2- 172.21.15.26:0/3474198768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555db38d5260 0x555db38f5950 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 >> 172.21.15.26:0/3474198768 conn(0x555db3800240 msgr2=0x555db38d7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 shutdown_connections 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 1 -- 172.21.15.26:0/3474198768 wait complete. 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.573+0000 7f256776db80 20 client.17478 trim_cache size 0 max 16384 2024-05-20T21:03:17.578 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25400b2880 msgr2=0x7f25400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25400b2880 0x7f25400b4d40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f254800b6f0 tx=0x7f254801a040 comp rx=0 tx=0).stop 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 msgr2=0x7f255819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 0x7f255819fcc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f254c009c50 tx=0x7f254c009f60 comp rx=0 tx=0).stop 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 shutdown_connections 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f25400b2880 0x7f25400b4d40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25581073f0 0x7f2558199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25581024a0 0x7f255819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 --2- 172.21.15.26:0/3888334906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2558102e60 0x7f255819fcc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 >> 172.21.15.26:0/3888334906 conn(0x7f25580784e0 msgr2=0x7f255806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 shutdown_connections 2024-05-20T21:03:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:17.574+0000 7f255ced7700 1 -- 172.21.15.26:0/3888334906 wait complete. 2024-05-20T21:03:18.283 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10.1 2024-05-20T21:03:18.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:03:18.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:03:18.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 155723 2.37 MB/sec execute 204 sec latency 16.463 ms 2024-05-20T21:03:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:19.426 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:19 smithi026 ceph-mon[31990]: pgmap v605: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T21:03:19.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:19 smithi055 ceph-mon[35213]: pgmap v605: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T21:03:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:19 smithi144 ceph-mon[35660]: pgmap v605: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-05-20T21:03:19.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 156180 2.38 MB/sec execute 205 sec latency 92.960 ms 2024-05-20T21:03:19.905 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:20.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 156648 2.38 MB/sec execute 206 sec latency 11.182 ms 2024-05-20T21:03:21.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:21 smithi144 ceph-mon[35660]: pgmap v606: 97 pgs: 97 active+clean; 25 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-20T21:03:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:21 smithi055 ceph-mon[35213]: pgmap v606: 97 pgs: 97 active+clean; 25 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-20T21:03:21.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:21 smithi026 ceph-mon[31990]: pgmap v606: 97 pgs: 97 active+clean; 25 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-20T21:03:21.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 157270 2.38 MB/sec execute 207 sec latency 18.962 ms 2024-05-20T21:03:21.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 -- 172.21.15.26:0/1914072946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 msgr2=0x7f10641081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 --2- 172.21.15.26:0/1914072946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f10641081c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f10580099c0 tx=0x7f1058009cd0 comp rx=0 tx=0).stop 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 -- 172.21.15.26:0/1914072946 shutdown_connections 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 --2- 172.21.15.26:0/1914072946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10641088f0 0x7f1064110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 --2- 172.21.15.26:0/1914072946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10641037f0 0x7f1064103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 --2- 172.21.15.26:0/1914072946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f10641081c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 -- 172.21.15.26:0/1914072946 >> 172.21.15.26:0/1914072946 conn(0x7f106407a6a0 msgr2=0x7f106407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 -- 172.21.15.26:0/1914072946 shutdown_connections 2024-05-20T21:03:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.847+0000 7f106a345700 1 -- 172.21.15.26:0/1914072946 wait complete. 2024-05-20T21:03:21.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.848+0000 7f106a345700 1 Processor -- start 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.848+0000 7f106a345700 1 -- start start 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f106a345700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10641037f0 0x7f106419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f106a345700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f106419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f106a345700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10641088f0 0x7f10641a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f106a345700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10641135b0 con 0x7f10641041a0 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f106a345700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1064113430 con 0x7f10641037f0 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f106a345700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1064113730 con 0x7f10641088f0 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f106419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f106419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48390/0 (socket says 172.21.15.26:48390) 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 -- 172.21.15.26:0/1097521384 learned_addr learned my addr 172.21.15.26:0/1097521384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f1063fff700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10641037f0 0x7f106419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10688e2700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10641088f0 0x7f10641a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 -- 172.21.15.26:0/1097521384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10641088f0 msgr2=0x7f10641a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10641088f0 0x7f10641a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 -- 172.21.15.26:0/1097521384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10641037f0 msgr2=0x7f106419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10641037f0 0x7f106419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.849+0000 7f10637fe700 1 -- 172.21.15.26:0/1097521384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10641a0d30 con 0x7f10641041a0 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.850+0000 7f10637fe700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f106419c260 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1058005910 tx=0x7f10580045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.850+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058026070 con 0x7f10641041a0 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.850+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f105800f460 con 0x7f10641041a0 2024-05-20T21:03:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.850+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10641a0fc0 con 0x7f10641041a0 2024-05-20T21:03:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.850+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058009e60 con 0x7f10641041a0 2024-05-20T21:03:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.850+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641a88a0 con 0x7f10641041a0 2024-05-20T21:03:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.851+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106406a230 con 0x7f10641041a0 2024-05-20T21:03:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.853+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1058029070 con 0x7f10641041a0 2024-05-20T21:03:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.854+0000 7f10617fa700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f104c0b2880 0x7f104c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.855+0000 7f1063fff700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f104c0b2880 0x7f104c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.855+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f10580e2d50 con 0x7f10641041a0 2024-05-20T21:03:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.855+0000 7f1063fff700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f104c0b2880 0x7f104c0b4d40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1054009680 tx=0x7f1054009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:21.858+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1058030080 con 0x7f10641041a0 2024-05-20T21:03:22.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.168+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10641a8a70 con 0x7f10641041a0 2024-05-20T21:03:22.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.169+0000 7f10617fa700 1 -- 172.21.15.26:0/1097521384 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f1058013070 con 0x7f10641041a0 2024-05-20T21:03:22.169 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:03:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:03:22.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f104c0b2880 msgr2=0x7f104c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f104c0b2880 0x7f104c0b4d40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1054009680 tx=0x7f1054009260 comp rx=0 tx=0).stop 2024-05-20T21:03:22.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 msgr2=0x7f106419c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f106419c260 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1058005910 tx=0x7f10580045e0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 shutdown_connections 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10641088f0 0x7f10641a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10641037f0 0x7f106419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f104c0b2880 0x7f104c0b4d40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 --2- 172.21.15.26:0/1097521384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10641041a0 0x7f106419c260 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 >> 172.21.15.26:0/1097521384 conn(0x7f106407a6a0 msgr2=0x7f1064101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 shutdown_connections 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.172+0000 7f106a345700 1 -- 172.21.15.26:0/1097521384 wait complete. 2024-05-20T21:03:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 -- 172.21.15.26:0/3851219454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f24103f90 msgr2=0x7f5f24110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/3851219454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f24103f90 0x7f5f24110470 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00b0a0 tx=0x7f5f1c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 -- 172.21.15.26:0/3851219454 shutdown_connections 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/3851219454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f24103f90 0x7f5f24110470 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/3851219454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f24108500 0x7f5f241088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/3851219454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f241035f0 0x7f5f24103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 -- 172.21.15.26:0/3851219454 >> 172.21.15.26:0/3851219454 conn(0x7f5f2407a6a0 msgr2=0x7f5f2407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 -- 172.21.15.26:0/3851219454 shutdown_connections 2024-05-20T21:03:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 -- 172.21.15.26:0/3851219454 wait complete. 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 Processor -- start 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.381+0000 7f5f2a45b700 1 -- start start 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f2a45b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 0x7f5f24075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f2a45b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f24103f90 0x7f5f24075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f2a45b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f24108500 0x7f5f240762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f2a45b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24113650 con 0x7f5f241035f0 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f2a45b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241134d0 con 0x7f5f24108500 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f2a45b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241137d0 con 0x7f5f24103f90 2024-05-20T21:03:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 0x7f5f24075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 0x7f5f24075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48406/0 (socket says 172.21.15.26:48406) 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 -- 172.21.15.26:0/2349803875 learned_addr learned my addr 172.21.15.26:0/2349803875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f24103f90 0x7f5f24075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f289f8700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f24108500 0x7f5f240762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 -- 172.21.15.26:0/2349803875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f24103f90 msgr2=0x7f5f24075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f24103f90 0x7f5f24075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 -- 172.21.15.26:0/2349803875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f24108500 msgr2=0x7f5f240762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f24108500 0x7f5f240762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 -- 172.21.15.26:0/2349803875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f241a8050 con 0x7f5f241035f0 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.382+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 0x7f5f24075840 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400e970 tx=0x7f5f1400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.383+0000 7f5f21ffb700 1 -- 172.21.15.26:0/2349803875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14004220 con 0x7f5f241035f0 2024-05-20T21:03:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.383+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f241a82e0 con 0x7f5f241035f0 2024-05-20T21:03:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.383+0000 7f5f21ffb700 1 -- 172.21.15.26:0/2349803875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f140043c0 con 0x7f5f241035f0 2024-05-20T21:03:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.383+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241a8870 con 0x7f5f241035f0 2024-05-20T21:03:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.383+0000 7f5f21ffb700 1 -- 172.21.15.26:0/2349803875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14003c10 con 0x7f5f241035f0 2024-05-20T21:03:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.386+0000 7f5f21ffb700 1 -- 172.21.15.26:0/2349803875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5f1400c7e0 con 0x7f5f241035f0 2024-05-20T21:03:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.387+0000 7f5f21ffb700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5f100b2980 0x7f5f100b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5f100b2980 0x7f5f100b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f21ffb700 1 -- 172.21.15.26:0/2349803875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5f140e23d0 con 0x7f5f241035f0 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5f100b2980 0x7f5f100b4e40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f2407ae00 tx=0x7f5f0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f34cf1b80 1 Processor -- start 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f34cf1b80 1 -- start start 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c15860 0x5565b4c15c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.388+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c19940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c19f60 0x5565b4c1a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5565b4b74c90 con 0x5565b4c16210 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5565b4b64830 con 0x5565b4c19f60 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565b4b68870 con 0x5565b4c15860 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c19940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c19940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48412/0 (socket says 172.21.15.26:48412) 2024-05-20T21:03:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f1bfff700 1 -- 172.21.15.26:0/2527592427 learned_addr learned my addr 172.21.15.26:0/2527592427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c15860 0x5565b4c15c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f289f8700 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c19f60 0x5565b4c1a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 -- 172.21.15.26:0/2527592427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c19f60 msgr2=0x5565b4c1a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c19f60 0x5565b4c1a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 -- 172.21.15.26:0/2527592427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 msgr2=0x5565b4c19940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c19940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 -- 172.21.15.26:0/2527592427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b4ab48d0 con 0x5565b4c15860 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c19940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.389+0000 7f5f23fff700 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c15860 0x5565b4c15c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f140a59b0 tx=0x7f5f140a9120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3981e48b177fef77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/2527592427 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f140a9440 con 0x5565b4c15860 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/2527592427 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f140aa030 con 0x5565b4c15860 2024-05-20T21:03:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/2527592427 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400cd00 con 0x5565b4c15860 2024-05-20T21:03:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/2527592427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c15860 msgr2=0x5565b4c15c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c15860 0x5565b4c15c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f140a59b0 tx=0x7f5f140a9120 comp rx=0 tx=0).stop 2024-05-20T21:03:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/2527592427 shutdown_connections 2024-05-20T21:03:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c19f60 0x5565b4c1a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c15860 0x5565b4c15c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/2527592427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c19940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.390+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/2527592427 >> 172.21.15.26:0/2527592427 conn(0x5565b4b43080 msgr2=0x5565b4c0fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.391+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/2527592427 shutdown_connections 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.391+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/2527592427 wait complete. 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.391+0000 7f5f34cf1b80 1 -- start start 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.391+0000 7f5f34cf1b80 1 Processor -- start 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.391+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.391+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c1ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5565b4a6afa0 con 0x5565b4c15860 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5565b4b067c0 con 0x5565b4c16210 2024-05-20T21:03:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565b4c22120 con 0x5565b4c19f60 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f34cf1b80 10 client.0 _ll_register_callbacks cb 0x5565b4aff920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c1e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39502/0 (socket says 172.21.15.26:39502) 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 -- 172.21.15.26:0/4110809917 learned_addr learned my addr 172.21.15.26:0/4110809917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f289f8700 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c1ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 -- 172.21.15.26:0/4110809917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 msgr2=0x5565b4c1ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c1ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 -- 172.21.15.26:0/4110809917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 msgr2=0x5565b4c21360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c21360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 -- 172.21.15.26:0/4110809917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b4b03690 con 0x5565b4c16210 2024-05-20T21:03:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c1e6b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004600 tx=0x7f5f0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e35e26eae7770b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.392+0000 7f5f037fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f037fe700 1 -- 172.21.15.26:0/4110809917 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c027420 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f037fe700 1 -- 172.21.15.26:0/4110809917 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5f0c01d410 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f140aa030 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f037fe700 1 -- 172.21.15.26:0/4110809917 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f0c01d610 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 10 client.26957 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565b4c2c870 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f037fe700 1 -- 172.21.15.26:0/4110809917 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f140aa030 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f037fe700 1 -- 172.21.15.26:0/4110809917 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5f0c02f020 con 0x5565b4c16210 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 10 client.26957 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 20 client.26957 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c2ee00 0x5565b4c31280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 4 client.26957 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.393+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565b4be19e0 con 0x5565b4c2ee00 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.394+0000 7f5f23fff700 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c2ee00 0x5565b4c31280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.394+0000 7f5f23fff700 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c2ee00 0x5565b4c31280 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f943bc1fb91935 server_cookie=df868d08f713d55f in_seq=0 out_seq=0 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.394+0000 7f5f037fe700 10 client.26957 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f037fe700 1 -- 172.21.15.26:0/4110809917 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5565b4be19e0 con 0x5565b4c2ee00 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f037fe700 10 client.26957 handle_command_reply: tid=0 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 client.26957 shutdown 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 msgr2=0x5565b4c1e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c1e6b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004600 tx=0x7f5f0c0047c0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 shutdown_connections 2024-05-20T21:03:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c2ee00 0x5565b4c31280 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c21360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f037fe700 0 client.26957 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c1ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/4110809917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c1e6b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 >> 172.21.15.26:0/4110809917 conn(0x5565b4b43080 msgr2=0x5565b4c0f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 shutdown_connections 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/4110809917 wait complete. 2024-05-20T21:03:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.395+0000 7f5f34cf1b80 20 client.26957 trim_cache size 0 max 16384 2024-05-20T21:03:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.411+0000 7f5f34cf1b80 1 Processor -- start 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 -- start start 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c36b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b2750 con 0x5565b4c15860 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241affe0 con 0x5565b4c16210 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241ade30 con 0x5565b4c19f60 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c36b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c0fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48438/0 (socket says 172.21.15.26:48438) 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c36b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39514/0 (socket says 172.21.15.26:39514) 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 -- 172.21.15.26:0/1134055075 learned_addr learned my addr 172.21.15.26:0/1134055075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f289f8700 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 -- 172.21.15.26:0/1134055075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 msgr2=0x5565b4c37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 -- 172.21.15.26:0/1134055075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 msgr2=0x5565b4c36b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c36b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.412+0000 7f5f23fff700 1 -- 172.21.15.26:0/1134055075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b4c1e480 con 0x5565b4c15860 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.413+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c0fc90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5f14091010 tx=0x7f5f140cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33607d4339fe7803 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.413+0000 7f5f19ffb700 1 -- 172.21.15.26:0/1134055075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f140a9930 con 0x5565b4c15860 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.413+0000 7f5f19ffb700 1 -- 172.21.15.26:0/1134055075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f140c2040 con 0x5565b4c15860 2024-05-20T21:03:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.413+0000 7f5f19ffb700 1 -- 172.21.15.26:0/1134055075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400cdd0 con 0x5565b4c15860 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.413+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1134055075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 msgr2=0x5565b4c0fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.413+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c0fc90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5f14091010 tx=0x7f5f140cc040 comp rx=0 tx=0).stop 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1134055075 shutdown_connections 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c15860 0x5565b4c0fc90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1134055075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c16210 0x5565b4c36b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1134055075 >> 172.21.15.26:0/1134055075 conn(0x5565b4b43080 msgr2=0x5565b4c1cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1134055075 shutdown_connections 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1134055075 wait complete. 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 -- start start 2024-05-20T21:03:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.414+0000 7f5f34cf1b80 1 Processor -- start 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c15860 0x5565b4c27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c24cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c25200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241ae4a0 con 0x5565b4c16210 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241af910 con 0x5565b4c15860 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241aeb70 con 0x5565b4c19f60 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f34cf1b80 10 client.0 _ll_register_callbacks cb 0x5565b4aff920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c24cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c24cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48452/0 (socket says 172.21.15.26:48452) 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 -- 172.21.15.26:0/1399364147 learned_addr learned my addr 172.21.15.26:0/1399364147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c15860 0x5565b4c27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f289f8700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c25200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 -- 172.21.15.26:0/1399364147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 msgr2=0x5565b4c25200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c25200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 -- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c15860 msgr2=0x5565b4c27920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c15860 0x5565b4c27920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 -- 172.21.15.26:0/1399364147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b4c25880 con 0x5565b4c16210 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f289f8700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c25200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c15860 0x5565b4c27920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f1bfff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c24cc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009b20 tx=0x7f5f0c027e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c530b026ea53dfe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.415+0000 7f5f037fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f037fe700 1 -- 172.21.15.26:0/1399364147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c01d410 con 0x5565b4c16210 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f037fe700 1 -- 172.21.15.26:0/1399364147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5f0c027630 con 0x5565b4c16210 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f140c2040 con 0x5565b4c16210 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f037fe700 1 -- 172.21.15.26:0/1399364147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f0c042050 con 0x5565b4c16210 2024-05-20T21:03:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 10 client.17502 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565b4c998f0 con 0x5565b4c16210 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f037fe700 1 -- 172.21.15.26:0/1399364147 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f140c2040 con 0x5565b4c16210 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f037fe700 1 -- 172.21.15.26:0/1399364147 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5f0c02f070 con 0x5565b4c16210 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 10 client.17502 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 20 client.17502 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c99fd0 0x5565b4c9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 4 client.17502 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565b4b76be0 con 0x5565b4c99fd0 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.416+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c99fd0 0x5565b4c9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.417+0000 7f5f23fff700 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c99fd0 0x5565b4c9c3f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e90e3f41a9aa8b server_cookie=5dbaa2dc194b4ffb in_seq=0 out_seq=0 2024-05-20T21:03:22.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.417+0000 7f5f037fe700 10 client.17502 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f037fe700 1 -- 172.21.15.26:0/1399364147 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565b4b76be0 con 0x5565b4c99fd0 2024-05-20T21:03:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f037fe700 10 client.17502 handle_command_reply: tid=0 2024-05-20T21:03:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 client.17502 shutdown 2024-05-20T21:03:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 msgr2=0x5565b4c24cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c24cc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009b20 tx=0x7f5f0c027e00 comp rx=0 tx=0).stop 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 shutdown_connections 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5565b4c99fd0 0x5565b4c9c3f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565b4c19f60 0x5565b4c25200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565b4c15860 0x5565b4c27920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 --2- 172.21.15.26:0/1399364147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565b4c16210 0x5565b4c24cc0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 >> 172.21.15.26:0/1399364147 conn(0x5565b4b43080 msgr2=0x5565b4c188b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f037fe700 0 client.17502 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 shutdown_connections 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 1 -- 172.21.15.26:0/1399364147 wait complete. 2024-05-20T21:03:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.418+0000 7f5f34cf1b80 20 client.17502 trim_cache size 0 max 16384 2024-05-20T21:03:22.423 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:03:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5f100b2980 msgr2=0x7f5f100b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5f100b2980 0x7f5f100b4e40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f2407ae00 tx=0x7f5f0c009260 comp rx=0 tx=0).stop 2024-05-20T21:03:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 msgr2=0x7f5f24075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 0x7f5f24075840 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400e970 tx=0x7f5f1400ec80 comp rx=0 tx=0).stop 2024-05-20T21:03:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 shutdown_connections 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5f100b2980 0x7f5f100b4e40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f241035f0 0x7f5f24075840 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f24108500 0x7f5f240762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 --2- 172.21.15.26:0/2349803875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f24103f90 0x7f5f24075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 >> 172.21.15.26:0/2349803875 conn(0x7f5f2407a6a0 msgr2=0x7f5f241017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 shutdown_connections 2024-05-20T21:03:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:22.419+0000 7f5f2a45b700 1 -- 172.21.15.26:0/2349803875 wait complete. 2024-05-20T21:03:22.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 157920 2.38 MB/sec execute 208 sec latency 26.583 ms 2024-05-20T21:03:23.163 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 130.1 --await-for 45.0' 2024-05-20T21:03:23.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 130.1 --await-for 45.0 2024-05-20T21:03:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:23 smithi055 ceph-mon[35213]: pgmap v607: 97 pgs: 97 active+clean; 25 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-05-20T21:03:23.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:23 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1097521384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:23.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:23 smithi026 ceph-mon[31990]: pgmap v607: 97 pgs: 97 active+clean; 25 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-05-20T21:03:23.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:23 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1097521384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:23 smithi144 ceph-mon[35660]: pgmap v607: 97 pgs: 97 active+clean; 25 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-05-20T21:03:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:23 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1097521384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:23.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 158565 2.38 MB/sec execute 209 sec latency 17.571 ms 2024-05-20T21:03:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:24.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 159404 2.40 MB/sec execute 210 sec latency 12.294 ms 2024-05-20T21:03:24.820 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:03:24.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:03:25.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:25 smithi026 ceph-mon[31990]: pgmap v608: 97 pgs: 97 active+clean; 26 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T21:03:25.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:25 smithi055 ceph-mon[35213]: pgmap v608: 97 pgs: 97 active+clean; 26 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T21:03:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:25 smithi144 ceph-mon[35660]: pgmap v608: 97 pgs: 97 active+clean; 26 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T21:03:25.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 159951 2.41 MB/sec execute 211 sec latency 14.155 ms 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 -- 172.21.15.26:0/241625777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 msgr2=0x7ffa0810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 --2- 172.21.15.26:0/241625777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa0810ac20 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009ca0 tx=0x7ffa0002a8c0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 -- 172.21.15.26:0/241625777 shutdown_connections 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 --2- 172.21.15.26:0/241625777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa0810ac20 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 --2- 172.21.15.26:0/241625777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 0x7ffa081082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 --2- 172.21.15.26:0/241625777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 0x7ffa081059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 -- 172.21.15.26:0/241625777 >> 172.21.15.26:0/241625777 conn(0x7ffa080fcf20 msgr2=0x7ffa080ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 -- 172.21.15.26:0/241625777 shutdown_connections 2024-05-20T21:03:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.632+0000 7ffa0e914640 1 -- 172.21.15.26:0/241625777 wait complete. 2024-05-20T21:03:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 Processor -- start 2024-05-20T21:03:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 -- start start 2024-05-20T21:03:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 0x7ffa08198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 0x7ffa081990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa081a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810e4e0 con 0x7ffa08108830 2024-05-20T21:03:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810e360 con 0x7ffa08105f00 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0e914640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810e660 con 0x7ffa081035d0 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0ceb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa081a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa077fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 0x7ffa081990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0ceb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa081a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48470/0 (socket says 172.21.15.26:48470) 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.633+0000 7ffa0ceb1640 1 -- 172.21.15.26:0/1126692388 learned_addr learned my addr 172.21.15.26:0/1126692388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 0x7ffa08198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa077fe640 1 -- 172.21.15.26:0/1126692388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 msgr2=0x7ffa08198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa077fe640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 0x7ffa08198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa077fe640 1 -- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 msgr2=0x7ffa081a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa077fe640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa081a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa077fe640 1 -- 172.21.15.26:0/1126692388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa081a07a0 con 0x7ffa08105f00 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 0x7ffa08198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa0ceb1640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa081a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa077fe640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 0x7ffa081990c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9c0 tx=0x7ff9fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa057fa640 1 -- 172.21.15.26:0/1126692388 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00ccc0 con 0x7ffa08105f00 2024-05-20T21:03:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa057fa640 1 -- 172.21.15.26:0/1126692388 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9fc0108c0 con 0x7ffa08105f00 2024-05-20T21:03:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa081a0a90 con 0x7ffa08105f00 2024-05-20T21:03:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa057fa640 1 -- 172.21.15.26:0/1126692388 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc018750 con 0x7ffa08105f00 2024-05-20T21:03:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.634+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081a0fe0 con 0x7ffa08105f00 2024-05-20T21:03:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa057fa640 1 -- 172.21.15.26:0/1126692388 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff9fc010430 con 0x7ffa08105f00 2024-05-20T21:03:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 Processor -- start 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 -- start start 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29b2220 0x55b9d29b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b9d291dc30 con 0x55b9d29b2220 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b9d290d7d0 con 0x55b9d29ad0b0 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.637+0000 7ffa1162c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b9d2911810 con 0x55b9d29ac6c0 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39562/0 (socket says 172.21.15.26:39562) 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29b2220 0x55b9d29b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 -- 172.21.15.26:0/1873893181 learned_addr learned my addr 172.21.15.26:0/1873893181 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa057fa640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9f00b28a0 0x7ff9f00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa0ceb1640 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 -- 172.21.15.26:0/1873893181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 msgr2=0x55b9d29acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 -- 172.21.15.26:0/1873893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29b2220 msgr2=0x55b9d29b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29b2220 0x55b9d29b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 -- 172.21.15.26:0/1873893181 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d2622210 con 0x55b9d29ad0b0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa0ceb1640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9f00b28a0 0x7ff9f00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa057fa640 1 -- 172.21.15.26:0/1126692388 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff9fc0e25c0 con 0x7ffa08105f00 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b1bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800cca0 tx=0x7ff9f800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1a63412c6ca8de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ff9ee7fc640 1 -- 172.21.15.26:0/1873893181 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8005220 con 0x55b9d29ad0b0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ff9ee7fc640 1 -- 172.21.15.26:0/1873893181 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9f8004c30 con 0x55b9d29ad0b0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.638+0000 7ff9ee7fc640 1 -- 172.21.15.26:0/1873893181 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8010750 con 0x55b9d29ad0b0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 -- 172.21.15.26:0/1873893181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 msgr2=0x55b9d29b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa0ceb1640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9f00b28a0 0x7ff9f00b4d60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffa00031840 tx=0x7ffa000095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b1bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800cca0 tx=0x7ff9f800e720 comp rx=0 tx=0).stop 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 -- 172.21.15.26:0/1873893181 shutdown_connections 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29b2220 0x55b9d29b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b1bb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1873893181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 -- 172.21.15.26:0/1873893181 >> 172.21.15.26:0/1873893181 conn(0x55b9d29a7d30 msgr2=0x55b9d29a8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 -- 172.21.15.26:0/1873893181 shutdown_connections 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 -- 172.21.15.26:0/1873893181 wait complete. 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.639+0000 7ffa1162c740 1 -- start start 2024-05-20T21:03:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 Processor -- start 2024-05-20T21:03:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b9d25f3790 con 0x55b9d29ad0b0 2024-05-20T21:03:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b9d28c9890 con 0x55b9d29b2220 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b9d29b51d0 con 0x55b9d29ac6c0 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa1162c740 10 client.0 _ll_register_callbacks cb 0x55b9d2621a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48488/0 (socket says 172.21.15.26:48488) 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa077fe640 1 -- 172.21.15.26:0/3171666224 learned_addr learned my addr 172.21.15.26:0/3171666224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.640+0000 7ffa0ceb1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa077fe640 1 -- 172.21.15.26:0/3171666224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 msgr2=0x55b9d29b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa077fe640 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa077fe640 1 -- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 msgr2=0x55b9d29bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa077fe640 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa077fe640 1 -- 172.21.15.26:0/3171666224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d26ee620 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa077fe640 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b6d30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0a5c00 tx=0x7ff9fc0e28c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317a5ab9e9cc1dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ff9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ff9e77fe640 1 -- 172.21.15.26:0/3171666224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc010720 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ff9e77fe640 1 -- 172.21.15.26:0/3171666224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff9fc0a8de0 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f8004c30 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ff9e77fe640 1 -- 172.21.15.26:0/3171666224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc0a5310 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa1162c740 10 client.17514 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9d29c0dc0 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.641+0000 7ff9e77fe640 1 -- 172.21.15.26:0/3171666224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9f8004c30 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ff9e77fe640 1 -- 172.21.15.26:0/3171666224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff9fc0e20d0 con 0x55b9d29ad0b0 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa1162c740 10 client.17514 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa1162c740 10 client.17514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa1162c740 20 client.17514 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa1162c740 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d29c24b0 0x55b9d29c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa1162c740 4 client.17514 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d291fb80 con 0x55b9d29c24b0 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa07fff640 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d29c24b0 0x55b9d29c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ffa07fff640 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d29c24b0 0x55b9d29c48a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f57c053f6f23f8 server_cookie=bc818622c5177dab in_seq=0 out_seq=0 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.642+0000 7ff9e77fe640 10 client.17514 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ff9e77fe640 1 -- 172.21.15.26:0/3171666224 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b9d291fb80 con 0x55b9d29c24b0 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ff9e77fe640 10 client.17514 handle_command_reply: tid=0 2024-05-20T21:03:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 client.17514 shutdown 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 msgr2=0x55b9d29b6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b6d30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0a5c00 tx=0x7ff9fc0e28c0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 shutdown_connections 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d29c24b0 0x55b9d29c48a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ad0b0 0x55b9d29b6d30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 --2- 172.21.15.26:0/3171666224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ac6c0 0x55b9d29b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 >> 172.21.15.26:0/3171666224 conn(0x55b9d29a7d30 msgr2=0x55b9d29a6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ff9e77fe640 0 client.17514 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 shutdown_connections 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 1 -- 172.21.15.26:0/3171666224 wait complete. 2024-05-20T21:03:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.643+0000 7ffa1162c740 20 client.17514 trim_cache size 0 max 16384 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 Processor -- start 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 -- start start 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d29ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ad0b0 0x55b9d29ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081ac840 con 0x55b9d29ac6c0 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081aa110 con 0x55b9d29b2220 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa1162c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081a7f60 con 0x55b9d29ad0b0 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d29ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d29ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48504/0 (socket says 172.21.15.26:48504) 2024-05-20T21:03:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 -- 172.21.15.26:0/1651341669 learned_addr learned my addr 172.21.15.26:0/1651341669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa0ceb1640 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 -- 172.21.15.26:0/1651341669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ad0b0 msgr2=0x55b9d29ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa077fe640 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ad0b0 0x55b9d29ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ad0b0 0x55b9d29ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 -- 172.21.15.26:0/1651341669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 msgr2=0x55b9d29aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.660+0000 7ffa07fff640 1 -- 172.21.15.26:0/1651341669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d29b15c0 con 0x55b9d29ac6c0 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa07fff640 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d29ca190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800ec20 tx=0x7ff9f800c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38667441bb1c1e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ff9edffb640 1 -- 172.21.15.26:0/1651341669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f802d070 con 0x55b9d29ac6c0 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ff9edffb640 1 -- 172.21.15.26:0/1651341669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9f8026b10 con 0x55b9d29ac6c0 2024-05-20T21:03:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ff9edffb640 1 -- 172.21.15.26:0/1651341669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8010720 con 0x55b9d29ac6c0 2024-05-20T21:03:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 -- 172.21.15.26:0/1651341669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 msgr2=0x55b9d29ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d29ca190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800ec20 tx=0x7ff9f800c590 comp rx=0 tx=0).stop 2024-05-20T21:03:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 -- 172.21.15.26:0/1651341669 shutdown_connections 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29b2220 0x55b9d29aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29ad0b0 0x55b9d29ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 --2- 172.21.15.26:0/1651341669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d29ca190 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.661+0000 7ffa1162c740 1 -- 172.21.15.26:0/1651341669 >> 172.21.15.26:0/1651341669 conn(0x55b9d29a7d30 msgr2=0x55b9d29ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 -- 172.21.15.26:0/1651341669 shutdown_connections 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 -- 172.21.15.26:0/1651341669 wait complete. 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 -- start start 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 Processor -- start 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d2a280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d2a285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29b2220 0x55b9d2a2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081a85d0 con 0x55b9d29ac6c0 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081a9a40 con 0x55b9d29ad0b0 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081a8ca0 con 0x55b9d29b2220 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa1162c740 10 client.0 _ll_register_callbacks cb 0x7ff9fc0e5d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d2a280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d2a280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48506/0 (socket says 172.21.15.26:48506) 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.662+0000 7ffa07fff640 1 -- 172.21.15.26:0/2797689500 learned_addr learned my addr 172.21.15.26:0/2797689500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa077fe640 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d2a285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa07fff640 1 -- 172.21.15.26:0/2797689500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29b2220 msgr2=0x55b9d2a2f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa07fff640 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29b2220 0x55b9d2a2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa07fff640 1 -- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 msgr2=0x55b9d2a285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa07fff640 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d2a285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa07fff640 1 -- 172.21.15.26:0/2797689500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d2a2fc20 con 0x55b9d29ac6c0 2024-05-20T21:03:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa07fff640 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d2a280b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8026530 tx=0x7ff9f80046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f67748cc4992ce server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ff9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ff9e77fe640 1 -- 172.21.15.26:0/2797689500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8035020 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ff9e77fe640 1 -- 172.21.15.26:0/2797689500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff9f80106a0 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f8026b10 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ff9e77fe640 1 -- 172.21.15.26:0/2797689500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9f802d040 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa1162c740 10 client.17520 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.663+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9d2a30f30 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ff9e77fe640 1 -- 172.21.15.26:0/2797689500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9f8026b10 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ff9e77fe640 1 -- 172.21.15.26:0/2797689500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff9f8039020 con 0x55b9d29ac6c0 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa1162c740 10 client.17520 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa1162c740 10 client.17520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa1162c740 20 client.17520 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa1162c740 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d2a31780 0x55b9d2a33ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa1162c740 4 client.17520 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:03:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9d2919360 con 0x55b9d2a31780 2024-05-20T21:03:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa077fe640 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d2a31780 0x55b9d2a33ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ffa077fe640 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d2a31780 0x55b9d2a33ba0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bc50494f86e5d9 server_cookie=3e9107e7ccf8c48a in_seq=0 out_seq=0 2024-05-20T21:03:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.664+0000 7ff9e77fe640 10 client.17520 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ff9e77fe640 1 -- 172.21.15.26:0/2797689500 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b9d2919360 con 0x55b9d2a31780 2024-05-20T21:03:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ff9e77fe640 10 client.17520 handle_command_reply: tid=0 2024-05-20T21:03:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 client.17520 shutdown 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 msgr2=0x55b9d2a280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d2a280b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8026530 tx=0x7ff9f80046e0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 shutdown_connections 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b9d2a31780 0x55b9d2a33ba0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9d29b2220 0x55b9d2a2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ff9e77fe640 0 client.17520 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b9d29ad0b0 0x55b9d2a285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 --2- 172.21.15.26:0/2797689500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9d29ac6c0 0x55b9d2a280b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 >> 172.21.15.26:0/2797689500 conn(0x55b9d29a7d30 msgr2=0x55b9d29aace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 shutdown_connections 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 1 -- 172.21.15.26:0/2797689500 wait complete. 2024-05-20T21:03:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.706+0000 7ffa1162c740 20 client.17520 trim_cache size 0 max 16384 2024-05-20T21:03:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:03:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:03:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:03:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9f00b28a0 msgr2=0x7ff9f00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9f00b28a0 0x7ff9f00b4d60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffa00031840 tx=0x7ffa000095c0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 msgr2=0x7ffa081990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 0x7ffa081990c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9c0 tx=0x7ff9fc00ee80 comp rx=0 tx=0).stop 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 shutdown_connections 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9f00b28a0 0x7ff9f00b4d60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa08108830 0x7ffa081a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa08105f00 0x7ffa081990c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 --2- 172.21.15.26:0/1126692388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa081035d0 0x7ffa08198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 >> 172.21.15.26:0/1126692388 conn(0x7ffa080fcf20 msgr2=0x7ffa08103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 shutdown_connections 2024-05-20T21:03:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:25.707+0000 7ffa0e914640 1 -- 172.21.15.26:0/1126692388 wait complete. 2024-05-20T21:03:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 -- 172.21.15.26:0/3569169986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 msgr2=0x7f19801099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 --2- 172.21.15.26:0/3569169986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 0x7f19801099b0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f197c009990 tx=0x7f197c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 -- 172.21.15.26:0/3569169986 shutdown_connections 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 --2- 172.21.15.26:0/3569169986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 0x7f19801099b0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 --2- 172.21.15.26:0/3569169986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006c4d0 0x7f19801103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 --2- 172.21.15.26:0/3569169986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006bb10 0x7f198006bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 -- 172.21.15.26:0/3569169986 >> 172.21.15.26:0/3569169986 conn(0x7f1980079480 msgr2=0x7f1980079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 -- 172.21.15.26:0/3569169986 shutdown_connections 2024-05-20T21:03:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.993+0000 7f1987b29700 1 -- 172.21.15.26:0/3569169986 wait complete. 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 Processor -- start 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 -- start start 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 0x7f1980102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 0x7f1980102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 0x7f1980102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19801135b0 con 0x7f19801095d0 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113430 con 0x7f198006c4d0 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f1987b29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113730 con 0x7f198006bb10 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f19850c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 0x7f1980102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f19858c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 0x7f1980102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f19860c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 0x7f1980102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f19850c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 0x7f1980102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39602/0 (socket says 172.21.15.26:39602) 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f19850c4700 1 -- 172.21.15.26:0/2471081685 learned_addr learned my addr 172.21.15.26:0/2471081685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.994+0000 7f19858c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 0x7f1980102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:34516/0 (socket says 172.21.15.26:34516) 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19850c4700 1 -- 172.21.15.26:0/2471081685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 msgr2=0x7f1980102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19850c4700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 0x7f1980102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19850c4700 1 -- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 msgr2=0x7f1980102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19850c4700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 0x7f1980102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19850c4700 1 -- 172.21.15.26:0/2471081685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801a7fd0 con 0x7f198006c4d0 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19850c4700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 0x7f1980102a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9a0 tx=0x7f197000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f19858c5700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 0x7f1980102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f1976ffd700 1 -- 172.21.15.26:0/2471081685 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000cbd0 con 0x7f198006c4d0 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f1976ffd700 1 -- 172.21.15.26:0/2471081685 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f197000cd70 con 0x7f198006c4d0 2024-05-20T21:03:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f1976ffd700 1 -- 172.21.15.26:0/2471081685 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970018680 con 0x7f198006c4d0 2024-05-20T21:03:25.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801a82c0 con 0x7f198006c4d0 2024-05-20T21:03:25.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.995+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a87b0 con 0x7f198006c4d0 2024-05-20T21:03:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:25.999+0000 7f1976ffd700 1 -- 172.21.15.26:0/2471081685 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1970010430 con 0x7f198006c4d0 2024-05-20T21:03:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 Processor -- start 2024-05-20T21:03:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 -- start start 2024-05-20T21:03:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 0x561533ae2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f1976ffd700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f196c0b29b0 0x7f196c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19860c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 0x561533ae2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19860c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 0x561533ae2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48544/0 (socket says 172.21.15.26:48544) 2024-05-20T21:03:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19860c6700 1 -- 172.21.15.26:0/1356258847 learned_addr learned my addr 172.21.15.26:0/1356258847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533addc80 0x561533ade100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561533a3bf00 con 0x561533ae27b0 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561533a2baa0 con 0x561533addc80 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.000+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561533a2fae0 con 0x561533ade640 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19858c5700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f196c0b29b0 0x7f196c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f1976ffd700 1 -- 172.21.15.26:0/2471081685 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f19700e25d0 con 0x7f198006c4d0 2024-05-20T21:03:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533addc80 0x561533ade100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19860c6700 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 -- 172.21.15.26:0/1356258847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 msgr2=0x561533aea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 -- 172.21.15.26:0/1356258847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 msgr2=0x561533ae2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 0x561533ae2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 -- 172.21.15.26:0/1356258847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615339d5140 con 0x561533addc80 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19860c6700 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 0x561533ae2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19858c5700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f196c0b29b0 0x7f196c0b4e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b2e0 tx=0x7f197c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.001+0000 7f19850c4700 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533addc80 0x561533ade100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f19700e0070 tx=0x7f19700a9610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae5757871715e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.002+0000 7f196bfff700 1 -- 172.21.15.26:0/1356258847 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700a96a0 con 0x561533addc80 2024-05-20T21:03:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.002+0000 7f196bfff700 1 -- 172.21.15.26:0/1356258847 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19700aa290 con 0x561533addc80 2024-05-20T21:03:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.002+0000 7f196bfff700 1 -- 172.21.15.26:0/1356258847 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700b29f0 con 0x561533addc80 2024-05-20T21:03:26.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.002+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533addc80 msgr2=0x561533ade100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.002+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533addc80 0x561533ade100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f19700e0070 tx=0x7f19700a9610 comp rx=0 tx=0).stop 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.002+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 shutdown_connections 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533ae27b0 0x561533ae2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1356258847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533addc80 0x561533ade100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 >> 172.21.15.26:0/1356258847 conn(0x561533a093d0 msgr2=0x561533ad8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 shutdown_connections 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 -- 172.21.15.26:0/1356258847 wait complete. 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 -- start start 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.003+0000 7f19923bfb80 1 Processor -- start 2024-05-20T21:03:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ade640 0x561533ae76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ae27b0 0x561533ae7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561533932130 con 0x561533addc80 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561533993f30 con 0x561533ade640 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561533aead10 con 0x561533ae27b0 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19923bfb80 10 client.0 _ll_register_callbacks cb 0x5615339c6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48548/0 (socket says 172.21.15.26:48548) 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 -- 172.21.15.26:0/3197226840 learned_addr learned my addr 172.21.15.26:0/3197226840 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19860c6700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ae27b0 0x561533ae7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19850c4700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ade640 0x561533ae76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 -- 172.21.15.26:0/3197226840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ae27b0 msgr2=0x561533ae7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ae27b0 0x561533ae7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 -- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ade640 msgr2=0x561533ae76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ade640 0x561533ae76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.004+0000 7f19858c5700 1 -- 172.21.15.26:0/3197226840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56153397ba60 con 0x561533addc80 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19850c4700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ade640 0x561533ae76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19858c5700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aea3e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f197c00f6e0 tx=0x7f197c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5beb189be0d381 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19697fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19697fa700 1 -- 172.21.15.26:0/3197226840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c00fe60 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19697fa700 1 -- 172.21.15.26:0/3197226840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f197c005260 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19700aa290 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19697fa700 1 -- 172.21.15.26:0/3197226840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f197c005460 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19923bfb80 10 client.17532 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.005+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615339e58f0 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19697fa700 1 -- 172.21.15.26:0/3197226840 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19700aa290 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19697fa700 1 -- 172.21.15.26:0/3197226840 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f197c022070 con 0x561533addc80 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19923bfb80 10 client.17532 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19923bfb80 10 client.17532 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:03:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19923bfb80 20 client.17532 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533af9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19923bfb80 4 client.17532 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561533a3de50 con 0x561533af78d0 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.006+0000 7f19850c4700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533af9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.007+0000 7f19697fa700 10 client.17532 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.007+0000 7f19850c4700 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533af9d50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc951ae80fe37bc server_cookie=7b3becba4977a410 in_seq=0 out_seq=0 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19697fa700 1 -- 172.21.15.26:0/3197226840 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561533a3de50 con 0x561533af78d0 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19697fa700 10 client.17532 handle_command_reply: tid=0 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 client.17532 shutdown 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 msgr2=0x561533aea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aea3e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f197c00f6e0 tx=0x7f197c009310 comp rx=0 tx=0).stop 2024-05-20T21:03:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 shutdown_connections 2024-05-20T21:03:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533af9d50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19697fa700 0 client.17532 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aea3e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ae27b0 0x561533ae7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.008+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3197226840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ade640 0x561533ae76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.009+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 >> 172.21.15.26:0/3197226840 conn(0x561533a093d0 msgr2=0x561533adcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.009+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 shutdown_connections 2024-05-20T21:03:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.009+0000 7f19923bfb80 1 -- 172.21.15.26:0/3197226840 wait complete. 2024-05-20T21:03:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.009+0000 7f19923bfb80 20 client.17532 trim_cache size 0 max 16384 2024-05-20T21:03:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.033+0000 7f19923bfb80 1 Processor -- start 2024-05-20T21:03:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.033+0000 7f19923bfb80 1 -- start start 2024-05-20T21:03:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533af3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533ad2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533aff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19923bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b2780 con 0x561533addc80 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19923bfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b0010 con 0x561533ae27b0 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19923bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19801ade60 con 0x561533ade640 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533af3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533af3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48558/0 (socket says 172.21.15.26:48558) 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 -- 172.21.15.26:0/1125207079 learned_addr learned my addr 172.21.15.26:0/1125207079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19860c6700 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533aff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19850c4700 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533ad2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 -- 172.21.15.26:0/1125207079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 msgr2=0x561533ad2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533ad2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 -- 172.21.15.26:0/1125207079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 msgr2=0x561533aff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533aff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 -- 172.21.15.26:0/1125207079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561533ae8620 con 0x561533addc80 2024-05-20T21:03:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19860c6700 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533aff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.034+0000 7f19858c5700 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533af3d70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f197c009990 tx=0x7f197c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b6fedaab8a29a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.035+0000 7f196b7fe700 1 -- 172.21.15.26:0/1125207079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c03d070 con 0x561533addc80 2024-05-20T21:03:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.035+0000 7f196b7fe700 1 -- 172.21.15.26:0/1125207079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f197c018e50 con 0x561533addc80 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.035+0000 7f196b7fe700 1 -- 172.21.15.26:0/1125207079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c038400 con 0x561533addc80 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.035+0000 7f19923bfb80 1 -- 172.21.15.26:0/1125207079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 msgr2=0x561533af3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.035+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533af3d70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f197c009990 tx=0x7f197c000bc0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 -- 172.21.15.26:0/1125207079 shutdown_connections 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533af3d70 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533aff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 --2- 172.21.15.26:0/1125207079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533ad2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 -- 172.21.15.26:0/1125207079 >> 172.21.15.26:0/1125207079 conn(0x561533a093d0 msgr2=0x561533ae17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 -- 172.21.15.26:0/1125207079 shutdown_connections 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 -- 172.21.15.26:0/1125207079 wait complete. 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 -- start start 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.036+0000 7f19923bfb80 1 Processor -- start 2024-05-20T21:03:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19923bfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533b61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19858c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19858c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48570/0 (socket says 172.21.15.26:48570) 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19858c5700 1 -- 172.21.15.26:0/3524145263 learned_addr learned my addr 172.21.15.26:0/3524145263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19801ae4d0 con 0x561533addc80 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f19801af940 con 0x561533ae27b0 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19801aeba0 con 0x561533ade640 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19850c4700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19858c5700 1 -- 172.21.15.26:0/3524145263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 msgr2=0x561533aee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.037+0000 7f19858c5700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19858c5700 1 -- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 msgr2=0x561533b61500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19858c5700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533b61500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19858c5700 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561533b61a40 con 0x561533addc80 2024-05-20T21:03:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19850c4700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19923bfb80 10 client.0 _ll_register_callbacks cb 0x5615339c6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197c018e50 con 0x561533addc80 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19858c5700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aee3a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f197c0095d0 tx=0x7f197c02fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ec80da74607d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19697fa700 10 client.17538 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19697fa700 1 -- 172.21.15.26:0/3524145263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c03d070 con 0x561533addc80 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19697fa700 1 -- 172.21.15.26:0/3524145263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f197c005260 con 0x561533addc80 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19697fa700 1 -- 172.21.15.26:0/3524145263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f197c0049c0 con 0x561533addc80 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19923bfb80 10 client.17538 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.038+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=30}) -- 0x561533b61a40 con 0x561533addc80 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19697fa700 1 -- 172.21.15.26:0/3524145263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f197c022030 con 0x561533addc80 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19923bfb80 10 client.17538 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19923bfb80 10 client.17538 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19923bfb80 20 client.17538 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533b64f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19923bfb80 4 client.17538 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.1, "await_for": 45.0}] 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.1, "await_for": 45.0}) -- 0x561533a37630 con 0x561533af78d0 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.039+0000 7f19850c4700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533b64f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.040+0000 7f19850c4700 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533b64f30 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f59d93e8a39724 server_cookie=b1136fd421da921b in_seq=0 out_seq=0 2024-05-20T21:03:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.040+0000 7f19697fa700 10 client.17538 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:26.062 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.182+0000 7f19697fa700 1 -- 172.21.15.26:0/3524145263 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561533a37630 con 0x561533af78d0 2024-05-20T21:03:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.182+0000 7f19697fa700 10 client.17538 handle_command_reply: tid=0 2024-05-20T21:03:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.182+0000 7f19923bfb80 1 client.17538 shutdown 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.182+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 msgr2=0x561533aee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.182+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aee3a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f197c0095d0 tx=0x7f197c02fb00 comp rx=0 tx=0).stop 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 shutdown_connections 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561533af78d0 0x561533b64f30 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561533addc80 0x561533aee3a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561533ae27b0 0x561533b61500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19697fa700 0 client.17538 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 --2- 172.21.15.26:0/3524145263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561533ade640 0x561533aee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 >> 172.21.15.26:0/3524145263 conn(0x561533a093d0 msgr2=0x561533ae8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:26.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 shutdown_connections 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 1 -- 172.21.15.26:0/3524145263 wait complete. 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.183+0000 7f19923bfb80 20 client.17538 trim_cache size 0 max 16384 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 26, 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:03:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "7ce51d22": { 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 26, 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.1, 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:03:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:26.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f196c0b29b0 msgr2=0x7f196c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f196c0b29b0 0x7f196c0b4e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b2e0 tx=0x7f197c01a040 comp rx=0 tx=0).stop 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 msgr2=0x7f1980102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 0x7f1980102a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9a0 tx=0x7f197000ecb0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 shutdown_connections 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f196c0b29b0 0x7f196c0b4e70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19801095d0 0x7f1980102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198006bb10 0x7f1980102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 --2- 172.21.15.26:0/2471081685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198006c4d0 0x7f1980102a40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 >> 172.21.15.26:0/2471081685 conn(0x7f1980079480 msgr2=0x7f1980101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 shutdown_connections 2024-05-20T21:03:26.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:26.184+0000 7f1987b29700 1 -- 172.21.15.26:0/2471081685 wait complete. 2024-05-20T21:03:26.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.40 MB/sec execute 212 sec latency 484.692 ms 2024-05-20T21:03:27.097 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7ce51d22', quiesce took 8.8 seconds. Will release after: 1.299999999999999 2024-05-20T21:03:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:27 smithi055 ceph-mon[35213]: pgmap v609: 97 pgs: 97 active+clean; 27 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-05-20T21:03:27.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:27 smithi026 ceph-mon[31990]: pgmap v609: 97 pgs: 97 active+clean; 27 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-05-20T21:03:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:27 smithi144 ceph-mon[35660]: pgmap v609: 97 pgs: 97 active+clean; 27 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-05-20T21:03:27.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.39 MB/sec execute 213 sec latency 1484.796 ms 2024-05-20T21:03:28.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:03:28.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.38 MB/sec execute 214 sec latency 2484.855 ms 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 -- 172.21.15.26:0/2650933370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85781084c0 msgr2=0x7f85781088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 --2- 172.21.15.26:0/2650933370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85781084c0 0x7f85781088a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8574009990 tx=0x7f8574009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 -- 172.21.15.26:0/2650933370 shutdown_connections 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 --2- 172.21.15.26:0/2650933370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85781084c0 0x7f85781088a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 --2- 172.21.15.26:0/2650933370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8578103600 0x7f857810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 --2- 172.21.15.26:0/2650933370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 0x7f85781030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.610+0000 7f857f8d8700 1 -- 172.21.15.26:0/2650933370 >> 172.21.15.26:0/2650933370 conn(0x7f8578078ab0 msgr2=0x7f85780ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.611+0000 7f857f8d8700 1 -- 172.21.15.26:0/2650933370 shutdown_connections 2024-05-20T21:03:28.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.611+0000 7f857f8d8700 1 -- 172.21.15.26:0/2650933370 wait complete. 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.611+0000 7f857f8d8700 1 Processor -- start 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.611+0000 7f857f8d8700 1 -- start start 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.611+0000 7f857f8d8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 0x7f857810fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857f8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 0x7f8578110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857f8d8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85781084c0 0x7f8578110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857f8d8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8578113680 con 0x7f8578103600 2024-05-20T21:03:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857f8d8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8578113500 con 0x7f85781084c0 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857f8d8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8578113800 con 0x7f8578102c40 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 0x7f8578110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 0x7f8578110130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48588/0 (socket says 172.21.15.26:48588) 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 -- 172.21.15.26:0/3961584822 learned_addr learned my addr 172.21.15.26:0/3961584822 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857d674700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 0x7f857810fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857de75700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85781084c0 0x7f8578110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 -- 172.21.15.26:0/3961584822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 msgr2=0x7f857810fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 0x7f857810fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 -- 172.21.15.26:0/3961584822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85781084c0 msgr2=0x7f8578110670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85781084c0 0x7f8578110670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 -- 172.21.15.26:0/3961584822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85781ac5a0 con 0x7f8578103600 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857d674700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 0x7f857810fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.612+0000 7f857ce73700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 0x7f8578110130 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f856800b530 tx=0x7f856800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.613+0000 7f856e7fc700 1 -- 172.21.15.26:0/3961584822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568015020 con 0x7f8578103600 2024-05-20T21:03:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.613+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85781ac830 con 0x7f8578103600 2024-05-20T21:03:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.613+0000 7f856e7fc700 1 -- 172.21.15.26:0/3961584822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8568011cc0 con 0x7f8578103600 2024-05-20T21:03:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.613+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781acdc0 con 0x7f8578103600 2024-05-20T21:03:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.613+0000 7f856e7fc700 1 -- 172.21.15.26:0/3961584822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568010d30 con 0x7f8578103600 2024-05-20T21:03:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.616+0000 7f856e7fc700 1 -- 172.21.15.26:0/3961584822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8568018020 con 0x7f8578103600 2024-05-20T21:03:28.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f856e7fc700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f85640b2900 0x7f85640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 Processor -- start 2024-05-20T21:03:28.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 -- start start 2024-05-20T21:03:28.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f856e7fc700 1 -- 172.21.15.26:0/3961584822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f85680e1a30 con 0x7f8578103600 2024-05-20T21:03:28.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f857d674700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f85640b2900 0x7f85640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e818cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 0x563e8e81c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e81d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e8e777d10 con 0x563e8e819290 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563e8e7678b0 con 0x563e8e81cfe0 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.617+0000 7f858a16eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e8e76b8f0 con 0x563e8e8188e0 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 0x563e8e81c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 0x563e8e81c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48592/0 (socket says 172.21.15.26:48592) 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 -- 172.21.15.26:0/1322014110 learned_addr learned my addr 172.21.15.26:0/1322014110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857de75700 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e81d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857d674700 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e818cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 -- 172.21.15.26:0/1322014110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 msgr2=0x563e8e818cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e818cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 -- 172.21.15.26:0/1322014110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 msgr2=0x563e8e81d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e81d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 -- 172.21.15.26:0/1322014110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8e6b7950 con 0x563e8e819290 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857d674700 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e818cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857d674700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f85640b2900 0x7f85640b4dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8574009750 tx=0x7f857400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f857ce73700 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 0x563e8e81c9c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f85680a8f20 tx=0x7f8568010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a0516c6805e007 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f85637fe700 1 -- 172.21.15.26:0/1322014110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680a9690 con 0x563e8e819290 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f85637fe700 1 -- 172.21.15.26:0/1322014110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85680a8f90 con 0x563e8e819290 2024-05-20T21:03:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.618+0000 7f85637fe700 1 -- 172.21.15.26:0/1322014110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680a8450 con 0x563e8e819290 2024-05-20T21:03:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 -- 172.21.15.26:0/1322014110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 msgr2=0x563e8e81c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 0x563e8e81c9c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f85680a8f20 tx=0x7f8568010fa0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 -- 172.21.15.26:0/1322014110 shutdown_connections 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e81d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e818cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 --2- 172.21.15.26:0/1322014110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e819290 0x563e8e81c9c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 -- 172.21.15.26:0/1322014110 >> 172.21.15.26:0/1322014110 conn(0x563e8e746100 msgr2=0x563e8e812f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 -- 172.21.15.26:0/1322014110 shutdown_connections 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.619+0000 7f858a16eb80 1 -- 172.21.15.26:0/1322014110 wait complete. 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 -- start start 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 Processor -- start 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e8243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e821710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e821c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e8e66e020 con 0x563e8e8188e0 2024-05-20T21:03:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563e8e709840 con 0x563e8e81cfe0 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e8e8251a0 con 0x563e8e819290 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f858a16eb80 10 client.0 _ll_register_callbacks cb 0x563e8e7029a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f857d674700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e8243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f857d674700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e8243c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48600/0 (socket says 172.21.15.26:48600) 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.620+0000 7f857d674700 1 -- 172.21.15.26:0/523326408 learned_addr learned my addr 172.21.15.26:0/523326408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857ce73700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e821710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857de75700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e821c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857d674700 1 -- 172.21.15.26:0/523326408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 msgr2=0x563e8e821710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857d674700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e821710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857d674700 1 -- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 msgr2=0x563e8e821c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857d674700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e821c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857d674700 1 -- 172.21.15.26:0/523326408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8e706710 con 0x563e8e8188e0 2024-05-20T21:03:28.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f857d674700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e8243c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f857806ae90 tx=0x7f8574009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd6b4113e36cae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f8560ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f8560ff9700 1 -- 172.21.15.26:0/523326408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574034060 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85680a8f90 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.621+0000 7f8560ff9700 1 -- 172.21.15.26:0/523326408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f857400be40 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f8560ff9700 1 -- 172.21.15.26:0/523326408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8574034060 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 10 client.17550 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e8e82f8f0 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f8560ff9700 1 -- 172.21.15.26:0/523326408 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85680a8f90 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f8560ff9700 1 -- 172.21.15.26:0/523326408 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8574038030 con 0x563e8e8188e0 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 10 client.17550 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 20 client.17550 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e831ed0 0x563e8e8342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 4 client.17550 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e8e7e4a60 con 0x563e8e831ed0 2024-05-20T21:03:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.622+0000 7f857ce73700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e831ed0 0x563e8e8342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.623+0000 7f8560ff9700 10 client.17550 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.623+0000 7f857ce73700 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e831ed0 0x563e8e8342f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b06a0f0344fb31 server_cookie=e1d39adc0f6d115f in_seq=0 out_seq=0 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f8560ff9700 1 -- 172.21.15.26:0/523326408 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563e8e7e4a60 con 0x563e8e831ed0 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f8560ff9700 10 client.17550 handle_command_reply: tid=0 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 client.17550 shutdown 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 msgr2=0x563e8e8243c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e8243c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f857806ae90 tx=0x7f8574009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 shutdown_connections 2024-05-20T21:03:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e831ed0 0x563e8e8342f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e8243c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e821c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 --2- 172.21.15.26:0/523326408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e821710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 >> 172.21.15.26:0/523326408 conn(0x563e8e746100 msgr2=0x563e8e81e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f8560ff9700 0 client.17550 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 shutdown_connections 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 1 -- 172.21.15.26:0/523326408 wait complete. 2024-05-20T21:03:28.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.624+0000 7f858a16eb80 20 client.17550 trim_cache size 0 max 16384 2024-05-20T21:03:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.646+0000 7f858a16eb80 1 Processor -- start 2024-05-20T21:03:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.646+0000 7f858a16eb80 1 -- start start 2024-05-20T21:03:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e821c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e839c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e83a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f858a16eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f855c010020 con 0x563e8e8188e0 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f858a16eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f855c00d8b0 con 0x563e8e81cfe0 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f858a16eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f855c00b6c0 con 0x563e8e819290 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e821c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e821c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48616/0 (socket says 172.21.15.26:48616) 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 -- 172.21.15.26:0/2807062262 learned_addr learned my addr 172.21.15.26:0/2807062262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857ce73700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e839c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857de75700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e83a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 -- 172.21.15.26:0/2807062262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 msgr2=0x563e8e839c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e839c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 -- 172.21.15.26:0/2807062262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 msgr2=0x563e8e83a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e83a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 -- 172.21.15.26:0/2807062262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8e813dc0 con 0x563e8e8188e0 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857ce73700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e839c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857de75700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e83a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f857d674700 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e821c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f8574005a60 tx=0x7f8574009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacccc682bd117b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.647+0000 7f8562ffd700 1 -- 172.21.15.26:0/2807062262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574034060 con 0x563e8e8188e0 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f8562ffd700 1 -- 172.21.15.26:0/2807062262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8574005c00 con 0x563e8e8188e0 2024-05-20T21:03:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f8562ffd700 1 -- 172.21.15.26:0/2807062262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857402f8a0 con 0x563e8e8188e0 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 -- 172.21.15.26:0/2807062262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 msgr2=0x563e8e821c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e821c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f8574005a60 tx=0x7f8574009310 comp rx=0 tx=0).stop 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 -- 172.21.15.26:0/2807062262 shutdown_connections 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e8188e0 0x563e8e821c80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e81cfe0 0x563e8e83a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 --2- 172.21.15.26:0/2807062262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e819290 0x563e8e839c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.648+0000 7f858a16eb80 1 -- 172.21.15.26:0/2807062262 >> 172.21.15.26:0/2807062262 conn(0x563e8e746100 msgr2=0x563e8e817f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 -- 172.21.15.26:0/2807062262 shutdown_connections 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 -- 172.21.15.26:0/2807062262 wait complete. 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 -- start start 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 Processor -- start 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e82ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e819290 0x563e8e827460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 0x563e8e827a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f855c00bd70 con 0x563e8e81cfe0 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f855c00d1e0 con 0x563e8e819290 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.649+0000 7f858a16eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f855c00c440 con 0x563e8e8188e0 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f858a16eb80 10 client.0 _ll_register_callbacks cb 0x563e8e7029a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857de75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 0x563e8e827a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857de75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 0x563e8e827a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48618/0 (socket says 172.21.15.26:48618) 2024-05-20T21:03:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857ce73700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e819290 0x563e8e827460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857de75700 1 -- 172.21.15.26:0/3507297812 learned_addr learned my addr 172.21.15.26:0/3507297812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e82ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 -- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e819290 msgr2=0x563e8e827460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e819290 0x563e8e827460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 -- 172.21.15.26:0/3507297812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 msgr2=0x563e8e827a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 0x563e8e827a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 -- 172.21.15.26:0/3507297812 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8e828260 con 0x563e8e8188e0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857de75700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 0x563e8e827a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f857d674700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e82ac20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8574009ce0 tx=0x7f857400f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1061d09b4a0631e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f8560ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.650+0000 7f8560ff9700 1 -- 172.21.15.26:0/3507297812 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574034030 con 0x563e8e8188e0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f8560ff9700 1 -- 172.21.15.26:0/3507297812 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f857402f4e0 con 0x563e8e8188e0 2024-05-20T21:03:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e8e779c60 con 0x563e8e8188e0 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f8560ff9700 1 -- 172.21.15.26:0/3507297812 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f857402f6e0 con 0x563e8e8188e0 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 10 client.26851 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e8e89db90 con 0x563e8e8188e0 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f8560ff9700 1 -- 172.21.15.26:0/3507297812 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563e8e779c60 con 0x563e8e8188e0 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f8560ff9700 1 -- 172.21.15.26:0/3507297812 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8574053020 con 0x563e8e8188e0 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 10 client.26851 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 10 client.26851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 20 client.26851 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e89e130 0x563e8e8a0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 4 client.26851 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.651+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563e8e773440 con 0x563e8e89e130 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.652+0000 7f857ce73700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e89e130 0x563e8e8a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.652+0000 7f8560ff9700 10 client.26851 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.652+0000 7f857ce73700 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e89e130 0x563e8e8a0550 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe4e18b3794abba server_cookie=847afcca3aeac3d3 in_seq=0 out_seq=0 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.653+0000 7f8560ff9700 1 -- 172.21.15.26:0/3507297812 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563e8e773440 con 0x563e8e89e130 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.653+0000 7f8560ff9700 10 client.26851 handle_command_reply: tid=0 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.653+0000 7f858a16eb80 1 client.26851 shutdown 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 msgr2=0x563e8e82ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e82ac20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8574009ce0 tx=0x7f857400f8d0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 shutdown_connections 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563e8e89e130 0x563e8e8a0550 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e8e81cfe0 0x563e8e827a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8e8188e0 0x563e8e82ac20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 --2- 172.21.15.26:0/3507297812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e8e819290 0x563e8e827460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 >> 172.21.15.26:0/3507297812 conn(0x563e8e746100 msgr2=0x563e8e822ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f8560ff9700 0 client.26851 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 shutdown_connections 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 1 -- 172.21.15.26:0/3507297812 wait complete. 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.654+0000 7f858a16eb80 20 client.26851 trim_cache size 0 max 16384 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "50f460ee-14e4-4033-8a50-7855764eb175", 2024-05-20T21:03:28.658 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:03:28.659 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f85640b2900 msgr2=0x7f85640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f85640b2900 0x7f85640b4dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8574009750 tx=0x7f857400b540 comp rx=0 tx=0).stop 2024-05-20T21:03:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 msgr2=0x7f8578110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 0x7f8578110130 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f856800b530 tx=0x7f856800b8f0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 shutdown_connections 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85781084c0 0x7f8578110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578102c40 0x7f857810fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f85640b2900 0x7f85640b4dc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 --2- 172.21.15.26:0/3961584822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8578103600 0x7f8578110130 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 >> 172.21.15.26:0/3961584822 conn(0x7f8578078ab0 msgr2=0x7f8578100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 shutdown_connections 2024-05-20T21:03:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:28.655+0000 7f857f8d8700 1 -- 172.21.15.26:0/3961584822 wait complete. 2024-05-20T21:03:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:29.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:29 smithi055 ceph-mon[35213]: pgmap v610: 97 pgs: 97 active+clean; 27 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T21:03:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:29 smithi026 ceph-mon[31990]: pgmap v610: 97 pgs: 97 active+clean; 27 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T21:03:29.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:03:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:29 smithi144 ceph-mon[35660]: pgmap v610: 97 pgs: 97 active+clean; 27 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-05-20T21:03:29.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.37 MB/sec execute 215 sec latency 3484.899 ms 2024-05-20T21:03:30.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:30 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:03:30.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:30 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:30.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:30 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:03:30.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:30 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:30.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:30 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:03:30.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:30 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:30.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:30 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:03:30.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:30 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:30.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:30 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:03:30.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:30 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:30.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:30 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:03:30.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:30 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:30.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.36 MB/sec execute 216 sec latency 4485.241 ms 2024-05-20T21:03:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:31 smithi055 ceph-mon[35213]: pgmap v611: 97 pgs: 97 active+clean; 27 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-20T21:03:31.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:31 smithi026 ceph-mon[31990]: pgmap v611: 97 pgs: 97 active+clean; 27 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-20T21:03:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:31 smithi144 ceph-mon[35660]: pgmap v611: 97 pgs: 97 active+clean; 27 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-20T21:03:31.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.35 MB/sec execute 217 sec latency 5485.293 ms 2024-05-20T21:03:32.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:32 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:03:32.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:32 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:03:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:32 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:03:32.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.34 MB/sec execute 218 sec latency 6485.382 ms 2024-05-20T21:03:33.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:33 smithi026 ceph-mon[31990]: pgmap v612: 97 pgs: 97 active+clean; 27 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-05-20T21:03:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:33 smithi055 ceph-mon[35213]: pgmap v612: 97 pgs: 97 active+clean; 27 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-05-20T21:03:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:33 smithi144 ceph-mon[35660]: pgmap v612: 97 pgs: 97 active+clean; 27 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-05-20T21:03:33.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.33 MB/sec execute 219 sec latency 7485.463 ms 2024-05-20T21:03:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:34.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.31 MB/sec execute 220 sec latency 8485.521 ms 2024-05-20T21:03:35.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:35 smithi026 ceph-mon[31990]: pgmap v613: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-05-20T21:03:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:35 smithi055 ceph-mon[35213]: pgmap v613: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-05-20T21:03:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 -- 172.21.15.26:0/3045820557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0103820 msgr2=0x7f4db0103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 --2- 172.21.15.26:0/3045820557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0103820 0x7f4db0103c00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4da00099c0 tx=0x7f4da0009cd0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 -- 172.21.15.26:0/3045820557 shutdown_connections 2024-05-20T21:03:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 --2- 172.21.15.26:0/3045820557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0103820 0x7f4db0103c00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 --2- 172.21.15.26:0/3045820557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4db0108910 0x7f4db0110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 --2- 172.21.15.26:0/3045820557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01041d0 0x7f4db01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 -- 172.21.15.26:0/3045820557 >> 172.21.15.26:0/3045820557 conn(0x7f4db007a6b0 msgr2=0x7f4db007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 -- 172.21.15.26:0/3045820557 shutdown_connections 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 -- 172.21.15.26:0/3045820557 wait complete. 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.541+0000 7f4db668d700 1 Processor -- start 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 -- start start 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db0103820 0x7f4db019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4db01041d0 0x7f4db019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 0x7f4db0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01135a0 con 0x7f4db0108910 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0113420 con 0x7f4db01041d0 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db668d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0113720 con 0x7f4db0103820 2024-05-20T21:03:35.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 0x7f4db0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4daf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4db01041d0 0x7f4db019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 0x7f4db0199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42054/0 (socket says 172.21.15.26:42054) 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 -- 172.21.15.26:0/2214062753 learned_addr learned my addr 172.21.15.26:0/2214062753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4daffff700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db0103820 0x7f4db019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 -- 172.21.15.26:0/2214062753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db0103820 msgr2=0x7f4db019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db0103820 0x7f4db019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 -- 172.21.15.26:0/2214062753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4db01041d0 msgr2=0x7f4db019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4db01041d0 0x7f4db019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.542+0000 7f4db4c2a700 1 -- 172.21.15.26:0/2214062753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0199fe0 con 0x7f4db0108910 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.543+0000 7f4db4c2a700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 0x7f4db0199880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4da800ef30 tx=0x7f4da800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.543+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da80092e0 con 0x7f4db0108910 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.543+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da8004bb0 con 0x7f4db0108910 2024-05-20T21:03:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.543+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db019a270 con 0x7f4db0108910 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.543+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004020 con 0x7f4db0108910 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.543+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a6590 con 0x7f4db0108910 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.544+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db010bb50 con 0x7f4db0108910 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.545+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4da801b030 con 0x7f4db0108910 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.546+0000 7f4dad7fa700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d980b2880 0x7f4d980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.546+0000 7f4daffff700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d980b2880 0x7f4d980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.546+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4da80e26f0 con 0x7f4db0108910 2024-05-20T21:03:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.547+0000 7f4daffff700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d980b2880 0x7f4d980b4d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4da0000c00 tx=0x7f4da0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:35.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:35 smithi144 ceph-mon[35660]: pgmap v613: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-05-20T21:03:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.549+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4da8022020 con 0x7f4db0108910 2024-05-20T21:03:35.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.30 MB/sec execute 221 sec latency 9485.567 ms 2024-05-20T21:03:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.828+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db019adb0 con 0x7f4db0108910 2024-05-20T21:03:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.829+0000 7f4dad7fa700 1 -- 172.21.15.26:0/2214062753 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f4da8010040 con 0x7f4db0108910 2024-05-20T21:03:35.830 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:03:35.830 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d980b2880 msgr2=0x7f4d980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d980b2880 0x7f4d980b4d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4da0000c00 tx=0x7f4da0005fb0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 msgr2=0x7f4db0199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 0x7f4db0199880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4da800ef30 tx=0x7f4da800c3a0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 shutdown_connections 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4d980b2880 0x7f4d980b4d40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4db0108910 0x7f4db0199880 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db0103820 0x7f4db019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 --2- 172.21.15.26:0/2214062753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4db01041d0 0x7f4db019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 >> 172.21.15.26:0/2214062753 conn(0x7f4db007a6b0 msgr2=0x7f4db01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 shutdown_connections 2024-05-20T21:03:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:35.832+0000 7f4db668d700 1 -- 172.21.15.26:0/2214062753 wait complete. 2024-05-20T21:03:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:03:36.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 -- 172.21.15.26:0/911623882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424101f70 msgr2=0x7f04241023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 --2- 172.21.15.26:0/911623882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424101f70 0x7f04241023f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0414009990 tx=0x7f0414009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 -- 172.21.15.26:0/911623882 shutdown_connections 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 --2- 172.21.15.26:0/911623882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0424102930 0x7f0424110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 --2- 172.21.15.26:0/911623882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0424109720 0x7f0424109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 --2- 172.21.15.26:0/911623882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424101f70 0x7f04241023f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 -- 172.21.15.26:0/911623882 >> 172.21.15.26:0/911623882 conn(0x7f04240776b0 msgr2=0x7f0424077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 -- 172.21.15.26:0/911623882 shutdown_connections 2024-05-20T21:03:36.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.259+0000 7f042b610700 1 -- 172.21.15.26:0/911623882 wait complete. 2024-05-20T21:03:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 Processor -- start 2024-05-20T21:03:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 -- start start 2024-05-20T21:03:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0424101f70 0x7f0424103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0424102930 0x7f0424103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 0x7f0424103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113680 con 0x7f0424109720 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113500 con 0x7f0424101f70 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f042b610700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113800 con 0x7f0424102930 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f0429bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 0x7f0424103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.260+0000 7f0429bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 0x7f0424103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42070/0 (socket says 172.21.15.26:42070) 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 -- 172.21.15.26:0/2535295108 learned_addr learned my addr 172.21.15.26:0/2535295108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0428bab700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0424102930 0x7f0424103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f04293ac700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0424101f70 0x7f0424103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 -- 172.21.15.26:0/2535295108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0424102930 msgr2=0x7f0424103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0424102930 0x7f0424103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 -- 172.21.15.26:0/2535295108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0424101f70 msgr2=0x7f0424103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0424101f70 0x7f0424103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 -- 172.21.15.26:0/2535295108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04241a8160 con 0x7f0424109720 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f0429bad700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 0x7f0424103bd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b970 tx=0x7f041c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f041a7fc700 1 -- 172.21.15.26:0/2535295108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00d860 con 0x7f0424109720 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04241a8450 con 0x7f0424109720 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241a8960 con 0x7f0424109720 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f041a7fc700 1 -- 172.21.15.26:0/2535295108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f041c00e430 con 0x7f0424109720 2024-05-20T21:03:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.261+0000 7f041a7fc700 1 -- 172.21.15.26:0/2535295108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c017660 con 0x7f0424109720 2024-05-20T21:03:36.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.264+0000 7f041a7fc700 1 -- 172.21.15.26:0/2535295108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f041c00da00 con 0x7f0424109720 2024-05-20T21:03:36.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f041a7fc700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04100b2900 0x7f04100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f04293ac700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04100b2900 0x7f04100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f041a7fc700 1 -- 172.21.15.26:0/2535295108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f041c0e1e30 con 0x7f0424109720 2024-05-20T21:03:36.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 Processor -- start 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 -- start start 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f04293ac700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04100b2900 0x7f04100b4dc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0420009a10 tx=0x7f0420009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381ea860 0x55f3381eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381ee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eef60 0x55f3381ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f338149c90 con 0x55f3381eb210 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f338139830 con 0x55f3381eef60 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.265+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f33813d870 con 0x55f3381ea860 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42080/0 (socket says 172.21.15.26:42080) 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 -- 172.21.15.26:0/3966174511 learned_addr learned my addr 172.21.15.26:0/3966174511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f04293ac700 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381ea860 0x55f3381eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0429bad700 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eef60 0x55f3381ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 -- 172.21.15.26:0/3966174511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381ea860 msgr2=0x55f3381eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381ea860 0x55f3381eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 -- 172.21.15.26:0/3966174511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eef60 msgr2=0x55f3381ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eef60 0x55f3381ef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 -- 172.21.15.26:0/3966174511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3380898d0 con 0x55f3381eb210 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0429bad700 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eef60 0x55f3381ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f0428bab700 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381ee940 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0414009990 tx=0x7f0414005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2ec31dc75d7358 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f040f7fe700 1 -- 172.21.15.26:0/3966174511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041400b9d0 con 0x55f3381eb210 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f040f7fe700 1 -- 172.21.15.26:0/3966174511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0414027070 con 0x55f3381eb210 2024-05-20T21:03:36.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.266+0000 7f040f7fe700 1 -- 172.21.15.26:0/3966174511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041400fd60 con 0x55f3381eb210 2024-05-20T21:03:36.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 -- 172.21.15.26:0/3966174511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 msgr2=0x55f3381ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381ee940 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0414009990 tx=0x7f0414005c70 comp rx=0 tx=0).stop 2024-05-20T21:03:36.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 -- 172.21.15.26:0/3966174511 shutdown_connections 2024-05-20T21:03:36.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eef60 0x55f3381ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381ea860 0x55f3381eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/3966174511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381ee940 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 -- 172.21.15.26:0/3966174511 >> 172.21.15.26:0/3966174511 conn(0x55f338118080 msgr2=0x55f3381e4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 -- 172.21.15.26:0/3966174511 shutdown_connections 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 -- 172.21.15.26:0/3966174511 wait complete. 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.267+0000 7f0435ea6b80 1 -- start start 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 Processor -- start 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 0x55f3381f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eb210 0x55f3381f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f33803ffa0 con 0x55f3381ea860 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f3380db7c0 con 0x55f3381eb210 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f3381f70c0 con 0x55f3381eef60 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0435ea6b80 10 client.0 _ll_register_callbacks cb 0x55f3380d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:36.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 0x55f3381f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 0x55f3381f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42090/0 (socket says 172.21.15.26:42090) 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 -- 172.21.15.26:0/4148516487 learned_addr learned my addr 172.21.15.26:0/4148516487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0428bab700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eb210 0x55f3381f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f0429bad700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 -- 172.21.15.26:0/4148516487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 msgr2=0x55f3381f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 -- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eb210 msgr2=0x55f3381f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.268+0000 7f04293ac700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eb210 0x55f3381f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f04293ac700 1 -- 172.21.15.26:0/4148516487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3380d8690 con 0x55f3381ea860 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f04293ac700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 0x55f3381f6350 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0420015500 tx=0x7f0420004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b508594b9dbfe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f040cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f040cff9700 1 -- 172.21.15.26:0/4148516487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420025070 con 0x55f3381ea860 2024-05-20T21:03:36.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f040cff9700 1 -- 172.21.15.26:0/4148516487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f0420015c80 con 0x55f3381ea860 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0414027070 con 0x55f3381ea860 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f040cff9700 1 -- 172.21.15.26:0/4148516487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f042001b040 con 0x55f3381ea860 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f0435ea6b80 10 client.17574 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f338201780 con 0x55f3381ea860 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f040cff9700 1 -- 172.21.15.26:0/4148516487 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0414027070 con 0x55f3381ea860 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.269+0000 7f040cff9700 1 -- 172.21.15.26:0/4148516487 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f0420006570 con 0x55f3381ea860 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0435ea6b80 10 client.17574 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0435ea6b80 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0435ea6b80 20 client.17574 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f338203db0 0x55f3382061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0435ea6b80 4 client.17574 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:36.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3381b69e0 con 0x55f338203db0 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0428bab700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f338203db0 0x55f3382061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f040cff9700 10 client.17574 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.270+0000 7f0428bab700 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f338203db0 0x55f3382061a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67878a04c8cfe53d server_cookie=591b3484adbe9f72 in_seq=0 out_seq=0 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.271+0000 7f040cff9700 1 -- 172.21.15.26:0/4148516487 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f3381b69e0 con 0x55f338203db0 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.271+0000 7f040cff9700 10 client.17574 handle_command_reply: tid=0 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.271+0000 7f0435ea6b80 1 client.17574 shutdown 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 msgr2=0x55f3381f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 0x55f3381f6350 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0420015500 tx=0x7f0420004860 comp rx=0 tx=0).stop 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 shutdown_connections 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f338203db0 0x55f3382061a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381ea860 0x55f3381f6350 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/4148516487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381eb210 0x55f3381f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f040cff9700 0 client.17574 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:36.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 >> 172.21.15.26:0/4148516487 conn(0x55f338118080 msgr2=0x55f3381f0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:36.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 shutdown_connections 2024-05-20T21:03:36.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 1 -- 172.21.15.26:0/4148516487 wait complete. 2024-05-20T21:03:36.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.272+0000 7f0435ea6b80 20 client.17574 trim_cache size 0 max 16384 2024-05-20T21:03:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 Processor -- start 2024-05-20T21:03:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 -- start start 2024-05-20T21:03:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eb210 0x55f3381f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 0x55f33820bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04241b2760 con 0x55f3381eef60 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04241afff0 con 0x55f3381ea860 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04241ade40 con 0x55f3381eb210 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0429bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 0x55f33820bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0429bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 0x55f33820bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42100/0 (socket says 172.21.15.26:42100) 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.293+0000 7f0429bad700 1 -- 172.21.15.26:0/1029468946 learned_addr learned my addr 172.21.15.26:0/1029468946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0428bab700 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eb210 0x55f3381f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f04293ac700 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0429bad700 1 -- 172.21.15.26:0/1029468946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eb210 msgr2=0x55f3381f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0429bad700 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eb210 0x55f3381f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0429bad700 1 -- 172.21.15.26:0/1029468946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 msgr2=0x55f3381e5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0429bad700 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381e5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0429bad700 1 -- 172.21.15.26:0/1029468946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3381e3c80 con 0x55f3381eef60 2024-05-20T21:03:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f0429bad700 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 0x55f33820bab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b940 tx=0x7f041c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950561d0d49e9dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f040effd700 1 -- 172.21.15.26:0/1029468946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0ad7a0 con 0x55f3381eef60 2024-05-20T21:03:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f040effd700 1 -- 172.21.15.26:0/1029468946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f041c0b2250 con 0x55f3381eef60 2024-05-20T21:03:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.294+0000 7f040effd700 1 -- 172.21.15.26:0/1029468946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0a98b0 con 0x55f3381eef60 2024-05-20T21:03:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1029468946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 msgr2=0x55f33820bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 0x55f33820bab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b940 tx=0x7f041c01b000 comp rx=0 tx=0).stop 2024-05-20T21:03:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1029468946 shutdown_connections 2024-05-20T21:03:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eef60 0x55f33820bab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1029468946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eb210 0x55f3381f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1029468946 >> 172.21.15.26:0/1029468946 conn(0x55f338118080 msgr2=0x55f3381f4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1029468946 shutdown_connections 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1029468946 wait complete. 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 -- start start 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.295+0000 7f0435ea6b80 1 Processor -- start 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04241ae4b0 con 0x55f3381eb210 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04241af920 con 0x55f3381ea860 2024-05-20T21:03:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04241aeb80 con 0x55f3381eef60 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0435ea6b80 10 client.0 _ll_register_callbacks cb 0x55f3380d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f04293ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381f9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42104/0 (socket says 172.21.15.26:42104) 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 -- 172.21.15.26:0/1939814020 learned_addr learned my addr 172.21.15.26:0/1939814020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0429bad700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 -- 172.21.15.26:0/1939814020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 msgr2=0x55f3381f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 -- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 msgr2=0x55f3381fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0428bab700 1 -- 172.21.15.26:0/1939814020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3381fa0f0 con 0x55f3381eb210 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f0429bad700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.296+0000 7f04293ac700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f0428bab700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381f9320 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f04140098e0 tx=0x7f0414009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e0833a6c104942 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f040cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f040cff9700 1 -- 172.21.15.26:0/1939814020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140059d0 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f040cff9700 1 -- 172.21.15.26:0/1939814020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f041400b7d0 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f041c0b2250 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f040cff9700 1 -- 172.21.15.26:0/1939814020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f041400fe20 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f0435ea6b80 10 client.17580 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f33826fa50 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f040cff9700 1 -- 172.21.15.26:0/1939814020 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f041c0b2250 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f040cff9700 1 -- 172.21.15.26:0/1939814020 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f0414036760 con 0x55f3381eb210 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f0435ea6b80 10 client.17580 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.297+0000 7f0435ea6b80 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f0435ea6b80 20 client.17580 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f33826fcd0 0x55f3382720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f0435ea6b80 4 client.17580 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f33814bbe0 con 0x55f33826fcd0 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f04293ac700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f33826fcd0 0x55f3382720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f04293ac700 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f33826fcd0 0x55f3382720c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d360c417a21606 server_cookie=ff2f7f0591bea1a1 in_seq=0 out_seq=0 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.298+0000 7f040cff9700 10 client.17580 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f040cff9700 1 -- 172.21.15.26:0/1939814020 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f33814bbe0 con 0x55f33826fcd0 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f040cff9700 10 client.17580 handle_command_reply: tid=0 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 client.17580 shutdown 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 msgr2=0x55f3381f9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381f9320 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f04140098e0 tx=0x7f0414009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 shutdown_connections 2024-05-20T21:03:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f33826fcd0 0x55f3382720c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3381eef60 0x55f3381f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3381ea860 0x55f3381fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 --2- 172.21.15.26:0/1939814020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3381eb210 0x55f3381f9320 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 >> 172.21.15.26:0/1939814020 conn(0x55f338118080 msgr2=0x55f3381edc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f040cff9700 0 client.17580 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 shutdown_connections 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 1 -- 172.21.15.26:0/1939814020 wait complete. 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.299+0000 7f0435ea6b80 20 client.17580 trim_cache size 0 max 16384 2024-05-20T21:03:36.304 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04100b2900 msgr2=0x7f04100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04100b2900 0x7f04100b4dc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0420009a10 tx=0x7f0420009190 comp rx=0 tx=0).stop 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 msgr2=0x7f0424103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:36.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 0x7f0424103bd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b970 tx=0x7f041c00bc80 comp rx=0 tx=0).stop 2024-05-20T21:03:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 shutdown_connections 2024-05-20T21:03:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04100b2900 0x7f04100b4dc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424109720 0x7f0424103bd0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0424101f70 0x7f0424103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 --2- 172.21.15.26:0/2535295108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0424102930 0x7f0424103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 >> 172.21.15.26:0/2535295108 conn(0x7f04240776b0 msgr2=0x7f0424100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 shutdown_connections 2024-05-20T21:03:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:36.300+0000 7f042b610700 1 -- 172.21.15.26:0/2535295108 wait complete. 2024-05-20T21:03:36.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:36 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2214062753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:36.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:36 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2214062753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:36.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:36 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2214062753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:36.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.29 MB/sec execute 222 sec latency 10485.603 ms 2024-05-20T21:03:36.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 7ce51d22 --release --await' 2024-05-20T21:03:36.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 7ce51d22 --release --await 2024-05-20T21:03:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:37 smithi055 ceph-mon[35213]: pgmap v614: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 609 KiB/s wr, 72 op/s 2024-05-20T21:03:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:37 smithi026 ceph-mon[31990]: pgmap v614: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 609 KiB/s wr, 72 op/s 2024-05-20T21:03:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:37 smithi144 ceph-mon[35660]: pgmap v614: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 609 KiB/s wr, 72 op/s 2024-05-20T21:03:37.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.28 MB/sec execute 223 sec latency 11485.715 ms 2024-05-20T21:03:37.594 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:03:37.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:03:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:38.409 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:38.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.27 MB/sec execute 224 sec latency 12485.825 ms 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 -- 172.21.15.26:0/810594654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70103ca0 msgr2=0x7f7c70104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 --2- 172.21.15.26:0/810594654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70103ca0 0x7f7c70104120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402a410 comp rx=0 tx=0).stop 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 -- 172.21.15.26:0/810594654 shutdown_connections 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 --2- 172.21.15.26:0/810594654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c70104660 0x7f7c7010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 --2- 172.21.15.26:0/810594654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70103ca0 0x7f7c70104120 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 --2- 172.21.15.26:0/810594654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c70102ab0 0x7f7c70102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 -- 172.21.15.26:0/810594654 >> 172.21.15.26:0/810594654 conn(0x7f7c700fe040 msgr2=0x7f7c70100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 -- 172.21.15.26:0/810594654 shutdown_connections 2024-05-20T21:03:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.802+0000 7f7c75b32640 1 -- 172.21.15.26:0/810594654 wait complete. 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 Processor -- start 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 -- start start 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c70102ab0 0x7f7c70198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c70103ca0 0x7f7c70199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 0x7f7c701a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010e500 con 0x7f7c70104660 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010e380 con 0x7f7c70103ca0 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c75b32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010e680 con 0x7f7c70102ab0 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 0x7f7c701a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 0x7f7c701a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42112/0 (socket says 172.21.15.26:42112) 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 -- 172.21.15.26:0/1894880079 learned_addr learned my addr 172.21.15.26:0/1894880079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6effd640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c70103ca0 0x7f7c70199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c70102ab0 0x7f7c70198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 -- 172.21.15.26:0/1894880079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c70102ab0 msgr2=0x7f7c70198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c70102ab0 0x7f7c70198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 -- 172.21.15.26:0/1894880079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c70103ca0 msgr2=0x7f7c70199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c70103ca0 0x7f7c70199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.803+0000 7f7c6ffff640 1 -- 172.21.15.26:0/1894880079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c701a0830 con 0x7f7c70104660 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.804+0000 7f7c6ffff640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 0x7f7c701a0190 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800cad0 tx=0x7f7c6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.804+0000 7f7c6cff9640 1 -- 172.21.15.26:0/1894880079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800ae20 con 0x7f7c70104660 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.804+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c701a0b20 con 0x7f7c70104660 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.804+0000 7f7c6cff9640 1 -- 172.21.15.26:0/1894880079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c68005020 con 0x7f7c70104660 2024-05-20T21:03:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.804+0000 7f7c6cff9640 1 -- 172.21.15.26:0/1894880079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800f6b0 con 0x7f7c70104660 2024-05-20T21:03:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.804+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c701a1070 con 0x7f7c70104660 2024-05-20T21:03:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.806+0000 7f7c7884e740 1 Processor -- start 2024-05-20T21:03:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.806+0000 7f7c6cff9640 1 -- 172.21.15.26:0/1894880079 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7c680051d0 con 0x7f7c70104660 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.806+0000 7f7c7884e740 1 -- start start 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c7884e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c245051c30 con 0x55c2450e10b0 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c7884e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c2450417d0 con 0x55c2450e6220 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c7884e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c245045810 con 0x55c2450e06c0 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42124/0 (socket says 172.21.15.26:42124) 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/334185482 learned_addr learned my addr 172.21.15.26:0/334185482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6effd640 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/334185482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 msgr2=0x55c2450e0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450e0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/334185482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 msgr2=0x55c2450e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/334185482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c244d56210 con 0x55c2450e10b0 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6effd640 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6cff9640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c580b28a0 0x7f7c580b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450e5bb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b7f0 tx=0x7f7c6000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35008fe61bcd0426 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c6effd640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c580b28a0 0x7f7c580b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.807+0000 7f7c55ffb640 1 -- 172.21.15.26:0/334185482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000d950 con 0x55c2450e10b0 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c6cff9640 1 -- 172.21.15.26:0/1894880079 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7c680e2760 con 0x7f7c70104660 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c55ffb640 1 -- 172.21.15.26:0/334185482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c60018070 con 0x55c2450e10b0 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c55ffb640 1 -- 172.21.15.26:0/334185482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60013da0 con 0x55c2450e10b0 2024-05-20T21:03:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 -- 172.21.15.26:0/334185482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 msgr2=0x55c2450e5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450e5bb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b7f0 tx=0x7f7c6000bcb0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c6effd640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c580b28a0 0x7f7c580b4d60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c64000c00 tx=0x7f7c64005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 -- 172.21.15.26:0/334185482 shutdown_connections 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450e5bb0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 --2- 172.21.15.26:0/334185482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 -- 172.21.15.26:0/334185482 >> 172.21.15.26:0/334185482 conn(0x55c2450dbd30 msgr2=0x55c2450dc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 -- 172.21.15.26:0/334185482 shutdown_connections 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.808+0000 7f7c7884e740 1 -- 172.21.15.26:0/334185482 wait complete. 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 -- start start 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 Processor -- start 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e10b0 0x55c2450ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c244d27790 con 0x55c2450e06c0 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c244ffd890 con 0x55c2450e6220 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c2450e91d0 con 0x55c2450e10b0 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c7884e740 10 client.0 _ll_register_callbacks cb 0x55c244d55a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c6f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c6f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450eeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42140/0 (socket says 172.21.15.26:42140) 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/474675432 learned_addr learned my addr 172.21.15.26:0/474675432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c6effd640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e10b0 0x55c2450ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.809+0000 7f7c6ffff640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/474675432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e10b0 msgr2=0x55c2450ef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e10b0 0x55c2450ef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 msgr2=0x55c2450f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c6f7fe640 1 -- 172.21.15.26:0/474675432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c244e22620 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450eeb90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7c60008010 tx=0x7f7c60013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200188e40995f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c4effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c4effd640 1 -- 172.21.15.26:0/474675432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000db50 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c4effd640 1 -- 172.21.15.26:0/474675432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7c6000dcf0 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c60018070 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c4effd640 1 -- 172.21.15.26:0/474675432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c6002d050 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c7884e740 10 client.17592 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2450f9060 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c4effd640 1 -- 172.21.15.26:0/474675432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c60018070 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.810+0000 7f7c4effd640 1 -- 172.21.15.26:0/474675432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7c60027450 con 0x55c2450e06c0 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c7884e740 10 client.17592 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c7884e740 10 client.17592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c7884e740 20 client.17592 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c7884e740 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2450fa4c0 0x55c2450fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c7884e740 4 client.17592 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c245053b80 con 0x55c2450fa4c0 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c6effd640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2450fa4c0 0x55c2450fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c6effd640 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2450fa4c0 0x55c2450fc8b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dffc3334cd390e0 server_cookie=1aa8b8890b33cf99 in_seq=0 out_seq=0 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.811+0000 7f7c4effd640 10 client.17592 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c4effd640 1 -- 172.21.15.26:0/474675432 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c245053b80 con 0x55c2450fa4c0 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c4effd640 10 client.17592 handle_command_reply: tid=0 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 client.17592 shutdown 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 msgr2=0x55c2450eeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450eeb90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7c60008010 tx=0x7f7c60013b20 comp rx=0 tx=0).stop 2024-05-20T21:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 shutdown_connections 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2450fa4c0 0x55c2450fc8b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e10b0 0x55c2450ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c4effd640 0 client.17592 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 --2- 172.21.15.26:0/474675432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450eeb90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 >> 172.21.15.26:0/474675432 conn(0x55c2450dbd30 msgr2=0x55c2450dafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 shutdown_connections 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 1 -- 172.21.15.26:0/474675432 wait complete. 2024-05-20T21:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.812+0000 7f7c7884e740 20 client.17592 trim_cache size 0 max 16384 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 Processor -- start 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 -- start start 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 0x55c2450f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e6220 0x55c2450ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701aa6d0 con 0x55c2450e06c0 2024-05-20T21:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701a7fa0 con 0x55c2450e10b0 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.828+0000 7f7c7884e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701a5df0 con 0x55c2450e6220 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 0x55c2450f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e6220 0x55c2450ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 0x55c2450f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49622/0 (socket says 172.21.15.26:49622) 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 -- 172.21.15.26:0/2010833755 learned_addr learned my addr 172.21.15.26:0/2010833755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 -- 172.21.15.26:0/2010833755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e6220 msgr2=0x55c2450ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e6220 0x55c2450ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 -- 172.21.15.26:0/2010833755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 msgr2=0x55c2450df320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450df320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 -- 172.21.15.26:0/2010833755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2450d76f0 con 0x55c2450e10b0 2024-05-20T21:03:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c6effd640 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 0x55c2450f60d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c6402a450 tx=0x7f7c6403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e7303cc4837dad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c557fa640 1 -- 172.21.15.26:0/2010833755 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c640043c0 con 0x55c2450e10b0 2024-05-20T21:03:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c557fa640 1 -- 172.21.15.26:0/2010833755 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c64003a80 con 0x55c2450e10b0 2024-05-20T21:03:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.829+0000 7f7c557fa640 1 -- 172.21.15.26:0/2010833755 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64051880 con 0x55c2450e10b0 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 -- 172.21.15.26:0/2010833755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 msgr2=0x55c2450f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 0x55c2450f60d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c6402a450 tx=0x7f7c6403e000 comp rx=0 tx=0).stop 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 -- 172.21.15.26:0/2010833755 shutdown_connections 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e6220 0x55c2450ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e10b0 0x55c2450f60d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2010833755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e06c0 0x55c2450df320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 -- 172.21.15.26:0/2010833755 >> 172.21.15.26:0/2010833755 conn(0x55c2450dbd30 msgr2=0x55c2450dec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 -- 172.21.15.26:0/2010833755 shutdown_connections 2024-05-20T21:03:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.830+0000 7f7c7884e740 1 -- 172.21.15.26:0/2010833755 wait complete. 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 -- start start 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 Processor -- start 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701a6460 con 0x55c2450e10b0 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701a78d0 con 0x55c2450e6220 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701a6b30 con 0x55c2450e06c0 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c7884e740 10 client.0 _ll_register_callbacks cb 0x7f7c6000b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42164/0 (socket says 172.21.15.26:42164) 2024-05-20T21:03:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c6effd640 1 -- 172.21.15.26:0/2181338653 learned_addr learned my addr 172.21.15.26:0/2181338653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.831+0000 7f7c6ffff640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c6effd640 1 -- 172.21.15.26:0/2181338653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 msgr2=0x55c2450f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c6effd640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c6effd640 1 -- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 msgr2=0x55c2450f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c6effd640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c6effd640 1 -- 172.21.15.26:0/2181338653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2450f4eb0 con 0x55c2450e10b0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c6effd640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450f38b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005e00 tx=0x7f7c640513b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eecae00cd7699ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c4effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c4effd640 1 -- 172.21.15.26:0/2181338653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003e40 con 0x55c2450e10b0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c4effd640 1 -- 172.21.15.26:0/2181338653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7c64057040 con 0x55c2450e10b0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c64003a80 con 0x55c2450e10b0 2024-05-20T21:03:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c4effd640 1 -- 172.21.15.26:0/2181338653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c640035f0 con 0x55c2450e10b0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c7884e740 10 client.17598 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2450f61f0 con 0x55c2450e10b0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.832+0000 7f7c4effd640 1 -- 172.21.15.26:0/2181338653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c64003a80 con 0x55c2450e10b0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c4effd640 1 -- 172.21.15.26:0/2181338653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7c64063050 con 0x55c2450e10b0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c7884e740 10 client.17598 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c7884e740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c7884e740 20 client.17598 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2451655a0 0x55c2451679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c7884e740 4 client.17598 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c24504d360 con 0x55c2451655a0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2451655a0 0x55c2451679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c6f7fe640 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2451655a0 0x55c2451679c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a301e4f416b7beed server_cookie=c3701a265bcc59f1 in_seq=0 out_seq=0 2024-05-20T21:03:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.833+0000 7f7c4effd640 10 client.17598 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.858+0000 7f7c4effd640 1 -- 172.21.15.26:0/2181338653 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c24504d360 con 0x55c2451655a0 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.858+0000 7f7c4effd640 10 client.17598 handle_command_reply: tid=0 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.858+0000 7f7c7884e740 1 client.17598 shutdown 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 msgr2=0x55c2450f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450f38b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005e00 tx=0x7f7c640513b0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 shutdown_connections 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c2451655a0 0x55c2451679c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c2450e6220 0x55c2450f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2450e10b0 0x55c2450f38b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c4effd640 0 client.17598 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 --2- 172.21.15.26:0/2181338653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2450e06c0 0x55c2450f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 >> 172.21.15.26:0/2181338653 conn(0x55c2450dbd30 msgr2=0x55c2450df4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 shutdown_connections 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 1 -- 172.21.15.26:0/2181338653 wait complete. 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c7884e740 20 client.17598 trim_cache size 0 max 16384 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:03:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:03:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:03:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c580b28a0 msgr2=0x7f7c580b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c75b32640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c580b28a0 0x7f7c580b4d60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c64000c00 tx=0x7f7c64005b50 comp rx=0 tx=0).stop 2024-05-20T21:03:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 msgr2=0x7f7c701a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.859+0000 7f7c75b32640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 0x7f7c701a0190 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800cad0 tx=0x7f7c6800cf90 comp rx=0 tx=0).stop 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 shutdown_connections 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c580b28a0 0x7f7c580b4d60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c70104660 0x7f7c701a0190 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c70103ca0 0x7f7c70199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 --2- 172.21.15.26:0/1894880079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c70102ab0 0x7f7c70198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 >> 172.21.15.26:0/1894880079 conn(0x7f7c700fe040 msgr2=0x7f7c70107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 shutdown_connections 2024-05-20T21:03:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:38.860+0000 7f7c75b32640 1 -- 172.21.15.26:0/1894880079 wait complete. 2024-05-20T21:03:39.344 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:39 smithi026 ceph-mon[31990]: pgmap v615: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 337 KiB/s wr, 67 op/s 2024-05-20T21:03:39.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:39 smithi055 ceph-mon[35213]: pgmap v615: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 337 KiB/s wr, 67 op/s 2024-05-20T21:03:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:39 smithi144 ceph-mon[35660]: pgmap v615: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 337 KiB/s wr, 67 op/s 2024-05-20T21:03:39.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160109 2.26 MB/sec execute 225 sec latency 13485.918 ms 2024-05-20T21:03:39.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 -- 172.21.15.26:0/1791613101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c06bd10 msgr2=0x7fce6c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:39.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 --2- 172.21.15.26:0/1791613101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c06bd10 0x7fce6c06c190 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fce600099c0 tx=0x7fce60009cd0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 -- 172.21.15.26:0/1791613101 shutdown_connections 2024-05-20T21:03:39.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 --2- 172.21.15.26:0/1791613101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 0x7fce6c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 --2- 172.21.15.26:0/1791613101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c109630 0x7fce6c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 --2- 172.21.15.26:0/1791613101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c06bd10 0x7fce6c06c190 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 -- 172.21.15.26:0/1791613101 >> 172.21.15.26:0/1791613101 conn(0x7fce6c0794e0 msgr2=0x7fce6c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:39.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 -- 172.21.15.26:0/1791613101 shutdown_connections 2024-05-20T21:03:39.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 -- 172.21.15.26:0/1791613101 wait complete. 2024-05-20T21:03:39.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.987+0000 7fce71d73700 1 Processor -- start 2024-05-20T21:03:39.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 -- start start 2024-05-20T21:03:39.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c06bd10 0x7fce6c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 0x7fce6c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 0x7fce6c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c113600 con 0x7fce6c109630 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c113480 con 0x7fce6c06bd10 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce71d73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c113780 con 0x7fce6c06c6d0 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce6b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c06bd10 0x7fce6c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 0x7fce6c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce6affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 0x7fce6c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce6b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c06bd10 0x7fce6c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53070/0 (socket says 172.21.15.26:53070) 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 0x7fce6c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43198/0 (socket says 172.21.15.26:43198) 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.988+0000 7fce6b7fe700 1 -- 172.21.15.26:0/350322539 learned_addr learned my addr 172.21.15.26:0/350322539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6bfff700 1 -- 172.21.15.26:0/350322539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 msgr2=0x7fce6c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6bfff700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 0x7fce6c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6bfff700 1 -- 172.21.15.26:0/350322539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c06bd10 msgr2=0x7fce6c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6bfff700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c06bd10 0x7fce6c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6bfff700 1 -- 172.21.15.26:0/350322539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c19a040 con 0x7fce6c109630 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6affd700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 0x7fce6c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.989+0000 7fce6bfff700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 0x7fce6c1998e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fce64007f10 tx=0x7fce6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.990+0000 7fce68ff9700 1 -- 172.21.15.26:0/350322539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004db0 con 0x7fce6c109630 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.990+0000 7fce68ff9700 1 -- 172.21.15.26:0/350322539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce64009e20 con 0x7fce6c109630 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.990+0000 7fce68ff9700 1 -- 172.21.15.26:0/350322539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64015400 con 0x7fce6c109630 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.990+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c19a330 con 0x7fce6c109630 2024-05-20T21:03:39.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.990+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c1a65f0 con 0x7fce6c109630 2024-05-20T21:03:39.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.993+0000 7fce68ff9700 1 -- 172.21.15.26:0/350322539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fce640155a0 con 0x7fce6c109630 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.993+0000 7fce68ff9700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce540b2900 0x7fce540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 Processor -- start 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce68ff9700 1 -- 172.21.15.26:0/350322539 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fce640e1ba0 con 0x7fce6c109630 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce540b2900 0x7fce540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 -- start start 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d70442720 0x557d70442b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 0x557d7043e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d7044a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557d7039be70 con 0x557d7043dbf0 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557d7038ba10 con 0x557d70442720 2024-05-20T21:03:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce7c609b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557d7038fa50 con 0x557d7043e5b0 2024-05-20T21:03:39.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 0x557d7043e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d70442720 0x557d70442b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d7044a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 0x557d7043e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43204/0 (socket says 172.21.15.26:43204) 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6affd700 1 -- 172.21.15.26:0/4011507811 learned_addr learned my addr 172.21.15.26:0/4011507811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.994+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce540b2900 0x7fce540b4dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005950 tx=0x7fce5c00f6a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6bfff700 1 -- 172.21.15.26:0/4011507811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d70442720 msgr2=0x557d70442b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6bfff700 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d70442720 0x557d70442b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6bfff700 1 -- 172.21.15.26:0/4011507811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 msgr2=0x557d7043e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6bfff700 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 0x557d7043e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6bfff700 1 -- 172.21.15.26:0/4011507811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d703350b0 con 0x557d7043e5b0 2024-05-20T21:03:39.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6affd700 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 0x557d7043e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:39.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce6bfff700 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d7044a620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce64000f80 tx=0x7fce640a8c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a6bb90e22f26b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:39.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce51ffb700 1 -- 172.21.15.26:0/4011507811 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640ad510 con 0x557d7043e5b0 2024-05-20T21:03:39.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce51ffb700 1 -- 172.21.15.26:0/4011507811 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce640b1fc0 con 0x557d7043e5b0 2024-05-20T21:03:39.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.995+0000 7fce51ffb700 1 -- 172.21.15.26:0/4011507811 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640a93c0 con 0x557d7043e5b0 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 -- 172.21.15.26:0/4011507811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 msgr2=0x557d7044a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d7044a620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce64000f80 tx=0x7fce640a8c70 comp rx=0 tx=0).stop 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 -- 172.21.15.26:0/4011507811 shutdown_connections 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d7044a620 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d70442720 0x557d70442b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4011507811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043dbf0 0x557d7043e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 -- 172.21.15.26:0/4011507811 >> 172.21.15.26:0/4011507811 conn(0x557d70369340 msgr2=0x557d704388e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 -- 172.21.15.26:0/4011507811 shutdown_connections 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.996+0000 7fce7c609b80 1 -- 172.21.15.26:0/4011507811 wait complete. 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 -- start start 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 Processor -- start 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043dbf0 0x557d7044a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043e5b0 0x557d704475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d70447b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557d702920a0 con 0x557d70442720 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557d702f3ea0 con 0x557d7043e5b0 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557d7044ac80 con 0x557d7043dbf0 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce7c609b80 10 client.0 _ll_register_callbacks cb 0x557d70326a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.997+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d70447b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d70447b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43206/0 (socket says 172.21.15.26:43206) 2024-05-20T21:03:40.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 -- 172.21.15.26:0/1614331087 learned_addr learned my addr 172.21.15.26:0/1614331087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043dbf0 0x557d7044a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6affd700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043e5b0 0x557d704475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 -- 172.21.15.26:0/1614331087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043dbf0 msgr2=0x557d7044a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043dbf0 0x557d7044a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 -- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043e5b0 msgr2=0x557d704475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043e5b0 0x557d704475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 -- 172.21.15.26:0/1614331087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d702db9d0 con 0x557d70442720 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce6bfff700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d70447b30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fce640df690 tx=0x7fce64004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67164f16371dcb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce4affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce4affd700 1 -- 172.21.15.26:0/1614331087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640c8060 con 0x557d70442720 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce4affd700 1 -- 172.21.15.26:0/1614331087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fce640a93c0 con 0x557d70442720 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce640b1fc0 con 0x557d70442720 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce4affd700 1 -- 172.21.15.26:0/1614331087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce64088050 con 0x557d70442720 2024-05-20T21:03:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce7c609b80 10 client.17610 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.998+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d70345860 con 0x557d70442720 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce4affd700 1 -- 172.21.15.26:0/1614331087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce640b1fc0 con 0x557d70442720 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce4affd700 1 -- 172.21.15.26:0/1614331087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fce6400db20 con 0x557d70442720 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce7c609b80 10 client.17610 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce7c609b80 10 client.17610 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce7c609b80 20 client.17610 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce7c609b80 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d7045fe80 0x557d70462270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce7c609b80 4 client.17610 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d7039ddc0 con 0x557d7045fe80 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:39.999+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d7045fe80 0x557d70462270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.000+0000 7fce4affd700 10 client.17610 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.000+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d7045fe80 0x557d70462270 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce92d962dbb3e400 server_cookie=24049842c33eb2f7 in_seq=0 out_seq=0 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.000+0000 7fce4affd700 1 -- 172.21.15.26:0/1614331087 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557d7039ddc0 con 0x557d7045fe80 2024-05-20T21:03:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.000+0000 7fce4affd700 10 client.17610 handle_command_reply: tid=0 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 client.17610 shutdown 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 msgr2=0x557d70447b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d70447b30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fce640df690 tx=0x7fce64004b40 comp rx=0 tx=0).stop 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 shutdown_connections 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d7045fe80 0x557d70462270 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d70447b30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043dbf0 0x557d7044a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 --2- 172.21.15.26:0/1614331087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043e5b0 0x557d704475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 >> 172.21.15.26:0/1614331087 conn(0x557d70369340 msgr2=0x557d7043cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce4affd700 0 client.17610 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 shutdown_connections 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 1 -- 172.21.15.26:0/1614331087 wait complete. 2024-05-20T21:03:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.001+0000 7fce7c609b80 20 client.17610 trim_cache size 0 max 16384 2024-05-20T21:03:40.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.021+0000 7fce7c609b80 1 Processor -- start 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.021+0000 7fce7c609b80 1 -- start start 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70443af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d70432e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d7043d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce7c609b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1ae380 con 0x557d70442720 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce7c609b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1abc10 con 0x557d7043dbf0 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce7c609b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1a9a60 con 0x557d7043e5b0 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d7043d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d7043d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43220/0 (socket says 172.21.15.26:43220) 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 -- 172.21.15.26:0/3411194635 learned_addr learned my addr 172.21.15.26:0/3411194635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70443af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6affd700 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d70432e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 -- 172.21.15.26:0/3411194635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 msgr2=0x557d70432e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d70432e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 -- 172.21.15.26:0/3411194635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 msgr2=0x557d70443af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70443af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 -- 172.21.15.26:0/3411194635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d70445b60 con 0x557d70442720 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6affd700 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d70432e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.022+0000 7fce6bfff700 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d7043d3a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fce640a90f0 tx=0x7fce640a8c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c7cabf9083f952 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce517fa700 1 -- 172.21.15.26:0/3411194635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640c8040 con 0x557d70442720 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce517fa700 1 -- 172.21.15.26:0/3411194635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce640b1fc0 con 0x557d70442720 2024-05-20T21:03:40.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce517fa700 1 -- 172.21.15.26:0/3411194635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640a93c0 con 0x557d70442720 2024-05-20T21:03:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 -- 172.21.15.26:0/3411194635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 msgr2=0x557d7043d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d7043d3a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fce640a90f0 tx=0x7fce640a8c70 comp rx=0 tx=0).stop 2024-05-20T21:03:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 -- 172.21.15.26:0/3411194635 shutdown_connections 2024-05-20T21:03:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d70442720 0x557d7043d3a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70443af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 --2- 172.21.15.26:0/3411194635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d7043e5b0 0x557d70432e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 -- 172.21.15.26:0/3411194635 >> 172.21.15.26:0/3411194635 conn(0x557d70369340 msgr2=0x557d70446640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 -- 172.21.15.26:0/3411194635 shutdown_connections 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.023+0000 7fce7c609b80 1 -- 172.21.15.26:0/3411194635 wait complete. 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 -- start start 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 Processor -- start 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70461480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 0x557d704619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d70442720 0x557d704c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1aa0d0 con 0x557d7043e5b0 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1ab540 con 0x557d7043dbf0 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1aa7a0 con 0x557d70442720 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce7c609b80 10 client.0 _ll_register_callbacks cb 0x557d70326a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce6affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 0x557d704619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce6affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 0x557d704619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43228/0 (socket says 172.21.15.26:43228) 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.024+0000 7fce6affd700 1 -- 172.21.15.26:0/4208867197 learned_addr learned my addr 172.21.15.26:0/4208867197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6bfff700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d70442720 0x557d704c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70461480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6affd700 1 -- 172.21.15.26:0/4208867197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d70442720 msgr2=0x557d704c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6affd700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d70442720 0x557d704c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6affd700 1 -- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 msgr2=0x557d70461480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6affd700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70461480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6affd700 1 -- 172.21.15.26:0/4208867197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d704c3cc0 con 0x557d7043e5b0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70461480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce6affd700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 0x557d704619c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fce600099c0 tx=0x7fce60005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c342084bbb586119 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce4affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce4affd700 1 -- 172.21.15.26:0/4208867197 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60003880 con 0x557d7043e5b0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce4affd700 1 -- 172.21.15.26:0/4208867197 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fce60003a20 con 0x557d7043e5b0 2024-05-20T21:03:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce640b1fc0 con 0x557d7043e5b0 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce4affd700 1 -- 172.21.15.26:0/4208867197 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce60034050 con 0x557d7043e5b0 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce7c609b80 10 client.17616 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.025+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d704cb010 con 0x557d7043e5b0 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce4affd700 1 -- 172.21.15.26:0/4208867197 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce640b1fc0 con 0x557d7043e5b0 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce4affd700 1 -- 172.21.15.26:0/4208867197 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fce60025430 con 0x557d7043e5b0 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce7c609b80 10 client.17616 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce7c609b80 10 client.17616 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce7c609b80 20 client.17616 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d704cb7a0 0x557d704cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce7c609b80 4 client.17616 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7ce51d22", "release": true, "await": true}] 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7ce51d22", "release": true, "await": true}) -- 0x557d703975a0 con 0x557d704cb7a0 2024-05-20T21:03:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.026+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d704cb7a0 0x557d704cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:40.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.027+0000 7fce6b7fe700 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d704cb7a0 0x557d704cdbc0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158798c59c5f9153 server_cookie=b949810aac4ab96e in_seq=0 out_seq=0 2024-05-20T21:03:40.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.027+0000 7fce4affd700 10 client.17616 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.039+0000 7fce4affd700 1 -- 172.21.15.26:0/4208867197 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557d703975a0 con 0x557d704cb7a0 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.039+0000 7fce4affd700 10 client.17616 handle_command_reply: tid=0 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.039+0000 7fce7c609b80 1 client.17616 shutdown 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.039+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 msgr2=0x557d704619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.039+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 0x557d704619c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fce600099c0 tx=0x7fce60005390 comp rx=0 tx=0).stop 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 shutdown_connections 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557d704cb7a0 0x557d704cdbc0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d70442720 0x557d704c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557d7043dbf0 0x557d70461480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 --2- 172.21.15.26:0/4208867197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d7043e5b0 0x557d704619c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 >> 172.21.15.26:0/4208867197 conn(0x557d70369340 msgr2=0x557d7045c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce4affd700 0 client.17616 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 shutdown_connections 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 1 -- 172.21.15.26:0/4208867197 wait complete. 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce7c609b80 20 client.17616 trim_cache size 0 max 16384 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:40.041 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 28, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "7ce51d22": { 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 28, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.1, 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:03:40.042 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:03:40.043 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.040+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce540b2900 msgr2=0x7fce540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce540b2900 0x7fce540b4dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005950 tx=0x7fce5c00f6a0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 msgr2=0x7fce6c1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 0x7fce6c1998e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fce64007f10 tx=0x7fce6400d3b0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 shutdown_connections 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fce540b2900 0x7fce540b4dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6c109630 0x7fce6c1998e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce6c06bd10 0x7fce6c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 --2- 172.21.15.26:0/350322539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c06c6d0 0x7fce6c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 >> 172.21.15.26:0/350322539 conn(0x7fce6c0794e0 msgr2=0x7fce6c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 shutdown_connections 2024-05-20T21:03:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:40.041+0000 7fce71d73700 1 -- 172.21.15.26:0/350322539 wait complete. 2024-05-20T21:03:40.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160397 2.25 MB/sec execute 226 sec latency 13956.681 ms 2024-05-20T21:03:41.339 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:41 smithi026 ceph-mon[31990]: pgmap v616: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 148 KiB/s wr, 32 op/s 2024-05-20T21:03:41.371 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7ce51d22', total seconds elapsed: 23.1 2024-05-20T21:03:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:41 smithi055 ceph-mon[35213]: pgmap v616: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 148 KiB/s wr, 32 op/s 2024-05-20T21:03:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:41 smithi144 ceph-mon[35660]: pgmap v616: 97 pgs: 97 active+clean; 28 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 148 KiB/s wr, 32 op/s 2024-05-20T21:03:41.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 161094 2.27 MB/sec execute 227 sec latency 17.553 ms 2024-05-20T21:03:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 -- 172.21.15.26:0/765123863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 msgr2=0x7fe340108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 --2- 172.21.15.26:0/765123863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe340108900 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009990 tx=0x7fe33c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 -- 172.21.15.26:0/765123863 shutdown_connections 2024-05-20T21:03:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 --2- 172.21.15.26:0/765123863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe340108900 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 --2- 172.21.15.26:0/765123863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe340102e00 0x7fe34010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 --2- 172.21.15.26:0/765123863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102440 0x7fe3401028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 -- 172.21.15.26:0/765123863 >> 172.21.15.26:0/765123863 conn(0x7fe3400793f0 msgr2=0x7fe340079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 -- 172.21.15.26:0/765123863 shutdown_connections 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.912+0000 7fe3483ae700 1 -- 172.21.15.26:0/765123863 wait complete. 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 Processor -- start 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 -- start start 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe340102440 0x7fe340199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102e00 0x7fe34019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe34019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112480 con 0x7fe340108520 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112300 con 0x7fe340102e00 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe3483ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112600 con 0x7fe340102440 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe34614a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe340102440 0x7fe340199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe34694b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe34019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe345949700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102e00 0x7fe34019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe34694b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe34019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43252/0 (socket says 172.21.15.26:43252) 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.913+0000 7fe34694b700 1 -- 172.21.15.26:0/1095244196 learned_addr learned my addr 172.21.15.26:0/1095244196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe34694b700 1 -- 172.21.15.26:0/1095244196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe340102440 msgr2=0x7fe340199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe34694b700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe340102440 0x7fe340199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe34694b700 1 -- 172.21.15.26:0/1095244196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102e00 msgr2=0x7fe34019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe34694b700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102e00 0x7fe34019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe34694b700 1 -- 172.21.15.26:0/1095244196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34019eb00 con 0x7fe340108520 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe345949700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102e00 0x7fe34019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.914+0000 7fe34694b700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe34019e400 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe338007d70 tx=0x7fe33800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.915+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338016020 con 0x7fe340108520 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.915+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34019edf0 con 0x7fe340108520 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.915+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe33800ac90 con 0x7fe340108520 2024-05-20T21:03:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.915+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380115e0 con 0x7fe340108520 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.915+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe340103fb0 con 0x7fe340108520 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.916+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34006a230 con 0x7fe340108520 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.917+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe338019020 con 0x7fe340108520 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.918+0000 7fe3377fe700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe32c0b2880 0x7fe32c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.918+0000 7fe34614a700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe32c0b2880 0x7fe32c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.918+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe3380e2700 con 0x7fe340108520 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.919+0000 7fe34614a700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe32c0b2880 0x7fe32c0b4d40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00b2e0 tx=0x7fe33c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:41.922+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe338020020 con 0x7fe340108520 2024-05-20T21:03:42.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.228+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe34019ae60 con 0x7fe340108520 2024-05-20T21:03:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.229+0000 7fe3377fe700 1 -- 172.21.15.26:0/1095244196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fe34019ae60 con 0x7fe340108520 2024-05-20T21:03:42.229 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:03:42.229 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe32c0b2880 msgr2=0x7fe32c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe32c0b2880 0x7fe32c0b4d40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00b2e0 tx=0x7fe33c005fb0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 msgr2=0x7fe34019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe34019e400 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe338007d70 tx=0x7fe33800a340 comp rx=0 tx=0).stop 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 shutdown_connections 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe32c0b2880 0x7fe32c0b4d40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe340108520 0x7fe34019e400 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe340102440 0x7fe340199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 --2- 172.21.15.26:0/1095244196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe340102e00 0x7fe34019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:42.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 >> 172.21.15.26:0/1095244196 conn(0x7fe3400793f0 msgr2=0x7fe34006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:42.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 shutdown_connections 2024-05-20T21:03:42.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:42.232+0000 7fe3483ae700 1 -- 172.21.15.26:0/1095244196 wait complete. 2024-05-20T21:03:42.233 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:03:42.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 161794 2.26 MB/sec execute 228 sec latency 11.781 ms 2024-05-20T21:03:43.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:03:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:43.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:43 smithi026 ceph-mon[31990]: pgmap v617: 97 pgs: 97 active+clean; 28 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 85 KiB/s wr, 25 op/s 2024-05-20T21:03:43.441 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:43 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1095244196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:43 smithi055 ceph-mon[35213]: pgmap v617: 97 pgs: 97 active+clean; 28 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 85 KiB/s wr, 25 op/s 2024-05-20T21:03:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:43 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1095244196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:43 smithi144 ceph-mon[35660]: pgmap v617: 97 pgs: 97 active+clean; 28 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 85 KiB/s wr, 25 op/s 2024-05-20T21:03:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:43 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1095244196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:03:43.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 162576 2.28 MB/sec execute 229 sec latency 18.450 ms 2024-05-20T21:03:44.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 163206 2.29 MB/sec execute 230 sec latency 18.428 ms 2024-05-20T21:03:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 -- 172.21.15.26:0/2502533348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f761406bbe0 msgr2=0x7f76141103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 --2- 172.21.15.26:0/2502533348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f761406bbe0 0x7f76141103d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b0a0 tx=0x7f760c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 -- 172.21.15.26:0/2502533348 shutdown_connections 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 --2- 172.21.15.26:0/2502533348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f761406bbe0 0x7f76141103d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 --2- 172.21.15.26:0/2502533348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76141095b0 0x7f7614109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 --2- 172.21.15.26:0/2502533348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f761406b220 0x7f761406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 -- 172.21.15.26:0/2502533348 >> 172.21.15.26:0/2502533348 conn(0x7f76140793f0 msgr2=0x7f7614079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 -- 172.21.15.26:0/2502533348 shutdown_connections 2024-05-20T21:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.158+0000 7f761c130700 1 -- 172.21.15.26:0/2502533348 wait complete. 2024-05-20T21:03:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 Processor -- start 2024-05-20T21:03:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 -- start start 2024-05-20T21:03:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f761406b220 0x7f761419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f761406bbe0 0x7f761419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 0x7f7614199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7614113580 con 0x7f76141095b0 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7614113400 con 0x7f761406b220 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761c130700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7614113700 con 0x7f761406bbe0 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761a6cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 0x7f7614199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761a6cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 0x7f7614199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43270/0 (socket says 172.21.15.26:43270) 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f761a6cd700 1 -- 172.21.15.26:0/1904915443 learned_addr learned my addr 172.21.15.26:0/1904915443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.159+0000 7f7619ecc700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f761406b220 0x7f761419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f76196cb700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f761406bbe0 0x7f761419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761a6cd700 1 -- 172.21.15.26:0/1904915443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f761406bbe0 msgr2=0x7f761419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f761406bbe0 0x7f761419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761a6cd700 1 -- 172.21.15.26:0/1904915443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f761406b220 msgr2=0x7f761419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f761406b220 0x7f761419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761a6cd700 1 -- 172.21.15.26:0/1904915443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761419a060 con 0x7f76141095b0 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 0x7f7614199900 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b780 tx=0x7f760c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f760affd700 1 -- 172.21.15.26:0/1904915443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c021070 con 0x7f76141095b0 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761419a2f0 con 0x7f76141095b0 2024-05-20T21:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f760affd700 1 -- 172.21.15.26:0/1904915443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f760c007b80 con 0x7f76141095b0 2024-05-20T21:03:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f760affd700 1 -- 172.21.15.26:0/1904915443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0052a0 con 0x7f76141095b0 2024-05-20T21:03:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.160+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76141a6610 con 0x7f76141095b0 2024-05-20T21:03:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.163+0000 7f760affd700 1 -- 172.21.15.26:0/1904915443 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f760c024070 con 0x7f76141095b0 2024-05-20T21:03:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.164+0000 7f760affd700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f76000b2900 0x7f76000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.164+0000 7f7619ecc700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f76000b2900 0x7f76000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.164+0000 7f760affd700 1 -- 172.21.15.26:0/1904915443 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f760c0e32a0 con 0x7f76141095b0 2024-05-20T21:03:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.164+0000 7f7619ecc700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f76000b2900 0x7f76000b4dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f761419aa40 tx=0x7f7610009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.164+0000 7f76269c6b80 1 Processor -- start 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 -- start start 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffb860 0x55eb09ffbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09fff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 0x55eb0a0003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb09f5ac90 con 0x55eb09ffff60 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55eb09f4a830 con 0x55eb09ffc210 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76269c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb09f4e870 con 0x55eb09ffb860 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76196cb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09fff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 0x55eb0a0003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76196cb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09fff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53154/0 (socket says 172.21.15.26:53154) 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 0x55eb0a0003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43282/0 (socket says 172.21.15.26:43282) 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f76196cb700 1 -- 172.21.15.26:0/1279237972 learned_addr learned my addr 172.21.15.26:0/1279237972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f7619ecc700 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffb860 0x55eb09ffbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 -- 172.21.15.26:0/1279237972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffb860 msgr2=0x55eb09ffbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffb860 0x55eb09ffbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 -- 172.21.15.26:0/1279237972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 msgr2=0x55eb09fff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09fff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.165+0000 7f761a6cd700 1 -- 172.21.15.26:0/1279237972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb09e9a8d0 con 0x55eb09ffff60 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.166+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 0x55eb0a0003a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f760c0e0d90 tx=0x7f760c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e62a23fa18abbdb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.166+0000 7f75fffff700 1 -- 172.21.15.26:0/1279237972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c02b070 con 0x55eb09ffff60 2024-05-20T21:03:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.166+0000 7f75fffff700 1 -- 172.21.15.26:0/1279237972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f760c0aaff0 con 0x55eb09ffff60 2024-05-20T21:03:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.166+0000 7f75fffff700 1 -- 172.21.15.26:0/1279237972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c090e60 con 0x55eb09ffff60 2024-05-20T21:03:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 -- 172.21.15.26:0/1279237972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 msgr2=0x55eb0a0003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 0x55eb0a0003a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f760c0e0d90 tx=0x7f760c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 -- 172.21.15.26:0/1279237972 shutdown_connections 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffff60 0x55eb0a0003a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffb860 0x55eb09ffbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1279237972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09fff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 -- 172.21.15.26:0/1279237972 >> 172.21.15.26:0/1279237972 conn(0x55eb09f29080 msgr2=0x55eb09ff5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 -- 172.21.15.26:0/1279237972 shutdown_connections 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 -- 172.21.15.26:0/1279237972 wait complete. 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 -- start start 2024-05-20T21:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.167+0000 7f76269c6b80 1 Processor -- start 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a007480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb0a0047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a004d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb09e50fa0 con 0x55eb09ffb860 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55eb09eec7c0 con 0x55eb09ffc210 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb0a007ee0 con 0x55eb09ffff60 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76269c6b80 10 client.0 _ll_register_callbacks cb 0x55eb09ee5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f7619ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f7619ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a007480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43296/0 (socket says 172.21.15.26:43296) 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f7619ecc700 1 -- 172.21.15.26:0/2569964545 learned_addr learned my addr 172.21.15.26:0/2569964545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76196cb700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb0a0047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f761a6cd700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.173 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:45 smithi026 ceph-mon[31990]: pgmap v618: 97 pgs: 97 active+clean; 29 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 208 KiB/s wr, 40 op/s 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76196cb700 1 -- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 msgr2=0x55eb0a004d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76196cb700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a004d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76196cb700 1 -- 172.21.15.26:0/2569964545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 msgr2=0x55eb0a007480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.168+0000 7f76196cb700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a007480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f76196cb700 1 -- 172.21.15.26:0/2569964545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb09ee9690 con 0x55eb09ffc210 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f7619ecc700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a007480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f76196cb700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb0a0047f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f760400df00 tx=0x7f7604012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e9e6413c706a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f75fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f75fd7fa700 1 -- 172.21.15.26:0/2569964545 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604004500 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760c0aaff0 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f75fd7fa700 1 -- 172.21.15.26:0/2569964545 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7604003680 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f75fd7fa700 1 -- 172.21.15.26:0/2569964545 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7604003880 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f76269c6b80 10 client.27068 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.169+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb0a0126d0 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f75fd7fa700 1 -- 172.21.15.26:0/2569964545 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f760c0aaff0 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f75fd7fa700 1 -- 172.21.15.26:0/2569964545 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7604019020 con 0x55eb09ffc210 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f76269c6b80 10 client.27068 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f76269c6b80 10 client.27068 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f76269c6b80 20 client.27068 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f76269c6b80 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a017100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f76269c6b80 4 client.27068 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb09fc79e0 con 0x55eb0a014d10 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.170+0000 7f7619ecc700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a017100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.171+0000 7f7619ecc700 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a017100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=151b396c8841762a server_cookie=f25c98cb70f81810 in_seq=0 out_seq=0 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.171+0000 7f75fd7fa700 10 client.27068 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.171+0000 7f75fd7fa700 1 -- 172.21.15.26:0/2569964545 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eb09fc79e0 con 0x55eb0a014d10 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.171+0000 7f75fd7fa700 10 client.27068 handle_command_reply: tid=0 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.171+0000 7f76269c6b80 1 client.27068 shutdown 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 msgr2=0x55eb0a0047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb0a0047f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f760400df00 tx=0x7f7604012730 comp rx=0 tx=0).stop 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 shutdown_connections 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a017100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a007480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a004d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f75fd7fa700 0 client.27068 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 --2- 172.21.15.26:0/2569964545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb0a0047f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 >> 172.21.15.26:0/2569964545 conn(0x55eb09f29080 msgr2=0x55eb09ff5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 shutdown_connections 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 1 -- 172.21.15.26:0/2569964545 wait complete. 2024-05-20T21:03:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.172+0000 7f76269c6b80 20 client.27068 trim_cache size 0 max 16384 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 Processor -- start 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 -- start start 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a010eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09ff5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a01c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76141b03e0 con 0x55eb09ffb860 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f76141adc70 con 0x55eb09ffc210 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.192+0000 7f76269c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76141abac0 con 0x55eb09ffff60 2024-05-20T21:03:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a010eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a010eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43304/0 (socket says 172.21.15.26:43304) 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 -- 172.21.15.26:0/3217969348 learned_addr learned my addr 172.21.15.26:0/3217969348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f76196cb700 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09ff5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f761a6cd700 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a01c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 -- 172.21.15.26:0/3217969348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 msgr2=0x55eb0a01c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a01c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 -- 172.21.15.26:0/3217969348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 msgr2=0x55eb09ff5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09ff5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 -- 172.21.15.26:0/3217969348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb09ff7970 con 0x55eb09ffb860 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f76196cb700 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09ff5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:45.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f7619ecc700 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a010eb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7610005930 tx=0x7f7610014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f558c50a58812256 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f75ff7fe700 1 -- 172.21.15.26:0/3217969348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610004420 con 0x55eb09ffb860 2024-05-20T21:03:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f75ff7fe700 1 -- 172.21.15.26:0/3217969348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7610004cc0 con 0x55eb09ffb860 2024-05-20T21:03:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.193+0000 7f75ff7fe700 1 -- 172.21.15.26:0/3217969348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100055e0 con 0x55eb09ffb860 2024-05-20T21:03:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 -- 172.21.15.26:0/3217969348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 msgr2=0x55eb0a010eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a010eb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7610005930 tx=0x7f7610014000 comp rx=0 tx=0).stop 2024-05-20T21:03:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 -- 172.21.15.26:0/3217969348 shutdown_connections 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a010eb0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffff60 0x55eb0a01c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 --2- 172.21.15.26:0/3217969348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffc210 0x55eb09ff5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 -- 172.21.15.26:0/3217969348 >> 172.21.15.26:0/3217969348 conn(0x55eb09f29080 msgr2=0x55eb09ffaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.194+0000 7f76269c6b80 1 -- 172.21.15.26:0/3217969348 shutdown_connections 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 -- 172.21.15.26:0/3217969348 wait complete. 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 -- start start 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 Processor -- start 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a077c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffc210 0x55eb0a0781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffff60 0x55eb0a00a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76141ac130 con 0x55eb09ffb860 2024-05-20T21:03:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f76141ad5a0 con 0x55eb09ffff60 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.195+0000 7f76269c6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76141ac800 con 0x55eb09ffc210 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f76269c6b80 10 client.0 _ll_register_callbacks cb 0x55eb09ee5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f7619ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a077c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f7619ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a077c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43310/0 (socket says 172.21.15.26:43310) 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f7619ecc700 1 -- 172.21.15.26:0/1021448609 learned_addr learned my addr 172.21.15.26:0/1021448609 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffff60 0x55eb0a00a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f76196cb700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffc210 0x55eb0a0781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 -- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffc210 msgr2=0x55eb0a0781d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffc210 0x55eb0a0781d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 -- 172.21.15.26:0/1021448609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 msgr2=0x55eb0a077c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a077c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 -- 172.21.15.26:0/1021448609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0a00adb0 con 0x55eb09ffff60 2024-05-20T21:03:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f761a6cd700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffff60 0x55eb0a00a6b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f760c0aaab0 tx=0x7f760c0aaee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a062ed72fc333bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f75fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f75fd7fa700 1 -- 172.21.15.26:0/1021448609 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c02b040 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7610004cc0 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.196+0000 7f75fd7fa700 1 -- 172.21.15.26:0/1021448609 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f760c0a6ee0 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f75fd7fa700 1 -- 172.21.15.26:0/1021448609 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f760c090e60 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 10 client.27074 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb0a07f480 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f75fd7fa700 1 -- 172.21.15.26:0/1021448609 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7610004cc0 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f75fd7fa700 1 -- 172.21.15.26:0/1021448609 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f760c011070 con 0x55eb09ffff60 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 10 client.27074 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 10 client.27074 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 20 client.27074 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a081770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 4 client.27074 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:03:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb09f5cbe0 con 0x55eb0a014d10 2024-05-20T21:03:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.197+0000 7f7619ecc700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a081770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f7619ecc700 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a081770 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1942231ea5175f86 server_cookie=7bb22f6d6fc88547 in_seq=0 out_seq=0 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f75fd7fa700 10 client.27074 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f75fd7fa700 1 -- 172.21.15.26:0/1021448609 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb09f5cbe0 con 0x55eb0a014d10 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f75fd7fa700 10 client.27074 handle_command_reply: tid=0 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 client.27074 shutdown 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffff60 msgr2=0x55eb0a00a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffff60 0x55eb0a00a6b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f760c0aaab0 tx=0x7f760c0aaee0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 shutdown_connections 2024-05-20T21:03:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55eb0a014d10 0x55eb0a081770 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb09ffb860 0x55eb0a077c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb09ffff60 0x55eb0a00a6b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f75fd7fa700 0 client.27074 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 --2- 172.21.15.26:0/1021448609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb09ffc210 0x55eb0a0781d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 >> 172.21.15.26:0/1021448609 conn(0x55eb09f29080 msgr2=0x55eb09ffe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 shutdown_connections 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 1 -- 172.21.15.26:0/1021448609 wait complete. 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.198+0000 7f76269c6b80 20 client.27074 trim_cache size 0 max 16384 2024-05-20T21:03:45.203 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:03:45.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f76000b2900 msgr2=0x7f76000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f76000b2900 0x7f76000b4dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f761419aa40 tx=0x7f7610009260 comp rx=0 tx=0).stop 2024-05-20T21:03:45.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 msgr2=0x7f7614199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:45.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 0x7f7614199900 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b780 tx=0x7f760c004d00 comp rx=0 tx=0).stop 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 shutdown_connections 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f76000b2900 0x7f76000b4dc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76141095b0 0x7f7614199900 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f761406b220 0x7f761419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 --2- 172.21.15.26:0/1904915443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f761406bbe0 0x7f761419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 >> 172.21.15.26:0/1904915443 conn(0x7f76140793f0 msgr2=0x7f7614101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 shutdown_connections 2024-05-20T21:03:45.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:45.199+0000 7f761c130700 1 -- 172.21.15.26:0/1904915443 wait complete. 2024-05-20T21:03:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:45 smithi055 ceph-mon[35213]: pgmap v618: 97 pgs: 97 active+clean; 29 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 208 KiB/s wr, 40 op/s 2024-05-20T21:03:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:45 smithi144 ceph-mon[35660]: pgmap v618: 97 pgs: 97 active+clean; 29 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 208 KiB/s wr, 40 op/s 2024-05-20T21:03:45.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 163659 2.29 MB/sec execute 231 sec latency 38.725 ms 2024-05-20T21:03:46.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:03:46.318 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:46.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 164240 2.29 MB/sec execute 232 sec latency 22.754 ms 2024-05-20T21:03:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:47 smithi055 ceph-mon[35213]: pgmap v619: 97 pgs: 97 active+clean; 28 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 468 KiB/s wr, 37 op/s 2024-05-20T21:03:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:47 smithi026 ceph-mon[31990]: pgmap v619: 97 pgs: 97 active+clean; 28 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 468 KiB/s wr, 37 op/s 2024-05-20T21:03:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:47 smithi144 ceph-mon[35660]: pgmap v619: 97 pgs: 97 active+clean; 28 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 468 KiB/s wr, 37 op/s 2024-05-20T21:03:47.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 164914 2.29 MB/sec execute 233 sec latency 16.517 ms 2024-05-20T21:03:48.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 -- 172.21.15.26:0/3883333145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 msgr2=0x7f92f0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 --2- 172.21.15.26:0/3883333145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f0108900 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009990 tx=0x7f92e0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 -- 172.21.15.26:0/3883333145 shutdown_connections 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 --2- 172.21.15.26:0/3883333145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f0108900 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 --2- 172.21.15.26:0/3883333145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f0102e00 0x7f92f010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 --2- 172.21.15.26:0/3883333145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92f0102440 0x7f92f01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 -- 172.21.15.26:0/3883333145 >> 172.21.15.26:0/3883333145 conn(0x7f92f00793f0 msgr2=0x7f92f0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 -- 172.21.15.26:0/3883333145 shutdown_connections 2024-05-20T21:03:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.178+0000 7f92f5a40700 1 -- 172.21.15.26:0/3883333145 wait complete. 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.179+0000 7f92f5a40700 1 Processor -- start 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.179+0000 7f92f5a40700 1 -- start start 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.179+0000 7f92f5a40700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92f0102440 0x7f92f019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.179+0000 7f92f5a40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f0102e00 0x7f92f019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.179+0000 7f92f5a40700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92f5a40700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92f006b6f0 con 0x7f92f0108520 2024-05-20T21:03:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92f5a40700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f92f006b570 con 0x7f92f0102440 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92f5a40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92f006b870 con 0x7f92f0102e00 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43320/0 (socket says 172.21.15.26:43320) 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 -- 172.21.15.26:0/324512311 learned_addr learned my addr 172.21.15.26:0/324512311 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92eeffd700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92f0102440 0x7f92f019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f0102e00 0x7f92f019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 -- 172.21.15.26:0/324512311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f0102e00 msgr2=0x7f92f019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f0102e00 0x7f92f019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 -- 172.21.15.26:0/324512311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92f0102440 msgr2=0x7f92f019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92f0102440 0x7f92f019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 -- 172.21.15.26:0/324512311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f019a040 con 0x7f92f0108520 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f01998e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f92e8007d70 tx=0x7f92e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:48.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.180+0000 7f92f4a3e700 1 -- 172.21.15.26:0/324512311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8016020 con 0x7f92f0108520 2024-05-20T21:03:48.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.181+0000 7f92f4a3e700 1 -- 172.21.15.26:0/324512311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92e800ad40 con 0x7f92f0108520 2024-05-20T21:03:48.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.181+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f019a330 con 0x7f92f0108520 2024-05-20T21:03:48.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.181+0000 7f92f4a3e700 1 -- 172.21.15.26:0/324512311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80115e0 con 0x7f92f0108520 2024-05-20T21:03:48.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.181+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a65f0 con 0x7f92f0108520 2024-05-20T21:03:48.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.184+0000 7f92f4a3e700 1 -- 172.21.15.26:0/324512311 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f92e8019020 con 0x7f92f0108520 2024-05-20T21:03:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.184+0000 7f92f4a3e700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92d80b2900 0x7f92d80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f92f4a3e700 1 -- 172.21.15.26:0/324512311 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f92e80e2100 con 0x7f92f0108520 2024-05-20T21:03:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f92eeffd700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92d80b2900 0x7f92d80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 Processor -- start 2024-05-20T21:03:48.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f92eeffd700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92d80b2900 0x7f92d80b4dc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f92e000b2e0 tx=0x7f92e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:48.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 -- start start 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b5324ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 0x560b5324e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324ef60 0x560b5324f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560b531a9c90 con 0x560b5324b210 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560b53199830 con 0x560b5324ef60 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.185+0000 7f93002d6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b5319d870 con 0x560b5324a860 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 0x560b5324e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 0x560b5324e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43334/0 (socket says 172.21.15.26:43334) 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 -- 172.21.15.26:0/786953610 learned_addr learned my addr 172.21.15.26:0/786953610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92eeffd700 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b5324ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324ef60 0x560b5324f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 -- 172.21.15.26:0/786953610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 msgr2=0x560b5324ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b5324ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 -- 172.21.15.26:0/786953610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324ef60 msgr2=0x560b5324f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324ef60 0x560b5324f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 -- 172.21.15.26:0/786953610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b530e98d0 con 0x560b5324b210 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92eeffd700 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b5324ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 0x560b5324e940 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009cb0 tx=0x7f92e40113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27752080577d90c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92d5ffb700 1 -- 172.21.15.26:0/786953610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4011d10 con 0x560b5324b210 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92d5ffb700 1 -- 172.21.15.26:0/786953610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92e40128e0 con 0x560b5324b210 2024-05-20T21:03:48.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.186+0000 7f92d5ffb700 1 -- 172.21.15.26:0/786953610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400f8a0 con 0x560b5324b210 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 -- 172.21.15.26:0/786953610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 msgr2=0x560b5324e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 0x560b5324e940 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009cb0 tx=0x7f92e40113e0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 -- 172.21.15.26:0/786953610 shutdown_connections 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324ef60 0x560b5324f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b5324ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 --2- 172.21.15.26:0/786953610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324b210 0x560b5324e940 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 -- 172.21.15.26:0/786953610 >> 172.21.15.26:0/786953610 conn(0x560b53178080 msgr2=0x560b53244e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 -- 172.21.15.26:0/786953610 shutdown_connections 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.187+0000 7f93002d6b80 1 -- 172.21.15.26:0/786953610 wait complete. 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 -- start start 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 Processor -- start 2024-05-20T21:03:48.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b53256410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53253710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b53253c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560b5309ffa0 con 0x560b5324ef60 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560b5313b7c0 con 0x560b5324a860 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b53257120 con 0x560b5324b210 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f93002d6b80 10 client.0 _ll_register_callbacks cb 0x560b53134920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.188+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b53253c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b53253c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43340/0 (socket says 172.21.15.26:43340) 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 -- 172.21.15.26:0/3179438233 learned_addr learned my addr 172.21.15.26:0/3179438233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92eeffd700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b53256410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53253710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 -- 172.21.15.26:0/3179438233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 msgr2=0x560b53253710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53253710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 -- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 msgr2=0x560b53256410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b53256410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 -- 172.21.15.26:0/3179438233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b53138690 con 0x560b5324ef60 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b53253c50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a93c0 tx=0x7f92e80a8eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233fba3bccbabac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ceffd700 1 -- 172.21.15.26:0/3179438233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80912c0 con 0x560b5324ef60 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e40128e0 con 0x560b5324ef60 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ceffd700 1 -- 172.21.15.26:0/3179438233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f92e80a5cb0 con 0x560b5324ef60 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f92ceffd700 1 -- 172.21.15.26:0/3179438233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92e80a9dc0 con 0x560b5324ef60 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f93002d6b80 10 client.17652 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.189+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b532618d0 con 0x560b5324ef60 2024-05-20T21:03:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f92ceffd700 1 -- 172.21.15.26:0/3179438233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92e40128e0 con 0x560b5324ef60 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f92ceffd700 1 -- 172.21.15.26:0/3179438233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f92e80beb10 con 0x560b5324ef60 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f93002d6b80 10 client.17652 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f93002d6b80 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f93002d6b80 20 client.17652 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f93002d6b80 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b53263e60 0x560b53266250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f93002d6b80 4 client.17652 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b532169e0 con 0x560b53263e60 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.190+0000 7f92eeffd700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b53263e60 0x560b53266250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.191+0000 7f92eeffd700 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b53263e60 0x560b53266250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f150d6b61f4972b server_cookie=d7d37bbf1b77f015 in_seq=0 out_seq=0 2024-05-20T21:03:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.191+0000 7f92ceffd700 10 client.17652 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.191+0000 7f92ceffd700 1 -- 172.21.15.26:0/3179438233 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560b532169e0 con 0x560b53263e60 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.191+0000 7f92ceffd700 10 client.17652 handle_command_reply: tid=0 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.191+0000 7f93002d6b80 1 client.17652 shutdown 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 msgr2=0x560b53253c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b53253c50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a93c0 tx=0x7f92e80a8eb0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 shutdown_connections 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b53263e60 0x560b53266250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b53253c50 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b53256410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 --2- 172.21.15.26:0/3179438233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53253710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f92ceffd700 0 client.17652 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 >> 172.21.15.26:0/3179438233 conn(0x560b53178080 msgr2=0x560b53250350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 shutdown_connections 2024-05-20T21:03:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 1 -- 172.21.15.26:0/3179438233 wait complete. 2024-05-20T21:03:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.192+0000 7f93002d6b80 20 client.17652 trim_cache size 0 max 16384 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 Processor -- start 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 -- start start 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b53253050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 0x560b53249d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5326ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b0590 con 0x560b5324ef60 2024-05-20T21:03:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ade20 con 0x560b5324b210 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f93002d6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01abc70 con 0x560b5324a860 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5326ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f92ee7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 0x560b53249d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f92ee7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 0x560b53249d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53224/0 (socket says 172.21.15.26:53224) 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f92ee7fc700 1 -- 172.21.15.26:0/2005667017 learned_addr learned my addr 172.21.15.26:0/2005667017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.209+0000 7f92eeffd700 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b53253050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ee7fc700 1 -- 172.21.15.26:0/2005667017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 msgr2=0x560b53253050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b53253050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ee7fc700 1 -- 172.21.15.26:0/2005667017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 msgr2=0x560b5326ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5326ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ee7fc700 1 -- 172.21.15.26:0/2005667017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b53245f70 con 0x560b5324b210 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5326ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 0x560b53249d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009cb0 tx=0x7f92e4009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fdfd986430516ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92d57fa700 1 -- 172.21.15.26:0/2005667017 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4011b90 con 0x560b5324b210 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92d57fa700 1 -- 172.21.15.26:0/2005667017 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92e402a070 con 0x560b5324b210 2024-05-20T21:03:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.210+0000 7f92d57fa700 1 -- 172.21.15.26:0/2005667017 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4003ad0 con 0x560b5324b210 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 -- 172.21.15.26:0/2005667017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 msgr2=0x560b53249d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 0x560b53249d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009cb0 tx=0x7f92e4009850 comp rx=0 tx=0).stop 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 -- 172.21.15.26:0/2005667017 shutdown_connections 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5326ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324a860 0x560b53253050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2005667017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324b210 0x560b53249d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 -- 172.21.15.26:0/2005667017 >> 172.21.15.26:0/2005667017 conn(0x560b53178080 msgr2=0x560b53249670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 -- 172.21.15.26:0/2005667017 shutdown_connections 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 -- 172.21.15.26:0/2005667017 wait complete. 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.211+0000 7f93002d6b80 1 -- start start 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 Processor -- start 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b5325c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53259cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5325a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ac2e0 con 0x560b5324ef60 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ad750 con 0x560b5324a860 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ac9b0 con 0x560b5324b210 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f93002d6b80 10 client.0 _ll_register_callbacks cb 0x560b53134920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b5325c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b5325c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:53232/0 (socket says 172.21.15.26:53232) 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 -- 172.21.15.26:0/2233775907 learned_addr learned my addr 172.21.15.26:0/2233775907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92ef7fe700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5325a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53259cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 -- 172.21.15.26:0/2233775907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 msgr2=0x560b53259cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53259cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 -- 172.21.15.26:0/2233775907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 msgr2=0x560b5325a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5325a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.212+0000 7f92eeffd700 1 -- 172.21.15.26:0/2233775907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5325a860 con 0x560b5324a860 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92eeffd700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b5325c910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2874461532847 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92ceffd700 1 -- 172.21.15.26:0/2233775907 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e000b8b0 con 0x560b5324a860 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92ceffd700 1 -- 172.21.15.26:0/2233775907 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f92e000ba50 con 0x560b5324a860 2024-05-20T21:03:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e402a070 con 0x560b5324a860 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92ceffd700 1 -- 172.21.15.26:0/2233775907 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92e000bc50 con 0x560b5324a860 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f93002d6b80 10 client.27092 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b532d2c30 con 0x560b5324a860 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92ceffd700 1 -- 172.21.15.26:0/2233775907 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92e402a070 con 0x560b5324a860 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f92ceffd700 1 -- 172.21.15.26:0/2233775907 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f92e003d420 con 0x560b5324a860 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f93002d6b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.213+0000 7f93002d6b80 10 client.27092 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f93002d6b80 20 client.27092 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b532d3340 0x560b532d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f93002d6b80 4 client.27092 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b531abbe0 con 0x560b532d3340 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b532d3340 0x560b532d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f92ceffd700 10 client.27092 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f92ee7fc700 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b532d3340 0x560b532d5760 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a439405cc4848bd server_cookie=e59be9be27b18894 in_seq=0 out_seq=0 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f92ceffd700 1 -- 172.21.15.26:0/2233775907 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b531abbe0 con 0x560b532d3340 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f92ceffd700 10 client.27092 handle_command_reply: tid=0 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.214+0000 7f93002d6b80 1 client.27092 shutdown 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 msgr2=0x560b5325c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b5325c910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 shutdown_connections 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x560b532d3340 0x560b532d5760 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b5324ef60 0x560b5325a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f92ceffd700 0 client.27092 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b5324a860 0x560b5325c910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 --2- 172.21.15.26:0/2233775907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b5324b210 0x560b53259cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 >> 172.21.15.26:0/2233775907 conn(0x560b53178080 msgr2=0x560b532557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 shutdown_connections 2024-05-20T21:03:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 1 -- 172.21.15.26:0/2233775907 wait complete. 2024-05-20T21:03:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.215+0000 7f93002d6b80 20 client.27092 trim_cache size 0 max 16384 2024-05-20T21:03:48.220 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92d80b2900 msgr2=0x7f92d80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92d80b2900 0x7f92d80b4dc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f92e000b2e0 tx=0x7f92e0005fb0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 msgr2=0x7f92f01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f01998e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f92e8007d70 tx=0x7f92e800a340 comp rx=0 tx=0).stop 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 shutdown_connections 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92d80b2900 0x7f92d80b4dc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f0108520 0x7f92f01998e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92f0102440 0x7f92f019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 --2- 172.21.15.26:0/324512311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f0102e00 0x7f92f019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 >> 172.21.15.26:0/324512311 conn(0x7f92f00793f0 msgr2=0x7f92f0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 shutdown_connections 2024-05-20T21:03:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:48.216+0000 7f92f5a40700 1 -- 172.21.15.26:0/324512311 wait complete. 2024-05-20T21:03:48.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 165572 2.29 MB/sec execute 234 sec latency 46.765 ms 2024-05-20T21:03:49.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:03:49.335 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:49 smithi055 ceph-mon[35213]: pgmap v620: 97 pgs: 97 active+clean; 28 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 629 KiB/s wr, 45 op/s 2024-05-20T21:03:49.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:49 smithi026 ceph-mon[31990]: pgmap v620: 97 pgs: 97 active+clean; 28 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 629 KiB/s wr, 45 op/s 2024-05-20T21:03:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:49 smithi144 ceph-mon[35660]: pgmap v620: 97 pgs: 97 active+clean; 28 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 629 KiB/s wr, 45 op/s 2024-05-20T21:03:49.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 166462 2.31 MB/sec execute 235 sec latency 19.735 ms 2024-05-20T21:03:50.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 166959 2.32 MB/sec execute 236 sec latency 16.828 ms 2024-05-20T21:03:51.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:51 smithi026 ceph-mon[31990]: pgmap v621: 97 pgs: 97 active+clean; 29 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 896 KiB/s wr, 66 op/s 2024-05-20T21:03:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 -- 172.21.15.26:0/3299752265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 msgr2=0x7fe8b8105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 --2- 172.21.15.26:0/3299752265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b8105bd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 -- 172.21.15.26:0/3299752265 shutdown_connections 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 --2- 172.21.15.26:0/3299752265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b8106110 0x7fe8b8112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 --2- 172.21.15.26:0/3299752265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b810b750 0x7fe8b810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 --2- 172.21.15.26:0/3299752265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b8105bd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 -- 172.21.15.26:0/3299752265 >> 172.21.15.26:0/3299752265 conn(0x7fe8b8101220 msgr2=0x7fe8b8103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 -- 172.21.15.26:0/3299752265 shutdown_connections 2024-05-20T21:03:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.219+0000 7fe8bf947700 1 -- 172.21.15.26:0/3299752265 wait complete. 2024-05-20T21:03:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.220+0000 7fe8bf947700 1 Processor -- start 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.220+0000 7fe8bf947700 1 -- start start 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bf947700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bf947700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8106110 0x7fe8b819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bf947700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b810b750 0x7fe8b81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bf947700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81157e0 con 0x7fe8b8105750 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bf947700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8115660 con 0x7fe8b810b750 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bf947700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8115960 con 0x7fe8b8106110 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44234/0 (socket says 172.21.15.26:44234) 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/53764038 learned_addr learned my addr 172.21.15.26:0/53764038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b810b750 0x7fe8b81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8106110 0x7fe8b819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bdee4700 1 -- 172.21.15.26:0/53764038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8106110 msgr2=0x7fe8b819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8106110 0x7fe8b819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bdee4700 1 -- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 msgr2=0x7fe8b819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bdee4700 1 -- 172.21.15.26:0/53764038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b81a0d40 con 0x7fe8b810b750 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8106110 0x7fe8b819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.221+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.222+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b810b750 0x7fe8b81a0600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000efc0 tx=0x7fe8b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.222+0000 7fe8ae7fc700 1 -- 172.21.15.26:0/53764038 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00092e0 con 0x7fe8b810b750 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.222+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b81a0fd0 con 0x7fe8b810b750 2024-05-20T21:03:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.222+0000 7fe8ae7fc700 1 -- 172.21.15.26:0/53764038 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b0004bb0 con 0x7fe8b810b750 2024-05-20T21:03:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.222+0000 7fe8ae7fc700 1 -- 172.21.15.26:0/53764038 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0004020 con 0x7fe8b810b750 2024-05-20T21:03:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.222+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81a88b0 con 0x7fe8b810b750 2024-05-20T21:03:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.225+0000 7fe8ae7fc700 1 -- 172.21.15.26:0/53764038 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe8b001b030 con 0x7fe8b810b750 2024-05-20T21:03:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.226+0000 7fe8ae7fc700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe8a40b2900 0x7fe8a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.226+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe8a40b2900 0x7fe8a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.226+0000 7fe8ae7fc700 1 -- 172.21.15.26:0/53764038 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe8b00e1cd0 con 0x7fe8b810b750 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.226+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe8a40b2900 0x7fe8a40b4dc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40103d0 tx=0x7fe8b4007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 Processor -- start 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 -- start start 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 0x55968d1d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d8210 0x55968d1db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1dbf60 0x55968d1dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55968d136c90 con 0x55968d1d7860 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55968d126830 con 0x55968d1dbf60 2024-05-20T21:03:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55968d12a870 con 0x55968d1d8210 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 0x55968d1d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 0x55968d1d7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44248/0 (socket says 172.21.15.26:44248) 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/1939148270 learned_addr learned my addr 172.21.15.26:0/1939148270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d8210 0x55968d1db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1dbf60 0x55968d1dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/1939148270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d8210 msgr2=0x55968d1db940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d8210 0x55968d1db940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/1939148270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1dbf60 msgr2=0x55968d1dc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.227+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1dbf60 0x55968d1dc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.228+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/1939148270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968d0768d0 con 0x55968d1d7860 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.228+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d8210 0x55968d1db940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.228+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 0x55968d1d7c40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4011790 tx=0x7fe8b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e3cb825b00688e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.228+0000 7fe8a37fe700 1 -- 172.21.15.26:0/1939148270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b401e420 con 0x55968d1d7860 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.228+0000 7fe8a37fe700 1 -- 172.21.15.26:0/1939148270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b401ea50 con 0x55968d1d7860 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.228+0000 7fe8a37fe700 1 -- 172.21.15.26:0/1939148270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4026b10 con 0x55968d1d7860 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/1939148270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 msgr2=0x55968d1d7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 0x55968d1d7c40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4011790 tx=0x7fe8b4013000 comp rx=0 tx=0).stop 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/1939148270 shutdown_connections 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d7860 0x55968d1d7c40 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1dbf60 0x55968d1dc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/1939148270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d8210 0x55968d1db940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/1939148270 >> 172.21.15.26:0/1939148270 conn(0x55968d105080 msgr2=0x55968d1d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/1939148270 shutdown_connections 2024-05-20T21:03:51.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/1939148270 wait complete. 2024-05-20T21:03:51.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 -- start start 2024-05-20T21:03:51.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.229+0000 7fe8ca1ddb80 1 Processor -- start 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d1e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55968d02cfa0 con 0x55968d1dbf60 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55968d0c87c0 con 0x55968d1d8210 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55968d1e4120 con 0x55968d1d7860 2024-05-20T21:03:51.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8ca1ddb80 10 client.0 _ll_register_callbacks cb 0x55968d0c1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36672/0 (socket says 172.21.15.26:36672) 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 -- 172.21.15.26:0/3825179615 learned_addr learned my addr 172.21.15.26:0/3825179615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d1e0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 -- 172.21.15.26:0/3825179615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 msgr2=0x55968d1e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 -- 172.21.15.26:0/3825179615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 msgr2=0x55968d1e0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d1e0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.230+0000 7fe8bcee2700 1 -- 172.21.15.26:0/3825179615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968d0c5690 con 0x55968d1d8210 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e0670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a245069c3a5870bb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:51.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/3825179615 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800b6b0 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/3825179615 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe8a800b850 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b401ea50 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/3825179615 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8a800ba50 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 10 client.27104 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55968d1ee8a0 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/3825179615 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b401ea50 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/3825179615 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fe8a800faf0 con 0x55968d1d8210 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 10 client.27104 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 10 client.27104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 20 client.27104 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.231+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d1f0e30 0x55968d1f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.232+0000 7fe8ca1ddb80 4 client.27104 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:51.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.232+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55968d1a39e0 con 0x55968d1f0e30 2024-05-20T21:03:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.232+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d1f0e30 0x55968d1f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.232+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d1f0e30 0x55968d1f3290 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9192746a0621eb4 server_cookie=1fbe7bcda66a95c3 in_seq=0 out_seq=0 2024-05-20T21:03:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.232+0000 7fe8a0ff9700 10 client.27104 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/3825179615 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55968d1a39e0 con 0x55968d1f0e30 2024-05-20T21:03:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8a0ff9700 10 client.27104 handle_command_reply: tid=0 2024-05-20T21:03:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 client.27104 shutdown 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 msgr2=0x55968d1e0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e0670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a80095c0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 shutdown_connections 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d1f0e30 0x55968d1f3290 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d1e0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/3825179615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e0670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 >> 172.21.15.26:0/3825179615 conn(0x55968d105080 msgr2=0x55968d1d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.233+0000 7fe8a0ff9700 0 client.27104 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.234+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 shutdown_connections 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.234+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/3825179615 wait complete. 2024-05-20T21:03:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.234+0000 7fe8ca1ddb80 20 client.27104 trim_cache size 0 max 16384 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 Processor -- start 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 -- start start 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 0x55968d1dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 0x55968d1e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1dbf60 0x55968d1d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00fde0 con 0x55968d1d8210 2024-05-20T21:03:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00d670 con 0x55968d1d7860 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00b4c0 con 0x55968d1dbf60 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8bcee2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 0x55968d1e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 0x55968d1dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 0x55968d1dafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36682/0 (socket says 172.21.15.26:36682) 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8bcee2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 0x55968d1e2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44258/0 (socket says 172.21.15.26:44258) 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/2532167095 learned_addr learned my addr 172.21.15.26:0/2532167095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.252+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1dbf60 0x55968d1d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/2532167095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1dbf60 msgr2=0x55968d1d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1dbf60 0x55968d1d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/2532167095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 msgr2=0x55968d1e2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 0x55968d1e2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bd6e3700 1 -- 172.21.15.26:0/2532167095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968d1debe0 con 0x55968d1d7860 2024-05-20T21:03:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1dbf60 0x55968d1d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:51.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bcee2700 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 0x55968d1e2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:51.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 0x55968d1dafc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4000c00 tx=0x7fe8b4029040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b26e8c33a5b5a59a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8a2ffd700 1 -- 172.21.15.26:0/2532167095 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b401e620 con 0x55968d1d7860 2024-05-20T21:03:51.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8a2ffd700 1 -- 172.21.15.26:0/2532167095 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b40054d0 con 0x55968d1d7860 2024-05-20T21:03:51.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.253+0000 7fe8a2ffd700 1 -- 172.21.15.26:0/2532167095 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4004ae0 con 0x55968d1d7860 2024-05-20T21:03:51.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/2532167095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 msgr2=0x55968d1dafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 0x55968d1dafc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4000c00 tx=0x7fe8b4029040 comp rx=0 tx=0).stop 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/2532167095 shutdown_connections 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1dbf60 0x55968d1d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d7860 0x55968d1dafc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/2532167095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1d8210 0x55968d1e2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/2532167095 >> 172.21.15.26:0/2532167095 conn(0x55968d105080 msgr2=0x55968d1da8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/2532167095 shutdown_connections 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/2532167095 wait complete. 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.254+0000 7fe8ca1ddb80 1 -- start start 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 Processor -- start 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d259ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00bb30 con 0x55968d1dbf60 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00cfa0 con 0x55968d1d8210 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00c200 con 0x55968d1d7860 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8ca1ddb80 10 client.0 _ll_register_callbacks cb 0x55968d0c1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8bdee4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d259ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8bcee2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8bd6e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8bdee4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d259ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44260/0 (socket says 172.21.15.26:44260) 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8bdee4700 1 -- 172.21.15.26:0/863976364 learned_addr learned my addr 172.21.15.26:0/863976364 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.255+0000 7fe8bcee2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36686/0 (socket says 172.21.15.26:36686) 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8bdee4700 1 -- 172.21.15.26:0/863976364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 msgr2=0x55968d1e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8bdee4700 1 -- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 msgr2=0x55968d1e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8bdee4700 1 -- 172.21.15.26:0/863976364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968d25a000 con 0x55968d1dbf60 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8bdee4700 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d259ac0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00a6590 tx=0x7fe8b00b1f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c236fdc4499c5b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/863976364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00a9a20 con 0x55968d1dbf60 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b40054d0 con 0x55968d1dbf60 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/863976364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe8b00a97b0 con 0x55968d1dbf60 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/863976364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8b00a9320 con 0x55968d1dbf60 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8ca1ddb80 10 client.17676 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.256+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55968d25b390 con 0x55968d1dbf60 2024-05-20T21:03:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/863976364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b40054d0 con 0x55968d1dbf60 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/863976364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fe8b00c3bf0 con 0x55968d1dbf60 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8ca1ddb80 10 client.17676 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8ca1ddb80 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8ca1ddb80 20 client.17676 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d25ba40 0x55968d25de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8ca1ddb80 4 client.17676 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55968d138be0 con 0x55968d25ba40 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.257+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d25ba40 0x55968d25de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.258+0000 7fe8bd6e3700 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d25ba40 0x55968d25de60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c238b2fb280deed server_cookie=26ab828f7163d6f2 in_seq=0 out_seq=0 2024-05-20T21:03:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.258+0000 7fe8a0ff9700 10 client.17676 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.258+0000 7fe8a0ff9700 1 -- 172.21.15.26:0/863976364 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55968d138be0 con 0x55968d25ba40 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.258+0000 7fe8a0ff9700 10 client.17676 handle_command_reply: tid=0 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.258+0000 7fe8ca1ddb80 1 client.17676 shutdown 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 msgr2=0x55968d259ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d259ac0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00a6590 tx=0x7fe8b00b1f40 comp rx=0 tx=0).stop 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 shutdown_connections 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55968d25ba40 0x55968d25de60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55968d1dbf60 0x55968d259ac0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55968d1d7860 0x55968d1e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 --2- 172.21.15.26:0/863976364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55968d1d8210 0x55968d1e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 >> 172.21.15.26:0/863976364 conn(0x55968d105080 msgr2=0x55968d1d6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8a0ff9700 0 client.17676 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 shutdown_connections 2024-05-20T21:03:51.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 1 -- 172.21.15.26:0/863976364 wait complete. 2024-05-20T21:03:51.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.259+0000 7fe8ca1ddb80 20 client.17676 trim_cache size 0 max 16384 2024-05-20T21:03:51.263 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:03:51.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe8a40b2900 msgr2=0x7fe8a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe8a40b2900 0x7fe8a40b4dc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40103d0 tx=0x7fe8b4007480 comp rx=0 tx=0).stop 2024-05-20T21:03:51.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b810b750 msgr2=0x7fe8b81a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b810b750 0x7fe8b81a0600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000efc0 tx=0x7fe8b000c3a0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 shutdown_connections 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe8a40b2900 0x7fe8a40b4dc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b8105750 0x7fe8b819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8b810b750 0x7fe8b81a0600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 --2- 172.21.15.26:0/53764038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8106110 0x7fe8b819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 >> 172.21.15.26:0/53764038 conn(0x7fe8b8101220 msgr2=0x7fe8b8101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 shutdown_connections 2024-05-20T21:03:51.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:51.260+0000 7fe8bf947700 1 -- 172.21.15.26:0/53764038 wait complete. 2024-05-20T21:03:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:51 smithi055 ceph-mon[35213]: pgmap v621: 97 pgs: 97 active+clean; 29 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 896 KiB/s wr, 66 op/s 2024-05-20T21:03:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:51 smithi144 ceph-mon[35660]: pgmap v621: 97 pgs: 97 active+clean; 29 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 896 KiB/s wr, 66 op/s 2024-05-20T21:03:51.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 167354 2.31 MB/sec execute 237 sec latency 20.473 ms 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 -- 172.21.15.26:0/2944674130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e0101d00 msgr2=0x7ff4e0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2944674130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e0101d00 0x7ff4e0102160 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009900 tx=0x7ff4c802a480 comp rx=0 tx=0).stop 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 -- 172.21.15.26:0/2944674130 shutdown_connections 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2944674130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e01026a0 0x7ff4e0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2944674130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e0101d00 0x7ff4e0102160 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2944674130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e01013a0 0x7ff4e01017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 -- 172.21.15.26:0/2944674130 >> 172.21.15.26:0/2944674130 conn(0x7ff4e00fcf20 msgr2=0x7ff4e00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 -- 172.21.15.26:0/2944674130 shutdown_connections 2024-05-20T21:03:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.962+0000 7ff4e4820640 1 -- 172.21.15.26:0/2944674130 wait complete. 2024-05-20T21:03:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 Processor -- start 2024-05-20T21:03:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 -- start start 2024-05-20T21:03:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e01013a0 0x7ff4e019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e0101d00 0x7ff4e019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 0x7ff4e01a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01106c0 con 0x7ff4e01026a0 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0110540 con 0x7ff4e0101d00 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4e4820640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0110840 con 0x7ff4e01013a0 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 0x7ff4e01a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 0x7ff4e01a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44278/0 (socket says 172.21.15.26:44278) 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4d7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e0101d00 0x7ff4e019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4ded9d640 1 -- 172.21.15.26:0/2692189397 learned_addr learned my addr 172.21.15.26:0/2692189397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4de59c640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e01013a0 0x7ff4e019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4d7fff640 1 -- 172.21.15.26:0/2692189397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e01013a0 msgr2=0x7ff4e019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e01013a0 0x7ff4e019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4d7fff640 1 -- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 msgr2=0x7ff4e01a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 0x7ff4e01a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4d7fff640 1 -- 172.21.15.26:0/2692189397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e01a2980 con 0x7ff4e0101d00 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.963+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 0x7ff4e01a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.964+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e0101d00 0x7ff4e019b2a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4c802a9a0 tx=0x7ff4c8003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.964+0000 7ff4d77fe640 1 -- 172.21.15.26:0/2692189397 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c803f070 con 0x7ff4e0101d00 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.964+0000 7ff4d77fe640 1 -- 172.21.15.26:0/2692189397 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4c8031da0 con 0x7ff4e0101d00 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.964+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e01a2c10 con 0x7ff4e0101d00 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.964+0000 7ff4d77fe640 1 -- 172.21.15.26:0/2692189397 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8035030 con 0x7ff4e0101d00 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.964+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01a30d0 con 0x7ff4e0101d00 2024-05-20T21:03:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.967+0000 7ff4d77fe640 1 -- 172.21.15.26:0/2692189397 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff4c8042070 con 0x7ff4e0101d00 2024-05-20T21:03:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.967+0000 7ff4e7614740 1 Processor -- start 2024-05-20T21:03:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.967+0000 7ff4e7614740 1 -- start start 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 0x555e4fae7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4e7614740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555e4fa58c30 con 0x555e4faed220 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4e7614740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555e4fa487d0 con 0x555e4fae76c0 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4e7614740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e4fa4c810 con 0x555e4fae80b0 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 0x555e4fae7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 0x555e4fae7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36724/0 (socket says 172.21.15.26:36724) 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 -- 172.21.15.26:0/1828903641 learned_addr learned my addr 172.21.15.26:0/1828903641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4d77fe640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4cc0b28a0 0x7ff4cc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4de59c640 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4cc0b28a0 0x7ff4cc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 -- 172.21.15.26:0/1828903641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 msgr2=0x555e4faecbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faecbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 -- 172.21.15.26:0/1828903641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 msgr2=0x555e4faef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faef6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 -- 172.21.15.26:0/1828903641 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4f75d210 con 0x555e4fae76c0 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4d77fe640 1 -- 172.21.15.26:0/2692189397 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff4c8102a60 con 0x7ff4e0101d00 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.968+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 0x555e4fae7ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8013ea0 tx=0x7ff4d8013f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75eb8a819dfa1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4d4ff9640 1 -- 172.21.15.26:0/1828903641 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8022020 con 0x555e4fae76c0 2024-05-20T21:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4cc0b28a0 0x7ff4cc0b4d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8023000 tx=0x7ff4d8021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4d4ff9640 1 -- 172.21.15.26:0/1828903641 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d8013110 con 0x555e4fae76c0 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4d4ff9640 1 -- 172.21.15.26:0/1828903641 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d801ed30 con 0x555e4fae76c0 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 -- 172.21.15.26:0/1828903641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 msgr2=0x555e4fae7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 0x555e4fae7ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8013ea0 tx=0x7ff4d8013f90 comp rx=0 tx=0).stop 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 -- 172.21.15.26:0/1828903641 shutdown_connections 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faecbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 --2- 172.21.15.26:0/1828903641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae76c0 0x555e4fae7ae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.969+0000 7ff4e7614740 1 -- 172.21.15.26:0/1828903641 >> 172.21.15.26:0/1828903641 conn(0x555e4fae2d30 msgr2=0x555e4fae3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 -- 172.21.15.26:0/1828903641 shutdown_connections 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 -- 172.21.15.26:0/1828903641 wait complete. 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 -- start start 2024-05-20T21:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 Processor -- start 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 0x555e4faf1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faf1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555e4f72e790 con 0x555e4fae76c0 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555e4fa04890 con 0x555e4faed220 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e4faf01d0 con 0x555e4fae80b0 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4de59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 0x555e4faf1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4e7614740 10 client.0 _ll_register_callbacks cb 0x555e4f75ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4de59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 0x555e4faf1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44304/0 (socket says 172.21.15.26:44304) 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4de59c640 1 -- 172.21.15.26:0/3505202543 learned_addr learned my addr 172.21.15.26:0/3505202543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.970+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faf1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4de59c640 1 -- 172.21.15.26:0/3505202543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 msgr2=0x555e4faf1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4de59c640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faf1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4de59c640 1 -- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 msgr2=0x555e4faf8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4de59c640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4de59c640 1 -- 172.21.15.26:0/3505202543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4f829620 con 0x555e4fae76c0 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4de59c640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 0x555e4faf1880 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000bad0 tx=0x7ff4d000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30641d4eb7f05920 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/3505202543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0003dc0 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/3505202543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff4d0011440 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d8013110 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/3505202543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4d0011680 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4e7614740 10 client.17688 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.971+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e4fafbeb0 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/3505202543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d8013110 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/3505202543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff4d0023020 con 0x555e4fae76c0 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4e7614740 10 client.17688 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4e7614740 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4e7614740 20 client.17688 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fafd5a0 0x555e4faff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4e7614740 4 client.17688 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e4fa5ab80 con 0x555e4fafd5a0 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.972+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fafd5a0 0x555e4faff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.973+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fafd5a0 0x555e4faff9f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b170c488ff2dadcd server_cookie=31b03b550b4e5bd6 in_seq=0 out_seq=0 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.973+0000 7ff4c5ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/3505202543 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555e4fa5ab80 con 0x555e4fafd5a0 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4c5ffb640 10 client.17688 handle_command_reply: tid=0 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 client.17688 shutdown 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 msgr2=0x555e4faf1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 0x555e4faf1880 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000bad0 tx=0x7ff4d000bf90 comp rx=0 tx=0).stop 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 shutdown_connections 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fafd5a0 0x555e4faff9f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4c5ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae80b0 0x555e4faf1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3505202543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae76c0 0x555e4faf1880 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 >> 172.21.15.26:0/3505202543 conn(0x555e4fae2d30 msgr2=0x555e4fae1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 shutdown_connections 2024-05-20T21:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 1 -- 172.21.15.26:0/3505202543 wait complete. 2024-05-20T21:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.974+0000 7ff4e7614740 20 client.17688 trim_cache size 0 max 16384 2024-05-20T21:03:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.992+0000 7ff4e7614740 1 Processor -- start 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.992+0000 7ff4e7614740 1 -- start start 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4fae6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae80b0 0x555e4faf8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4e7614740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01ae9c0 con 0x555e4faed220 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4e7614740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01ac290 con 0x555e4fae80b0 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4e7614740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01aa0e0 con 0x555e4fae76c0 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faf1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44308/0 (socket says 172.21.15.26:44308) 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 -- 172.21.15.26:0/3378265706 learned_addr learned my addr 172.21.15.26:0/3378265706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4de59c640 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4fae6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae80b0 0x555e4faf8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 -- 172.21.15.26:0/3378265706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 msgr2=0x555e4fae6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4fae6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 -- 172.21.15.26:0/3378265706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae80b0 msgr2=0x555e4faf8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae80b0 0x555e4faf8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 -- 172.21.15.26:0/3378265706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4faec680 con 0x555e4faed220 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4ded9d640 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faf1b80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff4d801efd0 tx=0x7ff4d80138e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb03fc99998999d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.993+0000 7ff4c7fff640 1 -- 172.21.15.26:0/3378265706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8036070 con 0x555e4faed220 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4c7fff640 1 -- 172.21.15.26:0/3378265706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d8013110 con 0x555e4faed220 2024-05-20T21:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4c7fff640 1 -- 172.21.15.26:0/3378265706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8022040 con 0x555e4faed220 2024-05-20T21:03:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 -- 172.21.15.26:0/3378265706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 msgr2=0x555e4faf1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faf1b80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff4d801efd0 tx=0x7ff4d80138e0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 -- 172.21.15.26:0/3378265706 shutdown_connections 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4faed220 0x555e4faf1b80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4fae80b0 0x555e4faf8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 --2- 172.21.15.26:0/3378265706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4fae6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 -- 172.21.15.26:0/3378265706 >> 172.21.15.26:0/3378265706 conn(0x555e4fae2d30 msgr2=0x555e4fae1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 -- 172.21.15.26:0/3378265706 shutdown_connections 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.994+0000 7ff4e7614740 1 -- 172.21.15.26:0/3378265706 wait complete. 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 -- start start 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 Processor -- start 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4faf6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 0x555e4faf6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01aa750 con 0x555e4fae80b0 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01abbc0 con 0x555e4faed220 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01aae20 con 0x555e4fae76c0 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4e7614740 10 client.0 _ll_register_callbacks cb 0x7ff4d0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 0x555e4faf6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4ded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 0x555e4faf6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44320/0 (socket says 172.21.15.26:44320) 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 -- 172.21.15.26:0/424597940 learned_addr learned my addr 172.21.15.26:0/424597940 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4de59c640 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4faf6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 -- 172.21.15.26:0/424597940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 msgr2=0x555e4faf6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4faf6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 -- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 msgr2=0x555e4faf7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.995+0000 7ff4d7fff640 1 -- 172.21.15.26:0/424597940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4faf7e10 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4d7fff640 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 0x555e4faf6880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000c00 tx=0x7ff4c8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a1cbb30c04e65d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/424597940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8049070 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/424597940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff4c803b4f0 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d8013110 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/424597940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4c80ca0d0 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 10 client.17694 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e4faf9190 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/424597940 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d8013110 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/424597940 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c80d2a80 con 0x555e4fae80b0 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 10 client.17694 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 10 client.17694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 20 client.17694 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fb689b0 0x555e4fb6add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 4 client.17694 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:03:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.996+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e4fa54360 con 0x555e4fb689b0 2024-05-20T21:03:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.997+0000 7ff4de59c640 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fb689b0 0x555e4fb6add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.997+0000 7ff4c5ffb640 10 client.17694 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:51.997+0000 7ff4de59c640 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fb689b0 0x555e4fb6add0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73eaf7ea3eca686f server_cookie=c3d787dee6d77b30 in_seq=0 out_seq=0 2024-05-20T21:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.034+0000 7ff4c5ffb640 1 -- 172.21.15.26:0/424597940 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555e4fa54360 con 0x555e4fb689b0 2024-05-20T21:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.034+0000 7ff4c5ffb640 10 client.17694 handle_command_reply: tid=0 2024-05-20T21:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.034+0000 7ff4e7614740 1 client.17694 shutdown 2024-05-20T21:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 msgr2=0x555e4faf6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 0x555e4faf6880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000c00 tx=0x7ff4c8005fd0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 shutdown_connections 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555e4fb689b0 0x555e4fb6add0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e4faed220 0x555e4faf7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e4fae80b0 0x555e4faf6880 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 --2- 172.21.15.26:0/424597940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e4fae76c0 0x555e4faf6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 >> 172.21.15.26:0/424597940 conn(0x555e4fae2d30 msgr2=0x555e4fae1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4c5ffb640 0 client.17694 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 shutdown_connections 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 1 -- 172.21.15.26:0/424597940 wait complete. 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e7614740 20 client.17694 trim_cache size 0 max 16384 2024-05-20T21:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:03:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:03:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:03:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4cc0b28a0 msgr2=0x7ff4cc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.035+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4cc0b28a0 0x7ff4cc0b4d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8023000 tx=0x7ff4d8021000 comp rx=0 tx=0).stop 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e0101d00 msgr2=0x7ff4e019b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e0101d00 0x7ff4e019b2a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4c802a9a0 tx=0x7ff4c8003c80 comp rx=0 tx=0).stop 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 shutdown_connections 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4cc0b28a0 0x7ff4cc0b4d60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4e01026a0 0x7ff4e01a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4e0101d00 0x7ff4e019b2a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 --2- 172.21.15.26:0/2692189397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e01013a0 0x7ff4e019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 >> 172.21.15.26:0/2692189397 conn(0x7ff4e00fcf20 msgr2=0x7ff4e0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 shutdown_connections 2024-05-20T21:03:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:03:52.036+0000 7ff4e4820640 1 -- 172.21.15.26:0/2692189397 wait complete. 2024-05-20T21:03:52.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 168012 2.31 MB/sec execute 238 sec latency 27.113 ms 2024-05-20T21:03:53.056 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:03:53.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:03:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:53 smithi055 ceph-mon[35213]: pgmap v622: 97 pgs: 97 active+clean; 28 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.1 MiB/s wr, 72 op/s 2024-05-20T21:03:53.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:53 smithi026 ceph-mon[31990]: pgmap v622: 97 pgs: 97 active+clean; 28 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.1 MiB/s wr, 72 op/s 2024-05-20T21:03:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:53 smithi144 ceph-mon[35660]: pgmap v622: 97 pgs: 97 active+clean; 28 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.1 MiB/s wr, 72 op/s 2024-05-20T21:03:53.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 168772 2.31 MB/sec execute 239 sec latency 12.916 ms 2024-05-20T21:03:54.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 169476 2.33 MB/sec execute 240 sec latency 15.795 ms 2024-05-20T21:03:55.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 -- 172.21.15.26:0/3965409136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 msgr2=0x7f8cbc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3965409136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc108900 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009990 tx=0x7f8cac009ca0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 -- 172.21.15.26:0/3965409136 shutdown_connections 2024-05-20T21:03:55.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3965409136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc108900 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3965409136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102e00 0x7f8cbc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3965409136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102440 0x7f8cbc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 -- 172.21.15.26:0/3965409136 >> 172.21.15.26:0/3965409136 conn(0x7f8cbc0793f0 msgr2=0x7f8cbc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 -- 172.21.15.26:0/3965409136 shutdown_connections 2024-05-20T21:03:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.193+0000 7f8cc2332700 1 -- 172.21.15.26:0/3965409136 wait complete. 2024-05-20T21:03:55.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 Processor -- start 2024-05-20T21:03:55.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 -- start start 2024-05-20T21:03:55.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102440 0x7f8cbc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102e00 0x7f8cbc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1124a0 con 0x7f8cbc108520 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112320 con 0x7f8cbc102440 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc2332700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112620 con 0x7f8cbc102e00 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc08cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc08cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44340/0 (socket says 172.21.15.26:44340) 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3239110790 learned_addr learned my addr 172.21.15.26:0/3239110790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.194+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102440 0x7f8cbc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102e00 0x7f8cbc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3239110790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102e00 msgr2=0x7f8cbc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102e00 0x7f8cbc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3239110790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102440 msgr2=0x7f8cbc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102440 0x7f8cbc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3239110790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc19a130 con 0x7f8cbc108520 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102440 0x7f8cbc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102e00 0x7f8cbc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc1999d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400ef30 tx=0x7f8cb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cb97fa700 1 -- 172.21.15.26:0/3239110790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb40092e0 con 0x7f8cbc108520 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cb97fa700 1 -- 172.21.15.26:0/3239110790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb4004bb0 con 0x7f8cbc108520 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc19a3c0 con 0x7f8cbc108520 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a6690 con 0x7f8cbc108520 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.195+0000 7f8cb97fa700 1 -- 172.21.15.26:0/3239110790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4004020 con 0x7f8cbc108520 2024-05-20T21:03:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.198+0000 7f8cb97fa700 1 -- 172.21.15.26:0/3239110790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8cb401b030 con 0x7f8cbc108520 2024-05-20T21:03:55.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.199+0000 7f8cb97fa700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ca40b2880 0x7f8ca40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.199+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ca40b2880 0x7f8ca40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.199+0000 7f8cb97fa700 1 -- 172.21.15.26:0/3239110790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8cb40e1d40 con 0x7f8cbc108520 2024-05-20T21:03:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.199+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ca40b2880 0x7f8ca40b4d40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00b2e0 tx=0x7f8cac01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.199+0000 7f8cccbc8b80 1 Processor -- start 2024-05-20T21:03:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 -- start start 2024-05-20T21:03:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f10887cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 0x555f1088b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f1088bfe0 0x555f1088c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555f107e6d10 con 0x555f10888290 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555f107d68b0 con 0x555f108878e0 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f107da8f0 con 0x555f1088bfe0 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 0x555f1088b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 0x555f1088b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44352/0 (socket says 172.21.15.26:44352) 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3516182446 learned_addr learned my addr 172.21.15.26:0/3516182446 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f10887cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f1088bfe0 0x555f1088c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3516182446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f1088bfe0 msgr2=0x555f1088c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f1088bfe0 0x555f1088c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3516182446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 msgr2=0x555f10887cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f10887cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3516182446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f10726950 con 0x555f10888290 2024-05-20T21:03:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.200+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 0x555f1088b9c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000bda0 tx=0x7f8cb000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c626a72d8ebbf765 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.201+0000 7f8ca27fc700 1 -- 172.21.15.26:0/3516182446 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0015070 con 0x555f10888290 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.201+0000 7f8ca27fc700 1 -- 172.21.15.26:0/3516182446 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb000fda0 con 0x555f10888290 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.201+0000 7f8ca27fc700 1 -- 172.21.15.26:0/3516182446 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00116a0 con 0x555f10888290 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.201+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3516182446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 msgr2=0x555f1088b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.201+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 0x555f1088b9c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000bda0 tx=0x7f8cb000f640 comp rx=0 tx=0).stop 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3516182446 shutdown_connections 2024-05-20T21:03:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f1088bfe0 0x555f1088c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f10887cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3516182446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f10888290 0x555f1088b9c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3516182446 >> 172.21.15.26:0/3516182446 conn(0x555f107b5100 msgr2=0x555f10881f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3516182446 shutdown_connections 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3516182446 wait complete. 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 -- start start 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.202+0000 7f8cccbc8b80 1 Processor -- start 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10893440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10890740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f10890c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555f106dd020 con 0x555f1088bfe0 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555f10778840 con 0x555f10888290 2024-05-20T21:03:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f108941a0 con 0x555f108878e0 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cccbc8b80 10 client.0 _ll_register_callbacks cb 0x555f107719a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cc08cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f10890c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10890740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10890740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36808/0 (socket says 172.21.15.26:36808) 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/1015154376 learned_addr learned my addr 172.21.15.26:0/1015154376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10893440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/1015154376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 msgr2=0x555f10893440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10893440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/1015154376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 msgr2=0x555f10890c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f10890c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/1015154376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f10775710 con 0x555f10888290 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.203+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f10890c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10890740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000ff90 tx=0x7f8cb001a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5525b183e6bd3a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8c9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/1015154376 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0028060 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/1015154376 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8cb001ae00 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb000fda0 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/1015154376 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cb0015070 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8cccbc8b80 10 client.27140 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f1089e880 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.204+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/1015154376 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb000fda0 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/1015154376 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8cb002c030 con 0x555f10888290 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cccbc8b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cccbc8b80 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cccbc8b80 20 client.27140 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f108a0d50 0x555f108a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cccbc8b80 4 client.27140 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f10853a60 con 0x555f108a0d50 2024-05-20T21:03:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.205+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f108a0d50 0x555f108a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.206+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f108a0d50 0x555f108a3140 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f297931db05e94 server_cookie=989a365886656728 in_seq=0 out_seq=0 2024-05-20T21:03:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.206+0000 7f8c9b7fe700 10 client.27140 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.206+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/1015154376 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555f10853a60 con 0x555f108a0d50 2024-05-20T21:03:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.206+0000 7f8c9b7fe700 10 client.27140 handle_command_reply: tid=0 2024-05-20T21:03:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.206+0000 7f8cccbc8b80 1 client.27140 shutdown 2024-05-20T21:03:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 msgr2=0x555f10890740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10890740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000ff90 tx=0x7f8cb001a690 comp rx=0 tx=0).stop 2024-05-20T21:03:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 shutdown_connections 2024-05-20T21:03:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f108a0d50 0x555f108a3140 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f10890c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10893440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/1015154376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10890740 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 >> 172.21.15.26:0/1015154376 conn(0x555f107b5100 msgr2=0x555f1088d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8c9b7fe700 0 client.27140 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 shutdown_connections 2024-05-20T21:03:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/1015154376 wait complete. 2024-05-20T21:03:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.207+0000 7f8cccbc8b80 20 client.27140 trim_cache size 0 max 16384 2024-05-20T21:03:55.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.231+0000 7f8cccbc8b80 1 Processor -- start 2024-05-20T21:03:55.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.231+0000 7f8cccbc8b80 1 -- start start 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.231+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f1088b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.231+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f10888290 0x555f1088b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.231+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ae430 con 0x555f1088bfe0 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1abcc0 con 0x555f108878e0 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1a9b10 con 0x555f10888290 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108a88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44370/0 (socket says 172.21.15.26:44370) 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3410107161 learned_addr learned my addr 172.21.15.26:0/3410107161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f1088b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f10888290 0x555f1088b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3410107161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f10888290 msgr2=0x555f1088b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f10888290 0x555f1088b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3410107161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 msgr2=0x555f1088b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f1088b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 -- 172.21.15.26:0/3410107161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1088ecb0 con 0x555f1088bfe0 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f1088b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f10888290 0x555f1088b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108a88d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40dfb50 tx=0x7f8cb40a98d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f07ad85e60522cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8ca1ffb700 1 -- 172.21.15.26:0/3410107161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb40a9a90 con 0x555f1088bfe0 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8ca1ffb700 1 -- 172.21.15.26:0/3410107161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb40a9390 con 0x555f1088bfe0 2024-05-20T21:03:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.232+0000 7f8ca1ffb700 1 -- 172.21.15.26:0/3410107161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb40a4d00 con 0x555f1088bfe0 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3410107161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 msgr2=0x555f108a88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108a88d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40dfb50 tx=0x7f8cb40a98d0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3410107161 shutdown_connections 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108a88d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f108878e0 0x555f1088b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3410107161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f10888290 0x555f1088b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3410107161 >> 172.21.15.26:0/3410107161 conn(0x555f107b5100 msgr2=0x555f1088ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3410107161 shutdown_connections 2024-05-20T21:03:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.233+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3410107161 wait complete. 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 -- start start 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 Processor -- start 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10899ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10896e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1aa180 con 0x555f1088bfe0 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ab5f0 con 0x555f10888290 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1aa850 con 0x555f108878e0 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cccbc8b80 10 client.0 _ll_register_callbacks cb 0x555f107719a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10896e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.234+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10896e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36834/0 (socket says 172.21.15.26:36834) 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3378771560 learned_addr learned my addr 172.21.15.26:0/3378771560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cc08cf700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10899ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3378771560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 msgr2=0x555f10899ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10899ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3378771560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 msgr2=0x555f108973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 -- 172.21.15.26:0/3378771560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f10897be0 con 0x555f10888290 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cbb7fe700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10896e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000f680 tx=0x7f8cb0004dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2679accb62cbc83 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8c9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/3378771560 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0028070 con 0x555f10888290 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/3378771560 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8cb0004710 con 0x555f10888290 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb40a9390 con 0x555f10888290 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/3378771560 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cb000fb40 con 0x555f10888290 2024-05-20T21:03:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cccbc8b80 10 client.27146 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.235+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f1090b780 con 0x555f10888290 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/3378771560 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb40a9390 con 0x555f10888290 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/3378771560 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8cb002c040 con 0x555f10888290 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cccbc8b80 10 client.27146 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cccbc8b80 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cccbc8b80 20 client.27146 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1090be60 0x555f1090e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cccbc8b80 4 client.27146 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f10892e60 con 0x555f1090be60 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1090be60 0x555f1090e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.237+0000 7f8c9b7fe700 10 client.27146 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.236+0000 7f8cbbfff700 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1090be60 0x555f1090e280 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2809f4332ad75c1c server_cookie=865087490883cb6b in_seq=0 out_seq=0 2024-05-20T21:03:55.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.237+0000 7f8c9b7fe700 1 -- 172.21.15.26:0/3378771560 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f10892e60 con 0x555f1090be60 2024-05-20T21:03:55.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.237+0000 7f8c9b7fe700 10 client.27146 handle_command_reply: tid=0 2024-05-20T21:03:55.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.237+0000 7f8cccbc8b80 1 client.27146 shutdown 2024-05-20T21:03:55.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 msgr2=0x555f10896e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10896e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000f680 tx=0x7f8cb0004dd0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 shutdown_connections 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1090be60 0x555f1090e280 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f1088bfe0 0x555f108973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f108878e0 0x555f10899ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8c9b7fe700 0 client.27146 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 --2- 172.21.15.26:0/3378771560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10888290 0x555f10896e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 >> 172.21.15.26:0/3378771560 conn(0x555f107b5100 msgr2=0x555f1088b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 shutdown_connections 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 1 -- 172.21.15.26:0/3378771560 wait complete. 2024-05-20T21:03:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.238+0000 7f8cccbc8b80 20 client.27146 trim_cache size 0 max 16384 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "5b4c8e78-a92e-4663-b2ee-f0d97ac7fbc9", 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ca40b2880 msgr2=0x7f8ca40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ca40b2880 0x7f8ca40b4d40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00b2e0 tx=0x7f8cac01a040 comp rx=0 tx=0).stop 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 msgr2=0x7f8cbc1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc1999d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400ef30 tx=0x7f8cb400c3a0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 shutdown_connections 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ca40b2880 0x7f8ca40b4d40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cbc108520 0x7f8cbc1999d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cbc102440 0x7f8cbc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 --2- 172.21.15.26:0/3239110790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cbc102e00 0x7f8cbc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 >> 172.21.15.26:0/3239110790 conn(0x7f8cbc0793f0 msgr2=0x7f8cbc06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 shutdown_connections 2024-05-20T21:03:55.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:55.239+0000 7f8cc2332700 1 -- 172.21.15.26:0/3239110790 wait complete. 2024-05-20T21:03:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:55 smithi055 ceph-mon[35213]: pgmap v623: 97 pgs: 97 active+clean; 27 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-05-20T21:03:55.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:55 smithi026 ceph-mon[31990]: pgmap v623: 97 pgs: 97 active+clean; 27 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-05-20T21:03:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:55 smithi144 ceph-mon[35660]: pgmap v623: 97 pgs: 97 active+clean; 27 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-05-20T21:03:55.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 170093 2.33 MB/sec execute 241 sec latency 15.678 ms 2024-05-20T21:03:56.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:03:56.345 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:03:56.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 170700 2.34 MB/sec execute 242 sec latency 15.533 ms 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:57 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:57 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:57 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:57 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:57 smithi055 ceph-mon[35213]: pgmap v624: 97 pgs: 97 active+clean; 28 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:57 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:03:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:57 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:03:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:57 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:57 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:03:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:57 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:57 smithi026 ceph-mon[31990]: pgmap v624: 97 pgs: 97 active+clean; 28 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-20T21:03:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:57 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:03:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:57 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:03:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:57 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:57 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:03:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:57 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:03:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:57 smithi144 ceph-mon[35660]: pgmap v624: 97 pgs: 97 active+clean; 28 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-20T21:03:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:57 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:03:57.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 171224 2.33 MB/sec execute 243 sec latency 18.018 ms 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 -- 172.21.15.26:0/4037228686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428102e00 msgr2=0x7fd42810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/4037228686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428102e00 0x7fd42810f2f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b0a0 tx=0x7fd42000b3b0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 -- 172.21.15.26:0/4037228686 shutdown_connections 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/4037228686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428102e00 0x7fd42810f2f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/4037228686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428108520 0x7fd428108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/4037228686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd428102440 0x7fd4281028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 -- 172.21.15.26:0/4037228686 >> 172.21.15.26:0/4037228686 conn(0x7fd4280793f0 msgr2=0x7fd428079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 -- 172.21.15.26:0/4037228686 shutdown_connections 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.252+0000 7fd42f1e6700 1 -- 172.21.15.26:0/4037228686 wait complete. 2024-05-20T21:03:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.253+0000 7fd42f1e6700 1 Processor -- start 2024-05-20T21:03:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.253+0000 7fd42f1e6700 1 -- start start 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.253+0000 7fd42f1e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd428102440 0x7fd42819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.253+0000 7fd42f1e6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 0x7fd42819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.253+0000 7fd42f1e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 0x7fd4281a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42f1e6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd428112480 con 0x7fd428108520 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42f1e6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd428112300 con 0x7fd428102e00 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42f1e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd428112600 con 0x7fd428102440 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 0x7fd4281a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd427fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 0x7fd42819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42cf82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd428102440 0x7fd42819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd427fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 0x7fd42819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36860/0 (socket says 172.21.15.26:36860) 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 0x7fd4281a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44388/0 (socket says 172.21.15.26:44388) 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd427fff700 1 -- 172.21.15.26:0/2828464007 learned_addr learned my addr 172.21.15.26:0/2828464007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 -- 172.21.15.26:0/2828464007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd428102440 msgr2=0x7fd42819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd428102440 0x7fd42819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 -- 172.21.15.26:0/2828464007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 msgr2=0x7fd42819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 0x7fd42819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 -- 172.21.15.26:0/2828464007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4281a0d20 con 0x7fd428108520 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd427fff700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 0x7fd42819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd42d783700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 0x7fd4281a05e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b0a0 tx=0x7fd420004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.254+0000 7fd425ffb700 1 -- 172.21.15.26:0/2828464007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420021070 con 0x7fd428108520 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.255+0000 7fd425ffb700 1 -- 172.21.15.26:0/2828464007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd420007b80 con 0x7fd428108520 2024-05-20T21:03:58.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.255+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4281a0fb0 con 0x7fd428108520 2024-05-20T21:03:58.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.255+0000 7fd425ffb700 1 -- 172.21.15.26:0/2828464007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420005040 con 0x7fd428108520 2024-05-20T21:03:58.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.255+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4281a8890 con 0x7fd428108520 2024-05-20T21:03:58.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.258+0000 7fd425ffb700 1 -- 172.21.15.26:0/2828464007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd420024070 con 0x7fd428108520 2024-05-20T21:03:58.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.258+0000 7fd425ffb700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4180b2900 0x7fd4180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.258+0000 7fd439a7cb80 1 Processor -- start 2024-05-20T21:03:58.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 -- start start 2024-05-20T21:03:58.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd42cf82700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4180b2900 0x7fd4180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd425ffb700 1 -- 172.21.15.26:0/2828464007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd42000f020 con 0x7fd428108520 2024-05-20T21:03:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd3fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a96dc94d70 con 0x55a96dd44a70 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a96dc84910 con 0x55a96dd45420 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a96dc88950 con 0x55a96dd3f6d0 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd42cf82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd427fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd42cf82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd44e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44398/0 (socket says 172.21.15.26:44398) 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd427fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd3f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36862/0 (socket says 172.21.15.26:36862) 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd42cf82700 1 -- 172.21.15.26:0/4215284575 learned_addr learned my addr 172.21.15.26:0/4215284575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd42d783700 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd3fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.259+0000 7fd42cf82700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4180b2900 0x7fd4180b4dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009c30 tx=0x7fd41c011440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:58.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd427fff700 1 -- 172.21.15.26:0/4215284575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 msgr2=0x55a96dd3fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd427fff700 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd3fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd427fff700 1 -- 172.21.15.26:0/4215284575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 msgr2=0x55a96dd44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd427fff700 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd44e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd427fff700 1 -- 172.21.15.26:0/4215284575 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a96dc2dfb0 con 0x55a96dd45420 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd42cf82700 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd427fff700 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd3f0b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd414005950 tx=0x7fd41400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664bdf266520a428 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd412ffd700 1 -- 172.21.15.26:0/4215284575 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414011640 con 0x55a96dd45420 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd412ffd700 1 -- 172.21.15.26:0/4215284575 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd414011c70 con 0x55a96dd45420 2024-05-20T21:03:58.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.260+0000 7fd412ffd700 1 -- 172.21.15.26:0/4215284575 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414019bf0 con 0x55a96dd45420 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 -- 172.21.15.26:0/4215284575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 msgr2=0x55a96dd3f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd3f0b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd414005950 tx=0x7fd41400bf90 comp rx=0 tx=0).stop 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 -- 172.21.15.26:0/4215284575 shutdown_connections 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd3fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/4215284575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd3f0b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 -- 172.21.15.26:0/4215284575 >> 172.21.15.26:0/4215284575 conn(0x55a96dc62240 msgr2=0x55a96dd313e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 -- 172.21.15.26:0/4215284575 shutdown_connections 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 -- 172.21.15.26:0/4215284575 wait complete. 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.261+0000 7fd439a7cb80 1 -- start start 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 Processor -- start 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 0x55a96dd4b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a96db8afa0 con 0x55a96dd3f6d0 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a96dbecda0 con 0x55a96dd45420 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a96dd4bea0 con 0x55a96dd44a70 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd42cf82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 0x55a96dd4b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd439a7cb80 10 client.0 _ll_register_callbacks cb 0x55a96dc1f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd42cf82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 0x55a96dd4b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44408/0 (socket says 172.21.15.26:44408) 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd42cf82700 1 -- 172.21.15.26:0/2294457913 learned_addr learned my addr 172.21.15.26:0/2294457913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd42d783700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.262+0000 7fd427fff700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42cf82700 1 -- 172.21.15.26:0/2294457913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 msgr2=0x55a96dd484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42cf82700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42cf82700 1 -- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 msgr2=0x55a96dd48a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42cf82700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd48a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42cf82700 1 -- 172.21.15.26:0/2294457913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a96dbd48d0 con 0x55a96dd3f6d0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42d783700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd42cf82700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 0x55a96dd4b1e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd41c01e9b0 tx=0x7fd41c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3bc13c7d289212 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd407fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd407fff700 1 -- 172.21.15.26:0/2294457913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c017040 con 0x55a96dd3f6d0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd407fff700 1 -- 172.21.15.26:0/2294457913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd41c004a30 con 0x55a96dd3f6d0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd414011c70 con 0x55a96dd3f6d0 2024-05-20T21:03:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd407fff700 1 -- 172.21.15.26:0/2294457913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd41c004c70 con 0x55a96dd3f6d0 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd439a7cb80 10 client.17724 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a96dc3e760 con 0x55a96dd3f6d0 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.263+0000 7fd407fff700 1 -- 172.21.15.26:0/2294457913 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd414011c70 con 0x55a96dd3f6d0 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd407fff700 1 -- 172.21.15.26:0/2294457913 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fd41c030020 con 0x55a96dd3f6d0 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd439a7cb80 10 client.17724 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd439a7cb80 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd439a7cb80 20 client.17724 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96dd5ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd439a7cb80 4 client.17724 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a96dc96cc0 con 0x55a96dd58a90 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd427fff700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96dd5ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd427fff700 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96dd5ae80 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a7e7d8582ee21a server_cookie=8256b7cf8d22a1b8 in_seq=0 out_seq=0 2024-05-20T21:03:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.264+0000 7fd407fff700 10 client.17724 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd407fff700 1 -- 172.21.15.26:0/2294457913 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a96dc96cc0 con 0x55a96dd58a90 2024-05-20T21:03:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd407fff700 10 client.17724 handle_command_reply: tid=0 2024-05-20T21:03:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 client.17724 shutdown 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 msgr2=0x55a96dd4b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 0x55a96dd4b1e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd41c01e9b0 tx=0x7fd41c0047c0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 shutdown_connections 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96dd5ae80 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd3f6d0 0x55a96dd4b1e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd407fff700 0 client.17724 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/2294457913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 >> 172.21.15.26:0/2294457913 conn(0x55a96dc62240 msgr2=0x55a96dd30af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 shutdown_connections 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 1 -- 172.21.15.26:0/2294457913 wait complete. 2024-05-20T21:03:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.266+0000 7fd439a7cb80 20 client.17724 trim_cache size 0 max 16384 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 Processor -- start 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 -- start start 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd45ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd31270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281ae250 con 0x55a96dd44a70 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281abae0 con 0x55a96dd45420 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.289+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd42810d2f0 con 0x55a96dd3f6d0 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd45ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd45ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44422/0 (socket says 172.21.15.26:44422) 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 -- 172.21.15.26:0/729777097 learned_addr learned my addr 172.21.15.26:0/729777097 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:58.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd42d783700 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd31270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd42cf82700 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 -- 172.21.15.26:0/729777097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 msgr2=0x55a96dd474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 -- 172.21.15.26:0/729777097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 msgr2=0x55a96dd31270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd31270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 -- 172.21.15.26:0/729777097 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a96dd32cd0 con 0x55a96dd44a70 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd42d783700 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd31270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd427fff700 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd45ec0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55a96dd32ea0 tx=0x7fd41400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3eff35f0d136eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd4127fc700 1 -- 172.21.15.26:0/729777097 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41402b040 con 0x55a96dd44a70 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd4127fc700 1 -- 172.21.15.26:0/729777097 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd414019cd0 con 0x55a96dd44a70 2024-05-20T21:03:58.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.290+0000 7fd4127fc700 1 -- 172.21.15.26:0/729777097 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41401f040 con 0x55a96dd44a70 2024-05-20T21:03:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 -- 172.21.15.26:0/729777097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 msgr2=0x55a96dd45ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd45ec0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55a96dd32ea0 tx=0x7fd41400b500 comp rx=0 tx=0).stop 2024-05-20T21:03:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 -- 172.21.15.26:0/729777097 shutdown_connections 2024-05-20T21:03:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd45420 0x55a96dd31270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd3f6d0 0x55a96dd474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/729777097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd44a70 0x55a96dd45ec0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 -- 172.21.15.26:0/729777097 >> 172.21.15.26:0/729777097 conn(0x55a96dc62240 msgr2=0x55a96dd36120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 -- 172.21.15.26:0/729777097 shutdown_connections 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.291+0000 7fd439a7cb80 1 -- 172.21.15.26:0/729777097 wait complete. 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 -- start start 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 Processor -- start 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd3f6d0 0x55a96dd51130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd51670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd45420 0x55a96dd4e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281a9fa0 con 0x55a96dd45420 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281ab410 con 0x55a96dd3f6d0 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281aa670 con 0x55a96dd44a70 2024-05-20T21:03:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd439a7cb80 10 client.0 _ll_register_callbacks cb 0x55a96dc1f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.292+0000 7fd42d783700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd45420 0x55a96dd4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42d783700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd45420 0x55a96dd4e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44436/0 (socket says 172.21.15.26:44436) 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42d783700 1 -- 172.21.15.26:0/3621455929 learned_addr learned my addr 172.21.15.26:0/3621455929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd3f6d0 0x55a96dd51130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd427fff700 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd51670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 -- 172.21.15.26:0/3621455929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 msgr2=0x55a96dd51670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd51670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 -- 172.21.15.26:0/3621455929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd45420 msgr2=0x55a96dd4e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd45420 0x55a96dd4e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 -- 172.21.15.26:0/3621455929 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a96dd4e6c0 con 0x55a96dd3f6d0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd42cf82700 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd3f6d0 0x55a96dd51130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0092a0 tx=0x7fd41c004ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdea310e3cd9aba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd407fff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:03:58.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd407fff700 1 -- 172.21.15.26:0/3621455929 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c036020 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd407fff700 1 -- 172.21.15.26:0/3621455929 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd41c005010 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.293+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd414019cd0 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd407fff700 1 -- 172.21.15.26:0/3621455929 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd41c017040 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 10 client.27164 fetch_fsmap learned FSMap version 30 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a96ddc3f20 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd407fff700 1 -- 172.21.15.26:0/3621455929 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd414019cd0 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd407fff700 1 -- 172.21.15.26:0/3621455929 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fd41c030050 con 0x55a96dd3f6d0 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 10 client.27164 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 20 client.27164 populate_metadata read hostname 'smithi026' 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96ddc6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:03:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 4 client.27164 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a96dc904a0 con 0x55a96dd58a90 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.294+0000 7fd427fff700 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96ddc6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd407fff700 10 client.27164 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd427fff700 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96ddc6610 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f561f3360dd23a7 server_cookie=2954a7b5444ac703 in_seq=0 out_seq=0 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd407fff700 1 -- 172.21.15.26:0/3621455929 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a96dc904a0 con 0x55a96dd58a90 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd407fff700 10 client.27164 handle_command_reply: tid=0 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 client.27164 shutdown 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd3f6d0 msgr2=0x55a96dd51130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd3f6d0 0x55a96dd51130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0092a0 tx=0x7fd41c004ae0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 shutdown_connections 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a96dd58a90 0x55a96ddc6610 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd407fff700 0 client.27164 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:03:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a96dd45420 0x55a96dd4e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a96dd3f6d0 0x55a96dd51130 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 --2- 172.21.15.26:0/3621455929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a96dd44a70 0x55a96dd51670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 >> 172.21.15.26:0/3621455929 conn(0x55a96dc62240 msgr2=0x55a96dd36120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 shutdown_connections 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.295+0000 7fd439a7cb80 1 -- 172.21.15.26:0/3621455929 wait complete. 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd439a7cb80 20 client.27164 trim_cache size 0 max 16384 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:03:58.300 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:03:58.301 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:03:58.301 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:03:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4180b2900 msgr2=0x7fd4180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4180b2900 0x7fd4180b4dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009c30 tx=0x7fd41c011440 comp rx=0 tx=0).stop 2024-05-20T21:03:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 msgr2=0x7fd4281a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:03:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 0x7fd4281a05e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b0a0 tx=0x7fd420004d00 comp rx=0 tx=0).stop 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 shutdown_connections 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4180b2900 0x7fd4180b4dc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd428108520 0x7fd4281a05e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd428102440 0x7fd42819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 --2- 172.21.15.26:0/2828464007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd428102e00 0x7fd42819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 >> 172.21.15.26:0/2828464007 conn(0x7fd4280793f0 msgr2=0x7fd42806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.296+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 shutdown_connections 2024-05-20T21:03:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:03:58.297+0000 7fd42f1e6700 1 -- 172.21.15.26:0/2828464007 wait complete. 2024-05-20T21:03:58.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 171911 2.34 MB/sec execute 244 sec latency 24.980 ms 2024-05-20T21:03:59.351 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:03:59.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:03:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:03:59 smithi055 ceph-mon[35213]: pgmap v625: 97 pgs: 97 active+clean; 28 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T21:03:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:03:59 smithi026 ceph-mon[31990]: pgmap v625: 97 pgs: 97 active+clean; 28 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T21:03:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:03:59 smithi144 ceph-mon[35660]: pgmap v625: 97 pgs: 97 active+clean; 28 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T21:03:59.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 172609 2.34 MB/sec execute 245 sec latency 44.904 ms 2024-05-20T21:03:59.635 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:00.129 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-20T21:04:00.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:04:00.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:00 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:04:00.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:00 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:04:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:00 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:04:00.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 173295 2.35 MB/sec execute 246 sec latency 19.767 ms 2024-05-20T21:04:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:01.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:01 smithi026 ceph-mon[31990]: pgmap v626: 97 pgs: 97 active+clean; 28 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-20T21:04:01.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:01 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:04:01.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:01 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:04:01.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:01 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:04:01.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:01 smithi055 ceph-mon[35213]: pgmap v626: 97 pgs: 97 active+clean; 28 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-20T21:04:01.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:01 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:04:01.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:01 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:04:01.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:01 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:04:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 -- 172.21.15.26:0/1401615989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81041a0 msgr2=0x7f3ee81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 --2- 172.21.15.26:0/1401615989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81041a0 0x7f3ee81081c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80099c0 tx=0x7f3ed8009cd0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 -- 172.21.15.26:0/1401615989 shutdown_connections 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 --2- 172.21.15.26:0/1401615989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee81088f0 0x7f3ee8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 --2- 172.21.15.26:0/1401615989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81037f0 0x7f3ee8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 --2- 172.21.15.26:0/1401615989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81041a0 0x7f3ee81081c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 -- 172.21.15.26:0/1401615989 >> 172.21.15.26:0/1401615989 conn(0x7f3ee807a6a0 msgr2=0x7f3ee807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 -- 172.21.15.26:0/1401615989 shutdown_connections 2024-05-20T21:04:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.494+0000 7f3ef079e700 1 -- 172.21.15.26:0/1401615989 wait complete. 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 Processor -- start 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 -- start start 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81037f0 0x7f3ee819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee81041a0 0x7f3ee819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 0x7f3ee81998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8113580 con 0x7f3ee81037f0 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8113400 con 0x7f3ee81041a0 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3ef079e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8113700 con 0x7f3ee81088f0 2024-05-20T21:04:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 0x7f3ee81998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.495+0000 7f3eedd39700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee81041a0 0x7f3ee819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 0x7f3ee81998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:33168/0 (socket says 172.21.15.26:33168) 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 -- 172.21.15.26:0/920031854 learned_addr learned my addr 172.21.15.26:0/920031854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eee53a700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81037f0 0x7f3ee819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 -- 172.21.15.26:0/920031854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee81041a0 msgr2=0x7f3ee819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee81041a0 0x7f3ee819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 -- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81037f0 msgr2=0x7f3ee819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81037f0 0x7f3ee819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 -- 172.21.15.26:0/920031854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee819a020 con 0x7f3ee81088f0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3eeed3b700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 0x7f3ee81998c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000ef30 tx=0x7f3ee000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00092e0 con 0x7f3ee81088f0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ee0004bb0 con 0x7f3ee81088f0 2024-05-20T21:04:01.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee819a2b0 con 0x7f3ee81088f0 2024-05-20T21:04:01.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.496+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0004020 con 0x7f3ee81088f0 2024-05-20T21:04:01.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.497+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee81a65d0 con 0x7f3ee81088f0 2024-05-20T21:04:01.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.498+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee810bac0 con 0x7f3ee81088f0 2024-05-20T21:04:01.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.499+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3ee001b030 con 0x7f3ee81088f0 2024-05-20T21:04:01.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.500+0000 7f3edf7fe700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ed40b2880 0x7f3ed40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:01.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.500+0000 7f3eee53a700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ed40b2880 0x7f3ed40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:01.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.500+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3ee00e1f00 con 0x7f3ee81088f0 2024-05-20T21:04:01.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.500+0000 7f3eee53a700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ed40b2880 0x7f3ed40b4d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ee819aa00 tx=0x7f3ee4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:01.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.503+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ee0022020 con 0x7f3ee81088f0 2024-05-20T21:04:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:01 smithi144 ceph-mon[35660]: pgmap v626: 97 pgs: 97 active+clean; 28 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-20T21:04:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:01 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:04:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:01 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:04:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:01 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:04:01.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 173906 2.37 MB/sec execute 247 sec latency 16.630 ms 2024-05-20T21:04:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.819+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ee819adf0 con 0x7f3ee81088f0 2024-05-20T21:04:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.819+0000 7f3edf7fe700 1 -- 172.21.15.26:0/920031854 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f3ee0010040 con 0x7f3ee81088f0 2024-05-20T21:04:01.819 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:01.819 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:01.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ed40b2880 msgr2=0x7f3ed40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:01.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ed40b2880 0x7f3ed40b4d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ee819aa00 tx=0x7f3ee4009260 comp rx=0 tx=0).stop 2024-05-20T21:04:01.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 msgr2=0x7f3ee81998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:01.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 0x7f3ee81998c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000ef30 tx=0x7f3ee000c3a0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 shutdown_connections 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ed40b2880 0x7f3ed40b4d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee81037f0 0x7f3ee819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee81088f0 0x7f3ee81998c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 --2- 172.21.15.26:0/920031854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee81041a0 0x7f3ee819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 >> 172.21.15.26:0/920031854 conn(0x7f3ee807a6a0 msgr2=0x7f3ee8101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 shutdown_connections 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:01.822+0000 7f3ef079e700 1 -- 172.21.15.26:0/920031854 wait complete. 2024-05-20T21:04:01.822 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:04:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:02 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/920031854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:02.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:02 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/920031854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:02.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:02 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/920031854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:02.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 174328 2.36 MB/sec execute 248 sec latency 36.169 ms 2024-05-20T21:04:03.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:03 smithi055 ceph-mon[35213]: pgmap v627: 97 pgs: 97 active+clean; 28 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-05-20T21:04:03.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:03 smithi026 ceph-mon[31990]: pgmap v627: 97 pgs: 97 active+clean; 28 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-05-20T21:04:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:03 smithi144 ceph-mon[35660]: pgmap v627: 97 pgs: 97 active+clean; 28 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-05-20T21:04:03.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 174977 2.36 MB/sec execute 249 sec latency 23.090 ms 2024-05-20T21:04:03.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 -- 172.21.15.26:0/2660601723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 msgr2=0x7f7af8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/2660601723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af8108900 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009990 tx=0x7f7ae8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 -- 172.21.15.26:0/2660601723 shutdown_connections 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/2660601723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af8108900 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/2660601723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 0x7f7af810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/2660601723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af8102440 0x7f7af81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 -- 172.21.15.26:0/2660601723 >> 172.21.15.26:0/2660601723 conn(0x7f7af80793f0 msgr2=0x7f7af8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:04.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 -- 172.21.15.26:0/2660601723 shutdown_connections 2024-05-20T21:04:04.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.145+0000 7f7afe1e2700 1 -- 172.21.15.26:0/2660601723 wait complete. 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 Processor -- start 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 -- start start 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af8102440 0x7f7af819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 0x7f7af819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af81a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8112480 con 0x7f7af8108520 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8112300 con 0x7f7af8102e00 2024-05-20T21:04:04.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7afe1e2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8112600 con 0x7f7af8102440 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7af7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af81a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7af6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 0x7f7af819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7af7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af81a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59396/0 (socket says 172.21.15.26:59396) 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.146+0000 7f7af7fff700 1 -- 172.21.15.26:0/3197807420 learned_addr learned my addr 172.21.15.26:0/3197807420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af77fe700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af8102440 0x7f7af819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af6ffd700 1 -- 172.21.15.26:0/3197807420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af8102440 msgr2=0x7f7af819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af6ffd700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af8102440 0x7f7af819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af6ffd700 1 -- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 msgr2=0x7f7af81a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af6ffd700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af81a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af6ffd700 1 -- 172.21.15.26:0/3197807420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af81a5fa0 con 0x7f7af8102e00 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af6ffd700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 0x7f7af819af20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00d750 tx=0x7f7aec00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec016070 con 0x7f7af8102e00 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af81a61d0 con 0x7f7af8102e00 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7aec004830 con 0x7f7af8102e00 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005500 con 0x7f7af8102e00 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.147+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af81a6730 con 0x7f7af8102e00 2024-05-20T21:04:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.149+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af810a9b0 con 0x7f7af8102e00 2024-05-20T21:04:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.150+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7aec004220 con 0x7f7af8102e00 2024-05-20T21:04:04.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.151+0000 7f7af4ff9700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ae00b2980 0x7f7ae00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:04.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.151+0000 7f7af77fe700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ae00b2980 0x7f7ae00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:04.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.151+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7aec0e1eb0 con 0x7f7af8102e00 2024-05-20T21:04:04.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.152+0000 7f7af77fe700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ae00b2980 0x7f7ae00b4e40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800b2e0 tx=0x7f7ae801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.156+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7aec01f020 con 0x7f7af8102e00 2024-05-20T21:04:04.181 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:04.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.478+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7af806a230 con 0x7f7af8102e00 2024-05-20T21:04:04.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.479+0000 7f7af4ff9700 1 -- 172.21.15.26:0/3197807420 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f7aec0a9b10 con 0x7f7af8102e00 2024-05-20T21:04:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:04.481 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:04.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.481+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ae00b2980 msgr2=0x7f7ae00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:04.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.481+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ae00b2980 0x7f7ae00b4e40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800b2e0 tx=0x7f7ae801a040 comp rx=0 tx=0).stop 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.481+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 msgr2=0x7f7af819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.481+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 0x7f7af819af20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00d750 tx=0x7f7aec00da60 comp rx=0 tx=0).stop 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 shutdown_connections 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ae00b2980 0x7f7ae00b4e40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7af8108520 0x7f7af81a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af8102440 0x7f7af819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 --2- 172.21.15.26:0/3197807420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af8102e00 0x7f7af819af20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 >> 172.21.15.26:0/3197807420 conn(0x7f7af80793f0 msgr2=0x7f7af8101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 shutdown_connections 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:04.482+0000 7f7afe1e2700 1 -- 172.21.15.26:0/3197807420 wait complete. 2024-05-20T21:04:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:04:04.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 175653 2.36 MB/sec execute 250 sec latency 24.129 ms 2024-05-20T21:04:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 -- 172.21.15.26:0/3145680017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c1019b0 msgr2=0x7f168c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 --2- 172.21.15.26:0/3145680017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c1019b0 0x7f168c101dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1680009900 tx=0x7f168002a410 comp rx=0 tx=0).stop 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 -- 172.21.15.26:0/3145680017 shutdown_connections 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 --2- 172.21.15.26:0/3145680017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c103560 0x7f168c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 --2- 172.21.15.26:0/3145680017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c102ba0 0x7f168c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 --2- 172.21.15.26:0/3145680017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c1019b0 0x7f168c101dd0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.134+0000 7f169302a640 1 -- 172.21.15.26:0/3145680017 >> 172.21.15.26:0/3145680017 conn(0x7f168c0fcf20 msgr2=0x7f168c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 -- 172.21.15.26:0/3145680017 shutdown_connections 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 -- 172.21.15.26:0/3145680017 wait complete. 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 Processor -- start 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 -- start start 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 0x7f168c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c102ba0 0x7f168c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c103560 0x7f168c1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f168c10d3b0 con 0x7f168c102ba0 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f168c10d230 con 0x7f168c1019b0 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f169302a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f168c10d530 con 0x7f168c103560 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f168bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c102ba0 0x7f168c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 0x7f168c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 0x7f168c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46656/0 (socket says 172.21.15.26:46656) 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 -- 172.21.15.26:0/1075774716 learned_addr learned my addr 172.21.15.26:0/1075774716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.135+0000 7f168bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c102ba0 0x7f168c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59416/0 (socket says 172.21.15.26:59416) 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f16915c7640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c103560 0x7f168c1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 -- 172.21.15.26:0/1075774716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c103560 msgr2=0x7f168c1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c103560 0x7f168c1a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 -- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c102ba0 msgr2=0x7f168c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c102ba0 0x7f168c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 -- 172.21.15.26:0/1075774716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c1a07f0 con 0x7f168c1019b0 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f16915c7640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c103560 0x7f168c1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1690dc6640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 0x7f168c198bd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1680031460 tx=0x7f1680003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1689ffb640 1 -- 172.21.15.26:0/1075774716 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168003f070 con 0x7f168c1019b0 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1689ffb640 1 -- 172.21.15.26:0/1075774716 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1680031ba0 con 0x7f168c1019b0 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c1a0a80 con 0x7f168c1019b0 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.136+0000 7f1689ffb640 1 -- 172.21.15.26:0/1075774716 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680035030 con 0x7f168c1019b0 2024-05-20T21:04:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.137+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c1a0f70 con 0x7f168c1019b0 2024-05-20T21:04:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.139+0000 7f1689ffb640 1 -- 172.21.15.26:0/1075774716 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1680042070 con 0x7f168c1019b0 2024-05-20T21:04:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.139+0000 7f1695e29740 1 Processor -- start 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.139+0000 7f1695e29740 1 -- start start 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1695e29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291cf6c0 0x5655291cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1695e29740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1695e29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291d76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1695e29740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565529140c30 con 0x5655291d00b0 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1695e29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5655291307d0 con 0x5655291d5220 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1695e29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565529134810 con 0x5655291cf6c0 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1689ffb640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f167c0b28a0 0x7f167c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59428/0 (socket says 172.21.15.26:59428) 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1690dc6640 1 -- 172.21.15.26:0/1282674441 learned_addr learned my addr 172.21.15.26:0/1282674441 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291d76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f16915c7640 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291cf6c0 0x5655291cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f16915c7640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f167c0b28a0 0x7f167c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 -- 172.21.15.26:0/1282674441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291cf6c0 msgr2=0x5655291cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291cf6c0 0x5655291cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 -- 172.21.15.26:0/1282674441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 msgr2=0x5655291d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 -- 172.21.15.26:0/1282674441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565528e45210 con 0x5655291d5220 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1689ffb640 1 -- 172.21.15.26:0/1075774716 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f16801029d0 con 0x7f168c1019b0 2024-05-20T21:04:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f168bfff640 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291d76b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1678009bf0 tx=0x7f1678011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7fdb5fe837ba794 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f16915c7640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f167c0b28a0 0x7f167c0b4d60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5655291d0910 tx=0x7f1684009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.140+0000 7f1676ffd640 1 -- 172.21.15.26:0/1282674441 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678004050 con 0x5655291d5220 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1676ffd640 1 -- 172.21.15.26:0/1282674441 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1678005020 con 0x5655291d5220 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1676ffd640 1 -- 172.21.15.26:0/1282674441 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678004ba0 con 0x5655291d5220 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 -- 172.21.15.26:0/1282674441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 msgr2=0x5655291d76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291d76b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1678009bf0 tx=0x7f1678011810 comp rx=0 tx=0).stop 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 -- 172.21.15.26:0/1282674441 shutdown_connections 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291d76b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 --2- 172.21.15.26:0/1282674441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291cf6c0 0x5655291cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 -- 172.21.15.26:0/1282674441 >> 172.21.15.26:0/1282674441 conn(0x5655291cad30 msgr2=0x5655291cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 -- 172.21.15.26:0/1282674441 shutdown_connections 2024-05-20T21:04:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.141+0000 7f1695e29740 1 -- 172.21.15.26:0/1282674441 wait complete. 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 -- start start 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 Processor -- start 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565528e16790 con 0x5655291d00b0 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5655290ec890 con 0x5655291cf6c0 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5655291d81d0 con 0x5655291d5220 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1695e29740 10 client.0 _ll_register_callbacks cb 0x565528e44a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f168bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f168bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59434/0 (socket says 172.21.15.26:59434) 2024-05-20T21:04:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f168bfff640 1 -- 172.21.15.26:0/3762237805 learned_addr learned my addr 172.21.15.26:0/3762237805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f16915c7640 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1690dc6640 1 -- 172.21.15.26:0/3762237805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 msgr2=0x5655291e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1690dc6640 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1690dc6640 1 -- 172.21.15.26:0/3762237805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 msgr2=0x5655291d9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1690dc6640 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.142+0000 7f1690dc6640 1 -- 172.21.15.26:0/3762237805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565528f11620 con 0x5655291cf6c0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1690dc6640 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16800ce500 tx=0x7f1680009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa3bc302ea40949 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f166bfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f166bfff640 1 -- 172.21.15.26:0/3762237805 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680049040 con 0x5655291cf6c0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f166bfff640 1 -- 172.21.15.26:0/3762237805 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f16800c6220 con 0x5655291cf6c0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1678005020 con 0x5655291cf6c0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f166bfff640 1 -- 172.21.15.26:0/3762237805 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16800c98e0 con 0x5655291cf6c0 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 10 client.27188 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5655291e3ce0 con 0x5655291cf6c0 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f166bfff640 1 -- 172.21.15.26:0/3762237805 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1678005020 con 0x5655291cf6c0 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f166bfff640 1 -- 172.21.15.26:0/3762237805 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f16800ca1f0 con 0x5655291cf6c0 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 10 client.27188 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 10 client.27188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 20 client.27188 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x5655291e7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 4 client.27188 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.143+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565529142b80 con 0x5655291e5440 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.144+0000 7f168bfff640 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x5655291e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.144+0000 7f166bfff640 10 client.27188 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.144+0000 7f168bfff640 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x5655291e7900 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0ef180d7ea3311 server_cookie=ebc53ddf15b0c370 in_seq=0 out_seq=0 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f166bfff640 1 -- 172.21.15.26:0/3762237805 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565529142b80 con 0x5655291e5440 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f166bfff640 10 client.27188 handle_command_reply: tid=0 2024-05-20T21:04:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 client.27188 shutdown 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 msgr2=0x5655291d9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16800ce500 tx=0x7f1680009260 comp rx=0 tx=0).stop 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 shutdown_connections 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x5655291e7900 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f166bfff640 0 client.27188 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291d9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 --2- 172.21.15.26:0/3762237805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.145+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 >> 172.21.15.26:0/3762237805 conn(0x5655291cad30 msgr2=0x5655291c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.146+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 shutdown_connections 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.146+0000 7f1695e29740 1 -- 172.21.15.26:0/3762237805 wait complete. 2024-05-20T21:04:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.146+0000 7f1695e29740 20 client.27188 trim_cache size 0 max 16384 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 Processor -- start 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 -- start start 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ac7b0 con 0x5655291d00b0 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1aa080 con 0x5655291cf6c0 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1695e29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a7ed0 con 0x5655291d5220 2024-05-20T21:04:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f168bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f168bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59440/0 (socket says 172.21.15.26:59440) 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.162+0000 7f168bfff640 1 -- 172.21.15.26:0/3193908974 learned_addr learned my addr 172.21.15.26:0/3193908974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f16915c7640 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f168bfff640 1 -- 172.21.15.26:0/3193908974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 msgr2=0x5655291c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f168bfff640 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f168bfff640 1 -- 172.21.15.26:0/3193908974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 msgr2=0x5655291d9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f168bfff640 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f168bfff640 1 -- 172.21.15.26:0/3193908974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655291cf220 con 0x5655291d00b0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f1690dc6640 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f168bfff640 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291e0cc0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1678004770 tx=0x7f1678005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af2f14a9bb37dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f16767fc640 1 -- 172.21.15.26:0/3193908974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678005440 con 0x5655291d00b0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f16767fc640 1 -- 172.21.15.26:0/3193908974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16780055e0 con 0x5655291d00b0 2024-05-20T21:04:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.163+0000 7f16767fc640 1 -- 172.21.15.26:0/3193908974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167802e620 con 0x5655291d00b0 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 -- 172.21.15.26:0/3193908974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 msgr2=0x5655291e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291e0cc0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1678004770 tx=0x7f1678005f90 comp rx=0 tx=0).stop 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 -- 172.21.15.26:0/3193908974 shutdown_connections 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d5220 0x5655291c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291d00b0 0x5655291e0cc0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 --2- 172.21.15.26:0/3193908974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291cf6c0 0x5655291d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 -- 172.21.15.26:0/3193908974 >> 172.21.15.26:0/3193908974 conn(0x5655291cad30 msgr2=0x5655291d8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 -- 172.21.15.26:0/3193908974 shutdown_connections 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 -- 172.21.15.26:0/3193908974 wait complete. 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 -- start start 2024-05-20T21:04:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.164+0000 7f1695e29740 1 Processor -- start 2024-05-20T21:04:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 0x5655291de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d00b0 0x5655291de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a8540 con 0x5655291cf6c0 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a99b0 con 0x5655291d5220 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a8c10 con 0x5655291d00b0 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 10 client.0 _ll_register_callbacks cb 0x7f1680005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 0x5655291de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 0x5655291de390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59450/0 (socket says 172.21.15.26:59450) 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 -- 172.21.15.26:0/582382211 learned_addr learned my addr 172.21.15.26:0/582382211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f16915c7640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f168bfff640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d00b0 0x5655291de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 -- 172.21.15.26:0/582382211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d00b0 msgr2=0x5655291de8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d00b0 0x5655291de8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 -- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 msgr2=0x5655291df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 -- 172.21.15.26:0/582382211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655291dfed0 con 0x5655291cf6c0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1690dc6640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 0x5655291de390 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f16801007e0 tx=0x7f16800c6450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c180e4bce4618f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f166bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f166bfff640 1 -- 172.21.15.26:0/582382211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680049030 con 0x5655291cf6c0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f166bfff640 1 -- 172.21.15.26:0/582382211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f16800d2b70 con 0x5655291cf6c0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56552913c360 con 0x5655291cf6c0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f166bfff640 1 -- 172.21.15.26:0/582382211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16800e9050 con 0x5655291cf6c0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 10 client.17760 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.165+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5655291dfed0 con 0x5655291cf6c0 2024-05-20T21:04:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f166bfff640 1 -- 172.21.15.26:0/582382211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56552913c360 con 0x5655291cf6c0 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f166bfff640 1 -- 172.21.15.26:0/582382211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f168003bbb0 con 0x5655291cf6c0 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f1695e29740 10 client.17760 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f1695e29740 10 client.17760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f1695e29740 20 client.17760 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f1695e29740 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x565529252720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f1695e29740 4 client.17760 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5655290b65f0 con 0x5655291e5440 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f168bfff640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x565529252720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f166bfff640 10 client.17760 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.166+0000 7f168bfff640 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x565529252720 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b362a3abd44718ea server_cookie=44bd7c40d57bc5 in_seq=0 out_seq=0 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f166bfff640 1 -- 172.21.15.26:0/582382211 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5655290b65f0 con 0x5655291e5440 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f166bfff640 10 client.17760 handle_command_reply: tid=0 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 client.17760 shutdown 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 msgr2=0x5655291de390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 0x5655291de390 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f16801007e0 tx=0x7f16800c6450 comp rx=0 tx=0).stop 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 shutdown_connections 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5655291e5440 0x565529252720 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655291d5220 0x5655291df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655291d00b0 0x5655291de8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f166bfff640 0 client.17760 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 --2- 172.21.15.26:0/582382211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5655291cf6c0 0x5655291de390 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.233+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 >> 172.21.15.26:0/582382211 conn(0x5655291cad30 msgr2=0x5655291d8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.234+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 shutdown_connections 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.234+0000 7f1695e29740 1 -- 172.21.15.26:0/582382211 wait complete. 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.234+0000 7f1695e29740 20 client.17760 trim_cache size 0 max 16384 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:04:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f167c0b28a0 msgr2=0x7f167c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f167c0b28a0 0x7f167c0b4d60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5655291d0910 tx=0x7f1684009500 comp rx=0 tx=0).stop 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 msgr2=0x7f168c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 0x7f168c198bd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1680031460 tx=0x7f1680003c10 comp rx=0 tx=0).stop 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 shutdown_connections 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f167c0b28a0 0x7f167c0b4d60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c103560 0x7f168c1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f168c102ba0 0x7f168c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 --2- 172.21.15.26:0/1075774716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f168c1019b0 0x7f168c198bd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 >> 172.21.15.26:0/1075774716 conn(0x7f168c0fcf20 msgr2=0x7f168c06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 shutdown_connections 2024-05-20T21:04:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:05.235+0000 7f169302a640 1 -- 172.21.15.26:0/1075774716 wait complete. 2024-05-20T21:04:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:05 smithi055 ceph-mon[35213]: pgmap v628: 97 pgs: 97 active+clean; 29 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-05-20T21:04:05.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:05 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3197807420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:05.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:05 smithi026 ceph-mon[31990]: pgmap v628: 97 pgs: 97 active+clean; 29 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-05-20T21:04:05.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:05 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3197807420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:05 smithi144 ceph-mon[35660]: pgmap v628: 97 pgs: 97 active+clean; 29 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-05-20T21:04:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:05 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3197807420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:05.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 176195 2.36 MB/sec execute 251 sec latency 66.524 ms 2024-05-20T21:04:06.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 176986 2.38 MB/sec execute 252 sec latency 85.965 ms 2024-05-20T21:04:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:07 smithi055 ceph-mon[35213]: pgmap v629: 97 pgs: 97 active+clean; 28 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:04:07.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:07 smithi026 ceph-mon[31990]: pgmap v629: 97 pgs: 97 active+clean; 28 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:04:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:07 smithi144 ceph-mon[35660]: pgmap v629: 97 pgs: 97 active+clean; 28 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:04:07.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 177386 2.38 MB/sec execute 253 sec latency 60.895 ms 2024-05-20T21:04:08.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 177842 2.38 MB/sec execute 254 sec latency 18.097 ms 2024-05-20T21:04:09.384 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:09 smithi026 ceph-mon[31990]: pgmap v630: 97 pgs: 97 active+clean; 30 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T21:04:09.414 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-20T21:04:09.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-20T21:04:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:09 smithi055 ceph-mon[35213]: pgmap v630: 97 pgs: 97 active+clean; 30 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T21:04:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:09 smithi144 ceph-mon[35660]: pgmap v630: 97 pgs: 97 active+clean; 30 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-20T21:04:09.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 178402 2.38 MB/sec execute 255 sec latency 56.607 ms 2024-05-20T21:04:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 -- 172.21.15.26:0/1732091336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c10fe90 msgr2=0x7fbe0c112280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 --2- 172.21.15.26:0/1732091336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c10fe90 0x7fbe0c112280 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b0a0 tx=0x7fbe0400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 -- 172.21.15.26:0/1732091336 shutdown_connections 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 --2- 172.21.15.26:0/1732091336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c10fe90 0x7fbe0c112280 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 --2- 172.21.15.26:0/1732091336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c104450 0x7fbe0c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 --2- 172.21.15.26:0/1732091336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 -- 172.21.15.26:0/1732091336 >> 172.21.15.26:0/1732091336 conn(0x7fbe0c0793f0 msgr2=0x7fbe0c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 -- 172.21.15.26:0/1732091336 shutdown_connections 2024-05-20T21:04:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.924+0000 7fbe13177700 1 -- 172.21.15.26:0/1732091336 wait complete. 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.925+0000 7fbe13177700 1 Processor -- start 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.925+0000 7fbe13177700 1 -- start start 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe13177700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe13177700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c104450 0x7fbe0c1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe13177700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c10fe90 0x7fbe0c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c104450 0x7fbe0c1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c1a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60548/0 (socket says 172.21.15.26:60548) 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe13177700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1157b0 con 0x7fbe0c104450 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe13177700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c115630 con 0x7fbe0c06c150 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe13177700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c115930 con 0x7fbe0c10fe90 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 -- 172.21.15.26:0/3121265132 learned_addr learned my addr 172.21.15.26:0/3121265132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe11714700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c10fe90 0x7fbe0c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 -- 172.21.15.26:0/3121265132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c10fe90 msgr2=0x7fbe0c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c10fe90 0x7fbe0c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 -- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c104450 msgr2=0x7fbe0c1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c104450 0x7fbe0c1a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 -- 172.21.15.26:0/3121265132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c19c3f0 con 0x7fbe0c06c150 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.926+0000 7fbe10f13700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c1a1a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e9a0 tx=0x7fbe0000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.927+0000 7fbe09ffb700 1 -- 172.21.15.26:0/3121265132 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cbd0 con 0x7fbe0c06c150 2024-05-20T21:04:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.927+0000 7fbe09ffb700 1 -- 172.21.15.26:0/3121265132 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe0000cd70 con 0x7fbe0c06c150 2024-05-20T21:04:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.927+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c19c680 con 0x7fbe0c06c150 2024-05-20T21:04:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.927+0000 7fbe09ffb700 1 -- 172.21.15.26:0/3121265132 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00018680 con 0x7fbe0c06c150 2024-05-20T21:04:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.927+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c1a89c0 con 0x7fbe0c06c150 2024-05-20T21:04:09.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.930+0000 7fbe09ffb700 1 -- 172.21.15.26:0/3121265132 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbe00010430 con 0x7fbe0c06c150 2024-05-20T21:04:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe09ffb700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbdfc0b2900 0x7fbdfc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe09ffb700 1 -- 172.21.15.26:0/3121265132 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fbe000e23e0 con 0x7fbe0c06c150 2024-05-20T21:04:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbdfc0b2900 0x7fbdfc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 Processor -- start 2024-05-20T21:04:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 -- start start 2024-05-20T21:04:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 0x55cfe9bcc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfe9b21d70 con 0x55cfe9bd1a70 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cfe9b11910 con 0x55cfe9bd2420 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.931+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cfe9b15950 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbdfc0b2900 0x7fbdfc0b4dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009680 tx=0x7fbdf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 0x55cfe9bcc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe11714700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 0x55cfe9bcc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60556/0 (socket says 172.21.15.26:60556) 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 -- 172.21.15.26:0/1235136767 learned_addr learned my addr 172.21.15.26:0/1235136767 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe10f13700 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 -- 172.21.15.26:0/1235136767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bcc6d0 msgr2=0x55cfe9bccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 -- 172.21.15.26:0/1235136767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 msgr2=0x55cfe9bd1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 -- 172.21.15.26:0/1235136767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe9abafb0 con 0x55cfe9bd2420 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe10f13700 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 0x55cfe9bcc0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8010c50 tx=0x7fbdf8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c38bd8db166d3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbdf6ffd700 1 -- 172.21.15.26:0/1235136767 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8023070 con 0x55cfe9bd2420 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbdf6ffd700 1 -- 172.21.15.26:0/1235136767 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdf801f450 con 0x55cfe9bd2420 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.932+0000 7fbdf6ffd700 1 -- 172.21.15.26:0/1235136767 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8011640 con 0x55cfe9bd2420 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1235136767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 msgr2=0x55cfe9bcc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 0x55cfe9bcc0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8010c50 tx=0x7fbdf8013000 comp rx=0 tx=0).stop 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1235136767 shutdown_connections 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1235136767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd2420 0x55cfe9bcc0b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1235136767 >> 172.21.15.26:0/1235136767 conn(0x55cfe9aef240 msgr2=0x55cfe9bbe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:09.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.933+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1235136767 shutdown_connections 2024-05-20T21:04:09.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.934+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1235136767 wait complete. 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.934+0000 7fbe1da0db80 1 -- start start 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.934+0000 7fbe1da0db80 1 Processor -- start 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bd8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bd5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfe9a17fa0 con 0x55cfe9bd1a70 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cfe9a79da0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cfe9bd8ea0 con 0x55cfe9bd2420 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe1da0db80 10 client.0 _ll_register_callbacks cb 0x55cfe9aac950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bd8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bd8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60570/0 (socket says 172.21.15.26:60570) 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 -- 172.21.15.26:0/3979535802 learned_addr learned my addr 172.21.15.26:0/3979535802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe11714700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bd5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 -- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 msgr2=0x55cfe9bd5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bd5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 -- 172.21.15.26:0/3979535802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 msgr2=0x55cfe9bd5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe10f13700 1 -- 172.21.15.26:0/3979535802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe9a618d0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.935+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:09.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbe10f13700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bd8250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe000dfed0 tx=0x7fbe000a9ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75dac07901bb4f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbdebfff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbdebfff700 1 -- 172.21.15.26:0/3979535802 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000cc2d0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbdebfff700 1 -- 172.21.15.26:0/3979535802 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fbe00010720 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf801f450 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbdebfff700 1 -- 172.21.15.26:0/3979535802 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe000aa0a0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbe1da0db80 10 client.27206 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfe9acb760 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.936+0000 7fbdebfff700 1 -- 172.21.15.26:0/3979535802 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdf801f450 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbdebfff700 1 -- 172.21.15.26:0/3979535802 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fbe000c6070 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbe1da0db80 10 client.27206 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbe1da0db80 10 client.27206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbe1da0db80 20 client.27206 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9be5b00 0x55cfe9be7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbe1da0db80 4 client.27206 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:09.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.937+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe9b23cc0 con 0x55cfe9be5b00 2024-05-20T21:04:09.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.938+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9be5b00 0x55cfe9be7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.938+0000 7fbdebfff700 10 client.27206 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:09.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.938+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9be5b00 0x55cfe9be7ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b445cad7d3148ed server_cookie=1870994caeff11fb in_seq=0 out_seq=0 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbdebfff700 1 -- 172.21.15.26:0/3979535802 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cfe9b23cc0 con 0x55cfe9be5b00 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbdebfff700 10 client.27206 handle_command_reply: tid=0 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 client.27206 shutdown 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 msgr2=0x55cfe9bd8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bd8250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe000dfed0 tx=0x7fbe000a9ee0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 shutdown_connections 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9be5b00 0x55cfe9be7ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bd5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bd8250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/3979535802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 >> 172.21.15.26:0/3979535802 conn(0x55cfe9aef240 msgr2=0x55cfe9bbdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbdebfff700 0 client.27206 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 shutdown_connections 2024-05-20T21:04:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.939+0000 7fbe1da0db80 1 -- 172.21.15.26:0/3979535802 wait complete. 2024-05-20T21:04:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.940+0000 7fbe1da0db80 20 client.27206 trim_cache size 0 max 16384 2024-05-20T21:04:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 Processor -- start 2024-05-20T21:04:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 -- start start 2024-05-20T21:04:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9be1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1ae3b0 con 0x55cfe9bd1a70 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1abc40 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c108b30 con 0x55cfe9bd2420 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33278/0 (socket says 172.21.15.26:33278) 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe0bfff700 1 -- 172.21.15.26:0/1791986353 learned_addr learned my addr 172.21.15.26:0/1791986353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.960+0000 7fbe10f13700 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9be1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe11714700 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe10f13700 1 -- 172.21.15.26:0/1791986353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 msgr2=0x55cfe9bbe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe10f13700 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bbe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe10f13700 1 -- 172.21.15.26:0/1791986353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 msgr2=0x55cfe9bd2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe10f13700 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe10f13700 1 -- 172.21.15.26:0/1791986353 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe9bbf5b0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbe10f13700 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9be1f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbe000a9930 tx=0x7fbe00010bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b982b543828cbb7d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbdf67fc700 1 -- 172.21.15.26:0/1791986353 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000a99a0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbdf67fc700 1 -- 172.21.15.26:0/1791986353 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe000add50 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.961+0000 7fbdf67fc700 1 -- 172.21.15.26:0/1791986353 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00010960 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1791986353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 msgr2=0x55cfe9be1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9be1f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbe000a9930 tx=0x7fbe00010bb0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1791986353 shutdown_connections 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bbe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9be1f70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/1791986353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bd2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1791986353 >> 172.21.15.26:0/1791986353 conn(0x55cfe9aef240 msgr2=0x55cfe9bd2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.962+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1791986353 shutdown_connections 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.963+0000 7fbe1da0db80 1 -- 172.21.15.26:0/1791986353 wait complete. 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.963+0000 7fbe1da0db80 1 -- start start 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.963+0000 7fbe1da0db80 1 Processor -- start 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bde160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe1da0db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bde8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1aa100 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1ab570 con 0x55cfe9bd1a70 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe1da0db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1aa7d0 con 0x55cfe9bd2420 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.964+0000 7fbe10f13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe1da0db80 10 client.0 _ll_register_callbacks cb 0x55cfe9aac950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33282/0 (socket says 172.21.15.26:33282) 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 -- 172.21.15.26:0/630474823 learned_addr learned my addr 172.21.15.26:0/630474823 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe11714700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bde8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bde160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 -- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 msgr2=0x55cfe9bde8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bde8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 -- 172.21.15.26:0/630474823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd1a70 msgr2=0x55cfe9bde160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bde160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe10f13700 1 -- 172.21.15.26:0/630474823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe9bec980 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe11714700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bde8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.965+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bde160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.966+0000 7fbe10f13700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bee080 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009230 tx=0x7fbe00010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f4a490f4ee1b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.966+0000 7fbdebfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.966+0000 7fbdebfff700 1 -- 172.21.15.26:0/630474823 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000a99a0 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.966+0000 7fbdebfff700 1 -- 172.21.15.26:0/630474823 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fbe000a8e10 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.966+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe000add50 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.966+0000 7fbdebfff700 1 -- 172.21.15.26:0/630474823 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe000b2800 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbe1da0db80 10 client.17775 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfe9bec980 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbdebfff700 1 -- 172.21.15.26:0/630474823 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe000add50 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbdebfff700 1 -- 172.21.15.26:0/630474823 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fbe000c6070 con 0x55cfe9bcc6d0 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbe1da0db80 10 client.17775 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:09.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbe1da0db80 10 client.17775 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:04:09.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbe1da0db80 20 client.17775 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:09.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.967+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9c4fd90 0x55cfe9c52180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:09.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.968+0000 7fbe1da0db80 4 client.17775 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:04:09.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.968+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfe9b1d4a0 con 0x55cfe9c4fd90 2024-05-20T21:04:09.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:09.969+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9c4fd90 0x55cfe9c52180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:10.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbe0bfff700 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9c4fd90 0x55cfe9c52180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47f2fc50a35b712b server_cookie=175009a6b21398d7 in_seq=0 out_seq=0 2024-05-20T21:04:10.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbdebfff700 10 client.17775 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:10.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbdebfff700 1 -- 172.21.15.26:0/630474823 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfe9b1d4a0 con 0x55cfe9c4fd90 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbdebfff700 10 client.17775 handle_command_reply: tid=0 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbe1da0db80 1 client.17775 shutdown 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 msgr2=0x55cfe9bee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.000+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bee080 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009230 tx=0x7fbe00010a90 comp rx=0 tx=0).stop 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 shutdown_connections 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cfe9c4fd90 0x55cfe9c52180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe9bcc6d0 0x55cfe9bee080 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfe9bd2420 0x55cfe9bde8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 --2- 172.21.15.26:0/630474823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cfe9bd1a70 0x55cfe9bde160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 >> 172.21.15.26:0/630474823 conn(0x55cfe9aef240 msgr2=0x55cfe9bd30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbdebfff700 0 client.17775 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 shutdown_connections 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 1 -- 172.21.15.26:0/630474823 wait complete. 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.001+0000 7fbe1da0db80 20 client.17775 trim_cache size 0 max 16384 2024-05-20T21:04:10.002 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:04:10.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.002+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbdfc0b2900 msgr2=0x7fbdfc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:10.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.002+0000 7fbe13177700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbdfc0b2900 0x7fbdfc0b4dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009680 tx=0x7fbdf8009260 comp rx=0 tx=0).stop 2024-05-20T21:04:10.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.002+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 msgr2=0x7fbe0c1a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:10.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.002+0000 7fbe13177700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c1a1a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e9a0 tx=0x7fbe0000ecb0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.002+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 shutdown_connections 2024-05-20T21:04:10.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.002+0000 7fbe13177700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c10fe90 0x7fbe0c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.003+0000 7fbe13177700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06c150 0x7fbe0c1a1a40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.003+0000 7fbe13177700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbdfc0b2900 0x7fbdfc0b4dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.003+0000 7fbe13177700 1 --2- 172.21.15.26:0/3121265132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c104450 0x7fbe0c1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.003+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 >> 172.21.15.26:0/3121265132 conn(0x7fbe0c0793f0 msgr2=0x7fbe0c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.003+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 shutdown_connections 2024-05-20T21:04:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:10.003+0000 7fbe13177700 1 -- 172.21.15.26:0/3121265132 wait complete. 2024-05-20T21:04:10.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 179016 2.38 MB/sec execute 256 sec latency 18.172 ms 2024-05-20T21:04:11.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:04:11.299 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:11 smithi026 ceph-mon[31990]: pgmap v631: 97 pgs: 97 active+clean; 26 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:04:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:11 smithi055 ceph-mon[35213]: pgmap v631: 97 pgs: 97 active+clean; 26 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:04:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:11 smithi144 ceph-mon[35660]: pgmap v631: 97 pgs: 97 active+clean; 26 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:04:11.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 179733 2.38 MB/sec execute 257 sec latency 17.403 ms 2024-05-20T21:04:12.073 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:12.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180630 2.40 MB/sec execute 258 sec latency 16.187 ms 2024-05-20T21:04:12.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.820+0000 7f3eb8f75700 1 -- 172.21.15.26:0/3992386898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102470 msgr2=0x7f3eb41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:12.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.820+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/3992386898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102470 0x7f3eb41028f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009990 tx=0x7f3ea8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 -- 172.21.15.26:0/3992386898 shutdown_connections 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/3992386898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb4102e30 0x7f3eb410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/3992386898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 0x7f3eb41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/3992386898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102470 0x7f3eb41028f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 -- 172.21.15.26:0/3992386898 >> 172.21.15.26:0/3992386898 conn(0x7f3eb4079480 msgr2=0x7f3eb4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 -- 172.21.15.26:0/3992386898 shutdown_connections 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 -- 172.21.15.26:0/3992386898 wait complete. 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.821+0000 7f3eb8f75700 1 Processor -- start 2024-05-20T21:04:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 -- start start 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb4102470 0x7f3eb419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 0x7f3eb419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 0x7f3eb4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4112530 con 0x7f3eb4102e30 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41123b0 con 0x7f3eb41073d0 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb8f75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41126b0 con 0x7f3eb4102470 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 0x7f3eb419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 0x7f3eb419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33300/0 (socket says 172.21.15.26:33300) 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/742335017 learned_addr learned my addr 172.21.15.26:0/742335017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 0x7f3eb4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb259c700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb4102470 0x7f3eb419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/742335017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb4102470 msgr2=0x7f3eb419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb4102470 0x7f3eb419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/742335017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 msgr2=0x7f3eb4199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 0x7f3eb4199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/742335017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb419a0d0 con 0x7f3eb4102e30 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.822+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 0x7f3eb4199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.823+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 0x7f3eb419fca0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b650 tx=0x7f3ea800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.823+0000 7f3ea37fe700 1 -- 172.21.15.26:0/742335017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8027070 con 0x7f3eb4102e30 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.823+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb419a360 con 0x7f3eb4102e30 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.823+0000 7f3ea37fe700 1 -- 172.21.15.26:0/742335017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea8022470 con 0x7f3eb4102e30 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.823+0000 7f3ea37fe700 1 -- 172.21.15.26:0/742335017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8017650 con 0x7f3eb4102e30 2024-05-20T21:04:12.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.823+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a6630 con 0x7f3eb4102e30 2024-05-20T21:04:12.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.825+0000 7f3ea37fe700 1 -- 172.21.15.26:0/742335017 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3ea8009d10 con 0x7f3eb4102e30 2024-05-20T21:04:12.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.826+0000 7f3ea37fe700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e9c0b2930 0x7f3e9c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3eb259c700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e9c0b2930 0x7f3e9c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ea37fe700 1 -- 172.21.15.26:0/742335017 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3ea80e2e70 con 0x7f3eb4102e30 2024-05-20T21:04:12.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ec380bb80 1 Processor -- start 2024-05-20T21:04:12.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3eb259c700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e9c0b2930 0x7f3e9c0b4df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4108a00 tx=0x7f3ea4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:12.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ec380bb80 1 -- start start 2024-05-20T21:04:12.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 0x55e282b51b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4cc80 0x55e282b4d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4d640 0x55e282b596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.827+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e282aaaf00 con 0x55e282b517b0 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e282a9aaa0 con 0x55e282b4cc80 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e282a9eae0 con 0x55e282b4d640 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 0x55e282b51b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4cc80 0x55e282b4d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 0x55e282b51b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33312/0 (socket says 172.21.15.26:33312) 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb259c700 1 -- 172.21.15.26:0/695961589 learned_addr learned my addr 172.21.15.26:0/695961589 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4d640 0x55e282b596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/695961589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4d640 msgr2=0x55e282b596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4d640 0x55e282b596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/695961589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 msgr2=0x55e282b51b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 0x55e282b51b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/695961589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e282a44140 con 0x55e282b4cc80 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb259c700 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 0x55e282b51b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4cc80 0x55e282b4d100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80e0910 tx=0x7f3ea80a9d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13c23cda1b7d89a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.828+0000 7f3ea0ff9700 1 -- 172.21.15.26:0/695961589 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80920e0 con 0x55e282b4cc80 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ea0ff9700 1 -- 172.21.15.26:0/695961589 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea80aa4c0 con 0x55e282b4cc80 2024-05-20T21:04:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ea0ff9700 1 -- 172.21.15.26:0/695961589 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80aa720 con 0x55e282b4cc80 2024-05-20T21:04:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 -- 172.21.15.26:0/695961589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4cc80 msgr2=0x55e282b4d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4cc80 0x55e282b4d100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80e0910 tx=0x7f3ea80a9d20 comp rx=0 tx=0).stop 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 -- 172.21.15.26:0/695961589 shutdown_connections 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b517b0 0x55e282b51b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4d640 0x55e282b596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/695961589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4cc80 0x55e282b4d100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 -- 172.21.15.26:0/695961589 >> 172.21.15.26:0/695961589 conn(0x55e282a783d0 msgr2=0x55e282b47970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.829+0000 7f3ec380bb80 1 -- 172.21.15.26:0/695961589 shutdown_connections 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 -- 172.21.15.26:0/695961589 wait complete. 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 -- start start 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 Processor -- start 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 0x55e282b59380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282b56680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b517b0 0x55e282b56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e2829a1130 con 0x55e282b4cc80 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e282a02f30 con 0x55e282b4d640 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e282a9aaa0 con 0x55e282b517b0 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3ec380bb80 10 client.0 _ll_register_callbacks cb 0x55e282a35ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 0x55e282b59380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.830+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 0x55e282b59380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33322/0 (socket says 172.21.15.26:33322) 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb259c700 1 -- 172.21.15.26:0/4122975586 learned_addr learned my addr 172.21.15.26:0/4122975586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282b56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b517b0 0x55e282b56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/4122975586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b517b0 msgr2=0x55e282b56bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b517b0 0x55e282b56bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/4122975586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 msgr2=0x55e282b59380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 0x55e282b59380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 -- 172.21.15.26:0/4122975586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2829eaa60 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b517b0 0x55e282b56bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb259c700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 0x55e282b59380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282b56680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80b3450 tx=0x7f3ea80a4e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c689e183f25a1f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3e99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3e99ffb700 1 -- 172.21.15.26:0/4122975586 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80c3060 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3e99ffb700 1 -- 172.21.15.26:0/4122975586 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3ea80aa930 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea80aa4c0 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3e99ffb700 1 -- 172.21.15.26:0/4122975586 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ea80b3290 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3ec380bb80 10 client.27221 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.831+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e282a548f0 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3e99ffb700 1 -- 172.21.15.26:0/4122975586 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea80aa4c0 con 0x55e282b4d640 2024-05-20T21:04:12.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3e99ffb700 1 -- 172.21.15.26:0/4122975586 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f3ea80a5e30 con 0x55e282b4d640 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3ec380bb80 10 client.27221 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3ec380bb80 10 client.27221 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3ec380bb80 20 client.27221 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b66400 0x55e282b687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3ec380bb80 4 client.27221 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e282aace50 con 0x55e282b66400 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3eb259c700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b66400 0x55e282b687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3e99ffb700 10 client.27221 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.832+0000 7f3eb259c700 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b66400 0x55e282b687f0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c712cda22c4471 server_cookie=dc7b5492d7f11e3 in_seq=0 out_seq=0 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3e99ffb700 1 -- 172.21.15.26:0/4122975586 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e282aace50 con 0x55e282b66400 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3e99ffb700 10 client.27221 handle_command_reply: tid=0 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 client.27221 shutdown 2024-05-20T21:04:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 msgr2=0x55e282b56680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282b56680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80b3450 tx=0x7f3ea80a4e10 comp rx=0 tx=0).stop 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 shutdown_connections 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b66400 0x55e282b687f0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4cc80 0x55e282b59380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b517b0 0x55e282b56bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4122975586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282b56680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 >> 172.21.15.26:0/4122975586 conn(0x55e282a783d0 msgr2=0x55e282b4bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3e99ffb700 0 client.27221 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 shutdown_connections 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4122975586 wait complete. 2024-05-20T21:04:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.834+0000 7f3ec380bb80 20 client.27221 trim_cache size 0 max 16384 2024-05-20T21:04:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.858+0000 7f3ec380bb80 1 Processor -- start 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.858+0000 7f3ec380bb80 1 -- start start 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 0x55e282b41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282b50f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4d640 0x55e282b6df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b0490 con 0x55e282b4d640 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41add20 con 0x55e282a783d0 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41abb70 con 0x55e282b4cc80 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 0x55e282b41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 0x55e282b41f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60626/0 (socket says 172.21.15.26:60626) 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4d640 0x55e282b6df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 -- 172.21.15.26:0/4283203522 learned_addr learned my addr 172.21.15.26:0/4283203522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282b50f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 -- 172.21.15.26:0/4283203522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 msgr2=0x55e282b50f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282b50f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 -- 172.21.15.26:0/4283203522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4d640 msgr2=0x55e282b6df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4d640 0x55e282b6df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 -- 172.21.15.26:0/4283203522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e282b58ec0 con 0x55e282a783d0 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4d640 0x55e282b6df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.859+0000 7f3eb259c700 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 0x55e282b41f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097d0 tx=0x7f3ea4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd49b111ecf3a46 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3e9bfff700 1 -- 172.21.15.26:0/4283203522 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011dc0 con 0x55e282a783d0 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3e9bfff700 1 -- 172.21.15.26:0/4283203522 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea40044b0 con 0x55e282a783d0 2024-05-20T21:04:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3e9bfff700 1 -- 172.21.15.26:0/4283203522 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea402a470 con 0x55e282a783d0 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4283203522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 msgr2=0x55e282b41f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 0x55e282b41f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097d0 tx=0x7f3ea4013000 comp rx=0 tx=0).stop 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4283203522 shutdown_connections 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282b4d640 0x55e282b6df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282a783d0 0x55e282b41f00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/4283203522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282b50f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.860+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4283203522 >> 172.21.15.26:0/4283203522 conn(0x55e282b68ae0 msgr2=0x55e282b50820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4283203522 shutdown_connections 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 -- 172.21.15.26:0/4283203522 wait complete. 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 -- start start 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 Processor -- start 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 0x55e282bcc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282bcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282bcb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ac1e0 con 0x55e282a783d0 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ad650 con 0x55e282b4d640 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ac8b0 con 0x55e282b4cc80 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.861+0000 7f3ec380bb80 10 client.0 _ll_register_callbacks cb 0x55e282a35ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282bcb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 0x55e282bcc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282bcb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60632/0 (socket says 172.21.15.26:60632) 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 -- 172.21.15.26:0/3647511320 learned_addr learned my addr 172.21.15.26:0/3647511320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 0x55e282bcc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33340/0 (socket says 172.21.15.26:33340) 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb1d9b700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282bcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 -- 172.21.15.26:0/3647511320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 msgr2=0x55e282bcae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282bcae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 -- 172.21.15.26:0/3647511320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 msgr2=0x55e282bcc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 0x55e282bcc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 -- 172.21.15.26:0/3647511320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e282bcb8a0 con 0x55e282b4d640 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb259c700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 0x55e282bcc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3eb2d9d700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282bcb360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3eac016360 tx=0x7f3eac016390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3622e7c58717e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3e99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3e99ffb700 1 -- 172.21.15.26:0/3647511320 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac01c070 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3e99ffb700 1 -- 172.21.15.26:0/3647511320 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3eac003a00 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.862+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea40044b0 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3e99ffb700 1 -- 172.21.15.26:0/3647511320 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3eac016e60 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 10 client.27227 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e282b5ea60 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3e99ffb700 1 -- 172.21.15.26:0/3647511320 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea40044b0 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3e99ffb700 1 -- 172.21.15.26:0/3647511320 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f3eac029050 con 0x55e282b4d640 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 10 client.27227 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 10 client.27227 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 20 client.27227 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b517b0 0x55e282b5f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 4 client.27227 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-20T21:04:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55e282aa6630 con 0x55e282b517b0 2024-05-20T21:04:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.863+0000 7f3eb259c700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b517b0 0x55e282b5f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.864+0000 7f3eb259c700 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b517b0 0x55e282b5f220 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79bde9095f5b4eb server_cookie=8e0a1b613af0ea47 in_seq=0 out_seq=0 2024-05-20T21:04:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:12.864+0000 7f3e99ffb700 10 client.27227 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.044+0000 7f3e99ffb700 1 -- 172.21.15.26:0/3647511320 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e282aa6630 con 0x55e282b517b0 2024-05-20T21:04:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.044+0000 7f3e99ffb700 10 client.27227 handle_command_reply: tid=0 2024-05-20T21:04:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.044+0000 7f3ec380bb80 1 client.27227 shutdown 2024-05-20T21:04:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 msgr2=0x55e282bcb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282bcb360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3eac016360 tx=0x7f3eac016390 comp rx=0 tx=0).stop 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 shutdown_connections 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e282b517b0 0x55e282b5f220 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e282a783d0 0x55e282bcc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e282b4d640 0x55e282bcb360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3e99ffb700 0 client.27227 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 --2- 172.21.15.26:0/3647511320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e282b4cc80 0x55e282bcae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 >> 172.21.15.26:0/3647511320 conn(0x55e282b68ae0 msgr2=0x55e282b57eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 shutdown_connections 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 1 -- 172.21.15.26:0/3647511320 wait complete. 2024-05-20T21:04:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.045+0000 7f3ec380bb80 20 client.27227 trim_cache size 0 max 16384 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 30, 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "a6088460": { 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 30, 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:04:13.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.0, 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e9c0b2930 msgr2=0x7f3e9c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e9c0b2930 0x7f3e9c0b4df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4108a00 tx=0x7f3ea4009260 comp rx=0 tx=0).stop 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 msgr2=0x7f3eb419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 0x7f3eb419fca0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b650 tx=0x7f3ea800bfc0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 shutdown_connections 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3eb41073d0 0x7f3eb4199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb4102470 0x7f3eb419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3e9c0b2930 0x7f3e9c0b4df0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 --2- 172.21.15.26:0/742335017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb4102e30 0x7f3eb419fca0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 >> 172.21.15.26:0/742335017 conn(0x7f3eb4079480 msgr2=0x7f3eb406be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 shutdown_connections 2024-05-20T21:04:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:13.046+0000 7f3eb8f75700 1 -- 172.21.15.26:0/742335017 wait complete. 2024-05-20T21:04:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:13 smithi055 ceph-mon[35213]: pgmap v632: 97 pgs: 97 active+clean; 26 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-20T21:04:13.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:13 smithi026 ceph-mon[31990]: pgmap v632: 97 pgs: 97 active+clean; 26 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-20T21:04:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:13 smithi144 ceph-mon[35660]: pgmap v632: 97 pgs: 97 active+clean; 26 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-20T21:04:13.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.39 MB/sec execute 259 sec latency 413.712 ms 2024-05-20T21:04:14.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.38 MB/sec execute 260 sec latency 1413.818 ms 2024-05-20T21:04:14.673 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a6088460', quiesce took 14.5 seconds. Will release after: -4.5 2024-05-20T21:04:14.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 -- 172.21.15.26:0/2233963770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f1406bc10 msgr2=0x7f8f1406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 --2- 172.21.15.26:0/2233963770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f1406bc10 0x7f8f1406c090 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b0a0 tx=0x7f8f0400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 -- 172.21.15.26:0/2233963770 shutdown_connections 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 --2- 172.21.15.26:0/2233963770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f1406c5d0 0x7f8f141103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 --2- 172.21.15.26:0/2233963770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f141095d0 0x7f8f141099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 --2- 172.21.15.26:0/2233963770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f1406bc10 0x7f8f1406c090 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 -- 172.21.15.26:0/2233963770 >> 172.21.15.26:0/2233963770 conn(0x7f8f140793f0 msgr2=0x7f8f14079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.216+0000 7f8f1b161700 1 -- 172.21.15.26:0/2233963770 shutdown_connections 2024-05-20T21:04:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.217+0000 7f8f1b161700 1 -- 172.21.15.26:0/2233963770 wait complete. 2024-05-20T21:04:15.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:15 smithi026 ceph-mon[31990]: pgmap v633: 97 pgs: 97 active+clean; 27 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:04:15.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.217+0000 7f8f1b161700 1 Processor -- start 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.217+0000 7f8f1b161700 1 -- start start 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.217+0000 7f8f1b161700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f1406bc10 0x7f8f1419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.217+0000 7f8f1b161700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f1406c5d0 0x7f8f1419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f1b161700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 0x7f8f14199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f1b161700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14113610 con 0x7f8f141095d0 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f1b161700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14113490 con 0x7f8f1406bc10 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f1b161700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14113790 con 0x7f8f1406c5d0 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 0x7f8f14199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 0x7f8f14199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33360/0 (socket says 172.21.15.26:33360) 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 -- 172.21.15.26:0/1962035237 learned_addr learned my addr 172.21.15.26:0/1962035237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f18efd700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f1406bc10 0x7f8f1419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f13fff700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f1406c5d0 0x7f8f1419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 -- 172.21.15.26:0/1962035237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f1406c5d0 msgr2=0x7f8f1419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f1406c5d0 0x7f8f1419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 -- 172.21.15.26:0/1962035237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f1406bc10 msgr2=0x7f8f1419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f1406bc10 0x7f8f1419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 -- 172.21.15.26:0/1962035237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1419a0f0 con 0x7f8f141095d0 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f196fe700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 0x7f8f14199990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c970 tx=0x7f8f0c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f11ffb700 1 -- 172.21.15.26:0/1962035237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004a30 con 0x7f8f141095d0 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f11ffb700 1 -- 172.21.15.26:0/1962035237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f0c0054b0 con 0x7f8f141095d0 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.218+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1419a380 con 0x7f8f141095d0 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.219+0000 7f8f11ffb700 1 -- 172.21.15.26:0/1962035237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00eac0 con 0x7f8f141095d0 2024-05-20T21:04:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.219+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a66a0 con 0x7f8f141095d0 2024-05-20T21:04:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.221+0000 7f8f11ffb700 1 -- 172.21.15.26:0/1962035237 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8f0c00ed80 con 0x7f8f141095d0 2024-05-20T21:04:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.222+0000 7f8f11ffb700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f000b2880 0x7f8f000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.222+0000 7f8f18efd700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f000b2880 0x7f8f000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.222+0000 7f8f11ffb700 1 -- 172.21.15.26:0/1962035237 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8f0c0e1a20 con 0x7f8f141095d0 2024-05-20T21:04:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.222+0000 7f8f18efd700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f000b2880 0x7f8f000b4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8f1419aad0 tx=0x7f8efc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.223+0000 7f8f259f7b80 1 Processor -- start 2024-05-20T21:04:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.223+0000 7f8f259f7b80 1 -- start start 2024-05-20T21:04:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.223+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed5860 0x56550fed5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.223+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 0x56550feda3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56550fe34c90 con 0x56550fed9f60 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56550fe24830 con 0x56550fed5860 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56550fe28870 con 0x56550fed6210 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 0x56550feda3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 0x56550feda3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33372/0 (socket says 172.21.15.26:33372) 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 -- 172.21.15.26:0/295973460 learned_addr learned my addr 172.21.15.26:0/295973460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f18efd700 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed5860 0x56550fed5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f13fff700 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 -- 172.21.15.26:0/295973460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 msgr2=0x56550fed9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 -- 172.21.15.26:0/295973460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed5860 msgr2=0x56550fed5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed5860 0x56550fed5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 -- 172.21.15.26:0/295973460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550fd748d0 con 0x56550fed9f60 2024-05-20T21:04:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f18efd700 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed5860 0x56550fed5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f196fe700 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 0x56550feda3a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0a5060 tx=0x7f8f0c0a8c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aebf27b2549b7e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f0affd700 1 -- 172.21.15.26:0/295973460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0c3070 con 0x56550fed9f60 2024-05-20T21:04:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f0affd700 1 -- 172.21.15.26:0/295973460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f0c0ad390 con 0x56550fed9f60 2024-05-20T21:04:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.224+0000 7f8f0affd700 1 -- 172.21.15.26:0/295973460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c01d030 con 0x56550fed9f60 2024-05-20T21:04:15.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 -- 172.21.15.26:0/295973460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 msgr2=0x56550feda3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 0x56550feda3a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0a5060 tx=0x7f8f0c0a8c20 comp rx=0 tx=0).stop 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 -- 172.21.15.26:0/295973460 shutdown_connections 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed9f60 0x56550feda3a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed5860 0x56550fed5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/295973460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 -- 172.21.15.26:0/295973460 >> 172.21.15.26:0/295973460 conn(0x56550fe03080 msgr2=0x56550fecfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 -- 172.21.15.26:0/295973460 shutdown_connections 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 -- 172.21.15.26:0/295973460 wait complete. 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.225+0000 7f8f259f7b80 1 -- start start 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 Processor -- start 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed6210 0x56550fede690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed9f60 0x56550fedebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56550fd2afa0 con 0x56550fed5860 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56550fdc67c0 con 0x56550fed6210 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56550fee2120 con 0x56550fed9f60 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f259f7b80 10 client.0 _ll_register_callbacks cb 0x56550fdbf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f18efd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f18efd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33374/0 (socket says 172.21.15.26:33374) 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f18efd700 1 -- 172.21.15.26:0/3970425517 learned_addr learned my addr 172.21.15.26:0/3970425517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed6210 0x56550fede690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.226+0000 7f8f196fe700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed9f60 0x56550fedebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f18efd700 1 -- 172.21.15.26:0/3970425517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed9f60 msgr2=0x56550fedebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed9f60 0x56550fedebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f18efd700 1 -- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed6210 msgr2=0x56550fede690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed6210 0x56550fede690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f18efd700 1 -- 172.21.15.26:0/3970425517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550fdc3690 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee1390 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00eea0 tx=0x7f8efc004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb1185468a70755 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8ef3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3970425517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc024020 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f0c0ad390 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3970425517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8efc00f040 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3970425517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8efc015aa0 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f259f7b80 10 client.17805 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.227+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56550feec830 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3970425517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f0c0ad390 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3970425517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8efc006250 con 0x56550fed5860 2024-05-20T21:04:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f259f7b80 10 client.17805 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f259f7b80 10 client.17805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f259f7b80 20 client.17805 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550feeee70 0x56550fef1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f259f7b80 4 client.17805 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56550fea19e0 con 0x56550feeee70 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.228+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550feeee70 0x56550fef1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.229+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550feeee70 0x56550fef1260 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3a1e443249dffff server_cookie=e5448e795abcd748 in_seq=0 out_seq=0 2024-05-20T21:04:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.229+0000 7f8ef3fff700 10 client.17805 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3970425517 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56550fea19e0 con 0x56550feeee70 2024-05-20T21:04:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8ef3fff700 10 client.17805 handle_command_reply: tid=0 2024-05-20T21:04:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 client.17805 shutdown 2024-05-20T21:04:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 msgr2=0x56550fee1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee1390 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00eea0 tx=0x7f8efc004860 comp rx=0 tx=0).stop 2024-05-20T21:04:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 shutdown_connections 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550feeee70 0x56550fef1260 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee1390 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed9f60 0x56550fedebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3970425517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed6210 0x56550fede690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 >> 172.21.15.26:0/3970425517 conn(0x56550fe03080 msgr2=0x56550fedb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8ef3fff700 0 client.17805 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.230+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 shutdown_connections 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.231+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3970425517 wait complete. 2024-05-20T21:04:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.231+0000 7f8f259f7b80 20 client.17805 trim_cache size 0 max 16384 2024-05-20T21:04:15.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 Processor -- start 2024-05-20T21:04:15.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 -- start start 2024-05-20T21:04:15.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed5860 0x56550fef6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 0x56550fef7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141b0570 con 0x56550fed6210 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141ade00 con 0x56550fed9f60 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.247+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141abc50 con 0x56550fed5860 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 0x56550fef7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 0x56550fef7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33380/0 (socket says 172.21.15.26:33380) 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 -- 172.21.15.26:0/3102194778 learned_addr learned my addr 172.21.15.26:0/3102194778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f196fe700 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed5860 0x56550fef6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 -- 172.21.15.26:0/3102194778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed5860 msgr2=0x56550fef6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed5860 0x56550fef6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 -- 172.21.15.26:0/3102194778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 msgr2=0x56550fed4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 -- 172.21.15.26:0/3102194778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550fedcc00 con 0x56550fed6210 2024-05-20T21:04:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed5860 0x56550fef6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 0x56550fef7090 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b070 tx=0x7f8f04003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7609a3350d4f0da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3102194778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04003b70 con 0x56550fed6210 2024-05-20T21:04:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3102194778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f04003d10 con 0x56550fed6210 2024-05-20T21:04:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.248+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3102194778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f040078e0 con 0x56550fed6210 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3102194778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 msgr2=0x56550fef7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 0x56550fef7090 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b070 tx=0x7f8f04003a00 comp rx=0 tx=0).stop 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3102194778 shutdown_connections 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed5860 0x56550fef6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3102194778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed6210 0x56550fef7090 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3102194778 >> 172.21.15.26:0/3102194778 conn(0x56550fe03080 msgr2=0x56550fedcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3102194778 shutdown_connections 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.249+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3102194778 wait complete. 2024-05-20T21:04:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 -- start start 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 Processor -- start 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141ac2c0 con 0x56550fed5860 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141ad730 con 0x56550fed9f60 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f259f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141ac990 con 0x56550fed6210 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f18efd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f18efd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33384/0 (socket says 172.21.15.26:33384) 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f18efd700 1 -- 172.21.15.26:0/3024429519 learned_addr learned my addr 172.21.15.26:0/3024429519 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.250+0000 7f8f196fe700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f259f7b80 10 client.0 _ll_register_callbacks cb 0x56550fdbf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f18efd700 1 -- 172.21.15.26:0/3024429519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 msgr2=0x56550fed6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f18efd700 1 -- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 msgr2=0x56550fed7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f18efd700 1 -- 172.21.15.26:0/3024429519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550fed79a0 con 0x56550fed5860 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f18efd700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee77d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00eea0 tx=0x7f8efc020c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1aa4f74e23a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8ef3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3024429519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc024050 con 0x56550fed5860 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f04003d10 con 0x56550fed5860 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3024429519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8efc033650 con 0x56550fed5860 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3024429519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8efc006d90 con 0x56550fed5860 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f259f7b80 10 client.17811 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.251+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56550ff596b0 con 0x56550fed5860 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3024429519 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f04003d10 con 0x56550fed5860 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3024429519 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8efc03d640 con 0x56550fed5860 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f259f7b80 10 client.17811 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f259f7b80 10 client.17811 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f259f7b80 20 client.17811 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550ff59d60 0x56550ff5c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f259f7b80 4 client.17811 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56550fe36be0 con 0x56550ff59d60 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.252+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550ff59d60 0x56550ff5c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f13fff700 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550ff59d60 0x56550ff5c180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6baa3441b7f2561d server_cookie=2083afb648fb9142 in_seq=0 out_seq=0 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8ef3fff700 10 client.17811 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8ef3fff700 1 -- 172.21.15.26:0/3024429519 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56550fe36be0 con 0x56550ff59d60 2024-05-20T21:04:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8ef3fff700 10 client.17811 handle_command_reply: tid=0 2024-05-20T21:04:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 client.17811 shutdown 2024-05-20T21:04:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 msgr2=0x56550fee77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee77d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00eea0 tx=0x7f8efc020c10 comp rx=0 tx=0).stop 2024-05-20T21:04:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 shutdown_connections 2024-05-20T21:04:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x56550ff59d60 0x56550ff5c180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56550fed5860 0x56550fee77d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56550fed9f60 0x56550fed7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8ef3fff700 0 client.17811 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 --2- 172.21.15.26:0/3024429519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56550fed6210 0x56550fed6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.253+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 >> 172.21.15.26:0/3024429519 conn(0x56550fe03080 msgr2=0x56550fed4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 shutdown_connections 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f259f7b80 1 -- 172.21.15.26:0/3024429519 wait complete. 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f259f7b80 20 client.17811 trim_cache size 0 max 16384 2024-05-20T21:04:15.258 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f000b2880 msgr2=0x7f8f000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f1b161700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f000b2880 0x7f8f000b4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8f1419aad0 tx=0x7f8efc009190 comp rx=0 tx=0).stop 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 msgr2=0x7f8f14199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.254+0000 7f8f1b161700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 0x7f8f14199990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c970 tx=0x7f8f0c00cc80 comp rx=0 tx=0).stop 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 shutdown_connections 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8f000b2880 0x7f8f000b4d40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f141095d0 0x7f8f14199990 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f1406bc10 0x7f8f1419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 --2- 172.21.15.26:0/1962035237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f1406c5d0 0x7f8f1419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 >> 172.21.15.26:0/1962035237 conn(0x7f8f140793f0 msgr2=0x7f8f14101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 shutdown_connections 2024-05-20T21:04:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:15.255+0000 7f8f1b161700 1 -- 172.21.15.26:0/1962035237 wait complete. 2024-05-20T21:04:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:15 smithi055 ceph-mon[35213]: pgmap v633: 97 pgs: 97 active+clean; 27 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:04:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:15 smithi144 ceph-mon[35660]: pgmap v633: 97 pgs: 97 active+clean; 27 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:04:15.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.37 MB/sec execute 261 sec latency 2413.931 ms 2024-05-20T21:04:16.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:04:16.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.36 MB/sec execute 262 sec latency 3414.037 ms 2024-05-20T21:04:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:17 smithi055 ceph-mon[35213]: pgmap v634: 97 pgs: 97 active+clean; 27 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-05-20T21:04:17.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:17 smithi026 ceph-mon[31990]: pgmap v634: 97 pgs: 97 active+clean; 27 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-05-20T21:04:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:17 smithi144 ceph-mon[35660]: pgmap v634: 97 pgs: 97 active+clean; 27 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-05-20T21:04:17.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.35 MB/sec execute 263 sec latency 4414.112 ms 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 -- 172.21.15.26:0/4159049583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 msgr2=0x7f7ebc101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/4159049583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc101dd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009900 tx=0x7f7eac02a410 comp rx=0 tx=0).stop 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 -- 172.21.15.26:0/4159049583 shutdown_connections 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/4159049583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 0x7f7ebc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/4159049583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ebc102ba0 0x7f7ebc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/4159049583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc101dd0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 -- 172.21.15.26:0/4159049583 >> 172.21.15.26:0/4159049583 conn(0x7f7ebc0fcf40 msgr2=0x7f7ebc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 -- 172.21.15.26:0/4159049583 shutdown_connections 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.327+0000 7f7ec0e75640 1 -- 172.21.15.26:0/4159049583 wait complete. 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 Processor -- start 2024-05-20T21:04:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 -- start start 2024-05-20T21:04:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ebc102ba0 0x7f7ebc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 0x7f7ebc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc10d390 con 0x7f7ebc1019b0 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc10d210 con 0x7f7ebc102ba0 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7ec0e75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc10d510 con 0x7f7ebc103560 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33404/0 (socket says 172.21.15.26:33404) 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7eba59c640 1 -- 172.21.15.26:0/1881571592 learned_addr learned my addr 172.21.15.26:0/1881571592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.328+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ebc102ba0 0x7f7ebc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 0x7f7ebc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eba59c640 1 -- 172.21.15.26:0/1881571592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 msgr2=0x7f7ebc19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eba59c640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 0x7f7ebc19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eba59c640 1 -- 172.21.15.26:0/1881571592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ebc102ba0 msgr2=0x7f7ebc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eba59c640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ebc102ba0 0x7f7ebc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eba59c640 1 -- 172.21.15.26:0/1881571592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc19f6b0 con 0x7f7ebc1019b0 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 0x7f7ebc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eba59c640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc197a90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005ab0 tx=0x7f7eac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eb37fe640 1 -- 172.21.15.26:0/1881571592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac03f030 con 0x7f7ebc1019b0 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eb37fe640 1 -- 172.21.15.26:0/1881571592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7eac003c90 con 0x7f7ebc1019b0 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc19f940 con 0x7f7ebc1019b0 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7eb37fe640 1 -- 172.21.15.26:0/1881571592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac035070 con 0x7f7ebc1019b0 2024-05-20T21:04:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.329+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc19fe30 con 0x7f7ebc1019b0 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 Processor -- start 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 -- start start 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7eb37fe640 1 -- 172.21.15.26:0/1881571592 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7eac042030 con 0x7f7ebc1019b0 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bc6c0 0x56059b1bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 0x56059b1c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56059b12dc30 con 0x56059b1c2220 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56059b11d7d0 con 0x56059b1bc6c0 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56059b121810 con 0x56059b1bd0b0 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7eb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 0x56059b1c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bc6c0 0x56059b1bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7eb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 0x56059b1c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33408/0 (socket says 172.21.15.26:33408) 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/3502296791 learned_addr learned my addr 172.21.15.26:0/3502296791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.332+0000 7f7eba59c640 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/3502296791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 msgr2=0x56059b1c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/3502296791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bc6c0 msgr2=0x56059b1bcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bc6c0 0x56059b1bcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/3502296791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059ae32210 con 0x56059b1c2220 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 0x56059b1c46b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400ed50 tx=0x7f7ea400a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc363c52c88f094e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb37fe640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ea80b28a0 0x7f7ea80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb0ff9640 1 -- 172.21.15.26:0/3502296791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea401e070 con 0x56059b1c2220 2024-05-20T21:04:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb0ff9640 1 -- 172.21.15.26:0/3502296791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ea400ad90 con 0x56059b1c2220 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb0ff9640 1 -- 172.21.15.26:0/3502296791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4017580 con 0x56059b1c2220 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ea80b28a0 0x7f7ea80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.333+0000 7f7eb37fe640 1 -- 172.21.15.26:0/1881571592 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7eac102d10 con 0x7f7ebc1019b0 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ea80b28a0 0x7f7ea80b4d60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56059b1bd910 tx=0x7f7eb4009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 -- 172.21.15.26:0/3502296791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 msgr2=0x56059b1c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 0x56059b1c46b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400ed50 tx=0x7f7ea400a680 comp rx=0 tx=0).stop 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 -- 172.21.15.26:0/3502296791 shutdown_connections 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1c2220 0x56059b1c46b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/3502296791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bc6c0 0x56059b1bcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 -- 172.21.15.26:0/3502296791 >> 172.21.15.26:0/3502296791 conn(0x56059b1b7d30 msgr2=0x56059b1b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 -- 172.21.15.26:0/3502296791 shutdown_connections 2024-05-20T21:04:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 -- 172.21.15.26:0/3502296791 wait complete. 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 -- start start 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.334+0000 7f7ec3b99740 1 Processor -- start 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56059ae03790 con 0x56059b1bc6c0 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56059b0d9890 con 0x56059b1c2220 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56059b1c51d0 con 0x56059b1bd0b0 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 10 client.0 _ll_register_callbacks cb 0x56059ae31a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33420/0 (socket says 172.21.15.26:33420) 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 -- 172.21.15.26:0/1974332007 learned_addr learned my addr 172.21.15.26:0/1974332007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 -- 172.21.15.26:0/1974332007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 msgr2=0x56059b1c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 -- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 msgr2=0x56059b1cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 -- 172.21.15.26:0/1974332007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059aefe620 con 0x56059b1bc6c0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7eba59c640 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1c67b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7eac004650 tx=0x7f7eac0c6250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512f62f7b7589bea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/1974332007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac049040 con 0x56059b1bc6c0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/1974332007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7eac0ca530 con 0x56059b1bc6c0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea400ad90 con 0x56059b1bc6c0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/1974332007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7eac032e60 con 0x56059b1bc6c0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 10 client.17823 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.335+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56059b1d0d10 con 0x56059b1bc6c0 2024-05-20T21:04:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/1974332007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ea400ad90 con 0x56059b1bc6c0 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/1974332007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7eac0d2ca0 con 0x56059b1bc6c0 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ec3b99740 10 client.17823 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ec3b99740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ec3b99740 20 client.17823 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b1d24e0 0x56059b1d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ec3b99740 4 client.17823 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059b12fb80 con 0x56059b1d24e0 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.336+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b1d24e0 0x56059b1d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b1d24e0 0x56059b1d4980 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175e26baef093cad server_cookie=2123a4fd51bbcd6c in_seq=0 out_seq=0 2024-05-20T21:04:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ea1ffb640 10 client.17823 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/1974332007 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56059b12fb80 con 0x56059b1d24e0 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ea1ffb640 10 client.17823 handle_command_reply: tid=0 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ec3b99740 1 client.17823 shutdown 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 msgr2=0x56059b1c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1c67b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7eac004650 tx=0x7f7eac0c6250 comp rx=0 tx=0).stop 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.337+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 shutdown_connections 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b1d24e0 0x56059b1d4980 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bd0b0 0x56059b1c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ea1ffb640 0 client.17823 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/1974332007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1c67b0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 >> 172.21.15.26:0/1974332007 conn(0x56059b1b7d30 msgr2=0x56059b1b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 shutdown_connections 2024-05-20T21:04:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 1 -- 172.21.15.26:0/1974332007 wait complete. 2024-05-20T21:04:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.338+0000 7f7ec3b99740 20 client.17823 trim_cache size 0 max 16384 2024-05-20T21:04:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 Processor -- start 2024-05-20T21:04:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 -- start start 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bd0b0 0x56059b1cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1c2220 0x56059b1c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1aa5a0 con 0x56059b1bc6c0 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1a7e70 con 0x56059b1bd0b0 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1a5cc0 con 0x56059b1c2220 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33428/0 (socket says 172.21.15.26:33428) 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eba59c640 1 -- 172.21.15.26:0/4251941056 learned_addr learned my addr 172.21.15.26:0/4251941056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eba59c640 1 -- 172.21.15.26:0/4251941056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1c2220 msgr2=0x56059b1c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1c2220 0x56059b1c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eba59c640 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1c2220 0x56059b1c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eba59c640 1 -- 172.21.15.26:0/4251941056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bd0b0 msgr2=0x56059b1cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.353+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bd0b0 0x56059b1cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7eba59c640 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bd0b0 0x56059b1cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7eba59c640 1 -- 172.21.15.26:0/4251941056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059b1b7b60 con 0x56059b1bc6c0 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7eba59c640 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1b2e40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000c00 tx=0x7f7eac1064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cb94a837ec41d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ea3fff640 1 -- 172.21.15.26:0/4251941056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac049070 con 0x56059b1bc6c0 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ea3fff640 1 -- 172.21.15.26:0/4251941056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7eac003e40 con 0x56059b1bc6c0 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ea3fff640 1 -- 172.21.15.26:0/4251941056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ca9d0 con 0x56059b1bc6c0 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 -- 172.21.15.26:0/4251941056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 msgr2=0x56059b1b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1b2e40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000c00 tx=0x7f7eac1064e0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 -- 172.21.15.26:0/4251941056 shutdown_connections 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1c2220 0x56059b1c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1bd0b0 0x56059b1cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/4251941056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bc6c0 0x56059b1b2e40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 -- 172.21.15.26:0/4251941056 >> 172.21.15.26:0/4251941056 conn(0x56059b1b7d30 msgr2=0x56059b1b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 -- 172.21.15.26:0/4251941056 shutdown_connections 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.354+0000 7f7ec3b99740 1 -- 172.21.15.26:0/4251941056 wait complete. 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 -- start start 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 Processor -- start 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bc6c0 0x56059b1cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 0x56059b1cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1a6330 con 0x56059b1bd0b0 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1a77a0 con 0x56059b1c2220 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1a6a00 con 0x56059b1bc6c0 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ec3b99740 10 client.0 _ll_register_callbacks cb 0x7f7eac106300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 0x56059b1cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 0x56059b1cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33442/0 (socket says 172.21.15.26:33442) 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/2709074841 learned_addr learned my addr 172.21.15.26:0/2709074841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eba59c640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bc6c0 0x56059b1cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/2709074841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bc6c0 msgr2=0x56059b1cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bc6c0 0x56059b1cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 msgr2=0x56059b1cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7eb9d9b640 1 -- 172.21.15.26:0/2709074841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059b1bdde0 con 0x56059b1bd0b0 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.355+0000 7f7ebad9d640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7eb9d9b640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 0x56059b1cb8d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400a6c0 tx=0x7f7ea4005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0cc648b92c205c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/2709074841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea401e040 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/2709074841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7ea4014040 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56059b129360 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/2709074841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ea4036050 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 10 client.17829 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56059b23d160 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/2709074841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56059b129360 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/2709074841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7ea4004860 con 0x56059b1bd0b0 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 10 client.17829 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 20 client.17829 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b23d760 0x56059b23fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 4 client.17829 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:04:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.356+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56059b0a35f0 con 0x56059b23d760 2024-05-20T21:04:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.357+0000 7f7eba59c640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b23d760 0x56059b23fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.357+0000 7f7ea1ffb640 10 client.17829 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.357+0000 7f7eba59c640 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b23d760 0x56059b23fb50 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f91574bffaf8200 server_cookie=54adccbfea04ccf5 in_seq=0 out_seq=0 2024-05-20T21:04:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ea1ffb640 1 -- 172.21.15.26:0/2709074841 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56059b0a35f0 con 0x56059b23d760 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ea1ffb640 10 client.17829 handle_command_reply: tid=0 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 client.17829 shutdown 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 msgr2=0x56059b1cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 0x56059b1cb8d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400a6c0 tx=0x7f7ea4005d00 comp rx=0 tx=0).stop 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 shutdown_connections 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56059b23d760 0x56059b23fb50 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56059b1c2220 0x56059b1cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56059b1bd0b0 0x56059b1cb8d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 --2- 172.21.15.26:0/2709074841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56059b1bc6c0 0x56059b1cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ea1ffb640 0 client.17829 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 >> 172.21.15.26:0/2709074841 conn(0x56059b1b7d30 msgr2=0x56059b1b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 shutdown_connections 2024-05-20T21:04:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 1 -- 172.21.15.26:0/2709074841 wait complete. 2024-05-20T21:04:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.390+0000 7f7ec3b99740 20 client.17829 trim_cache size 0 max 16384 2024-05-20T21:04:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:04:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:04:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ea80b28a0 msgr2=0x7f7ea80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ea80b28a0 0x7f7ea80b4d60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56059b1bd910 tx=0x7f7eb4009700 comp rx=0 tx=0).stop 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 msgr2=0x7f7ebc197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc197a90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005ab0 tx=0x7f7eac0042f0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 shutdown_connections 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7ea80b28a0 0x7f7ea80b4d60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ebc103560 0x7f7ebc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ebc102ba0 0x7f7ebc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 --2- 172.21.15.26:0/1881571592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc1019b0 0x7f7ebc197a90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 >> 172.21.15.26:0/1881571592 conn(0x7f7ebc0fcf40 msgr2=0x7f7ebc105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 shutdown_connections 2024-05-20T21:04:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:18.391+0000 7f7ec0e75640 1 -- 172.21.15.26:0/1881571592 wait complete. 2024-05-20T21:04:18.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.35 MB/sec execute 264 sec latency 5414.146 ms 2024-05-20T21:04:18.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 -- 172.21.15.26:0/2005008796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806bcf0 msgr2=0x7fe1e806c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/2005008796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806bcf0 0x7fe1e806c0d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400b0a0 tx=0x7fe1e400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 -- 172.21.15.26:0/2005008796 shutdown_connections 2024-05-20T21:04:18.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/2005008796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806bcf0 0x7fe1e806c0d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/2005008796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e810ff90 0x7fe1e8112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/2005008796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e806c610 0x7fe1e810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 -- 172.21.15.26:0/2005008796 >> 172.21.15.26:0/2005008796 conn(0x7fe1e80794e0 msgr2=0x7fe1e80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 -- 172.21.15.26:0/2005008796 shutdown_connections 2024-05-20T21:04:18.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 -- 172.21.15.26:0/2005008796 wait complete. 2024-05-20T21:04:18.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 Processor -- start 2024-05-20T21:04:18.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.649+0000 7fe1ef3be700 1 -- start start 2024-05-20T21:04:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ef3be700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e806bcf0 0x7fe1e819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ef3be700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 0x7fe1e819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ef3be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e810ff90 0x7fe1e81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ef3be700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8115810 con 0x7fe1e806c610 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ef3be700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8115690 con 0x7fe1e810ff90 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ef3be700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8115990 con 0x7fe1e806bcf0 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 0x7fe1e819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 0x7fe1e819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33464/0 (socket says 172.21.15.26:33464) 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 -- 172.21.15.26:0/1609329469 learned_addr learned my addr 172.21.15.26:0/1609329469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ed15a700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e806bcf0 0x7fe1e819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ed95b700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e810ff90 0x7fe1e81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 -- 172.21.15.26:0/1609329469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e806bcf0 msgr2=0x7fe1e819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e806bcf0 0x7fe1e819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 -- 172.21.15.26:0/1609329469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e810ff90 msgr2=0x7fe1e81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e810ff90 0x7fe1e81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 -- 172.21.15.26:0/1609329469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e81a0d80 con 0x7fe1e806c610 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ed95b700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e810ff90 0x7fe1e81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1ec959700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 0x7fe1e819c300 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800d950 tx=0x7fe1d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.650+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8016070 con 0x7fe1e806c610 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.651+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e81a1010 con 0x7fe1e806c610 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.651+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1d8009dd0 con 0x7fe1e806c610 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.651+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800f3c0 con 0x7fe1e806c610 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.651+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e81a88f0 con 0x7fe1e806c610 2024-05-20T21:04:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.652+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e806a230 con 0x7fe1e806c610 2024-05-20T21:04:18.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.653+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe1d8019070 con 0x7fe1e806c610 2024-05-20T21:04:18.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.654+0000 7fe1de7fc700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1d40b2880 0x7fe1d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:18.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.654+0000 7fe1ed15a700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1d40b2880 0x7fe1d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:18.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.654+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe1d80e1f40 con 0x7fe1e806c610 2024-05-20T21:04:18.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.654+0000 7fe1ed15a700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1d40b2880 0x7fe1d40b4d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009e60 tx=0x7fe1e400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.658+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1d80a9ba0 con 0x7fe1e806c610 2024-05-20T21:04:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.934+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1e802e560 con 0x7fe1e806c610 2024-05-20T21:04:18.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.935+0000 7fe1de7fc700 1 -- 172.21.15.26:0/1609329469 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fe1d80a94a0 con 0x7fe1e806c610 2024-05-20T21:04:18.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:18.935 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1d40b2880 msgr2=0x7fe1d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1d40b2880 0x7fe1d40b4d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009e60 tx=0x7fe1e400b7b0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 msgr2=0x7fe1e819c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 0x7fe1e819c300 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800d950 tx=0x7fe1d800dc60 comp rx=0 tx=0).stop 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 shutdown_connections 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1e810ff90 0x7fe1e81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e806bcf0 0x7fe1e819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1d40b2880 0x7fe1d40b4d40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.937+0000 7fe1ef3be700 1 --2- 172.21.15.26:0/1609329469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e806c610 0x7fe1e819c300 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.938+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 >> 172.21.15.26:0/1609329469 conn(0x7fe1e80794e0 msgr2=0x7fe1e8111630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.938+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 shutdown_connections 2024-05-20T21:04:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:18.938+0000 7fe1ef3be700 1 -- 172.21.15.26:0/1609329469 wait complete. 2024-05-20T21:04:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:04:19.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:19 smithi055 ceph-mon[35213]: pgmap v635: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-20T21:04:19.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:19 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1609329469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:19.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:19 smithi026 ceph-mon[31990]: pgmap v635: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-20T21:04:19.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:19 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1609329469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:19 smithi144 ceph-mon[35660]: pgmap v635: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-20T21:04:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:19 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1609329469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:19.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.34 MB/sec execute 265 sec latency 6414.196 ms 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.734+0000 7f9d9170d700 1 -- 172.21.15.26:0/1456528090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 msgr2=0x7f9d8c06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.734+0000 7f9d9170d700 1 --2- 172.21.15.26:0/1456528090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c06c0f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009990 tx=0x7f9d80009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 -- 172.21.15.26:0/1456528090 shutdown_connections 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 --2- 172.21.15.26:0/1456528090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8c06c630 0x7f9d8c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 --2- 172.21.15.26:0/1456528090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d8c1095b0 0x7f9d8c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 --2- 172.21.15.26:0/1456528090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c06c0f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 -- 172.21.15.26:0/1456528090 >> 172.21.15.26:0/1456528090 conn(0x7f9d8c0793f0 msgr2=0x7f9d8c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 -- 172.21.15.26:0/1456528090 shutdown_connections 2024-05-20T21:04:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 -- 172.21.15.26:0/1456528090 wait complete. 2024-05-20T21:04:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 Processor -- start 2024-05-20T21:04:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.735+0000 7f9d9170d700 1 -- start start 2024-05-20T21:04:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d9170d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d9170d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8c06c630 0x7f9d8c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d9170d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d8c1095b0 0x7f9d8c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d9170d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1135d0 con 0x7f9d8c06bc70 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d9170d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c113450 con 0x7f9d8c1095b0 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d9170d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c113750 con 0x7f9d8c06c630 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43332/0 (socket says 172.21.15.26:43332) 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 -- 172.21.15.26:0/2570397103 learned_addr learned my addr 172.21.15.26:0/2570397103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d8c1095b0 0x7f9d8c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8c06c630 0x7f9d8c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 -- 172.21.15.26:0/2570397103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8c06c630 msgr2=0x7f9d8c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8c06c630 0x7f9d8c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 -- 172.21.15.26:0/2570397103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d8c1095b0 msgr2=0x7f9d8c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d8c1095b0 0x7f9d8c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 -- 172.21.15.26:0/2570397103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c19a070 con 0x7f9d8c06bc70 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.736+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c19f750 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00c9b0 tx=0x7f9d7c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.737+0000 7f9d73fff700 1 -- 172.21.15.26:0/2570397103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c007b00 con 0x7f9d8c06bc70 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.737+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c19a360 con 0x7f9d8c06bc70 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.737+0000 7f9d73fff700 1 -- 172.21.15.26:0/2570397103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7c007ca0 con 0x7f9d8c06bc70 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.737+0000 7f9d73fff700 1 -- 172.21.15.26:0/2570397103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c018660 con 0x7f9d8c06bc70 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.737+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a6620 con 0x7f9d8c06bc70 2024-05-20T21:04:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.739+0000 7f9d73fff700 1 -- 172.21.15.26:0/2570397103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9d7c00f450 con 0x7f9d8c06bc70 2024-05-20T21:04:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d73fff700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d740b2880 0x7f9d740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d740b2880 0x7f9d740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d73fff700 1 -- 172.21.15.26:0/2570397103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9d7c0e2260 con 0x7f9d8c06bc70 2024-05-20T21:04:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 Processor -- start 2024-05-20T21:04:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 -- start start 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d740b2880 0x7f9d740b4d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005f50 tx=0x7f9d80005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a394050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a397d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a3987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c6a2eec90 con 0x563c6a393c70 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563c6a2de830 con 0x563c6a394620 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563c6a2e2870 con 0x563c6a398370 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.741+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a394050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a394050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43344/0 (socket says 172.21.15.26:43344) 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 -- 172.21.15.26:0/326575930 learned_addr learned my addr 172.21.15.26:0/326575930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a397d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a3987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 -- 172.21.15.26:0/326575930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 msgr2=0x563c6a3987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a3987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 -- 172.21.15.26:0/326575930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 msgr2=0x563c6a397d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a397d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 -- 172.21.15.26:0/326575930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6a22e8d0 con 0x563c6a393c70 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d8affd700 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a394050 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0a58a0 tx=0x7f9d7c0a9090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0b6af2d30aa508 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d717fa700 1 -- 172.21.15.26:0/326575930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0a9f20 con 0x563c6a393c70 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d717fa700 1 -- 172.21.15.26:0/326575930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7c0a9820 con 0x563c6a393c70 2024-05-20T21:04:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.742+0000 7f9d717fa700 1 -- 172.21.15.26:0/326575930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0a9a80 con 0x563c6a393c70 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/326575930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 msgr2=0x563c6a394050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a394050 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0a58a0 tx=0x7f9d7c0a9090 comp rx=0 tx=0).stop 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/326575930 shutdown_connections 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a394050 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a3987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/326575930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a397d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/326575930 >> 172.21.15.26:0/326575930 conn(0x563c6a2bd080 msgr2=0x563c6a389e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/326575930 shutdown_connections 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/326575930 wait complete. 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.743+0000 7f9d9bfa3b80 1 -- start start 2024-05-20T21:04:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 Processor -- start 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a393c70 0x563c6a39f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a39cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 0x563c6a39d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c6a1e4fa0 con 0x563c6a398370 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563c6a2807c0 con 0x563c6a394620 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563c6a3a0530 con 0x563c6a393c70 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d9bfa3b80 10 client.0 _ll_register_callbacks cb 0x563c6a279920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 0x563c6a39d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 0x563c6a39d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43352/0 (socket says 172.21.15.26:43352) 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 -- 172.21.15.26:0/2786536071 learned_addr learned my addr 172.21.15.26:0/2786536071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a393c70 0x563c6a39f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a39cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 -- 172.21.15.26:0/2786536071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a393c70 msgr2=0x563c6a39f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a393c70 0x563c6a39f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 -- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 msgr2=0x563c6a39cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a39cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.744+0000 7f9d8b7fe700 1 -- 172.21.15.26:0/2786536071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6a27d690 con 0x563c6a398370 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 0x563c6a39d1b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9d84011d00 tx=0x7f9d8400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322104dfe30abf78 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2786536071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84019070 con 0x563c6a398370 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2786536071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9d8400fa10 con 0x563c6a398370 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c0a9820 con 0x563c6a398370 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2786536071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d84025050 con 0x563c6a398370 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 10 client.17847 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c6a3aae60 con 0x563c6a398370 2024-05-20T21:04:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2786536071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7c0a9820 con 0x563c6a398370 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2786536071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9d84015c60 con 0x563c6a398370 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 10 client.17847 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 20 client.17847 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.745+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a3ad440 0x563c6a3af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.746+0000 7f9d9bfa3b80 4 client.17847 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.746+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6a35b9e0 con 0x563c6a3ad440 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.746+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a3ad440 0x563c6a3af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.746+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a3ad440 0x563c6a3af830 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70802e6e9cd1b315 server_cookie=eb8ec4d0a5ff5987 in_seq=0 out_seq=0 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.746+0000 7f9d6eff5700 10 client.17847 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2786536071 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563c6a35b9e0 con 0x563c6a3ad440 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d6eff5700 10 client.17847 handle_command_reply: tid=0 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 client.17847 shutdown 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 msgr2=0x563c6a39d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 0x563c6a39d1b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9d84011d00 tx=0x7f9d8400f4b0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 shutdown_connections 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a3ad440 0x563c6a3af830 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a398370 0x563c6a39d1b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a393c70 0x563c6a39f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2786536071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a394620 0x563c6a39cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 >> 172.21.15.26:0/2786536071 conn(0x563c6a2bd080 msgr2=0x563c6a389510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d6eff5700 0 client.17847 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 shutdown_connections 2024-05-20T21:04:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.747+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2786536071 wait complete. 2024-05-20T21:04:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.748+0000 7f9d9bfa3b80 20 client.17847 trim_cache size 0 max 16384 2024-05-20T21:04:19.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 Processor -- start 2024-05-20T21:04:19.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 -- start start 2024-05-20T21:04:19.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a389c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a394620 0x563c6a38a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a398370 0x563c6a39a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ae390 con 0x563c6a393c70 2024-05-20T21:04:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1abc20 con 0x563c6a398370 2024-05-20T21:04:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1a9a70 con 0x563c6a394620 2024-05-20T21:04:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a389c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a389c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43364/0 (socket says 172.21.15.26:43364) 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8affd700 1 -- 172.21.15.26:0/3155128113 learned_addr learned my addr 172.21.15.26:0/3155128113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a394620 0x563c6a38a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8affd700 1 -- 172.21.15.26:0/3155128113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a394620 msgr2=0x563c6a38a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8affd700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a394620 0x563c6a38a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.763+0000 7f9d8affd700 1 -- 172.21.15.26:0/3155128113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a398370 msgr2=0x563c6a39a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a398370 0x563c6a39a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d8affd700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a398370 0x563c6a39a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d8affd700 1 -- 172.21.15.26:0/3155128113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6a3980c0 con 0x563c6a393c70 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a394620 0x563c6a38a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a398370 0x563c6a39a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d8affd700 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a389c90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0a97b0 tx=0x7f9d7c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f892b856f1393c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d70ff9700 1 -- 172.21.15.26:0/3155128113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b2680 con 0x563c6a393c70 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d70ff9700 1 -- 172.21.15.26:0/3155128113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7c0a9f20 con 0x563c6a393c70 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.764+0000 7f9d70ff9700 1 -- 172.21.15.26:0/3155128113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b21f0 con 0x563c6a393c70 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/3155128113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 msgr2=0x563c6a389c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a389c90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0a97b0 tx=0x7f9d7c0cc040 comp rx=0 tx=0).stop 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/3155128113 shutdown_connections 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a393c70 0x563c6a389c90 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a398370 0x563c6a39a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/3155128113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a394620 0x563c6a38a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/3155128113 >> 172.21.15.26:0/3155128113 conn(0x563c6a2bd080 msgr2=0x563c6a38ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/3155128113 shutdown_connections 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/3155128113 wait complete. 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 -- start start 2024-05-20T21:04:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.765+0000 7f9d9bfa3b80 1 Processor -- start 2024-05-20T21:04:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a393c70 0x563c6a3a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 0x563c6a3a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a415fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1aa0e0 con 0x563c6a394620 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ab550 con 0x563c6a393c70 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1aa7b0 con 0x563c6a398370 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d9bfa3b80 10 client.0 _ll_register_callbacks cb 0x563c6a279920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 0x563c6a3a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 0x563c6a3a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43372/0 (socket says 172.21.15.26:43372) 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 -- 172.21.15.26:0/2411959664 learned_addr learned my addr 172.21.15.26:0/2411959664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a393c70 0x563c6a3a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a415fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 -- 172.21.15.26:0/2411959664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 msgr2=0x563c6a415fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a415fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 -- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a393c70 msgr2=0x563c6a3a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a393c70 0x563c6a3a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 -- 172.21.15.26:0/2411959664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6a4164f0 con 0x563c6a394620 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8b7fe700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a415fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a393c70 0x563c6a3a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d8a7fc700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 0x563c6a3a3090 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c079a60 tx=0x7f9d80009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d4e00b9630005c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.766+0000 7f9d6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2411959664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80036060 con 0x563c6a394620 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2411959664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9d8000bb60 con 0x563c6a394620 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c0a9f20 con 0x563c6a394620 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2411959664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d8000bd60 con 0x563c6a394620 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 10 client.17853 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c6a4176c0 con 0x563c6a394620 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2411959664 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7c0a9f20 con 0x563c6a394620 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2411959664 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9d80030030 con 0x563c6a394620 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 10 client.17853 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 20 client.17853 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a417dd0 0x563c6a41a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 4 client.17853 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.767+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c6a2f0be0 con 0x563c6a417dd0 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a417dd0 0x563c6a41a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d8affd700 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a417dd0 0x563c6a41a1f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c1c2a2c89f13c4 server_cookie=bf3bb936d70d803f in_seq=0 out_seq=0 2024-05-20T21:04:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d6eff5700 10 client.17853 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d6eff5700 1 -- 172.21.15.26:0/2411959664 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c6a2f0be0 con 0x563c6a417dd0 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d6eff5700 10 client.17853 handle_command_reply: tid=0 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 client.17853 shutdown 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 msgr2=0x563c6a3a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 0x563c6a3a3090 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c079a60 tx=0x7f9d80009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 shutdown_connections 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563c6a417dd0 0x563c6a41a1f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c6a398370 0x563c6a415fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c6a393c70 0x563c6a3a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 --2- 172.21.15.26:0/2411959664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c6a394620 0x563c6a3a3090 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.768+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 >> 172.21.15.26:0/2411959664 conn(0x563c6a2bd080 msgr2=0x563c6a396cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d6eff5700 0 client.17853 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 shutdown_connections 2024-05-20T21:04:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9bfa3b80 1 -- 172.21.15.26:0/2411959664 wait complete. 2024-05-20T21:04:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9bfa3b80 20 client.17853 trim_cache size 0 max 16384 2024-05-20T21:04:19.775 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d740b2880 msgr2=0x7f9d740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d740b2880 0x7f9d740b4d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005f50 tx=0x7f9d80005e80 comp rx=0 tx=0).stop 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 msgr2=0x7f9d8c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c19f750 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00c9b0 tx=0x7f9d7c00cd70 comp rx=0 tx=0).stop 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 shutdown_connections 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9d740b2880 0x7f9d740b4d40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8c06bc70 0x7f9d8c19f750 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d8c1095b0 0x7f9d8c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 --2- 172.21.15.26:0/2570397103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8c06c630 0x7f9d8c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.769+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 >> 172.21.15.26:0/2570397103 conn(0x7f9d8c0793f0 msgr2=0x7f9d8c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.770+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 shutdown_connections 2024-05-20T21:04:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:19.770+0000 7f9d9170d700 1 -- 172.21.15.26:0/2570397103 wait complete. 2024-05-20T21:04:20.566 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id a6088460 --release --await' 2024-05-20T21:04:20.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id a6088460 --release --await 2024-05-20T21:04:20.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.33 MB/sec execute 266 sec latency 7414.228 ms 2024-05-20T21:04:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:21 smithi055 ceph-mon[35213]: pgmap v636: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 984 KiB/s wr, 65 op/s 2024-05-20T21:04:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:21 smithi026 ceph-mon[31990]: pgmap v636: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 984 KiB/s wr, 65 op/s 2024-05-20T21:04:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:21 smithi144 ceph-mon[35660]: pgmap v636: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 984 KiB/s wr, 65 op/s 2024-05-20T21:04:21.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.32 MB/sec execute 267 sec latency 8414.315 ms 2024-05-20T21:04:22.261 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:04:22.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:04:22.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.31 MB/sec execute 268 sec latency 9414.342 ms 2024-05-20T21:04:22.707 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:23.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:23 smithi026 ceph-mon[31990]: pgmap v637: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 828 KiB/s wr, 59 op/s 2024-05-20T21:04:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:23 smithi055 ceph-mon[35213]: pgmap v637: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 828 KiB/s wr, 59 op/s 2024-05-20T21:04:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:23 smithi144 ceph-mon[35660]: pgmap v637: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 828 KiB/s wr, 59 op/s 2024-05-20T21:04:23.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180690 2.30 MB/sec execute 269 sec latency 10414.409 ms 2024-05-20T21:04:24.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 -- 172.21.15.26:0/136742619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c8109630 msgr2=0x7fa8c8109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/136742619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c8109630 0x7fa8c8109a10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80099c0 tx=0x7fa8b8009cd0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 -- 172.21.15.26:0/136742619 shutdown_connections 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/136742619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c8109630 0x7fa8c8109a10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/136742619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8c806c6d0 0x7fa8c8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/136742619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c806bd10 0x7fa8c806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 -- 172.21.15.26:0/136742619 >> 172.21.15.26:0/136742619 conn(0x7fa8c80794e0 msgr2=0x7fa8c80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 -- 172.21.15.26:0/136742619 shutdown_connections 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.431+0000 7fa8cde4d700 1 -- 172.21.15.26:0/136742619 wait complete. 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 Processor -- start 2024-05-20T21:04:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 -- start start 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c806bd10 0x7fa8c819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 0x7fa8c819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8c8109630 0x7fa8c8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8113600 con 0x7fa8c806c6d0 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8113480 con 0x7fa8c8109630 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8cde4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8113780 con 0x7fa8c806bd10 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 0x7fa8c819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 0x7fa8c819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43378/0 (socket says 172.21.15.26:43378) 2024-05-20T21:04:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/4003808220 learned_addr learned my addr 172.21.15.26:0/4003808220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8c7fff700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8c8109630 0x7fa8c8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.432+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c806bd10 0x7fa8c819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/4003808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c806bd10 msgr2=0x7fa8c819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c806bd10 0x7fa8c819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/4003808220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8c8109630 msgr2=0x7fa8c8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8c8109630 0x7fa8c8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/4003808220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c819a060 con 0x7fa8c806c6d0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 0x7fa8c819fc80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00d980 tx=0x7fa8bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c4ff9700 1 -- 172.21.15.26:0/4003808220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc017070 con 0x7fa8c806c6d0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c4ff9700 1 -- 172.21.15.26:0/4003808220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc005020 con 0x7fa8c806c6d0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c819a350 con 0x7fa8c806c6d0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8c4ff9700 1 -- 172.21.15.26:0/4003808220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00f980 con 0x7fa8c806c6d0 2024-05-20T21:04:24.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.433+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81a6610 con 0x7fa8c806c6d0 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.436+0000 7fa8c4ff9700 1 -- 172.21.15.26:0/4003808220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa8bc004a30 con 0x7fa8c806c6d0 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.436+0000 7fa8c4ff9700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8b00b2880 0x7fa8b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.436+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8b00b2880 0x7fa8b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.436+0000 7fa8c4ff9700 1 -- 172.21.15.26:0/4003808220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa8bc0e2040 con 0x7fa8c806c6d0 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 Processor -- start 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 -- start start 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8b00b2880 0x7fa8b00b4d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009990 tx=0x7fa8b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd9c0 0x55a4941e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a494136e70 con 0x55a4941dd000 2024-05-20T21:04:24.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a494126a10 con 0x55a4941e1b30 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a49412aa50 con 0x55a4941dd9c0 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941dd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43392/0 (socket says 172.21.15.26:43392) 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/1278638144 learned_addr learned my addr 172.21.15.26:0/1278638144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8c7fff700 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd9c0 0x55a4941e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.437+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/1278638144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd9c0 msgr2=0x55a4941e9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd9c0 0x55a4941e9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/1278638144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 msgr2=0x55a4941e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/1278638144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4940d00b0 con 0x55a4941dd000 2024-05-20T21:04:24.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941dd480 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0a93a0 tx=0x7fa8bc004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25636fd4b0f8ba93 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:24.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8adffb700 1 -- 172.21.15.26:0/1278638144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0a96a0 con 0x55a4941dd000 2024-05-20T21:04:24.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8adffb700 1 -- 172.21.15.26:0/1278638144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc0ada50 con 0x55a4941dd000 2024-05-20T21:04:24.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.438+0000 7fa8adffb700 1 -- 172.21.15.26:0/1278638144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0b2070 con 0x55a4941dd000 2024-05-20T21:04:24.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/1278638144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 msgr2=0x55a4941dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941dd480 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0a93a0 tx=0x7fa8bc004e50 comp rx=0 tx=0).stop 2024-05-20T21:04:24.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/1278638144 shutdown_connections 2024-05-20T21:04:24.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd9c0 0x55a4941e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/1278638144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941dd480 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/1278638144 >> 172.21.15.26:0/1278638144 conn(0x55a494104340 msgr2=0x55a4941d38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/1278638144 shutdown_connections 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/1278638144 wait complete. 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 -- start start 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.439+0000 7fa8d86e3b80 1 Processor -- start 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd000 0x55a4941e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a49402d0a0 con 0x55a4941dd9c0 2024-05-20T21:04:24.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a49408eea0 con 0x55a4941e1b30 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a4941e9fd0 con 0x55a4941dd000 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 10 client.0 _ll_register_callbacks cb 0x55a4940c1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941e69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43394/0 (socket says 172.21.15.26:43394) 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/455107772 learned_addr learned my addr 172.21.15.26:0/455107772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd000 0x55a4941e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c7fff700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/455107772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd000 msgr2=0x55a4941e96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd000 0x55a4941e96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 msgr2=0x55a4941e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/455107772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4940769d0 con 0x55a4941dd9c0 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd000 0x55a4941e96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941e69a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0a9350 tx=0x7fa8bc0a9f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241beed2bb4968af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/455107772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0c8060 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/455107772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa8bc0b2070 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.440+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8bc0ada50 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/455107772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc088050 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 10 client.17865 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4940e0860 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/455107772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc0ada50 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/455107772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fa8bc06d030 con 0x55a4941dd9c0 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 10 client.17865 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 10 client.17865 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:04:24.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 20 client.17865 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a4941f69b0 0x55a4941f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 4 client.17865 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a494138dc0 con 0x55a4941f69b0 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.441+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a4941f69b0 0x55a4941f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.442+0000 7fa8a6ffd700 10 client.17865 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.442+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a4941f69b0 0x55a4941f8da0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b914f11cb772546c server_cookie=d3bfdb88e2cffa4b in_seq=0 out_seq=0 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/455107772 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a494138dc0 con 0x55a4941f69b0 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8a6ffd700 10 client.17865 handle_command_reply: tid=0 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 client.17865 shutdown 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 msgr2=0x55a4941e69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941e69a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0a9350 tx=0x7fa8bc0a9f10 comp rx=0 tx=0).stop 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 shutdown_connections 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a4941f69b0 0x55a4941f8da0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941e1b30 0x55a4941e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941dd000 0x55a4941e96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/455107772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941e69a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8a6ffd700 0 client.17865 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 >> 172.21.15.26:0/455107772 conn(0x55a494104340 msgr2=0x55a4941dbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 shutdown_connections 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/455107772 wait complete. 2024-05-20T21:04:24.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.443+0000 7fa8d86e3b80 20 client.17865 trim_cache size 0 max 16384 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 Processor -- start 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 -- start start 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd9c0 0x55a4941e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81ae310 con 0x55a4941dd000 2024-05-20T21:04:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81abba0 con 0x55a4941dd9c0 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81a99f0 con 0x55a4941e1b30 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43406/0 (socket says 172.21.15.26:43406) 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c77fe700 1 -- 172.21.15.26:0/920642899 learned_addr learned my addr 172.21.15.26:0/920642899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c7fff700 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd9c0 0x55a4941e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c77fe700 1 -- 172.21.15.26:0/920642899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 msgr2=0x55a4941e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.464+0000 7fa8c77fe700 1 -- 172.21.15.26:0/920642899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd9c0 msgr2=0x55a4941e0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.465+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd9c0 0x55a4941e0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.465+0000 7fa8c77fe700 1 -- 172.21.15.26:0/920642899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4941e9ea0 con 0x55a4941dd000 2024-05-20T21:04:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.465+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941cde70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b580 tx=0x7fa8b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256ba7aa67eb1ed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.465+0000 7fa8ad7fa700 1 -- 172.21.15.26:0/920642899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003680 con 0x55a4941dd000 2024-05-20T21:04:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.465+0000 7fa8ad7fa700 1 -- 172.21.15.26:0/920642899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8b8036070 con 0x55a4941dd000 2024-05-20T21:04:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.465+0000 7fa8ad7fa700 1 -- 172.21.15.26:0/920642899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80377c0 con 0x55a4941dd000 2024-05-20T21:04:24.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/920642899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 msgr2=0x55a4941cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941cde70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b580 tx=0x7fa8b801f000 comp rx=0 tx=0).stop 2024-05-20T21:04:24.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/920642899 shutdown_connections 2024-05-20T21:04:24.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd000 0x55a4941cde70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/920642899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd9c0 0x55a4941e0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/920642899 >> 172.21.15.26:0/920642899 conn(0x55a494104340 msgr2=0x55a4941dc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/920642899 shutdown_connections 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/920642899 wait complete. 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 -- start start 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.466+0000 7fa8d86e3b80 1 Processor -- start 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd000 0x55a4941fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81aa060 con 0x55a4941dd9c0 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81ab4d0 con 0x55a4941dd000 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81aa730 con 0x55a4941e1b30 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8d86e3b80 10 client.0 _ll_register_callbacks cb 0x55a4940c1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43410/0 (socket says 172.21.15.26:43410) 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/3847930997 learned_addr learned my addr 172.21.15.26:0/3847930997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c7fff700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd000 0x55a4941fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/3847930997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 msgr2=0x55a4941f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd000 msgr2=0x55a4941fc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd000 0x55a4941fc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 -- 172.21.15.26:0/3847930997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4941f9950 con 0x55a4941dd9c0 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd000 0x55a4941fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8c6ffd700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941f8da0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0cc170 tx=0x7fa8bc0a9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e85df505413cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.467+0000 7fa8a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/3847930997 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0912c0 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b8036070 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/3847930997 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa8bc0b2070 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/3847930997 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc0c8070 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 10 client.17871 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4942620c0 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/3847930997 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8b8036070 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/3847930997 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fa8bc06d040 con 0x55a4941dd9c0 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 10 client.17871 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 10 client.17871 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 20 client.17871 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a494262520 0x55a494264910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 4 client.17871 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a6088460", "release": true, "await": true}] 2024-05-20T21:04:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a6088460", "release": true, "await": true}) -- 0x55a4941325a0 con 0x55a494262520 2024-05-20T21:04:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.468+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a494262520 0x55a494264910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.469+0000 7fa8a6ffd700 10 client.17871 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.469+0000 7fa8c77fe700 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a494262520 0x55a494264910 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d311bc47167e37c server_cookie=77a64242a9f93358 in_seq=0 out_seq=0 2024-05-20T21:04:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.481+0000 7fa8a6ffd700 1 -- 172.21.15.26:0/3847930997 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a4941325a0 con 0x55a494262520 2024-05-20T21:04:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.481+0000 7fa8a6ffd700 10 client.17871 handle_command_reply: tid=0 2024-05-20T21:04:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.481+0000 7fa8d86e3b80 1 client.17871 shutdown 2024-05-20T21:04:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 msgr2=0x55a4941f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941f8da0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0cc170 tx=0x7fa8bc0a9230 comp rx=0 tx=0).stop 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 shutdown_connections 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a494262520 0x55a494264910 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4941e1b30 0x55a4941f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a4941dd000 0x55a4941fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8a6ffd700 0 client.17871 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 --2- 172.21.15.26:0/3847930997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4941dd9c0 0x55a4941f8da0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 >> 172.21.15.26:0/3847930997 conn(0x55a494104340 msgr2=0x55a4941e0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 shutdown_connections 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 1 -- 172.21.15.26:0/3847930997 wait complete. 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8d86e3b80 20 client.17871 trim_cache size 0 max 16384 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:04:24.484 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 32, 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "a6088460": { 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 32, 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.0, 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:04:24.485 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:24.486 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8b00b2880 msgr2=0x7fa8b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8b00b2880 0x7fa8b00b4d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009990 tx=0x7fa8b801a040 comp rx=0 tx=0).stop 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 msgr2=0x7fa8c819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.482+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 0x7fa8c819fc80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00d980 tx=0x7fa8bc00dc90 comp rx=0 tx=0).stop 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 shutdown_connections 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8c8109630 0x7fa8c8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c806bd10 0x7fa8c819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa8b00b2880 0x7fa8b00b4d40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 --2- 172.21.15.26:0/4003808220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c806c6d0 0x7fa8c819fc80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 >> 172.21.15.26:0/4003808220 conn(0x7fa8c80794e0 msgr2=0x7fa8c8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 shutdown_connections 2024-05-20T21:04:24.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:24.483+0000 7fa8cde4d700 1 -- 172.21.15.26:0/4003808220 wait complete. 2024-05-20T21:04:24.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180699 2.29 MB/sec execute 270 sec latency 11324.838 ms 2024-05-20T21:04:25.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 -- 172.21.15.26:0/3564510905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c108520 msgr2=0x7faf7c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 --2- 172.21.15.26:0/3564510905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c108520 0x7faf7c108900 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faf78009990 tx=0x7faf78009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 -- 172.21.15.26:0/3564510905 shutdown_connections 2024-05-20T21:04:25.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 --2- 172.21.15.26:0/3564510905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c108520 0x7faf7c108900 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 --2- 172.21.15.26:0/3564510905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102e00 0x7faf7c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 --2- 172.21.15.26:0/3564510905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102440 0x7faf7c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 -- 172.21.15.26:0/3564510905 >> 172.21.15.26:0/3564510905 conn(0x7faf7c0793f0 msgr2=0x7faf7c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:25.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 -- 172.21.15.26:0/3564510905 shutdown_connections 2024-05-20T21:04:25.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.186+0000 7faf83612700 1 -- 172.21.15.26:0/3564510905 wait complete. 2024-05-20T21:04:25.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 Processor -- start 2024-05-20T21:04:25.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 -- start start 2024-05-20T21:04:25.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 0x7faf7c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 0x7faf7c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c108520 0x7faf7c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1124a0 con 0x7faf7c102e00 2024-05-20T21:04:25.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c112320 con 0x7faf7c102440 2024-05-20T21:04:25.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf83612700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c112620 con 0x7faf7c108520 2024-05-20T21:04:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 0x7faf7c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf813ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 0x7faf7c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf813ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 0x7faf7c104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:34934/0 (socket says 172.21.15.26:34934) 2024-05-20T21:04:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 0x7faf7c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43428/0 (socket says 172.21.15.26:43428) 2024-05-20T21:04:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.187+0000 7faf813ae700 1 -- 172.21.15.26:0/1231406873 learned_addr learned my addr 172.21.15.26:0/1231406873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf81baf700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c108520 0x7faf7c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf80bad700 1 -- 172.21.15.26:0/1231406873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c108520 msgr2=0x7faf7c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf80bad700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c108520 0x7faf7c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf80bad700 1 -- 172.21.15.26:0/1231406873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 msgr2=0x7faf7c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf80bad700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 0x7faf7c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf80bad700 1 -- 172.21.15.26:0/1231406873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c1a80a0 con 0x7faf7c102e00 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf813ae700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 0x7faf7c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf80bad700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 0x7faf7c1045c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00d750 tx=0x7faf6c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf727fc700 1 -- 172.21.15.26:0/1231406873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c016070 con 0x7faf7c102e00 2024-05-20T21:04:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c1a8390 con 0x7faf7c102e00 2024-05-20T21:04:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf727fc700 1 -- 172.21.15.26:0/1231406873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf6c005020 con 0x7faf7c102e00 2024-05-20T21:04:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf727fc700 1 -- 172.21.15.26:0/1231406873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00f980 con 0x7faf7c102e00 2024-05-20T21:04:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.188+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1a88d0 con 0x7faf7c102e00 2024-05-20T21:04:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.191+0000 7faf727fc700 1 -- 172.21.15.26:0/1231406873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7faf6c004a30 con 0x7faf7c102e00 2024-05-20T21:04:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.192+0000 7faf727fc700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf680b2900 0x7faf680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.192+0000 7faf727fc700 1 -- 172.21.15.26:0/1231406873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faf6c0e1a40 con 0x7faf7c102e00 2024-05-20T21:04:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.192+0000 7faf813ae700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf680b2900 0x7faf680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.192+0000 7faf8dea8b80 1 Processor -- start 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.192+0000 7faf8dea8b80 1 -- start start 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.192+0000 7faf813ae700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf680b2900 0x7faf680b4dc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf78009750 tx=0x7faf7801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d06cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d0b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5596e3c65d10 con 0x5596e3d07290 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5596e3c558b0 con 0x5596e3d068e0 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596e3c598f0 con 0x5596e3d0afe0 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf813ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d06cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43430/0 (socket says 172.21.15.26:43430) 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 -- 172.21.15.26:0/3650926205 learned_addr learned my addr 172.21.15.26:0/3650926205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf813ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d06cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:34946/0 (socket says 172.21.15.26:34946) 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf81baf700 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d0b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 -- 172.21.15.26:0/3650926205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 msgr2=0x5596e3d0b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d0b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 -- 172.21.15.26:0/3650926205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 msgr2=0x5596e3d06cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d06cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 -- 172.21.15.26:0/3650926205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e3ba5950 con 0x5596e3d07290 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf813ae700 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d06cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.193+0000 7faf80bad700 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0a9c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0a5080 tx=0x7faf6c0a8880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a8ca874a6e3da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf677fe700 1 -- 172.21.15.26:0/3650926205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c01f040 con 0x5596e3d07290 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf677fe700 1 -- 172.21.15.26:0/3650926205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf6c0b1e60 con 0x5596e3d07290 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf677fe700 1 -- 172.21.15.26:0/3650926205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0a4970 con 0x5596e3d07290 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf8dea8b80 1 -- 172.21.15.26:0/3650926205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 msgr2=0x5596e3d0a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0a9c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0a5080 tx=0x7faf6c0a8880 comp rx=0 tx=0).stop 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf8dea8b80 1 -- 172.21.15.26:0/3650926205 shutdown_connections 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d0b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d06cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.194+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/3650926205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0a9c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.195+0000 7faf8dea8b80 1 -- 172.21.15.26:0/3650926205 >> 172.21.15.26:0/3650926205 conn(0x5596e3c34100 msgr2=0x5596e3d00f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.195+0000 7faf8dea8b80 1 -- 172.21.15.26:0/3650926205 shutdown_connections 2024-05-20T21:04:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.195+0000 7faf8dea8b80 1 -- 172.21.15.26:0/3650926205 wait complete. 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.195+0000 7faf8dea8b80 1 -- start start 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.195+0000 7faf8dea8b80 1 Processor -- start 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d068e0 0x5596e3d123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0afe0 0x5596e3d0fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5596e3b5c020 con 0x5596e3d07290 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5596e3bf7840 con 0x5596e3d0afe0 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43440/0 (socket says 172.21.15.26:43440) 2024-05-20T21:04:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596e3d131a0 con 0x5596e3d068e0 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 -- 172.21.15.26:0/2994606579 learned_addr learned my addr 172.21.15.26:0/2994606579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf813ae700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d068e0 0x5596e3d123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf81baf700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0afe0 0x5596e3d0fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf8dea8b80 10 client.0 _ll_register_callbacks cb 0x5596e3bf09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 -- 172.21.15.26:0/2994606579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d068e0 msgr2=0x5596e3d123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d068e0 0x5596e3d123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 -- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0afe0 msgr2=0x5596e3d0fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0afe0 0x5596e3d0fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 -- 172.21.15.26:0/2994606579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e3bf4710 con 0x5596e3d07290 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf813ae700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d068e0 0x5596e3d123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf81baf700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0afe0 0x5596e3d0fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf80bad700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0f6d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0a6270 tx=0x7faf6c0b1c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ab0cf823180f3e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.196+0000 7faf64ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf64ff9700 1 -- 172.21.15.26:0/2994606579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c01f040 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf64ff9700 1 -- 172.21.15.26:0/2994606579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faf6c0a8260 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf6c0b1e60 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf64ff9700 1 -- 172.21.15.26:0/2994606579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf6c0a8460 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 10 client.17883 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596e3d1d840 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf64ff9700 1 -- 172.21.15.26:0/2994606579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf6c0b1e60 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf64ff9700 1 -- 172.21.15.26:0/2994606579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7faf6c0a7cd0 con 0x5596e3d07290 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 10 client.17883 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 20 client.17883 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d1fd10 0x5596e3d22100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 4 client.17883 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.197+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e3cd2a60 con 0x5596e3d1fd10 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.198+0000 7faf813ae700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d1fd10 0x5596e3d22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.198+0000 7faf64ff9700 10 client.17883 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.198+0000 7faf813ae700 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d1fd10 0x5596e3d22100 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733a5995aab72427 server_cookie=aa8bc1de509d604c in_seq=0 out_seq=0 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf64ff9700 1 -- 172.21.15.26:0/2994606579 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5596e3cd2a60 con 0x5596e3d1fd10 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf64ff9700 10 client.17883 handle_command_reply: tid=0 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 client.17883 shutdown 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 msgr2=0x5596e3d0f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0f6d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0a6270 tx=0x7faf6c0b1c80 comp rx=0 tx=0).stop 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 shutdown_connections 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d1fd10 0x5596e3d22100 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0afe0 0x5596e3d0fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d068e0 0x5596e3d123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/2994606579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d07290 0x5596e3d0f6d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 >> 172.21.15.26:0/2994606579 conn(0x5596e3c34100 msgr2=0x5596e3d0c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf64ff9700 0 client.17883 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 shutdown_connections 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 1 -- 172.21.15.26:0/2994606579 wait complete. 2024-05-20T21:04:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.199+0000 7faf8dea8b80 20 client.17883 trim_cache size 0 max 16384 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.222+0000 7faf8dea8b80 1 Processor -- start 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 -- start start 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 0x5596e3d24d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d25280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0e6d0 0x5596e3d08930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000fd90 con 0x5596e3d068e0 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000d620 con 0x5596e3d0e6d0 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000b470 con 0x5596e3d0afe0 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 0x5596e3d24d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 0x5596e3d24d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43456/0 (socket says 172.21.15.26:43456) 2024-05-20T21:04:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 -- 172.21.15.26:0/4166467318 learned_addr learned my addr 172.21.15.26:0/4166467318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf81baf700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0e6d0 0x5596e3d08930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf80bad700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d25280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 -- 172.21.15.26:0/4166467318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 msgr2=0x5596e3d25280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d25280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 -- 172.21.15.26:0/4166467318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0e6d0 msgr2=0x5596e3d08930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0e6d0 0x5596e3d08930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf813ae700 1 -- 172.21.15.26:0/4166467318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e3d0ad00 con 0x5596e3d068e0 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.223+0000 7faf80bad700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d25280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.224+0000 7faf81baf700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0e6d0 0x5596e3d08930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.224+0000 7faf813ae700 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 0x5596e3d24d40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7faf78005850 tx=0x7faf7801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43ca05e7f9a7311 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.224+0000 7faf66ffd700 1 -- 172.21.15.26:0/4166467318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780044a0 con 0x5596e3d068e0 2024-05-20T21:04:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.224+0000 7faf66ffd700 1 -- 172.21.15.26:0/4166467318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf78004640 con 0x5596e3d068e0 2024-05-20T21:04:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.224+0000 7faf66ffd700 1 -- 172.21.15.26:0/4166467318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78026690 con 0x5596e3d068e0 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4166467318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 msgr2=0x5596e3d24d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 0x5596e3d24d40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7faf78005850 tx=0x7faf7801f000 comp rx=0 tx=0).stop 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4166467318 shutdown_connections 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d068e0 0x5596e3d24d40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d0e6d0 0x5596e3d08930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4166467318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0afe0 0x5596e3d25280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4166467318 >> 172.21.15.26:0/4166467318 conn(0x5596e3c34100 msgr2=0x5596e3d0dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4166467318 shutdown_connections 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4166467318 wait complete. 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 -- start start 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.225+0000 7faf8dea8b80 1 Processor -- start 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d25280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 0x5596e3d23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0e6d0 0x5596e3d23b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000bae0 con 0x5596e3d0afe0 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000cf50 con 0x5596e3d068e0 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000c1b0 con 0x5596e3d0e6d0 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf8dea8b80 10 client.0 _ll_register_callbacks cb 0x5596e3bf09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 0x5596e3d23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf813ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d25280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 0x5596e3d23630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43464/0 (socket says 172.21.15.26:43464) 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 -- 172.21.15.26:0/4119969600 learned_addr learned my addr 172.21.15.26:0/4119969600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf81baf700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0e6d0 0x5596e3d23b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 -- 172.21.15.26:0/4119969600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0e6d0 msgr2=0x5596e3d23b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0e6d0 0x5596e3d23b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 -- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 msgr2=0x5596e3d25280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d25280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.226+0000 7faf80bad700 1 -- 172.21.15.26:0/4119969600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e3d241b0 con 0x5596e3d0afe0 2024-05-20T21:04:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf81baf700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0e6d0 0x5596e3d23b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf80bad700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 0x5596e3d23630 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00ddb0 tx=0x7faf6c0a9700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e883cde0a89e1fa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf64ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf64ff9700 1 -- 172.21.15.26:0/4119969600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c01f040 con 0x5596e3d0afe0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf64ff9700 1 -- 172.21.15.26:0/4119969600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faf6c0a55f0 con 0x5596e3d0afe0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf78004640 con 0x5596e3d0afe0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf64ff9700 1 -- 172.21.15.26:0/4119969600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf6c0a8dd0 con 0x5596e3d0afe0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf8dea8b80 10 client.17889 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596e3d241b0 con 0x5596e3d0afe0 2024-05-20T21:04:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf64ff9700 1 -- 172.21.15.26:0/4119969600 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf78004640 con 0x5596e3d0afe0 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf64ff9700 1 -- 172.21.15.26:0/4119969600 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7faf6c00fcb0 con 0x5596e3d0afe0 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf8dea8b80 10 client.17889 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf8dea8b80 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.227+0000 7faf8dea8b80 20 client.17889 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.228+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d8b720 0x5596e3d8db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.228+0000 7faf8dea8b80 4 client.17889 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.228+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5596e3c67c60 con 0x5596e3d8b720 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.228+0000 7faf813ae700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d8b720 0x5596e3d8db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.228+0000 7faf64ff9700 10 client.17889 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.228+0000 7faf813ae700 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d8b720 0x5596e3d8db40 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a22a087e629b46d server_cookie=1026bc17fd72f928 in_seq=0 out_seq=0 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf64ff9700 1 -- 172.21.15.26:0/4119969600 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5596e3c67c60 con 0x5596e3d8b720 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf64ff9700 10 client.17889 handle_command_reply: tid=0 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 client.17889 shutdown 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 msgr2=0x5596e3d23630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 0x5596e3d23630 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00ddb0 tx=0x7faf6c0a9700 comp rx=0 tx=0).stop 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 shutdown_connections 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5596e3d8b720 0x5596e3d8db40 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596e3d0e6d0 0x5596e3d23b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e3d068e0 0x5596e3d25280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 --2- 172.21.15.26:0/4119969600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596e3d0afe0 0x5596e3d23630 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 >> 172.21.15.26:0/4119969600 conn(0x5596e3c34100 msgr2=0x5596e3d0fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.229+0000 7faf64ff9700 0 client.17889 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.230+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 shutdown_connections 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.230+0000 7faf8dea8b80 1 -- 172.21.15.26:0/4119969600 wait complete. 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.230+0000 7faf8dea8b80 20 client.17889 trim_cache size 0 max 16384 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:25.237 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:04:25.238 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "08a2dbf1-54a8-40bf-9a84-1e76d629adfe", 2024-05-20T21:04:25.238 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:04:25.238 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf680b2900 msgr2=0x7faf680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf680b2900 0x7faf680b4dc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf78009750 tx=0x7faf7801a040 comp rx=0 tx=0).stop 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 msgr2=0x7faf7c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 0x7faf7c1045c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00d750 tx=0x7faf6c00da60 comp rx=0 tx=0).stop 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 shutdown_connections 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c108520 0x7faf7c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf7c102440 0x7faf7c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf680b2900 0x7faf680b4dc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 --2- 172.21.15.26:0/1231406873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf7c102e00 0x7faf7c1045c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 >> 172.21.15.26:0/1231406873 conn(0x7faf7c0793f0 msgr2=0x7faf7c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 shutdown_connections 2024-05-20T21:04:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:25.231+0000 7faf83612700 1 -- 172.21.15.26:0/1231406873 wait complete. 2024-05-20T21:04:25.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:25 smithi055 ceph-mon[35213]: pgmap v638: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 619 KiB/s wr, 60 op/s 2024-05-20T21:04:25.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:25 smithi026 ceph-mon[31990]: pgmap v638: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 619 KiB/s wr, 60 op/s 2024-05-20T21:04:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:25 smithi144 ceph-mon[35660]: pgmap v638: 97 pgs: 97 active+clean; 27 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 619 KiB/s wr, 60 op/s 2024-05-20T21:04:25.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 181295 2.30 MB/sec execute 271 sec latency 17.746 ms 2024-05-20T21:04:25.864 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a6088460', total seconds elapsed: 25.7 2024-05-20T21:04:26.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:26 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:04:26.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:26 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:26.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:26 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:04:26.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:26 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:26.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:26 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:04:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:26 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:04:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:26 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:26 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:04:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:26 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:26.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:26 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:04:26.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 181851 2.30 MB/sec execute 272 sec latency 23.007 ms 2024-05-20T21:04:26.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:04:26.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:26 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:04:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:26 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:26 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:04:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:26 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:26.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:26 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:04:26.890 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:27.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:27 smithi026 ceph-mon[31990]: pgmap v639: 97 pgs: 97 active+clean; 27 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 329 KiB/s wr, 31 op/s 2024-05-20T21:04:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:27 smithi144 ceph-mon[35660]: pgmap v639: 97 pgs: 97 active+clean; 27 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 329 KiB/s wr, 31 op/s 2024-05-20T21:04:27.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 182473 2.30 MB/sec execute 273 sec latency 17.485 ms 2024-05-20T21:04:27.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:27 smithi055 ceph-mon[35213]: pgmap v639: 97 pgs: 97 active+clean; 27 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 329 KiB/s wr, 31 op/s 2024-05-20T21:04:28.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 183135 2.30 MB/sec execute 274 sec latency 33.461 ms 2024-05-20T21:04:28.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 -- 172.21.15.26:0/625805248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 msgr2=0x7f04d4103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 --2- 172.21.15.26:0/625805248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4103a50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009990 tx=0x7f04c8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 -- 172.21.15.26:0/625805248 shutdown_connections 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 --2- 172.21.15.26:0/625805248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4103f90 0x7f04d4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 --2- 172.21.15.26:0/625805248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04d4108500 0x7f04d41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 --2- 172.21.15.26:0/625805248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4103a50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 -- 172.21.15.26:0/625805248 >> 172.21.15.26:0/625805248 conn(0x7f04d407a6a0 msgr2=0x7f04d407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 -- 172.21.15.26:0/625805248 shutdown_connections 2024-05-20T21:04:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 -- 172.21.15.26:0/625805248 wait complete. 2024-05-20T21:04:28.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.751+0000 7f04da0be700 1 Processor -- start 2024-05-20T21:04:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 -- start start 2024-05-20T21:04:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04d4103f90 0x7f04d419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4108500 0x7f04d419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4113690 con 0x7f04d41035f0 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4113510 con 0x7f04d4108500 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04da0be700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4113810 con 0x7f04d4103f90 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43476/0 (socket says 172.21.15.26:43476) 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04d37fe700 1 -- 172.21.15.26:0/1529658708 learned_addr learned my addr 172.21.15.26:0/1529658708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04d3fff700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4108500 0x7f04d419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04d2ffd700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04d4103f90 0x7f04d419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.752+0000 7f04d3fff700 1 -- 172.21.15.26:0/1529658708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04d4103f90 msgr2=0x7f04d419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d3fff700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04d4103f90 0x7f04d419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d3fff700 1 -- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 msgr2=0x7f04d4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d3fff700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d3fff700 1 -- 172.21.15.26:0/1529658708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d419eb60 con 0x7f04d4108500 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d3fff700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4108500 0x7f04d419e460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00efc0 tx=0x7f04cc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d0ff9700 1 -- 172.21.15.26:0/1529658708 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0093a0 con 0x7f04d4108500 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d419edf0 con 0x7f04d4108500 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d0ff9700 1 -- 172.21.15.26:0/1529658708 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04cc00c9f0 con 0x7f04d4108500 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04d0ff9700 1 -- 172.21.15.26:0/1529658708 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004420 con 0x7f04d4108500 2024-05-20T21:04:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.753+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41a6710 con 0x7f04d4108500 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.756+0000 7f04d0ff9700 1 -- 172.21.15.26:0/1529658708 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f04cc01b030 con 0x7f04d4108500 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.756+0000 7f04d0ff9700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04bc0b2900 0x7f04bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04d37fe700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04bc0b2900 0x7f04bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04d0ff9700 1 -- 172.21.15.26:0/1529658708 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f04cc0e1940 con 0x7f04d4108500 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 Processor -- start 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04d37fe700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04bc0b2900 0x7f04bc0b4dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f04c400ac60 tx=0x7f04c400a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 -- start start 2024-05-20T21:04:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b49700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b43d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b50080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563561aa1d70 con 0x563561b438a0 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563561a91910 con 0x563561b44260 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.757+0000 7f04e4954b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563561a95950 con 0x563561b49320 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b43d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b50080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b50080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35014/0 (socket says 172.21.15.26:35014) 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b43d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43488/0 (socket says 172.21.15.26:43488) 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 -- 172.21.15.26:0/2449860210 learned_addr learned my addr 172.21.15.26:0/2449860210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d37fe700 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b49700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 -- 172.21.15.26:0/2449860210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 msgr2=0x563561b49700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b49700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 -- 172.21.15.26:0/2449860210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 msgr2=0x563561b43d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b43d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 -- 172.21.15.26:0/2449860210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563561a3afb0 con 0x563561b44260 2024-05-20T21:04:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04d3fff700 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b50080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0a8f20 tx=0x7f04cc0a89b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=643e873d79493f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04b9ffb700 1 -- 172.21.15.26:0/2449860210 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0a8f90 con 0x563561b44260 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.758+0000 7f04b9ffb700 1 -- 172.21.15.26:0/2449860210 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04cc0ad340 con 0x563561b44260 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04b9ffb700 1 -- 172.21.15.26:0/2449860210 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0a9450 con 0x563561b44260 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 -- 172.21.15.26:0/2449860210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 msgr2=0x563561b50080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b50080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0a8f20 tx=0x7f04cc0a89b0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 -- 172.21.15.26:0/2449860210 shutdown_connections 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b50080 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b49700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2449860210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b43d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 -- 172.21.15.26:0/2449860210 >> 172.21.15.26:0/2449860210 conn(0x563561a6f240 msgr2=0x563561b3e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 -- 172.21.15.26:0/2449860210 shutdown_connections 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.759+0000 7f04e4954b80 1 -- 172.21.15.26:0/2449860210 wait complete. 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 -- start start 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 Processor -- start 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b4fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b4d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b4d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563561997fa0 con 0x563561b438a0 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5635619f9da0 con 0x563561b44260 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563561b506e0 con 0x563561b49320 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04e4954b80 10 client.0 _ll_register_callbacks cb 0x563561a2c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b4fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04d2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b4d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b4fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43494/0 (socket says 172.21.15.26:43494) 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.760+0000 7f04d37fe700 1 -- 172.21.15.26:0/3718082864 learned_addr learned my addr 172.21.15.26:0/3718082864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d3fff700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b4d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d37fe700 1 -- 172.21.15.26:0/3718082864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 msgr2=0x563561b4d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d37fe700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b4d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d37fe700 1 -- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 msgr2=0x563561b4d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d37fe700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b4d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d37fe700 1 -- 172.21.15.26:0/3718082864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635619e18d0 con 0x563561b438a0 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d2ffd700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b4d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04d37fe700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b4fdf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f04c401da90 tx=0x7f04c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7b9718040cbde5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04b2ffd700 1 -- 172.21.15.26:0/3718082864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c402a060 con 0x563561b438a0 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04cc0ad340 con 0x563561b438a0 2024-05-20T21:04:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04b2ffd700 1 -- 172.21.15.26:0/3718082864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f04c401b850 con 0x563561b438a0 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04b2ffd700 1 -- 172.21.15.26:0/3718082864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04c4016070 con 0x563561b438a0 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04e4954b80 10 client.17901 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.761+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563561a4b760 con 0x563561b438a0 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04b2ffd700 1 -- 172.21.15.26:0/3718082864 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04cc0ad340 con 0x563561b438a0 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04b2ffd700 1 -- 172.21.15.26:0/3718082864 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f04c402e030 con 0x563561b438a0 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04e4954b80 10 client.17901 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04e4954b80 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04e4954b80 20 client.17901 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04e4954b80 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561b5d310 0x563561b5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04e4954b80 4 client.17901 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563561aa3cc0 con 0x563561b5d310 2024-05-20T21:04:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.762+0000 7f04d2ffd700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561b5d310 0x563561b5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.763+0000 7f04d2ffd700 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561b5d310 0x563561b5f740 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f269346f68420185 server_cookie=28eda105ee174b04 in_seq=0 out_seq=0 2024-05-20T21:04:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.763+0000 7f04b2ffd700 10 client.17901 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04b2ffd700 1 -- 172.21.15.26:0/3718082864 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563561aa3cc0 con 0x563561b5d310 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04b2ffd700 10 client.17901 handle_command_reply: tid=0 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 client.17901 shutdown 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 msgr2=0x563561b4fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b4fdf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f04c401da90 tx=0x7f04c40047c0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 shutdown_connections 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561b5d310 0x563561b5f740 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b438a0 0x563561b4fdf0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b49320 0x563561b4d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 --2- 172.21.15.26:0/3718082864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b4d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 >> 172.21.15.26:0/3718082864 conn(0x563561a6f240 msgr2=0x563561b3daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04b2ffd700 0 client.17901 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 shutdown_connections 2024-05-20T21:04:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 1 -- 172.21.15.26:0/3718082864 wait complete. 2024-05-20T21:04:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.764+0000 7f04e4954b80 20 client.17901 trim_cache size 0 max 16384 2024-05-20T21:04:28.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 Processor -- start 2024-05-20T21:04:28.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 -- start start 2024-05-20T21:04:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 0x563561b59740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b44260 0x563561b3e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b47460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41b0660 con 0x563561b49320 2024-05-20T21:04:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41adef0 con 0x563561b438a0 2024-05-20T21:04:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04e4954b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41abd40 con 0x563561b44260 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 0x563561b59740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 0x563561b59740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:35022/0 (socket says 172.21.15.26:35022) 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04d37fe700 1 -- 172.21.15.26:0/287195749 learned_addr learned my addr 172.21.15.26:0/287195749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04d3fff700 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b47460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04d2ffd700 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b44260 0x563561b3e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.791+0000 7f04d37fe700 1 -- 172.21.15.26:0/287195749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b44260 msgr2=0x563561b3e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04d37fe700 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b44260 0x563561b3e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04d37fe700 1 -- 172.21.15.26:0/287195749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 msgr2=0x563561b47460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04d37fe700 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b47460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04d37fe700 1 -- 172.21.15.26:0/287195749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563561b5ade0 con 0x563561b438a0 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04d37fe700 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 0x563561b59740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f04c400aa40 tx=0x7f04c400aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13535f06bfba6280 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04b97fa700 1 -- 172.21.15.26:0/287195749 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4010430 con 0x563561b438a0 2024-05-20T21:04:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04b97fa700 1 -- 172.21.15.26:0/287195749 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04c402f670 con 0x563561b438a0 2024-05-20T21:04:28.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.792+0000 7f04b97fa700 1 -- 172.21.15.26:0/287195749 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c401b8b0 con 0x563561b438a0 2024-05-20T21:04:28.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 -- 172.21.15.26:0/287195749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 msgr2=0x563561b59740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 0x563561b59740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f04c400aa40 tx=0x7f04c400aee0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 -- 172.21.15.26:0/287195749 shutdown_connections 2024-05-20T21:04:28.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b47460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b438a0 0x563561b59740 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 --2- 172.21.15.26:0/287195749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b44260 0x563561b3e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 -- 172.21.15.26:0/287195749 >> 172.21.15.26:0/287195749 conn(0x563561a6f240 msgr2=0x563561b46d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 -- 172.21.15.26:0/287195749 shutdown_connections 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 -- 172.21.15.26:0/287195749 wait complete. 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 -- start start 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.793+0000 7f04e4954b80 1 Processor -- start 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b438a0 0x563561b55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b56170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b64820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41ac3b0 con 0x563561b49320 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41ad820 con 0x563561b44260 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41aca80 con 0x563561b438a0 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04e4954b80 10 client.0 _ll_register_callbacks cb 0x563561a2c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b64820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b438a0 0x563561b55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b64820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43514/0 (socket says 172.21.15.26:43514) 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 -- 172.21.15.26:0/2029325348 learned_addr learned my addr 172.21.15.26:0/2029325348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d2ffd700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b56170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 -- 172.21.15.26:0/2029325348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b438a0 msgr2=0x563561b55c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b438a0 0x563561b55c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 -- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 msgr2=0x563561b56170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b56170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d3fff700 1 -- 172.21.15.26:0/2029325348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563561b65020 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.794+0000 7f04d37fe700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b438a0 0x563561b55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04d3fff700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b64820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0a5010 tx=0x7f04cc00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78771c93c647eba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04b2ffd700 1 -- 172.21.15.26:0/2029325348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0ad340 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04b2ffd700 1 -- 172.21.15.26:0/2029325348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f04cc0b1ae0 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c402f670 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04b2ffd700 1 -- 172.21.15.26:0/2029325348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04cc0a8f90 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 10 client.17907 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563561bc7b40 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04b2ffd700 1 -- 172.21.15.26:0/2029325348 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04c402f670 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04b2ffd700 1 -- 172.21.15.26:0/2029325348 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f04cc0a4a90 con 0x563561b49320 2024-05-20T21:04:28.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 10 client.17907 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 20 client.17907 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561bc7e20 0x563561bca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 4 client.17907 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.795+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563561a9d4a0 con 0x563561bc7e20 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04d37fe700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561bc7e20 0x563561bca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04d37fe700 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561bc7e20 0x563561bca210 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee3a5710b785fa7 server_cookie=f48c26ade8520547 in_seq=0 out_seq=0 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04b2ffd700 10 client.17907 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04b2ffd700 1 -- 172.21.15.26:0/2029325348 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563561a9d4a0 con 0x563561bc7e20 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04b2ffd700 10 client.17907 handle_command_reply: tid=0 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 client.17907 shutdown 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 msgr2=0x563561b64820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b64820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0a5010 tx=0x7f04cc00cea0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 shutdown_connections 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563561bc7e20 0x563561bca210 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563561b49320 0x563561b64820 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563561b438a0 0x563561b55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 --2- 172.21.15.26:0/2029325348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563561b44260 0x563561b56170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04b2ffd700 0 client.17907 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.796+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 >> 172.21.15.26:0/2029325348 conn(0x563561a6f240 msgr2=0x563561b475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 shutdown_connections 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04e4954b80 1 -- 172.21.15.26:0/2029325348 wait complete. 2024-05-20T21:04:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04e4954b80 20 client.17907 trim_cache size 0 max 16384 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04bc0b2900 msgr2=0x7f04bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04bc0b2900 0x7f04bc0b4dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f04c400ac60 tx=0x7f04c400a430 comp rx=0 tx=0).stop 2024-05-20T21:04:28.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4108500 msgr2=0x7f04d419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4108500 0x7f04d419e460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00efc0 tx=0x7f04cc00c3a0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 shutdown_connections 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04bc0b2900 0x7f04bc0b4dc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d41035f0 0x7f04d4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04d4108500 0x7f04d419e460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 --2- 172.21.15.26:0/1529658708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04d4103f90 0x7f04d419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.797+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 >> 172.21.15.26:0/1529658708 conn(0x7f04d407a6a0 msgr2=0x7f04d41016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.798+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 shutdown_connections 2024-05-20T21:04:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:28.798+0000 7f04da0be700 1 -- 172.21.15.26:0/1529658708 wait complete. 2024-05-20T21:04:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:29 smithi026 ceph-mon[31990]: pgmap v640: 97 pgs: 97 active+clean; 29 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 329 KiB/s wr, 46 op/s 2024-05-20T21:04:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:29 smithi144 ceph-mon[35660]: pgmap v640: 97 pgs: 97 active+clean; 29 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 329 KiB/s wr, 46 op/s 2024-05-20T21:04:29.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 183900 2.31 MB/sec execute 275 sec latency 17.700 ms 2024-05-20T21:04:29.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:04:29.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:04:29.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:29 smithi055 ceph-mon[35213]: pgmap v640: 97 pgs: 97 active+clean; 29 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 329 KiB/s wr, 46 op/s 2024-05-20T21:04:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:30.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 184489 2.32 MB/sec execute 276 sec latency 18.034 ms 2024-05-20T21:04:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/4130488588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4108790 msgr2=0x7f2ea410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/4130488588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4108790 0x7f2ea410abe0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009ca0 tx=0x7f2e9c02a8c0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/4130488588 shutdown_connections 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/4130488588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4108790 0x7f2ea410abe0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/4130488588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ea4105e60 0x7f2ea4108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/4130488588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea4103530 0x7f2ea4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/4130488588 >> 172.21.15.26:0/4130488588 conn(0x7f2ea40fcf20 msgr2=0x7f2ea40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/4130488588 shutdown_connections 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/4130488588 wait complete. 2024-05-20T21:04:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 Processor -- start 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.492+0000 7f2ea9d0a640 1 -- start start 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea9d0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 0x7f2ea4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea9d0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea4105e60 0x7f2ea4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea9d0a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ea4108790 0x7f2ea41a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea9d0a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410e500 con 0x7f2ea4103530 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea9d0a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410e380 con 0x7f2ea4108790 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea9d0a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410e680 con 0x7f2ea4105e60 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 0x7f2ea4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 0x7f2ea4198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36916/0 (socket says 172.21.15.26:36916) 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 -- 172.21.15.26:0/2773226001 learned_addr learned my addr 172.21.15.26:0/2773226001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ea4108790 0x7f2ea41a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea4105e60 0x7f2ea4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 -- 172.21.15.26:0/2773226001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea4105e60 msgr2=0x7f2ea4199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea4105e60 0x7f2ea4199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 -- 172.21.15.26:0/2773226001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ea4108790 msgr2=0x7f2ea41a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ea4108790 0x7f2ea41a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 -- 172.21.15.26:0/2773226001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea41a0740 con 0x7f2ea4103530 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.493+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 0x7f2ea4198b20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c970 tx=0x7f2e9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.494+0000 7f2ea17fa640 1 -- 172.21.15.26:0/2773226001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94007bb0 con 0x7f2ea4103530 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.494+0000 7f2ea17fa640 1 -- 172.21.15.26:0/2773226001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e94007d50 con 0x7f2ea4103530 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.494+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea41a0a30 con 0x7f2ea4103530 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.494+0000 7f2ea17fa640 1 -- 172.21.15.26:0/2773226001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94004e40 con 0x7f2ea4103530 2024-05-20T21:04:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.494+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41a0f80 con 0x7f2ea4103530 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.496+0000 7f2ea17fa640 1 -- 172.21.15.26:0/2773226001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2e94005020 con 0x7f2ea4103530 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.496+0000 7f2eaca1e740 1 Processor -- start 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 -- start start 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 0x564e350d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d14c0 0x564e350d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d6630 0x564e350d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e3503dc30 con 0x564e350d0ad0 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564e3502d7d0 con 0x564e350d14c0 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564e35031810 con 0x564e350d6630 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d14c0 0x564e350d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 0x564e350d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 0x564e350d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36924/0 (socket says 172.21.15.26:36924) 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea3fff640 1 -- 172.21.15.26:0/1368584710 learned_addr learned my addr 172.21.15.26:0/1368584710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d6630 0x564e350d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea37fe640 1 -- 172.21.15.26:0/1368584710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d6630 msgr2=0x564e350d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d6630 0x564e350d8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea37fe640 1 -- 172.21.15.26:0/1368584710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 msgr2=0x564e350d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 0x564e350d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea37fe640 1 -- 172.21.15.26:0/1368584710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e34d42210 con 0x564e350d14c0 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.497+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 0x564e350d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2ea17fa640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2e900b2920 0x7f2e900b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d14c0 0x564e350d5fc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e940df9a0 tx=0x7f2e940ea5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3bf2d0609507f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2e900b2920 0x7f2e900b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2ea17fa640 1 -- 172.21.15.26:0/2773226001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f2e940e1f30 con 0x7f2ea4103530 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2e99ffb640 1 -- 172.21.15.26:0/1368584710 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e940a53b0 con 0x564e350d14c0 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2e99ffb640 1 -- 172.21.15.26:0/1368584710 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e940055d0 con 0x564e350d14c0 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.498+0000 7f2e99ffb640 1 -- 172.21.15.26:0/1368584710 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e940a8c60 con 0x564e350d14c0 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2e900b2920 0x7f2e900b4de0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c005fe0 tx=0x7f2e9c005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1368584710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d14c0 msgr2=0x564e350d5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d14c0 0x564e350d5fc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e940df9a0 tx=0x7f2e940ea5f0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1368584710 shutdown_connections 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d6630 0x564e350d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d14c0 0x564e350d5fc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1368584710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d0ad0 0x564e350d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1368584710 >> 172.21.15.26:0/1368584710 conn(0x564e350c7d30 msgr2=0x564e350c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1368584710 shutdown_connections 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1368584710 wait complete. 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.499+0000 7f2eaca1e740 1 -- start start 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 Processor -- start 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d0ad0 0x564e350dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 0x564e350db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d6630 0x564e350e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e34d13790 con 0x564e350d14c0 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564e34fe9890 con 0x564e350d6630 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564e350d95e0 con 0x564e350d0ad0 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2eaca1e740 10 client.0 _ll_register_callbacks cb 0x564e34d41a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 0x564e350db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 0x564e350db130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36936/0 (socket says 172.21.15.26:36936) 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 -- 172.21.15.26:0/3138566373 learned_addr learned my addr 172.21.15.26:0/3138566373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d6630 0x564e350e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d0ad0 0x564e350dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 -- 172.21.15.26:0/3138566373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d0ad0 msgr2=0x564e350dabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d0ad0 0x564e350dabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 -- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d6630 msgr2=0x564e350e2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d6630 0x564e350e2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.500+0000 7f2e9bfff640 1 -- 172.21.15.26:0/3138566373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e34e0e620 con 0x564e350d14c0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 0x564e350db130 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c013710 tx=0x7f2e8c013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211595ff9bffbf97 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e82ffd640 1 -- 172.21.15.26:0/3138566373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c020060 con 0x564e350d14c0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e82ffd640 1 -- 172.21.15.26:0/3138566373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f2e8c011d70 con 0x564e350d14c0 2024-05-20T21:04:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e3503fb80 con 0x564e350d14c0 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e82ffd640 1 -- 172.21.15.26:0/3138566373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e8c00f450 con 0x564e350d14c0 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 10 client.17919 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564e350e5220 con 0x564e350d14c0 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e82ffd640 1 -- 172.21.15.26:0/3138566373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564e3503fb80 con 0x564e350d14c0 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2e82ffd640 1 -- 172.21.15.26:0/3138566373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f2e8c024030 con 0x564e350d14c0 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 10 client.17919 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 20 client.17919 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.501+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e350e69b0 0x564e350e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.502+0000 7f2eaca1e740 4 client.17919 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.502+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e35039360 con 0x564e350e69b0 2024-05-20T21:04:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.502+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e350e69b0 0x564e350e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.502+0000 7f2e82ffd640 10 client.17919 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.502+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e350e69b0 0x564e350e8e20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea55b74e1997a90d server_cookie=e7e5efc2b22b95ba in_seq=0 out_seq=0 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.503+0000 7f2e82ffd640 1 -- 172.21.15.26:0/3138566373 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564e35039360 con 0x564e350e69b0 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.503+0000 7f2e82ffd640 10 client.17919 handle_command_reply: tid=0 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.503+0000 7f2eaca1e740 1 client.17919 shutdown 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.503+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 msgr2=0x564e350db130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.503+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 0x564e350db130 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c013710 tx=0x7f2e8c013bd0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.503+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 shutdown_connections 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e350e69b0 0x564e350e8e20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d6630 0x564e350e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d14c0 0x564e350db130 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/3138566373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d0ad0 0x564e350dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 >> 172.21.15.26:0/3138566373 conn(0x564e350c7d30 msgr2=0x564e350c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2e82ffd640 0 client.17919 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 shutdown_connections 2024-05-20T21:04:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 1 -- 172.21.15.26:0/3138566373 wait complete. 2024-05-20T21:04:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.504+0000 7f2eaca1e740 20 client.17919 trim_cache size 0 max 16384 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 Processor -- start 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 -- start start 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41aa580 con 0x564e350d6630 2024-05-20T21:04:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41a7e50 con 0x564e350d0ad0 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41a5ca0 con 0x564e350d14c0 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36940/0 (socket says 172.21.15.26:36940) 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350d5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54312/0 (socket says 172.21.15.26:54312) 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2ea3fff640 1 -- 172.21.15.26:0/2125904002 learned_addr learned my addr 172.21.15.26:0/2125904002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.521+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea3fff640 1 -- 172.21.15.26:0/2125904002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 msgr2=0x564e350e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350e2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea3fff640 1 -- 172.21.15.26:0/2125904002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 msgr2=0x564e350d5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350d5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea3fff640 1 -- 172.21.15.26:0/2125904002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e350dba90 con 0x564e350d6630 2024-05-20T21:04:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350caa10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009ca0 tx=0x7f2e9c044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3decdea79c691762 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2e997fa640 1 -- 172.21.15.26:0/2125904002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c004040 con 0x564e350d6630 2024-05-20T21:04:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2e997fa640 1 -- 172.21.15.26:0/2125904002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c036050 con 0x564e350d6630 2024-05-20T21:04:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.522+0000 7f2e997fa640 1 -- 172.21.15.26:0/2125904002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c030a50 con 0x564e350d6630 2024-05-20T21:04:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 -- 172.21.15.26:0/2125904002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 msgr2=0x564e350caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350caa10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009ca0 tx=0x7f2e9c044000 comp rx=0 tx=0).stop 2024-05-20T21:04:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 -- 172.21.15.26:0/2125904002 shutdown_connections 2024-05-20T21:04:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350caa10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/2125904002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 -- 172.21.15.26:0/2125904002 >> 172.21.15.26:0/2125904002 conn(0x564e350c7d30 msgr2=0x564e350c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 -- 172.21.15.26:0/2125904002 shutdown_connections 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 -- 172.21.15.26:0/2125904002 wait complete. 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.523+0000 7f2eaca1e740 1 -- start start 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 Processor -- start 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41a6310 con 0x564e350d6630 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41a7780 con 0x564e350d0ad0 2024-05-20T21:04:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41a69e0 con 0x564e350d14c0 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2eaca1e740 10 client.0 _ll_register_callbacks cb 0x7f2e8c009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350df6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54324/0 (socket says 172.21.15.26:54324) 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea37fe640 1 -- 172.21.15.26:0/1763819150 learned_addr learned my addr 172.21.15.26:0/1763819150 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36944/0 (socket says 172.21.15.26:36944) 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2e9bfff640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 -- 172.21.15.26:0/1763819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 msgr2=0x564e350dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 -- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 msgr2=0x564e350df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350df6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea3fff640 1 -- 172.21.15.26:0/1763819150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e350c7b60 con 0x564e350d6630 2024-05-20T21:04:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.524+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2ea3fff640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350e0ba0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c005e30 tx=0x7f2e9c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82effa90a6cd017 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2e82ffd640 1 -- 172.21.15.26:0/1763819150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c030be0 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2e82ffd640 1 -- 172.21.15.26:0/1763819150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f2e9c030d80 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9c036050 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2e82ffd640 1 -- 172.21.15.26:0/1763819150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e9c0098b0 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 10 client.17925 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564e350e22d0 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2e82ffd640 1 -- 172.21.15.26:0/1763819150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c036050 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2e82ffd640 1 -- 172.21.15.26:0/1763819150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f2e9c067050 con 0x564e350d6630 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 10 client.17925 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 10 client.17925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 20 client.17925 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e35151a20 0x564e35153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 4 client.17925 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.525+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e34fb35f0 con 0x564e35151a20 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.526+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e35151a20 0x564e35153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.526+0000 7f2ea37fe640 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e35151a20 0x564e35153e40 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1889bc1d9346170 server_cookie=61fff56c4e94e1fc in_seq=0 out_seq=0 2024-05-20T21:04:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.526+0000 7f2e82ffd640 10 client.17925 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:31 smithi144 ceph-mon[35660]: pgmap v641: 97 pgs: 97 active+clean; 29 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 342 KiB/s wr, 42 op/s 2024-05-20T21:04:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2e82ffd640 1 -- 172.21.15.26:0/1763819150 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564e34fb35f0 con 0x564e35151a20 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2e82ffd640 10 client.17925 handle_command_reply: tid=0 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 client.17925 shutdown 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 msgr2=0x564e350e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350e0ba0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c005e30 tx=0x7f2e9c009540 comp rx=0 tx=0).stop 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 shutdown_connections 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564e35151a20 0x564e35153e40 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e350d6630 0x564e350e0ba0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e350d14c0 0x564e350dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 --2- 172.21.15.26:0/1763819150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564e350d0ad0 0x564e350df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 >> 172.21.15.26:0/1763819150 conn(0x564e350c7d30 msgr2=0x564e350d58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.561+0000 7f2e82ffd640 0 client.17925 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 shutdown_connections 2024-05-20T21:04:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2eaca1e740 1 -- 172.21.15.26:0/1763819150 wait complete. 2024-05-20T21:04:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2eaca1e740 20 client.17925 trim_cache size 0 max 16384 2024-05-20T21:04:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:04:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:04:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2e900b2920 msgr2=0x7f2e900b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2e900b2920 0x7f2e900b4de0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c005fe0 tx=0x7f2e9c005f70 comp rx=0 tx=0).stop 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 msgr2=0x7f2ea4198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 0x7f2ea4198b20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c970 tx=0x7f2e9400ce30 comp rx=0 tx=0).stop 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 shutdown_connections 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2e900b2920 0x7f2e900b4de0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ea4108790 0x7f2ea41a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea4105e60 0x7f2ea4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 --2- 172.21.15.26:0/2773226001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea4103530 0x7f2ea4198b20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 >> 172.21.15.26:0/2773226001 conn(0x7f2ea40fcf20 msgr2=0x7f2ea4103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 shutdown_connections 2024-05-20T21:04:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:31.562+0000 7f2ea9d0a640 1 -- 172.21.15.26:0/2773226001 wait complete. 2024-05-20T21:04:31.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 184957 2.32 MB/sec execute 277 sec latency 36.900 ms 2024-05-20T21:04:31.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:31 smithi026 ceph-mon[31990]: pgmap v641: 97 pgs: 97 active+clean; 29 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 342 KiB/s wr, 42 op/s 2024-05-20T21:04:31.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:31 smithi055 ceph-mon[35213]: pgmap v641: 97 pgs: 97 active+clean; 29 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 342 KiB/s wr, 42 op/s 2024-05-20T21:04:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 -- 172.21.15.26:0/229863864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 msgr2=0x7fabb8109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/229863864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 0x7fabb8109980 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 -- 172.21.15.26:0/229863864 shutdown_connections 2024-05-20T21:04:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/229863864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 0x7fabb8109980 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/229863864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/229863864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb806bab0 0x7fabb806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 -- 172.21.15.26:0/229863864 >> 172.21.15.26:0/229863864 conn(0x7fabb80793b0 msgr2=0x7fabb80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 -- 172.21.15.26:0/229863864 shutdown_connections 2024-05-20T21:04:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.835+0000 7fabbd2b4700 1 -- 172.21.15.26:0/229863864 wait complete. 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 Processor -- start 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 -- start start 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb806bab0 0x7fabb819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 0x7fabb8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8113560 con 0x7fabb81095a0 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81133e0 con 0x7fabb806c470 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabbd2b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81136e0 con 0x7fabb806bab0 2024-05-20T21:04:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabb67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabb6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb806bab0 0x7fabb819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabb67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54354/0 (socket says 172.21.15.26:54354) 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.836+0000 7fabb67fc700 1 -- 172.21.15.26:0/2568059973 learned_addr learned my addr 172.21.15.26:0/2568059973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb77fe700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 0x7fabb8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb67fc700 1 -- 172.21.15.26:0/2568059973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb806bab0 msgr2=0x7fabb819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb67fc700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb806bab0 0x7fabb819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb67fc700 1 -- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 msgr2=0x7fabb8199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb67fc700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 0x7fabb8199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb67fc700 1 -- 172.21.15.26:0/2568059973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb8199fc0 con 0x7fabb806c470 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabb67fc700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb819fbe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c9b0 tx=0x7fabac00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac007b00 con 0x7fabb806c470 2024-05-20T21:04:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb819a2b0 con 0x7fabb806c470 2024-05-20T21:04:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabac00ce40 con 0x7fabb806c470 2024-05-20T21:04:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb8102430 con 0x7fabb806c470 2024-05-20T21:04:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.837+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac018660 con 0x7fabb806c470 2024-05-20T21:04:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.838+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba4005330 con 0x7fabb806c470 2024-05-20T21:04:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.840+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fabac007ca0 con 0x7fabb806c470 2024-05-20T21:04:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.841+0000 7fab9ffff700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faba00b2900 0x7faba00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.841+0000 7fabb6ffd700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faba00b2900 0x7faba00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.841+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fabac0e2310 con 0x7fabb806c470 2024-05-20T21:04:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.841+0000 7fabb6ffd700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faba00b2900 0x7faba00b4dc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faba800b2e0 tx=0x7faba8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:31.845+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabac014030 con 0x7fabb806c470 2024-05-20T21:04:32.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.166+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faba4005620 con 0x7fabb806c470 2024-05-20T21:04:32.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.167+0000 7fab9ffff700 1 -- 172.21.15.26:0/2568059973 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fabac0aa080 con 0x7fabb806c470 2024-05-20T21:04:32.167 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:32.168 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faba00b2900 msgr2=0x7faba00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faba00b2900 0x7faba00b4dc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faba800b2e0 tx=0x7faba8005fb0 comp rx=0 tx=0).stop 2024-05-20T21:04:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 msgr2=0x7fabb819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb819fbe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c9b0 tx=0x7fabac00ccc0 comp rx=0 tx=0).stop 2024-05-20T21:04:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 shutdown_connections 2024-05-20T21:04:32.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faba00b2900 0x7faba00b4dc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabb81095a0 0x7fabb8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb806bab0 0x7fabb819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 --2- 172.21.15.26:0/2568059973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb806c470 0x7fabb819fbe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 >> 172.21.15.26:0/2568059973 conn(0x7fabb80793b0 msgr2=0x7fabb8101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 shutdown_connections 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:32.170+0000 7fabbd2b4700 1 -- 172.21.15.26:0/2568059973 wait complete. 2024-05-20T21:04:32.171 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:04:32.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:32 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2568059973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:32.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:32 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2568059973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:32.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 185564 2.32 MB/sec execute 278 sec latency 22.320 ms 2024-05-20T21:04:32.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:32 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2568059973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:32.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:04:33.266 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:33 smithi144 ceph-mon[35660]: pgmap v642: 97 pgs: 97 active+clean; 28 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 527 KiB/s wr, 47 op/s 2024-05-20T21:04:33.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 186303 2.32 MB/sec execute 279 sec latency 14.935 ms 2024-05-20T21:04:33.595 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:33 smithi026 ceph-mon[31990]: pgmap v642: 97 pgs: 97 active+clean; 28 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 527 KiB/s wr, 47 op/s 2024-05-20T21:04:33.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:33 smithi055 ceph-mon[35213]: pgmap v642: 97 pgs: 97 active+clean; 28 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 527 KiB/s wr, 47 op/s 2024-05-20T21:04:34.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 186909 2.32 MB/sec execute 280 sec latency 74.105 ms 2024-05-20T21:04:35.408 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:35 smithi026 ceph-mon[31990]: pgmap v643: 97 pgs: 97 active+clean; 26 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 925 KiB/s wr, 86 op/s 2024-05-20T21:04:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:35 smithi144 ceph-mon[35660]: pgmap v643: 97 pgs: 97 active+clean; 26 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 925 KiB/s wr, 86 op/s 2024-05-20T21:04:35.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 187761 2.34 MB/sec execute 281 sec latency 15.281 ms 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1287305320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c801035f0 msgr2=0x7f5c80103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1287305320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c801035f0 0x7f5c80103a50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098f0 tx=0x7f5c70009c00 comp rx=0 tx=0).stop 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1287305320 shutdown_connections 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1287305320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 0x7f5c80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1287305320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c80108500 0x7f5c801088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1287305320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c801035f0 0x7f5c80103a50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.671+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1287305320 >> 172.21.15.26:0/1287305320 conn(0x7f5c8007a6a0 msgr2=0x7f5c8007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.672+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1287305320 shutdown_connections 2024-05-20T21:04:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.672+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1287305320 wait complete. 2024-05-20T21:04:35.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.672+0000 7f5c87bbc700 1 Processor -- start 2024-05-20T21:04:35.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.672+0000 7f5c87bbc700 1 -- start start 2024-05-20T21:04:35.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c87bbc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 0x7f5c8019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c87bbc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 0x7f5c8019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c87bbc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c80108500 0x7f5c80199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c87bbc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c80113650 con 0x7f5c80108500 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c87bbc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801134d0 con 0x7f5c801035f0 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c87bbc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801137d0 con 0x7f5c80103f90 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 0x7f5c8019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 0x7f5c8019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54368/0 (socket says 172.21.15.26:54368) 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 -- 172.21.15.26:0/1129419923 learned_addr learned my addr 172.21.15.26:0/1129419923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c86159700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c80108500 0x7f5c80199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85157700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 0x7f5c8019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 -- 172.21.15.26:0/1129419923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 msgr2=0x7f5c8019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 0x7f5c8019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 -- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c80108500 msgr2=0x7f5c80199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c80108500 0x7f5c80199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 -- 172.21.15.26:0/1129419923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8019a0f0 con 0x7f5c801035f0 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85157700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 0x7f5c8019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.673+0000 7f5c85958700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 0x7f5c8019f7d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00ebf0 tx=0x7f5c7c00ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.674+0000 7f5c76ffd700 1 -- 172.21.15.26:0/1129419923 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00cd40 con 0x7f5c801035f0 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.674+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8019a3e0 con 0x7f5c801035f0 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.674+0000 7f5c76ffd700 1 -- 172.21.15.26:0/1129419923 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c7c0108c0 con 0x7f5c801035f0 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.674+0000 7f5c76ffd700 1 -- 172.21.15.26:0/1129419923 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0187e0 con 0x7f5c801035f0 2024-05-20T21:04:35.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.674+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c801a66a0 con 0x7f5c801035f0 2024-05-20T21:04:35.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.677+0000 7f5c76ffd700 1 -- 172.21.15.26:0/1129419923 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5c7c01f020 con 0x7f5c801035f0 2024-05-20T21:04:35.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.678+0000 7f5c76ffd700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c6c0b2880 0x7f5c6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.678+0000 7f5c85157700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c6c0b2880 0x7f5c6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.678+0000 7f5c76ffd700 1 -- 172.21.15.26:0/1129419923 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5c7c0e2490 con 0x7f5c801035f0 2024-05-20T21:04:35.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.678+0000 7f5c92452b80 1 Processor -- start 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 -- start start 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85157700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c6c0b2880 0x7f5c6c0b4d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009230 tx=0x7f5c7000fc80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc530b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52acb0 0x55b3dc52b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc537490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3dc484d70 con 0x55b3dc530730 2024-05-20T21:04:35.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b3dc474910 con 0x55b3dc52b670 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c92452b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3dc478950 con 0x55b3dc52acb0 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc530b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc530b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36976/0 (socket says 172.21.15.26:36976) 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 -- 172.21.15.26:0/2851125179 learned_addr learned my addr 172.21.15.26:0/2851125179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c86159700 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc537490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85157700 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52acb0 0x55b3dc52b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 -- 172.21.15.26:0/2851125179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52acb0 msgr2=0x55b3dc52b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52acb0 0x55b3dc52b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 -- 172.21.15.26:0/2851125179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 msgr2=0x55b3dc537490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc537490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c85958700 1 -- 172.21.15.26:0/2851125179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3dc41dfb0 con 0x55b3dc530730 2024-05-20T21:04:35.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.679+0000 7f5c86159700 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc537490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:35.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.680+0000 7f5c85958700 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc530b10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0a6cc0 tx=0x7f5c7c0a95e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894a3f452dd8f150 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:35.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.680+0000 7f5c6bfff700 1 -- 172.21.15.26:0/2851125179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0a5050 con 0x55b3dc530730 2024-05-20T21:04:35.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.680+0000 7f5c6bfff700 1 -- 172.21.15.26:0/2851125179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c7c0b2070 con 0x55b3dc530730 2024-05-20T21:04:35.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.680+0000 7f5c6bfff700 1 -- 172.21.15.26:0/2851125179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0b28b0 con 0x55b3dc530730 2024-05-20T21:04:35.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 -- 172.21.15.26:0/2851125179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 msgr2=0x55b3dc530b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc530b10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0a6cc0 tx=0x7f5c7c0a95e0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 -- 172.21.15.26:0/2851125179 shutdown_connections 2024-05-20T21:04:35.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc530b10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc537490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2851125179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52acb0 0x55b3dc52b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 -- 172.21.15.26:0/2851125179 >> 172.21.15.26:0/2851125179 conn(0x55b3dc452240 msgr2=0x55b3dc5213e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 -- 172.21.15.26:0/2851125179 shutdown_connections 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 -- 172.21.15.26:0/2851125179 wait complete. 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.681+0000 7f5c92452b80 1 -- start start 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 Processor -- start 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc537110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc534410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc534950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3dc37afa0 con 0x55b3dc52acb0 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b3dc3dcda0 con 0x55b3dc52b670 2024-05-20T21:04:35.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3dc537a30 con 0x55b3dc530730 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c92452b80 10 client.0 _ll_register_callbacks cb 0x55b3dc40f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c85958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc537110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c85157700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc534410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c85958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc537110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36986/0 (socket says 172.21.15.26:36986) 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c85157700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc534410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54380/0 (socket says 172.21.15.26:54380) 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c85958700 1 -- 172.21.15.26:0/2068101597 learned_addr learned my addr 172.21.15.26:0/2068101597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.682+0000 7f5c86159700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc534950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85958700 1 -- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 msgr2=0x55b3dc534950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85958700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc534950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85958700 1 -- 172.21.15.26:0/2068101597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 msgr2=0x55b3dc534410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85958700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc534410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85958700 1 -- 172.21.15.26:0/2068101597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dc3c48d0 con 0x55b3dc52acb0 2024-05-20T21:04:35.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85157700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc534410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c85958700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc537110 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0b22a0 tx=0x7f5c7c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a7c599e8a0040d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c697fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c697fa700 1 -- 172.21.15.26:0/2068101597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0c3060 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c697fa700 1 -- 172.21.15.26:0/2068101597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5c7c0b2420 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7c0b2070 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c697fa700 1 -- 172.21.15.26:0/2068101597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c7c0e22e0 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c92452b80 10 client.17943 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3dc42e760 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.683+0000 7f5c697fa700 1 -- 172.21.15.26:0/2068101597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c7c0b2070 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c697fa700 1 -- 172.21.15.26:0/2068101597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5c7c0104a0 con 0x55b3dc52acb0 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c92452b80 10 client.17943 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c92452b80 10 client.17943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:04:35.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c92452b80 20 client.17943 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:35.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc544410 0x55b3dc546800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c92452b80 4 client.17943 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:35.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3dc486cc0 con 0x55b3dc544410 2024-05-20T21:04:35.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.684+0000 7f5c85157700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc544410 0x55b3dc546800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.685+0000 7f5c697fa700 10 client.17943 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:35.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.685+0000 7f5c85157700 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc544410 0x55b3dc546800 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e5b6c92eefef90f server_cookie=91aeb41d82ad8a01 in_seq=0 out_seq=0 2024-05-20T21:04:35.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.685+0000 7f5c697fa700 1 -- 172.21.15.26:0/2068101597 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b3dc486cc0 con 0x55b3dc544410 2024-05-20T21:04:35.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.685+0000 7f5c697fa700 10 client.17943 handle_command_reply: tid=0 2024-05-20T21:04:35.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.685+0000 7f5c92452b80 1 client.17943 shutdown 2024-05-20T21:04:35.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 msgr2=0x55b3dc537110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc537110 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0b22a0 tx=0x7f5c7c00c800 comp rx=0 tx=0).stop 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 shutdown_connections 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc544410 0x55b3dc546800 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc537110 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc534950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 --2- 172.21.15.26:0/2068101597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc534410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c697fa700 0 client.17943 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 >> 172.21.15.26:0/2068101597 conn(0x55b3dc452240 msgr2=0x55b3dc520af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 shutdown_connections 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 1 -- 172.21.15.26:0/2068101597 wait complete. 2024-05-20T21:04:35.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.686+0000 7f5c92452b80 20 client.17943 trim_cache size 0 max 16384 2024-05-20T21:04:35.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:35 smithi055 ceph-mon[35213]: pgmap v643: 97 pgs: 97 active+clean; 26 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 925 KiB/s wr, 86 op/s 2024-05-20T21:04:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 Processor -- start 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 -- start start 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc529ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc52a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc52a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801ae4b0 con 0x55b3dc52acb0 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801abd40 con 0x55b3dc52b670 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c92452b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801a9b90 con 0x55b3dc530730 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc529ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85958700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc529ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36988/0 (socket says 172.21.15.26:36988) 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85958700 1 -- 172.21.15.26:0/4294142302 learned_addr learned my addr 172.21.15.26:0/4294142302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85157700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc52a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c86159700 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc52a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85157700 1 -- 172.21.15.26:0/4294142302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 msgr2=0x55b3dc52a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85157700 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc52a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85157700 1 -- 172.21.15.26:0/4294142302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 msgr2=0x55b3dc529ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85157700 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc529ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85157700 1 -- 172.21.15.26:0/4294142302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3dc532980 con 0x55b3dc52b670 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.705+0000 7f5c85958700 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc529ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.706+0000 7f5c85157700 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc52a1e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c80109b70 tx=0x7f5c70021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbaf757baa221d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.706+0000 7f5c6b7fe700 1 -- 172.21.15.26:0/4294142302 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700054a0 con 0x55b3dc52b670 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.706+0000 7f5c6b7fe700 1 -- 172.21.15.26:0/4294142302 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c70005640 con 0x55b3dc52b670 2024-05-20T21:04:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.706+0000 7f5c6b7fe700 1 -- 172.21.15.26:0/4294142302 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000ba40 con 0x55b3dc52b670 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 -- 172.21.15.26:0/4294142302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 msgr2=0x55b3dc52a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc52a1e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c80109b70 tx=0x7f5c70021000 comp rx=0 tx=0).stop 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 -- 172.21.15.26:0/4294142302 shutdown_connections 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc52acb0 0x55b3dc529ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc530730 0x55b3dc52a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 --2- 172.21.15.26:0/4294142302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52b670 0x55b3dc52a1e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 -- 172.21.15.26:0/4294142302 >> 172.21.15.26:0/4294142302 conn(0x55b3dc452240 msgr2=0x55b3dc5364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 -- 172.21.15.26:0/4294142302 shutdown_connections 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 -- 172.21.15.26:0/4294142302 wait complete. 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 -- start start 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.707+0000 7f5c92452b80 1 Processor -- start 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 0x55b3dc549810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52b670 0x55b3dc544b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc5450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801aa200 con 0x55b3dc530730 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801ab670 con 0x55b3dc52acb0 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801aa8d0 con 0x55b3dc52b670 2024-05-20T21:04:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c92452b80 10 client.0 _ll_register_callbacks cb 0x55b3dc40f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 0x55b3dc549810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 0x55b3dc549810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54404/0 (socket says 172.21.15.26:54404) 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 -- 172.21.15.26:0/3455707278 learned_addr learned my addr 172.21.15.26:0/3455707278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c86159700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc5450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85157700 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52b670 0x55b3dc544b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 -- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52b670 msgr2=0x55b3dc544b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52b670 0x55b3dc544b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 -- 172.21.15.26:0/3455707278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 msgr2=0x55b3dc5450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc5450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 -- 172.21.15.26:0/3455707278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dc545610 con 0x55b3dc52acb0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c85958700 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 0x55b3dc549810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c010890 tx=0x7f5c7c0049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b8cc37f7ebc25dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.708+0000 7f5c697fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c697fa700 1 -- 172.21.15.26:0/3455707278 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0a9820 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c697fa700 1 -- 172.21.15.26:0/3455707278 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5c7c0b2630 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c70005640 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c697fa700 1 -- 172.21.15.26:0/3455707278 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c7c0a5910 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 10 client.27356 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3dc5ae950 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c697fa700 1 -- 172.21.15.26:0/3455707278 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c70005640 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c697fa700 1 -- 172.21.15.26:0/3455707278 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5c7c06f210 con 0x55b3dc52acb0 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 10 client.27356 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 10 client.27356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 20 client.27356 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc5aecf0 0x55b3dc5b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 4 client.27356 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.709+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3dc4804a0 con 0x55b3dc5aecf0 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c85157700 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc5aecf0 0x55b3dc5b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c697fa700 10 client.27356 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c85157700 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc5aecf0 0x55b3dc5b1110 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=110489703da49f21 server_cookie=efc7ef8a9f62e5a0 in_seq=0 out_seq=0 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c697fa700 1 -- 172.21.15.26:0/3455707278 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3dc4804a0 con 0x55b3dc5aecf0 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c697fa700 10 client.27356 handle_command_reply: tid=0 2024-05-20T21:04:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c92452b80 1 client.27356 shutdown 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 msgr2=0x55b3dc549810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.710+0000 7f5c92452b80 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 0x55b3dc549810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c010890 tx=0x7f5c7c0049e0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 shutdown_connections 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55b3dc5aecf0 0x55b3dc5b1110 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dc530730 0x55b3dc5450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b3dc52acb0 0x55b3dc549810 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c697fa700 0 client.27356 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:04:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 --2- 172.21.15.26:0/3455707278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3dc52b670 0x55b3dc544b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 >> 172.21.15.26:0/3455707278 conn(0x55b3dc452240 msgr2=0x55b3dc531420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 shutdown_connections 2024-05-20T21:04:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 1 -- 172.21.15.26:0/3455707278 wait complete. 2024-05-20T21:04:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c92452b80 20 client.27356 trim_cache size 0 max 16384 2024-05-20T21:04:35.716 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c6c0b2880 msgr2=0x7f5c6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c6c0b2880 0x7f5c6c0b4d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009230 tx=0x7f5c7000fc80 comp rx=0 tx=0).stop 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 msgr2=0x7f5c8019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.711+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 0x7f5c8019f7d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00ebf0 tx=0x7f5c7c00ef00 comp rx=0 tx=0).stop 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 shutdown_connections 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c6c0b2880 0x7f5c6c0b4d40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c80108500 0x7f5c80199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c801035f0 0x7f5c8019f7d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 --2- 172.21.15.26:0/1129419923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c80103f90 0x7f5c8019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 >> 172.21.15.26:0/1129419923 conn(0x7f5c8007a6a0 msgr2=0x7f5c80101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 shutdown_connections 2024-05-20T21:04:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:35.712+0000 7f5c87bbc700 1 -- 172.21.15.26:0/1129419923 wait complete. 2024-05-20T21:04:36.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:04:36.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 188281 2.35 MB/sec execute 282 sec latency 17.430 ms 2024-05-20T21:04:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:37 smithi026 ceph-mon[31990]: pgmap v644: 97 pgs: 97 active+clean; 27 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-05-20T21:04:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:37 smithi144 ceph-mon[35660]: pgmap v644: 97 pgs: 97 active+clean; 27 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-05-20T21:04:37.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 188727 2.34 MB/sec execute 283 sec latency 19.784 ms 2024-05-20T21:04:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:37 smithi055 ceph-mon[35213]: pgmap v644: 97 pgs: 97 active+clean; 27 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-05-20T21:04:38.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 189411 2.35 MB/sec execute 284 sec latency 23.133 ms 2024-05-20T21:04:38.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 -- 172.21.15.26:0/4213475543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f861806b220 msgr2=0x7f861806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 --2- 172.21.15.26:0/4213475543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f861806b220 0x7f861806b6a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8608009990 tx=0x7f8608009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 -- 172.21.15.26:0/4213475543 shutdown_connections 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 --2- 172.21.15.26:0/4213475543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f86181103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 --2- 172.21.15.26:0/4213475543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86181095b0 0x7f8618109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 --2- 172.21.15.26:0/4213475543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f861806b220 0x7f861806b6a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 -- 172.21.15.26:0/4213475543 >> 172.21.15.26:0/4213475543 conn(0x7f86180793f0 msgr2=0x7f8618079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 -- 172.21.15.26:0/4213475543 shutdown_connections 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.604+0000 7f861fd36700 1 -- 172.21.15.26:0/4213475543 wait complete. 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 Processor -- start 2024-05-20T21:04:38.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 -- start start 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 0x7f8618102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f8618102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86181095b0 0x7f8618102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86181135f0 con 0x7f86181095b0 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8618113470 con 0x7f861806bbe0 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861d2d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f8618102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861dad2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 0x7f8618102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861fd36700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8618113770 con 0x7f861806b220 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861d2d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f8618102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54432/0 (socket says 172.21.15.26:54432) 2024-05-20T21:04:38.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861dad2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 0x7f8618102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49440/0 (socket says 172.21.15.26:49440) 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861d2d1700 1 -- 172.21.15.26:0/348200972 learned_addr learned my addr 172.21.15.26:0/348200972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.605+0000 7f861e2d3700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86181095b0 0x7f8618102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861e2d3700 1 -- 172.21.15.26:0/348200972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 msgr2=0x7f8618102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861e2d3700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 0x7f8618102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861e2d3700 1 -- 172.21.15.26:0/348200972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 msgr2=0x7f8618102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861e2d3700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f8618102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861e2d3700 1 -- 172.21.15.26:0/348200972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86181a80b0 con 0x7f86181095b0 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861dad2700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 0x7f8618102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861d2d1700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f8618102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861e2d3700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86181095b0 0x7f8618102f90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f861000efc0 tx=0x7f861000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f860effd700 1 -- 172.21.15.26:0/348200972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100092e0 con 0x7f86181095b0 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f860effd700 1 -- 172.21.15.26:0/348200972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8610004bb0 con 0x7f86181095b0 2024-05-20T21:04:38.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f860effd700 1 -- 172.21.15.26:0/348200972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610004020 con 0x7f86181095b0 2024-05-20T21:04:38.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.606+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86181a8340 con 0x7f86181095b0 2024-05-20T21:04:38.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.607+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181a8830 con 0x7f86181095b0 2024-05-20T21:04:38.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.609+0000 7f860effd700 1 -- 172.21.15.26:0/348200972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f861001b030 con 0x7f86181095b0 2024-05-20T21:04:38.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.610+0000 7f860effd700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86040b2880 0x7f86040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.610+0000 7f861dad2700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86040b2880 0x7f86040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.610+0000 7f860effd700 1 -- 172.21.15.26:0/348200972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f86100e1a00 con 0x7f86181095b0 2024-05-20T21:04:38.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.611+0000 7f861dad2700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86040b2880 0x7f86040b4d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8614009fd0 tx=0x7f8614009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:38.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.611+0000 7f862a5ccb80 1 Processor -- start 2024-05-20T21:04:38.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.611+0000 7f862a5ccb80 1 -- start start 2024-05-20T21:04:38.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.611+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f44c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.611+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f48940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559017ea3c90 con 0x559017f45210 2024-05-20T21:04:38.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559017e93830 con 0x559017f44860 2024-05-20T21:04:38.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559017e97870 con 0x559017f48f60 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f44c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f44c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54440/0 (socket says 172.21.15.26:54440) 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 -- 172.21.15.26:0/282021839 learned_addr learned my addr 172.21.15.26:0/282021839 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861d2d1700 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f48940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861e2d3700 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 -- 172.21.15.26:0/282021839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 msgr2=0x559017f493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 -- 172.21.15.26:0/282021839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 msgr2=0x559017f48940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f48940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 -- 172.21.15.26:0/282021839 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559017de38d0 con 0x559017f44860 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f861dad2700 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f44c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86140043c0 tx=0x7f8614014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea40e1bcea7dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:38.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.612+0000 7f8603fff700 1 -- 172.21.15.26:0/282021839 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614003d10 con 0x559017f44860 2024-05-20T21:04:38.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f8603fff700 1 -- 172.21.15.26:0/282021839 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8614026890 con 0x559017f44860 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f8603fff700 1 -- 172.21.15.26:0/282021839 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861401e750 con 0x559017f44860 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 -- 172.21.15.26:0/282021839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 msgr2=0x559017f44c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f44c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86140043c0 tx=0x7f8614014000 comp rx=0 tx=0).stop 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 -- 172.21.15.26:0/282021839 shutdown_connections 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f44c40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/282021839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f48940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.613+0000 7f862a5ccb80 1 -- 172.21.15.26:0/282021839 >> 172.21.15.26:0/282021839 conn(0x559017e72080 msgr2=0x559017f3ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.614+0000 7f862a5ccb80 1 -- 172.21.15.26:0/282021839 shutdown_connections 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.614+0000 7f862a5ccb80 1 -- 172.21.15.26:0/282021839 wait complete. 2024-05-20T21:04:38.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.614+0000 7f862a5ccb80 1 -- start start 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.614+0000 7f862a5ccb80 1 Processor -- start 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f44860 0x559017f50380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f45210 0x559017f4d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f4dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559017d99fa0 con 0x559017f48f60 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559017e357c0 con 0x559017f45210 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559017f51120 con 0x559017f44860 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f862a5ccb80 10 client.0 _ll_register_callbacks cb 0x559017e2e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861e2d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f4dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861e2d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f4dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37036/0 (socket says 172.21.15.26:37036) 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861e2d3700 1 -- 172.21.15.26:0/4177191303 learned_addr learned my addr 172.21.15.26:0/4177191303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861dad2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f44860 0x559017f50380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861d2d1700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f45210 0x559017f4d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861dad2700 1 -- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f45210 msgr2=0x559017f4d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861dad2700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f45210 0x559017f4d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861dad2700 1 -- 172.21.15.26:0/4177191303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 msgr2=0x559017f4dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861dad2700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f4dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861dad2700 1 -- 172.21.15.26:0/4177191303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559017e32690 con 0x559017f44860 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861d2d1700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f45210 0x559017f4d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.615+0000 7f861e2d3700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f4dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f861dad2700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f44860 0x559017f50380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8614025000 tx=0x7f8614003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8f31b4bdd5e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f86017fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f86017fa700 1 -- 172.21.15.26:0/4177191303 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614026c50 con 0x559017f44860 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8614026890 con 0x559017f44860 2024-05-20T21:04:38.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f86017fa700 1 -- 172.21.15.26:0/4177191303 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8614030420 con 0x559017f44860 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f86017fa700 1 -- 172.21.15.26:0/4177191303 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8614039050 con 0x559017f44860 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f862a5ccb80 10 client.27187 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559017f5b830 con 0x559017f44860 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f86017fa700 1 -- 172.21.15.26:0/4177191303 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8614026890 con 0x559017f44860 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.616+0000 7f86017fa700 1 -- 172.21.15.26:0/4177191303 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f861401e510 con 0x559017f44860 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f862a5ccb80 10 client.27187 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f862a5ccb80 10 client.27187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f862a5ccb80 20 client.27187 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017f5ddb0 0x559017f601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f862a5ccb80 4 client.27187 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559017f109e0 con 0x559017f5ddb0 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f861d2d1700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017f5ddb0 0x559017f601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f861d2d1700 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017f5ddb0 0x559017f601a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=988b019eba053cfa server_cookie=1e1152148e02d991 in_seq=0 out_seq=0 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.617+0000 7f86017fa700 10 client.27187 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f86017fa700 1 -- 172.21.15.26:0/4177191303 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559017f109e0 con 0x559017f5ddb0 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f86017fa700 10 client.27187 handle_command_reply: tid=0 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 client.27187 shutdown 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f44860 msgr2=0x559017f50380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f44860 0x559017f50380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8614025000 tx=0x7f8614003d40 comp rx=0 tx=0).stop 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 shutdown_connections 2024-05-20T21:04:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017f5ddb0 0x559017f601a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f4dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f44860 0x559017f50380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/4177191303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f45210 0x559017f4d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f86017fa700 0 client.27187 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:38.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.618+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 >> 172.21.15.26:0/4177191303 conn(0x559017e72080 msgr2=0x559017f3e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:38.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.619+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 shutdown_connections 2024-05-20T21:04:38.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.619+0000 7f862a5ccb80 1 -- 172.21.15.26:0/4177191303 wait complete. 2024-05-20T21:04:38.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.619+0000 7f862a5ccb80 20 client.27187 trim_cache size 0 max 16384 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.638+0000 7f862a5ccb80 1 Processor -- start 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.638+0000 7f862a5ccb80 1 -- start start 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f47f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f3f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b28a0 con 0x559017f45210 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b0130 con 0x559017f44860 2024-05-20T21:04:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86181adf80 con 0x559017f48f60 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f3ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37042/0 (socket says 172.21.15.26:37042) 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 -- 172.21.15.26:0/706013372 learned_addr learned my addr 172.21.15.26:0/706013372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861e2d3700 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f3f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861dad2700 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f47f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 -- 172.21.15.26:0/706013372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 msgr2=0x559017f3f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f3f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 -- 172.21.15.26:0/706013372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 msgr2=0x559017f47f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f47f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 -- 172.21.15.26:0/706013372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559017f48cb0 con 0x559017f45210 2024-05-20T21:04:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.639+0000 7f861d2d1700 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f3ec90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f860800b650 tx=0x7f860801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd333b8968496182 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.640+0000 7f86037fe700 1 -- 172.21.15.26:0/706013372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608005690 con 0x559017f45210 2024-05-20T21:04:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.640+0000 7f86037fe700 1 -- 172.21.15.26:0/706013372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8608003b10 con 0x559017f45210 2024-05-20T21:04:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.640+0000 7f86037fe700 1 -- 172.21.15.26:0/706013372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800f930 con 0x559017f45210 2024-05-20T21:04:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 -- 172.21.15.26:0/706013372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 msgr2=0x559017f3ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f3ec90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f860800b650 tx=0x7f860801a040 comp rx=0 tx=0).stop 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 -- 172.21.15.26:0/706013372 shutdown_connections 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f48f60 0x559017f3f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017f47f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/706013372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f45210 0x559017f3ec90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 -- 172.21.15.26:0/706013372 >> 172.21.15.26:0/706013372 conn(0x559017e72080 msgr2=0x559017f43f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 -- 172.21.15.26:0/706013372 shutdown_connections 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 -- 172.21.15.26:0/706013372 wait complete. 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 -- start start 2024-05-20T21:04:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.641+0000 7f862a5ccb80 1 Processor -- start 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017fc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f45210 0x559017fc1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f62a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86181ae5f0 con 0x559017f48f60 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86181afa60 con 0x559017f44860 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86181aecc0 con 0x559017f45210 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f862a5ccb80 10 client.0 _ll_register_callbacks cb 0x559017e2e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017fc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017fc0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54472/0 (socket says 172.21.15.26:54472) 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 -- 172.21.15.26:0/2340718560 learned_addr learned my addr 172.21.15.26:0/2340718560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861e2d3700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f62a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861d2d1700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f45210 0x559017fc1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 -- 172.21.15.26:0/2340718560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f45210 msgr2=0x559017fc1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f45210 0x559017fc1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 -- 172.21.15.26:0/2340718560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 msgr2=0x559017f62a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f62a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 -- 172.21.15.26:0/2340718560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559017f632c0 con 0x559017f44860 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861e2d3700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f62a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f861dad2700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017fc0d30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8614005bb0 tx=0x7f861401ead0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0b93c8fdf952f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f86017fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.642+0000 7f86017fa700 1 -- 172.21.15.26:0/2340718560 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614041070 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8608003b10 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f86017fa700 1 -- 172.21.15.26:0/2340718560 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f861402c040 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f86017fa700 1 -- 172.21.15.26:0/2340718560 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f861404d050 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 10 client.27374 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559017fc8460 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f86017fa700 1 -- 172.21.15.26:0/2340718560 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8608003b10 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f86017fa700 1 -- 172.21.15.26:0/2340718560 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8614030440 con 0x559017f44860 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 10 client.27374 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 10 client.27374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 20 client.27374 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017fc8950 0x559017fcad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 4 client.27374 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559017ea5be0 con 0x559017fc8950 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.643+0000 7f861d2d1700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017fc8950 0x559017fcad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f86017fa700 10 client.27374 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f861d2d1700 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017fc8950 0x559017fcad40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=797b5c593dec433a server_cookie=16facd377b8311c4 in_seq=0 out_seq=0 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f86017fa700 1 -- 172.21.15.26:0/2340718560 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559017ea5be0 con 0x559017fc8950 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f86017fa700 10 client.27374 handle_command_reply: tid=0 2024-05-20T21:04:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 client.27374 shutdown 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 msgr2=0x559017fc0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017fc0d30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8614005bb0 tx=0x7f861401ead0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 shutdown_connections 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559017fc8950 0x559017fcad40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559017f48f60 0x559017f62a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559017f44860 0x559017fc0d30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 --2- 172.21.15.26:0/2340718560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559017f45210 0x559017fc1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f86017fa700 0 client.27374 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 >> 172.21.15.26:0/2340718560 conn(0x559017e72080 msgr2=0x559017f4eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.644+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 shutdown_connections 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f862a5ccb80 1 -- 172.21.15.26:0/2340718560 wait complete. 2024-05-20T21:04:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f862a5ccb80 20 client.27374 trim_cache size 0 max 16384 2024-05-20T21:04:38.653 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86040b2880 msgr2=0x7f86040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86040b2880 0x7f86040b4d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8614009fd0 tx=0x7f8614009260 comp rx=0 tx=0).stop 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86181095b0 msgr2=0x7f8618102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86181095b0 0x7f8618102f90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f861000efc0 tx=0x7f861000c3a0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 shutdown_connections 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86040b2880 0x7f86040b4d40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86181095b0 0x7f8618102f90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f861806b220 0x7f8618102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 --2- 172.21.15.26:0/348200972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f861806bbe0 0x7f8618102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:38.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.645+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 >> 172.21.15.26:0/348200972 conn(0x7f86180793f0 msgr2=0x7f8618101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:38.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.646+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 shutdown_connections 2024-05-20T21:04:38.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:38.646+0000 7f861fd36700 1 -- 172.21.15.26:0/348200972 wait complete. 2024-05-20T21:04:39.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:39 smithi026 ceph-mon[31990]: pgmap v645: 97 pgs: 97 active+clean; 28 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-20T21:04:39.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:04:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:39 smithi144 ceph-mon[35660]: pgmap v645: 97 pgs: 97 active+clean; 28 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-20T21:04:39.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 190097 2.34 MB/sec execute 285 sec latency 17.795 ms 2024-05-20T21:04:39.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:39 smithi055 ceph-mon[35213]: pgmap v645: 97 pgs: 97 active+clean; 28 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-20T21:04:39.783 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:40.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 190624 2.34 MB/sec execute 286 sec latency 69.803 ms 2024-05-20T21:04:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:41 smithi144 ceph-mon[35660]: pgmap v646: 97 pgs: 97 active+clean; 27 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-05-20T21:04:41.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 191320 2.36 MB/sec execute 287 sec latency 116.844 ms 2024-05-20T21:04:41.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:41 smithi055 ceph-mon[35213]: pgmap v646: 97 pgs: 97 active+clean; 27 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-05-20T21:04:41.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:41 smithi026 ceph-mon[31990]: pgmap v646: 97 pgs: 97 active+clean; 27 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-05-20T21:04:42.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 191831 2.36 MB/sec execute 288 sec latency 108.005 ms 2024-05-20T21:04:43.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:43 smithi026 ceph-mon[31990]: pgmap v647: 97 pgs: 97 active+clean; 27 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:04:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:43 smithi144 ceph-mon[35660]: pgmap v647: 97 pgs: 97 active+clean; 27 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:04:43.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 192286 2.36 MB/sec execute 289 sec latency 19.860 ms 2024-05-20T21:04:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:43 smithi055 ceph-mon[35213]: pgmap v647: 97 pgs: 97 active+clean; 27 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:04:43.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 -- 172.21.15.26:0/3037601591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06bbe0 msgr2=0x7f919c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 --2- 172.21.15.26:0/3037601591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06bbe0 0x7f919c1103d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f919400b0a0 tx=0x7f919400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 -- 172.21.15.26:0/3037601591 shutdown_connections 2024-05-20T21:04:43.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 --2- 172.21.15.26:0/3037601591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06bbe0 0x7f919c1103d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 --2- 172.21.15.26:0/3037601591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f919c1095b0 0x7f919c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 --2- 172.21.15.26:0/3037601591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06b220 0x7f919c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 -- 172.21.15.26:0/3037601591 >> 172.21.15.26:0/3037601591 conn(0x7f919c0793f0 msgr2=0x7f919c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.911+0000 7f91a3c25700 1 -- 172.21.15.26:0/3037601591 shutdown_connections 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 -- 172.21.15.26:0/3037601591 wait complete. 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 Processor -- start 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 -- start start 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06b220 0x7f919c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 0x7f919c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f919c1095b0 0x7f919c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f919c113570 con 0x7f919c06b220 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1133f0 con 0x7f919c06bbe0 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.912+0000 7f91a3c25700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1136f0 con 0x7f919c1095b0 2024-05-20T21:04:43.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a19c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06b220 0x7f919c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 0x7f919c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 0x7f919c102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54570/0 (socket says 172.21.15.26:54570) 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a19c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06b220 0x7f919c102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52730/0 (socket says 172.21.15.26:52730) 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 -- 172.21.15.26:0/1997901694 learned_addr learned my addr 172.21.15.26:0/1997901694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a21c2700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f919c1095b0 0x7f919c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 -- 172.21.15.26:0/1997901694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f919c1095b0 msgr2=0x7f919c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f919c1095b0 0x7f919c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 -- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06b220 msgr2=0x7f919c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06b220 0x7f919c102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 -- 172.21.15.26:0/1997901694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c1a7fd0 con 0x7f919c06bbe0 2024-05-20T21:04:43.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a11c0700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 0x7f919c102a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9a0 tx=0x7f918c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:43.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f9192ffd700 1 -- 172.21.15.26:0/1997901694 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00ca10 con 0x7f919c06bbe0 2024-05-20T21:04:43.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f9192ffd700 1 -- 172.21.15.26:0/1997901694 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f918c00cbb0 con 0x7f919c06bbe0 2024-05-20T21:04:43.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.913+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c1a8260 con 0x7f919c06bbe0 2024-05-20T21:04:43.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.914+0000 7f9192ffd700 1 -- 172.21.15.26:0/1997901694 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c018680 con 0x7f919c06bbe0 2024-05-20T21:04:43.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.914+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c1a87f0 con 0x7f919c06bbe0 2024-05-20T21:04:43.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.916+0000 7f9192ffd700 1 -- 172.21.15.26:0/1997901694 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f918c010430 con 0x7f919c06bbe0 2024-05-20T21:04:43.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.917+0000 7f9192ffd700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b2900 0x7f91880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b2900 0x7f91880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f9192ffd700 1 -- 172.21.15.26:0/1997901694 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f918c0e23c0 con 0x7f919c06bbe0 2024-05-20T21:04:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 Processor -- start 2024-05-20T21:04:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 -- start start 2024-05-20T21:04:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b2900 0x7f91880b4dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9198005950 tx=0x7f919800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b18a0 0x5561288b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b2260 0x5561288be080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55612880fd70 con 0x5561288b7320 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5561287ff910 con 0x5561288b18a0 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556128803950 con 0x5561288b2260 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.918+0000 7f91a19c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52732/0 (socket says 172.21.15.26:52732) 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 -- 172.21.15.26:0/4009279427 learned_addr learned my addr 172.21.15.26:0/4009279427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a11c0700 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b18a0 0x5561288b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a21c2700 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b2260 0x5561288be080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 -- 172.21.15.26:0/4009279427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b2260 msgr2=0x5561288be080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b2260 0x5561288be080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 -- 172.21.15.26:0/4009279427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b18a0 msgr2=0x5561288b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b18a0 0x5561288b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 -- 172.21.15.26:0/4009279427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561287a8fb0 con 0x5561288b7320 2024-05-20T21:04:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a11c0700 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b18a0 0x5561288b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f91a19c1700 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288b7700 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f919800fd10 tx=0x7f9198012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3c160b1ea69f6a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f9187fff700 1 -- 172.21.15.26:0/4009279427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198016070 con 0x5561288b7320 2024-05-20T21:04:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f9187fff700 1 -- 172.21.15.26:0/4009279427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198010c70 con 0x5561288b7320 2024-05-20T21:04:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.919+0000 7f9187fff700 1 -- 172.21.15.26:0/4009279427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198025ca0 con 0x5561288b7320 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/4009279427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 msgr2=0x5561288b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288b7700 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f919800fd10 tx=0x7f9198012000 comp rx=0 tx=0).stop 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/4009279427 shutdown_connections 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288b7700 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b2260 0x5561288be080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/4009279427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b18a0 0x5561288b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/4009279427 >> 172.21.15.26:0/4009279427 conn(0x5561287dd240 msgr2=0x5561288ac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.920+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/4009279427 shutdown_connections 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/4009279427 wait complete. 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 -- start start 2024-05-20T21:04:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 Processor -- start 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288bb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556128705fa0 con 0x5561288b7320 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556128767da0 con 0x5561288b2260 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.921+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5561288be6e0 con 0x5561288b18a0 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91ae4bbb80 10 client.0 _ll_register_callbacks cb 0x55612879a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288bb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288bb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52748/0 (socket says 172.21.15.26:52748) 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 -- 172.21.15.26:0/1925422362 learned_addr learned my addr 172.21.15.26:0/1925422362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a11c0700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 -- 172.21.15.26:0/1925422362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 msgr2=0x5561288bdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288bdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 -- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 msgr2=0x5561288bb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288bb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 -- 172.21.15.26:0/1925422362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612874f8d0 con 0x5561288b7320 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a11c0700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288bdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91a21c2700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288bb580 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f919400bf90 tx=0x7f9194004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae4e685a822fb0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91857fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.922+0000 7f91857fa700 1 -- 172.21.15.26:0/1925422362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91940076e0 con 0x5561288b7320 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91857fa700 1 -- 172.21.15.26:0/1925422362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9194007880 con 0x5561288b7320 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9198010c70 con 0x5561288b7320 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91857fa700 1 -- 172.21.15.26:0/1925422362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9194007a80 con 0x5561288b7320 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 10 client.17979 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:43.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561287b9760 con 0x5561288b7320 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91857fa700 1 -- 172.21.15.26:0/1925422362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198010c70 con 0x5561288b7320 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91857fa700 1 -- 172.21.15.26:0/1925422362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9194030050 con 0x5561288b7320 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 10 client.17979 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 20 client.17979 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5561288cb190 0x5561288cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 4 client.17979 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.923+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556128811cc0 con 0x5561288cb190 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.924+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5561288cb190 0x5561288cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.924+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5561288cb190 0x5561288cd580 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe6e55d18204586 server_cookie=b6e760ff45879129 in_seq=0 out_seq=0 2024-05-20T21:04:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.924+0000 7f91857fa700 10 client.17979 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91857fa700 1 -- 172.21.15.26:0/1925422362 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556128811cc0 con 0x5561288cb190 2024-05-20T21:04:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91857fa700 10 client.17979 handle_command_reply: tid=0 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 client.17979 shutdown 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 msgr2=0x5561288bb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288bb580 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f919400bf90 tx=0x7f9194004ac0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 shutdown_connections 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5561288cb190 0x5561288cd580 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b7320 0x5561288bb580 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288bdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91857fa700 0 client.17979 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1925422362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 >> 172.21.15.26:0/1925422362 conn(0x5561287dd240 msgr2=0x5561288abaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 shutdown_connections 2024-05-20T21:04:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1925422362 wait complete. 2024-05-20T21:04:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.925+0000 7f91ae4bbb80 20 client.17979 trim_cache size 0 max 16384 2024-05-20T21:04:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.942+0000 7f91ae4bbb80 1 Processor -- start 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 -- start start 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 0x5561288b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b7320 0x5561288d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b26f0 con 0x5561288b18a0 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1aff80 con 0x5561288b2260 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1addd0 con 0x5561288b7320 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 0x5561288b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 0x5561288b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52752/0 (socket says 172.21.15.26:52752) 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 -- 172.21.15.26:0/1480306141 learned_addr learned my addr 172.21.15.26:0/1480306141 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a11c0700 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a21c2700 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b7320 0x5561288d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 -- 172.21.15.26:0/1480306141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b7320 msgr2=0x5561288d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b7320 0x5561288d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 -- 172.21.15.26:0/1480306141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 msgr2=0x5561288ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 -- 172.21.15.26:0/1480306141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561288cdbe0 con 0x5561288b18a0 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.943+0000 7f91a19c1700 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 0x5561288b68a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f919800fd10 tx=0x7f919800a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43e0655972eff69 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.944+0000 7f91877fe700 1 -- 172.21.15.26:0/1480306141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198016070 con 0x5561288b18a0 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.944+0000 7f91877fe700 1 -- 172.21.15.26:0/1480306141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198025d30 con 0x5561288b18a0 2024-05-20T21:04:43.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.944+0000 7f91877fe700 1 -- 172.21.15.26:0/1480306141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91980054d0 con 0x5561288b18a0 2024-05-20T21:04:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.944+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1480306141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 msgr2=0x5561288b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.944+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 0x5561288b68a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f919800fd10 tx=0x7f919800a370 comp rx=0 tx=0).stop 2024-05-20T21:04:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1480306141 shutdown_connections 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b18a0 0x5561288b68a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b7320 0x5561288d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/1480306141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b2260 0x5561288ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1480306141 >> 172.21.15.26:0/1480306141 conn(0x5561287dd240 msgr2=0x5561288b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1480306141 shutdown_connections 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/1480306141 wait complete. 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 -- start start 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 Processor -- start 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.945+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 0x5561288d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b7320 0x5561288d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1ae440 con 0x5561288b2260 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1af8b0 con 0x5561288b7320 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1aeb10 con 0x5561288b18a0 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 10 client.0 _ll_register_callbacks cb 0x55612879a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 0x5561288d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 0x5561288d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52764/0 (socket says 172.21.15.26:52764) 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 -- 172.21.15.26:0/608836219 learned_addr learned my addr 172.21.15.26:0/608836219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a19c1700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a21c2700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b7320 0x5561288d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 -- 172.21.15.26:0/608836219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 msgr2=0x5561288c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 -- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b7320 msgr2=0x5561288d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b7320 0x5561288d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 -- 172.21.15.26:0/608836219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561288d3010 con 0x5561288b2260 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a21c2700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b7320 0x5561288d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91a11c0700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 0x5561288d24e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f918c0a9860 tx=0x7f918c0a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb0106cb7505de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91857fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91857fa700 1 -- 172.21.15.26:0/608836219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0a9920 con 0x5561288b2260 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91857fa700 1 -- 172.21.15.26:0/608836219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f918c0a5f10 con 0x5561288b2260 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.946+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9198025d30 con 0x5561288b2260 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91857fa700 1 -- 172.21.15.26:0/608836219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f918c0adcd0 con 0x5561288b2260 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 10 client.17985 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556128935b70 con 0x5561288b2260 2024-05-20T21:04:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91857fa700 1 -- 172.21.15.26:0/608836219 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198025d30 con 0x5561288b2260 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91857fa700 1 -- 172.21.15.26:0/608836219 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f918c0a5290 con 0x5561288b2260 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 10 client.17985 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 20 client.17985 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556128936280 0x5561289386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 4 client.17985 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55612880b4a0 con 0x556128936280 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.947+0000 7f91a19c1700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556128936280 0x5561289386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91a19c1700 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556128936280 0x5561289386a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c75f6c2976082cf server_cookie=3304148355adf024 in_seq=0 out_seq=0 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91857fa700 10 client.17985 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91857fa700 1 -- 172.21.15.26:0/608836219 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55612880b4a0 con 0x556128936280 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91857fa700 10 client.17985 handle_command_reply: tid=0 2024-05-20T21:04:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 client.17985 shutdown 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 msgr2=0x5561288d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 0x5561288d24e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f918c0a9860 tx=0x7f918c0a5910 comp rx=0 tx=0).stop 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 shutdown_connections 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556128936280 0x5561289386a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5561288b7320 0x5561288d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561288b18a0 0x5561288c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 --2- 172.21.15.26:0/608836219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561288b2260 0x5561288d24e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91857fa700 0 client.17985 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 >> 172.21.15.26:0/608836219 conn(0x5561287dd240 msgr2=0x5561288b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 shutdown_connections 2024-05-20T21:04:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 1 -- 172.21.15.26:0/608836219 wait complete. 2024-05-20T21:04:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.948+0000 7f91ae4bbb80 20 client.17985 trim_cache size 0 max 16384 2024-05-20T21:04:43.955 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:04:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b2900 msgr2=0x7f91880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b2900 0x7f91880b4dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9198005950 tx=0x7f919800a300 comp rx=0 tx=0).stop 2024-05-20T21:04:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 msgr2=0x7f919c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 0x7f919c102a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9a0 tx=0x7f918c00ecb0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 shutdown_connections 2024-05-20T21:04:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f91880b2900 0x7f91880b4dc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c06b220 0x7f919c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f919c1095b0 0x7f919c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 --2- 172.21.15.26:0/1997901694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919c06bbe0 0x7f919c102a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:43.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 >> 172.21.15.26:0/1997901694 conn(0x7f919c0793f0 msgr2=0x7f919c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:43.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 shutdown_connections 2024-05-20T21:04:43.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:43.949+0000 7f91a3c25700 1 -- 172.21.15.26:0/1997901694 wait complete. 2024-05-20T21:04:44.436 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-20T21:04:44.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:04:44.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 192972 2.37 MB/sec execute 290 sec latency 18.339 ms 2024-05-20T21:04:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 -- 172.21.15.26:0/3663943311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 msgr2=0x7f46b8104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/3663943311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b8104130 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009900 tx=0x7f46ac02a410 comp rx=0 tx=0).stop 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 -- 172.21.15.26:0/3663943311 shutdown_connections 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/3663943311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8104670 0x7f46b810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/3663943311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b8104130 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/3663943311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46b8102ac0 0x7f46b8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 -- 172.21.15.26:0/3663943311 >> 172.21.15.26:0/3663943311 conn(0x7f46b80fe010 msgr2=0x7f46b8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 -- 172.21.15.26:0/3663943311 shutdown_connections 2024-05-20T21:04:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.665+0000 7f46bd6e1640 1 -- 172.21.15.26:0/3663943311 wait complete. 2024-05-20T21:04:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 Processor -- start 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 -- start start 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46b8102ac0 0x7f46b8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b81990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8104670 0x7f46b81a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810e4f0 con 0x7f46b8103cb0 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810e370 con 0x7f46b8102ac0 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46bd6e1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810e670 con 0x7f46b8104670 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46b67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b81990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46b67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b81990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52780/0 (socket says 172.21.15.26:52780) 2024-05-20T21:04:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.666+0000 7f46b67fc640 1 -- 172.21.15.26:0/2078010957 learned_addr learned my addr 172.21.15.26:0/2078010957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46b8102ac0 0x7f46b8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b77fe640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8104670 0x7f46b81a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b67fc640 1 -- 172.21.15.26:0/2078010957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8104670 msgr2=0x7f46b81a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b67fc640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8104670 0x7f46b81a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b67fc640 1 -- 172.21.15.26:0/2078010957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46b8102ac0 msgr2=0x7f46b8198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b67fc640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46b8102ac0 0x7f46b8198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b67fc640 1 -- 172.21.15.26:0/2078010957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b81a0780 con 0x7f46b8103cb0 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46b67fc640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b81990a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f46ac031460 tx=0x7f46ac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f469ffff640 1 -- 172.21.15.26:0/2078010957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac03f070 con 0x7f46b8103cb0 2024-05-20T21:04:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f469ffff640 1 -- 172.21.15.26:0/2078010957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ac031ba0 con 0x7f46b8103cb0 2024-05-20T21:04:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b81a0a10 con 0x7f46b8103cb0 2024-05-20T21:04:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f469ffff640 1 -- 172.21.15.26:0/2078010957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac035030 con 0x7f46b8103cb0 2024-05-20T21:04:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.667+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81a0f50 con 0x7f46b8103cb0 2024-05-20T21:04:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 Processor -- start 2024-05-20T21:04:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 -- start start 2024-05-20T21:04:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 0x55ef10ccfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10cd5220 0x55ef10cd76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ef10c40c30 con 0x55ef10ccf6c0 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ef10c307d0 con 0x55ef10cd5220 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46c03fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef10c34810 con 0x55ef10cd00b0 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 0x55ef10ccfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 0x55ef10ccfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52792/0 (socket says 172.21.15.26:52792) 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f46b77fe640 1 -- 172.21.15.26:0/3148361323 learned_addr learned my addr 172.21.15.26:0/3148361323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.670+0000 7f469ffff640 1 -- 172.21.15.26:0/2078010957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f46ac042070 con 0x7f46b8103cb0 2024-05-20T21:04:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b77fe640 1 -- 172.21.15.26:0/3148361323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 msgr2=0x55ef10cd4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b77fe640 1 -- 172.21.15.26:0/3148361323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10cd5220 msgr2=0x55ef10cd76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10cd5220 0x55ef10cd76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b77fe640 1 -- 172.21.15.26:0/3148361323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef10945210 con 0x55ef10ccf6c0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 0x55ef10ccfae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009fd0 tx=0x7f46b00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3803a41e115952 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f469d7fa640 1 -- 172.21.15.26:0/3148361323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0004d20 con 0x55ef10ccf6c0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f469d7fa640 1 -- 172.21.15.26:0/3148361323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0004050 con 0x55ef10ccf6c0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f469d7fa640 1 -- 172.21.15.26:0/3148361323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0005260 con 0x55ef10ccf6c0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.671+0000 7f469ffff640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46a00b2920 0x7f46a00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46a00b2920 0x7f46a00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f469ffff640 1 -- 172.21.15.26:0/2078010957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f46ac102910 con 0x7f46b8103cb0 2024-05-20T21:04:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46a00b2920 0x7f46a00b4de0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55ef10cd04b0 tx=0x7f46a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 -- 172.21.15.26:0/3148361323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 msgr2=0x55ef10ccfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 0x55ef10ccfae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009fd0 tx=0x7f46b00118d0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 -- 172.21.15.26:0/3148361323 shutdown_connections 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10cd5220 0x55ef10cd76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3148361323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10ccf6c0 0x55ef10ccfae0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 -- 172.21.15.26:0/3148361323 >> 172.21.15.26:0/3148361323 conn(0x55ef10ccad30 msgr2=0x55ef10ccb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 -- 172.21.15.26:0/3148361323 shutdown_connections 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 -- 172.21.15.26:0/3148361323 wait complete. 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.672+0000 7f46c03fe740 1 -- start start 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 Processor -- start 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10cd9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd5220 0x55ef10ce0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ef10916790 con 0x55ef10cd5220 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ef10bec890 con 0x55ef10ccf6c0 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef10cd81d0 con 0x55ef10cd00b0 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46c03fe740 10 client.0 _ll_register_callbacks cb 0x55ef10944a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10cd9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd5220 0x55ef10ce0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10cd9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54654/0 (socket says 172.21.15.26:54654) 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 -- 172.21.15.26:0/3880403091 learned_addr learned my addr 172.21.15.26:0/3880403091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b67fc640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 -- 172.21.15.26:0/3880403091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 msgr2=0x55ef10cd9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 -- 172.21.15.26:0/3880403091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd5220 msgr2=0x55ef10ce0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd5220 0x55ef10ce0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b6ffd640 1 -- 172.21.15.26:0/3880403091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef10a11620 con 0x55ef10ccf6c0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.673+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd5220 0x55ef10ce0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10cd9840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46a8013010 tx=0x7f46a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207d715c15457d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f469aff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f469aff5640 1 -- 172.21.15.26:0/3880403091 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a802e060 con 0x55ef10ccf6c0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f469aff5640 1 -- 172.21.15.26:0/3880403091 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f46a8018030 con 0x55ef10ccf6c0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b0004050 con 0x55ef10ccf6c0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f469aff5640 1 -- 172.21.15.26:0/3880403091 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46a8009db0 con 0x55ef10ccf6c0 2024-05-20T21:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 10 client.27401 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef10ce3e10 con 0x55ef10ccf6c0 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f469aff5640 1 -- 172.21.15.26:0/3880403091 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0004050 con 0x55ef10ccf6c0 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f469aff5640 1 -- 172.21.15.26:0/3880403091 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f46a8032030 con 0x55ef10ccf6c0 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 10 client.27401 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 10 client.27401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 20 client.27401 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.674+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10ce5550 0x55ef10ce7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.675+0000 7f46c03fe740 4 client.27401 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.675+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef10c42b80 con 0x55ef10ce5550 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.675+0000 7f46b67fc640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10ce5550 0x55ef10ce7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.675+0000 7f46b67fc640 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10ce5550 0x55ef10ce7940 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef845f95f9337a1 server_cookie=fd822e65e56371a8 in_seq=0 out_seq=0 2024-05-20T21:04:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.675+0000 7f469aff5640 10 client.27401 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f469aff5640 1 -- 172.21.15.26:0/3880403091 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ef10c42b80 con 0x55ef10ce5550 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f469aff5640 10 client.27401 handle_command_reply: tid=0 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 client.27401 shutdown 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 msgr2=0x55ef10cd9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10cd9840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46a8013010 tx=0x7f46a8004730 comp rx=0 tx=0).stop 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 shutdown_connections 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10ce5550 0x55ef10ce7940 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f469aff5640 0 client.27401 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd5220 0x55ef10ce0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3880403091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10cd9840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 >> 172.21.15.26:0/3880403091 conn(0x55ef10ccad30 msgr2=0x55ef10cc9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 shutdown_connections 2024-05-20T21:04:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.676+0000 7f46c03fe740 1 -- 172.21.15.26:0/3880403091 wait complete. 2024-05-20T21:04:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.677+0000 7f46c03fe740 20 client.27401 trim_cache size 0 max 16384 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 Processor -- start 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 -- start start 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccad30 0x55ef10cd9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10ce0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ac810 con 0x55ef10cd00b0 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81aa0e0 con 0x55ef10ccf6c0 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46c03fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81a7f30 con 0x55ef10ccad30 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52806/0 (socket says 172.21.15.26:52806) 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 -- 172.21.15.26:0/1716347438 learned_addr learned my addr 172.21.15.26:0/1716347438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b67fc640 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10ce0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccad30 0x55ef10cd9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 -- 172.21.15.26:0/1716347438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccad30 msgr2=0x55ef10cd9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccad30 0x55ef10cd9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 -- 172.21.15.26:0/1716347438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 msgr2=0x55ef10ce0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10ce0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.693+0000 7f46b77fe640 1 -- 172.21.15.26:0/1716347438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef10cd4600 con 0x55ef10cd00b0 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46b77fe640 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55ef10cd0910 tx=0x7f46b00045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a81a86fe2ad5366 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f469cff9640 1 -- 172.21.15.26:0/1716347438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0003a30 con 0x55ef10cd00b0 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f469cff9640 1 -- 172.21.15.26:0/1716347438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0003bd0 con 0x55ef10cd00b0 2024-05-20T21:04:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f469cff9640 1 -- 172.21.15.26:0/1716347438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0005540 con 0x55ef10cd00b0 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 -- 172.21.15.26:0/1716347438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 msgr2=0x55ef10cd9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55ef10cd0910 tx=0x7f46b00045c0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 -- 172.21.15.26:0/1716347438 shutdown_connections 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cd9990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccf6c0 0x55ef10ce0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 --2- 172.21.15.26:0/1716347438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccad30 0x55ef10cd9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.694+0000 7f46c03fe740 1 -- 172.21.15.26:0/1716347438 >> 172.21.15.26:0/1716347438 conn(0x55ef10cd4cf0 msgr2=0x55ef10ccb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 -- 172.21.15.26:0/1716347438 shutdown_connections 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 -- 172.21.15.26:0/1716347438 wait complete. 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 -- start start 2024-05-20T21:04:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 Processor -- start 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccad30 0x55ef10cde7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccf6c0 0x55ef10cded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cdfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81a85a0 con 0x55ef10cd00b0 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81a9a10 con 0x55ef10ccad30 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81a8c70 con 0x55ef10ccf6c0 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46c03fe740 10 client.0 _ll_register_callbacks cb 0x7f46a8009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cdfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccad30 0x55ef10cde7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccad30 0x55ef10cde7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54676/0 (socket says 172.21.15.26:54676) 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cdfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52822/0 (socket says 172.21.15.26:52822) 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46b6ffd640 1 -- 172.21.15.26:0/3338327724 learned_addr learned my addr 172.21.15.26:0/3338327724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.695+0000 7f46b67fc640 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccf6c0 0x55ef10cded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46b77fe640 1 -- 172.21.15.26:0/3338327724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccf6c0 msgr2=0x55ef10cded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccf6c0 0x55ef10cded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46b77fe640 1 -- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccad30 msgr2=0x55ef10cde7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccad30 0x55ef10cde7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46b77fe640 1 -- 172.21.15.26:0/3338327724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef10ce0330 con 0x55ef10cd00b0 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46b77fe640 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cdfd40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f46b0006010 tx=0x7f46b0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281f921040df1514 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f469aff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f469aff5640 1 -- 172.21.15.26:0/3338327724 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b003a060 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f469aff5640 1 -- 172.21.15.26:0/3338327724 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f46b00047f0 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b0003bd0 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f469aff5640 1 -- 172.21.15.26:0/3338327724 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46b0003a30 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46c03fe740 10 client.18003 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef10d503d0 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.696+0000 7f469aff5640 1 -- 172.21.15.26:0/3338327724 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0003bd0 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f469aff5640 1 -- 172.21.15.26:0/3338327724 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f46b003e030 con 0x55ef10cd00b0 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46c03fe740 10 client.18003 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46c03fe740 10 client.18003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46c03fe740 20 client.18003 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10cd5220 0x55ef10d52b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46c03fe740 4 client.18003 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:04:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef10c3c360 con 0x55ef10cd5220 2024-05-20T21:04:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10cd5220 0x55ef10d52b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f46b6ffd640 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10cd5220 0x55ef10d52b40 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd4f0d98acf4a73 server_cookie=58ef7c61c1e17c27 in_seq=0 out_seq=0 2024-05-20T21:04:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.697+0000 7f469aff5640 10 client.18003 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f469aff5640 1 -- 172.21.15.26:0/3338327724 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef10c3c360 con 0x55ef10cd5220 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f469aff5640 10 client.18003 handle_command_reply: tid=0 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 client.18003 shutdown 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 msgr2=0x55ef10cdfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cdfd40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f46b0006010 tx=0x7f46b0005b00 comp rx=0 tx=0).stop 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 shutdown_connections 2024-05-20T21:04:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef10cd5220 0x55ef10d52b40 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef10cd00b0 0x55ef10cdfd40 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef10ccf6c0 0x55ef10cded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 --2- 172.21.15.26:0/3338327724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef10ccad30 0x55ef10cde7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f469aff5640 0 client.18003 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 >> 172.21.15.26:0/3338327724 conn(0x55ef10cd4cf0 msgr2=0x55ef10cd9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 shutdown_connections 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 1 -- 172.21.15.26:0/3338327724 wait complete. 2024-05-20T21:04:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.745+0000 7f46c03fe740 20 client.18003 trim_cache size 0 max 16384 2024-05-20T21:04:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:04:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:04:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46a00b2920 msgr2=0x7f46a00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46a00b2920 0x7f46a00b4de0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55ef10cd04b0 tx=0x7f46a8005f90 comp rx=0 tx=0).stop 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 msgr2=0x7f46b81990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b81990a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f46ac031460 tx=0x7f46ac003c10 comp rx=0 tx=0).stop 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 shutdown_connections 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f46a00b2920 0x7f46a00b4de0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8104670 0x7f46b81a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b8103cb0 0x7f46b81990a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 --2- 172.21.15.26:0/2078010957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46b8102ac0 0x7f46b8198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.746+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 >> 172.21.15.26:0/2078010957 conn(0x7f46b80fe010 msgr2=0x7f46b8107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.747+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 shutdown_connections 2024-05-20T21:04:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:44.747+0000 7f46bd6e1640 1 -- 172.21.15.26:0/2078010957 wait complete. 2024-05-20T21:04:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:45.430 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:45 smithi026 ceph-mon[31990]: pgmap v648: 97 pgs: 97 active+clean; 27 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-20T21:04:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:45 smithi144 ceph-mon[35660]: pgmap v648: 97 pgs: 97 active+clean; 27 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-20T21:04:45.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 193603 2.36 MB/sec execute 291 sec latency 40.327 ms 2024-05-20T21:04:45.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:45 smithi055 ceph-mon[35213]: pgmap v648: 97 pgs: 97 active+clean; 27 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-20T21:04:45.932 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:04:45.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:04:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:46.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 194165 2.36 MB/sec execute 292 sec latency 18.217 ms 2024-05-20T21:04:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 -- 172.21.15.26:0/306831046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68108620 msgr2=0x7fad68108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/306831046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68108620 0x7fad68108a00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fad58009990 tx=0x7fad58009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 -- 172.21.15.26:0/306831046 shutdown_connections 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/306831046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68108620 0x7fad68108a00 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/306831046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad681028c0 0x7fad68110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/306831046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad68101f00 0x7fad68102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.912+0000 7fad6e2dc700 1 -- 172.21.15.26:0/306831046 >> 172.21.15.26:0/306831046 conn(0x7fad68077780 msgr2=0x7fad68077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.913+0000 7fad6e2dc700 1 -- 172.21.15.26:0/306831046 shutdown_connections 2024-05-20T21:04:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.913+0000 7fad6e2dc700 1 -- 172.21.15.26:0/306831046 wait complete. 2024-05-20T21:04:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.913+0000 7fad6e2dc700 1 Processor -- start 2024-05-20T21:04:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 -- start start 2024-05-20T21:04:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 0x7fad681030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad681028c0 0x7fad68103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad68108620 0x7fad68103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fad68113660 con 0x7fad68101f00 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fad681134e0 con 0x7fad68108620 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6e2dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad681137e0 con 0x7fad681028c0 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 0x7fad681030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 0x7fad681030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52834/0 (socket says 172.21.15.26:52834) 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 -- 172.21.15.26:0/1752810173 learned_addr learned my addr 172.21.15.26:0/1752810173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6c879700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad68108620 0x7fad68103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad5ffff700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad681028c0 0x7fad68103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 -- 172.21.15.26:0/1752810173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad681028c0 msgr2=0x7fad68103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad681028c0 0x7fad68103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 -- 172.21.15.26:0/1752810173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad68108620 msgr2=0x7fad68103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad68108620 0x7fad68103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad67fff700 1 -- 172.21.15.26:0/1752810173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad681a8160 con 0x7fad68101f00 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.914+0000 7fad6c879700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad68108620 0x7fad68103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.915+0000 7fad67fff700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 0x7fad681030e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fad580059d0 tx=0x7fad5800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.915+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58027070 con 0x7fad68101f00 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.915+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad681a83f0 con 0x7fad68101f00 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.915+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681a88a0 con 0x7fad68101f00 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.915+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad58022470 con 0x7fad68101f00 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.915+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58017650 con 0x7fad68101f00 2024-05-20T21:04:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.916+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad48005330 con 0x7fad68101f00 2024-05-20T21:04:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.918+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fad58009d10 con 0x7fad68101f00 2024-05-20T21:04:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.918+0000 7fad65ffb700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fad540b2900 0x7fad540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.919+0000 7fad5ffff700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fad540b2900 0x7fad540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.919+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fad580e2fd0 con 0x7fad68101f00 2024-05-20T21:04:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.919+0000 7fad5ffff700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fad540b2900 0x7fad540b4dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fad50006fd0 tx=0x7fad50008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:46.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:46.923+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad58013070 con 0x7fad68101f00 2024-05-20T21:04:47.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.246+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad48005620 con 0x7fad68101f00 2024-05-20T21:04:47.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.247+0000 7fad65ffb700 1 -- 172.21.15.26:0/1752810173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fad580aad20 con 0x7fad68101f00 2024-05-20T21:04:47.247 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:47.247 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:47.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fad540b2900 msgr2=0x7fad540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:47.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fad540b2900 0x7fad540b4dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fad50006fd0 tx=0x7fad50008040 comp rx=0 tx=0).stop 2024-05-20T21:04:47.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 msgr2=0x7fad681030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 0x7fad681030e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fad580059d0 tx=0x7fad5800bfc0 comp rx=0 tx=0).stop 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 shutdown_connections 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fad540b2900 0x7fad540b4dc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad68101f00 0x7fad681030e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad68108620 0x7fad68103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 --2- 172.21.15.26:0/1752810173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad681028c0 0x7fad68103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 >> 172.21.15.26:0/1752810173 conn(0x7fad68077780 msgr2=0x7fad68100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 shutdown_connections 2024-05-20T21:04:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:47.249+0000 7fad6e2dc700 1 -- 172.21.15.26:0/1752810173 wait complete. 2024-05-20T21:04:47.251 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:04:47.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:47 smithi026 ceph-mon[31990]: pgmap v649: 97 pgs: 97 active+clean; 28 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:04:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:47 smithi144 ceph-mon[35660]: pgmap v649: 97 pgs: 97 active+clean; 28 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:04:47.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 194882 2.37 MB/sec execute 293 sec latency 58.547 ms 2024-05-20T21:04:47.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:47 smithi055 ceph-mon[35213]: pgmap v649: 97 pgs: 97 active+clean; 28 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:04:48.222 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-20T21:04:48.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-20T21:04:48.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1752810173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1752810173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:48.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 195479 2.38 MB/sec execute 294 sec latency 15.968 ms 2024-05-20T21:04:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1752810173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 -- 172.21.15.26:0/2043217840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 msgr2=0x7f818c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 --2- 172.21.15.26:0/2043217840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c1028c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81780098f0 tx=0x7f8178009c00 comp rx=0 tx=0).stop 2024-05-20T21:04:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 -- 172.21.15.26:0/2043217840 shutdown_connections 2024-05-20T21:04:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 --2- 172.21.15.26:0/2043217840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c102e00 0x7f818c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 --2- 172.21.15.26:0/2043217840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c108520 0x7f818c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 --2- 172.21.15.26:0/2043217840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c1028c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 -- 172.21.15.26:0/2043217840 >> 172.21.15.26:0/2043217840 conn(0x7f818c0793f0 msgr2=0x7f818c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 -- 172.21.15.26:0/2043217840 shutdown_connections 2024-05-20T21:04:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.401+0000 7f81930d3700 1 -- 172.21.15.26:0/2043217840 wait complete. 2024-05-20T21:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 Processor -- start 2024-05-20T21:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 -- start start 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c102e00 0x7f818c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c108520 0x7f818c1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1124d0 con 0x7f818c108520 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f818c112350 con 0x7f818c102440 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f81930d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818c112650 con 0x7f818c102e00 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f8190e6f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f8190e6f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54722/0 (socket says 172.21.15.26:54722) 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f8190e6f700 1 -- 172.21.15.26:0/4238617362 learned_addr learned my addr 172.21.15.26:0/4238617362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f818bfff700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c102e00 0x7f818c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.402+0000 7f8191670700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c108520 0x7f818c1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8190e6f700 1 -- 172.21.15.26:0/4238617362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c102e00 msgr2=0x7f818c19afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8190e6f700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c102e00 0x7f818c19afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8190e6f700 1 -- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c108520 msgr2=0x7f818c1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8190e6f700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c108520 0x7f818c1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8190e6f700 1 -- 172.21.15.26:0/4238617362 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c1a6060 con 0x7f818c102440 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8191670700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c108520 0x7f818c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8190e6f700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c19aaa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9a0 tx=0x7f818000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8189ffb700 1 -- 172.21.15.26:0/4238617362 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000cbd0 con 0x7f818c102440 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1a62f0 con 0x7f818c102440 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8189ffb700 1 -- 172.21.15.26:0/4238617362 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f818000cd70 con 0x7f818c102440 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f8189ffb700 1 -- 172.21.15.26:0/4238617362 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180018680 con 0x7f818c102440 2024-05-20T21:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.403+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1a6850 con 0x7f818c102440 2024-05-20T21:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.406+0000 7f8189ffb700 1 -- 172.21.15.26:0/4238617362 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8180018820 con 0x7f818c102440 2024-05-20T21:04:49.406 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:49 smithi026 ceph-mon[31990]: pgmap v650: 97 pgs: 97 active+clean; 25 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-20T21:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.408+0000 7f8189ffb700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f817c0b2900 0x7f817c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.408+0000 7f818bfff700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f817c0b2900 0x7f817c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.408+0000 7f8189ffb700 1 -- 172.21.15.26:0/4238617362 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f81800e2400 con 0x7f818c102440 2024-05-20T21:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.408+0000 7f819d969b80 1 Processor -- start 2024-05-20T21:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.408+0000 7f819d969b80 1 -- start start 2024-05-20T21:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f818bfff700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f817c0b2900 0x7f817c0b4dc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f81780098c0 tx=0x7f8178019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f819d969b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff98fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f819d969b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ff8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f819d969b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ff9f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f819d969b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55836fee8df0 con 0x55836ff8b2e0 2024-05-20T21:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f819d969b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55836fed8990 con 0x55836ff98bd0 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f819d969b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55836fedc9d0 con 0x55836ff8a920 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff98fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff98fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54732/0 (socket says 172.21.15.26:54732) 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 -- 172.21.15.26:0/3139542947 learned_addr learned my addr 172.21.15.26:0/3139542947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f818bfff700 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ff8ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8191670700 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ff9f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 -- 172.21.15.26:0/3139542947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 msgr2=0x55836ff8ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ff8ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 -- 172.21.15.26:0/3139542947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 msgr2=0x55836ff9f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ff9f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8190e6f700 1 -- 172.21.15.26:0/3139542947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836fe82030 con 0x55836ff98bd0 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.409+0000 7f8191670700 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ff9f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.410+0000 7f8190e6f700 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff98fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81800a5a40 tx=0x7f81800a9130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3db5e75befe35e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.410+0000 7f8176ffd700 1 -- 172.21.15.26:0/3139542947 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800b2170 con 0x55836ff98bd0 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.410+0000 7f8176ffd700 1 -- 172.21.15.26:0/3139542947 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81800a94d0 con 0x55836ff98bd0 2024-05-20T21:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.410+0000 7f8176ffd700 1 -- 172.21.15.26:0/3139542947 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800cc2f0 con 0x55836ff98bd0 2024-05-20T21:04:49.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 -- 172.21.15.26:0/3139542947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 msgr2=0x55836ff98fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff98fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81800a5a40 tx=0x7f81800a9130 comp rx=0 tx=0).stop 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 -- 172.21.15.26:0/3139542947 shutdown_connections 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ff9f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff98fb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 --2- 172.21.15.26:0/3139542947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ff8ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 -- 172.21.15.26:0/3139542947 >> 172.21.15.26:0/3139542947 conn(0x55836feb62c0 msgr2=0x55836ff85460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 -- 172.21.15.26:0/3139542947 shutdown_connections 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 -- 172.21.15.26:0/3139542947 wait complete. 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.411+0000 7f819d969b80 1 -- start start 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 Processor -- start 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff8a920 0x55836ff9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff9c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff98bd0 0x55836ff9cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55836fddf020 con 0x55836ff98bd0 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55836fe40e20 con 0x55836ff8a920 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55836ff9feb0 con 0x55836ff8b2e0 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f819d969b80 10 client.0 _ll_register_callbacks cb 0x55836fe739d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.412+0000 7f8191670700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff98bd0 0x55836ff9cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8191670700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff98bd0 0x55836ff9cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52870/0 (socket says 172.21.15.26:52870) 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8191670700 1 -- 172.21.15.26:0/1458139891 learned_addr learned my addr 172.21.15.26:0/1458139891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8190e6f700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff8a920 0x55836ff9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f818bfff700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff9c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8190e6f700 1 -- 172.21.15.26:0/1458139891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 msgr2=0x55836ff9c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8190e6f700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff9c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8190e6f700 1 -- 172.21.15.26:0/1458139891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff98bd0 msgr2=0x55836ff9cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8190e6f700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff98bd0 0x55836ff9cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.413+0000 7f8190e6f700 1 -- 172.21.15.26:0/1458139891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836fe28950 con 0x55836ff8a920 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f8190e6f700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff8a920 0x55836ff9f6d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55836ff875a0 tx=0x7f81800a9ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c931e1c0aef670e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f816bfff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f816bfff700 1 -- 172.21.15.26:0/1458139891 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800c4070 con 0x55836ff8a920 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81800a94d0 con 0x55836ff8a920 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f816bfff700 1 -- 172.21.15.26:0/1458139891 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f81800b2390 con 0x55836ff8a920 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f816bfff700 1 -- 172.21.15.26:0/1458139891 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8180087020 con 0x55836ff8a920 2024-05-20T21:04:49.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f819d969b80 10 client.27425 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.414+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55836fe927e0 con 0x55836ff8a920 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f816bfff700 1 -- 172.21.15.26:0/1458139891 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81800a94d0 con 0x55836ff8a920 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f816bfff700 1 -- 172.21.15.26:0/1458139891 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f81800a5330 con 0x55836ff8a920 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f819d969b80 10 client.27425 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f819d969b80 10 client.27425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f819d969b80 20 client.27425 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f819d969b80 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55836ffacc10 0x55836ffaf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f819d969b80 4 client.27425 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55836feead40 con 0x55836ffacc10 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.415+0000 7f818bfff700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55836ffacc10 0x55836ffaf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.416+0000 7f816bfff700 10 client.27425 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.416+0000 7f818bfff700 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55836ffacc10 0x55836ffaf000 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2c37ba7b4ef61f server_cookie=3942dc8a442f5d92 in_seq=0 out_seq=0 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f816bfff700 1 -- 172.21.15.26:0/1458139891 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55836feead40 con 0x55836ffacc10 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f816bfff700 10 client.27425 handle_command_reply: tid=0 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 client.27425 shutdown 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff8a920 msgr2=0x55836ff9f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff8a920 0x55836ff9f6d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55836ff875a0 tx=0x7f81800a9ed0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 shutdown_connections 2024-05-20T21:04:49.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55836ffacc10 0x55836ffaf000 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff98bd0 0x55836ff9cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f816bfff700 0 client.27425 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff8a920 0x55836ff9f6d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 --2- 172.21.15.26:0/1458139891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff9c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.417+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 >> 172.21.15.26:0/1458139891 conn(0x55836feb62c0 msgr2=0x55836ff85280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.418+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 shutdown_connections 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.418+0000 7f819d969b80 1 -- 172.21.15.26:0/1458139891 wait complete. 2024-05-20T21:04:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.418+0000 7f819d969b80 20 client.27425 trim_cache size 0 max 16384 2024-05-20T21:04:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 Processor -- start 2024-05-20T21:04:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 -- start start 2024-05-20T21:04:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8a920 0x55836ff852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff99fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1ae3e0 con 0x55836ff8a920 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1abc70 con 0x55836ff98bd0 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f819d969b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a9ac0 con 0x55836ff8b2e0 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f8191670700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f8191670700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff9a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54748/0 (socket says 172.21.15.26:54748) 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f8191670700 1 -- 172.21.15.26:0/89414473 learned_addr learned my addr 172.21.15.26:0/89414473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f8190e6f700 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8a920 0x55836ff852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.443+0000 7f818bfff700 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff99fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f8191670700 1 -- 172.21.15.26:0/89414473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 msgr2=0x55836ff99fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f8191670700 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff99fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f8191670700 1 -- 172.21.15.26:0/89414473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8a920 msgr2=0x55836ff852f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f8191670700 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8a920 0x55836ff852f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f8191670700 1 -- 172.21.15.26:0/89414473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836ff87130 con 0x55836ff98bd0 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f8191670700 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff9a510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f81840076f0 tx=0x7f81840099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc4d142c7fa8800 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f81767fc700 1 -- 172.21.15.26:0/89414473 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184003a00 con 0x55836ff98bd0 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f81767fc700 1 -- 172.21.15.26:0/89414473 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8184003ba0 con 0x55836ff98bd0 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.444+0000 7f81767fc700 1 -- 172.21.15.26:0/89414473 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184017700 con 0x55836ff98bd0 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 -- 172.21.15.26:0/89414473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 msgr2=0x55836ff9a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff9a510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f81840076f0 tx=0x7f81840099e0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 -- 172.21.15.26:0/89414473 shutdown_connections 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8a920 0x55836ff852f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55836ff9a510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 --2- 172.21.15.26:0/89414473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8b2e0 0x55836ff99fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 -- 172.21.15.26:0/89414473 >> 172.21.15.26:0/89414473 conn(0x55836feb62c0 msgr2=0x55836ff998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 -- 172.21.15.26:0/89414473 shutdown_connections 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.445+0000 7f819d969b80 1 -- 172.21.15.26:0/89414473 wait complete. 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 -- start start 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 Processor -- start 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ffa19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ffa1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55837000f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1aa130 con 0x55836ff8b2e0 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1ab5a0 con 0x55836ff98bd0 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1aa800 con 0x55836ff8a920 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f819d969b80 10 client.0 _ll_register_callbacks cb 0x55836fe739d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f8191670700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55837000f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f818bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ffa1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f8191670700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55837000f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54760/0 (socket says 172.21.15.26:54760) 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.446+0000 7f8191670700 1 -- 172.21.15.26:0/266311344 learned_addr learned my addr 172.21.15.26:0/266311344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f8190e6f700 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ffa19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f818bfff700 1 -- 172.21.15.26:0/266311344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 msgr2=0x55836ffa19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f818bfff700 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ffa19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f818bfff700 1 -- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 msgr2=0x55837000f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f818bfff700 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55837000f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f818bfff700 1 -- 172.21.15.26:0/266311344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836ff9fa40 con 0x55836ff8b2e0 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f818bfff700 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ffa1ee0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8178009fd0 tx=0x7f8178009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a190fd66e597660b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f816bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f816bfff700 1 -- 172.21.15.26:0/266311344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81780054c0 con 0x55836ff8b2e0 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f816bfff700 1 -- 172.21.15.26:0/266311344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8178005660 con 0x55836ff8b2e0 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.447+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8184003ba0 con 0x55836ff8b2e0 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f816bfff700 1 -- 172.21.15.26:0/266311344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8178041050 con 0x55836ff8b2e0 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 10 client.18027 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558370018140 con 0x55836ff8b2e0 2024-05-20T21:04:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f816bfff700 1 -- 172.21.15.26:0/266311344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8184003ba0 con 0x55836ff8b2e0 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f816bfff700 1 -- 172.21.15.26:0/266311344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f817803a060 con 0x55836ff8b2e0 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 10 client.18027 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 20 client.18027 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558370018840 0x55837001ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 4 client.18027 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55836fee4520 con 0x558370018840 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.448+0000 7f8190e6f700 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558370018840 0x55837001ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.449+0000 7f816bfff700 10 client.18027 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.449+0000 7f8190e6f700 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558370018840 0x55837001ac30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73808193a575addb server_cookie=d8d77c8bf3091362 in_seq=0 out_seq=0 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f816bfff700 1 -- 172.21.15.26:0/266311344 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55836fee4520 con 0x558370018840 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f816bfff700 10 client.18027 handle_command_reply: tid=0 2024-05-20T21:04:49.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 client.18027 shutdown 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 msgr2=0x55836ffa1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ffa1ee0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8178009fd0 tx=0x7f8178009c00 comp rx=0 tx=0).stop 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 shutdown_connections 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558370018840 0x55837001ac30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55836ff98bd0 0x55837000f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55836ff8a920 0x55836ffa19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f816bfff700 0 client.18027 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 --2- 172.21.15.26:0/266311344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55836ff8b2e0 0x55836ffa1ee0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 >> 172.21.15.26:0/266311344 conn(0x55836feb62c0 msgr2=0x55836ff9a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 shutdown_connections 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.450+0000 7f819d969b80 1 -- 172.21.15.26:0/266311344 wait complete. 2024-05-20T21:04:49.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f819d969b80 20 client.18027 trim_cache size 0 max 16384 2024-05-20T21:04:49.454 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:04:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "275e5845-8850-4705-a61b-42ff627f5510", 2024-05-20T21:04:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:04:49.454 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f817c0b2900 msgr2=0x7f817c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f817c0b2900 0x7f817c0b4dc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f81780098c0 tx=0x7f8178019040 comp rx=0 tx=0).stop 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 msgr2=0x7f818c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c19aaa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9a0 tx=0x7f818000ecb0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 shutdown_connections 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f817c0b2900 0x7f817c0b4dc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f818c108520 0x7f818c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f818c102440 0x7f818c19aaa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 --2- 172.21.15.26:0/4238617362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c102e00 0x7f818c19afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:49.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.451+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 >> 172.21.15.26:0/4238617362 conn(0x7f818c0793f0 msgr2=0x7f818c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:49.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.452+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 shutdown_connections 2024-05-20T21:04:49.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:49.452+0000 7f81930d3700 1 -- 172.21.15.26:0/4238617362 wait complete. 2024-05-20T21:04:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:49 smithi144 ceph-mon[35660]: pgmap v650: 97 pgs: 97 active+clean; 25 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-20T21:04:49.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 195987 2.38 MB/sec execute 295 sec latency 20.595 ms 2024-05-20T21:04:49.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:49 smithi055 ceph-mon[35213]: pgmap v650: 97 pgs: 97 active+clean; 25 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-20T21:04:50.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:50 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:04:50.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:50 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:50.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:50 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:04:50.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:50 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:50 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:04:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:50 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:50 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:04:50.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:50 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:50.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 196689 2.38 MB/sec execute 296 sec latency 14.713 ms 2024-05-20T21:04:50.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:50 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:04:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:50 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:50 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:04:50.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:50 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:04:50.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:04:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:51 smithi144 ceph-mon[35660]: pgmap v651: 97 pgs: 97 active+clean; 25 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-05-20T21:04:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:51 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:04:51.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197404 2.38 MB/sec execute 297 sec latency 17.813 ms 2024-05-20T21:04:51.659 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:51 smithi026 ceph-mon[31990]: pgmap v651: 97 pgs: 97 active+clean; 25 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-05-20T21:04:51.660 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:51 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:04:51.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:51 smithi055 ceph-mon[35213]: pgmap v651: 97 pgs: 97 active+clean; 25 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-05-20T21:04:51.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:51 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:04:52.048 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 -- 172.21.15.26:0/2526230562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c06b480 msgr2=0x7f376c06b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 --2- 172.21.15.26:0/2526230562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c06b480 0x7f376c06b860 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f37680099c0 tx=0x7f3768009cd0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 -- 172.21.15.26:0/2526230562 shutdown_connections 2024-05-20T21:04:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 --2- 172.21.15.26:0/2526230562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c06b480 0x7f376c06b860 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 --2- 172.21.15.26:0/2526230562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f376c108900 0x7f376c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 --2- 172.21.15.26:0/2526230562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06be30 0x7f376c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 -- 172.21.15.26:0/2526230562 >> 172.21.15.26:0/2526230562 conn(0x7f376c0793e0 msgr2=0x7f376c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 -- 172.21.15.26:0/2526230562 shutdown_connections 2024-05-20T21:04:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.078+0000 7f377461e700 1 -- 172.21.15.26:0/2526230562 wait complete. 2024-05-20T21:04:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 Processor -- start 2024-05-20T21:04:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 -- start start 2024-05-20T21:04:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06b480 0x7f376c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f376c06be30 0x7f376c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c108900 0x7f376c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1135e0 con 0x7f376c108900 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f376c113460 con 0x7f376c06be30 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f377461e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f376c113760 con 0x7f376c06b480 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f37723ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06b480 0x7f376c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f37723ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06b480 0x7f376c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:50408/0 (socket says 172.21.15.26:50408) 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f37723ba700 1 -- 172.21.15.26:0/3065084782 learned_addr learned my addr 172.21.15.26:0/3065084782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f3772bbb700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c108900 0x7f376c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.079+0000 7f3771bb9700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f376c06be30 0x7f376c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f3772bbb700 1 -- 172.21.15.26:0/3065084782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06b480 msgr2=0x7f376c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f3772bbb700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06b480 0x7f376c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f3772bbb700 1 -- 172.21.15.26:0/3065084782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f376c06be30 msgr2=0x7f376c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f3772bbb700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f376c06be30 0x7f376c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f3772bbb700 1 -- 172.21.15.26:0/3065084782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c199fc0 con 0x7f376c108900 2024-05-20T21:04:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f3772bbb700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c108900 0x7f376c199860 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f376400efc0 tx=0x7f376400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f37637fe700 1 -- 172.21.15.26:0/3065084782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640092e0 con 0x7f376c108900 2024-05-20T21:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c19a250 con 0x7f376c108900 2024-05-20T21:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f37637fe700 1 -- 172.21.15.26:0/3065084782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764004bb0 con 0x7f376c108900 2024-05-20T21:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f37637fe700 1 -- 172.21.15.26:0/3065084782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764004020 con 0x7f376c108900 2024-05-20T21:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.080+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c102440 con 0x7f376c108900 2024-05-20T21:04:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.083+0000 7f37637fe700 1 -- 172.21.15.26:0/3065084782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f376401b030 con 0x7f376c108900 2024-05-20T21:04:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.084+0000 7f37637fe700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37580b2930 0x7f37580b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.084+0000 7f37637fe700 1 -- 172.21.15.26:0/3065084782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f37640e1c90 con 0x7f376c108900 2024-05-20T21:04:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.084+0000 7f37723ba700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37580b2930 0x7f37580b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 Processor -- start 2024-05-20T21:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 -- start start 2024-05-20T21:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f37723ba700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37580b2930 0x7f37580b4df0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3768003760 tx=0x7f3768005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cd5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74ccfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd0330 0x563a74cdc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a74c2ee20 con 0x563a74cd5340 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.085+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563a74c1e9c0 con 0x563a74ccf970 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a74c22a00 con 0x563a74cd0330 2024-05-20T21:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f37723ba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cd5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74ccfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f37723ba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cd5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38388/0 (socket says 172.21.15.26:38388) 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f37723ba700 1 -- 172.21.15.26:0/3425273971 learned_addr learned my addr 172.21.15.26:0/3425273971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3772bbb700 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd0330 0x563a74cdc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 -- 172.21.15.26:0/3425273971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd0330 msgr2=0x563a74cdc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd0330 0x563a74cdc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 -- 172.21.15.26:0/3425273971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 msgr2=0x563a74cd5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cd5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 -- 172.21.15.26:0/3425273971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a74b6ea60 con 0x563a74ccf970 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f37723ba700 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cd5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.086+0000 7f3771bb9700 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74ccfdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f375c005950 tx=0x7f375c00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b013fe4d220a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.087+0000 7f3760ff9700 1 -- 172.21.15.26:0/3425273971 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c010c70 con 0x563a74ccf970 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.087+0000 7f3760ff9700 1 -- 172.21.15.26:0/3425273971 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f375c010e10 con 0x563a74ccf970 2024-05-20T21:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.087+0000 7f3760ff9700 1 -- 172.21.15.26:0/3425273971 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c005020 con 0x563a74ccf970 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 -- 172.21.15.26:0/3425273971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 msgr2=0x563a74ccfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74ccfdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f375c005950 tx=0x7f375c00bf60 comp rx=0 tx=0).stop 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 -- 172.21.15.26:0/3425273971 shutdown_connections 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cd5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd0330 0x563a74cdc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/3425273971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74ccfdf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 -- 172.21.15.26:0/3425273971 >> 172.21.15.26:0/3425273971 conn(0x563a74bfd210 msgr2=0x563a74cca280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 -- 172.21.15.26:0/3425273971 shutdown_connections 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 -- 172.21.15.26:0/3425273971 wait complete. 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 -- start start 2024-05-20T21:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.088+0000 7f377eeb4b80 1 Processor -- start 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cd90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd5340 0x563a74cd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a74b25130 con 0x563a74cd0330 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563a74bc0950 con 0x563a74ccf970 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a74cdc710 con 0x563a74cd5340 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f377eeb4b80 10 client.0 _ll_register_callbacks cb 0x563a74bb9ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cd90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f37723ba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cd90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38400/0 (socket says 172.21.15.26:38400) 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f3771bb9700 1 -- 172.21.15.26:0/2425874980 learned_addr learned my addr 172.21.15.26:0/2425874980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f37723ba700 1 -- 172.21.15.26:0/2425874980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd5340 msgr2=0x563a74cd9620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f37723ba700 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd5340 0x563a74cd9620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f37723ba700 1 -- 172.21.15.26:0/2425874980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 msgr2=0x563a74cd90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f37723ba700 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cd90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.089+0000 7f37723ba700 1 -- 172.21.15.26:0/2425874980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a74bbd820 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f37723ba700 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdbde0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3768005930 tx=0x7f376801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8831d7eee014e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f3755ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f3755ffb700 1 -- 172.21.15.26:0/2425874980 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768024410 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f3755ffb700 1 -- 172.21.15.26:0/2425874980 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f37680245b0 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c010e10 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f3755ffb700 1 -- 172.21.15.26:0/2425874980 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37680247b0 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f377eeb4b80 10 client.27443 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a74ce6e80 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f3755ffb700 1 -- 172.21.15.26:0/2425874980 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3768035070 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.090+0000 7f3755ffb700 1 -- 172.21.15.26:0/2425874980 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f3768030850 con 0x563a74ccf970 2024-05-20T21:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f377eeb4b80 10 client.27443 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f377eeb4b80 10 client.27443 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f377eeb4b80 20 client.27443 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74ce94b0 0x563a74ceb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f377eeb4b80 4 client.27443 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f375c010e10 con 0x563a74ce94b0 2024-05-20T21:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f3771bb9700 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74ce94b0 0x563a74ceb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f3755ffb700 10 client.27443 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.091+0000 7f3771bb9700 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74ce94b0 0x563a74ceb8a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c19b2bca617348 server_cookie=25d9c93b62ee062d in_seq=0 out_seq=0 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f3755ffb700 1 -- 172.21.15.26:0/2425874980 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f375c010e10 con 0x563a74ce94b0 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f3755ffb700 10 client.27443 handle_command_reply: tid=0 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f377eeb4b80 1 client.27443 shutdown 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 msgr2=0x563a74cdbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdbde0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3768005930 tx=0x7f376801a050 comp rx=0 tx=0).stop 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 shutdown_connections 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.092+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74ce94b0 0x563a74ceb8a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f3755ffb700 0 client.27443 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74cd5340 0x563a74cd9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdbde0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/2425874980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cd90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 >> 172.21.15.26:0/2425874980 conn(0x563a74bfd210 msgr2=0x563a74cc4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 shutdown_connections 2024-05-20T21:04:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 1 -- 172.21.15.26:0/2425874980 wait complete. 2024-05-20T21:04:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.093+0000 7f377eeb4b80 20 client.27443 trim_cache size 0 max 16384 2024-05-20T21:04:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 Processor -- start 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 -- start start 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ccf970 0x563a74cd9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cdb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 0x563a74cd1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f375000fdf0 con 0x563a74cd0330 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f375000d680 con 0x563a74cd5340 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f375000b4d0 con 0x563a74ccf970 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cdb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f3772bbb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 0x563a74cd1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cdb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38416/0 (socket says 172.21.15.26:38416) 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f3772bbb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 0x563a74cd1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:45730/0 (socket says 172.21.15.26:45730) 2024-05-20T21:04:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f3771bb9700 1 -- 172.21.15.26:0/765704927 learned_addr learned my addr 172.21.15.26:0/765704927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.121+0000 7f37723ba700 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ccf970 0x563a74cd9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3771bb9700 1 -- 172.21.15.26:0/765704927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ccf970 msgr2=0x563a74cd9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3771bb9700 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ccf970 0x563a74cd9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3771bb9700 1 -- 172.21.15.26:0/765704927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 msgr2=0x563a74cd1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3771bb9700 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 0x563a74cd1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3771bb9700 1 -- 172.21.15.26:0/765704927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a74cd7670 con 0x563a74cd0330 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3772bbb700 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 0x563a74cd1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3771bb9700 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cdb020 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f375c00bfd0 tx=0x7f375c005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c03760b3e6b63a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3757fff700 1 -- 172.21.15.26:0/765704927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c02e070 con 0x563a74cd0330 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3757fff700 1 -- 172.21.15.26:0/765704927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f375c0298c0 con 0x563a74cd0330 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.122+0000 7f3757fff700 1 -- 172.21.15.26:0/765704927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c027930 con 0x563a74cd0330 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 -- 172.21.15.26:0/765704927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 msgr2=0x563a74cdb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cdb020 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f375c00bfd0 tx=0x7f375c005d00 comp rx=0 tx=0).stop 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 -- 172.21.15.26:0/765704927 shutdown_connections 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74cd5340 0x563a74cd1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ccf970 0x563a74cd9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/765704927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd0330 0x563a74cdb020 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 -- 172.21.15.26:0/765704927 >> 172.21.15.26:0/765704927 conn(0x563a74bfd210 msgr2=0x563a74cda910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 -- 172.21.15.26:0/765704927 shutdown_connections 2024-05-20T21:04:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 -- 172.21.15.26:0/765704927 wait complete. 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.123+0000 7f377eeb4b80 1 -- start start 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 Processor -- start 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cdcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ce94b0 0x563a74cdd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f375000bb40 con 0x563a74cd5340 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f375000cfb0 con 0x563a74ccf970 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f375000c210 con 0x563a74ce94b0 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f377eeb4b80 10 client.0 _ll_register_callbacks cb 0x563a74bb9ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cdcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f37723ba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f3771bb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cdcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38430/0 (socket says 172.21.15.26:38430) 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f3771bb9700 1 -- 172.21.15.26:0/679794701 learned_addr learned my addr 172.21.15.26:0/679794701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f3772bbb700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ce94b0 0x563a74cdd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f37723ba700 1 -- 172.21.15.26:0/679794701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ce94b0 msgr2=0x563a74cdd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f37723ba700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ce94b0 0x563a74cdd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f37723ba700 1 -- 172.21.15.26:0/679794701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 msgr2=0x563a74cdcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f37723ba700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cdcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.124+0000 7f37723ba700 1 -- 172.21.15.26:0/679794701 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a74d54060 con 0x563a74ccf970 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3771bb9700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cdcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3772bbb700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ce94b0 0x563a74cdd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f37723ba700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdc9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3768005f50 tx=0x7f3768024740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6b6b258ce2d376 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3755ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3755ffb700 1 -- 172.21.15.26:0/679794701 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768003e90 con 0x563a74ccf970 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c0298c0 con 0x563a74ccf970 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3755ffb700 1 -- 172.21.15.26:0/679794701 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3768030610 con 0x563a74ccf970 2024-05-20T21:04:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3755ffb700 1 -- 172.21.15.26:0/679794701 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3768024e30 con 0x563a74ccf970 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f377eeb4b80 10 client.27449 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a74d552a0 con 0x563a74ccf970 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3755ffb700 1 -- 172.21.15.26:0/679794701 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f375c0298c0 con 0x563a74ccf970 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.125+0000 7f3755ffb700 1 -- 172.21.15.26:0/679794701 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f376804b020 con 0x563a74ccf970 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f377eeb4b80 10 client.27449 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f377eeb4b80 10 client.27449 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f377eeb4b80 20 client.27449 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74d55990 0x563a74d57db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f377eeb4b80 4 client.27449 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x563a74c9bb70 con 0x563a74d55990 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f3771bb9700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74d55990 0x563a74d57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f3771bb9700 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74d55990 0x563a74d57db0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7144cf85f4174b2a server_cookie=b0187afd6e060799 in_seq=0 out_seq=0 2024-05-20T21:04:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.126+0000 7f3755ffb700 10 client.27449 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.298+0000 7f3755ffb700 1 -- 172.21.15.26:0/679794701 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563a74c9bb70 con 0x563a74d55990 2024-05-20T21:04:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.298+0000 7f3755ffb700 10 client.27449 handle_command_reply: tid=0 2024-05-20T21:04:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.298+0000 7f377eeb4b80 1 client.27449 shutdown 2024-05-20T21:04:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 msgr2=0x563a74cdc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdc9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3768005f50 tx=0x7f3768024740 comp rx=0 tx=0).stop 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 shutdown_connections 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a74d55990 0x563a74d57db0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a74ce94b0 0x563a74cdd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a74ccf970 0x563a74cdc9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 --2- 172.21.15.26:0/679794701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a74cd5340 0x563a74cdcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 >> 172.21.15.26:0/679794701 conn(0x563a74bfd210 msgr2=0x563a74cc4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f3755ffb700 0 client.27449 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 shutdown_connections 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 1 -- 172.21.15.26:0/679794701 wait complete. 2024-05-20T21:04:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.299+0000 7f377eeb4b80 20 client.27449 trim_cache size 0 max 16384 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 34, 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "1e6d7737": { 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 34, 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:04:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.0, 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:04:52.303 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:52.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37580b2930 msgr2=0x7f37580b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37580b2930 0x7f37580b4df0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3768003760 tx=0x7f3768005fd0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c108900 msgr2=0x7f376c199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:52.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c108900 0x7f376c199860 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f376400efc0 tx=0x7f376400c3a0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 shutdown_connections 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37580b2930 0x7f37580b4df0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c108900 0x7f376c199860 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f376c06b480 0x7f376c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 --2- 172.21.15.26:0/3065084782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f376c06be30 0x7f376c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 >> 172.21.15.26:0/3065084782 conn(0x7f376c0793e0 msgr2=0x7f376c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 shutdown_connections 2024-05-20T21:04:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:52.300+0000 7f377461e700 1 -- 172.21.15.26:0/3065084782 wait complete. 2024-05-20T21:04:52.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.38 MB/sec execute 298 sec latency 399.590 ms 2024-05-20T21:04:53.186 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1e6d7737', quiesce took 8.7 seconds. Will release after: 1.3000000000000007 2024-05-20T21:04:53.403 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:53 smithi026 ceph-mon[31990]: pgmap v652: 97 pgs: 97 active+clean; 26 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:04:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:53 smithi144 ceph-mon[35660]: pgmap v652: 97 pgs: 97 active+clean; 26 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:04:53.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.37 MB/sec execute 299 sec latency 1399.985 ms 2024-05-20T21:04:53.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:53 smithi055 ceph-mon[35213]: pgmap v652: 97 pgs: 97 active+clean; 26 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:04:54.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:04:54.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.36 MB/sec execute 300 sec latency 2400.038 ms 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 -- 172.21.15.26:0/307243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806c5b0 msgr2=0x7fdb0810fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 --2- 172.21.15.26:0/307243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806c5b0 0x7fdb0810fae0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009990 tx=0x7fdafc009ca0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 -- 172.21.15.26:0/307243 shutdown_connections 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 --2- 172.21.15.26:0/307243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb08110020 0x7fdb08112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 --2- 172.21.15.26:0/307243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0806bc90 0x7fdb0806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 --2- 172.21.15.26:0/307243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806c5b0 0x7fdb0810fae0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 -- 172.21.15.26:0/307243 >> 172.21.15.26:0/307243 conn(0x7fdb08079410 msgr2=0x7fdb08079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 -- 172.21.15.26:0/307243 shutdown_connections 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 -- 172.21.15.26:0/307243 wait complete. 2024-05-20T21:04:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.731+0000 7fdb0db32700 1 Processor -- start 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 -- start start 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0806bc90 0x7fdb0819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0806c5b0 0x7fdb0819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 0x7fdb081a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08115870 con 0x7fdb08110020 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081156f0 con 0x7fdb0806bc90 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb0db32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081159f0 con 0x7fdb0806c5b0 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 0x7fdb081a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.732+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 0x7fdb081a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38448/0 (socket says 172.21.15.26:38448) 2024-05-20T21:04:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 -- 172.21.15.26:0/3533811627 learned_addr learned my addr 172.21.15.26:0/3533811627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb077fe700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0806bc90 0x7fdb0819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0806c5b0 0x7fdb0819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 -- 172.21.15.26:0/3533811627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0806c5b0 msgr2=0x7fdb0819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0806c5b0 0x7fdb0819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 -- 172.21.15.26:0/3533811627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0806bc90 msgr2=0x7fdb0819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0806bc90 0x7fdb0819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 -- 172.21.15.26:0/3533811627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb081a0d90 con 0x7fdb08110020 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb07fff700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 0x7fdb081a0650 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b9e0 tx=0x7fdb0000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb04ff9700 1 -- 172.21.15.26:0/3533811627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00019070 con 0x7fdb08110020 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb04ff9700 1 -- 172.21.15.26:0/3533811627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb0000cdc0 con 0x7fdb08110020 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.733+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb081a1080 con 0x7fdb08110020 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.734+0000 7fdb04ff9700 1 -- 172.21.15.26:0/3533811627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00014e50 con 0x7fdb08110020 2024-05-20T21:04:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.734+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb081a8900 con 0x7fdb08110020 2024-05-20T21:04:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.737+0000 7fdb04ff9700 1 -- 172.21.15.26:0/3533811627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdb0001c070 con 0x7fdb08110020 2024-05-20T21:04:54.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 Processor -- start 2024-05-20T21:04:54.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb04ff9700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf00b2a20 0x7fdaf00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 -- start start 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb04ff9700 1 -- 172.21.15.26:0/3533811627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fdb000e1620 con 0x7fdb08110020 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf00b2a20 0x7fdaf00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3a20 0x5639389f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563938949c90 con 0x5639389f3070 2024-05-20T21:04:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563938939830 con 0x5639389f7770 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56393893d870 con 0x5639389f3a20 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38464/0 (socket says 172.21.15.26:38464) 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb077fe700 1 -- 172.21.15.26:0/2755894313 learned_addr learned my addr 172.21.15.26:0/2755894313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.738+0000 7fdb07fff700 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb077fe700 1 -- 172.21.15.26:0/2755894313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3a20 msgr2=0x5639389f7150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb077fe700 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3a20 0x5639389f7150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb077fe700 1 -- 172.21.15.26:0/2755894313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 msgr2=0x5639389f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb077fe700 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf00b2a20 0x7fdaf00b4ee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b5c0 tx=0x7fdafc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb077fe700 1 -- 172.21.15.26:0/2755894313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639388898d0 con 0x5639389f3070 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb07fff700 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:04:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdb077fe700 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389f3450 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800ad70 tx=0x7fdaf800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec63664f323222 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:54.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdaedffb700 1 -- 172.21.15.26:0/2755894313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8004020 con 0x5639389f3070 2024-05-20T21:04:54.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdaedffb700 1 -- 172.21.15.26:0/2755894313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf8003680 con 0x5639389f3070 2024-05-20T21:04:54.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.739+0000 7fdaedffb700 1 -- 172.21.15.26:0/2755894313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80186a0 con 0x5639389f3070 2024-05-20T21:04:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 -- 172.21.15.26:0/2755894313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 msgr2=0x5639389f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389f3450 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800ad70 tx=0x7fdaf800f6a0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 -- 172.21.15.26:0/2755894313 shutdown_connections 2024-05-20T21:04:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389f3450 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/2755894313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3a20 0x5639389f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 -- 172.21.15.26:0/2755894313 >> 172.21.15.26:0/2755894313 conn(0x563938918080 msgr2=0x5639389e4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 -- 172.21.15.26:0/2755894313 shutdown_connections 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.740+0000 7fdb183c8b80 1 -- 172.21.15.26:0/2755894313 wait complete. 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 -- start start 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 Processor -- start 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389feae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x5639389fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f7770 0x5639389fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56393883ffa0 con 0x5639389f3070 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5639388db7c0 con 0x5639389f3a20 2024-05-20T21:04:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5639389ff930 con 0x5639389f7770 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb183c8b80 10 client.0 _ll_register_callbacks cb 0x5639388d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389feae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38476/0 (socket says 172.21.15.26:38476) 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb077fe700 1 -- 172.21.15.26:0/1570562681 learned_addr learned my addr 172.21.15.26:0/1570562681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.741+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x5639389fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb07fff700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f7770 0x5639389fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb077fe700 1 -- 172.21.15.26:0/1570562681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f7770 msgr2=0x5639389fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb077fe700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f7770 0x5639389fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb077fe700 1 -- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 msgr2=0x5639389fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb077fe700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x5639389fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb077fe700 1 -- 172.21.15.26:0/1570562681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639388d8690 con 0x5639389f3070 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb077fe700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389feae0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8005c60 tx=0x7fdaf8004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b5c819a3f44b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdae6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdae6ffd700 1 -- 172.21.15.26:0/1570562681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8026020 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdae6ffd700 1 -- 172.21.15.26:0/1570562681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fdaf8005040 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf8003680 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdae6ffd700 1 -- 172.21.15.26:0/1570562681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdaf8031050 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.742+0000 7fdb183c8b80 10 client.18057 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563938a09fb0 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdae6ffd700 1 -- 172.21.15.26:0/1570562681 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf8003680 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdae6ffd700 1 -- 172.21.15.26:0/1570562681 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fdaf8004af0 con 0x5639389f3070 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 10 client.18057 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 20 client.18057 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a0c530 0x563938a0e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 4 client.18057 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639389b69e0 con 0x563938a0c530 2024-05-20T21:04:54.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.743+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a0c530 0x563938a0e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.744+0000 7fdae6ffd700 10 client.18057 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.744+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a0c530 0x563938a0e920 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4b31c85436bc8d server_cookie=7a86d34f691c8968 in_seq=0 out_seq=0 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdae6ffd700 1 -- 172.21.15.26:0/1570562681 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5639389b69e0 con 0x563938a0c530 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdae6ffd700 10 client.18057 handle_command_reply: tid=0 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 client.18057 shutdown 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 msgr2=0x5639389feae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389feae0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8005c60 tx=0x7fdaf8004230 comp rx=0 tx=0).stop 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 shutdown_connections 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a0c530 0x563938a0e920 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3070 0x5639389feae0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdae6ffd700 0 client.18057 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f7770 0x5639389fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/1570562681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x5639389fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 >> 172.21.15.26:0/1570562681 conn(0x563938918080 msgr2=0x5639389e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:54.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 shutdown_connections 2024-05-20T21:04:54.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 1 -- 172.21.15.26:0/1570562681 wait complete. 2024-05-20T21:04:54.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.745+0000 7fdb183c8b80 20 client.18057 trim_cache size 0 max 16384 2024-05-20T21:04:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 Processor -- start 2024-05-20T21:04:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 -- start start 2024-05-20T21:04:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x5639389fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 0x5639389fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081ae470 con 0x5639389f3a20 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081abd00 con 0x5639389f7770 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081088a0 con 0x5639389f3070 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 0x5639389fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 0x5639389fc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38478/0 (socket says 172.21.15.26:38478) 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 -- 172.21.15.26:0/677022003 learned_addr learned my addr 172.21.15.26:0/677022003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb07fff700 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb077fe700 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x5639389fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 -- 172.21.15.26:0/677022003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 msgr2=0x5639389fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x5639389fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 -- 172.21.15.26:0/677022003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 msgr2=0x5639389f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.768+0000 7fdb06ffd700 1 -- 172.21.15.26:0/677022003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639389f9db0 con 0x5639389f3a20 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.769+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 0x5639389fc2e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0093d0 tx=0x7fdafc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a10e78e0f670ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.769+0000 7fdaed7fa700 1 -- 172.21.15.26:0/677022003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc036070 con 0x5639389f3a20 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.769+0000 7fdaed7fa700 1 -- 172.21.15.26:0/677022003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdafc00b950 con 0x5639389f3a20 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.769+0000 7fdaed7fa700 1 -- 172.21.15.26:0/677022003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0047d0 con 0x5639389f3a20 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 -- 172.21.15.26:0/677022003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 msgr2=0x5639389fc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 0x5639389fc2e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0093d0 tx=0x7fdafc01e000 comp rx=0 tx=0).stop 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 -- 172.21.15.26:0/677022003 shutdown_connections 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f7770 0x5639389f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x5639389fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/677022003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f3a20 0x5639389fc2e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 -- 172.21.15.26:0/677022003 >> 172.21.15.26:0/677022003 conn(0x563938918080 msgr2=0x5639389fb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 -- 172.21.15.26:0/677022003 shutdown_connections 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 -- 172.21.15.26:0/677022003 wait complete. 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 -- start start 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.770+0000 7fdb183c8b80 1 Processor -- start 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x563938a05190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x563938a01520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 0x563938a01a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081aa1c0 con 0x5639389f7770 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081ab630 con 0x5639389f3a20 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081aa890 con 0x5639389f3070 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb183c8b80 10 client.0 _ll_register_callbacks cb 0x5639388d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:54.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 0x563938a01a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 0x563938a01a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38490/0 (socket says 172.21.15.26:38490) 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 -- 172.21.15.26:0/378622454 learned_addr learned my addr 172.21.15.26:0/378622454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb06ffd700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x563938a01520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb077fe700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x563938a05190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 -- 172.21.15.26:0/378622454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 msgr2=0x563938a05190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x563938a05190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 -- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 msgr2=0x563938a01520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x563938a01520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb07fff700 1 -- 172.21.15.26:0/378622454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563938a02110 con 0x5639389f7770 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.771+0000 7fdb077fe700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x563938a05190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb07fff700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 0x563938a01a60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000bfd0 tx=0x7fdb00004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c5cb3a3a2cb6d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdae6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdae6ffd700 1 -- 172.21.15.26:0/378622454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb000c1070 con 0x5639389f7770 2024-05-20T21:04:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdae6ffd700 1 -- 172.21.15.26:0/378622454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fdb000b1760 con 0x5639389f7770 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdafc00b950 con 0x5639389f7770 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdae6ffd700 1 -- 172.21.15.26:0/378622454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb000a8690 con 0x5639389f7770 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb183c8b80 10 client.18063 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563938a77d00 con 0x5639389f7770 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdae6ffd700 1 -- 172.21.15.26:0/378622454 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdafc00b950 con 0x5639389f7770 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdae6ffd700 1 -- 172.21.15.26:0/378622454 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fdb000a7de0 con 0x5639389f7770 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb183c8b80 10 client.18063 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb183c8b80 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.772+0000 7fdb183c8b80 20 client.18063 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a78400 0x563938a7a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdb183c8b80 4 client.18063 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56393894bbe0 con 0x563938a78400 2024-05-20T21:04:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdb077fe700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a78400 0x563938a7a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdae6ffd700 10 client.18063 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdb077fe700 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a78400 0x563938a7a820 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1cde30060aff548 server_cookie=958025294a115f64 in_seq=0 out_seq=0 2024-05-20T21:04:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdae6ffd700 1 -- 172.21.15.26:0/378622454 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56393894bbe0 con 0x563938a78400 2024-05-20T21:04:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdae6ffd700 10 client.18063 handle_command_reply: tid=0 2024-05-20T21:04:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.773+0000 7fdb183c8b80 1 client.18063 shutdown 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 msgr2=0x563938a01a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 0x563938a01a60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000bfd0 tx=0x7fdb00004be0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 shutdown_connections 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563938a78400 0x563938a7a820 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639389f7770 0x563938a01a60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5639389f3070 0x563938a05190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 --2- 172.21.15.26:0/378622454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5639389f3a20 0x563938a01520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdae6ffd700 0 client.18063 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 >> 172.21.15.26:0/378622454 conn(0x563938918080 msgr2=0x5639389fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 shutdown_connections 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 1 -- 172.21.15.26:0/378622454 wait complete. 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.774+0000 7fdb183c8b80 20 client.18063 trim_cache size 0 max 16384 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:04:54.780 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf00b2a20 msgr2=0x7fdaf00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf00b2a20 0x7fdaf00b4ee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b5c0 tx=0x7fdafc01a040 comp rx=0 tx=0).stop 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 msgr2=0x7fdb081a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 0x7fdb081a0650 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b9e0 tx=0x7fdb0000bcf0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 shutdown_connections 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fdaf00b2a20 0x7fdaf00b4ee0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08110020 0x7fdb081a0650 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0806bc90 0x7fdb0819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 --2- 172.21.15.26:0/3533811627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0806c5b0 0x7fdb0819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 >> 172.21.15.26:0/3533811627 conn(0x7fdb08079410 msgr2=0x7fdb081114f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 shutdown_connections 2024-05-20T21:04:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:54.775+0000 7fdb0db32700 1 -- 172.21.15.26:0/3533811627 wait complete. 2024-05-20T21:04:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:55 smithi144 ceph-mon[35660]: pgmap v653: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-20T21:04:55.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.35 MB/sec execute 301 sec latency 3400.114 ms 2024-05-20T21:04:55.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:55 smithi026 ceph-mon[31990]: pgmap v653: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-20T21:04:55.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:55 smithi055 ceph-mon[35213]: pgmap v653: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-20T21:04:55.706 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:04:55.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:04:56.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.35 MB/sec execute 302 sec latency 4400.202 ms 2024-05-20T21:04:56.844 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:57.354 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:57 smithi026 ceph-mon[31990]: pgmap v654: 97 pgs: 97 active+clean; 26 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-05-20T21:04:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:57 smithi144 ceph-mon[35660]: pgmap v654: 97 pgs: 97 active+clean; 26 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-05-20T21:04:57.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.34 MB/sec execute 303 sec latency 5400.282 ms 2024-05-20T21:04:57.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:57 smithi055 ceph-mon[35213]: pgmap v654: 97 pgs: 97 active+clean; 26 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-05-20T21:04:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 -- 172.21.15.26:0/932317816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244102c50 msgr2=0x7f52441030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 --2- 172.21.15.26:0/932317816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244102c50 0x7f52441030d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5238009900 tx=0x7f523802a410 comp rx=0 tx=0).stop 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 -- 172.21.15.26:0/932317816 shutdown_connections 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 --2- 172.21.15.26:0/932317816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 0x7f5244109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 --2- 172.21.15.26:0/932317816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244102c50 0x7f52441030d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 --2- 172.21.15.26:0/932317816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5244101a60 0x7f5244101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 -- 172.21.15.26:0/932317816 >> 172.21.15.26:0/932317816 conn(0x7f52440fcfd0 msgr2=0x7f52440ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 -- 172.21.15.26:0/932317816 shutdown_connections 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.845+0000 7f5249a08640 1 -- 172.21.15.26:0/932317816 wait complete. 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 Processor -- start 2024-05-20T21:04:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 -- start start 2024-05-20T21:04:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 0x7f52441059f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5244102c50 0x7f5244104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 0x7f5244104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f524410d420 con 0x7f5244101a60 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f524410d2a0 con 0x7f5244103610 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5249a08640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f524410d5a0 con 0x7f5244102c50 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5242ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 0x7f52441059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5242ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 0x7f52441059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38506/0 (socket says 172.21.15.26:38506) 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f5242ffd640 1 -- 172.21.15.26:0/2509066625 learned_addr learned my addr 172.21.15.26:0/2509066625 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f52427fc640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5244102c50 0x7f5244104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.846+0000 7f52437fe640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 0x7f5244104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5242ffd640 1 -- 172.21.15.26:0/2509066625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5244102c50 msgr2=0x7f5244104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5242ffd640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5244102c50 0x7f5244104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5242ffd640 1 -- 172.21.15.26:0/2509066625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 msgr2=0x7f5244104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5242ffd640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 0x7f5244104570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5242ffd640 1 -- 172.21.15.26:0/2509066625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5244104e00 con 0x7f5244101a60 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f52437fe640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 0x7f5244104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5242ffd640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 0x7f52441059f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f523400d950 tx=0x7f523400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5248a06640 1 -- 172.21.15.26:0/2509066625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234017070 con 0x7f5244101a60 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5248a06640 1 -- 172.21.15.26:0/2509066625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5234009c30 con 0x7f5244101a60 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5248a06640 1 -- 172.21.15.26:0/2509066625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523400f620 con 0x7f5244101a60 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441a2c40 con 0x7f5244101a60 2024-05-20T21:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.847+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a3110 con 0x7f5244101a60 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.849+0000 7f5248a06640 1 -- 172.21.15.26:0/2509066625 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f523401a070 con 0x7f5244101a60 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 Processor -- start 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 -- start start 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97983ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a97988bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a9798b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a978f4c30 con 0x563a979836c0 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563a978e47d0 con 0x563a97989220 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f524c7f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a978e8810 con 0x563a979840b0 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97983ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97983ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38510/0 (socket says 172.21.15.26:38510) 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 -- 172.21.15.26:0/1107440372 learned_addr learned my addr 172.21.15.26:0/1107440372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52427fc640 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a9798b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f5242ffd640 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a97988bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 -- 172.21.15.26:0/1107440372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 msgr2=0x563a97988bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a97988bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 -- 172.21.15.26:0/1107440372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 msgr2=0x563a9798b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a9798b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52437fe640 1 -- 172.21.15.26:0/1107440372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a975f9210 con 0x563a979836c0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.850+0000 7f52427fc640 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a9798b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f52437fe640 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97983ae0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f523c007f40 tx=0x7f523c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d12f04af899b0d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f5248a06640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f522c0b28a0 0x7f522c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f5242ffd640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f522c0b28a0 0x7f522c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f5229ffb640 1 -- 172.21.15.26:0/1107440372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c014070 con 0x563a979836c0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f5229ffb640 1 -- 172.21.15.26:0/1107440372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f523c004d20 con 0x563a979836c0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f5229ffb640 1 -- 172.21.15.26:0/1107440372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c005450 con 0x563a979836c0 2024-05-20T21:04:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.851+0000 7f5248a06640 1 -- 172.21.15.26:0/2509066625 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f52340e1cf0 con 0x7f5244101a60 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- 172.21.15.26:0/1107440372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 msgr2=0x563a97983ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97983ae0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f523c007f40 tx=0x7f523c00e7e0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f5242ffd640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f522c0b28a0 0x7f522c0b4d60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f52340a6500 tx=0x7f523400bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- 172.21.15.26:0/1107440372 shutdown_connections 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a9798b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a97988bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- 172.21.15.26:0/1107440372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97983ae0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- 172.21.15.26:0/1107440372 >> 172.21.15.26:0/1107440372 conn(0x563a9797ed30 msgr2=0x563a9797f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- 172.21.15.26:0/1107440372 shutdown_connections 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- 172.21.15.26:0/1107440372 wait complete. 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- start start 2024-05-20T21:04:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 Processor -- start 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a9798d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9798dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a97994dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a975ca790 con 0x563a979836c0 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563a978a0890 con 0x563a97989220 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.852+0000 7f524c7f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a9798c1d0 con 0x563a979840b0 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f524c7f3740 10 client.0 _ll_register_callbacks cb 0x563a975f8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a9798d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a9798d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38518/0 (socket says 172.21.15.26:38518) 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 -- 172.21.15.26:0/3929962708 learned_addr learned my addr 172.21.15.26:0/3929962708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52427fc640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9798dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a97994dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 -- 172.21.15.26:0/3929962708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 msgr2=0x563a9798dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9798dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 -- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 msgr2=0x563a97994dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a97994dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 -- 172.21.15.26:0/3929962708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a976c5620 con 0x563a979836c0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f52437fe640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a9798d840 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f523c007310 tx=0x7f523c0072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7817ac2ed47c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f5222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f5222ffd640 1 -- 172.21.15.26:0/3929962708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c014070 con 0x563a979836c0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f5222ffd640 1 -- 172.21.15.26:0/3929962708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f523c010a50 con 0x563a979836c0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.853+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523c004d20 con 0x563a979836c0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f5222ffd640 1 -- 172.21.15.26:0/3929962708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f523c010c50 con 0x563a979836c0 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 10 client.18075 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a97997da0 con 0x563a979836c0 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f5222ffd640 1 -- 172.21.15.26:0/3929962708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f523c004d20 con 0x563a979836c0 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f5222ffd640 1 -- 172.21.15.26:0/3929962708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f523c005290 con 0x563a979836c0 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 10 client.18075 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 10 client.18075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 20 client.18075 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97999150 0x563a9799b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 4 client.18075 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a978f6b80 con 0x563a97999150 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.854+0000 7f52427fc640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97999150 0x563a9799b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.855+0000 7f52427fc640 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97999150 0x563a9799b540 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c261ce4c3cd9ddd8 server_cookie=26e930fc937a09b3 in_seq=0 out_seq=0 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.855+0000 7f5222ffd640 10 client.18075 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.855+0000 7f5222ffd640 1 -- 172.21.15.26:0/3929962708 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a978f6b80 con 0x563a97999150 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.855+0000 7f5222ffd640 10 client.18075 handle_command_reply: tid=0 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.855+0000 7f524c7f3740 1 client.18075 shutdown 2024-05-20T21:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 msgr2=0x563a9798d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a9798d840 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f523c007310 tx=0x7f523c0072a0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 shutdown_connections 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97999150 0x563a9799b540 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a97989220 0x563a97994dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9798dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f5222ffd640 0 client.18075 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 --2- 172.21.15.26:0/3929962708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a9798d840 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 >> 172.21.15.26:0/3929962708 conn(0x563a9797ed30 msgr2=0x563a9797dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 shutdown_connections 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 1 -- 172.21.15.26:0/3929962708 wait complete. 2024-05-20T21:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.856+0000 7f524c7f3740 20 client.18075 trim_cache size 0 max 16384 2024-05-20T21:04:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 Processor -- start 2024-05-20T21:04:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 -- start start 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a9797ed30 0x563a9798dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97994d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9797de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52441ae9e0 con 0x563a979836c0 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52441ac2b0 con 0x563a9797ed30 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f524c7f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52441aa100 con 0x563a979840b0 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f52427fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97994d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f52427fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97994d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38522/0 (socket says 172.21.15.26:38522) 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f52427fc640 1 -- 172.21.15.26:0/221448803 learned_addr learned my addr 172.21.15.26:0/221448803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f52437fe640 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a9797ed30 0x563a9798dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.871+0000 7f52437fe640 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9797de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52427fc640 1 -- 172.21.15.26:0/221448803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 msgr2=0x563a9797de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52427fc640 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9797de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52427fc640 1 -- 172.21.15.26:0/221448803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a9797ed30 msgr2=0x563a9798dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52427fc640 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a9797ed30 0x563a9798dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52427fc640 1 -- 172.21.15.26:0/221448803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9799b8b0 con 0x563a979836c0 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52437fe640 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9797de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52427fc640 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97994d80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5238000c00 tx=0x7f5238004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ff73de922348e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52297fa640 1 -- 172.21.15.26:0/221448803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238035040 con 0x563a979836c0 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52297fa640 1 -- 172.21.15.26:0/221448803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5238003dc0 con 0x563a979836c0 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.872+0000 7f52297fa640 1 -- 172.21.15.26:0/221448803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238031500 con 0x563a979836c0 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- 172.21.15.26:0/221448803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 msgr2=0x563a97994d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97994d80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5238000c00 tx=0x7f5238004230 comp rx=0 tx=0).stop 2024-05-20T21:04:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- 172.21.15.26:0/221448803 shutdown_connections 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a979840b0 0x563a9797de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979836c0 0x563a97994d80 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- 172.21.15.26:0/221448803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a9797ed30 0x563a9798dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- 172.21.15.26:0/221448803 >> 172.21.15.26:0/221448803 conn(0x563a97988cf0 msgr2=0x563a9797d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- 172.21.15.26:0/221448803 shutdown_connections 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- 172.21.15.26:0/221448803 wait complete. 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- start start 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 Processor -- start 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a9797ed30 0x563a97992280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a979836c0 0x563a979927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 0x563a97993780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52441aa770 con 0x563a979840b0 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52441abbe0 con 0x563a979836c0 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.873+0000 7f524c7f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52441aae40 con 0x563a9797ed30 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f524c7f3740 10 client.0 _ll_register_callbacks cb 0x7f523c007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 0x563a97993780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 0x563a97993780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38528/0 (socket says 172.21.15.26:38528) 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 -- 172.21.15.26:0/2877781731 learned_addr learned my addr 172.21.15.26:0/2877781731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52427fc640 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a979836c0 0x563a979927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 -- 172.21.15.26:0/2877781731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a9797ed30 msgr2=0x563a97992280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a9797ed30 0x563a97992280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 -- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a979836c0 msgr2=0x563a979927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a979836c0 0x563a979927c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 -- 172.21.15.26:0/2877781731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a97993d70 con 0x563a979840b0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f52437fe640 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 0x563a97993780 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f523c007890 tx=0x7f523c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a54bbac1888f908 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f5222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f5222ffd640 1 -- 172.21.15.26:0/2877781731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c004fb0 con 0x563a979840b0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f5222ffd640 1 -- 172.21.15.26:0/2877781731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f523c005150 con 0x563a979840b0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5238003dc0 con 0x563a979840b0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f5222ffd640 1 -- 172.21.15.26:0/2877781731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f523c045050 con 0x563a979840b0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f524c7f3740 10 client.18081 fetch_fsmap learned FSMap version 30 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.874+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a97995080 con 0x563a979840b0 2024-05-20T21:04:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f5222ffd640 1 -- 172.21.15.26:0/2877781731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5238003dc0 con 0x563a979840b0 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f5222ffd640 1 -- 172.21.15.26:0/2877781731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f523c032470 con 0x563a979840b0 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f524c7f3740 10 client.18081 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f524c7f3740 10 client.18081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f524c7f3740 20 client.18081 populate_metadata read hostname 'smithi026' 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f524c7f3740 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97989220 0x563a97a06300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f524c7f3740 4 client.18081 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a978f0360 con 0x563a97989220 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f52427fc640 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97989220 0x563a97a06300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f52427fc640 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97989220 0x563a97a06300 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0795a4e5c5622f server_cookie=1b32138e7c7bf03d in_seq=0 out_seq=0 2024-05-20T21:04:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.875+0000 7f5222ffd640 10 client.18081 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f5222ffd640 1 -- 172.21.15.26:0/2877781731 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563a978f0360 con 0x563a97989220 2024-05-20T21:04:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f5222ffd640 10 client.18081 handle_command_reply: tid=0 2024-05-20T21:04:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 client.18081 shutdown 2024-05-20T21:04:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 msgr2=0x563a97993780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 0x563a97993780 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f523c007890 tx=0x7f523c005ae0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 shutdown_connections 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a97989220 0x563a97a06300 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a979840b0 0x563a97993780 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a979836c0 0x563a979927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 --2- 172.21.15.26:0/2877781731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a9797ed30 0x563a97992280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f5222ffd640 0 client.18081 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 >> 172.21.15.26:0/2877781731 conn(0x563a97988cf0 msgr2=0x563a9797d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 shutdown_connections 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 1 -- 172.21.15.26:0/2877781731 wait complete. 2024-05-20T21:04:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.902+0000 7f524c7f3740 20 client.18081 trim_cache size 0 max 16384 2024-05-20T21:04:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:04:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:04:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f522c0b28a0 msgr2=0x7f522c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f522c0b28a0 0x7f522c0b4d60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f52340a6500 tx=0x7f523400bf60 comp rx=0 tx=0).stop 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 msgr2=0x7f52441059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 0x7f52441059f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f523400d950 tx=0x7f523400de10 comp rx=0 tx=0).stop 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 shutdown_connections 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f522c0b28a0 0x7f522c0b4d60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5244103610 0x7f5244104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5244102c50 0x7f5244104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 --2- 172.21.15.26:0/2509066625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5244101a60 0x7f52441059f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 >> 172.21.15.26:0/2509066625 conn(0x7f52440fcfd0 msgr2=0x7f52440fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 shutdown_connections 2024-05-20T21:04:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:04:57.903+0000 7f5249a08640 1 -- 172.21.15.26:0/2509066625 wait complete. 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 -- 172.21.15.26:0/2616071664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102e00 msgr2=0x7f2f4410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/2616071664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102e00 0x7f2f4410f2f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00b0a0 tx=0x7f2f3c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 -- 172.21.15.26:0/2616071664 shutdown_connections 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/2616071664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102e00 0x7f2f4410f2f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/2616071664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f44108520 0x7f2f44108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/2616071664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f44102440 0x7f2f441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 -- 172.21.15.26:0/2616071664 >> 172.21.15.26:0/2616071664 conn(0x7f2f440793f0 msgr2=0x7f2f44079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 -- 172.21.15.26:0/2616071664 shutdown_connections 2024-05-20T21:04:58.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 -- 172.21.15.26:0/2616071664 wait complete. 2024-05-20T21:04:58.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 Processor -- start 2024-05-20T21:04:58.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.011+0000 7f2f49c7e700 1 -- start start 2024-05-20T21:04:58.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f49c7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 0x7f2f4419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:58.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f49c7e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f44102e00 0x7f2f4419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:58.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f49c7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f44108520 0x7f2f441999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:58.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f49c7e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44112500 con 0x7f2f44102440 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f49c7e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44112380 con 0x7f2f44102e00 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f49c7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44112680 con 0x7f2f44108520 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 0x7f2f4419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 0x7f2f4419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38548/0 (socket says 172.21.15.26:38548) 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 -- 172.21.15.26:0/585789417 learned_addr learned my addr 172.21.15.26:0/585789417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f43fff700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f44108520 0x7f2f441999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f42ffd700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f44102e00 0x7f2f4419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 -- 172.21.15.26:0/585789417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f44108520 msgr2=0x7f2f441999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f44108520 0x7f2f441999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 -- 172.21.15.26:0/585789417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f44102e00 msgr2=0x7f2f4419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f44102e00 0x7f2f4419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.012+0000 7f2f437fe700 1 -- 172.21.15.26:0/585789417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4419a100 con 0x7f2f44102440 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.013+0000 7f2f437fe700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 0x7f2f4419f790 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d950 tx=0x7f2f3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.013+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34016070 con 0x7f2f44102440 2024-05-20T21:04:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.013+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f3400fd20 con 0x7f2f44102440 2024-05-20T21:04:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.013+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4419a390 con 0x7f2f44102440 2024-05-20T21:04:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.013+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400e3b0 con 0x7f2f44102440 2024-05-20T21:04:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.013+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a6660 con 0x7f2f44102440 2024-05-20T21:04:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.014+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4406a230 con 0x7f2f44102440 2024-05-20T21:04:58.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.015+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2f34019070 con 0x7f2f44102440 2024-05-20T21:04:58.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.016+0000 7f2f40ff9700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f2c0b2880 0x7f2f2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:58.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.016+0000 7f2f42ffd700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f2c0b2880 0x7f2f2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:58.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.016+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f2f340e1d30 con 0x7f2f44102440 2024-05-20T21:04:58.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.017+0000 7f2f42ffd700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f2c0b2880 0x7f2f2c0b4d40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2f44079ba0 tx=0x7f2f38009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.019+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f340a9990 con 0x7f2f44102440 2024-05-20T21:04:58.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.290+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f4402e560 con 0x7f2f44102440 2024-05-20T21:04:58.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.291+0000 7f2f40ff9700 1 -- 172.21.15.26:0/585789417 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f2f340a9290 con 0x7f2f44102440 2024-05-20T21:04:58.290 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:58.290 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:58.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f2c0b2880 msgr2=0x7f2f2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:58.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f2c0b2880 0x7f2f2c0b4d40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2f44079ba0 tx=0x7f2f38009190 comp rx=0 tx=0).stop 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 msgr2=0x7f2f4419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 0x7f2f4419f790 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d950 tx=0x7f2f3400dc60 comp rx=0 tx=0).stop 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 shutdown_connections 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f2f2c0b2880 0x7f2f2c0b4d40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f44102440 0x7f2f4419f790 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f44108520 0x7f2f441999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 --2- 172.21.15.26:0/585789417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f44102e00 0x7f2f4419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 >> 172.21.15.26:0/585789417 conn(0x7f2f440793f0 msgr2=0x7f2f4406b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 shutdown_connections 2024-05-20T21:04:58.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:58.293+0000 7f2f49c7e700 1 -- 172.21.15.26:0/585789417 wait complete. 2024-05-20T21:04:58.294 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:04:58.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.33 MB/sec execute 304 sec latency 6400.326 ms 2024-05-20T21:04:59.203 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 1e6d7737 --release --await' 2024-05-20T21:04:59.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 1e6d7737 --release --await 2024-05-20T21:04:59.440 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:59 smithi026 ceph-mon[31990]: pgmap v655: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-05-20T21:04:59.440 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:04:59 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/585789417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:59 smithi144 ceph-mon[35660]: pgmap v655: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-05-20T21:04:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:04:59 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/585789417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:59.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.32 MB/sec execute 305 sec latency 7400.408 ms 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 -- 172.21.15.26:0/3218679254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c103820 msgr2=0x7efe5c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 --2- 172.21.15.26:0/3218679254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c103820 0x7efe5c103c00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0099c0 tx=0x7efe4c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 -- 172.21.15.26:0/3218679254 shutdown_connections 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 --2- 172.21.15.26:0/3218679254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c103820 0x7efe5c103c00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 --2- 172.21.15.26:0/3218679254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c108910 0x7efe5c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 --2- 172.21.15.26:0/3218679254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c1041d0 0x7efe5c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 -- 172.21.15.26:0/3218679254 >> 172.21.15.26:0/3218679254 conn(0x7efe5c07a6b0 msgr2=0x7efe5c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 -- 172.21.15.26:0/3218679254 shutdown_connections 2024-05-20T21:04:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.698+0000 7efe629e3700 1 -- 172.21.15.26:0/3218679254 wait complete. 2024-05-20T21:04:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 Processor -- start 2024-05-20T21:04:59.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 -- start start 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c103820 0x7efe5c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1041d0 0x7efe5c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 0x7efe5c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1135a0 con 0x7efe5c108910 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c113420 con 0x7efe5c103820 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.699+0000 7efe629e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c113720 con 0x7efe5c1041d0 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 0x7efe5c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:59.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 0x7efe5c0762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43110/0 (socket says 172.21.15.26:43110) 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 -- 172.21.15.26:0/1061551054 learned_addr learned my addr 172.21.15.26:0/1061551054 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe5b7fe700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1041d0 0x7efe5c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe5bfff700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c103820 0x7efe5c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 -- 172.21.15.26:0/1061551054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1041d0 msgr2=0x7efe5c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1041d0 0x7efe5c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 -- 172.21.15.26:0/1061551054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c103820 msgr2=0x7efe5c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c103820 0x7efe5c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 -- 172.21.15.26:0/1061551054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1a7fc0 con 0x7efe5c108910 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe60f80700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 0x7efe5c0762c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7efe5400ada0 tx=0x7efe5400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54017020 con 0x7efe5c108910 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1a82b0 con 0x7efe5c108910 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5400ce10 con 0x7efe5c108910 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54005280 con 0x7efe5c108910 2024-05-20T21:04:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.700+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a87f0 con 0x7efe5c108910 2024-05-20T21:04:59.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:59 smithi055 ceph-mon[35213]: pgmap v655: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-05-20T21:04:59.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:04:59 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/585789417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:04:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.702+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c10bb50 con 0x7efe5c108910 2024-05-20T21:04:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.703+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efe5400c980 con 0x7efe5c108910 2024-05-20T21:04:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.703+0000 7efe597fa700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe440b2880 0x7efe440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:04:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.704+0000 7efe5bfff700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe440b2880 0x7efe440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:04:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.704+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efe540e19c0 con 0x7efe5c108910 2024-05-20T21:04:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.704+0000 7efe5bfff700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe440b2880 0x7efe440b4d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000c00 tx=0x7efe4c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:04:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.708+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe54021020 con 0x7efe5c108910 2024-05-20T21:04:59.742 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:04:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.983+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5c10b530 con 0x7efe5c108910 2024-05-20T21:04:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.984+0000 7efe597fa700 1 -- 172.21.15.26:0/1061551054 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7efe540a96b0 con 0x7efe5c108910 2024-05-20T21:04:59.983 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:04:59.983 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:04:59.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe440b2880 msgr2=0x7efe440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:59.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe440b2880 0x7efe440b4d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000c00 tx=0x7efe4c01a040 comp rx=0 tx=0).stop 2024-05-20T21:04:59.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 msgr2=0x7efe5c0762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:04:59.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 0x7efe5c0762c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7efe5400ada0 tx=0x7efe5400c330 comp rx=0 tx=0).stop 2024-05-20T21:04:59.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 shutdown_connections 2024-05-20T21:04:59.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efe440b2880 0x7efe440b4d40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe5c108910 0x7efe5c0762c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe5c103820 0x7efe5c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 --2- 172.21.15.26:0/1061551054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c1041d0 0x7efe5c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 >> 172.21.15.26:0/1061551054 conn(0x7efe5c07a6b0 msgr2=0x7efe5c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 shutdown_connections 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:04:59.986+0000 7efe629e3700 1 -- 172.21.15.26:0/1061551054 wait complete. 2024-05-20T21:04:59.986 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:05:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:00 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1061551054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:00.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.32 MB/sec execute 306 sec latency 8400.469 ms 2024-05-20T21:05:00.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:00 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1061551054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:00.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:00 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1061551054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:01.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:05:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:01 smithi026 ceph-mon[31990]: pgmap v656: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 879 KiB/s wr, 72 op/s 2024-05-20T21:05:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:01 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:05:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:01 smithi144 ceph-mon[35660]: pgmap v656: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 879 KiB/s wr, 72 op/s 2024-05-20T21:05:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:01 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:05:01.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197691 2.31 MB/sec execute 307 sec latency 9400.563 ms 2024-05-20T21:05:01.691 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:01 smithi055 ceph-mon[35213]: pgmap v656: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 879 KiB/s wr, 72 op/s 2024-05-20T21:05:01.691 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:01 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:05:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.166+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/3515580586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806c470 msgr2=0x7f6bc81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.166+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/3515580586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806c470 0x7f6bc81103c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b0a0 tx=0x7f6bc000b3b0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/3515580586 shutdown_connections 2024-05-20T21:05:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/3515580586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806c470 0x7f6bc81103c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/3515580586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 0x7f6bc8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/3515580586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc806bab0 0x7f6bc806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/3515580586 >> 172.21.15.26:0/3515580586 conn(0x7f6bc80793b0 msgr2=0x7f6bc80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/3515580586 shutdown_connections 2024-05-20T21:05:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.167+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/3515580586 wait complete. 2024-05-20T21:05:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.168+0000 7f6bcd0d8700 1 Processor -- start 2024-05-20T21:05:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.168+0000 7f6bcd0d8700 1 -- start start 2024-05-20T21:05:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bcd0d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 0x7f6bc819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bcd0d8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc806c470 0x7f6bc819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bcd0d8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 0x7f6bc8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bcd0d8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc8113570 con 0x7f6bc806bab0 2024-05-20T21:05:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bcd0d8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81133f0 con 0x7f6bc81095a0 2024-05-20T21:05:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bcd0d8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81136f0 con 0x7f6bc806c470 2024-05-20T21:05:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bc6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 0x7f6bc819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bc6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 0x7f6bc819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43140/0 (socket says 172.21.15.26:43140) 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bc6d9d700 1 -- 172.21.15.26:0/1784731791 learned_addr learned my addr 172.21.15.26:0/1784731791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 0x7f6bc8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.169+0000 7f6bc659c700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc806c470 0x7f6bc819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc759e700 1 -- 172.21.15.26:0/1784731791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc806c470 msgr2=0x7f6bc819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc806c470 0x7f6bc819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc759e700 1 -- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 msgr2=0x7f6bc819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 0x7f6bc819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc759e700 1 -- 172.21.15.26:0/1784731791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc819a050 con 0x7f6bc81095a0 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 0x7f6bc819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 0x7f6bc8199870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b0a0 tx=0x7f6bc0004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6baffff700 1 -- 172.21.15.26:0/1784731791 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0021070 con 0x7f6bc81095a0 2024-05-20T21:05:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6baffff700 1 -- 172.21.15.26:0/1784731791 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc0007b80 con 0x7f6bc81095a0 2024-05-20T21:05:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc819a2e0 con 0x7f6bc81095a0 2024-05-20T21:05:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6baffff700 1 -- 172.21.15.26:0/1784731791 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0005040 con 0x7f6bc81095a0 2024-05-20T21:05:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.170+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc81a6500 con 0x7f6bc81095a0 2024-05-20T21:05:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.173+0000 7f6baffff700 1 -- 172.21.15.26:0/1784731791 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6bc0024070 con 0x7f6bc81095a0 2024-05-20T21:05:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.173+0000 7f6baffff700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bb00b2880 0x7f6bb00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.174+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bb00b2880 0x7f6bb00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.174+0000 7f6baffff700 1 -- 172.21.15.26:0/1784731791 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6bc000f020 con 0x7f6bc81095a0 2024-05-20T21:05:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.174+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bb00b2880 0x7f6bb00b4d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800f4d0 tx=0x7f6bb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.175+0000 7f6bd796eb80 1 Processor -- start 2024-05-20T21:05:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 -- start start 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b05d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fb41a64d90 con 0x55fb41b0b2b0 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fb41a54930 con 0x55fb41b062a0 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fb41a58970 con 0x55fb41b058e0 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43156/0 (socket says 172.21.15.26:43156) 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 -- 172.21.15.26:0/1677046170 learned_addr learned my addr 172.21.15.26:0/1677046170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc659c700 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b05d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 -- 172.21.15.26:0/1677046170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 msgr2=0x55fb41b05d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b05d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 -- 172.21.15.26:0/1677046170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 msgr2=0x55fb41b120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.176+0000 7f6bc6d9d700 1 -- 172.21.15.26:0/1677046170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb419a49d0 con 0x55fb41b0b2b0 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.177+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0b690 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8010750 tx=0x7f6bb8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ceb5c0dec2a5d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.177+0000 7f6bad7fa700 1 -- 172.21.15.26:0/1677046170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8016070 con 0x55fb41b0b2b0 2024-05-20T21:05:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.177+0000 7f6bad7fa700 1 -- 172.21.15.26:0/1677046170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bb8025900 con 0x55fb41b0b2b0 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bad7fa700 1 -- 172.21.15.26:0/1677046170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb801ba80 con 0x55fb41b0b2b0 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1677046170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 msgr2=0x55fb41b0b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0b690 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8010750 tx=0x7f6bb8012000 comp rx=0 tx=0).stop 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1677046170 shutdown_connections 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0b690 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1677046170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b05d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1677046170 >> 172.21.15.26:0/1677046170 conn(0x55fb41a33180 msgr2=0x55fb41b001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1677046170 shutdown_connections 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.178+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1677046170 wait complete. 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 -- start start 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 Processor -- start 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b0f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fb4195b0a0 con 0x55fb41b0b2b0 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fb419f68c0 con 0x55fb41b062a0 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fb41b12790 con 0x55fb41b058e0 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bc759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bc659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b0f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bc659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b0f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46408/0 (socket says 172.21.15.26:46408) 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bc659c700 1 -- 172.21.15.26:0/588815206 learned_addr learned my addr 172.21.15.26:0/588815206 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.179+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bd796eb80 10 client.0 _ll_register_callbacks cb 0x55fb419efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bc759e700 1 -- 172.21.15.26:0/588815206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 msgr2=0x55fb41b11da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bc759e700 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b11da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bc759e700 1 -- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 msgr2=0x55fb41b0f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bc759e700 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b0f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bc759e700 1 -- 172.21.15.26:0/588815206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb419f3790 con 0x55fb41b0b2b0 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bc759e700 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0f5e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6bc001dfd0 tx=0x7f6bc00aa7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d1882ebb42dbad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6baaff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6baaff5700 1 -- 172.21.15.26:0/588815206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00ab470 con 0x55fb41b0b2b0 2024-05-20T21:05:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6baaff5700 1 -- 172.21.15.26:0/588815206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6bc00c5360 con 0x55fb41b0b2b0 2024-05-20T21:05:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb8025900 con 0x55fb41b0b2b0 2024-05-20T21:05:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.180+0000 7f6baaff5700 1 -- 172.21.15.26:0/588815206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6bc00aad70 con 0x55fb41b0b2b0 2024-05-20T21:05:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 10 client.18105 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb41b1ce90 con 0x55fb41b0b2b0 2024-05-20T21:05:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6baaff5700 1 -- 172.21.15.26:0/588815206 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc00c2030 con 0x55fb41b0b2b0 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6baaff5700 1 -- 172.21.15.26:0/588815206 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f6bc0011070 con 0x55fb41b0b2b0 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 10 client.18105 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 10 client.18105 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 20 client.18105 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b21890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 4 client.18105 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.181+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6bb8025900 con 0x55fb41b1f4a0 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.182+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b21890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.182+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b21890 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49cc369854f8e23b server_cookie=644552277515a504 in_seq=0 out_seq=0 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.182+0000 7f6baaff5700 10 client.18105 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6baaff5700 1 -- 172.21.15.26:0/588815206 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f6bb8025900 con 0x55fb41b1f4a0 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6baaff5700 10 client.18105 handle_command_reply: tid=0 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 client.18105 shutdown 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 msgr2=0x55fb41b0f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0f5e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6bc001dfd0 tx=0x7f6bc00aa7b0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 shutdown_connections 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b21890 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b0f5e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6baaff5700 0 client.18105 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b11da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/588815206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b062a0 0x55fb41b0f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 >> 172.21.15.26:0/588815206 conn(0x55fb41a33180 msgr2=0x55fb41afaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 shutdown_connections 2024-05-20T21:05:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.183+0000 7f6bd796eb80 1 -- 172.21.15.26:0/588815206 wait complete. 2024-05-20T21:05:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.184+0000 7f6bd796eb80 20 client.18105 trim_cache size 0 max 16384 2024-05-20T21:05:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.209+0000 7f6bd796eb80 1 Processor -- start 2024-05-20T21:05:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.209+0000 7f6bd796eb80 1 -- start start 2024-05-20T21:05:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.209+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b058e0 0x55fb41b0a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.209+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b062a0 0x55fb41b118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.209+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b27180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81b03f0 con 0x55fb41b0b2b0 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81adc80 con 0x55fb41b058e0 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81abab0 con 0x55fb41b062a0 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b27180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43178/0 (socket says 172.21.15.26:43178) 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 -- 172.21.15.26:0/1331645332 learned_addr learned my addr 172.21.15.26:0/1331645332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b058e0 0x55fb41b0a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc659c700 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b062a0 0x55fb41b118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 -- 172.21.15.26:0/1331645332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b062a0 msgr2=0x55fb41b118e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b062a0 0x55fb41b118e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 -- 172.21.15.26:0/1331645332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b058e0 msgr2=0x55fb41b0a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b058e0 0x55fb41b0a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 -- 172.21.15.26:0/1331645332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb41afc9f0 con 0x55fb41b0b2b0 2024-05-20T21:05:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bc759e700 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b27180 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b070 tx=0x7f6bc001da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8d85551624c230 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bacff9700 1 -- 172.21.15.26:0/1331645332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00af120 con 0x55fb41b0b2b0 2024-05-20T21:05:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bacff9700 1 -- 172.21.15.26:0/1331645332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc00c2020 con 0x55fb41b0b2b0 2024-05-20T21:05:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.210+0000 7f6bacff9700 1 -- 172.21.15.26:0/1331645332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc001d450 con 0x55fb41b0b2b0 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1331645332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 msgr2=0x55fb41b27180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b27180 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b070 tx=0x7f6bc001da40 comp rx=0 tx=0).stop 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1331645332 shutdown_connections 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b0b2b0 0x55fb41b27180 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b058e0 0x55fb41b0a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/1331645332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b062a0 0x55fb41b118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1331645332 >> 172.21.15.26:0/1331645332 conn(0x55fb41a33180 msgr2=0x55fb41b09620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1331645332 shutdown_connections 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.211+0000 7f6bd796eb80 1 -- 172.21.15.26:0/1331645332 wait complete. 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 -- start start 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 Processor -- start 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b17cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 0x55fb41b18230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b0b2b0 0x55fb41b14b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81ac140 con 0x55fb41b062a0 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81ad5b0 con 0x55fb41b0b2b0 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc81ac810 con 0x55fb41b058e0 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bd796eb80 10 client.0 _ll_register_callbacks cb 0x55fb419efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 0x55fb41b18230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b17cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b17cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:57826/0 (socket says 172.21.15.26:57826) 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 0x55fb41b18230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43188/0 (socket says 172.21.15.26:43188) 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc6d9d700 1 -- 172.21.15.26:0/3343411890 learned_addr learned my addr 172.21.15.26:0/3343411890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc759e700 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b0b2b0 0x55fb41b14b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc659c700 1 -- 172.21.15.26:0/3343411890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 msgr2=0x55fb41b17cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc659c700 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b17cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc659c700 1 -- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b0b2b0 msgr2=0x55fb41b14b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.212+0000 7f6bc659c700 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b0b2b0 0x55fb41b14b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6bc659c700 1 -- 172.21.15.26:0/3343411890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb41b15380 con 0x55fb41b062a0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6bc659c700 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 0x55fb41b18230 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55fb41b10300 tx=0x7f6bbc012780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd48ca36b6880c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6baaff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6baaff5700 1 -- 172.21.15.26:0/3343411890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc004020 con 0x55fb41b062a0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6baaff5700 1 -- 172.21.15.26:0/3343411890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6bbc0041c0 con 0x55fb41b062a0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bc00c2020 con 0x55fb41b062a0 2024-05-20T21:05:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6baaff5700 1 -- 172.21.15.26:0/3343411890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6bbc0043c0 con 0x55fb41b062a0 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6bd796eb80 10 client.18111 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb41b8adb0 con 0x55fb41b062a0 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.213+0000 7f6baaff5700 1 -- 172.21.15.26:0/3343411890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc00c2020 con 0x55fb41b062a0 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6baaff5700 1 -- 172.21.15.26:0/3343411890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f6bbc005390 con 0x55fb41b062a0 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bd796eb80 10 client.18111 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bd796eb80 10 client.18111 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bd796eb80 20 client.18111 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b8d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bd796eb80 4 client.18111 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1e6d7737", "release": true, "await": true}] 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1e6d7737", "release": true, "await": true}) -- 0x55fb41ad1ae0 con 0x55fb41b1f4a0 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b8d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6baaff5700 10 client.18111 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.214+0000 7f6bc6d9d700 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b8d550 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec9767a5de05383 server_cookie=16354ad1b0b16f23 in_seq=0 out_seq=0 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.235+0000 7f6baaff5700 1 -- 172.21.15.26:0/3343411890 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fb41ad1ae0 con 0x55fb41b1f4a0 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.235+0000 7f6baaff5700 10 client.18111 handle_command_reply: tid=0 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.235+0000 7f6bd796eb80 1 client.18111 shutdown 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.235+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 msgr2=0x55fb41b18230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.235+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 0x55fb41b18230 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55fb41b10300 tx=0x7f6bbc012780 comp rx=0 tx=0).stop 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 shutdown_connections 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fb41b1f4a0 0x55fb41b8d550 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb41b0b2b0 0x55fb41b14b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb41b058e0 0x55fb41b17cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 --2- 172.21.15.26:0/3343411890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb41b062a0 0x55fb41b18230 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 >> 172.21.15.26:0/3343411890 conn(0x55fb41a33180 msgr2=0x55fb41b09620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6baaff5700 0 client.18111 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 shutdown_connections 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 1 -- 172.21.15.26:0/3343411890 wait complete. 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bd796eb80 20 client.18111 trim_cache size 0 max 16384 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:05:02.236 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 36, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "1e6d7737": { 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 36, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 130.0, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:05:02.237 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:02.238 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bb00b2880 msgr2=0x7f6bb00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bb00b2880 0x7f6bb00b4d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800f4d0 tx=0x7f6bb8005f90 comp rx=0 tx=0).stop 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 msgr2=0x7f6bc8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.236+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 0x7f6bc8199870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b0a0 tx=0x7f6bc0004ce0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 shutdown_connections 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bb00b2880 0x7f6bb00b4d40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc806bab0 0x7f6bc819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bc81095a0 0x7f6bc8199870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 --2- 172.21.15.26:0/1784731791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc806c470 0x7f6bc819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 >> 172.21.15.26:0/1784731791 conn(0x7f6bc80793b0 msgr2=0x7f6bc8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 shutdown_connections 2024-05-20T21:05:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:02.237+0000 7f6bcd0d8700 1 -- 172.21.15.26:0/1784731791 wait complete. 2024-05-20T21:05:02.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 198030 2.31 MB/sec execute 308 sec latency 10067.260 ms 2024-05-20T21:05:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:03 smithi026 ceph-mon[31990]: pgmap v657: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 669 KiB/s wr, 60 op/s 2024-05-20T21:05:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:03 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:03 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:03 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:03 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:03 smithi144 ceph-mon[35660]: pgmap v657: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 669 KiB/s wr, 60 op/s 2024-05-20T21:05:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:03 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:03 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:03 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:03 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 198628 2.32 MB/sec execute 309 sec latency 17.852 ms 2024-05-20T21:05:03.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:03 smithi055 ceph-mon[35213]: pgmap v657: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 669 KiB/s wr, 60 op/s 2024-05-20T21:05:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:03 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:03 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:03 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:03 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:03.928 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1e6d7737', total seconds elapsed: 19.5 2024-05-20T21:05:04.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 199086 2.32 MB/sec execute 310 sec latency 18.262 ms 2024-05-20T21:05:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.280+0000 7efd4b810700 1 -- 172.21.15.26:0/1577806575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102e00 msgr2=0x7efd4410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.280+0000 7efd4b810700 1 --2- 172.21.15.26:0/1577806575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102e00 0x7efd4410f2f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00b0a0 tx=0x7efd3c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 -- 172.21.15.26:0/1577806575 shutdown_connections 2024-05-20T21:05:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 --2- 172.21.15.26:0/1577806575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102e00 0x7efd4410f2f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 --2- 172.21.15.26:0/1577806575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd44108520 0x7efd44108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 --2- 172.21.15.26:0/1577806575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd44102440 0x7efd441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 -- 172.21.15.26:0/1577806575 >> 172.21.15.26:0/1577806575 conn(0x7efd440793f0 msgr2=0x7efd44079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 -- 172.21.15.26:0/1577806575 shutdown_connections 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 -- 172.21.15.26:0/1577806575 wait complete. 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 Processor -- start 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.281+0000 7efd4b810700 1 -- start start 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 0x7efd4419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd44102e00 0x7efd4419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd44108520 0x7efd441a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd441124d0 con 0x7efd44102440 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd44112350 con 0x7efd44102e00 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd44112650 con 0x7efd44108520 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 0x7efd4419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 0x7efd4419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43210/0 (socket says 172.21.15.26:43210) 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 -- 172.21.15.26:0/2046780011 learned_addr learned my addr 172.21.15.26:0/2046780011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd48dab700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd44102e00 0x7efd4419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd49dad700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd44108520 0x7efd441a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 -- 172.21.15.26:0/2046780011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd44108520 msgr2=0x7efd441a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd44108520 0x7efd441a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 -- 172.21.15.26:0/2046780011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd44102e00 msgr2=0x7efd4419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd44102e00 0x7efd4419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 -- 172.21.15.26:0/2046780011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd441a5f30 con 0x7efd44102440 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd495ac700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 0x7efd4419a970 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efd4000eb30 tx=0x7efd4000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd3a7fc700 1 -- 172.21.15.26:0/2046780011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4000cbd0 con 0x7efd44102440 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd441a6100 con 0x7efd44102440 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd3a7fc700 1 -- 172.21.15.26:0/2046780011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd400044b0 con 0x7efd44102440 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd3a7fc700 1 -- 172.21.15.26:0/2046780011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40003dc0 con 0x7efd44102440 2024-05-20T21:05:05.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.282+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd441a6630 con 0x7efd44102440 2024-05-20T21:05:05.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.286+0000 7efd3a7fc700 1 -- 172.21.15.26:0/2046780011 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efd40003f60 con 0x7efd44102440 2024-05-20T21:05:05.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.287+0000 7efd3a7fc700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efd300b2880 0x7efd300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.287+0000 7efd48dab700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efd300b2880 0x7efd300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.287+0000 7efd3a7fc700 1 -- 172.21.15.26:0/2046780011 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efd400e2410 con 0x7efd44102440 2024-05-20T21:05:05.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd48dab700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efd300b2880 0x7efd300b4d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efd44079a60 tx=0x7efd34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:05.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 Processor -- start 2024-05-20T21:05:05.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 -- start start 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a29727dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297281940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a2972823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2971dcc90 con 0x55a29727d860 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a2971cc830 con 0x55a29727e210 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd560a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2971d0870 con 0x55a297281f60 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd495ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a29727dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd48dab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297281940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd495ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a29727dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43224/0 (socket says 172.21.15.26:43224) 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd495ac700 1 -- 172.21.15.26:0/1167628072 learned_addr learned my addr 172.21.15.26:0/1167628072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.288+0000 7efd49dad700 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a2972823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd48dab700 1 -- 172.21.15.26:0/1167628072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 msgr2=0x55a2972823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd48dab700 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a2972823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd48dab700 1 -- 172.21.15.26:0/1167628072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 msgr2=0x55a29727dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd48dab700 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a29727dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd48dab700 1 -- 172.21.15.26:0/1167628072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29711c8d0 con 0x55a29727e210 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd495ac700 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a29727dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd48dab700 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297281940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd34011c60 tx=0x7efd34014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9dca80fa3f59ed server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd2f7fe700 1 -- 172.21.15.26:0/1167628072 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34004230 con 0x55a29727e210 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd2f7fe700 1 -- 172.21.15.26:0/1167628072 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd340043d0 con 0x55a29727e210 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.289+0000 7efd2f7fe700 1 -- 172.21.15.26:0/1167628072 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34003ca0 con 0x55a29727e210 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 -- 172.21.15.26:0/1167628072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 msgr2=0x55a297281940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297281940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd34011c60 tx=0x7efd34014000 comp rx=0 tx=0).stop 2024-05-20T21:05:05.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 -- 172.21.15.26:0/1167628072 shutdown_connections 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a29727dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a2972823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1167628072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297281940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 -- 172.21.15.26:0/1167628072 >> 172.21.15.26:0/1167628072 conn(0x55a2971ab080 msgr2=0x55a297277e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 -- 172.21.15.26:0/1167628072 shutdown_connections 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 -- 172.21.15.26:0/1167628072 wait complete. 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.290+0000 7efd560a6b80 1 -- start start 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 Processor -- start 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a297289420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297286720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a297286c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2970d2fa0 con 0x55a29727d860 2024-05-20T21:05:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a29716e7c0 con 0x55a29727e210 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a29728a120 con 0x55a297281f60 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd495ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a297289420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd560a6b80 10 client.0 _ll_register_callbacks cb 0x55a297167920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd495ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a297289420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43234/0 (socket says 172.21.15.26:43234) 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd495ac700 1 -- 172.21.15.26:0/4228504714 learned_addr learned my addr 172.21.15.26:0/4228504714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd48dab700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297286720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd49dad700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a297286c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd495ac700 1 -- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 msgr2=0x55a297286c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.291+0000 7efd495ac700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a297286c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd495ac700 1 -- 172.21.15.26:0/4228504714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 msgr2=0x55a297286720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd495ac700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297286720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd495ac700 1 -- 172.21.15.26:0/4228504714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29716b690 con 0x55a29727d860 2024-05-20T21:05:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd48dab700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297286720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd495ac700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a297289420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efd400099b0 tx=0x7efd400a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebae5ef95b04e482 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd2cff9700 1 -- 172.21.15.26:0/4228504714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400a9140 con 0x55a29727d860 2024-05-20T21:05:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd2cff9700 1 -- 172.21.15.26:0/4228504714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efd400cc2d0 con 0x55a29727d860 2024-05-20T21:05:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd340043d0 con 0x55a29727d860 2024-05-20T21:05:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd2cff9700 1 -- 172.21.15.26:0/4228504714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd400a5160 con 0x55a29727d860 2024-05-20T21:05:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd560a6b80 10 client.18123 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a297298d10 con 0x55a29727d860 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.292+0000 7efd2cff9700 1 -- 172.21.15.26:0/4228504714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd340043d0 con 0x55a29727d860 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd2cff9700 1 -- 172.21.15.26:0/4228504714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7efd4006f250 con 0x55a29727d860 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd560a6b80 10 client.18123 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd560a6b80 10 client.18123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd560a6b80 20 client.18123 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd560a6b80 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29729b2a0 0x55a29729d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd560a6b80 4 client.18123 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2972499e0 con 0x55a29729b2a0 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd48dab700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29729b2a0 0x55a29729d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd48dab700 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29729b2a0 0x55a29729d6d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65ee0a45c6123963 server_cookie=ca8cd85a9f7039ea in_seq=0 out_seq=0 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.293+0000 7efd2cff9700 10 client.18123 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:05:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd2cff9700 1 -- 172.21.15.26:0/4228504714 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a2972499e0 con 0x55a29729b2a0 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd2cff9700 10 client.18123 handle_command_reply: tid=0 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 client.18123 shutdown 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 msgr2=0x55a297289420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a297289420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efd400099b0 tx=0x7efd400a9890 comp rx=0 tx=0).stop 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 shutdown_connections 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29729b2a0 0x55a29729d6d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a29727d860 0x55a297289420 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a297281f60 0x55a297286c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd2cff9700 0 client.18123 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 --2- 172.21.15.26:0/4228504714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727e210 0x55a297286720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.294+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 >> 172.21.15.26:0/4228504714 conn(0x55a2971ab080 msgr2=0x55a297283350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.295+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 shutdown_connections 2024-05-20T21:05:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.295+0000 7efd560a6b80 1 -- 172.21.15.26:0/4228504714 wait complete. 2024-05-20T21:05:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.295+0000 7efd560a6b80 20 client.18123 trim_cache size 0 max 16384 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 Processor -- start 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 -- start start 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 0x55a297278470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2971ab080 0x55a2972858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727d860 0x55a297288660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd441ae390 con 0x55a2971ab080 2024-05-20T21:05:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd441abc20 con 0x55a29727d860 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd560a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd441a9a90 con 0x55a2972779b0 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd48dab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2971ab080 0x55a2972858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd49dad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727d860 0x55a297288660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd495ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 0x55a297278470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd495ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 0x55a297278470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:57860/0 (socket says 172.21.15.26:57860) 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd49dad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727d860 0x55a297288660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46458/0 (socket says 172.21.15.26:46458) 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd48dab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2971ab080 0x55a2972858a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43242/0 (socket says 172.21.15.26:43242) 2024-05-20T21:05:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.312+0000 7efd495ac700 1 -- 172.21.15.26:0/1102793604 learned_addr learned my addr 172.21.15.26:0/1102793604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:05.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd495ac700 1 -- 172.21.15.26:0/1102793604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727d860 msgr2=0x55a297288660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd495ac700 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727d860 0x55a297288660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd495ac700 1 -- 172.21.15.26:0/1102793604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2971ab080 msgr2=0x55a2972858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd495ac700 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2971ab080 0x55a2972858a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd495ac700 1 -- 172.21.15.26:0/1102793604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a297278d40 con 0x55a2972779b0 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd495ac700 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 0x55a297278470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd400e0220 tx=0x7efd40004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd94aa53a025ac0f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd2effd700 1 -- 172.21.15.26:0/1102793604 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400a9320 con 0x55a2972779b0 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd2effd700 1 -- 172.21.15.26:0/1102793604 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd400c3070 con 0x55a2972779b0 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.313+0000 7efd2effd700 1 -- 172.21.15.26:0/1102793604 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4006f250 con 0x55a2972779b0 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 -- 172.21.15.26:0/1102793604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 msgr2=0x55a297278470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 0x55a297278470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd400e0220 tx=0x7efd40004660 comp rx=0 tx=0).stop 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 -- 172.21.15.26:0/1102793604 shutdown_connections 2024-05-20T21:05:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a29727d860 0x55a297288660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2972779b0 0x55a297278470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 --2- 172.21.15.26:0/1102793604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2971ab080 0x55a2972858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 -- 172.21.15.26:0/1102793604 >> 172.21.15.26:0/1102793604 conn(0x55a297274b00 msgr2=0x55a297287f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 -- 172.21.15.26:0/1102793604 shutdown_connections 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.314+0000 7efd560a6b80 1 -- 172.21.15.26:0/1102793604 wait complete. 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 -- start start 2024-05-20T21:05:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 Processor -- start 2024-05-20T21:05:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 0x55a29727f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2972779b0 0x55a29727fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29727d860 0x55a297280220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd441aa0e0 con 0x55a2972779b0 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd441ab550 con 0x55a2971ab080 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd441aa7b0 con 0x55a29727d860 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd560a6b80 10 client.0 _ll_register_callbacks cb 0x55a297167920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd495ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 0x55a29727f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd48dab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2972779b0 0x55a29727fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd495ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 0x55a29727f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46470/0 (socket says 172.21.15.26:46470) 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd49dad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29727d860 0x55a297280220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.315+0000 7efd495ac700 1 -- 172.21.15.26:0/329388214 learned_addr learned my addr 172.21.15.26:0/329388214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd495ac700 1 -- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29727d860 msgr2=0x55a297280220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd495ac700 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29727d860 0x55a297280220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd495ac700 1 -- 172.21.15.26:0/329388214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2972779b0 msgr2=0x55a29727fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:05.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd495ac700 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2972779b0 0x55a29727fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd495ac700 1 -- 172.21.15.26:0/329388214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a297305080 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd495ac700 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 0x55a29727f7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd40019fd0 tx=0x7efd400a9890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcf427e40f05659 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd2cff9700 1 -- 172.21.15.26:0/329388214 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400a9320 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd400c3070 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd2cff9700 1 -- 172.21.15.26:0/329388214 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efd400a5fc0 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd2cff9700 1 -- 172.21.15.26:0/329388214 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd400b2170 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd560a6b80 10 client.27524 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.316+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a297306360 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd2cff9700 1 -- 172.21.15.26:0/329388214 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd400c3070 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd2cff9700 1 -- 172.21.15.26:0/329388214 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7efd400a5340 con 0x55a2971ab080 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd560a6b80 10 client.27524 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:05.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd560a6b80 10 client.27524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd560a6b80 20 client.27524 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd560a6b80 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29727e210 0x55a297308640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd560a6b80 4 client.27524 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2971debe0 con 0x55a29727e210 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd48dab700 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29727e210 0x55a297308640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd48dab700 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29727e210 0x55a297308640 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6cb7f91a11a80747 server_cookie=28145c1d010623cb in_seq=0 out_seq=0 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.317+0000 7efd2cff9700 10 client.27524 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd2cff9700 1 -- 172.21.15.26:0/329388214 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2971debe0 con 0x55a29727e210 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd2cff9700 10 client.27524 handle_command_reply: tid=0 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 client.27524 shutdown 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 msgr2=0x55a29727f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 0x55a29727f7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd40019fd0 tx=0x7efd400a9890 comp rx=0 tx=0).stop 2024-05-20T21:05:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 shutdown_connections 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55a29727e210 0x55a297308640 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29727d860 0x55a297280220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a2971ab080 0x55a29727f7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 --2- 172.21.15.26:0/329388214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2972779b0 0x55a29727fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 >> 172.21.15.26:0/329388214 conn(0x55a297274b00 msgr2=0x55a29727cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd2cff9700 0 client.27524 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 shutdown_connections 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 1 -- 172.21.15.26:0/329388214 wait complete. 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.318+0000 7efd560a6b80 20 client.27524 trim_cache size 0 max 16384 2024-05-20T21:05:05.322 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efd300b2880 msgr2=0x7efd300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efd300b2880 0x7efd300b4d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efd44079a60 tx=0x7efd34009260 comp rx=0 tx=0).stop 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 msgr2=0x7efd4419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 0x7efd4419a970 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efd4000eb30 tx=0x7efd4000ee40 comp rx=0 tx=0).stop 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 shutdown_connections 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efd300b2880 0x7efd300b4d40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd44102440 0x7efd4419a970 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd44108520 0x7efd441a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 --2- 172.21.15.26:0/2046780011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd44102e00 0x7efd4419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:05.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 >> 172.21.15.26:0/2046780011 conn(0x7efd440793f0 msgr2=0x7efd44100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:05.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 shutdown_connections 2024-05-20T21:05:05.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:05.319+0000 7efd4b810700 1 -- 172.21.15.26:0/2046780011 wait complete. 2024-05-20T21:05:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:05 smithi144 ceph-mon[35660]: pgmap v658: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 411 KiB/s wr, 56 op/s 2024-05-20T21:05:05.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 199585 2.32 MB/sec execute 311 sec latency 34.410 ms 2024-05-20T21:05:05.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:05 smithi026 ceph-mon[31990]: pgmap v658: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 411 KiB/s wr, 56 op/s 2024-05-20T21:05:05.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:05 smithi055 ceph-mon[35213]: pgmap v658: 97 pgs: 97 active+clean; 25 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 411 KiB/s wr, 56 op/s 2024-05-20T21:05:06.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:05:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:06.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 200233 2.32 MB/sec execute 312 sec latency 20.241 ms 2024-05-20T21:05:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:07 smithi144 ceph-mon[35660]: pgmap v659: 97 pgs: 97 active+clean; 27 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 389 KiB/s wr, 26 op/s 2024-05-20T21:05:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:07 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:07 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:07.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 200825 2.32 MB/sec execute 313 sec latency 21.417 ms 2024-05-20T21:05:07.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:07 smithi026 ceph-mon[31990]: pgmap v659: 97 pgs: 97 active+clean; 27 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 389 KiB/s wr, 26 op/s 2024-05-20T21:05:07.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:07.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:07.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:07 smithi055 ceph-mon[35213]: pgmap v659: 97 pgs: 97 active+clean; 27 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 389 KiB/s wr, 26 op/s 2024-05-20T21:05:07.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:07 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:07.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:07 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:08.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:05:08.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:05:08.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:08 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 -- 172.21.15.26:0/1631633045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4102720 msgr2=0x7f21f4102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1631633045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4102720 0x7f21f4102ba0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009990 tx=0x7f21e8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 -- 172.21.15.26:0/1631633045 shutdown_connections 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1631633045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1631633045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4107460 0x7f21f4107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1631633045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4102720 0x7f21f4102ba0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 -- 172.21.15.26:0/1631633045 >> 172.21.15.26:0/1631633045 conn(0x7f21f4079300 msgr2=0x7f21f4079710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 -- 172.21.15.26:0/1631633045 shutdown_connections 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.539+0000 7f21f8b74700 1 -- 172.21.15.26:0/1631633045 wait complete. 2024-05-20T21:05:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 Processor -- start 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 -- start start 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4102720 0x7f21f419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4107460 0x7f21f41a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4112650 con 0x7f21f4107460 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41124d0 con 0x7f21f41030e0 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.540+0000 7f21f8b74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41127d0 con 0x7f21f4102720 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4107460 0x7f21f41a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46486/0 (socket says 172.21.15.26:46486) 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 -- 172.21.15.26:0/1414015263 learned_addr learned my addr 172.21.15.26:0/1414015263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f259c700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4102720 0x7f21f419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 -- 172.21.15.26:0/1414015263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4102720 msgr2=0x7f21f419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4102720 0x7f21f419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 -- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4107460 msgr2=0x7f21f41a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4107460 0x7f21f41a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 -- 172.21.15.26:0/1414015263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41a6030 con 0x7f21f41030e0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f259c700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4102720 0x7f21f419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f419afb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21e80098e0 tx=0x7f21e80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21e37fe700 1 -- 172.21.15.26:0/1414015263 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8027070 con 0x7f21f41030e0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21e37fe700 1 -- 172.21.15.26:0/1414015263 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21e8022470 con 0x7f21f41030e0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21e37fe700 1 -- 172.21.15.26:0/1414015263 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8017650 con 0x7f21f41030e0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.541+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41a6200 con 0x7f21f41030e0 2024-05-20T21:05:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.542+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a6600 con 0x7f21f41030e0 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.544+0000 7f21e37fe700 1 -- 172.21.15.26:0/1414015263 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f21e802a070 con 0x7f21f41030e0 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.545+0000 7f21e37fe700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21dc0b2900 0x7f21dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.545+0000 7f21f259c700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21dc0b2900 0x7f21dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.545+0000 7f21e37fe700 1 -- 172.21.15.26:0/1414015263 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f21e80e31f0 con 0x7f21f41030e0 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.545+0000 7f220340ab80 1 Processor -- start 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.545+0000 7f21f259c700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21dc0b2900 0x7f21dc0b4dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f21e400bb90 tx=0x7f21e4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 -- start start 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e930902d70 con 0x55e9309ae610 2024-05-20T21:05:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e9308f2910 con 0x55e9309aefc0 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f220340ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e9308f6950 con 0x55e9309a92d0 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43278/0 (socket says 172.21.15.26:43278) 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 -- 172.21.15.26:0/1346545242 learned_addr learned my addr 172.21.15.26:0/1346545242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 -- 172.21.15.26:0/1346545242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 msgr2=0x55e9309a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 -- 172.21.15.26:0/1346545242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 msgr2=0x55e9309a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f259c700 1 -- 172.21.15.26:0/1346545242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93089bfb0 con 0x55e9309ae610 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.546+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.547+0000 7f21f259c700 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ae9f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f21e400b970 tx=0x7f21e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9b9c47f9f465dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.547+0000 7f21e0ff9700 1 -- 172.21.15.26:0/1346545242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4022070 con 0x55e9309ae610 2024-05-20T21:05:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.547+0000 7f21e0ff9700 1 -- 172.21.15.26:0/1346545242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21e401e450 con 0x55e9309ae610 2024-05-20T21:05:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.547+0000 7f21e0ff9700 1 -- 172.21.15.26:0/1346545242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e401d610 con 0x55e9309ae610 2024-05-20T21:05:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 -- 172.21.15.26:0/1346545242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 msgr2=0x55e9309ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ae9f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f21e400b970 tx=0x7f21e4012000 comp rx=0 tx=0).stop 2024-05-20T21:05:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 -- 172.21.15.26:0/1346545242 shutdown_connections 2024-05-20T21:05:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ae9f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 --2- 172.21.15.26:0/1346545242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 -- 172.21.15.26:0/1346545242 >> 172.21.15.26:0/1346545242 conn(0x55e9308d0240 msgr2=0x55e93099f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 -- 172.21.15.26:0/1346545242 shutdown_connections 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 -- 172.21.15.26:0/1346545242 wait complete. 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.548+0000 7f220340ab80 1 -- start start 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 Processor -- start 2024-05-20T21:05:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e9307f8fa0 con 0x55e9309ae610 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e93085ada0 con 0x55e9309aefc0 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f220340ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e9309b5aa0 con 0x55e9309a92d0 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43282/0 (socket says 172.21.15.26:43282) 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f21f1d9b700 1 -- 172.21.15.26:0/2501352498 learned_addr learned my addr 172.21.15.26:0/2501352498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f21f259c700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.549+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f220340ab80 10 client.0 _ll_register_callbacks cb 0x55e93088d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f1d9b700 1 -- 172.21.15.26:0/2501352498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 msgr2=0x55e9309b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f1d9b700 1 -- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 msgr2=0x55e9309b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f1d9b700 1 -- 172.21.15.26:0/2501352498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9308428d0 con 0x55e9309ae610 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309b20f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f21e80e1000 tx=0x7f21e80aa4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8805f3705fad8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2501352498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80aebf0 con 0x55e9309ae610 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2501352498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f21e80cd0b0 con 0x55e9309ae610 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e401e450 con 0x55e9309ae610 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2501352498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f21e80b3610 con 0x55e9309ae610 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f220340ab80 10 client.18141 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.550+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e9308ac760 con 0x55e9309ae610 2024-05-20T21:05:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2501352498 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21e80aaaa0 con 0x55e9309ae610 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2501352498 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f21e80a9aa0 con 0x55e9309ae610 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f220340ab80 10 client.18141 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f220340ab80 10 client.18141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f220340ab80 20 client.18141 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f220340ab80 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e9309c2680 0x55e9309c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f220340ab80 4 client.18141 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f21e401e450 con 0x55e9309c2680 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.551+0000 7f21f259c700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e9309c2680 0x55e9309c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.552+0000 7f21d9ffb700 10 client.18141 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:05:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.552+0000 7f21f259c700 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e9309c2680 0x55e9309c4aa0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0e836f9f7459b9 server_cookie=dea7fb30524219 in_seq=0 out_seq=0 2024-05-20T21:05:08.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:05:08.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:05:08.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:08 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.552+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2501352498 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f21e401e450 con 0x55e9309c2680 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.552+0000 7f21d9ffb700 10 client.18141 handle_command_reply: tid=0 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.552+0000 7f220340ab80 1 client.18141 shutdown 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 msgr2=0x55e9309b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309b20f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f21e80e1000 tx=0x7f21e80aa4a0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 shutdown_connections 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e9309c2680 0x55e9309c4aa0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309aefc0 0x55e9309b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 --2- 172.21.15.26:0/2501352498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309b20f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f21d9ffb700 0 client.18141 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 >> 172.21.15.26:0/2501352498 conn(0x55e9308d0240 msgr2=0x55e93099eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 shutdown_connections 2024-05-20T21:05:08.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 1 -- 172.21.15.26:0/2501352498 wait complete. 2024-05-20T21:05:08.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.553+0000 7f220340ab80 20 client.18141 trim_cache size 0 max 16384 2024-05-20T21:05:08.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 201522 2.33 MB/sec execute 314 sec latency 23.859 ms 2024-05-20T21:05:08.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.572+0000 7f220340ab80 1 Processor -- start 2024-05-20T21:05:08.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.572+0000 7f220340ab80 1 -- start start 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f220340ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309a92d0 0x55e9309beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f220340ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f220340ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309aefc0 0x55e9309afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f220340ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ae310 con 0x55e9309ae610 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f220340ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41abba0 con 0x55e9309a92d0 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f220340ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41a99f0 con 0x55e9309aefc0 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43294/0 (socket says 172.21.15.26:43294) 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f1d9b700 1 -- 172.21.15.26:0/4008829719 learned_addr learned my addr 172.21.15.26:0/4008829719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f259c700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309a92d0 0x55e9309beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f259c700 1 -- 172.21.15.26:0/4008829719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309aefc0 msgr2=0x55e9309afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309aefc0 0x55e9309afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f259c700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309aefc0 0x55e9309afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f259c700 1 -- 172.21.15.26:0/4008829719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 msgr2=0x55e9309ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f259c700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f259c700 1 -- 172.21.15.26:0/4008829719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9309b0660 con 0x55e9309a92d0 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f1d9b700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.573+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309aefc0 0x55e9309afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.574+0000 7f21f259c700 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309a92d0 0x55e9309beb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f21e400f640 tx=0x7f21e4015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64fd3cdcc790fde server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.574+0000 7f21dbfff700 1 -- 172.21.15.26:0/4008829719 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4022040 con 0x55e9309a92d0 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.574+0000 7f21dbfff700 1 -- 172.21.15.26:0/4008829719 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21e4011ce0 con 0x55e9309a92d0 2024-05-20T21:05:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.574+0000 7f21dbfff700 1 -- 172.21.15.26:0/4008829719 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4005460 con 0x55e9309a92d0 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- 172.21.15.26:0/4008829719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309a92d0 msgr2=0x55e9309beb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309a92d0 0x55e9309beb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f21e400f640 tx=0x7f21e4015000 comp rx=0 tx=0).stop 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- 172.21.15.26:0/4008829719 shutdown_connections 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309aefc0 0x55e9309afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309a92d0 0x55e9309beb40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- 172.21.15.26:0/4008829719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309ae610 0x55e9309ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- 172.21.15.26:0/4008829719 >> 172.21.15.26:0/4008829719 conn(0x55e9308d0240 msgr2=0x55e9309ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- 172.21.15.26:0/4008829719 shutdown_connections 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- 172.21.15.26:0/4008829719 wait complete. 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- start start 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 Processor -- start 2024-05-20T21:05:08.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309ae610 0x55e9309c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 0x55e9309ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41aa060 con 0x55e9309aefc0 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ab4d0 con 0x55e9309ae610 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.575+0000 7f220340ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41aa730 con 0x55e9309a92d0 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f220340ab80 10 client.0 _ll_register_callbacks cb 0x55e93088d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 0x55e9309ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309ae610 0x55e9309c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309ae610 0x55e9309c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46540/0 (socket says 172.21.15.26:46540) 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 0x55e9309ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43300/0 (socket says 172.21.15.26:43300) 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f1d9b700 1 -- 172.21.15.26:0/2444611581 learned_addr learned my addr 172.21.15.26:0/2444611581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f259c700 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 -- 172.21.15.26:0/2444611581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 msgr2=0x55e9309bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 -- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309ae610 msgr2=0x55e9309c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309ae610 0x55e9309c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 -- 172.21.15.26:0/2444611581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9309ca8d0 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21f2d9d700 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 0x55e9309ca0c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x55e9309b3370 tx=0x7f21ec007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8d82303689f49a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2444611581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec003c00 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4011ce0 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.576+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2444611581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f21ec003da0 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2444611581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f21ec028050 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 10 client.18147 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e930a2d230 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2444611581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21e4011ce0 con 0x55e9309aefc0 2024-05-20T21:05:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2444611581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f21ec004650 con 0x55e9309aefc0 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 10 client.18147 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 10 client.18147 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 20 client.18147 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e930a2d8f0 0x55e930a2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 4 client.18147 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e930904cc0 con 0x55e930a2d8f0 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.577+0000 7f21f259c700 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e930a2d8f0 0x55e930a2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f21f259c700 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e930a2d8f0 0x55e930a2fd10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ced6e105bf87a6 server_cookie=fb0a1ba53d1581fb in_seq=0 out_seq=0 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f21d9ffb700 10 client.18147 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f21d9ffb700 1 -- 172.21.15.26:0/2444611581 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e930904cc0 con 0x55e930a2d8f0 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f21d9ffb700 10 client.18147 handle_command_reply: tid=0 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 client.18147 shutdown 2024-05-20T21:05:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 msgr2=0x55e9309ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 0x55e9309ca0c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x55e9309b3370 tx=0x7f21ec007dc0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 shutdown_connections 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55e930a2d8f0 0x55e930a2fd10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e9309aefc0 0x55e9309ca0c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9309a92d0 0x55e9309bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f21d9ffb700 0 client.18147 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 --2- 172.21.15.26:0/2444611581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e9309ae610 0x55e9309c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 >> 172.21.15.26:0/2444611581 conn(0x55e9308d0240 msgr2=0x55e9309a4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.578+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 shutdown_connections 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f220340ab80 1 -- 172.21.15.26:0/2444611581 wait complete. 2024-05-20T21:05:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f220340ab80 20 client.18147 trim_cache size 0 max 16384 2024-05-20T21:05:08.583 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:05:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21dc0b2900 msgr2=0x7f21dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21dc0b2900 0x7f21dc0b4dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f21e400bb90 tx=0x7f21e4005d80 comp rx=0 tx=0).stop 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 msgr2=0x7f21f419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f419afb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21e80098e0 tx=0x7f21e80057c0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 shutdown_connections 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21dc0b2900 0x7f21dc0b4dc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f4107460 0x7f21f41a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.579+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21f4102720 0x7f21f419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.580+0000 7f21f8b74700 1 --2- 172.21.15.26:0/1414015263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21f41030e0 0x7f21f419afb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.580+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 >> 172.21.15.26:0/1414015263 conn(0x7f21f4079300 msgr2=0x7f21f4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.580+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 shutdown_connections 2024-05-20T21:05:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:08.580+0000 7f21f8b74700 1 -- 172.21.15.26:0/1414015263 wait complete. 2024-05-20T21:05:08.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:05:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:05:08.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:08 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:05:09.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:05:09.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:09 smithi144 ceph-mon[35660]: pgmap v660: 97 pgs: 97 active+clean; 28 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 459 KiB/s wr, 48 op/s 2024-05-20T21:05:09.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 202132 2.34 MB/sec execute 315 sec latency 17.626 ms 2024-05-20T21:05:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:09.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:09 smithi055 ceph-mon[35213]: pgmap v660: 97 pgs: 97 active+clean; 28 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 459 KiB/s wr, 48 op/s 2024-05-20T21:05:09.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:09 smithi026 ceph-mon[31990]: pgmap v660: 97 pgs: 97 active+clean; 28 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 459 KiB/s wr, 48 op/s 2024-05-20T21:05:10.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 202637 2.34 MB/sec execute 316 sec latency 11.395 ms 2024-05-20T21:05:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 -- 172.21.15.26:0/2509645022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 msgr2=0x7ff868101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 --2- 172.21.15.26:0/2509645022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff868101dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff858009900 tx=0x7ff85802a410 comp rx=0 tx=0).stop 2024-05-20T21:05:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 -- 172.21.15.26:0/2509645022 shutdown_connections 2024-05-20T21:05:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 --2- 172.21.15.26:0/2509645022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff868103560 0x7ff868109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 --2- 172.21.15.26:0/2509645022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff868103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 --2- 172.21.15.26:0/2509645022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff868101dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 -- 172.21.15.26:0/2509645022 >> 172.21.15.26:0/2509645022 conn(0x7ff8680fcf20 msgr2=0x7ff8680ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 -- 172.21.15.26:0/2509645022 shutdown_connections 2024-05-20T21:05:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.001+0000 7ff86cdda640 1 -- 172.21.15.26:0/2509645022 wait complete. 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 Processor -- start 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 -- start start 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff86806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff86819d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff868103560 0x7ff86819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff86810d360 con 0x7ff868102ba0 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff86810d1e0 con 0x7ff8681019b0 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86cdda640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff86810d4e0 con 0x7ff868103560 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff86819d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff86659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff86806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff86819d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37332/0 (socket says 172.21.15.26:37332) 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 -- 172.21.15.26:0/4149498072 learned_addr learned my addr 172.21.15.26:0/4149498072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff866d9d640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff868103560 0x7ff86819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 -- 172.21.15.26:0/4149498072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff868103560 msgr2=0x7ff86819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff868103560 0x7ff86819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 -- 172.21.15.26:0/4149498072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 msgr2=0x7ff86806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.002+0000 7ff865d9b640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff86806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff865d9b640 1 -- 172.21.15.26:0/4149498072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681a07c0 con 0x7ff868102ba0 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff86659c640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff86806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff865d9b640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff86819d8c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e9f0 tx=0x7ff85c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff8577fe640 1 -- 172.21.15.26:0/4149498072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00cd80 con 0x7ff868102ba0 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681a0ab0 con 0x7ff868102ba0 2024-05-20T21:05:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff8577fe640 1 -- 172.21.15.26:0/4149498072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff85c0108c0 con 0x7ff868102ba0 2024-05-20T21:05:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff8577fe640 1 -- 172.21.15.26:0/4149498072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0187c0 con 0x7ff868102ba0 2024-05-20T21:05:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.003+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681a1050 con 0x7ff868102ba0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.005+0000 7ff8577fe640 1 -- 172.21.15.26:0/4149498072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff85c010430 con 0x7ff868102ba0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 Processor -- start 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 -- start start 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e2bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e30fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e33ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563318d98c30 con 0x563318e2c4c0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563318d887d0 con 0x563318e31630 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563318d8c810 con 0x563318e2bad0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e30fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff866d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e2bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff865d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e33ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e30fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37340/0 (socket says 172.21.15.26:37340) 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.006+0000 7ff86659c640 1 -- 172.21.15.26:0/3768506971 learned_addr learned my addr 172.21.15.26:0/3768506971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff866d9d640 1 -- 172.21.15.26:0/3768506971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 msgr2=0x563318e33ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff866d9d640 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e33ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff866d9d640 1 -- 172.21.15.26:0/3768506971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 msgr2=0x563318e30fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff866d9d640 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e30fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff866d9d640 1 -- 172.21.15.26:0/3768506971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563318a9d210 con 0x563318e2bad0 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff8577fe640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8500b28a0 0x7ff8500b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff8577fe640 1 -- 172.21.15.26:0/4149498072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff85c0e2600 con 0x7ff868102ba0 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff866d9d640 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e2bef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff860009f30 tx=0x7ff860010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54274eb2f0a3b5a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff86659c640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8500b28a0 0x7ff8500b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff854ff9640 1 -- 172.21.15.26:0/3768506971 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860016070 con 0x563318e2bad0 2024-05-20T21:05:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff854ff9640 1 -- 172.21.15.26:0/3768506971 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff860004d20 con 0x563318e2bad0 2024-05-20T21:05:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.007+0000 7ff854ff9640 1 -- 172.21.15.26:0/3768506971 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8600126b0 con 0x563318e2bad0 2024-05-20T21:05:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3768506971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 msgr2=0x563318e2bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86659c640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8500b28a0 0x7ff8500b4d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x563318e2c8c0 tx=0x7ff858031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e2bef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff860009f30 tx=0x7ff860010800 comp rx=0 tx=0).stop 2024-05-20T21:05:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3768506971 shutdown_connections 2024-05-20T21:05:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e33ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e30fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3768506971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e2bef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3768506971 >> 172.21.15.26:0/3768506971 conn(0x563318e22d30 msgr2=0x563318e23130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3768506971 shutdown_connections 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.008+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3768506971 wait complete. 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 -- start start 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 Processor -- start 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e2c4c0 0x563318e36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e31630 0x563318e3d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563318a6e790 con 0x563318e2bad0 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563318d44890 con 0x563318e2c4c0 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563318e345e0 con 0x563318e31630 2024-05-20T21:05:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86fbc6740 10 client.0 _ll_register_callbacks cb 0x563318a9ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e35c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37348/0 (socket says 172.21.15.26:37348) 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 -- 172.21.15.26:0/3613364383 learned_addr learned my addr 172.21.15.26:0/3613364383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e2c4c0 0x563318e36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff866d9d640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e31630 0x563318e3d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 -- 172.21.15.26:0/3613364383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e31630 msgr2=0x563318e3d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e31630 0x563318e3d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 -- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e2c4c0 msgr2=0x563318e36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e2c4c0 0x563318e36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff86659c640 1 -- 172.21.15.26:0/3613364383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563318b69620 con 0x563318e2bad0 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.009+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e2c4c0 0x563318e36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff86659c640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e35c50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff85802a930 tx=0x7ff858009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d23816eb7ac45e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff84dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff84dffb640 1 -- 172.21.15.26:0/3613364383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858035040 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff84dffb640 1 -- 172.21.15.26:0/3613364383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff8580035f0 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff860004d20 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff84dffb640 1 -- 172.21.15.26:0/3613364383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8580037f0 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff86fbc6740 10 client.18159 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563318e40220 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff84dffb640 1 -- 172.21.15.26:0/3613364383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff860004d20 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.010+0000 7ff84dffb640 1 -- 172.21.15.26:0/3613364383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff858044070 con 0x563318e2bad0 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff86fbc6740 10 client.18159 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff86fbc6740 10 client.18159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff86fbc6740 20 client.18159 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318e41960 0x563318e43d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff86fbc6740 4 client.18159 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563318d9ab80 con 0x563318e41960 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318e41960 0x563318e43d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318e41960 0x563318e43d70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd4faa4c7d5b08e server_cookie=deb330b391bba4f5 in_seq=0 out_seq=0 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.011+0000 7ff84dffb640 10 client.18159 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff84dffb640 1 -- 172.21.15.26:0/3613364383 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563318d9ab80 con 0x563318e41960 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff84dffb640 10 client.18159 handle_command_reply: tid=0 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 client.18159 shutdown 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 msgr2=0x563318e35c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e35c50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff85802a930 tx=0x7ff858009260 comp rx=0 tx=0).stop 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 shutdown_connections 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318e41960 0x563318e43d70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e31630 0x563318e3d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e2c4c0 0x563318e36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3613364383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e35c50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 >> 172.21.15.26:0/3613364383 conn(0x563318e22d30 msgr2=0x563318e21fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff84dffb640 0 client.18159 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 shutdown_connections 2024-05-20T21:05:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.013+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3613364383 wait complete. 2024-05-20T21:05:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.014+0000 7ff86fbc6740 20 client.18159 trim_cache size 0 max 16384 2024-05-20T21:05:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.031+0000 7ff86fbc6740 1 Processor -- start 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.031+0000 7ff86fbc6740 1 -- start start 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e26520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e3d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681aa640 con 0x563318e2c4c0 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681a7f10 con 0x563318e31630 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681a5d60 con 0x563318e2bad0 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e3d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e3d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37362/0 (socket says 172.21.15.26:37362) 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 -- 172.21.15.26:0/3774220942 learned_addr learned my addr 172.21.15.26:0/3774220942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff866d9d640 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff86659c640 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e26520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 -- 172.21.15.26:0/3774220942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 msgr2=0x563318e26520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e26520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 -- 172.21.15.26:0/3774220942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 msgr2=0x563318e35f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e35f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 -- 172.21.15.26:0/3774220942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563318e364b0 con 0x563318e2c4c0 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff865d9b640 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e3d190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e9c0 tx=0x7ff85c0e4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ee63e002073104 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff84ffff640 1 -- 172.21.15.26:0/3774220942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c010720 con 0x563318e2c4c0 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff84ffff640 1 -- 172.21.15.26:0/3774220942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff85c0b2360 con 0x563318e2c4c0 2024-05-20T21:05:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.032+0000 7ff84ffff640 1 -- 172.21.15.26:0/3774220942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0a5400 con 0x563318e2c4c0 2024-05-20T21:05:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3774220942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 msgr2=0x563318e3d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e3d190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e9c0 tx=0x7ff85c0e4040 comp rx=0 tx=0).stop 2024-05-20T21:05:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3774220942 shutdown_connections 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e35f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2c4c0 0x563318e3d190 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/3774220942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2bad0 0x563318e26520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3774220942 >> 172.21.15.26:0/3774220942 conn(0x563318e22d30 msgr2=0x563318e35860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3774220942 shutdown_connections 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.033+0000 7ff86fbc6740 1 -- 172.21.15.26:0/3774220942 wait complete. 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 -- start start 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 Processor -- start 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e3a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2c4c0 0x563318e3ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e3bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681a63d0 con 0x563318e2bad0 2024-05-20T21:05:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681a7840 con 0x563318e31630 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681a6aa0 con 0x563318e2c4c0 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86fbc6740 10 client.0 _ll_register_callbacks cb 0x7ff858032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e3a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e3a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37374/0 (socket says 172.21.15.26:37374) 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 -- 172.21.15.26:0/1247842584 learned_addr learned my addr 172.21.15.26:0/1247842584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff866d9d640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e3bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff865d9b640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2c4c0 0x563318e3ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 -- 172.21.15.26:0/1247842584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2c4c0 msgr2=0x563318e3ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2c4c0 0x563318e3ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 -- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 msgr2=0x563318e3bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e3bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.034+0000 7ff86659c640 1 -- 172.21.15.26:0/1247842584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563318e3c0c0 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86659c640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e3a5d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff85803dcb0 tx=0x7ff858003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ac99ae2c8bd086 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff84dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff84dffb640 1 -- 172.21.15.26:0/1247842584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85805e030 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563318d94360 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff84dffb640 1 -- 172.21.15.26:0/1247842584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff858003990 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff84dffb640 1 -- 172.21.15.26:0/1247842584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff858035050 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 10 client.18165 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563318e3d440 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff84dffb640 1 -- 172.21.15.26:0/1247842584 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563318d94360 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff84dffb640 1 -- 172.21.15.26:0/1247842584 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff858044040 con 0x563318e2bad0 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 10 client.18165 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 10 client.18165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 20 client.18165 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318eacc40 0x563318eaf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 4 client.18165 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:05:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.035+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563318d0e5f0 con 0x563318eacc40 2024-05-20T21:05:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.036+0000 7ff865d9b640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318eacc40 0x563318eaf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.036+0000 7ff84dffb640 10 client.18165 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.036+0000 7ff865d9b640 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318eacc40 0x563318eaf060 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e2f6311799fb1b server_cookie=c7258e40c8a97ce8 in_seq=0 out_seq=0 2024-05-20T21:05:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff84dffb640 1 -- 172.21.15.26:0/1247842584 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563318d0e5f0 con 0x563318eacc40 2024-05-20T21:05:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff84dffb640 10 client.18165 handle_command_reply: tid=0 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 client.18165 shutdown 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 msgr2=0x563318e3a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e3a5d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff85803dcb0 tx=0x7ff858003670 comp rx=0 tx=0).stop 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 shutdown_connections 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563318eacc40 0x563318eaf060 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563318e31630 0x563318e3bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563318e2c4c0 0x563318e3ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 --2- 172.21.15.26:0/1247842584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563318e2bad0 0x563318e3a5d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 >> 172.21.15.26:0/1247842584 conn(0x563318e22d30 msgr2=0x563318e21720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff84dffb640 0 client.18165 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 shutdown_connections 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 1 -- 172.21.15.26:0/1247842584 wait complete. 2024-05-20T21:05:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.080+0000 7ff86fbc6740 20 client.18165 trim_cache size 0 max 16384 2024-05-20T21:05:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:05:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:05:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8500b28a0 msgr2=0x7ff8500b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8500b28a0 0x7ff8500b4d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x563318e2c8c0 tx=0x7ff858031400 comp rx=0 tx=0).stop 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 msgr2=0x7ff86819d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff86819d8c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e9f0 tx=0x7ff85c00eeb0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 shutdown_connections 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8500b28a0 0x7ff8500b4d60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff868103560 0x7ff86819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff868102ba0 0x7ff86819d8c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 --2- 172.21.15.26:0/4149498072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8681019b0 0x7ff86806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 >> 172.21.15.26:0/4149498072 conn(0x7ff8680fcf20 msgr2=0x7ff868106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 shutdown_connections 2024-05-20T21:05:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:11.081+0000 7ff86cdda640 1 -- 172.21.15.26:0/4149498072 wait complete. 2024-05-20T21:05:11.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:11 smithi026 ceph-mon[31990]: pgmap v661: 97 pgs: 97 active+clean; 29 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 693 KiB/s wr, 50 op/s 2024-05-20T21:05:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:11 smithi144 ceph-mon[35660]: pgmap v661: 97 pgs: 97 active+clean; 29 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 693 KiB/s wr, 50 op/s 2024-05-20T21:05:11.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 203132 2.33 MB/sec execute 317 sec latency 32.903 ms 2024-05-20T21:05:11.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 -- 172.21.15.26:0/1095078247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4102e00 msgr2=0x7f12e410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 --2- 172.21.15.26:0/1095078247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4102e00 0x7f12e410f2f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00b0a0 tx=0x7f12dc00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 -- 172.21.15.26:0/1095078247 shutdown_connections 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 --2- 172.21.15.26:0/1095078247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4102e00 0x7f12e410f2f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 --2- 172.21.15.26:0/1095078247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4108520 0x7f12e4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 --2- 172.21.15.26:0/1095078247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4102440 0x7f12e41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 -- 172.21.15.26:0/1095078247 >> 172.21.15.26:0/1095078247 conn(0x7f12e40793f0 msgr2=0x7f12e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 -- 172.21.15.26:0/1095078247 shutdown_connections 2024-05-20T21:05:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.664+0000 7f12ebb25700 1 -- 172.21.15.26:0/1095078247 wait complete. 2024-05-20T21:05:11.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 Processor -- start 2024-05-20T21:05:11.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 -- start start 2024-05-20T21:05:11.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 0x7f12e419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4102e00 0x7f12e419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4108520 0x7f12e4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41124a0 con 0x7f12e4108520 2024-05-20T21:05:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4112320 con 0x7f12e4102440 2024-05-20T21:05:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.665+0000 7f12ebb25700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4112620 con 0x7f12e4102e00 2024-05-20T21:05:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 0x7f12e419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 0x7f12e419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60004/0 (socket says 172.21.15.26:60004) 2024-05-20T21:05:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 -- 172.21.15.26:0/3930620906 learned_addr learned my addr 172.21.15.26:0/3930620906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12ea0c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4108520 0x7f12e4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e90c0700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4102e00 0x7f12e419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 -- 172.21.15.26:0/3930620906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4102e00 msgr2=0x7f12e419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4102e00 0x7f12e419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 -- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4108520 msgr2=0x7f12e4199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4108520 0x7f12e4199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 -- 172.21.15.26:0/3930620906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e419a080 con 0x7f12e4102440 2024-05-20T21:05:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12e98c1700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 0x7f12e419f710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f12e000e9a0 tx=0x7f12e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12daffd700 1 -- 172.21.15.26:0/3930620906 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e000ca10 con 0x7f12e4102440 2024-05-20T21:05:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.666+0000 7f12daffd700 1 -- 172.21.15.26:0/3930620906 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12e000cbb0 con 0x7f12e4102440 2024-05-20T21:05:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.667+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e419a310 con 0x7f12e4102440 2024-05-20T21:05:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.667+0000 7f12daffd700 1 -- 172.21.15.26:0/3930620906 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0018680 con 0x7f12e4102440 2024-05-20T21:05:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.667+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e4103fb0 con 0x7f12e4102440 2024-05-20T21:05:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.670+0000 7f12daffd700 1 -- 172.21.15.26:0/3930620906 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f12e0010430 con 0x7f12e4102440 2024-05-20T21:05:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.670+0000 7f12daffd700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f12d00b2880 0x7f12d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.670+0000 7f12e90c0700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f12d00b2880 0x7f12d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12daffd700 1 -- 172.21.15.26:0/3930620906 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f12e00e23b0 con 0x7f12e4102440 2024-05-20T21:05:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 Processor -- start 2024-05-20T21:05:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12e90c0700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f12d00b2880 0x7f12d00b4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f12e4079ba0 tx=0x7f12d400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 -- start start 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48e860 0x562a8b48ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b492940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b4933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a8b3edc90 con 0x562a8b48f210 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a8b3dd830 con 0x562a8b48e860 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.671+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a8b3e1870 con 0x562a8b492f60 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b492940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b492940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37404/0 (socket says 172.21.15.26:37404) 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 -- 172.21.15.26:0/1989487565 learned_addr learned my addr 172.21.15.26:0/1989487565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48e860 0x562a8b48ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b4933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 -- 172.21.15.26:0/1989487565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 msgr2=0x562a8b4933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b4933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 -- 172.21.15.26:0/1989487565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48e860 msgr2=0x562a8b48ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48e860 0x562a8b48ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 -- 172.21.15.26:0/1989487565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8b32d8d0 con 0x562a8b48f210 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b4933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b492940 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f12d400b670 tx=0x7f12d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd396e92d632a2e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.672+0000 7f12cffff700 1 -- 172.21.15.26:0/1989487565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d40118d0 con 0x562a8b48f210 2024-05-20T21:05:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12cffff700 1 -- 172.21.15.26:0/1989487565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d4004830 con 0x562a8b48f210 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12cffff700 1 -- 172.21.15.26:0/1989487565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4005350 con 0x562a8b48f210 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1989487565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 msgr2=0x562a8b492940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b492940 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f12d400b670 tx=0x7f12d4013000 comp rx=0 tx=0).stop 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1989487565 shutdown_connections 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b4933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48e860 0x562a8b48ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1989487565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b492940 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.673+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1989487565 >> 172.21.15.26:0/1989487565 conn(0x562a8b3bc080 msgr2=0x562a8b488e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.674+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1989487565 shutdown_connections 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.674+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1989487565 wait complete. 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.674+0000 7f12f63bbb80 1 -- start start 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.674+0000 7f12f63bbb80 1 Processor -- start 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b49a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b497670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 0x562a8b497bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a8b2e3fa0 con 0x562a8b492f60 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a8b37f7c0 con 0x562a8b48f210 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a8b49b120 con 0x562a8b48e860 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12f63bbb80 10 client.0 _ll_register_callbacks cb 0x562a8b378920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 0x562a8b497bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 0x562a8b497bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37414/0 (socket says 172.21.15.26:37414) 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 -- 172.21.15.26:0/1888507612 learned_addr learned my addr 172.21.15.26:0/1888507612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b497670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b49a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 -- 172.21.15.26:0/1888507612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 msgr2=0x562a8b49a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b49a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 -- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 msgr2=0x562a8b497670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b497670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12ea0c2700 1 -- 172.21.15.26:0/1888507612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8b37c690 con 0x562a8b492f60 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.675+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b49a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 0x562a8b497bb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00bd90 tx=0x7f12dc00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bad9f1cd8ca1a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1888507612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc02c060 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d4004830 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1888507612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f12dc01c6c0 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1888507612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12dc01c8c0 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12f63bbb80 10 client.18177 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a8b4a5860 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.676+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1888507612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d4004830 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1888507612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f12dc030030 con 0x562a8b492f60 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12f63bbb80 10 client.18177 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12f63bbb80 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12f63bbb80 20 client.18177 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b4a7df0 0x562a8b4aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12f63bbb80 4 client.18177 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a8b45a9e0 con 0x562a8b4a7df0 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b4a7df0 0x562a8b4aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12cd7fa700 10 client.18177 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.677+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b4a7df0 0x562a8b4aa210 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb6af5e7e8c6ae5 server_cookie=a494ba372e4aaee8 in_seq=0 out_seq=0 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1888507612 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562a8b45a9e0 con 0x562a8b4a7df0 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12cd7fa700 10 client.18177 handle_command_reply: tid=0 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 client.18177 shutdown 2024-05-20T21:05:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 msgr2=0x562a8b497bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 0x562a8b497bb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00bd90 tx=0x7f12dc00bdc0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 shutdown_connections 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b4a7df0 0x562a8b4aa210 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b492f60 0x562a8b497bb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b49a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1888507612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b497670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12cd7fa700 0 client.18177 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 >> 172.21.15.26:0/1888507612 conn(0x562a8b3bc080 msgr2=0x562a8b488510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.678+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 shutdown_connections 2024-05-20T21:05:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.679+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1888507612 wait complete. 2024-05-20T21:05:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.679+0000 7f12f63bbb80 20 client.18177 trim_cache size 0 max 16384 2024-05-20T21:05:11.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.696+0000 7f12f63bbb80 1 Processor -- start 2024-05-20T21:05:11.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.696+0000 7f12f63bbb80 1 -- start start 2024-05-20T21:05:11.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b491fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b4afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b492f60 0x562a8b4b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41ae350 con 0x562a8b48f210 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41abbe0 con 0x562a8b492f60 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41a9a30 con 0x562a8b48e860 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e90c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b4afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b491fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b491fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38044/0 (socket says 172.21.15.26:38044) 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e90c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b4afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37420/0 (socket says 172.21.15.26:37420) 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 -- 172.21.15.26:0/738295315 learned_addr learned my addr 172.21.15.26:0/738295315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b492f60 0x562a8b4b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 -- 172.21.15.26:0/738295315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b492f60 msgr2=0x562a8b4b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b492f60 0x562a8b4b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 -- 172.21.15.26:0/738295315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 msgr2=0x562a8b4afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b4afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 -- 172.21.15.26:0/738295315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8b495c20 con 0x562a8b48e860 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e90c0700 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b4afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12e98c1700 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b491fc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12e00a59f0 tx=0x7f12e001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5b5a81c5b972d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.697+0000 7f12cf7fe700 1 -- 172.21.15.26:0/738295315 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00aa070 con 0x562a8b48e860 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12cf7fe700 1 -- 172.21.15.26:0/738295315 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12e0010e50 con 0x562a8b48e860 2024-05-20T21:05:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12cf7fe700 1 -- 172.21.15.26:0/738295315 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00a9bd0 con 0x562a8b48e860 2024-05-20T21:05:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 -- 172.21.15.26:0/738295315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 msgr2=0x562a8b491fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b491fc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12e00a59f0 tx=0x7f12e001a040 comp rx=0 tx=0).stop 2024-05-20T21:05:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 -- 172.21.15.26:0/738295315 shutdown_connections 2024-05-20T21:05:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b492f60 0x562a8b4b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b48e860 0x562a8b491fc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/738295315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48f210 0x562a8b4afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 -- 172.21.15.26:0/738295315 >> 172.21.15.26:0/738295315 conn(0x562a8b3bc080 msgr2=0x562a8b4918b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 -- 172.21.15.26:0/738295315 shutdown_connections 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.698+0000 7f12f63bbb80 1 -- 172.21.15.26:0/738295315 wait complete. 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 -- start start 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 Processor -- start 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 0x562a8b4a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b49dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b49e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41aa0a0 con 0x562a8b48e860 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41ab510 con 0x562a8b48f210 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41aa770 con 0x562a8b492f60 2024-05-20T21:05:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12f63bbb80 10 client.0 _ll_register_callbacks cb 0x562a8b378920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12e98c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 0x562a8b4a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12e98c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 0x562a8b4a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37422/0 (socket says 172.21.15.26:37422) 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 -- 172.21.15.26:0/1213780634 learned_addr learned my addr 172.21.15.26:0/1213780634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.699+0000 7f12e90c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b49dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12ea0c2700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b49e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 -- 172.21.15.26:0/1213780634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 msgr2=0x562a8b49e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b49e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 -- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 msgr2=0x562a8b49dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b49dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 -- 172.21.15.26:0/1213780634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8b49ea20 con 0x562a8b48e860 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b49dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12e98c1700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 0x562a8b4a09f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f12e00dfea0 tx=0x7f12e00a9870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8565e12fb1db4c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1213780634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00c3060 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1213780634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f12e00a9de0 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e0010e50 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1213780634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12e00add20 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12f63bbb80 10 client.18183 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.700+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a8b512950 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1213780634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12e0010e50 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1213780634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f12e00b2340 con 0x562a8b48e860 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12f63bbb80 10 client.18183 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12f63bbb80 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12f63bbb80 20 client.18183 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b512f50 0x562a8b515370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12f63bbb80 4 client.18183 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:05:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a8b3efbe0 con 0x562a8b512f50 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.701+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b512f50 0x562a8b515370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12cd7fa700 10 client.18183 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12e90c0700 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b512f50 0x562a8b515370 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e85ad633a607769 server_cookie=6bba368dedc025cb in_seq=0 out_seq=0 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12cd7fa700 1 -- 172.21.15.26:0/1213780634 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a8b3efbe0 con 0x562a8b512f50 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12cd7fa700 10 client.18183 handle_command_reply: tid=0 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 client.18183 shutdown 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 msgr2=0x562a8b4a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 0x562a8b4a09f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f12e00dfea0 tx=0x7f12e00a9870 comp rx=0 tx=0).stop 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 shutdown_connections 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562a8b512f50 0x562a8b515370 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a8b48e860 0x562a8b4a09f0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a8b492f60 0x562a8b49e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 --2- 172.21.15.26:0/1213780634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a8b48f210 0x562a8b49dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 >> 172.21.15.26:0/1213780634 conn(0x562a8b3bc080 msgr2=0x562a8b492140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12cd7fa700 0 client.18183 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.702+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 shutdown_connections 2024-05-20T21:05:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12f63bbb80 1 -- 172.21.15.26:0/1213780634 wait complete. 2024-05-20T21:05:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12f63bbb80 20 client.18183 trim_cache size 0 max 16384 2024-05-20T21:05:11.709 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:05:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f12d00b2880 msgr2=0x7f12d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f12d00b2880 0x7f12d00b4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f12e4079ba0 tx=0x7f12d400b3c0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 msgr2=0x7f12e419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 0x7f12e419f710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f12e000e9a0 tx=0x7f12e000ecb0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 shutdown_connections 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f12d00b2880 0x7f12d00b4d40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e4108520 0x7f12e4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e4102440 0x7f12e419f710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 --2- 172.21.15.26:0/3930620906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4102e00 0x7f12e419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 >> 172.21.15.26:0/3930620906 conn(0x7f12e40793f0 msgr2=0x7f12e406b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 shutdown_connections 2024-05-20T21:05:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:11.703+0000 7f12ebb25700 1 -- 172.21.15.26:0/3930620906 wait complete. 2024-05-20T21:05:11.711 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:11 smithi055 ceph-mon[35213]: pgmap v661: 97 pgs: 97 active+clean; 29 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 693 KiB/s wr, 50 op/s 2024-05-20T21:05:12.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 203833 2.34 MB/sec execute 318 sec latency 18.445 ms 2024-05-20T21:05:13.534 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:05:13.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:05:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:13 smithi144 ceph-mon[35660]: pgmap v662: 97 pgs: 97 active+clean; 28 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 957 KiB/s wr, 57 op/s 2024-05-20T21:05:13.559 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:13 smithi026 ceph-mon[31990]: pgmap v662: 97 pgs: 97 active+clean; 28 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 957 KiB/s wr, 57 op/s 2024-05-20T21:05:13.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 204556 2.34 MB/sec execute 319 sec latency 17.684 ms 2024-05-20T21:05:13.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:13 smithi055 ceph-mon[35213]: pgmap v662: 97 pgs: 97 active+clean; 28 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 957 KiB/s wr, 57 op/s 2024-05-20T21:05:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:14.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 205373 2.35 MB/sec execute 320 sec latency 63.219 ms 2024-05-20T21:05:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:15 smithi144 ceph-mon[35660]: pgmap v663: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-05-20T21:05:15.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 205819 2.36 MB/sec execute 321 sec latency 72.324 ms 2024-05-20T21:05:15.644 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:15 smithi026 ceph-mon[31990]: pgmap v663: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-05-20T21:05:15.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:15 smithi055 ceph-mon[35213]: pgmap v663: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-05-20T21:05:16.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 206191 2.35 MB/sec execute 322 sec latency 69.979 ms 2024-05-20T21:05:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:17 smithi144 ceph-mon[35660]: pgmap v664: 97 pgs: 97 active+clean; 25 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-20T21:05:17.551 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:17 smithi026 ceph-mon[31990]: pgmap v664: 97 pgs: 97 active+clean; 25 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-20T21:05:17.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 206676 2.35 MB/sec execute 323 sec latency 26.873 ms 2024-05-20T21:05:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 -- 172.21.15.26:0/4206751571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102440 msgr2=0x7f4dd01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/4206751571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102440 0x7f4dd01028c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009990 tx=0x7f4db8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 -- 172.21.15.26:0/4206751571 shutdown_connections 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/4206751571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd0102e00 0x7f4dd010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/4206751571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 0x7f4dd0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/4206751571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102440 0x7f4dd01028c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 -- 172.21.15.26:0/4206751571 >> 172.21.15.26:0/4206751571 conn(0x7f4dd00793f0 msgr2=0x7f4dd0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 -- 172.21.15.26:0/4206751571 shutdown_connections 2024-05-20T21:05:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.575+0000 7f4dd62f4700 1 -- 172.21.15.26:0/4206751571 wait complete. 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 Processor -- start 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 -- start start 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd0102440 0x7f4dd0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 0x7f4dd01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 0x7f4dd0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01124a0 con 0x7f4dd0102e00 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0112320 con 0x7f4dd0108520 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dd62f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0112620 con 0x7f4dd0102440 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.576+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 0x7f4dd01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 0x7f4dd01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37454/0 (socket says 172.21.15.26:37454) 2024-05-20T21:05:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 -- 172.21.15.26:0/497361903 learned_addr learned my addr 172.21.15.26:0/497361903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dd4891700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 0x7f4dd0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dcffff700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd0102440 0x7f4dd0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 -- 172.21.15.26:0/497361903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd0102440 msgr2=0x7f4dd0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd0102440 0x7f4dd0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 -- 172.21.15.26:0/497361903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 msgr2=0x7f4dd0104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 0x7f4dd0104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 -- 172.21.15.26:0/497361903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd01a8060 con 0x7f4dd0102e00 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dd4891700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 0x7f4dd0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 0x7f4dd01045c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4db8006010 tx=0x7f4db80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dcdffb700 1 -- 172.21.15.26:0/497361903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8027070 con 0x7f4dd0102e00 2024-05-20T21:05:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dcdffb700 1 -- 172.21.15.26:0/497361903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4db8022470 con 0x7f4dd0102e00 2024-05-20T21:05:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dcdffb700 1 -- 172.21.15.26:0/497361903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8017650 con 0x7f4dd0102e00 2024-05-20T21:05:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.577+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd01a82f0 con 0x7f4dd0102e00 2024-05-20T21:05:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.578+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a86f0 con 0x7f4dd0102e00 2024-05-20T21:05:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.581+0000 7f4dcdffb700 1 -- 172.21.15.26:0/497361903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4db802a070 con 0x7f4dd0102e00 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.581+0000 7f4de0b8ab80 1 Processor -- start 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.581+0000 7f4dcdffb700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 -- start start 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dcffff700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dcdffb700 1 -- 172.21.15.26:0/497361903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4db80e2f70 con 0x7f4dd0102e00 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fdd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557a25f33d10 con 0x557a25fe17f0 2024-05-20T21:05:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557a25f238b0 con 0x557a25fddaa0 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a25f278f0 con 0x557a25fdd0f0 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dd4891700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dd4891700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37462/0 (socket says 172.21.15.26:37462) 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dd4891700 1 -- 172.21.15.26:0/2115937416 learned_addr learned my addr 172.21.15.26:0/2115937416 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dcffff700 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fdd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dcffff700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009bf0 tx=0x7f4dc0011440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dc7fff700 1 -- 172.21.15.26:0/2115937416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 msgr2=0x557a25fdd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fdd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dc7fff700 1 -- 172.21.15.26:0/2115937416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 msgr2=0x557a25fe1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.582+0000 7f4dc7fff700 1 -- 172.21.15.26:0/2115937416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a25e73950 con 0x557a25fddaa0 2024-05-20T21:05:17.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.583+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe11d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4db80e0a60 tx=0x7f4db80a9f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=806ca900b9e66aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:17.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.583+0000 7f4dc67fc700 1 -- 172.21.15.26:0/2115937416 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db80aacc0 con 0x557a25fddaa0 2024-05-20T21:05:17.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.583+0000 7f4dc67fc700 1 -- 172.21.15.26:0/2115937416 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4db80aa5c0 con 0x557a25fddaa0 2024-05-20T21:05:17.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.583+0000 7f4dc67fc700 1 -- 172.21.15.26:0/2115937416 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db80aa820 con 0x557a25fddaa0 2024-05-20T21:05:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2115937416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 msgr2=0x557a25fe11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe11d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4db80e0a60 tx=0x7f4db80a9f90 comp rx=0 tx=0).stop 2024-05-20T21:05:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2115937416 shutdown_connections 2024-05-20T21:05:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fdd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2115937416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe11d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2115937416 >> 172.21.15.26:0/2115937416 conn(0x557a25f02100 msgr2=0x557a25fcef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2115937416 shutdown_connections 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2115937416 wait complete. 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.584+0000 7f4de0b8ab80 1 -- start start 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 Processor -- start 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fe8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557a25e2a020 con 0x557a25fe17f0 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557a25ec5840 con 0x557a25fddaa0 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a25fe9770 con 0x557a25fdd0f0 2024-05-20T21:05:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4de0b8ab80 10 client.0 _ll_register_callbacks cb 0x557a25ebe9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dd4891700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dd4891700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37468/0 (socket says 172.21.15.26:37468) 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60080/0 (socket says 172.21.15.26:60080) 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dd4891700 1 -- 172.21.15.26:0/1434208355 learned_addr learned my addr 172.21.15.26:0/1434208355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dcffff700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fe8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 -- 172.21.15.26:0/1434208355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 msgr2=0x557a25fe8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fe8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 -- 172.21.15.26:0/1434208355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 msgr2=0x557a25fe64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dc7fff700 1 -- 172.21.15.26:0/1434208355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a25ec2710 con 0x557a25fddaa0 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.585+0000 7f4dd4891700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe5fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4db80b3550 tx=0x7f4db80ae9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0818af9cfa9af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4daf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4daf7fe700 1 -- 172.21.15.26:0/1434208355 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db80c3070 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4daf7fe700 1 -- 172.21.15.26:0/1434208355 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4db80aaa30 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db80aa5c0 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4daf7fe700 1 -- 172.21.15.26:0/1434208355 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4db80aacc0 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 10 client.27587 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a25ff3de0 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4daf7fe700 1 -- 172.21.15.26:0/1434208355 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4db80aa5c0 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4daf7fe700 1 -- 172.21.15.26:0/1434208355 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4db80a4ce0 con 0x557a25fddaa0 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 10 client.27587 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 10 client.27587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 20 client.27587 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a25ff62b0 0x557a25ff86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.586+0000 7f4de0b8ab80 4 client.27587 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.587+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a25fa0a60 con 0x557a25ff62b0 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.587+0000 7f4dcffff700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a25ff62b0 0x557a25ff86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.587+0000 7f4dcffff700 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a25ff62b0 0x557a25ff86a0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e2dd42f6ceed12 server_cookie=7033151912919f8f in_seq=0 out_seq=0 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.587+0000 7f4daf7fe700 10 client.27587 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4daf7fe700 1 -- 172.21.15.26:0/1434208355 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557a25fa0a60 con 0x557a25ff62b0 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4daf7fe700 10 client.27587 handle_command_reply: tid=0 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 client.27587 shutdown 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 msgr2=0x557a25fe5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe5fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4db80b3550 tx=0x7f4db80ae9e0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 shutdown_connections 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a25ff62b0 0x557a25ff86a0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fe17f0 0x557a25fe64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fdd0f0 0x557a25fe8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/1434208355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fddaa0 0x557a25fe5fb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 >> 172.21.15.26:0/1434208355 conn(0x557a25f02100 msgr2=0x557a25fe2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4daf7fe700 0 client.27587 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.588+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 shutdown_connections 2024-05-20T21:05:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.589+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/1434208355 wait complete. 2024-05-20T21:05:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.589+0000 7f4de0b8ab80 20 client.27587 trim_cache size 0 max 16384 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 Processor -- start 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 -- start start 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fe8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25ffde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25ffe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01ae320 con 0x557a25fddaa0 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01abbb0 con 0x557a25fdd0f0 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd019da60 con 0x557a25fe17f0 2024-05-20T21:05:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25ffde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25ffde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37476/0 (socket says 172.21.15.26:37476) 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dc7fff700 1 -- 172.21.15.26:0/3523916864 learned_addr learned my addr 172.21.15.26:0/3523916864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.611+0000 7f4dcffff700 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fe8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dd4891700 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25ffe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dcffff700 1 -- 172.21.15.26:0/3523916864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 msgr2=0x557a25ffe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dcffff700 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25ffe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dcffff700 1 -- 172.21.15.26:0/3523916864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 msgr2=0x557a25ffde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dcffff700 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25ffde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dcffff700 1 -- 172.21.15.26:0/3523916864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a25fe68b0 con 0x557a25fdd0f0 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25ffde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dcffff700 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fe8020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x557a25ff8d50 tx=0x7f4dc001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92a070947c3547e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dc5ffb700 1 -- 172.21.15.26:0/3523916864 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0031070 con 0x557a25fdd0f0 2024-05-20T21:05:17.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dc5ffb700 1 -- 172.21.15.26:0/3523916864 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dc0011e50 con 0x557a25fdd0f0 2024-05-20T21:05:17.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.612+0000 7f4dc5ffb700 1 -- 172.21.15.26:0/3523916864 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0017040 con 0x557a25fdd0f0 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/3523916864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 msgr2=0x557a25fe8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fe8020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x557a25ff8d50 tx=0x7f4dc001c000 comp rx=0 tx=0).stop 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/3523916864 shutdown_connections 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25ffe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fe8020 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/3523916864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25ffde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/3523916864 >> 172.21.15.26:0/3523916864 conn(0x557a25f02100 msgr2=0x557a25fd3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/3523916864 shutdown_connections 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/3523916864 wait complete. 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.613+0000 7f4de0b8ab80 1 -- start start 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 Processor -- start 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25fedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25fee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01aa070 con 0x557a25fddaa0 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01ab4e0 con 0x557a25fdd0f0 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01aa740 con 0x557a25fe17f0 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4de0b8ab80 10 client.0 _ll_register_callbacks cb 0x557a25ebe9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25fedcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fed7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60092/0 (socket says 172.21.15.26:60092) 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25fedcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37488/0 (socket says 172.21.15.26:37488) 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dcffff700 1 -- 172.21.15.26:0/2409463406 learned_addr learned my addr 172.21.15.26:0/2409463406 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dd4891700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25fee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 -- 172.21.15.26:0/2409463406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 msgr2=0x557a25fee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25fee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 -- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 msgr2=0x557a25fed7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fed7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dc7fff700 1 -- 172.21.15.26:0/2409463406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a26060d80 con 0x557a25fddaa0 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.614+0000 7f4dcffff700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4dc7fff700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25fedcf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4db80a7830 tx=0x7f4db80e0a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913639ceb943c1af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4daf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4daf7fe700 1 -- 172.21.15.26:0/2409463406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db80c3070 con 0x557a25fddaa0 2024-05-20T21:05:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4daf7fe700 1 -- 172.21.15.26:0/2409463406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4db80b2f00 con 0x557a25fddaa0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc0011e50 con 0x557a25fddaa0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4daf7fe700 1 -- 172.21.15.26:0/2409463406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4db80b3100 con 0x557a25fddaa0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4de0b8ab80 10 client.18201 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a26060d80 con 0x557a25fddaa0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4daf7fe700 1 -- 172.21.15.26:0/2409463406 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dc0011e50 con 0x557a25fddaa0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.615+0000 7f4daf7fe700 1 -- 172.21.15.26:0/2409463406 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4db80a4ce0 con 0x557a25fddaa0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4de0b8ab80 10 client.18201 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4de0b8ab80 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4de0b8ab80 20 client.18201 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a260623d0 0x557a260647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4de0b8ab80 4 client.18201 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a25f35c60 con 0x557a260623d0 2024-05-20T21:05:17.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4dcffff700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a260623d0 0x557a260647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4dcffff700 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a260623d0 0x557a260647f0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7457d5c313c852f3 server_cookie=a8eb2eaaacdf773e in_seq=0 out_seq=0 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.616+0000 7f4daf7fe700 10 client.18201 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4daf7fe700 1 -- 172.21.15.26:0/2409463406 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a25f35c60 con 0x557a260623d0 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4daf7fe700 10 client.18201 handle_command_reply: tid=0 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 client.18201 shutdown 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 msgr2=0x557a25fedcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25fedcf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4db80a7830 tx=0x7f4db80e0a60 comp rx=0 tx=0).stop 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 shutdown_connections 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557a260623d0 0x557a260647f0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25fe17f0 0x557a25fee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557a25fdd0f0 0x557a25fed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4daf7fe700 0 client.18201 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 --2- 172.21.15.26:0/2409463406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a25fddaa0 0x557a25fedcf0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 >> 172.21.15.26:0/2409463406 conn(0x557a25f02100 msgr2=0x557a25fe0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 shutdown_connections 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.617+0000 7f4de0b8ab80 1 -- 172.21.15.26:0/2409463406 wait complete. 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4de0b8ab80 20 client.18201 trim_cache size 0 max 16384 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "309ff63f-7cee-43b7-81e7-2e06c8cfcfb2", 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:05:17.623 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:05:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4dbc0b2880 msgr2=0x7f4dbc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009bf0 tx=0x7f4dc0011440 comp rx=0 tx=0).stop 2024-05-20T21:05:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 msgr2=0x7f4dd01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 0x7f4dd01045c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4db8006010 tx=0x7f4db80057c0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 shutdown_connections 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dd0108520 0x7f4dd0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd0102440 0x7f4dd0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 --2- 172.21.15.26:0/497361903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0102e00 0x7f4dd01045c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.618+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 >> 172.21.15.26:0/497361903 conn(0x7f4dd00793f0 msgr2=0x7f4dd006b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.619+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 shutdown_connections 2024-05-20T21:05:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:17.619+0000 7f4dd62f4700 1 -- 172.21.15.26:0/497361903 wait complete. 2024-05-20T21:05:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:17 smithi055 ceph-mon[35213]: pgmap v664: 97 pgs: 97 active+clean; 25 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-20T21:05:18.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:05:18.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:18 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:05:18.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:18 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:05:18.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:18 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:05:18.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:18 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:05:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:18 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:05:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:18 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:05:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:18 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:05:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:18 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:05:18.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 207350 2.35 MB/sec execute 324 sec latency 18.754 ms 2024-05-20T21:05:18.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:18 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:05:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:18 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:05:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:18 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:05:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:18 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:05:18.778 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:19.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:19 smithi026 ceph-mon[31990]: pgmap v665: 97 pgs: 97 active+clean; 26 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-05-20T21:05:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:19 smithi144 ceph-mon[35660]: pgmap v665: 97 pgs: 97 active+clean; 26 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-05-20T21:05:19.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 208075 2.35 MB/sec execute 325 sec latency 17.539 ms 2024-05-20T21:05:19.601 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:19 smithi055 ceph-mon[35213]: pgmap v665: 97 pgs: 97 active+clean; 26 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-05-20T21:05:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/2391665873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 msgr2=0x7fc6b406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/2391665873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b406c090 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009990 tx=0x7fc6a8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/2391665873 shutdown_connections 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/2391665873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 0x7fc6b41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/2391665873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 0x7fc6b41099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/2391665873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b406c090 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/2391665873 >> 172.21.15.26:0/2391665873 conn(0x7fc6b40793f0 msgr2=0x7fc6b4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/2391665873 shutdown_connections 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.559+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/2391665873 wait complete. 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 Processor -- start 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 -- start start 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 0x7fc6b419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 0x7fc6b41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41135b0 con 0x7fc6b406bc10 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4113430 con 0x7fc6b406c5d0 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6ba1b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4113730 con 0x7fc6b41095d0 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57500/0 (socket says 172.21.15.26:57500) 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.560+0000 7fc6b37fe700 1 -- 172.21.15.26:0/3138215060 learned_addr learned my addr 172.21.15.26:0/3138215060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b3fff700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 0x7fc6b41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 0x7fc6b419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b37fe700 1 -- 172.21.15.26:0/3138215060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 msgr2=0x7fc6b41999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 0x7fc6b41999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b37fe700 1 -- 172.21.15.26:0/3138215060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 msgr2=0x7fc6b419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 0x7fc6b419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b37fe700 1 -- 172.21.15.26:0/3138215060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b419a110 con 0x7fc6b406bc10 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b3fff700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 0x7fc6b41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 0x7fc6b419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b419f7f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400e9c0 tx=0x7fc6a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b0ff9700 1 -- 172.21.15.26:0/3138215060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a400cbd0 con 0x7fc6b406bc10 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b0ff9700 1 -- 172.21.15.26:0/3138215060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a400ee60 con 0x7fc6b406bc10 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6b0ff9700 1 -- 172.21.15.26:0/3138215060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40054b0 con 0x7fc6b406bc10 2024-05-20T21:05:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b419a400 con 0x7fc6b406bc10 2024-05-20T21:05:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.561+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b41a66c0 con 0x7fc6b406bc10 2024-05-20T21:05:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.564+0000 7fc6b0ff9700 1 -- 172.21.15.26:0/3138215060 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc6a4021440 con 0x7fc6b406bc10 2024-05-20T21:05:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.565+0000 7fc6b0ff9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc69c0b2880 0x7fc69c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.565+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc69c0b2880 0x7fc69c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.565+0000 7fc6c4a4fb80 1 Processor -- start 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.565+0000 7fc6b0ff9700 1 -- 172.21.15.26:0/3138215060 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fc6a40e2550 con 0x7fc6b406bc10 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.565+0000 7fc6c4a4fb80 1 -- start start 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade52a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade4d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5607adda6d70 con 0x5607ade52670 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5607add96910 con 0x5607ade4ccb0 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5607add9a950 con 0x5607ade4d670 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade4d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade4d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60688/0 (socket says 172.21.15.26:60688) 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/2527207430 learned_addr learned my addr 172.21.15.26:0/2527207430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc69c0b2880 0x7fc69c0b4d40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4079b20 tx=0x7fc6a801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b3fff700 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade52a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/2527207430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 msgr2=0x5607ade59460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade59460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/2527207430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 msgr2=0x5607ade52a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade52a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.566+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/2527207430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607add3ffb0 con 0x5607ade4ccb0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.567+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade4d130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009990 tx=0x7fc6a801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e9c687c22e131d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.567+0000 7fc699ffb700 1 -- 172.21.15.26:0/2527207430 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005050 con 0x5607ade4ccb0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.567+0000 7fc699ffb700 1 -- 172.21.15.26:0/2527207430 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a8005680 con 0x5607ade4ccb0 2024-05-20T21:05:20.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.567+0000 7fc699ffb700 1 -- 172.21.15.26:0/2527207430 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80038e0 con 0x5607ade4ccb0 2024-05-20T21:05:20.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2527207430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 msgr2=0x5607ade4d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade4d130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009990 tx=0x7fc6a801f000 comp rx=0 tx=0).stop 2024-05-20T21:05:20.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2527207430 shutdown_connections 2024-05-20T21:05:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade52a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade59460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2527207430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade4d130 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2527207430 >> 172.21.15.26:0/2527207430 conn(0x5607add74240 msgr2=0x5607ade433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2527207430 shutdown_connections 2024-05-20T21:05:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2527207430 wait complete. 2024-05-20T21:05:20.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 208894 2.36 MB/sec execute 326 sec latency 18.342 ms 2024-05-20T21:05:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 -- start start 2024-05-20T21:05:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.568+0000 7fc6c4a4fb80 1 Processor -- start 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade56420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5607adc9cfa0 con 0x5607ade52670 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5607adcfeda0 con 0x5607ade4ccb0 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5607ade59b00 con 0x5607ade4d670 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6c4a4fb80 10 client.0 _ll_register_callbacks cb 0x5607add31950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade56960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57528/0 (socket says 172.21.15.26:57528) 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b3fff700 1 -- 172.21.15.26:0/1858002753 learned_addr learned my addr 172.21.15.26:0/1858002753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade56420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 -- 172.21.15.26:0/1858002753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 msgr2=0x5607ade56420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade56420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 -- 172.21.15.26:0/1858002753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 msgr2=0x5607ade56960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade56960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 -- 172.21.15.26:0/1858002753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607adce68d0 con 0x5607ade4ccb0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade56420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade59120 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40e0360 tx=0x7fc6a40b2380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a6e81b55073e77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.569+0000 7fc692ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc692ffd700 1 -- 172.21.15.26:0/1858002753 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40c3060 con 0x5607ade4ccb0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc692ffd700 1 -- 172.21.15.26:0/1858002753 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fc6a40e20f0 con 0x5607ade4ccb0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a8005680 con 0x5607ade4ccb0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc692ffd700 1 -- 172.21.15.26:0/1858002753 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6a40e22f0 con 0x5607ade4ccb0 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 10 client.27605 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607add50760 con 0x5607ade4ccb0 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc692ffd700 1 -- 172.21.15.26:0/1858002753 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a8005680 con 0x5607ade4ccb0 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc692ffd700 1 -- 172.21.15.26:0/1858002753 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fc6a40a5640 con 0x5607ade4ccb0 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 10 client.27605 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 10 client.27605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 20 client.27605 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607ade66650 0x5607ade68aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 4 client.27605 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.570+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607adda8cc0 con 0x5607ade66650 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.571+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607ade66650 0x5607ade68aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.571+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607ade66650 0x5607ade68aa0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1507a8ca7f96c142 server_cookie=4118174324df4bbc in_seq=0 out_seq=0 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.571+0000 7fc692ffd700 10 client.27605 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc692ffd700 1 -- 172.21.15.26:0/1858002753 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5607adda8cc0 con 0x5607ade66650 2024-05-20T21:05:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc692ffd700 10 client.27605 handle_command_reply: tid=0 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 client.27605 shutdown 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 msgr2=0x5607ade59120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade59120 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40e0360 tx=0x7fc6a40b2380 comp rx=0 tx=0).stop 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 shutdown_connections 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607ade66650 0x5607ade68aa0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade56960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade59120 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc692ffd700 0 client.27605 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/1858002753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade56420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 >> 172.21.15.26:0/1858002753 conn(0x5607add74240 msgr2=0x5607ade42af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 shutdown_connections 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.572+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/1858002753 wait complete. 2024-05-20T21:05:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.573+0000 7fc6c4a4fb80 20 client.27605 trim_cache size 0 max 16384 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 Processor -- start 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 -- start start 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade62ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade47fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41ae330 con 0x5607ade52670 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41abbc0 con 0x5607ade4ccb0 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41a9a10 con 0x5607ade4d670 2024-05-20T21:05:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade47fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade62ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade62ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60698/0 (socket says 172.21.15.26:60698) 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.597+0000 7fc6b37fe700 1 -- 172.21.15.26:0/2665777174 learned_addr learned my addr 172.21.15.26:0/2665777174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b37fe700 1 -- 172.21.15.26:0/2665777174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 msgr2=0x5607ade43270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade43270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b37fe700 1 -- 172.21.15.26:0/2665777174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 msgr2=0x5607ade47fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade47fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b37fe700 1 -- 172.21.15.26:0/2665777174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ade69120 con 0x5607ade4ccb0 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade62ba0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40cc010 tx=0x7fc6a400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=384209930dfdba7f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6997fa700 1 -- 172.21.15.26:0/2665777174 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40a5640 con 0x5607ade4ccb0 2024-05-20T21:05:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6997fa700 1 -- 172.21.15.26:0/2665777174 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a40c9070 con 0x5607ade4ccb0 2024-05-20T21:05:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.598+0000 7fc6997fa700 1 -- 172.21.15.26:0/2665777174 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40af0b0 con 0x5607ade4ccb0 2024-05-20T21:05:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2665777174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 msgr2=0x5607ade62ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade62ba0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40cc010 tx=0x7fc6a400cf90 comp rx=0 tx=0).stop 2024-05-20T21:05:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2665777174 shutdown_connections 2024-05-20T21:05:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade47fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4ccb0 0x5607ade62ba0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/2665777174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4d670 0x5607ade43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2665777174 >> 172.21.15.26:0/2665777174 conn(0x5607add74240 msgr2=0x5607ade47890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2665777174 shutdown_connections 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/2665777174 wait complete. 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.599+0000 7fc6c4a4fb80 1 -- start start 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 Processor -- start 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4ccb0 0x5607ade6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 0x5607ade6e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade6e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41aa080 con 0x5607ade52670 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41ab4f0 con 0x5607ade4d670 2024-05-20T21:05:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41aa750 con 0x5607ade4ccb0 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6c4a4fb80 10 client.0 _ll_register_callbacks cb 0x5607add31950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 0x5607ade6e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade6e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 0x5607ade6e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60706/0 (socket says 172.21.15.26:60706) 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/4137778414 learned_addr learned my addr 172.21.15.26:0/4137778414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4ccb0 0x5607ade6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/4137778414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4ccb0 msgr2=0x5607ade6de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4ccb0 0x5607ade6de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/4137778414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 msgr2=0x5607ade6e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade6e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.600+0000 7fc6b2ffd700 1 -- 172.21.15.26:0/4137778414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ade5d1e0 con 0x5607ade4d670 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc6b2ffd700 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 0x5607ade6e3a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005ef0 tx=0x7fc6a8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f88bd28167d5ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc692ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc692ffd700 1 -- 172.21.15.26:0/4137778414 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a803d060 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a40c9070 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc692ffd700 1 -- 172.21.15.26:0/4137778414 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fc6a8003c60 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc692ffd700 1 -- 172.21.15.26:0/4137778414 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6a8003e60 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc6c4a4fb80 10 client.27611 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607aded1500 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc692ffd700 1 -- 172.21.15.26:0/4137778414 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a40c9070 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.601+0000 7fc692ffd700 1 -- 172.21.15.26:0/4137778414 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fc6a8041030 con 0x5607ade4d670 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6c4a4fb80 10 client.27611 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6c4a4fb80 10 client.27611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6c4a4fb80 20 client.27611 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607aded1790 0x5607aded3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6c4a4fb80 4 client.27611 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:05:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607adda24a0 con 0x5607aded1790 2024-05-20T21:05:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607aded1790 0x5607aded3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc6b37fe700 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607aded1790 0x5607aded3b80 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5768ace954bed841 server_cookie=10908f48431bd71a in_seq=0 out_seq=0 2024-05-20T21:05:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.602+0000 7fc692ffd700 10 client.27611 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc692ffd700 1 -- 172.21.15.26:0/4137778414 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5607adda24a0 con 0x5607aded1790 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc692ffd700 10 client.27611 handle_command_reply: tid=0 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 client.27611 shutdown 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 msgr2=0x5607ade6e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 0x5607ade6e3a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005ef0 tx=0x7fc6a8005e80 comp rx=0 tx=0).stop 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 shutdown_connections 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5607aded1790 0x5607aded3b80 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ade52670 0x5607ade6e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607ade4ccb0 0x5607ade6de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 --2- 172.21.15.26:0/4137778414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607ade4d670 0x5607ade6e3a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 >> 172.21.15.26:0/4137778414 conn(0x5607add74240 msgr2=0x5607ade48120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc692ffd700 0 client.27611 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 shutdown_connections 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 1 -- 172.21.15.26:0/4137778414 wait complete. 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.603+0000 7fc6c4a4fb80 20 client.27611 trim_cache size 0 max 16384 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "309ff63f-7cee-43b7-81e7-2e06c8cfcfb2": { 2024-05-20T21:05:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "309ff63f-7cee-43b7-81e7-2e06c8cfcfb2", 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc69c0b2880 msgr2=0x7fc69c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc69c0b2880 0x7fc69c0b4d40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4079b20 tx=0x7fc6a801a040 comp rx=0 tx=0).stop 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 msgr2=0x7fc6b419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b419f7f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400e9c0 tx=0x7fc6a400ecd0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 shutdown_connections 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc69c0b2880 0x7fc69c0b4d40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b406bc10 0x7fc6b419f7f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6b41095d0 0x7fc6b41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 --2- 172.21.15.26:0/3138215060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b406c5d0 0x7fc6b419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 >> 172.21.15.26:0/3138215060 conn(0x7fc6b40793f0 msgr2=0x7fc6b4101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 shutdown_connections 2024-05-20T21:05:20.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:20.604+0000 7fc6ba1b9700 1 -- 172.21.15.26:0/3138215060 wait complete. 2024-05-20T21:05:21.410 INFO:tasks.cephfs.filesystem:scrub status for tag:309ff63f-7cee-43b7-81e7-2e06c8cfcfb2 - {'path': '/', 'tag': '309ff63f-7cee-43b7-81e7-2e06c8cfcfb2', 'options': 'recursive,force'} 2024-05-20T21:05:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:21 smithi144 ceph-mon[35660]: pgmap v666: 97 pgs: 97 active+clean; 27 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:05:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:21 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:05:21.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 209404 2.37 MB/sec execute 327 sec latency 18.017 ms 2024-05-20T21:05:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:21 smithi055 ceph-mon[35213]: pgmap v666: 97 pgs: 97 active+clean; 27 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:05:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:21 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:05:21.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:21 smithi026 ceph-mon[31990]: pgmap v666: 97 pgs: 97 active+clean; 27 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:05:21.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:21 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:05:22.500 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-20T21:05:22.500 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.55 2024-05-20T21:05:22.500 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:05:22.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 209853 2.37 MB/sec execute 328 sec latency 10.732 ms 2024-05-20T21:05:22.761 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:23.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:23 smithi026 ceph-mon[31990]: pgmap v667: 97 pgs: 97 active+clean; 26 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-05-20T21:05:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:23 smithi144 ceph-mon[35660]: pgmap v667: 97 pgs: 97 active+clean; 26 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-05-20T21:05:23.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 210457 2.37 MB/sec execute 329 sec latency 29.071 ms 2024-05-20T21:05:23.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:23 smithi055 ceph-mon[35213]: pgmap v667: 97 pgs: 97 active+clean; 26 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-05-20T21:05:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 -- 172.21.15.26:0/2617677307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538103530 msgr2=0x7fe538105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/2617677307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538103530 0x7fe538105920 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe528009900 tx=0x7fe52802a410 comp rx=0 tx=0).stop 2024-05-20T21:05:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 -- 172.21.15.26:0/2617677307 shutdown_connections 2024-05-20T21:05:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/2617677307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe538108790 0x7fe53810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/2617677307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe538105e60 0x7fe538108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/2617677307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538103530 0x7fe538105920 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 -- 172.21.15.26:0/2617677307 >> 172.21.15.26:0/2617677307 conn(0x7fe5380fcf20 msgr2=0x7fe5380ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 -- 172.21.15.26:0/2617677307 shutdown_connections 2024-05-20T21:05:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.180+0000 7fe53c9a4640 1 -- 172.21.15.26:0/2617677307 wait complete. 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 Processor -- start 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 -- start start 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe538103530 0x7fe538102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 0x7fe5381014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe538108790 0x7fe538101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe53810e500 con 0x7fe538105e60 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe53810e380 con 0x7fe538108790 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53c9a4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe53810e680 con 0x7fe538103530 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 0x7fe5381014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 0x7fe5381014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57562/0 (socket says 172.21.15.26:57562) 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 -- 172.21.15.26:0/3584417905 learned_addr learned my addr 172.21.15.26:0/3584417905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe536d9d640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe538108790 0x7fe538101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe53659c640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe538103530 0x7fe538102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 -- 172.21.15.26:0/3584417905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe538103530 msgr2=0x7fe538102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe538103530 0x7fe538102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 -- 172.21.15.26:0/3584417905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe538108790 msgr2=0x7fe538101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe538108790 0x7fe538101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.181+0000 7fe535d9b640 1 -- 172.21.15.26:0/3584417905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5381022b0 con 0x7fe538105e60 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.182+0000 7fe535d9b640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 0x7fe5381014e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d770 tx=0x7fe52c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.182+0000 7fe5277fe640 1 -- 172.21.15.26:0/3584417905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c017070 con 0x7fe538105e60 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.182+0000 7fe5277fe640 1 -- 172.21.15.26:0/3584417905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe52c009c30 con 0x7fe538105e60 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.182+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5381a2ca0 con 0x7fe538105e60 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.182+0000 7fe5277fe640 1 -- 172.21.15.26:0/3584417905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00f620 con 0x7fe538105e60 2024-05-20T21:05:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.182+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5381a31a0 con 0x7fe538105e60 2024-05-20T21:05:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe5277fe640 1 -- 172.21.15.26:0/3584417905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe52c01a070 con 0x7fe538105e60 2024-05-20T21:05:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 Processor -- start 2024-05-20T21:05:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 -- start start 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 0x5562db04aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04b0b0 0x5562db04fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db050220 0x5562db0526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5562dafbbc30 con 0x5562db050220 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5562dafab7d0 con 0x5562db04a6c0 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53f798740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5562dafaf810 con 0x5562db04b0b0 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe536d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 0x5562db04aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe536d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 0x5562db04aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60716/0 (socket says 172.21.15.26:60716) 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe536d9d640 1 -- 172.21.15.26:0/2040270303 learned_addr learned my addr 172.21.15.26:0/2040270303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe535d9b640 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db050220 0x5562db0526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.185+0000 7fe53659c640 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04b0b0 0x5562db04fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe536d9d640 1 -- 172.21.15.26:0/2040270303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04b0b0 msgr2=0x5562db04fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04b0b0 0x5562db04fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe536d9d640 1 -- 172.21.15.26:0/2040270303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db050220 msgr2=0x5562db0526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db050220 0x5562db0526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe536d9d640 1 -- 172.21.15.26:0/2040270303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562dacc0210 con 0x5562db04a6c0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe535d9b640 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db050220 0x5562db0526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe5277fe640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe5200b28a0 0x7fe5200b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 0x5562db04aae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fd0 tx=0x7fe5300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfaa83b928b4d1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe53659c640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe5200b28a0 0x7fe5200b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe5277fe640 1 -- 172.21.15.26:0/3584417905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe52c0e1ca0 con 0x7fe538105e60 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe524ff9640 1 -- 172.21.15.26:0/2040270303 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530004d20 con 0x5562db04a6c0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe524ff9640 1 -- 172.21.15.26:0/2040270303 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe530004050 con 0x5562db04a6c0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.186+0000 7fe524ff9640 1 -- 172.21.15.26:0/2040270303 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530005260 con 0x5562db04a6c0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53659c640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe5200b28a0 0x7fe5200b4d60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe528005ab0 tx=0x7fe528005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 -- 172.21.15.26:0/2040270303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 msgr2=0x5562db04aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 0x5562db04aae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fd0 tx=0x7fe5300118d0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 -- 172.21.15.26:0/2040270303 shutdown_connections 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db050220 0x5562db0526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04b0b0 0x5562db04fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 --2- 172.21.15.26:0/2040270303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04a6c0 0x5562db04aae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 -- 172.21.15.26:0/2040270303 >> 172.21.15.26:0/2040270303 conn(0x5562db045d30 msgr2=0x5562db046130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 -- 172.21.15.26:0/2040270303 shutdown_connections 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.187+0000 7fe53f798740 1 -- 172.21.15.26:0/2040270303 wait complete. 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 -- start start 2024-05-20T21:05:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 Processor -- start 2024-05-20T21:05:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 0x5562db054d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db050220 0x5562db05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5562dac91790 con 0x5562db04b0b0 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5562daf67890 con 0x5562db050220 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5562db0531d0 con 0x5562db04a6c0 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53f798740 10 client.0 _ll_register_callbacks cb 0x5562dacbfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe535d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 0x5562db054d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe535d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 0x5562db054d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57580/0 (socket says 172.21.15.26:57580) 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe535d9b640 1 -- 172.21.15.26:0/2425885877 learned_addr learned my addr 172.21.15.26:0/2425885877 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db050220 0x5562db05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53659c640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe536d9d640 1 -- 172.21.15.26:0/2425885877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04a6c0 msgr2=0x5562db054810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe536d9d640 1 -- 172.21.15.26:0/2425885877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 msgr2=0x5562db054d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 0x5562db054d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe536d9d640 1 -- 172.21.15.26:0/2425885877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562dad8c620 con 0x5562db050220 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe53659c640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.188+0000 7fe535d9b640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 0x5562db054d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe536d9d640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db050220 0x5562db05bd90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe530011dd0 tx=0x7fe530004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebcedb6e8418bd75 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe51dffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe51dffb640 1 -- 172.21.15.26:0/2425885877 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530028020 con 0x5562db050220 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe51dffb640 1 -- 172.21.15.26:0/2425885877 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe530003c40 con 0x5562db050220 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe530004050 con 0x5562db050220 2024-05-20T21:05:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe51dffb640 1 -- 172.21.15.26:0/2425885877 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe530033050 con 0x5562db050220 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe53f798740 10 client.27623 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562db05ed70 con 0x5562db050220 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe51dffb640 1 -- 172.21.15.26:0/2425885877 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe530004050 con 0x5562db050220 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.189+0000 7fe51dffb640 1 -- 172.21.15.26:0/2425885877 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fe53000f4e0 con 0x5562db050220 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53f798740 10 client.27623 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53f798740 10 client.27623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53f798740 20 client.27623 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53f798740 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db060350 0x5562db062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53f798740 4 client.27623 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562dafbdb80 con 0x5562db060350 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53659c640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db060350 0x5562db062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe53659c640 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db060350 0x5562db062740 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2ab2471b94f073 server_cookie=c481b15cab42f44b in_seq=0 out_seq=0 2024-05-20T21:05:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.190+0000 7fe51dffb640 10 client.27623 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe51dffb640 1 -- 172.21.15.26:0/2425885877 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5562dafbdb80 con 0x5562db060350 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe51dffb640 10 client.27623 handle_command_reply: tid=0 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 client.27623 shutdown 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db050220 msgr2=0x5562db05bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db050220 0x5562db05bd90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe530011dd0 tx=0x7fe530004cf0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 shutdown_connections 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db060350 0x5562db062740 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db050220 0x5562db05bd90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe51dffb640 0 client.27623 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04b0b0 0x5562db054d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 --2- 172.21.15.26:0/2425885877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 >> 172.21.15.26:0/2425885877 conn(0x5562db045d30 msgr2=0x5562db044fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 shutdown_connections 2024-05-20T21:05:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 1 -- 172.21.15.26:0/2425885877 wait complete. 2024-05-20T21:05:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.191+0000 7fe53f798740 20 client.27623 trim_cache size 0 max 16384 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.209+0000 7fe53f798740 1 Processor -- start 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.209+0000 7fe53f798740 1 -- start start 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53f798740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53f798740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db05bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53f798740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db054d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53f798740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381aea10 con 0x5562db04a6c0 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53f798740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381ac2e0 con 0x5562db04b0b0 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53f798740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381aa130 con 0x5562db050220 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57582/0 (socket says 172.21.15.26:57582) 2024-05-20T21:05:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 -- 172.21.15.26:0/258752966 learned_addr learned my addr 172.21.15.26:0/258752966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe535d9b640 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db05bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe536d9d640 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db054d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 -- 172.21.15.26:0/258752966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 msgr2=0x5562db054d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db054d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 -- 172.21.15.26:0/258752966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 msgr2=0x5562db05bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db05bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 -- 172.21.15.26:0/258752966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562db04aaa0 con 0x5562db04a6c0 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe535d9b640 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db05bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.210+0000 7fe53659c640 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db054810 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe5280096c0 tx=0x7fe528040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1cf83d3169156 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe51ffff640 1 -- 172.21.15.26:0/258752966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528035040 con 0x5562db04a6c0 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe51ffff640 1 -- 172.21.15.26:0/258752966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe528031500 con 0x5562db04a6c0 2024-05-20T21:05:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe51ffff640 1 -- 172.21.15.26:0/258752966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280514c0 con 0x5562db04a6c0 2024-05-20T21:05:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 -- 172.21.15.26:0/258752966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 msgr2=0x5562db054810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db054810 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe5280096c0 tx=0x7fe528040000 comp rx=0 tx=0).stop 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 -- 172.21.15.26:0/258752966 shutdown_connections 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db054d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db05bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 --2- 172.21.15.26:0/258752966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db054810 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 -- 172.21.15.26:0/258752966 >> 172.21.15.26:0/258752966 conn(0x5562db045d30 msgr2=0x5562db04c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 -- 172.21.15.26:0/258752966 shutdown_connections 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.211+0000 7fe53f798740 1 -- 172.21.15.26:0/258752966 wait complete. 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 -- start start 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 Processor -- start 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db058dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db059310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db05a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381aa7a0 con 0x5562db04a6c0 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381abc10 con 0x5562db04b0b0 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381aae70 con 0x5562db050220 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53f798740 10 client.0 _ll_register_callbacks cb 0x7fe528009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db058dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db058dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57584/0 (socket says 172.21.15.26:57584) 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 -- 172.21.15.26:0/1160398331 learned_addr learned my addr 172.21.15.26:0/1160398331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe535d9b640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db059310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe536d9d640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db05a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 -- 172.21.15.26:0/1160398331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 msgr2=0x5562db05a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db05a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 -- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 msgr2=0x5562db059310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db059310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe53659c640 1 -- 172.21.15.26:0/1160398331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562db05a910 con 0x5562db04a6c0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.212+0000 7fe536d9d640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db05a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53659c640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db058dd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe5280044b0 tx=0x7fe5280044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ade71ccb8c748d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe51dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe51dffb640 1 -- 172.21.15.26:0/1160398331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52805e060 con 0x5562db04a6c0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe51dffb640 1 -- 172.21.15.26:0/1160398331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe528003cc0 con 0x5562db04a6c0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe528031500 con 0x5562db04a6c0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe51dffb640 1 -- 172.21.15.26:0/1160398331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe528035070 con 0x5562db04a6c0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 10 client.18237 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562db05bc90 con 0x5562db04a6c0 2024-05-20T21:05:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe51dffb640 1 -- 172.21.15.26:0/1160398331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe528031500 con 0x5562db04a6c0 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe51dffb640 1 -- 172.21.15.26:0/1160398331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fe528062030 con 0x5562db04a6c0 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 10 client.18237 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 10 client.18237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 20 client.18237 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db0cb2f0 0x5562db0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 4 client.18237 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.213+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562dafb7360 con 0x5562db0cb2f0 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.214+0000 7fe535d9b640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db0cb2f0 0x5562db0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.214+0000 7fe51dffb640 10 client.18237 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.214+0000 7fe535d9b640 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db0cb2f0 0x5562db0cd710 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d9000f31c4679e server_cookie=a64a9d71c0254f12 in_seq=0 out_seq=0 2024-05-20T21:05:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe51dffb640 1 -- 172.21.15.26:0/1160398331 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5562dafb7360 con 0x5562db0cb2f0 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe51dffb640 10 client.18237 handle_command_reply: tid=0 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 client.18237 shutdown 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 msgr2=0x5562db058dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db058dd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe5280044b0 tx=0x7fe5280044e0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 shutdown_connections 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5562db0cb2f0 0x5562db0cd710 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562db050220 0x5562db05a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5562db04b0b0 0x5562db059310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe51dffb640 0 client.18237 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 --2- 172.21.15.26:0/1160398331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562db04a6c0 0x5562db058dd0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 >> 172.21.15.26:0/1160398331 conn(0x5562db045d30 msgr2=0x5562db0546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 shutdown_connections 2024-05-20T21:05:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 1 -- 172.21.15.26:0/1160398331 wait complete. 2024-05-20T21:05:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.269+0000 7fe53f798740 20 client.18237 trim_cache size 0 max 16384 2024-05-20T21:05:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:05:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:05:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe5200b28a0 msgr2=0x7fe5200b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe5200b28a0 0x7fe5200b4d60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe528005ab0 tx=0x7fe528005a20 comp rx=0 tx=0).stop 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 msgr2=0x7fe5381014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 0x7fe5381014e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d770 tx=0x7fe52c00dc30 comp rx=0 tx=0).stop 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 shutdown_connections 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe5200b28a0 0x7fe5200b4d60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe538108790 0x7fe538101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe538105e60 0x7fe5381014e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 --2- 172.21.15.26:0/3584417905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe538103530 0x7fe538102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 >> 172.21.15.26:0/3584417905 conn(0x7fe5380fcf20 msgr2=0x7fe5381040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 shutdown_connections 2024-05-20T21:05:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:24.270+0000 7fe53c9a4640 1 -- 172.21.15.26:0/3584417905 wait complete. 2024-05-20T21:05:24.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 -- 172.21.15.26:0/3584462653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 msgr2=0x7f1108108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 --2- 172.21.15.26:0/3584462653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f1108108900 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f1104009990 tx=0x7f1104009ca0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 -- 172.21.15.26:0/3584462653 shutdown_connections 2024-05-20T21:05:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 --2- 172.21.15.26:0/3584462653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f1108108900 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 --2- 172.21.15.26:0/3584462653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102e00 0x7f110810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 --2- 172.21.15.26:0/3584462653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102440 0x7f11081028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 -- 172.21.15.26:0/3584462653 >> 172.21.15.26:0/3584462653 conn(0x7f11080793f0 msgr2=0x7f1108079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 -- 172.21.15.26:0/3584462653 shutdown_connections 2024-05-20T21:05:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.524+0000 7f110f555700 1 -- 172.21.15.26:0/3584462653 wait complete. 2024-05-20T21:05:24.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 Processor -- start 2024-05-20T21:05:24.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 -- start start 2024-05-20T21:05:24.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102440 0x7f110819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102e00 0x7f110819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f11081a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1108115780 con 0x7f1108108520 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1108115600 con 0x7f1108102e00 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110f555700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1108115900 con 0x7f1108102440 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110daf2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f11081a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110d2f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102440 0x7f110819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110caf0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102e00 0x7f110819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110daf2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f11081a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57604/0 (socket says 172.21.15.26:57604) 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.525+0000 7f110daf2700 1 -- 172.21.15.26:0/3755658391 learned_addr learned my addr 172.21.15.26:0/3755658391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110daf2700 1 -- 172.21.15.26:0/3755658391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102440 msgr2=0x7f110819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110daf2700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102440 0x7f110819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110daf2700 1 -- 172.21.15.26:0/3755658391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102e00 msgr2=0x7f110819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110daf2700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102e00 0x7f110819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110daf2700 1 -- 172.21.15.26:0/3755658391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081a0d30 con 0x7f1108108520 2024-05-20T21:05:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110caf0700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102e00 0x7f110819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110d2f1700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102440 0x7f110819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110daf2700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f11081a05f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1100007ee0 tx=0x7f110000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110000dd20 con 0x7f1108108520 2024-05-20T21:05:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a1020 con 0x7f1108108520 2024-05-20T21:05:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1100004020 con 0x7f1108108520 2024-05-20T21:05:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100015660 con 0x7f1108108520 2024-05-20T21:05:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.526+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a88a0 con 0x7f1108108520 2024-05-20T21:05:24.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.527+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110810a9b0 con 0x7f1108108520 2024-05-20T21:05:24.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.529+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f11000041d0 con 0x7f1108108520 2024-05-20T21:05:24.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.530+0000 7f10fe7fc700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10f40b2860 0x7f10f40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:24.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.530+0000 7f110d2f1700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10f40b2860 0x7f10f40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:24.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.530+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f11000e1ba0 con 0x7f1108108520 2024-05-20T21:05:24.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.531+0000 7f110d2f1700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10f40b2860 0x7f10f40b4d20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f110400b2e0 tx=0x7f1104005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:24.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.535+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1100010070 con 0x7f1108108520 2024-05-20T21:05:24.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 211146 2.37 MB/sec execute 330 sec latency 32.034 ms 2024-05-20T21:05:24.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.837+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f110810a390 con 0x7f1108108520 2024-05-20T21:05:24.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.838+0000 7f10fe7fc700 1 -- 172.21.15.26:0/3755658391 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f11000a9860 con 0x7f1108108520 2024-05-20T21:05:24.838 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:05:24.838 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10f40b2860 msgr2=0x7f10f40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10f40b2860 0x7f10f40b4d20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f110400b2e0 tx=0x7f1104005fb0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 msgr2=0x7f11081a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f11081a05f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1100007ee0 tx=0x7f110000d3b0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 shutdown_connections 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10f40b2860 0x7f10f40b4d20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1108108520 0x7f11081a05f0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1108102440 0x7f110819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 --2- 172.21.15.26:0/3755658391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1108102e00 0x7f110819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:24.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 >> 172.21.15.26:0/3755658391 conn(0x7f11080793f0 msgr2=0x7f1108101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:24.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 shutdown_connections 2024-05-20T21:05:24.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:24.841+0000 7f110f555700 1 -- 172.21.15.26:0/3755658391 wait complete. 2024-05-20T21:05:24.842 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:05:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:25 smithi026 ceph-mon[31990]: pgmap v668: 97 pgs: 97 active+clean; 28 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-05-20T21:05:25.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:25 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3755658391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:25.508 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 146.55 --await-for 45.0' 2024-05-20T21:05:25.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 146.55 --await-for 45.0 2024-05-20T21:05:25.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 211829 2.37 MB/sec execute 331 sec latency 19.648 ms 2024-05-20T21:05:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:25 smithi055 ceph-mon[35213]: pgmap v668: 97 pgs: 97 active+clean; 28 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-05-20T21:05:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:25 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3755658391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:25.786 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:25.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:25 smithi144 ceph-mon[35660]: pgmap v668: 97 pgs: 97 active+clean; 28 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-05-20T21:05:25.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:25 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3755658391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:26.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 212591 2.38 MB/sec execute 332 sec latency 18.101 ms 2024-05-20T21:05:27.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213164 2.39 MB/sec execute 333 sec latency 16.597 ms 2024-05-20T21:05:27.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:27 smithi026 ceph-mon[31990]: pgmap v669: 97 pgs: 97 active+clean; 28 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:05:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:27 smithi055 ceph-mon[35213]: pgmap v669: 97 pgs: 97 active+clean; 28 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 -- 172.21.15.26:0/605120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 msgr2=0x7fed0c06bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 --2- 172.21.15.26:0/605120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c06bf90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009990 tx=0x7fecfc009ca0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 -- 172.21.15.26:0/605120 shutdown_connections 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 --2- 172.21.15.26:0/605120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c06c4d0 0x7fed0c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 --2- 172.21.15.26:0/605120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0c1095d0 0x7fed0c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 --2- 172.21.15.26:0/605120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c06bf90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 -- 172.21.15.26:0/605120 >> 172.21.15.26:0/605120 conn(0x7fed0c079480 msgr2=0x7fed0c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.727+0000 7fed13fd9700 1 -- 172.21.15.26:0/605120 shutdown_connections 2024-05-20T21:05:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.728+0000 7fed13fd9700 1 -- 172.21.15.26:0/605120 wait complete. 2024-05-20T21:05:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.728+0000 7fed13fd9700 1 Processor -- start 2024-05-20T21:05:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.728+0000 7fed13fd9700 1 -- start start 2024-05-20T21:05:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.728+0000 7fed13fd9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.728+0000 7fed13fd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0c06c4d0 0x7fed0c102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.728+0000 7fed13fd9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c1095d0 0x7fed0c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed13fd9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c113590 con 0x7fed0c06bb10 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed13fd9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c113410 con 0x7fed0c1095d0 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed13fd9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c113710 con 0x7fed0c06c4d0 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57628/0 (socket says 172.21.15.26:57628) 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 -- 172.21.15.26:0/197823541 learned_addr learned my addr 172.21.15.26:0/197823541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed12576700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c1095d0 0x7fed0c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11574700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0c06c4d0 0x7fed0c102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 -- 172.21.15.26:0/197823541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0c06c4d0 msgr2=0x7fed0c102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0c06c4d0 0x7fed0c102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 -- 172.21.15.26:0/197823541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c1095d0 msgr2=0x7fed0c102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c1095d0 0x7fed0c102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 -- 172.21.15.26:0/197823541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c1a7fb0 con 0x7fed0c06bb10 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed12576700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c1095d0 0x7fed0c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed11d75700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c102500 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fed0800e9a0 tx=0x7fed0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed02ffd700 1 -- 172.21.15.26:0/197823541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0800ca10 con 0x7fed0c06bb10 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.729+0000 7fed02ffd700 1 -- 172.21.15.26:0/197823541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed0800cbb0 con 0x7fed0c06bb10 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.730+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c1a82a0 con 0x7fed0c06bb10 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.730+0000 7fed02ffd700 1 -- 172.21.15.26:0/197823541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08018680 con 0x7fed0c06bb10 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.730+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c1a87b0 con 0x7fed0c06bb10 2024-05-20T21:05:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.732+0000 7fed02ffd700 1 -- 172.21.15.26:0/197823541 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fed08010430 con 0x7fed0c06bb10 2024-05-20T21:05:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.733+0000 7fed02ffd700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fecf80b2930 0x7fecf80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.733+0000 7fed02ffd700 1 -- 172.21.15.26:0/197823541 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fed080e23b0 con 0x7fed0c06bb10 2024-05-20T21:05:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.733+0000 7fed11574700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fecf80b2930 0x7fecf80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 Processor -- start 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 -- start start 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed11574700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fecf80b2930 0x7fecf80b4df0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed0c103f90 tx=0x7fecfc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 0x555ad5868ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5863c90 0x555ad5864110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5864650 0x555ad58706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555ad57c1f10 con 0x555ad58687c0 2024-05-20T21:05:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555ad57b1ab0 con 0x555ad5863c90 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ad57b5af0 con 0x555ad5864650 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 0x555ad5868ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 0x555ad5868ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57640/0 (socket says 172.21.15.26:57640) 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed11d75700 1 -- 172.21.15.26:0/1983508553 learned_addr learned my addr 172.21.15.26:0/1983508553 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.734+0000 7fed11574700 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5863c90 0x555ad5864110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed12576700 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5864650 0x555ad58706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed11d75700 1 -- 172.21.15.26:0/1983508553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5864650 msgr2=0x555ad58706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed11d75700 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5864650 0x555ad58706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed11d75700 1 -- 172.21.15.26:0/1983508553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5863c90 msgr2=0x555ad5864110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed11d75700 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5863c90 0x555ad5864110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed11d75700 1 -- 172.21.15.26:0/1983508553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad575b150 con 0x555ad58687c0 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fed11d75700 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 0x555ad5868ba0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fed080a9670 tx=0x7fed080a9570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5a16b755d7555 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fecf7fff700 1 -- 172.21.15.26:0/1983508553 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed080a9970 con 0x555ad58687c0 2024-05-20T21:05:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fecf7fff700 1 -- 172.21.15.26:0/1983508553 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed08010e50 con 0x555ad58687c0 2024-05-20T21:05:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.735+0000 7fecf7fff700 1 -- 172.21.15.26:0/1983508553 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed080cc2b0 con 0x555ad58687c0 2024-05-20T21:05:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 -- 172.21.15.26:0/1983508553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 msgr2=0x555ad5868ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 0x555ad5868ba0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fed080a9670 tx=0x7fed080a9570 comp rx=0 tx=0).stop 2024-05-20T21:05:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 -- 172.21.15.26:0/1983508553 shutdown_connections 2024-05-20T21:05:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad58687c0 0x555ad5868ba0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5864650 0x555ad58706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/1983508553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5863c90 0x555ad5864110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 -- 172.21.15.26:0/1983508553 >> 172.21.15.26:0/1983508553 conn(0x555ad578f3e0 msgr2=0x555ad585e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 -- 172.21.15.26:0/1983508553 shutdown_connections 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.736+0000 7fed1e86fb80 1 -- 172.21.15.26:0/1983508553 wait complete. 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 -- start start 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 Processor -- start 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5863c90 0x555ad5870370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 0x555ad586d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad58687c0 0x555ad586dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555ad56b8140 con 0x555ad5864650 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555ad5719f40 con 0x555ad58687c0 2024-05-20T21:05:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ad5870d20 con 0x555ad5863c90 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed1e86fb80 10 client.0 _ll_register_callbacks cb 0x555ad574caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed11574700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 0x555ad586d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed12576700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad58687c0 0x555ad586dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed11574700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 0x555ad586d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57644/0 (socket says 172.21.15.26:57644) 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed11574700 1 -- 172.21.15.26:0/3768744678 learned_addr learned my addr 172.21.15.26:0/3768744678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.737+0000 7fed11d75700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5863c90 0x555ad5870370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed12576700 1 -- 172.21.15.26:0/3768744678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5863c90 msgr2=0x555ad5870370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed12576700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5863c90 0x555ad5870370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed12576700 1 -- 172.21.15.26:0/3768744678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 msgr2=0x555ad586d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed12576700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 0x555ad586d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed12576700 1 -- 172.21.15.26:0/3768744678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad5701a70 con 0x555ad58687c0 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed11574700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 0x555ad586d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed12576700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad58687c0 0x555ad586dbb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fed04004090 tx=0x7fed040040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f991185964da65 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fecf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fecf57fa700 1 -- 172.21.15.26:0/3768744678 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0401a070 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fecf57fa700 1 -- 172.21.15.26:0/3768744678 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fed04004db0 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed08010e50 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.738+0000 7fecf57fa700 1 -- 172.21.15.26:0/3768744678 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed04025050 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 10 client.27647 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ad576b900 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fecf57fa700 1 -- 172.21.15.26:0/3768744678 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed08010e50 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fecf57fa700 1 -- 172.21.15.26:0/3768744678 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fed0400e4f0 con 0x555ad58687c0 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 10 client.27647 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 10 client.27647 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 20 client.27647 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad587d8c0 0x555ad587fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 4 client.27647 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ad57c3e60 con 0x555ad587d8c0 2024-05-20T21:05:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.739+0000 7fed11d75700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad587d8c0 0x555ad587fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.740+0000 7fed11d75700 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad587d8c0 0x555ad587fce0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32d131efbd3bca7 server_cookie=33af7fec3f31b82e in_seq=0 out_seq=0 2024-05-20T21:05:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.740+0000 7fecf57fa700 10 client.27647 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fecf57fa700 1 -- 172.21.15.26:0/3768744678 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555ad57c3e60 con 0x555ad587d8c0 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fecf57fa700 10 client.27647 handle_command_reply: tid=0 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 client.27647 shutdown 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad58687c0 msgr2=0x555ad586dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad58687c0 0x555ad586dbb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fed04004090 tx=0x7fed040040c0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 shutdown_connections 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad587d8c0 0x555ad587fce0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad58687c0 0x555ad586dbb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad5863c90 0x555ad5870370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fecf57fa700 0 client.27647 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/3768744678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5864650 0x555ad586d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 >> 172.21.15.26:0/3768744678 conn(0x555ad578f3e0 msgr2=0x555ad5862c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 shutdown_connections 2024-05-20T21:05:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 1 -- 172.21.15.26:0/3768744678 wait complete. 2024-05-20T21:05:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.741+0000 7fed1e86fb80 20 client.27647 trim_cache size 0 max 16384 2024-05-20T21:05:27.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.765+0000 7fed1e86fb80 1 Processor -- start 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.765+0000 7fed1e86fb80 1 -- start start 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5858f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5862c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad58631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1b2800 con 0x555ad5863c90 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1b0090 con 0x555ad5864650 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1adee0 con 0x555ad58687c0 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5858f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5858f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57658/0 (socket says 172.21.15.26:57658) 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 -- 172.21.15.26:0/862793440 learned_addr learned my addr 172.21.15.26:0/862793440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11574700 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5862c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed12576700 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad58631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 -- 172.21.15.26:0/862793440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 msgr2=0x555ad58631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad58631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 -- 172.21.15.26:0/862793440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 msgr2=0x555ad5862c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5862c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 -- 172.21.15.26:0/862793440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad5870b50 con 0x555ad5863c90 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.766+0000 7fed11d75700 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5858f10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fed080e01c0 tx=0x7fed08023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f818c86e65be4c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fecf77fe700 1 -- 172.21.15.26:0/862793440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed080b27d0 con 0x555ad5863c90 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fecf77fe700 1 -- 172.21.15.26:0/862793440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed08010e50 con 0x555ad5863c90 2024-05-20T21:05:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fecf77fe700 1 -- 172.21.15.26:0/862793440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed080b2340 con 0x555ad5863c90 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 -- 172.21.15.26:0/862793440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 msgr2=0x555ad5858f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5858f10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fed080e01c0 tx=0x7fed08023040 comp rx=0 tx=0).stop 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 -- 172.21.15.26:0/862793440 shutdown_connections 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5858f10 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad58631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/862793440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5862c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.767+0000 7fed1e86fb80 1 -- 172.21.15.26:0/862793440 >> 172.21.15.26:0/862793440 conn(0x555ad578f3e0 msgr2=0x555ad5867870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 -- 172.21.15.26:0/862793440 shutdown_connections 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 -- 172.21.15.26:0/862793440 wait complete. 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 -- start start 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 Processor -- start 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5876150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5876690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.768+0000 7fed1e86fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad5872f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1ae550 con 0x555ad5863c90 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1af9c0 con 0x555ad5864650 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed1e86fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1aec20 con 0x555ad58687c0 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5876150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed1e86fb80 10 client.0 _ll_register_callbacks cb 0x555ad574caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5876150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57662/0 (socket says 172.21.15.26:57662) 2024-05-20T21:05:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 -- 172.21.15.26:0/58842892 learned_addr learned my addr 172.21.15.26:0/58842892 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11574700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5876690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed12576700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad5872f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 -- 172.21.15.26:0/58842892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 msgr2=0x555ad5872f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad5872f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 -- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 msgr2=0x555ad5876690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5876690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 -- 172.21.15.26:0/58842892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad5873690 con 0x555ad5863c90 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed12576700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad5872f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11574700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5876690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed11d75700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5876150 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fed080a9430 tx=0x7fed080adda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871a17ac67aa737d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fecf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fecf57fa700 1 -- 172.21.15.26:0/58842892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0806f060 con 0x555ad5863c90 2024-05-20T21:05:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fecf57fa700 1 -- 172.21.15.26:0/58842892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fed0806f200 con 0x555ad5863c90 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.769+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed08010e50 con 0x555ad5863c90 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fecf57fa700 1 -- 172.21.15.26:0/58842892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed0806f400 con 0x555ad5863c90 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 10 client.18261 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ad58e8e90 con 0x555ad5863c90 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fecf57fa700 1 -- 172.21.15.26:0/58842892 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed08010e50 con 0x555ad5863c90 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fecf57fa700 1 -- 172.21.15.26:0/58842892 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fed080a8bd0 con 0x555ad5863c90 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 10 client.18261 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 10 client.18261 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 20 client.18261 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad58e90c0 0x555ad58eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 4 client.18261 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.55, "await_for": 45.0}] 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.55, "await_for": 45.0}) -- 0x555ad57bd640 con 0x555ad58e90c0 2024-05-20T21:05:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.770+0000 7fed11574700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad58e90c0 0x555ad58eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:27.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.771+0000 7fecf57fa700 10 client.18261 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:27.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.771+0000 7fed11574700 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad58e90c0 0x555ad58eb4b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537ffe0d22017301 server_cookie=61908c56ab9a80d3 in_seq=0 out_seq=0 2024-05-20T21:05:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:27 smithi144 ceph-mon[35660]: pgmap v669: 97 pgs: 97 active+clean; 28 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-20T21:05:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fecf57fa700 1 -- 172.21.15.26:0/58842892 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ad57bd640 con 0x555ad58e90c0 2024-05-20T21:05:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fecf57fa700 10 client.18261 handle_command_reply: tid=0 2024-05-20T21:05:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 client.18261 shutdown 2024-05-20T21:05:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 msgr2=0x555ad5876150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5876150 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fed080a9430 tx=0x7fed080adda0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 shutdown_connections 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555ad58e90c0 0x555ad58eb4b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ad5863c90 0x555ad5876150 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ad58687c0 0x555ad5872f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fecf57fa700 0 client.18261 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 --2- 172.21.15.26:0/58842892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555ad5864650 0x555ad5876690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 >> 172.21.15.26:0/58842892 conn(0x555ad578f3e0 msgr2=0x555ad5862c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 shutdown_connections 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 1 -- 172.21.15.26:0/58842892 wait complete. 2024-05-20T21:05:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.927+0000 7fed1e86fb80 20 client.18261 trim_cache size 0 max 16384 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 38, 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "8724a85b": { 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 38, 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:05:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 146.6, 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fecf80b2930 msgr2=0x7fecf80b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fecf80b2930 0x7fecf80b4df0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed0c103f90 tx=0x7fecfc005fb0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 msgr2=0x7fed0c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c102500 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fed0800e9a0 tx=0x7fed0800ecb0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 shutdown_connections 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fecf80b2930 0x7fecf80b4df0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed0c06bb10 0x7fed0c102500 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed0c1095d0 0x7fed0c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 --2- 172.21.15.26:0/197823541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0c06c4d0 0x7fed0c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 >> 172.21.15.26:0/197823541 conn(0x7fed0c079480 msgr2=0x7fed0c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 shutdown_connections 2024-05-20T21:05:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:27.928+0000 7fed13fd9700 1 -- 172.21.15.26:0/197823541 wait complete. 2024-05-20T21:05:28.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.38 MB/sec execute 334 sec latency 647.815 ms 2024-05-20T21:05:28.767 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8724a85b', quiesce took 6.3 seconds. Will release after: 20.25 2024-05-20T21:05:29.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.38 MB/sec execute 335 sec latency 1647.927 ms 2024-05-20T21:05:29.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:29 smithi055 ceph-mon[35213]: pgmap v670: 97 pgs: 97 active+clean; 24 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-05-20T21:05:29.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:29 smithi026 ceph-mon[31990]: pgmap v670: 97 pgs: 97 active+clean; 24 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-05-20T21:05:29.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:29 smithi144 ceph-mon[35660]: pgmap v670: 97 pgs: 97 active+clean; 24 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-05-20T21:05:30.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.37 MB/sec execute 336 sec latency 2647.989 ms 2024-05-20T21:05:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:31 smithi055 ceph-mon[35213]: pgmap v671: 97 pgs: 97 active+clean; 25 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:05:31.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.36 MB/sec execute 337 sec latency 3648.102 ms 2024-05-20T21:05:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:31 smithi026 ceph-mon[31990]: pgmap v671: 97 pgs: 97 active+clean; 25 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:05:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:31 smithi144 ceph-mon[35660]: pgmap v671: 97 pgs: 97 active+clean; 25 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:05:32.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.35 MB/sec execute 338 sec latency 4648.153 ms 2024-05-20T21:05:33.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.35 MB/sec execute 339 sec latency 5648.265 ms 2024-05-20T21:05:33.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:33 smithi055 ceph-mon[35213]: pgmap v672: 97 pgs: 97 active+clean; 25 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:05:33.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:33 smithi026 ceph-mon[31990]: pgmap v672: 97 pgs: 97 active+clean; 25 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:05:33.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:33 smithi144 ceph-mon[35660]: pgmap v672: 97 pgs: 97 active+clean; 25 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:05:34.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.34 MB/sec execute 340 sec latency 6648.377 ms 2024-05-20T21:05:35.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.33 MB/sec execute 341 sec latency 7648.456 ms 2024-05-20T21:05:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:35 smithi055 ceph-mon[35213]: pgmap v673: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-20T21:05:35.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:35 smithi026 ceph-mon[31990]: pgmap v673: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-20T21:05:35.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:35 smithi144 ceph-mon[35660]: pgmap v673: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-20T21:05:36.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.33 MB/sec execute 342 sec latency 8648.552 ms 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 -- 172.21.15.26:0/1888624668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958103610 msgr2=0x7f7958109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 --2- 172.21.15.26:0/1888624668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958103610 0x7f7958109ea0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002a410 comp rx=0 tx=0).stop 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 -- 172.21.15.26:0/1888624668 shutdown_connections 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 --2- 172.21.15.26:0/1888624668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958103610 0x7f7958109ea0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 --2- 172.21.15.26:0/1888624668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958102c50 0x7f79581030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 --2- 172.21.15.26:0/1888624668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 0x7f7958101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 -- 172.21.15.26:0/1888624668 >> 172.21.15.26:0/1888624668 conn(0x7f79580fcfd0 msgr2=0x7f79580ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 -- 172.21.15.26:0/1888624668 shutdown_connections 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.373+0000 7f795ec6b640 1 -- 172.21.15.26:0/1888624668 wait complete. 2024-05-20T21:05:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 Processor -- start 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 -- start start 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 0x7f795810c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 0x7f795810aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958103610 0x7f795810af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f795810e4f0 con 0x7f7958102c50 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f795810e370 con 0x7f7958103610 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f795ec6b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f795810e670 con 0x7f7958101a60 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f7957fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 0x7f795810aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f7957fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 0x7f795810aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54646/0 (socket says 172.21.15.26:54646) 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.374+0000 7f7957fff640 1 -- 172.21.15.26:0/3082815491 learned_addr learned my addr 172.21.15.26:0/3082815491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f795d208640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958103610 0x7f795810af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f795ca07640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 0x7f795810c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7957fff640 1 -- 172.21.15.26:0/3082815491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 msgr2=0x7f795810c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7957fff640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 0x7f795810c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7957fff640 1 -- 172.21.15.26:0/3082815491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958103610 msgr2=0x7f795810af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7957fff640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958103610 0x7f795810af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7957fff640 1 -- 172.21.15.26:0/3082815491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795810b7f0 con 0x7f7958102c50 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f795d208640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958103610 0x7f795810af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f795ca07640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 0x7f795810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7957fff640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 0x7f795810aa20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f794800ebc0 tx=0x7f794800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7955ffb640 1 -- 172.21.15.26:0/3082815491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948010630 con 0x7f7958102c50 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7955ffb640 1 -- 172.21.15.26:0/3082815491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7948010c60 con 0x7f7958102c50 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79581a2c10 con 0x7f7958102c50 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f7955ffb640 1 -- 172.21.15.26:0/3082815491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948018b00 con 0x7f7958102c50 2024-05-20T21:05:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.375+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a31b0 con 0x7f7958102c50 2024-05-20T21:05:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7955ffb640 1 -- 172.21.15.26:0/3082815491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7948018ca0 con 0x7f7958102c50 2024-05-20T21:05:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 Processor -- start 2024-05-20T21:05:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 -- start start 2024-05-20T21:05:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639962fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399634bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x5563996376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5563995a0c30 con 0x55639962f6c0 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5563995907d0 con 0x5563996300b0 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7961988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556399594810 con 0x556399635220 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f795d208640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639962fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f795d208640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639962fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54656/0 (socket says 172.21.15.26:54656) 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f795d208640 1 -- 172.21.15.26:0/673916182 learned_addr learned my addr 172.21.15.26:0/673916182 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f795ca07640 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399634bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.378+0000 7f7957fff640 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x5563996376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795d208640 1 -- 172.21.15.26:0/673916182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 msgr2=0x5563996376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795d208640 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x5563996376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795d208640 1 -- 172.21.15.26:0/673916182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 msgr2=0x556399634bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795d208640 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399634bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795d208640 1 -- 172.21.15.26:0/673916182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563992a5210 con 0x55639962f6c0 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795ca07640 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399634bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f7955ffb640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f79440b2920 0x7f79440b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795d208640 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639962fae0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f7950031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b713c07c07484a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f795ca07640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f79440b2920 0x7f79440b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f7955ffb640 1 -- 172.21.15.26:0/3082815491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f79480e2900 con 0x7f7958102c50 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f794effd640 1 -- 172.21.15.26:0/673916182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950035030 con 0x55639962f6c0 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f794effd640 1 -- 172.21.15.26:0/673916182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7950004030 con 0x55639962f6c0 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.379+0000 7f794effd640 1 -- 172.21.15.26:0/673916182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950003a60 con 0x55639962f6c0 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 -- 172.21.15.26:0/673916182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 msgr2=0x55639962fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f795ca07640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f79440b2920 0x7f79440b4de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x5563996304b0 tx=0x7f7940005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639962fae0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f7950031580 comp rx=0 tx=0).stop 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 -- 172.21.15.26:0/673916182 shutdown_connections 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x5563996376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399634bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 --2- 172.21.15.26:0/673916182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639962fae0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 -- 172.21.15.26:0/673916182 >> 172.21.15.26:0/673916182 conn(0x55639962ad30 msgr2=0x55639962b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 -- 172.21.15.26:0/673916182 shutdown_connections 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 -- 172.21.15.26:0/673916182 wait complete. 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.380+0000 7f7961988740 1 -- start start 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 Processor -- start 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55639962f6c0 0x556399639790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 0x556399639cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x556399640d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556399276790 con 0x5563996300b0 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55639954c890 con 0x55639962f6c0 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556399638170 con 0x556399635220 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7961988740 10 client.0 _ll_register_callbacks cb 0x5563992a4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7957fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 0x556399639cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7957fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 0x556399639cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54672/0 (socket says 172.21.15.26:54672) 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f7957fff640 1 -- 172.21.15.26:0/156240869 learned_addr learned my addr 172.21.15.26:0/156240869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f795ca07640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55639962f6c0 0x556399639790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.381+0000 7f795d208640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x556399640d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7957fff640 1 -- 172.21.15.26:0/156240869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 msgr2=0x556399640d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7957fff640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x556399640d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7957fff640 1 -- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55639962f6c0 msgr2=0x556399639790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7957fff640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55639962f6c0 0x556399639790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7957fff640 1 -- 172.21.15.26:0/156240869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556399371620 con 0x5563996300b0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f795ca07640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55639962f6c0 0x556399639790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7957fff640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 0x556399639cd0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f79480a7130 tx=0x7f794800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e96f92c99266f0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7937fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7937fff640 1 -- 172.21.15.26:0/156240869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480cc040 con 0x5563996300b0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7937fff640 1 -- 172.21.15.26:0/156240869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f79480109e0 con 0x5563996300b0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950004030 con 0x5563996300b0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7937fff640 1 -- 172.21.15.26:0/156240869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79480a9630 con 0x5563996300b0 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7961988740 10 client.18273 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556399643d60 con 0x5563996300b0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7937fff640 1 -- 172.21.15.26:0/156240869 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7950004030 con 0x5563996300b0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7937fff640 1 -- 172.21.15.26:0/156240869 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f79480a5830 con 0x5563996300b0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.382+0000 7f7961988740 10 client.18273 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f7961988740 10 client.18273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f7961988740 20 client.18273 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f7961988740 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996453e0 0x556399647840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f7961988740 4 client.18273 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563995a2b80 con 0x5563996453e0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f795ca07640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996453e0 0x556399647840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f795ca07640 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996453e0 0x556399647840 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6279b242c0e85 server_cookie=f2a2d1bf746af4d in_seq=0 out_seq=0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.383+0000 7f7937fff640 10 client.18273 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7937fff640 1 -- 172.21.15.26:0/156240869 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5563995a2b80 con 0x5563996453e0 2024-05-20T21:05:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7937fff640 10 client.18273 handle_command_reply: tid=0 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 client.18273 shutdown 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 msgr2=0x556399639cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 0x556399639cd0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f79480a7130 tx=0x7f794800ef90 comp rx=0 tx=0).stop 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 shutdown_connections 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996453e0 0x556399647840 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x556399640d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563996300b0 0x556399639cd0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7937fff640 0 client.18273 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 --2- 172.21.15.26:0/156240869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55639962f6c0 0x556399639790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 >> 172.21.15.26:0/156240869 conn(0x55639962ad30 msgr2=0x556399629fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.384+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 shutdown_connections 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.385+0000 7f7961988740 1 -- 172.21.15.26:0/156240869 wait complete. 2024-05-20T21:05:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.385+0000 7f7961988740 20 client.18273 trim_cache size 0 max 16384 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 Processor -- start 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 -- start start 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55639962f6c0 0x55639962e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399640cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 0x55639964c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79581aea10 con 0x556399635220 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f79581ac2e0 con 0x5563996300b0 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f7961988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79581aa130 con 0x55639962f6c0 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f795d208640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 0x55639964c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f795d208640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 0x55639964c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54688/0 (socket says 172.21.15.26:54688) 2024-05-20T21:05:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.400+0000 7f795d208640 1 -- 172.21.15.26:0/1250485031 learned_addr learned my addr 172.21.15.26:0/1250485031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795d208640 1 -- 172.21.15.26:0/1250485031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55639962f6c0 msgr2=0x55639962e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795ca07640 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55639962f6c0 0x55639962e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f7957fff640 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399640cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795d208640 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55639962f6c0 0x55639962e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795d208640 1 -- 172.21.15.26:0/1250485031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 msgr2=0x556399640cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795d208640 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399640cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795d208640 1 -- 172.21.15.26:0/1250485031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556399630940 con 0x556399635220 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f795d208640 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 0x55639964c100 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f7950031900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e20f4011265baae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f794e7fc640 1 -- 172.21.15.26:0/1250485031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950035040 con 0x556399635220 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f794e7fc640 1 -- 172.21.15.26:0/1250485031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7950032bc0 con 0x556399635220 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f794e7fc640 1 -- 172.21.15.26:0/1250485031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950003c10 con 0x556399635220 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.401+0000 7f7961988740 1 -- 172.21.15.26:0/1250485031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 msgr2=0x55639964c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 0x55639964c100 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f7950031900 comp rx=0 tx=0).stop 2024-05-20T21:05:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- 172.21.15.26:0/1250485031 shutdown_connections 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556399635220 0x55639964c100 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x556399640cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- 172.21.15.26:0/1250485031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55639962f6c0 0x55639962e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- 172.21.15.26:0/1250485031 >> 172.21.15.26:0/1250485031 conn(0x55639962ad30 msgr2=0x55639962dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- 172.21.15.26:0/1250485031 shutdown_connections 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- 172.21.15.26:0/1250485031 wait complete. 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- start start 2024-05-20T21:05:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 Processor -- start 2024-05-20T21:05:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639963e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x55639963e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x55639963f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79581aa7a0 con 0x55639962f6c0 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f79581abc10 con 0x5563996300b0 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79581aae70 con 0x556399635220 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f7961988740 10 client.0 _ll_register_callbacks cb 0x7f794800eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f795ca07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639963e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.402+0000 7f795ca07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639963e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54704/0 (socket says 172.21.15.26:54704) 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 -- 172.21.15.26:0/1702608963 learned_addr learned my addr 172.21.15.26:0/1702608963 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7957fff640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x55639963e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 -- 172.21.15.26:0/1702608963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 msgr2=0x55639963f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795d208640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x55639963f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x55639963f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 -- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 msgr2=0x55639963e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x55639963e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 -- 172.21.15.26:0/1702608963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55639963fca0 con 0x55639962f6c0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7957fff640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x55639963e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f795ca07640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639963e160 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f794002cfd0 tx=0x7f7940004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf8f63185f6546e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7937fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7937fff640 1 -- 172.21.15.26:0/1702608963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940003b80 con 0x55639962f6c0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7937fff640 1 -- 172.21.15.26:0/1702608963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7940003d20 con 0x55639962f6c0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950032bc0 con 0x55639962f6c0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7937fff640 1 -- 172.21.15.26:0/1702608963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f794003c050 con 0x55639962f6c0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7961988740 10 client.18279 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.403+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556399641080 con 0x55639962f6c0 2024-05-20T21:05:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7937fff640 1 -- 172.21.15.26:0/1702608963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7950032bc0 con 0x55639962f6c0 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7937fff640 1 -- 172.21.15.26:0/1702608963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f79400215a0 con 0x55639962f6c0 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7961988740 10 client.18279 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7961988740 10 client.18279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7961988740 20 client.18279 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7961988740 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996b07a0 0x5563996b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7961988740 4 client.18279 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55639959c360 con 0x5563996b07a0 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7957fff640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996b07a0 0x5563996b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7937fff640 10 client.18279 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.404+0000 7f7957fff640 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996b07a0 0x5563996b2bc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4076993b52ebb2c server_cookie=6be447c180b67959 in_seq=0 out_seq=0 2024-05-20T21:05:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7937fff640 1 -- 172.21.15.26:0/1702608963 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55639959c360 con 0x5563996b07a0 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7937fff640 10 client.18279 handle_command_reply: tid=0 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 client.18279 shutdown 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 msgr2=0x55639963e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639963e160 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f794002cfd0 tx=0x7f7940004730 comp rx=0 tx=0).stop 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 shutdown_connections 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5563996b07a0 0x5563996b2bc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556399635220 0x55639963f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5563996300b0 0x55639963e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7937fff640 0 client.18279 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 --2- 172.21.15.26:0/1702608963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55639962f6c0 0x55639963e160 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 >> 172.21.15.26:0/1702608963 conn(0x55639962ad30 msgr2=0x556399629720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 shutdown_connections 2024-05-20T21:05:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.434+0000 7f7961988740 1 -- 172.21.15.26:0/1702608963 wait complete. 2024-05-20T21:05:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f7961988740 20 client.18279 trim_cache size 0 max 16384 2024-05-20T21:05:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:05:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:05:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:05:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f79440b2920 msgr2=0x7f79440b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f79440b2920 0x7f79440b4de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x5563996304b0 tx=0x7f7940005f90 comp rx=0 tx=0).stop 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 msgr2=0x7f795810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 0x7f795810aa20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f794800ebc0 tx=0x7f794800c6a0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 shutdown_connections 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f79440b2920 0x7f79440b4de0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7958103610 0x7f795810af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7958102c50 0x7f795810aa20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 --2- 172.21.15.26:0/3082815491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958101a60 0x7f795810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 >> 172.21.15.26:0/3082815491 conn(0x7f79580fcfd0 msgr2=0x7f7958105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 shutdown_connections 2024-05-20T21:05:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:37.435+0000 7f795ec6b640 1 -- 172.21.15.26:0/3082815491 wait complete. 2024-05-20T21:05:37.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.32 MB/sec execute 343 sec latency 9648.668 ms 2024-05-20T21:05:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:37 smithi055 ceph-mon[35213]: pgmap v674: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 871 KiB/s wr, 75 op/s 2024-05-20T21:05:37.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:37 smithi026 ceph-mon[31990]: pgmap v674: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 871 KiB/s wr, 75 op/s 2024-05-20T21:05:37.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:37 smithi144 ceph-mon[35660]: pgmap v674: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 871 KiB/s wr, 75 op/s 2024-05-20T21:05:38.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.31 MB/sec execute 344 sec latency 10648.770 ms 2024-05-20T21:05:39.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.31 MB/sec execute 345 sec latency 11648.903 ms 2024-05-20T21:05:39.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:39 smithi055 ceph-mon[35213]: pgmap v675: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 677 KiB/s wr, 68 op/s 2024-05-20T21:05:39.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:39 smithi026 ceph-mon[31990]: pgmap v675: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 677 KiB/s wr, 68 op/s 2024-05-20T21:05:39.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:39 smithi144 ceph-mon[35660]: pgmap v675: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 677 KiB/s wr, 68 op/s 2024-05-20T21:05:40.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.30 MB/sec execute 346 sec latency 12649.019 ms 2024-05-20T21:05:41.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.29 MB/sec execute 347 sec latency 13649.111 ms 2024-05-20T21:05:41.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:41 smithi055 ceph-mon[35213]: pgmap v676: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 392 KiB/s wr, 34 op/s 2024-05-20T21:05:41.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:41 smithi026 ceph-mon[31990]: pgmap v676: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 392 KiB/s wr, 34 op/s 2024-05-20T21:05:41.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:41 smithi144 ceph-mon[35660]: pgmap v676: 97 pgs: 97 active+clean; 25 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 392 KiB/s wr, 34 op/s 2024-05-20T21:05:42.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.29 MB/sec execute 348 sec latency 14649.235 ms 2024-05-20T21:05:43.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.28 MB/sec execute 349 sec latency 15649.362 ms 2024-05-20T21:05:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:43 smithi055 ceph-mon[35213]: pgmap v677: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 188 KiB/s wr, 27 op/s 2024-05-20T21:05:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:43 smithi026 ceph-mon[31990]: pgmap v677: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 188 KiB/s wr, 27 op/s 2024-05-20T21:05:43.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:43 smithi144 ceph-mon[35660]: pgmap v677: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 188 KiB/s wr, 27 op/s 2024-05-20T21:05:44.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.27 MB/sec execute 350 sec latency 16649.451 ms 2024-05-20T21:05:45.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.27 MB/sec execute 351 sec latency 17649.586 ms 2024-05-20T21:05:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:45 smithi055 ceph-mon[35213]: pgmap v678: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-05-20T21:05:45.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:45 smithi026 ceph-mon[31990]: pgmap v678: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-05-20T21:05:45.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:45 smithi144 ceph-mon[35660]: pgmap v678: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-05-20T21:05:46.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.26 MB/sec execute 352 sec latency 18649.691 ms 2024-05-20T21:05:47.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.25 MB/sec execute 353 sec latency 19649.811 ms 2024-05-20T21:05:47.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:47 smithi055 ceph-mon[35213]: pgmap v679: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.2 KiB/s wr, 17 op/s 2024-05-20T21:05:47.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:47 smithi026 ceph-mon[31990]: pgmap v679: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.2 KiB/s wr, 17 op/s 2024-05-20T21:05:47.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:47 smithi144 ceph-mon[35660]: pgmap v679: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.2 KiB/s wr, 17 op/s 2024-05-20T21:05:48.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.25 MB/sec execute 354 sec latency 20649.932 ms 2024-05-20T21:05:49.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:05:49.284 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:49.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.24 MB/sec execute 355 sec latency 21649.985 ms 2024-05-20T21:05:49.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:49 smithi026 ceph-mon[31990]: pgmap v680: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.0 KiB/s wr, 18 op/s 2024-05-20T21:05:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:49 smithi055 ceph-mon[35213]: pgmap v680: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.0 KiB/s wr, 18 op/s 2024-05-20T21:05:49.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:49 smithi144 ceph-mon[35660]: pgmap v680: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.0 KiB/s wr, 18 op/s 2024-05-20T21:05:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 -- 172.21.15.26:0/1407707763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0105e90 msgr2=0x7fccc0108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 --2- 172.21.15.26:0/1407707763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0105e90 0x7fccc0108280 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009900 tx=0x7fcca802a410 comp rx=0 tx=0).stop 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 -- 172.21.15.26:0/1407707763 shutdown_connections 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 --2- 172.21.15.26:0/1407707763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc01087c0 0x7fccc010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 --2- 172.21.15.26:0/1407707763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0105e90 0x7fccc0108280 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 --2- 172.21.15.26:0/1407707763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc0103560 0x7fccc0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 -- 172.21.15.26:0/1407707763 >> 172.21.15.26:0/1407707763 conn(0x7fccc00fcf50 msgr2=0x7fccc00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 -- 172.21.15.26:0/1407707763 shutdown_connections 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 -- 172.21.15.26:0/1407707763 wait complete. 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 Processor -- start 2024-05-20T21:05:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.537+0000 7fccc5324640 1 -- start start 2024-05-20T21:05:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccc5324640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 0x7fccc0102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccc5324640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc0105e90 0x7fccc0101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccc5324640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc01087c0 0x7fccc0101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccc5324640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010e4c0 con 0x7fccc0103560 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccc5324640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010e340 con 0x7fccc01087c0 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccc5324640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010e640 con 0x7fccc0105e90 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 0x7fccc0102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 0x7fccc0102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53454/0 (socket says 172.21.15.26:53454) 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 -- 172.21.15.26:0/2820372863 learned_addr learned my addr 172.21.15.26:0/2820372863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccb7fff640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc0105e90 0x7fccc0101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 -- 172.21.15.26:0/2820372863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc0105e90 msgr2=0x7fccc0101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc0105e90 0x7fccc0101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc01087c0 0x7fccc0101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 -- 172.21.15.26:0/2820372863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc01087c0 msgr2=0x7fccc0101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc01087c0 0x7fccc0101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 -- 172.21.15.26:0/2820372863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc01022e0 con 0x7fccc0103560 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc01087c0 0x7fccc0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbeffd640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 0x7fccc0102ed0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fccb000d950 tx=0x7fccb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.538+0000 7fccbcff9640 1 -- 172.21.15.26:0/2820372863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0017070 con 0x7fccc0103560 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.539+0000 7fccbcff9640 1 -- 172.21.15.26:0/2820372863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccb0009c30 con 0x7fccc0103560 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.539+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc01a2cb0 con 0x7fccc0103560 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.539+0000 7fccbcff9640 1 -- 172.21.15.26:0/2820372863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb000f620 con 0x7fccc0103560 2024-05-20T21:05:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.539+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc01a31d0 con 0x7fccc0103560 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.541+0000 7fccbcff9640 1 -- 172.21.15.26:0/2820372863 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fccb0010460 con 0x7fccc0103560 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.541+0000 7fccc8043740 1 Processor -- start 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.541+0000 7fccc8043740 1 -- start start 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccc8043740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f1f6c0 0x559865f1fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccc8043740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccc8043740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f25220 0x559865f276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccc8043740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559865e90c30 con 0x559865f200b0 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccc8043740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559865e807d0 con 0x559865f1f6c0 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccc8043740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559865e84810 con 0x559865f25220 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f24bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53464/0 (socket says 172.21.15.26:53464) 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 -- 172.21.15.26:0/4072728788 learned_addr learned my addr 172.21.15.26:0/4072728788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f1f6c0 0x559865f1fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccb7fff640 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f25220 0x559865f276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 -- 172.21.15.26:0/4072728788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f25220 msgr2=0x559865f276b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f25220 0x559865f276b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 -- 172.21.15.26:0/4072728788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f1f6c0 msgr2=0x559865f1fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f1f6c0 0x559865f1fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbeffd640 1 -- 172.21.15.26:0/4072728788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559865b95210 con 0x559865f200b0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.542+0000 7fccbcff9640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccac0b28a0 0x7fccac0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.543+0000 7fccbeffd640 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f24bb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fccb00ea160 tx=0x7fccb00ea620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581ee41d47aa6a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.543+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccac0b28a0 0x7fccac0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.543+0000 7fccbcff9640 1 -- 172.21.15.26:0/2820372863 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fccb00e1f30 con 0x7fccc0103560 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.543+0000 7fccb57fa640 1 -- 172.21.15.26:0/4072728788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a53b0 con 0x559865f200b0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.543+0000 7fccb57fa640 1 -- 172.21.15.26:0/4072728788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccb00eae50 con 0x559865f200b0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.543+0000 7fccb57fa640 1 -- 172.21.15.26:0/4072728788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a8c60 con 0x559865f200b0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 -- 172.21.15.26:0/4072728788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 msgr2=0x559865f24bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccac0b28a0 0x7fccac0b4d60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x559865f20910 tx=0x7fccb8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f24bb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fccb00ea160 tx=0x7fccb00ea620 comp rx=0 tx=0).stop 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 -- 172.21.15.26:0/4072728788 shutdown_connections 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f25220 0x559865f276b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f24bb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 --2- 172.21.15.26:0/4072728788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f1f6c0 0x559865f1fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 -- 172.21.15.26:0/4072728788 >> 172.21.15.26:0/4072728788 conn(0x559865f1ad30 msgr2=0x559865f1b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 -- 172.21.15.26:0/4072728788 shutdown_connections 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 -- 172.21.15.26:0/4072728788 wait complete. 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 -- start start 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 Processor -- start 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.544+0000 7fccc8043740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccc8043740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccc8043740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559865b66790 con 0x559865f200b0 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccc8043740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559865e3c890 con 0x559865f25220 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccc8043740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559865f281d0 con 0x559865f1f6c0 2024-05-20T21:05:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccc8043740 10 client.0 _ll_register_callbacks cb 0x559865b94a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f29d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53480/0 (socket says 172.21.15.26:53480) 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 -- 172.21.15.26:0/1724238704 learned_addr learned my addr 172.21.15.26:0/1724238704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccbeffd640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 -- 172.21.15.26:0/1724238704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 msgr2=0x559865f297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 -- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 msgr2=0x559865f30d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 -- 172.21.15.26:0/1724238704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559865c61620 con 0x559865f200b0 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccb7fff640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f29d00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fcca8000c00 tx=0x7fcca8031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b0cd5b76323748 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fcca67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fcca67fc640 1 -- 172.21.15.26:0/1724238704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8035030 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fcca67fc640 1 -- 172.21.15.26:0/1724238704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcca8004240 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.545+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb00eae50 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fcca67fc640 1 -- 172.21.15.26:0/1724238704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcca8004440 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 10 client.18291 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559865f33d90 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fcca67fc640 1 -- 172.21.15.26:0/1724238704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccb00eae50 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fcca67fc640 1 -- 172.21.15.26:0/1724238704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcca804d050 con 0x559865f200b0 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 10 client.18291 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 20 client.18291 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865f354d0 0x559865f378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 4 client.18291 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559865e92b80 con 0x559865f354d0 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.546+0000 7fccbeffd640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865f354d0 0x559865f378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.547+0000 7fcca67fc640 10 client.18291 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.547+0000 7fccbeffd640 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865f354d0 0x559865f378c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d8593d3035d54 server_cookie=735a8d04ae57cf5e in_seq=0 out_seq=0 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fcca67fc640 1 -- 172.21.15.26:0/1724238704 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559865e92b80 con 0x559865f354d0 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fcca67fc640 10 client.18291 handle_command_reply: tid=0 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 client.18291 shutdown 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 msgr2=0x559865f29d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f29d00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fcca8000c00 tx=0x7fcca8031820 comp rx=0 tx=0).stop 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 shutdown_connections 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865f354d0 0x559865f378c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f29d00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fcca67fc640 0 client.18291 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 --2- 172.21.15.26:0/1724238704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 >> 172.21.15.26:0/1724238704 conn(0x559865f1ad30 msgr2=0x559865f19fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 shutdown_connections 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 1 -- 172.21.15.26:0/1724238704 wait complete. 2024-05-20T21:05:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.548+0000 7fccc8043740 20 client.18291 trim_cache size 0 max 16384 2024-05-20T21:05:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 Processor -- start 2024-05-20T21:05:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 -- start start 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f21300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01ae970 con 0x559865f200b0 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01ac240 con 0x559865f25220 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccc8043740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01aa090 con 0x559865f1f6c0 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f21840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f21840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53482/0 (socket says 172.21.15.26:53482) 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 -- 172.21.15.26:0/1096242410 learned_addr learned my addr 172.21.15.26:0/1096242410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccbeffd640 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f21300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 -- 172.21.15.26:0/1096242410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 msgr2=0x559865f21300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f21300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 -- 172.21.15.26:0/1096242410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 msgr2=0x559865f30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.564+0000 7fccb7fff640 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.565+0000 7fccb7fff640 1 -- 172.21.15.26:0/1096242410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559865f1ab60 con 0x559865f200b0 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.565+0000 7fccbeffd640 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f21300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.565+0000 7fccb7fff640 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f21840 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fcca803c910 tx=0x7fcca8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab91d420b7b6b012 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.565+0000 7fccb4ff9640 1 -- 172.21.15.26:0/1096242410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8053030 con 0x559865f200b0 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.565+0000 7fccb4ff9640 1 -- 172.21.15.26:0/1096242410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcca8040070 con 0x559865f200b0 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.565+0000 7fccb4ff9640 1 -- 172.21.15.26:0/1096242410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8035030 con 0x559865f200b0 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- 172.21.15.26:0/1096242410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 msgr2=0x559865f21840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f21840 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fcca803c910 tx=0x7fcca8009520 comp rx=0 tx=0).stop 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- 172.21.15.26:0/1096242410 shutdown_connections 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f200b0 0x559865f21840 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- 172.21.15.26:0/1096242410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f1f6c0 0x559865f21300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- 172.21.15.26:0/1096242410 >> 172.21.15.26:0/1096242410 conn(0x559865f1ad30 msgr2=0x559865f15e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- 172.21.15.26:0/1096242410 shutdown_connections 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- 172.21.15.26:0/1096242410 wait complete. 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- start start 2024-05-20T21:05:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 Processor -- start 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 0x559865f2e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f200b0 0x559865f2e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01aa700 con 0x559865f1f6c0 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01abb70 con 0x559865f25220 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01aadd0 con 0x559865f200b0 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.566+0000 7fccc8043740 10 client.0 _ll_register_callbacks cb 0x7fccb80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 0x559865f2e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 0x559865f2e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53484/0 (socket says 172.21.15.26:53484) 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 -- 172.21.15.26:0/3112388726 learned_addr learned my addr 172.21.15.26:0/3112388726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbf7fe640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccb7fff640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f200b0 0x559865f2e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 -- 172.21.15.26:0/3112388726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f200b0 msgr2=0x559865f2e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f200b0 0x559865f2e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 -- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 msgr2=0x559865f2f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f2f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 -- 172.21.15.26:0/3112388726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559865f2fd90 con 0x559865f1f6c0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccbeffd640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 0x559865f2e250 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fccb00a91f0 tx=0x7fccb00e2270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc095857231ea94 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fcca67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fcca67fc640 1 -- 172.21.15.26:0/3112388726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00c3060 con 0x559865f1f6c0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fcca67fc640 1 -- 172.21.15.26:0/3112388726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fccb0091030 con 0x559865f1f6c0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.567+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca8040070 con 0x559865f1f6c0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fcca67fc640 1 -- 172.21.15.26:0/3112388726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccb0020040 con 0x559865f1f6c0 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 10 client.18297 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559865fa0080 con 0x559865f1f6c0 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fcca67fc640 1 -- 172.21.15.26:0/3112388726 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcca8040070 con 0x559865f1f6c0 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fcca67fc640 1 -- 172.21.15.26:0/3112388726 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fccb00e5ac0 con 0x559865f1f6c0 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 10 client.18297 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 10 client.18297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 20 client.18297 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865fa0410 0x559865fa2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 4 client.18297 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.568+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559865e8c360 con 0x559865fa0410 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.569+0000 7fccb7fff640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865fa0410 0x559865fa2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.569+0000 7fcca67fc640 10 client.18297 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.569+0000 7fccb7fff640 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865fa0410 0x559865fa2800 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1500e80208fc6dc6 server_cookie=e697877db2262171 in_seq=0 out_seq=0 2024-05-20T21:05:50.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.24 MB/sec execute 356 sec latency 22650.024 ms 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fcca67fc640 1 -- 172.21.15.26:0/3112388726 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559865e8c360 con 0x559865fa0410 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fcca67fc640 10 client.18297 handle_command_reply: tid=0 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 client.18297 shutdown 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 msgr2=0x559865f2e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 0x559865f2e250 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fccb00a91f0 tx=0x7fccb00e2270 comp rx=0 tx=0).stop 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 shutdown_connections 2024-05-20T21:05:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559865fa0410 0x559865fa2800 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559865f25220 0x559865f2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559865f200b0 0x559865f2e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fcca67fc640 0 client.18297 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 --2- 172.21.15.26:0/3112388726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559865f1f6c0 0x559865f2e250 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.584+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 >> 172.21.15.26:0/3112388726 conn(0x559865f1ad30 msgr2=0x559865f29690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 shutdown_connections 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc8043740 1 -- 172.21.15.26:0/3112388726 wait complete. 2024-05-20T21:05:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc8043740 20 client.18297 trim_cache size 0 max 16384 2024-05-20T21:05:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:05:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:05:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccac0b28a0 msgr2=0x7fccac0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccac0b28a0 0x7fccac0b4d60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x559865f20910 tx=0x7fccb8009500 comp rx=0 tx=0).stop 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 msgr2=0x7fccc0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 0x7fccc0102ed0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fccb000d950 tx=0x7fccb000de10 comp rx=0 tx=0).stop 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 shutdown_connections 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccac0b28a0 0x7fccac0b4d60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc01087c0 0x7fccc0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc0105e90 0x7fccc0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 --2- 172.21.15.26:0/2820372863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0103560 0x7fccc0102ed0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 >> 172.21.15.26:0/2820372863 conn(0x7fccc00fcf50 msgr2=0x7fccc0104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.585+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 shutdown_connections 2024-05-20T21:05:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:05:50.586+0000 7fccc5324640 1 -- 172.21.15.26:0/2820372863 wait complete. 2024-05-20T21:05:51.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 -- 172.21.15.26:0/689563564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 msgr2=0x7f6b7c1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/689563564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c1070a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6b700099c0 tx=0x7f6b70009cd0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 -- 172.21.15.26:0/689563564 shutdown_connections 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/689563564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7c1077d0 0x7f6b7c10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/689563564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 0x7f6b7c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/689563564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c1070a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 -- 172.21.15.26:0/689563564 >> 172.21.15.26:0/689563564 conn(0x7f6b7c0793f0 msgr2=0x7f6b7c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 -- 172.21.15.26:0/689563564 shutdown_connections 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.110+0000 7f6b82c8d700 1 -- 172.21.15.26:0/689563564 wait complete. 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 Processor -- start 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 -- start start 2024-05-20T21:05:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 0x7f6b7c104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7c1077d0 0x7f6b7c104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1124c0 con 0x7f6b7c103430 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c112340 con 0x7f6b7c1077d0 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b82c8d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c112640 con 0x7f6b7c102a80 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b7bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b7bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53494/0 (socket says 172.21.15.26:53494) 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b7bfff700 1 -- 172.21.15.26:0/2420871697 learned_addr learned my addr 172.21.15.26:0/2420871697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b8122a700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7c1077d0 0x7f6b7c104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.111+0000 7f6b80a29700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 0x7f6b7c104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b7bfff700 1 -- 172.21.15.26:0/2420871697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 msgr2=0x7f6b7c104140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b7bfff700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 0x7f6b7c104140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b7bfff700 1 -- 172.21.15.26:0/2420871697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7c1077d0 msgr2=0x7f6b7c104bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b7bfff700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7c1077d0 0x7f6b7c104bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b7bfff700 1 -- 172.21.15.26:0/2420871697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c1a80d0 con 0x7f6b7c103430 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b80a29700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 0x7f6b7c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b7bfff700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c104680 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6b700098e0 tx=0x7f6b700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70027070 con 0x7f6b7c103430 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b70003fb0 con 0x7f6b7c103430 2024-05-20T21:05:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c1a8360 con 0x7f6b7c103430 2024-05-20T21:05:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70022c40 con 0x7f6b7c103430 2024-05-20T21:05:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.112+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1a8810 con 0x7f6b7c103430 2024-05-20T21:05:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.113+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c103b80 con 0x7f6b7c103430 2024-05-20T21:05:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.115+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6b7002a070 con 0x7f6b7c103430 2024-05-20T21:05:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.116+0000 7f6b79ffb700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6b640b2900 0x7f6b640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.116+0000 7f6b80a29700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6b640b2900 0x7f6b640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.116+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6b700e32e0 con 0x7f6b7c103430 2024-05-20T21:05:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.116+0000 7f6b80a29700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6b640b2900 0x7f6b640b4dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00bcf0 tx=0x7f6b6c00b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.120+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b70013070 con 0x7f6b7c103430 2024-05-20T21:05:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.393+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b7c1a8b30 con 0x7f6b7c103430 2024-05-20T21:05:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.394+0000 7f6b79ffb700 1 -- 172.21.15.26:0/2420871697 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f6b700043a0 con 0x7f6b7c103430 2024-05-20T21:05:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:05:51.396 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:05:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6b640b2900 msgr2=0x7f6b640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6b640b2900 0x7f6b640b4dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00bcf0 tx=0x7f6b6c00b2c0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 msgr2=0x7f6b7c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c104680 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6b700098e0 tx=0x7f6b700047b0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 shutdown_connections 2024-05-20T21:05:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7c1077d0 0x7f6b7c104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7c102a80 0x7f6b7c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6b640b2900 0x7f6b640b4dc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 --2- 172.21.15.26:0/2420871697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7c103430 0x7f6b7c104680 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 >> 172.21.15.26:0/2420871697 conn(0x7f6b7c0793f0 msgr2=0x7f6b7c06c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 shutdown_connections 2024-05-20T21:05:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:51.396+0000 7f6b82c8d700 1 -- 172.21.15.26:0/2420871697 wait complete. 2024-05-20T21:05:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:05:51.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:05:51.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.23 MB/sec execute 357 sec latency 23650.054 ms 2024-05-20T21:05:51.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:51 smithi144 ceph-mon[35660]: pgmap v681: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-05-20T21:05:51.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:51 smithi055 ceph-mon[35213]: pgmap v681: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-05-20T21:05:51.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:51 smithi026 ceph-mon[31990]: pgmap v681: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-05-20T21:05:52.032 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 8724a85b --release --await' 2024-05-20T21:05:52.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 8724a85b --release --await 2024-05-20T21:05:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:52.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.22 MB/sec execute 358 sec latency 24650.155 ms 2024-05-20T21:05:52.598 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2420871697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:52.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:52 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2420871697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:52.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:52 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2420871697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:05:53.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.22 MB/sec execute 359 sec latency 25650.197 ms 2024-05-20T21:05:53.581 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:53 smithi026 ceph-mon[31990]: pgmap v682: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-20T21:05:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:53 smithi055 ceph-mon[35213]: pgmap v682: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-20T21:05:53.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:53 smithi144 ceph-mon[35660]: pgmap v682: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 -- 172.21.15.26:0/3076757356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 msgr2=0x7f9b54110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 --2- 172.21.15.26:0/3076757356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b54110400 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00b0d0 tx=0x7f9b4c00b3e0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 -- 172.21.15.26:0/3076757356 shutdown_connections 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 --2- 172.21.15.26:0/3076757356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b54110400 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 --2- 172.21.15.26:0/3076757356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b54109630 0x7f9b54109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 --2- 172.21.15.26:0/3076757356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5406bd10 0x7f9b5406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 -- 172.21.15.26:0/3076757356 >> 172.21.15.26:0/3076757356 conn(0x7f9b540794e0 msgr2=0x7f9b540798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 -- 172.21.15.26:0/3076757356 shutdown_connections 2024-05-20T21:05:53.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.824+0000 7f9b5acee700 1 -- 172.21.15.26:0/3076757356 wait complete. 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 Processor -- start 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 -- start start 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5406bd10 0x7f9b5419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b5419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54109630 0x7f9b54199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b54113600 con 0x7f9b5406c6d0 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b54113480 con 0x7f9b5406bd10 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5acee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b54113780 con 0x7f9b54109630 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b5419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b5419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53508/0 (socket says 172.21.15.26:53508) 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 -- 172.21.15.26:0/471188530 learned_addr learned my addr 172.21.15.26:0/471188530 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5406bd10 0x7f9b5419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b5928b700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54109630 0x7f9b54199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 -- 172.21.15.26:0/471188530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54109630 msgr2=0x7f9b54199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54109630 0x7f9b54199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 -- 172.21.15.26:0/471188530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5406bd10 msgr2=0x7f9b5419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5406bd10 0x7f9b5419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.825+0000 7f9b4bfff700 1 -- 172.21.15.26:0/471188530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b54199fe0 con 0x7f9b5406c6d0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.826+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b5419fc00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400d980 tx=0x7f9b4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.826+0000 7f9b49ffb700 1 -- 172.21.15.26:0/471188530 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44017070 con 0x7f9b5406c6d0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.826+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5419a2d0 con 0x7f9b5406c6d0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.826+0000 7f9b49ffb700 1 -- 172.21.15.26:0/471188530 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b44005020 con 0x7f9b5406c6d0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.826+0000 7f9b49ffb700 1 -- 172.21.15.26:0/471188530 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4400f980 con 0x7f9b5406c6d0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.826+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b541a6590 con 0x7f9b5406c6d0 2024-05-20T21:05:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.828+0000 7f9b49ffb700 1 -- 172.21.15.26:0/471188530 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9b44004a30 con 0x7f9b5406c6d0 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.829+0000 7f9b49ffb700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9b3c0b2880 0x7f9b3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.829+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9b3c0b2880 0x7f9b3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.829+0000 7f9b49ffb700 1 -- 172.21.15.26:0/471188530 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9b440e2040 con 0x7f9b5406c6d0 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 Processor -- start 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 -- start start 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9b3c0b2880 0x7f9b3c0b4d40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9b5419a9c0 tx=0x7f9b5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c916c0 0x55e401c91aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c8bcb0 0x55e401c8c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c98460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e401be5d70 con 0x55e401c8c670 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e401bd5910 con 0x55e401c8bcb0 2024-05-20T21:05:53.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b65584b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e401bd9950 con 0x55e401c916c0 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c98460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c98460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53516/0 (socket says 172.21.15.26:53516) 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 -- 172.21.15.26:0/2210694156 learned_addr learned my addr 172.21.15.26:0/2210694156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 -- 172.21.15.26:0/2210694156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c916c0 msgr2=0x55e401c91aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c916c0 0x55e401c91aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c8bcb0 0x55e401c8c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c916c0 0x55e401c91aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 -- 172.21.15.26:0/2210694156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c8bcb0 msgr2=0x55e401c8c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c8bcb0 0x55e401c8c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 -- 172.21.15.26:0/2210694156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e401b7efb0 con 0x55e401c8c670 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.830+0000 7f9b5928b700 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c98460 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009ce0 tx=0x7f9b4c009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99afa0bdcc081a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.831+0000 7f9b3affd700 1 -- 172.21.15.26:0/2210694156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c009250 con 0x55e401c8c670 2024-05-20T21:05:53.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.831+0000 7f9b3affd700 1 -- 172.21.15.26:0/2210694156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b4c020070 con 0x55e401c8c670 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.831+0000 7f9b3affd700 1 -- 172.21.15.26:0/2210694156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c005830 con 0x55e401c8c670 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 -- 172.21.15.26:0/2210694156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 msgr2=0x55e401c98460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c98460 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009ce0 tx=0x7f9b4c009d10 comp rx=0 tx=0).stop 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 -- 172.21.15.26:0/2210694156 shutdown_connections 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c98460 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c916c0 0x55e401c91aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 --2- 172.21.15.26:0/2210694156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c8bcb0 0x55e401c8c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 -- 172.21.15.26:0/2210694156 >> 172.21.15.26:0/2210694156 conn(0x55e401bb3240 msgr2=0x55e401c823e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 -- 172.21.15.26:0/2210694156 shutdown_connections 2024-05-20T21:05:53.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 -- 172.21.15.26:0/2210694156 wait complete. 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 -- start start 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.832+0000 7f9b65584b80 1 Processor -- start 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e401adbfa0 con 0x55e401c8bcb0 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e401b3dda0 con 0x55e401c916c0 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e401c98a90 con 0x55e401c8c670 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b65584b80 10 client.0 _ll_register_callbacks cb 0x55e401b70950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53532/0 (socket says 172.21.15.26:53532) 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 -- 172.21.15.26:0/4139649875 learned_addr learned my addr 172.21.15.26:0/4139649875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:53.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b5928b700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 -- 172.21.15.26:0/4139649875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 msgr2=0x55e401c953b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c953b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 -- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 msgr2=0x55e401c958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 -- 172.21.15.26:0/4139649875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e401b258d0 con 0x55e401c8bcb0 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b5928b700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c980b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f9b500116c0 tx=0x7f9b500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee915679556848d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.833+0000 7f9b33fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b33fff700 1 -- 172.21.15.26:0/4139649875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5001e830 con 0x55e401c8bcb0 2024-05-20T21:05:53.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b33fff700 1 -- 172.21.15.26:0/4139649875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9b5001e9d0 con 0x55e401c8bcb0 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4c020070 con 0x55e401c8bcb0 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b33fff700 1 -- 172.21.15.26:0/4139649875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b50010e50 con 0x55e401c8bcb0 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 10 client.18315 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e401b8f760 con 0x55e401c8bcb0 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b33fff700 1 -- 172.21.15.26:0/4139649875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b4c020070 con 0x55e401c8bcb0 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b33fff700 1 -- 172.21.15.26:0/4139649875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9b50031410 con 0x55e401c8bcb0 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 10 client.18315 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 10 client.18315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 20 client.18315 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401ca5670 0x55e401ca7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 4 client.18315 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.834+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e401be7cc0 con 0x55e401ca5670 2024-05-20T21:05:53.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.835+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401ca5670 0x55e401ca7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.835+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401ca5670 0x55e401ca7b00 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff97b9dc861f4e6 server_cookie=61be861a3fb3be05 in_seq=0 out_seq=0 2024-05-20T21:05:53.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.835+0000 7f9b33fff700 10 client.18315 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b33fff700 1 -- 172.21.15.26:0/4139649875 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e401be7cc0 con 0x55e401ca5670 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b33fff700 10 client.18315 handle_command_reply: tid=0 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 client.18315 shutdown 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 msgr2=0x55e401c980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c980b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f9b500116c0 tx=0x7f9b500047c0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 shutdown_connections 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401ca5670 0x55e401ca7b00 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b33fff700 0 client.18315 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c980b0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 --2- 172.21.15.26:0/4139649875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 >> 172.21.15.26:0/4139649875 conn(0x55e401bb3240 msgr2=0x55e401c81af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 shutdown_connections 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 1 -- 172.21.15.26:0/4139649875 wait complete. 2024-05-20T21:05:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.836+0000 7f9b65584b80 20 client.18315 trim_cache size 0 max 16384 2024-05-20T21:05:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 Processor -- start 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 -- start start 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c82270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401cad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541ae140 con 0x55e401c8bcb0 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541ab9d0 con 0x55e401c916c0 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b65584b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541a9820 con 0x55e401c8c670 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b58a8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c82270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b58a8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c82270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53542/0 (socket says 172.21.15.26:53542) 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b58a8a700 1 -- 172.21.15.26:0/3235699862 learned_addr learned my addr 172.21.15.26:0/3235699862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b5928b700 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401cad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.858+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b58a8a700 1 -- 172.21.15.26:0/3235699862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 msgr2=0x55e401c972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b58a8a700 1 -- 172.21.15.26:0/3235699862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 msgr2=0x55e401cad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401cad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b58a8a700 1 -- 172.21.15.26:0/3235699862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e401c842e0 con 0x55e401c8bcb0 2024-05-20T21:05:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c82270 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f9b50011010 tx=0x7f9b50031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36227d774c2d25df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b3a7fc700 1 -- 172.21.15.26:0/3235699862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50031a20 con 0x55e401c8bcb0 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b3a7fc700 1 -- 172.21.15.26:0/3235699862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b50022070 con 0x55e401c8bcb0 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.859+0000 7f9b3a7fc700 1 -- 172.21.15.26:0/3235699862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5001ee60 con 0x55e401c8bcb0 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 -- 172.21.15.26:0/3235699862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 msgr2=0x55e401c82270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c82270 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f9b50011010 tx=0x7f9b50031df0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 -- 172.21.15.26:0/3235699862 shutdown_connections 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8bcb0 0x55e401c82270 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401cad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3235699862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8c670 0x55e401c972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 -- 172.21.15.26:0/3235699862 >> 172.21.15.26:0/3235699862 conn(0x55e401bb3240 msgr2=0x55e401c96be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 -- 172.21.15.26:0/3235699862 shutdown_connections 2024-05-20T21:05:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 -- 172.21.15.26:0/3235699862 wait complete. 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 -- start start 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.860+0000 7f9b65584b80 1 Processor -- start 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8bcb0 0x55e401c9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c9e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541a9e90 con 0x55e401c8c670 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541ab300 con 0x55e401c916c0 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541aa560 con 0x55e401c8bcb0 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b65584b80 10 client.0 _ll_register_callbacks cb 0x55e401b70950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c9e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c9e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53558/0 (socket says 172.21.15.26:53558) 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 -- 172.21.15.26:0/3619950231 learned_addr learned my addr 172.21.15.26:0/3619950231 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8bcb0 0x55e401c9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b5928b700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 -- 172.21.15.26:0/3619950231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8bcb0 msgr2=0x55e401c9dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8bcb0 0x55e401c9dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 -- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 msgr2=0x55e401c9acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c9acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 -- 172.21.15.26:0/3619950231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e401c9b230 con 0x55e401c8c670 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b5928b700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b4bfff700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c9e260 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9b440a5720 tx=0x7f9b440057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7f65fc380516fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.861+0000 7f9b33fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b33fff700 1 -- 172.21.15.26:0/3619950231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4406b070 con 0x55e401c8c670 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b33fff700 1 -- 172.21.15.26:0/3619950231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9b440a9900 con 0x55e401c8c670 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b50022070 con 0x55e401c8c670 2024-05-20T21:05:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b33fff700 1 -- 172.21.15.26:0/3619950231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b440a96a0 con 0x55e401c8c670 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 10 client.18321 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e401d10b00 con 0x55e401c8c670 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b33fff700 1 -- 172.21.15.26:0/3619950231 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b50022070 con 0x55e401c8c670 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b33fff700 1 -- 172.21.15.26:0/3619950231 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9b440b2070 con 0x55e401c8c670 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 10 client.18321 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 20 client.18321 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401d111c0 0x55e401d135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 4 client.18321 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.862+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e401be14a0 con 0x55e401d111c0 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401d111c0 0x55e401d135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b58a8a700 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401d111c0 0x55e401d135e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881cb425087d58db server_cookie=b4d57b1a83b6ebd7 in_seq=0 out_seq=0 2024-05-20T21:05:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b33fff700 10 client.18321 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b33fff700 1 -- 172.21.15.26:0/3619950231 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e401be14a0 con 0x55e401d111c0 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b33fff700 10 client.18321 handle_command_reply: tid=0 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 client.18321 shutdown 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 msgr2=0x55e401c9e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c9e260 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9b440a5720 tx=0x7f9b440057a0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 shutdown_connections 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e401d111c0 0x55e401d135e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b33fff700 0 client.18321 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e401c916c0 0x55e401c9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e401c8bcb0 0x55e401c9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 --2- 172.21.15.26:0/3619950231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e401c8c670 0x55e401c9e260 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.863+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 >> 172.21.15.26:0/3619950231 conn(0x55e401bb3240 msgr2=0x55e401c8f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 shutdown_connections 2024-05-20T21:05:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b65584b80 1 -- 172.21.15.26:0/3619950231 wait complete. 2024-05-20T21:05:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b65584b80 20 client.18321 trim_cache size 0 max 16384 2024-05-20T21:05:53.869 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:05:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:05:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:05:53.869 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9b3c0b2880 msgr2=0x7f9b3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9b3c0b2880 0x7f9b3c0b4d40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9b5419a9c0 tx=0x7f9b5000b410 comp rx=0 tx=0).stop 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 msgr2=0x7f9b5419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b5419fc00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400d980 tx=0x7f9b4400dc90 comp rx=0 tx=0).stop 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 shutdown_connections 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54109630 0x7f9b54199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5406bd10 0x7f9b5419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9b3c0b2880 0x7f9b3c0b4d40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 --2- 172.21.15.26:0/471188530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5406c6d0 0x7f9b5419fc00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 >> 172.21.15.26:0/471188530 conn(0x7f9b540794e0 msgr2=0x7f9b541010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 shutdown_connections 2024-05-20T21:05:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:53.864+0000 7f9b5acee700 1 -- 172.21.15.26:0/471188530 wait complete. 2024-05-20T21:05:54.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.21 MB/sec execute 360 sec latency 26650.269 ms 2024-05-20T21:05:54.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:54 smithi026 ceph-mon[31990]: pgmap v683: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-20T21:05:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:54 smithi055 ceph-mon[35213]: pgmap v683: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-20T21:05:54.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:54 smithi144 ceph-mon[35660]: pgmap v683: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-20T21:05:55.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213250 2.20 MB/sec execute 361 sec latency 27650.317 ms 2024-05-20T21:05:55.588 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:05:55.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:05:56.083 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 -- 172.21.15.26:0/3510130154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f485006bbe0 msgr2=0x7f48501103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 --2- 172.21.15.26:0/3510130154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f485006bbe0 0x7f48501103d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f484800b0a0 tx=0x7f484800b3b0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 -- 172.21.15.26:0/3510130154 shutdown_connections 2024-05-20T21:05:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 --2- 172.21.15.26:0/3510130154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f485006bbe0 0x7f48501103d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 --2- 172.21.15.26:0/3510130154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48501095b0 0x7f4850109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 --2- 172.21.15.26:0/3510130154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f485006b220 0x7f485006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 -- 172.21.15.26:0/3510130154 >> 172.21.15.26:0/3510130154 conn(0x7f48500793f0 msgr2=0x7f4850079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 -- 172.21.15.26:0/3510130154 shutdown_connections 2024-05-20T21:05:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.095+0000 7f4855fb0700 1 -- 172.21.15.26:0/3510130154 wait complete. 2024-05-20T21:05:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.096+0000 7f4855fb0700 1 Processor -- start 2024-05-20T21:05:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 -- start start 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485006b220 0x7f4850102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f485006bbe0 0x7f4850102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 0x7f4850102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4850113590 con 0x7f48501095b0 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4850113410 con 0x7f485006bbe0 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f4855fb0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4850113710 con 0x7f485006b220 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 0x7f4850102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 0x7f4850102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53582/0 (socket says 172.21.15.26:53582) 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 -- 172.21.15.26:0/1439746796 learned_addr learned my addr 172.21.15.26:0/1439746796 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484effd700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f485006bbe0 0x7f4850102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485006b220 0x7f4850102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 -- 172.21.15.26:0/1439746796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485006b220 msgr2=0x7f4850102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485006b220 0x7f4850102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 -- 172.21.15.26:0/1439746796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f485006bbe0 msgr2=0x7f4850102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.097+0000 7f484ffff700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f485006bbe0 0x7f4850102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f484ffff700 1 -- 172.21.15.26:0/1439746796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48501a7ff0 con 0x7f48501095b0 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485006b220 0x7f4850102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f484ffff700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 0x7f4850102f90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f484800b830 tx=0x7f4848004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f484cff9700 1 -- 172.21.15.26:0/1439746796 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848021070 con 0x7f48501095b0 2024-05-20T21:05:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48501a8280 con 0x7f48501095b0 2024-05-20T21:05:56.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f484cff9700 1 -- 172.21.15.26:0/1439746796 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4848007b80 con 0x7f48501095b0 2024-05-20T21:05:56.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f484cff9700 1 -- 172.21.15.26:0/1439746796 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480052a0 con 0x7f48501095b0 2024-05-20T21:05:56.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.098+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a86d0 con 0x7f48501095b0 2024-05-20T21:05:56.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.101+0000 7f484cff9700 1 -- 172.21.15.26:0/1439746796 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4848024070 con 0x7f48501095b0 2024-05-20T21:05:56.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.102+0000 7f484cff9700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48380b2880 0x7f48380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.102+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48380b2880 0x7f48380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.102+0000 7f484cff9700 1 -- 172.21.15.26:0/1439746796 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f48480e34b0 con 0x7f48501095b0 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 Processor -- start 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 -- start start 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48380b2880 0x7f48380b4d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4840005e10 tx=0x7f4840005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 0x55a8e021db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e0218bf0 0x55a8e0219070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0225620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8e0176e70 con 0x55a8e021d720 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a8e0166a10 con 0x55a8e02195b0 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f4860846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8e016aa50 con 0x55a8e0218bf0 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 0x55a8e021db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 0x55a8e021db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53586/0 (socket says 172.21.15.26:53586) 2024-05-20T21:05:56.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 -- 172.21.15.26:0/1281572624 learned_addr learned my addr 172.21.15.26:0/1281572624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484ffff700 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0225620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484effd700 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e0218bf0 0x55a8e0219070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 -- 172.21.15.26:0/1281572624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e0218bf0 msgr2=0x55a8e0219070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e0218bf0 0x55a8e0219070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 -- 172.21.15.26:0/1281572624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 msgr2=0x55a8e0225620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0225620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.103+0000 7f484f7fe700 1 -- 172.21.15.26:0/1281572624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e01100b0 con 0x55a8e021d720 2024-05-20T21:05:56.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.104+0000 7f484f7fe700 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 0x55a8e021db00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f484000fda0 tx=0x7f4840012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8a91d226ae575c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.104+0000 7f4835ffb700 1 -- 172.21.15.26:0/1281572624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840022070 con 0x55a8e021d720 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.104+0000 7f4835ffb700 1 -- 172.21.15.26:0/1281572624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4840011bb0 con 0x55a8e021d720 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.104+0000 7f4835ffb700 1 -- 172.21.15.26:0/1281572624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484001d610 con 0x55a8e021d720 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 -- 172.21.15.26:0/1281572624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 msgr2=0x55a8e021db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 0x55a8e021db00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f484000fda0 tx=0x7f4840012000 comp rx=0 tx=0).stop 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 -- 172.21.15.26:0/1281572624 shutdown_connections 2024-05-20T21:05:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e021d720 0x55a8e021db00 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0225620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 --2- 172.21.15.26:0/1281572624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e0218bf0 0x55a8e0219070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 -- 172.21.15.26:0/1281572624 >> 172.21.15.26:0/1281572624 conn(0x55a8e0144340 msgr2=0x55a8e02138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 -- 172.21.15.26:0/1281572624 shutdown_connections 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.105+0000 7f4860846b80 1 -- 172.21.15.26:0/1281572624 wait complete. 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.106+0000 7f4860846b80 1 -- start start 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.106+0000 7f4860846b80 1 Processor -- start 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.106+0000 7f4860846b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e02253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f4860846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e02195b0 0x55a8e02226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f4860846b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e021d720 0x55a8e0222c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f4860846b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8e006d0a0 con 0x55a8e0218bf0 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f4860846b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a8e00ceea0 con 0x55a8e021d720 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e02253e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e02253e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53596/0 (socket says 172.21.15.26:53596) 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484f7fe700 1 -- 172.21.15.26:0/2495796113 learned_addr learned my addr 172.21.15.26:0/2495796113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8e0225c80 con 0x55a8e02195b0 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e021d720 0x55a8e0222c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484effd700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e02195b0 0x55a8e02226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f4860846b80 10 client.0 _ll_register_callbacks cb 0x55a8e0101a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 -- 172.21.15.26:0/2495796113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e02195b0 msgr2=0x55a8e02226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e02195b0 0x55a8e02226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 -- 172.21.15.26:0/2495796113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 msgr2=0x55a8e02253e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e02253e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 -- 172.21.15.26:0/2495796113 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e00b69d0 con 0x55a8e021d720 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484f7fe700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e02253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484effd700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e02195b0 0x55a8e02226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f484ffff700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e021d720 0x55a8e0222c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48480e0ef0 tx=0x7f484801df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=522fab162655115 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.107+0000 7f482effd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f482effd700 1 -- 172.21.15.26:0/2495796113 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484802b070 con 0x55a8e021d720 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f482effd700 1 -- 172.21.15.26:0/2495796113 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f48480cd300 con 0x55a8e021d720 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4840011bb0 con 0x55a8e021d720 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f482effd700 1 -- 172.21.15.26:0/2495796113 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48480aaaa0 con 0x55a8e021d720 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 10 client.27701 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8e0120860 con 0x55a8e021d720 2024-05-20T21:05:56.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f482effd700 1 -- 172.21.15.26:0/2495796113 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4840011bb0 con 0x55a8e021d720 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f482effd700 1 -- 172.21.15.26:0/2495796113 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4848011070 con 0x55a8e021d720 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 10 client.27701 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 10 client.27701 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 20 client.27701 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e023b280 0x55a8e023d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.108+0000 7f4860846b80 4 client.27701 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.109+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e0178dc0 con 0x55a8e023b280 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.109+0000 7f484f7fe700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e023b280 0x55a8e023d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.109+0000 7f484f7fe700 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e023b280 0x55a8e023d6c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d82c0e8f08d1cf server_cookie=331ef4989e1d7551 in_seq=0 out_seq=0 2024-05-20T21:05:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.109+0000 7f482effd700 10 client.27701 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:56.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f482effd700 1 -- 172.21.15.26:0/2495796113 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8e0178dc0 con 0x55a8e023b280 2024-05-20T21:05:56.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f482effd700 10 client.27701 handle_command_reply: tid=0 2024-05-20T21:05:56.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 client.27701 shutdown 2024-05-20T21:05:56.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e021d720 msgr2=0x55a8e0222c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e021d720 0x55a8e0222c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48480e0ef0 tx=0x7f484801df90 comp rx=0 tx=0).stop 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 shutdown_connections 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e023b280 0x55a8e023d6c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e02253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e021d720 0x55a8e0222c20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 --2- 172.21.15.26:0/2495796113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e02195b0 0x55a8e02226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 >> 172.21.15.26:0/2495796113 conn(0x55a8e0144340 msgr2=0x55a8e0217be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f482effd700 0 client.27701 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 shutdown_connections 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.110+0000 7f4860846b80 1 -- 172.21.15.26:0/2495796113 wait complete. 2024-05-20T21:05:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.111+0000 7f4860846b80 20 client.27701 trim_cache size 0 max 16384 2024-05-20T21:05:56.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 Processor -- start 2024-05-20T21:05:56.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 -- start start 2024-05-20T21:05:56.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e0218bf0 0x55a8e021cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 0x55a8e020de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0242fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48501ae1b0 con 0x55a8e02195b0 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48501aba40 con 0x55a8e0218bf0 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.135+0000 7f4860846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f485019d890 con 0x55a8e021d720 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 0x55a8e020de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 0x55a8e020de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53602/0 (socket says 172.21.15.26:53602) 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 -- 172.21.15.26:0/151261661 learned_addr learned my addr 172.21.15.26:0/151261661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484ffff700 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0242fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484f7fe700 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e0218bf0 0x55a8e021cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 -- 172.21.15.26:0/151261661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 msgr2=0x55a8e0242fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0242fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 -- 172.21.15.26:0/151261661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e0218bf0 msgr2=0x55a8e021cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e0218bf0 0x55a8e021cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 -- 172.21.15.26:0/151261661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e0223220 con 0x55a8e02195b0 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484f7fe700 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e0218bf0 0x55a8e021cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f484effd700 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 0x55a8e020de70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f48440105a0 tx=0x7f484400a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b15086277602796 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f48357fa700 1 -- 172.21.15.26:0/151261661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844004460 con 0x55a8e02195b0 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f48357fa700 1 -- 172.21.15.26:0/151261661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f484401f070 con 0x55a8e02195b0 2024-05-20T21:05:56.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.136+0000 7f48357fa700 1 -- 172.21.15.26:0/151261661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484401a880 con 0x55a8e02195b0 2024-05-20T21:05:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 -- 172.21.15.26:0/151261661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 msgr2=0x55a8e020de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 0x55a8e020de70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f48440105a0 tx=0x7f484400a380 comp rx=0 tx=0).stop 2024-05-20T21:05:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 -- 172.21.15.26:0/151261661 shutdown_connections 2024-05-20T21:05:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0242fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e0218bf0 0x55a8e021cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 --2- 172.21.15.26:0/151261661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e02195b0 0x55a8e020de70 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 -- 172.21.15.26:0/151261661 >> 172.21.15.26:0/151261661 conn(0x55a8e0144340 msgr2=0x55a8e021c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 -- 172.21.15.26:0/151261661 shutdown_connections 2024-05-20T21:05:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.137+0000 7f4860846b80 1 -- 172.21.15.26:0/151261661 wait complete. 2024-05-20T21:05:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 -- start start 2024-05-20T21:05:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 Processor -- start 2024-05-20T21:05:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e0233c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0230770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0230cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48501a9f00 con 0x55a8e0218bf0 2024-05-20T21:05:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48501ab370 con 0x55a8e02195b0 2024-05-20T21:05:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48501aa5d0 con 0x55a8e021d720 2024-05-20T21:05:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f4860846b80 10 client.0 _ll_register_callbacks cb 0x55a8e0101a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:05:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e0233c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e0233c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53612/0 (socket says 172.21.15.26:53612) 2024-05-20T21:05:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f484f7fe700 1 -- 172.21.15.26:0/3306282306 learned_addr learned my addr 172.21.15.26:0/3306282306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.138+0000 7f484ffff700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0230cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484effd700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0230770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484f7fe700 1 -- 172.21.15.26:0/3306282306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 msgr2=0x55a8e0230cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484f7fe700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0230cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484f7fe700 1 -- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 msgr2=0x55a8e0230770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:05:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484f7fe700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0230770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484f7fe700 1 -- 172.21.15.26:0/3306282306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e0225c30 con 0x55a8e0218bf0 2024-05-20T21:05:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484ffff700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0230cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:05:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f484f7fe700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e0233c60 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x55a8e023df40 tx=0x7f484000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90def8b7f9e5b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f482effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:05:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f482effd700 1 -- 172.21.15.26:0/3306282306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840022040 con 0x55a8e0218bf0 2024-05-20T21:05:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f482effd700 1 -- 172.21.15.26:0/3306282306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4840005240 con 0x55a8e0218bf0 2024-05-20T21:05:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484401f070 con 0x55a8e0218bf0 2024-05-20T21:05:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f482effd700 1 -- 172.21.15.26:0/3306282306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4840011e40 con 0x55a8e0218bf0 2024-05-20T21:05:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f4860846b80 10 client.18339 fetch_fsmap learned FSMap version 30 2024-05-20T21:05:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8e02a6b30 con 0x55a8e0218bf0 2024-05-20T21:05:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.139+0000 7f482effd700 1 -- 172.21.15.26:0/3306282306 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f484401f070 con 0x55a8e0218bf0 2024-05-20T21:05:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f482effd700 1 -- 172.21.15.26:0/3306282306 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4840046050 con 0x55a8e0218bf0 2024-05-20T21:05:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f4860846b80 10 client.18339 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:05:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f4860846b80 10 client.18339 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:05:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f4860846b80 20 client.18339 populate_metadata read hostname 'smithi026' 2024-05-20T21:05:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f4860846b80 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e02a7080 0x55a8e02a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f4860846b80 4 client.18339 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8724a85b", "release": true, "await": true}] 2024-05-20T21:05:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8724a85b", "release": true, "await": true}) -- 0x55a8e01725a0 con 0x55a8e02a7080 2024-05-20T21:05:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f484effd700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e02a7080 0x55a8e02a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f484effd700 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e02a7080 0x55a8e02a94a0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0aa6f08c820692 server_cookie=892fbd8415c9ac6b in_seq=0 out_seq=0 2024-05-20T21:05:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.140+0000 7f482effd700 10 client.18339 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:56.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f482effd700 1 -- 172.21.15.26:0/3306282306 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a8e01725a0 con 0x55a8e02a7080 2024-05-20T21:05:56.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f482effd700 10 client.18339 handle_command_reply: tid=0 2024-05-20T21:05:56.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 client.18339 shutdown 2024-05-20T21:05:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 msgr2=0x55a8e0233c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e0233c60 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x55a8e023df40 tx=0x7f484000fda0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 shutdown_connections 2024-05-20T21:05:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a8e02a7080 0x55a8e02a94a0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8e0218bf0 0x55a8e0233c60 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8e021d720 0x55a8e0230cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f482effd700 0 client.18339 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:05:56.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 --2- 172.21.15.26:0/3306282306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a8e02195b0 0x55a8e0230770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 >> 172.21.15.26:0/3306282306 conn(0x55a8e0144340 msgr2=0x55a8e0223f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:56.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 shutdown_connections 2024-05-20T21:05:56.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 1 -- 172.21.15.26:0/3306282306 wait complete. 2024-05-20T21:05:56.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.153+0000 7f4860846b80 20 client.18339 trim_cache size 0 max 16384 2024-05-20T21:05:56.157 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:05:56.158 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:05:56.158 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:05:56.158 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 40, 2024-05-20T21:05:56.158 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:05:56.158 INFO:teuthology.orchestra.run.smithi026.stdout: "8724a85b": { 2024-05-20T21:05:56.158 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 40, 2024-05-20T21:05:56.159 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:05:56.159 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:05:56.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:05:56.159 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:05:56.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:05:56.160 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:05:56.160 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 146.6, 2024-05-20T21:05:56.160 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:05:56.160 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:05:56.160 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:05:56.161 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:05:56.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:05:56.161 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:05:56.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:56.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:56.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:56.162 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:56.162 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:05:56.162 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:05:56.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48380b2880 msgr2=0x7f48380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48380b2880 0x7f48380b4d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4840005e10 tx=0x7f4840005d80 comp rx=0 tx=0).stop 2024-05-20T21:05:56.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 msgr2=0x7f4850102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:56.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 0x7f4850102f90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f484800b830 tx=0x7f4848004d00 comp rx=0 tx=0).stop 2024-05-20T21:05:56.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 shutdown_connections 2024-05-20T21:05:56.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f48380b2880 0x7f48380b4d40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48501095b0 0x7f4850102f90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485006b220 0x7f4850102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 --2- 172.21.15.26:0/1439746796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f485006bbe0 0x7f4850102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:56.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 >> 172.21.15.26:0/1439746796 conn(0x7f48500793f0 msgr2=0x7f4850101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:56.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 shutdown_connections 2024-05-20T21:05:56.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:56.154+0000 7f4855fb0700 1 -- 172.21.15.26:0/1439746796 wait complete. 2024-05-20T21:05:56.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213424 2.20 MB/sec execute 362 sec latency 28227.364 ms 2024-05-20T21:05:57.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:56 smithi026 ceph-mon[31990]: pgmap v684: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-05-20T21:05:57.159 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8724a85b', total seconds elapsed: 34.7 2024-05-20T21:05:57.159 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.55 2024-05-20T21:05:57.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:05:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:56 smithi144 ceph-mon[35660]: pgmap v684: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-05-20T21:05:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:56 smithi055 ceph-mon[35213]: pgmap v684: 97 pgs: 97 active+clean; 25 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-05-20T21:05:57.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 214074 2.20 MB/sec execute 363 sec latency 17.605 ms 2024-05-20T21:05:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:05:58.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 214819 2.20 MB/sec execute 364 sec latency 17.112 ms 2024-05-20T21:05:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 -- 172.21.15.26:0/569155602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b7806b220 msgr2=0x7f3b7806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 --2- 172.21.15.26:0/569155602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b7806b220 0x7f3b7806b6a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009990 tx=0x7f3b68009ca0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 -- 172.21.15.26:0/569155602 shutdown_connections 2024-05-20T21:05:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 --2- 172.21.15.26:0/569155602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b7806bbe0 0x7f3b781103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 --2- 172.21.15.26:0/569155602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b781095b0 0x7f3b78109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 --2- 172.21.15.26:0/569155602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b7806b220 0x7f3b7806b6a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.072+0000 7f3b7f598700 1 -- 172.21.15.26:0/569155602 >> 172.21.15.26:0/569155602 conn(0x7f3b780793f0 msgr2=0x7f3b78079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.073+0000 7f3b7f598700 1 -- 172.21.15.26:0/569155602 shutdown_connections 2024-05-20T21:05:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.073+0000 7f3b7f598700 1 -- 172.21.15.26:0/569155602 wait complete. 2024-05-20T21:05:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.073+0000 7f3b7f598700 1 Processor -- start 2024-05-20T21:05:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.073+0000 7f3b7f598700 1 -- start start 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7f598700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b7806b220 0x7f3b7819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7f598700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b7806bbe0 0x7f3b7819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7f598700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b781095b0 0x7f3b781999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7f598700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781135f0 con 0x7f3b781095b0 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7f598700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78113470 con 0x7f3b7806bbe0 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7f598700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78113770 con 0x7f3b7806b220 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7db35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b781095b0 0x7f3b781999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b7806bbe0 0x7f3b7819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7db35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b781095b0 0x7f3b781999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53628/0 (socket says 172.21.15.26:53628) 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7db35700 1 -- 172.21.15.26:0/3161664751 learned_addr learned my addr 172.21.15.26:0/3161664751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7d334700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b7806b220 0x7f3b7819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 -- 172.21.15.26:0/3161664751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b7806b220 msgr2=0x7f3b7819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b7806b220 0x7f3b7819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 -- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b781095b0 msgr2=0x7f3b781999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b781095b0 0x7f3b781999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 -- 172.21.15.26:0/3161664751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7819a110 con 0x7f3b7806bbe0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b7cb33700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b7806bbe0 0x7f3b7819fd30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009990 tx=0x7f3b680057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.074+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68026070 con 0x7f3b7806bbe0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.075+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7819a3a0 con 0x7f3b7806bbe0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.075+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b68021410 con 0x7f3b7806bbe0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.075+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a66c0 con 0x7f3b7806bbe0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.075+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68017650 con 0x7f3b7806bbe0 2024-05-20T21:05:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.075+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7806a230 con 0x7f3b7806bbe0 2024-05-20T21:05:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.077+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3b68029070 con 0x7f3b7806bbe0 2024-05-20T21:05:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.078+0000 7f3b6e7fc700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3b640b2900 0x7f3b640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:05:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.078+0000 7f3b7d334700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3b640b2900 0x7f3b640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:05:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.078+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3b680e3380 con 0x7f3b7806bbe0 2024-05-20T21:05:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.078+0000 7f3b7d334700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3b640b2900 0x7f3b640b4dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3b7819aaf0 tx=0x7f3b74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:05:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.081+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b68030080 con 0x7f3b7806bbe0 2024-05-20T21:05:59.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:58 smithi026 ceph-mon[31990]: pgmap v685: 97 pgs: 97 active+clean; 26 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-05-20T21:05:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:58 smithi144 ceph-mon[35660]: pgmap v685: 97 pgs: 97 active+clean; 26 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-05-20T21:05:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.399+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b7819aee0 con 0x7f3b7806bbe0 2024-05-20T21:05:59.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.400+0000 7f3b6e7fc700 1 -- 172.21.15.26:0/3161664751 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f3b68013070 con 0x7f3b7806bbe0 2024-05-20T21:05:59.400 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:05:59.400 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:05:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3b640b2900 msgr2=0x7f3b640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3b640b2900 0x7f3b640b4dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3b7819aaf0 tx=0x7f3b74009260 comp rx=0 tx=0).stop 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b7806bbe0 msgr2=0x7f3b7819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b7806bbe0 0x7f3b7819fd30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009990 tx=0x7f3b680057c0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 shutdown_connections 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3b640b2900 0x7f3b640b4dc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b781095b0 0x7f3b781999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b7806b220 0x7f3b7819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 --2- 172.21.15.26:0/3161664751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b7806bbe0 0x7f3b7819fd30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.403+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 >> 172.21.15.26:0/3161664751 conn(0x7f3b780793f0 msgr2=0x7f3b781012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.404+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 shutdown_connections 2024-05-20T21:05:59.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:05:59.404+0000 7f3b7f598700 1 -- 172.21.15.26:0/3161664751 wait complete. 2024-05-20T21:05:59.404 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:05:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:58 smithi055 ceph-mon[35213]: pgmap v685: 97 pgs: 97 active+clean; 26 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-05-20T21:05:59.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 215455 2.20 MB/sec execute 365 sec latency 17.567 ms 2024-05-20T21:06:00.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:05:59 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3161664751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:00.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:05:59 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3161664751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:00.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:05:59 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3161664751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:00.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 216196 2.21 MB/sec execute 366 sec latency 18.822 ms 2024-05-20T21:06:00.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:06:01.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:00 smithi026 ceph-mon[31990]: pgmap v686: 97 pgs: 97 active+clean; 26 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-05-20T21:06:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:00 smithi144 ceph-mon[35660]: pgmap v686: 97 pgs: 97 active+clean; 26 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-05-20T21:06:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 -- 172.21.15.26:0/2043255749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 msgr2=0x7fb25006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 --2- 172.21.15.26:0/2043255749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25006bf30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb240009990 tx=0x7fb240009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 -- 172.21.15.26:0/2043255749 shutdown_connections 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 --2- 172.21.15.26:0/2043255749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb25006c470 0x7fb2501103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 --2- 172.21.15.26:0/2043255749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2501095a0 0x7fb250109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 --2- 172.21.15.26:0/2043255749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25006bf30 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 -- 172.21.15.26:0/2043255749 >> 172.21.15.26:0/2043255749 conn(0x7fb2500793b0 msgr2=0x7fb2500797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 -- 172.21.15.26:0/2043255749 shutdown_connections 2024-05-20T21:06:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.350+0000 7fb258095700 1 -- 172.21.15.26:0/2043255749 wait complete. 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 Processor -- start 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 -- start start 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 0x7fb25019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2501095a0 0x7fb2501a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb250113640 con 0x7fb25006bab0 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501134c0 con 0x7fb25006c470 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb258095700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501137c0 con 0x7fb2501095a0 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb255630700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 0x7fb25019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb255e31700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb256632700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2501095a0 0x7fb2501a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb255630700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 0x7fb25019c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43902/0 (socket says 172.21.15.26:43902) 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb255630700 1 -- 172.21.15.26:0/3555649179 learned_addr learned my addr 172.21.15.26:0/3555649179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.351+0000 7fb255e31700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35510/0 (socket says 172.21.15.26:35510) 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb255630700 1 -- 172.21.15.26:0/3555649179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2501095a0 msgr2=0x7fb2501a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb255630700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2501095a0 0x7fb2501a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb255630700 1 -- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 msgr2=0x7fb25019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb255630700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb255630700 1 -- 172.21.15.26:0/3555649179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2501a0e00 con 0x7fb25006c470 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb255630700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 0x7fb25019c330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b650 tx=0x7fb24000bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240027070 con 0x7fb25006c470 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.352+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb240022470 con 0x7fb25006c470 2024-05-20T21:06:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.353+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240017650 con 0x7fb25006c470 2024-05-20T21:06:01.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.353+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501a1090 con 0x7fb25006c470 2024-05-20T21:06:01.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.353+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501a8970 con 0x7fb25006c470 2024-05-20T21:06:01.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.354+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25006a230 con 0x7fb25006c470 2024-05-20T21:06:01.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.356+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb24002a070 con 0x7fb25006c470 2024-05-20T21:06:01.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.357+0000 7fb246ffd700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb23c0b2900 0x7fb23c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:01.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.357+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2400e3290 con 0x7fb25006c470 2024-05-20T21:06:01.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.357+0000 7fb255e31700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb23c0b2900 0x7fb23c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:01.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.357+0000 7fb255e31700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb23c0b2900 0x7fb23c0b4dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009c60 tx=0x7fb24c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:01.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.360+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb240013070 con 0x7fb25006c470 2024-05-20T21:06:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:00 smithi055 ceph-mon[35213]: pgmap v686: 97 pgs: 97 active+clean; 26 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-05-20T21:06:01.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 216763 2.22 MB/sec execute 367 sec latency 18.686 ms 2024-05-20T21:06:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.673+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb25019d150 con 0x7fb25006c470 2024-05-20T21:06:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.674+0000 7fb246ffd700 1 -- 172.21.15.26:0/3555649179 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fb25019d150 con 0x7fb25006c470 2024-05-20T21:06:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:06:01.675 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb23c0b2900 msgr2=0x7fb23c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb23c0b2900 0x7fb23c0b4dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009c60 tx=0x7fb24c009260 comp rx=0 tx=0).stop 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 msgr2=0x7fb25019c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 0x7fb25019c330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b650 tx=0x7fb24000bfc0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 shutdown_connections 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb23c0b2900 0x7fb23c0b4dc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25006bab0 0x7fb25019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2501095a0 0x7fb2501a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 --2- 172.21.15.26:0/3555649179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25006c470 0x7fb25019c330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 >> 172.21.15.26:0/3555649179 conn(0x7fb2500793b0 msgr2=0x7fb25010e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 shutdown_connections 2024-05-20T21:06:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:01.676+0000 7fb258095700 1 -- 172.21.15.26:0/3555649179 wait complete. 2024-05-20T21:06:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:06:02.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:01 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3555649179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:01 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3555649179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:02.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:01 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3555649179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:02.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 217289 2.22 MB/sec execute 368 sec latency 20.752 ms 2024-05-20T21:06:03.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:02 smithi026 ceph-mon[31990]: pgmap v687: 97 pgs: 97 active+clean; 27 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 412 KiB/s wr, 27 op/s 2024-05-20T21:06:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:02 smithi144 ceph-mon[35660]: pgmap v687: 97 pgs: 97 active+clean; 27 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 412 KiB/s wr, 27 op/s 2024-05-20T21:06:03.414 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 146.55 --await-for 45.0' 2024-05-20T21:06:03.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 146.55 --await-for 45.0 2024-05-20T21:06:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:02 smithi055 ceph-mon[35213]: pgmap v687: 97 pgs: 97 active+clean; 27 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 412 KiB/s wr, 27 op/s 2024-05-20T21:06:03.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 217980 2.22 MB/sec execute 369 sec latency 13.708 ms 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.698+0000 7fee621a4640 1 -- 172.21.15.26:0/213080727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1112d0 msgr2=0x7fee5c1136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.698+0000 7fee621a4640 1 --2- 172.21.15.26:0/213080727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1112d0 0x7fee5c1136c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a8c0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 -- 172.21.15.26:0/213080727 shutdown_connections 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 --2- 172.21.15.26:0/213080727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c1112d0 0x7fee5c1136c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 --2- 172.21.15.26:0/213080727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c10a3b0 0x7fee5c110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 --2- 172.21.15.26:0/213080727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee5c109a50 0x7fee5c109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 -- 172.21.15.26:0/213080727 >> 172.21.15.26:0/213080727 conn(0x7fee5c06fac0 msgr2=0x7fee5c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 -- 172.21.15.26:0/213080727 shutdown_connections 2024-05-20T21:06:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 -- 172.21.15.26:0/213080727 wait complete. 2024-05-20T21:06:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 Processor -- start 2024-05-20T21:06:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.699+0000 7fee621a4640 1 -- start start 2024-05-20T21:06:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee621a4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee5c109a50 0x7fee5c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee621a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 0x7fee5c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee621a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c1112d0 0x7fee5c1a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee621a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c116cf0 con 0x7fee5c10a3b0 2024-05-20T21:06:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee621a4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c116b70 con 0x7fee5c1112d0 2024-05-20T21:06:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee621a4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c116e70 con 0x7fee5c109a50 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 0x7fee5c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 0x7fee5c1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35534/0 (socket says 172.21.15.26:35534) 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 -- 172.21.15.26:0/1192965245 learned_addr learned my addr 172.21.15.26:0/1192965245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee5c109a50 0x7fee5c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 -- 172.21.15.26:0/1192965245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee5c109a50 msgr2=0x7fee5c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee5c109a50 0x7fee5c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 -- 172.21.15.26:0/1192965245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c1112d0 msgr2=0x7fee5c1a8850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c1112d0 0x7fee5c1a8850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 -- 172.21.15.26:0/1192965245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c1a8ec0 con 0x7fee5c10a3b0 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.700+0000 7fee5affd640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 0x7fee5c1a17e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fee5000d950 tx=0x7fee5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.701+0000 7fee58ff9640 1 -- 172.21.15.26:0/1192965245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50017030 con 0x7fee5c10a3b0 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.701+0000 7fee58ff9640 1 -- 172.21.15.26:0/1192965245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee50004020 con 0x7fee5c10a3b0 2024-05-20T21:06:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.701+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c1a91b0 con 0x7fee5c10a3b0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.701+0000 7fee58ff9640 1 -- 172.21.15.26:0/1192965245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500055d0 con 0x7fee5c10a3b0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.701+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1a9750 con 0x7fee5c10a3b0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.703+0000 7fee58ff9640 1 -- 172.21.15.26:0/1192965245 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fee50009a30 con 0x7fee5c10a3b0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 Processor -- start 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 -- start start 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b1afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b1dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563619a82c30 con 0x563619b1b630 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563619a727d0 con 0x563619b164c0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee64f96740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563619a76810 con 0x563619b15ad0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee58ff9640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee440b2920 0x7fee440b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee5b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b1afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b1dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee5b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b1afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43926/0 (socket says 172.21.15.26:43926) 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee5b7fe640 1 -- 172.21.15.26:0/2965967665 learned_addr learned my addr 172.21.15.26:0/2965967665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.704+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee440b2920 0x7fee440b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee58ff9640 1 -- 172.21.15.26:0/1192965245 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fee500e1cc0 con 0x7fee5c10a3b0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5b7fe640 1 -- 172.21.15.26:0/2965967665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 msgr2=0x563619b15ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b15ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5b7fe640 1 -- 172.21.15.26:0/2965967665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 msgr2=0x563619b1dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b1dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5b7fe640 1 -- 172.21.15.26:0/2965967665 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563619787210 con 0x563619b164c0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee440b2920 0x7fee440b4de0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee540319e0 tx=0x7fee54009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b1afc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00cbf0 tx=0x7fee4c00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6065d171f7d19ac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee41ffb640 1 -- 172.21.15.26:0/2965967665 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c010430 con 0x563619b164c0 2024-05-20T21:06:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee41ffb640 1 -- 172.21.15.26:0/2965967665 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee4c010a60 con 0x563619b164c0 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.705+0000 7fee41ffb640 1 -- 172.21.15.26:0/2965967665 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c019910 con 0x563619b164c0 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 -- 172.21.15.26:0/2965967665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 msgr2=0x563619b1afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b1afc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00cbf0 tx=0x7fee4c00e350 comp rx=0 tx=0).stop 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 -- 172.21.15.26:0/2965967665 shutdown_connections 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b1dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b1afc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 --2- 172.21.15.26:0/2965967665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b15ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 -- 172.21.15.26:0/2965967665 >> 172.21.15.26:0/2965967665 conn(0x563619b0cd30 msgr2=0x563619b0d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 -- 172.21.15.26:0/2965967665 shutdown_connections 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 -- 172.21.15.26:0/2965967665 wait complete. 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.706+0000 7fee64f96740 1 -- start start 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 Processor -- start 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b1fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b20130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b27170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee5affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b20130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee5affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b20130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43942/0 (socket says 172.21.15.26:43942) 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee5b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b1fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee5b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b1fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:60010/0 (socket says 172.21.15.26:60010) 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee5affd640 1 -- 172.21.15.26:0/3883079530 learned_addr learned my addr 172.21.15.26:0/3883079530 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563619758790 con 0x563619b1b630 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563619a2e890 con 0x563619b164c0 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.707+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563619b1e5e0 con 0x563619b15ad0 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee64f96740 10 client.0 _ll_register_callbacks cb 0x563619786a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5bfff640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b27170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5affd640 1 -- 172.21.15.26:0/3883079530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 msgr2=0x563619b1fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5affd640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b1fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5affd640 1 -- 172.21.15.26:0/3883079530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 msgr2=0x563619b27170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5affd640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b27170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5affd640 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563619853620 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5bfff640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b27170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee5affd640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b20130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fee50005fd0 tx=0x7fee500a8cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e040c41a7715b5ad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee3affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee3affd640 1 -- 172.21.15.26:0/3883079530 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500c3060 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee3affd640 1 -- 172.21.15.26:0/3883079530 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fee50009d20 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4c010a60 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee3affd640 1 -- 172.21.15.26:0/3883079530 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee500ca050 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee64f96740 10 client.27728 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.708+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563619b2a1c0 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee3affd640 1 -- 172.21.15.26:0/3883079530 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee4c010a60 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee3affd640 1 -- 172.21.15.26:0/3883079530 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fee500a5510 con 0x563619b164c0 2024-05-20T21:06:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee64f96740 10 client.27728 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee64f96740 10 client.27728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee64f96740 20 client.27728 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee64f96740 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b2b910 0x563619b2dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee64f96740 4 client.27728 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563619a84b80 con 0x563619b2b910 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b2b910 0x563619b2dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee3affd640 10 client.27728 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.709+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b2b910 0x563619b2dd40 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc770558a259cc8 server_cookie=d053db7d2191cf50 in_seq=0 out_seq=0 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee3affd640 1 -- 172.21.15.26:0/3883079530 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563619a84b80 con 0x563619b2b910 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee3affd640 10 client.27728 handle_command_reply: tid=0 2024-05-20T21:06:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 client.27728 shutdown 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 msgr2=0x563619b20130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b20130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fee50005fd0 tx=0x7fee500a8cc0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 shutdown_connections 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b2b910 0x563619b2dd40 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b27170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b20130 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee3affd640 0 client.27728 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 --2- 172.21.15.26:0/3883079530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b1fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.710+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 >> 172.21.15.26:0/3883079530 conn(0x563619b0cd30 msgr2=0x563619b0bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.711+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 shutdown_connections 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.711+0000 7fee64f96740 1 -- 172.21.15.26:0/3883079530 wait complete. 2024-05-20T21:06:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.711+0000 7fee64f96740 20 client.27728 trim_cache size 0 max 16384 2024-05-20T21:06:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 Processor -- start 2024-05-20T21:06:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 -- start start 2024-05-20T21:06:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1b2da0 con 0x563619b1b630 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1b0670 con 0x563619b164c0 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee64f96740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1ae4c0 con 0x563619b15ad0 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b0fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35558/0 (socket says 172.21.15.26:35558) 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee5bfff640 1 -- 172.21.15.26:0/2903756325 learned_addr learned my addr 172.21.15.26:0/2903756325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee5affd640 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.726+0000 7fee5bfff640 1 -- 172.21.15.26:0/2903756325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 msgr2=0x563619b27f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5bfff640 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b27f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5bfff640 1 -- 172.21.15.26:0/2903756325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 msgr2=0x563619b27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5bfff640 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b27130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5bfff640 1 -- 172.21.15.26:0/2903756325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563619b086f0 con 0x563619b1b630 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5affd640 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee5bfff640 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b0fc40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fee54032bb0 tx=0x7fee5403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9612a0b6e9a27f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee417fa640 1 -- 172.21.15.26:0/2903756325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54003db0 con 0x563619b1b630 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee417fa640 1 -- 172.21.15.26:0/2903756325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee5404b070 con 0x563619b1b630 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.727+0000 7fee417fa640 1 -- 172.21.15.26:0/2903756325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54030b10 con 0x563619b1b630 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- 172.21.15.26:0/2903756325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 msgr2=0x563619b0fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b0fc40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fee54032bb0 tx=0x7fee5403d000 comp rx=0 tx=0).stop 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- 172.21.15.26:0/2903756325 shutdown_connections 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b1b630 0x563619b0fc40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b164c0 0x563619b27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- 172.21.15.26:0/2903756325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b15ad0 0x563619b27f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- 172.21.15.26:0/2903756325 >> 172.21.15.26:0/2903756325 conn(0x563619b0cd30 msgr2=0x563619b27800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- 172.21.15.26:0/2903756325 shutdown_connections 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- 172.21.15.26:0/2903756325 wait complete. 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- start start 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 Processor -- start 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 0x563619b24270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b164c0 0x563619b247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b1b630 0x563619b257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1aeb30 con 0x563619b15ad0 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1affa0 con 0x563619b1b630 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1af200 con 0x563619b164c0 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee64f96740 10 client.0 _ll_register_callbacks cb 0x7fee50005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 0x563619b24270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 0x563619b24270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35568/0 (socket says 172.21.15.26:35568) 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.728+0000 7fee5b7fe640 1 -- 172.21.15.26:0/3135813074 learned_addr learned my addr 172.21.15.26:0/3135813074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee5b7fe640 1 -- 172.21.15.26:0/3135813074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b164c0 msgr2=0x563619b247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b164c0 0x563619b247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee5b7fe640 1 -- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b1b630 msgr2=0x563619b257c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b1b630 0x563619b257c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee5b7fe640 1 -- 172.21.15.26:0/3135813074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563619b25db0 con 0x563619b15ad0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 0x563619b24270 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000c00 tx=0x7fee4c00e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa7d59fb6d9ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee3affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee3affd640 1 -- 172.21.15.26:0/3135813074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c005040 con 0x563619b15ad0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee3affd640 1 -- 172.21.15.26:0/3135813074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fee4c0051e0 con 0x563619b15ad0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5404b070 con 0x563619b15ad0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee3affd640 1 -- 172.21.15.26:0/3135813074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee4c0053e0 con 0x563619b15ad0 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee64f96740 10 client.18369 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563619b27060 con 0x563619b15ad0 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.729+0000 7fee3affd640 1 -- 172.21.15.26:0/3135813074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee5404b070 con 0x563619b15ad0 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee3affd640 1 -- 172.21.15.26:0/3135813074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fee4c01a410 con 0x563619b15ad0 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee64f96740 10 client.18369 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee64f96740 10 client.18369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee64f96740 20 client.18369 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee64f96740 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b965c0 0x563619b989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee64f96740 4 client.18369 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563619a7e360 con 0x563619b965c0 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee5affd640 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b965c0 0x563619b989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee3affd640 10 client.18369 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.730+0000 7fee5affd640 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b965c0 0x563619b989b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a24735755a4504 server_cookie=75e83f06ed648e36 in_seq=0 out_seq=0 2024-05-20T21:06:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee3affd640 1 -- 172.21.15.26:0/3135813074 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563619a7e360 con 0x563619b965c0 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee3affd640 10 client.18369 handle_command_reply: tid=0 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 client.18369 shutdown 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 msgr2=0x563619b24270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 0x563619b24270 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000c00 tx=0x7fee4c00e920 comp rx=0 tx=0).stop 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 shutdown_connections 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563619b965c0 0x563619b989b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563619b1b630 0x563619b257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563619b164c0 0x563619b247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 --2- 172.21.15.26:0/3135813074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563619b15ad0 0x563619b24270 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee3affd640 0 client.18369 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.768+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 >> 172.21.15.26:0/3135813074 conn(0x563619b0cd30 msgr2=0x563619b0fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 shutdown_connections 2024-05-20T21:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee64f96740 1 -- 172.21.15.26:0/3135813074 wait complete. 2024-05-20T21:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee64f96740 20 client.18369 trim_cache size 0 max 16384 2024-05-20T21:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee440b2920 msgr2=0x7fee440b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee440b2920 0x7fee440b4de0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee540319e0 tx=0x7fee54009360 comp rx=0 tx=0).stop 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 msgr2=0x7fee5c1a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 0x7fee5c1a17e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fee5000d950 tx=0x7fee5000de10 comp rx=0 tx=0).stop 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 shutdown_connections 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fee440b2920 0x7fee440b4de0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c1112d0 0x7fee5c1a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c10a3b0 0x7fee5c1a17e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 --2- 172.21.15.26:0/1192965245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee5c109a50 0x7fee5c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 >> 172.21.15.26:0/1192965245 conn(0x7fee5c06fac0 msgr2=0x7fee5c0716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 shutdown_connections 2024-05-20T21:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:03.769+0000 7fee621a4640 1 -- 172.21.15.26:0/1192965245 wait complete. 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 -- 172.21.15.26:0/1701170136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0108500 msgr2=0x7f22c01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 --2- 172.21.15.26:0/1701170136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0108500 0x7f22c01088e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0098f0 tx=0x7f22bc009c00 comp rx=0 tx=0).stop 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 -- 172.21.15.26:0/1701170136 shutdown_connections 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 --2- 172.21.15.26:0/1701170136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22c0103f90 0x7f22c0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 --2- 172.21.15.26:0/1701170136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0108500 0x7f22c01088e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 --2- 172.21.15.26:0/1701170136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c01035f0 0x7f22c0103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.947+0000 7f22c7f76700 1 -- 172.21.15.26:0/1701170136 >> 172.21.15.26:0/1701170136 conn(0x7f22c007a6a0 msgr2=0x7f22c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.948+0000 7f22c7f76700 1 -- 172.21.15.26:0/1701170136 shutdown_connections 2024-05-20T21:06:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.948+0000 7f22c7f76700 1 -- 172.21.15.26:0/1701170136 wait complete. 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.948+0000 7f22c7f76700 1 Processor -- start 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.948+0000 7f22c7f76700 1 -- start start 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c7f76700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22c01035f0 0x7f22c019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c7f76700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0103f90 0x7f22c019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c7f76700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 0x7f22c01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c7f76700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0113620 con 0x7f22c0108500 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c7f76700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01134a0 con 0x7f22c0103f90 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c7f76700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01137a0 con 0x7f22c01035f0 2024-05-20T21:06:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c6513700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 0x7f22c01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c6513700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 0x7f22c01999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35582/0 (socket says 172.21.15.26:35582) 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c6513700 1 -- 172.21.15.26:0/353895164 learned_addr learned my addr 172.21.15.26:0/353895164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c5d12700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22c01035f0 0x7f22c019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c5511700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0103f90 0x7f22c019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.949+0000 7f22c6513700 1 -- 172.21.15.26:0/353895164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22c01035f0 msgr2=0x7f22c019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c6513700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22c01035f0 0x7f22c019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c6513700 1 -- 172.21.15.26:0/353895164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0103f90 msgr2=0x7f22c019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c6513700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0103f90 0x7f22c019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c6513700 1 -- 172.21.15.26:0/353895164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c019a140 con 0x7f22c0108500 2024-05-20T21:06:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c6513700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 0x7f22c01999e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f22b8007f10 tx=0x7f22b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c5511700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0103f90 0x7f22c019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22b6ffd700 1 -- 172.21.15.26:0/353895164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8009e20 con 0x7f22c0108500 2024-05-20T21:06:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22b6ffd700 1 -- 172.21.15.26:0/353895164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22b8004bb0 con 0x7f22c0108500 2024-05-20T21:06:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.950+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c019a430 con 0x7f22c0108500 2024-05-20T21:06:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.951+0000 7f22b6ffd700 1 -- 172.21.15.26:0/353895164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8005840 con 0x7f22c0108500 2024-05-20T21:06:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.951+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c01a66f0 con 0x7f22c0108500 2024-05-20T21:06:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.954+0000 7f22b6ffd700 1 -- 172.21.15.26:0/353895164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f22b801a020 con 0x7f22c0108500 2024-05-20T21:06:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.955+0000 7f22b6ffd700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22ac0b29d0 0x7f22ac0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.955+0000 7f22b6ffd700 1 -- 172.21.15.26:0/353895164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22b80e1750 con 0x7f22c0108500 2024-05-20T21:06:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.955+0000 7f22c5d12700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22ac0b29d0 0x7f22ac0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 Processor -- start 2024-05-20T21:06:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 -- start start 2024-05-20T21:06:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22c5d12700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22ac0b29d0 0x7f22ac0b4e90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c010 tx=0x7f22bc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28ce9860 0x55ad28ce9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 0x55ad28ced940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ad28c48c90 con 0x55ad28cea210 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ad28c38830 con 0x55ad28cedf60 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22d280cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad28c3c870 con 0x55ad28ce9860 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.956+0000 7f22c5511700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 0x55ad28ced940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 0x55ad28ced940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35586/0 (socket says 172.21.15.26:35586) 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 -- 172.21.15.26:0/3269567455 learned_addr learned my addr 172.21.15.26:0/3269567455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c6513700 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5d12700 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28ce9860 0x55ad28ce9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 -- 172.21.15.26:0/3269567455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28ce9860 msgr2=0x55ad28ce9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28ce9860 0x55ad28ce9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 -- 172.21.15.26:0/3269567455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 msgr2=0x55ad28cee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 -- 172.21.15.26:0/3269567455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad28b888d0 con 0x55ad28cea210 2024-05-20T21:06:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c6513700 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22c5511700 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 0x55ad28ced940 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f22b0011630 tx=0x7f22b0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a9066981269e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.957+0000 7f22abfff700 1 -- 172.21.15.26:0/3269567455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004a30 con 0x55ad28cea210 2024-05-20T21:06:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22abfff700 1 -- 172.21.15.26:0/3269567455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22b00054b0 con 0x55ad28cea210 2024-05-20T21:06:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22abfff700 1 -- 172.21.15.26:0/3269567455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0016880 con 0x55ad28cea210 2024-05-20T21:06:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 -- 172.21.15.26:0/3269567455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 msgr2=0x55ad28ced940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 0x55ad28ced940 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f22b0011630 tx=0x7f22b0011940 comp rx=0 tx=0).stop 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 -- 172.21.15.26:0/3269567455 shutdown_connections 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28ce9860 0x55ad28ce9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3269567455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cea210 0x55ad28ced940 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.958+0000 7f22d280cb80 1 -- 172.21.15.26:0/3269567455 >> 172.21.15.26:0/3269567455 conn(0x55ad28c17080 msgr2=0x55ad28ce3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.959+0000 7f22d280cb80 1 -- 172.21.15.26:0/3269567455 shutdown_connections 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.959+0000 7f22d280cb80 1 -- 172.21.15.26:0/3269567455 wait complete. 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.959+0000 7f22d280cb80 1 -- start start 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.959+0000 7f22d280cb80 1 Processor -- start 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cf5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cea210 0x55ad28cf2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cedf60 0x55ad28cf2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ad28b3efa0 con 0x55ad28ce9860 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ad28bda7c0 con 0x55ad28cea210 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad28cf6120 con 0x55ad28cedf60 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cf5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22d280cb80 10 client.0 _ll_register_callbacks cb 0x55ad28bd3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cf5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35600/0 (socket says 172.21.15.26:35600) 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 -- 172.21.15.26:0/3640404175 learned_addr learned my addr 172.21.15.26:0/3640404175 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c6513700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cedf60 0x55ad28cf2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5511700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cea210 0x55ad28cf2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 -- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cedf60 msgr2=0x55ad28cf2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cedf60 0x55ad28cf2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 -- 172.21.15.26:0/3640404175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cea210 msgr2=0x55ad28cf2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cea210 0x55ad28cf2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 -- 172.21.15.26:0/3640404175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad28bd7690 con 0x55ad28ce9860 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22c5d12700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cf5350 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f22c01047d0 tx=0x7f22bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283312e5416f51ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.960+0000 7f22a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22a97fa700 1 -- 172.21.15.26:0/3640404175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc005400 con 0x55ad28ce9860 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22b00054b0 con 0x55ad28ce9860 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22a97fa700 1 -- 172.21.15.26:0/3640404175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22bc0055a0 con 0x55ad28ce9860 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22a97fa700 1 -- 172.21.15.26:0/3640404175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22bc003680 con 0x55ad28ce9860 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22d280cb80 10 client.18381 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad28d007c0 con 0x55ad28ce9860 2024-05-20T21:06:03.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.961+0000 7f22a97fa700 1 -- 172.21.15.26:0/3640404175 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22b00054b0 con 0x55ad28ce9860 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22a97fa700 1 -- 172.21.15.26:0/3640404175 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f22bc038020 con 0x55ad28ce9860 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22d280cb80 10 client.18381 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22d280cb80 10 client.18381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22d280cb80 20 client.18381 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d02da0 0x55ad28d05190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22d280cb80 4 client.18381 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad28cb59e0 con 0x55ad28d02da0 2024-05-20T21:06:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:03.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.962+0000 7f22c5511700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d02da0 0x55ad28d05190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.963+0000 7f22a97fa700 10 client.18381 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.963+0000 7f22c5511700 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d02da0 0x55ad28d05190 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be21f695570297ef server_cookie=17489b6c6285cc66 in_seq=0 out_seq=0 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22a97fa700 1 -- 172.21.15.26:0/3640404175 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ad28cb59e0 con 0x55ad28d02da0 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22a97fa700 10 client.18381 handle_command_reply: tid=0 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 client.18381 shutdown 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 msgr2=0x55ad28cf5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cf5350 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f22c01047d0 tx=0x7f22bc009260 comp rx=0 tx=0).stop 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 shutdown_connections 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d02da0 0x55ad28d05190 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cf5350 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cedf60 0x55ad28cf2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22a97fa700 0 client.18381 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 --2- 172.21.15.26:0/3640404175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cea210 0x55ad28cf2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.964+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 >> 172.21.15.26:0/3640404175 conn(0x55ad28c17080 msgr2=0x55ad28ce3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.965+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 shutdown_connections 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.965+0000 7f22d280cb80 1 -- 172.21.15.26:0/3640404175 wait complete. 2024-05-20T21:06:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.965+0000 7f22d280cb80 20 client.18381 trim_cache size 0 max 16384 2024-05-20T21:06:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 Processor -- start 2024-05-20T21:06:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 -- start start 2024-05-20T21:06:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 0x55ad28cf4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28d0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cedf60 0x55ad28d0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01ae340 con 0x55ad28cedf60 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01abbd0 con 0x55ad28ce9860 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22d280cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01a9a20 con 0x55ad28cea210 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22c5d12700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 0x55ad28cf4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22c5d12700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 0x55ad28cf4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44002/0 (socket says 172.21.15.26:44002) 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22c5d12700 1 -- 172.21.15.26:0/1129955544 learned_addr learned my addr 172.21.15.26:0/1129955544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22c6513700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cedf60 0x55ad28d0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.986+0000 7f22c5511700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28d0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5d12700 1 -- 172.21.15.26:0/1129955544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 msgr2=0x55ad28d0aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5d12700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28d0aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5d12700 1 -- 172.21.15.26:0/1129955544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cedf60 msgr2=0x55ad28d0afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5d12700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cedf60 0x55ad28d0afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5d12700 1 -- 172.21.15.26:0/1129955544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad28cf0bc0 con 0x55ad28ce9860 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c6513700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cedf60 0x55ad28d0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5511700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28d0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22c5d12700 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 0x55ad28cf4590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00ba20 tx=0x7f22bc00b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355f1568957163c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22ab7fe700 1 -- 172.21.15.26:0/1129955544 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00bc10 con 0x55ad28ce9860 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22ab7fe700 1 -- 172.21.15.26:0/1129955544 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22bc03f020 con 0x55ad28ce9860 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.987+0000 7f22ab7fe700 1 -- 172.21.15.26:0/1129955544 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc005440 con 0x55ad28ce9860 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 -- 172.21.15.26:0/1129955544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 msgr2=0x55ad28cf4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 0x55ad28cf4590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00ba20 tx=0x7f22bc00b770 comp rx=0 tx=0).stop 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 -- 172.21.15.26:0/1129955544 shutdown_connections 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28cedf60 0x55ad28d0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28ce9860 0x55ad28cf4590 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 --2- 172.21.15.26:0/1129955544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28d0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 -- 172.21.15.26:0/1129955544 >> 172.21.15.26:0/1129955544 conn(0x55ad28c17080 msgr2=0x55ad28cf3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 -- 172.21.15.26:0/1129955544 shutdown_connections 2024-05-20T21:06:03.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.988+0000 7f22d280cb80 1 -- 172.21.15.26:0/1129955544 wait complete. 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 -- start start 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 Processor -- start 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cfb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28cf8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cf9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01aa090 con 0x55ad28ce9860 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01ab500 con 0x55ad28cedf60 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01aa760 con 0x55ad28cea210 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22d280cb80 10 client.0 _ll_register_callbacks cb 0x55ad28bd3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22c5d12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cfb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22c5d12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cfb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35616/0 (socket says 172.21.15.26:35616) 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22c5d12700 1 -- 172.21.15.26:0/533190706 learned_addr learned my addr 172.21.15.26:0/533190706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22c6513700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cf9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.989+0000 7f22c5511700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28cf8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5d12700 1 -- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 msgr2=0x55ad28cf8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5d12700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28cf8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5d12700 1 -- 172.21.15.26:0/533190706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 msgr2=0x55ad28cf9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5d12700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cf9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5d12700 1 -- 172.21.15.26:0/533190706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad28cf9940 con 0x55ad28ce9860 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5511700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28cf8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c6513700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cf9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22c5d12700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cfb960 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f22bc005610 tx=0x7f22bc005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ad39c756a4d176 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22a97fa700 1 -- 172.21.15.26:0/533190706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc048060 con 0x55ad28ce9860 2024-05-20T21:06:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22a97fa700 1 -- 172.21.15.26:0/533190706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22bc02c040 con 0x55ad28ce9860 2024-05-20T21:06:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.990+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22bc03f020 con 0x55ad28ce9860 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.993+0000 7f22a97fa700 1 -- 172.21.15.26:0/533190706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22bc003a30 con 0x55ad28ce9860 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.993+0000 7f22d280cb80 10 client.18387 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.993+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad28d76080 con 0x55ad28ce9860 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22a97fa700 1 -- 172.21.15.26:0/533190706 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22bc03f020 con 0x55ad28ce9860 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22a97fa700 1 -- 172.21.15.26:0/533190706 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f22bc038070 con 0x55ad28ce9860 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22d280cb80 10 client.18387 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22d280cb80 10 client.18387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22d280cb80 20 client.18387 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22d280cb80 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d765f0 0x55ad28d78a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22d280cb80 4 client.18387 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad28c4abe0 con 0x55ad28d765f0 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.994+0000 7f22c5511700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d765f0 0x55ad28d78a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22a97fa700 10 client.18387 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22c5511700 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d765f0 0x55ad28d78a10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=900b3bdd62e405d3 server_cookie=917bd9316df37d95 in_seq=0 out_seq=0 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22a97fa700 1 -- 172.21.15.26:0/533190706 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad28c4abe0 con 0x55ad28d765f0 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22a97fa700 10 client.18387 handle_command_reply: tid=0 2024-05-20T21:06:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 client.18387 shutdown 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 msgr2=0x55ad28cfb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cfb960 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f22bc005610 tx=0x7f22bc005640 comp rx=0 tx=0).stop 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 shutdown_connections 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ad28d765f0 0x55ad28d78a10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad28ce9860 0x55ad28cfb960 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ad28cedf60 0x55ad28cf9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22a97fa700 0 client.18387 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 --2- 172.21.15.26:0/533190706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad28cea210 0x55ad28cf8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.995+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 >> 172.21.15.26:0/533190706 conn(0x55ad28c17080 msgr2=0x55ad28ce8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.996+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 shutdown_connections 2024-05-20T21:06:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.996+0000 7f22d280cb80 1 -- 172.21.15.26:0/533190706 wait complete. 2024-05-20T21:06:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.996+0000 7f22d280cb80 20 client.18387 trim_cache size 0 max 16384 2024-05-20T21:06:03.997 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.996+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22ac0b29d0 msgr2=0x7f22ac0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.996+0000 7f22c7f76700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22ac0b29d0 0x7f22ac0b4e90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c010 tx=0x7f22bc019040 comp rx=0 tx=0).stop 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 msgr2=0x7f22c01999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 0x7f22c01999e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f22b8007f10 tx=0x7f22b800d3b0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 shutdown_connections 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22ac0b29d0 0x7f22ac0b4e90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22c0108500 0x7f22c01999e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22c01035f0 0x7f22c019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 --2- 172.21.15.26:0/353895164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c0103f90 0x7f22c019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 >> 172.21.15.26:0/353895164 conn(0x7f22c007a6a0 msgr2=0x7f22c0101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 shutdown_connections 2024-05-20T21:06:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:03.997+0000 7f22c7f76700 1 -- 172.21.15.26:0/353895164 wait complete. 2024-05-20T21:06:04.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 218645 2.22 MB/sec execute 370 sec latency 34.918 ms 2024-05-20T21:06:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:04 smithi144 ceph-mon[35660]: pgmap v688: 97 pgs: 97 active+clean; 27 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 748 KiB/s wr, 53 op/s 2024-05-20T21:06:05.379 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:04 smithi026 ceph-mon[31990]: pgmap v688: 97 pgs: 97 active+clean; 27 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 748 KiB/s wr, 53 op/s 2024-05-20T21:06:05.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:06:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:04 smithi055 ceph-mon[35213]: pgmap v688: 97 pgs: 97 active+clean; 27 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 748 KiB/s wr, 53 op/s 2024-05-20T21:06:05.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219420 2.23 MB/sec execute 371 sec latency 19.139 ms 2024-05-20T21:06:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:06.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219972 2.24 MB/sec execute 372 sec latency 78.416 ms 2024-05-20T21:06:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 -- 172.21.15.26:0/1381023772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 msgr2=0x7ff96810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 --2- 172.21.15.26:0/1381023772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff96810f2f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff96000b0d0 tx=0x7ff96000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 -- 172.21.15.26:0/1381023772 shutdown_connections 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 --2- 172.21.15.26:0/1381023772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff96810f2f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 --2- 172.21.15.26:0/1381023772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff968102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 --2- 172.21.15.26:0/1381023772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff968107070 0x7ff9681074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 -- 172.21.15.26:0/1381023772 >> 172.21.15.26:0/1381023772 conn(0x7ff9680793f0 msgr2=0x7ff968079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 -- 172.21.15.26:0/1381023772 shutdown_connections 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.453+0000 7ff96edc9700 1 -- 172.21.15.26:0/1381023772 wait complete. 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 Processor -- start 2024-05-20T21:06:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 -- start start 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff96819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff968107070 0x7ff96819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff968199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681124a0 con 0x7ff968107a30 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff968112320 con 0x7ff968102ae0 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.454+0000 7ff96edc9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff968112620 con 0x7ff968107070 2024-05-20T21:06:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff96819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96d366700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff968199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff96819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:44036/0 (socket says 172.21.15.26:44036) 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 -- 172.21.15.26:0/3580774748 learned_addr learned my addr 172.21.15.26:0/3580774748 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff967fff700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff968107070 0x7ff96819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 -- 172.21.15.26:0/3580774748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff968107070 msgr2=0x7ff96819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff968107070 0x7ff96819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 -- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 msgr2=0x7ff968199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff968199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 -- 172.21.15.26:0/3580774748 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96819a030 con 0x7ff968102ae0 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96d366700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff968199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96cb65700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff96819f680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e9a0 tx=0x7ff95c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff965ffb700 1 -- 172.21.15.26:0/3580774748 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00ca10 con 0x7ff968102ae0 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96819a320 con 0x7ff968102ae0 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff965ffb700 1 -- 172.21.15.26:0/3580774748 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff95c00cbb0 con 0x7ff968102ae0 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.455+0000 7ff965ffb700 1 -- 172.21.15.26:0/3580774748 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c018680 con 0x7ff968102ae0 2024-05-20T21:06:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.456+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9681a64e0 con 0x7ff968102ae0 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.458+0000 7ff965ffb700 1 -- 172.21.15.26:0/3580774748 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff95c010430 con 0x7ff968102ae0 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.459+0000 7ff965ffb700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9580b2930 0x7ff9580b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff967fff700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9580b2930 0x7ff9580b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff965ffb700 1 -- 172.21.15.26:0/3580774748 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff95c0e2470 con 0x7ff968102ae0 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 Processor -- start 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 -- start start 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff967fff700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9580b2930 0x7ff9580b4df0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff968079a60 tx=0x7ff95400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f32730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f2ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f39180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d694e8be30 con 0x55d694f32350 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d694e7b9d0 con 0x55d694f2d340 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff97965fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d694e7fa10 con 0x55d694f2c980 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff96cb65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f32730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff96cb65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f32730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35636/0 (socket says 172.21.15.26:35636) 2024-05-20T21:06:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff96cb65700 1 -- 172.21.15.26:0/1446496873 learned_addr learned my addr 172.21.15.26:0/1446496873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.460+0000 7ff96d366700 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f39180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff967fff700 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f2ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff96cb65700 1 -- 172.21.15.26:0/1446496873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 msgr2=0x55d694f2ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff96cb65700 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f2ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff96cb65700 1 -- 172.21.15.26:0/1446496873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 msgr2=0x55d694f39180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff96cb65700 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f39180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff96cb65700 1 -- 172.21.15.26:0/1446496873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d694dcba70 con 0x55d694f32350 2024-05-20T21:06:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff967fff700 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f2ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff96cb65700 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f32730 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e970 tx=0x7ff95c0b21f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc9ad5465ffad4a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff952ffd700 1 -- 172.21.15.26:0/1446496873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0aa0d0 con 0x55d694f32350 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff952ffd700 1 -- 172.21.15.26:0/1446496873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff95c0a99d0 con 0x55d694f32350 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.461+0000 7ff952ffd700 1 -- 172.21.15.26:0/1446496873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0cc330 con 0x55d694f32350 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 -- 172.21.15.26:0/1446496873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 msgr2=0x55d694f32730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f32730 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e970 tx=0x7ff95c0b21f0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 -- 172.21.15.26:0/1446496873 shutdown_connections 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f32730 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f39180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 --2- 172.21.15.26:0/1446496873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f2ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 -- 172.21.15.26:0/1446496873 >> 172.21.15.26:0/1446496873 conn(0x55d694e5a220 msgr2=0x55d694f27290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 -- 172.21.15.26:0/1446496873 shutdown_connections 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 -- 172.21.15.26:0/1446496873 wait complete. 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 -- start start 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.462+0000 7ff97965fb80 1 Processor -- start 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f38e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 0x55d694f36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f32350 0x55d694f36670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d694d82140 con 0x55d694f2d340 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d694e1d960 con 0x55d694f32350 2024-05-20T21:06:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d694f39720 con 0x55d694f2c980 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff97965fb80 10 client.0 _ll_register_callbacks cb 0x55d694e16ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff967fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 0x55d694f36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff967fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 0x55d694f36130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35650/0 (socket says 172.21.15.26:35650) 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff967fff700 1 -- 172.21.15.26:0/426916637 learned_addr learned my addr 172.21.15.26:0/426916637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96d366700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f32350 0x55d694f36670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96cb65700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f38e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96d366700 1 -- 172.21.15.26:0/426916637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 msgr2=0x55d694f38e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96d366700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f38e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96d366700 1 -- 172.21.15.26:0/426916637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 msgr2=0x55d694f36130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96d366700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 0x55d694f36130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff96d366700 1 -- 172.21.15.26:0/426916637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d694e1a830 con 0x55d694f32350 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.463+0000 7ff967fff700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 0x55d694f36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff96d366700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f32350 0x55d694f36670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff96000b770 tx=0x7ff960007710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b1c75534651891 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff947fff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff947fff700 1 -- 172.21.15.26:0/426916637 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960003c10 con 0x55d694f32350 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c0a99d0 con 0x55d694f32350 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff947fff700 1 -- 172.21.15.26:0/426916637 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff960003db0 con 0x55d694f32350 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff947fff700 1 -- 172.21.15.26:0/426916637 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff960007880 con 0x55d694f32350 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff97965fb80 10 client.27758 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d694f43ea0 con 0x55d694f32350 2024-05-20T21:06:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.464+0000 7ff947fff700 1 -- 172.21.15.26:0/426916637 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff95c0a99d0 con 0x55d694f32350 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff947fff700 1 -- 172.21.15.26:0/426916637 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff96002f050 con 0x55d694f32350 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff97965fb80 10 client.27758 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff97965fb80 10 client.27758 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff97965fb80 20 client.27758 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff97965fb80 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694f46480 0x55d694f48870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff97965fb80 4 client.27758 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d694ef8b80 con 0x55d694f46480 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff96cb65700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694f46480 0x55d694f48870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff96cb65700 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694f46480 0x55d694f48870 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b36d9574f28f4c server_cookie=faa27aba97c8fe48 in_seq=0 out_seq=0 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.465+0000 7ff947fff700 10 client.27758 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff947fff700 1 -- 172.21.15.26:0/426916637 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d694ef8b80 con 0x55d694f46480 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff947fff700 10 client.27758 handle_command_reply: tid=0 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 client.27758 shutdown 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f32350 msgr2=0x55d694f36670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f32350 0x55d694f36670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff96000b770 tx=0x7ff960007710 comp rx=0 tx=0).stop 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 shutdown_connections 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694f46480 0x55d694f48870 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f32350 0x55d694f36670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f38e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 --2- 172.21.15.26:0/426916637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2d340 0x55d694f36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 >> 172.21.15.26:0/426916637 conn(0x55d694e5a220 msgr2=0x55d694f21d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff947fff700 0 client.27758 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 shutdown_connections 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.466+0000 7ff97965fb80 1 -- 172.21.15.26:0/426916637 wait complete. 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.467+0000 7ff97965fb80 20 client.27758 trim_cache size 0 max 16384 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 Processor -- start 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 -- start start 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 0x55d694f350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f30540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f32350 0x55d694f4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681b0300 con 0x55d694f2c980 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681adb90 con 0x55d694f2d340 2024-05-20T21:06:06.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff97965fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681ab9e0 con 0x55d694f32350 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 0x55d694f350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 0x55d694f350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35654/0 (socket says 172.21.15.26:35654) 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 -- 172.21.15.26:0/585787236 learned_addr learned my addr 172.21.15.26:0/585787236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff967fff700 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f30540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96d366700 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f32350 0x55d694f4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 -- 172.21.15.26:0/585787236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f32350 msgr2=0x55d694f4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f32350 0x55d694f4e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 -- 172.21.15.26:0/585787236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 msgr2=0x55d694f30540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f30540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.491+0000 7ff96cb65700 1 -- 172.21.15.26:0/585787236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d694f277d0 con 0x55d694f2c980 2024-05-20T21:06:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.492+0000 7ff96cb65700 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 0x55d694f350b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0a96d0 tx=0x7ff95c091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9dd55e35d0ced6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.492+0000 7ff9527fc700 1 -- 172.21.15.26:0/585787236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0b2830 con 0x55d694f2c980 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.492+0000 7ff9527fc700 1 -- 172.21.15.26:0/585787236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff95c08f070 con 0x55d694f2c980 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.492+0000 7ff9527fc700 1 -- 172.21.15.26:0/585787236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0cc0d0 con 0x55d694f2c980 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 -- 172.21.15.26:0/585787236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 msgr2=0x55d694f350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 0x55d694f350b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0a96d0 tx=0x7ff95c091040 comp rx=0 tx=0).stop 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 -- 172.21.15.26:0/585787236 shutdown_connections 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f2c980 0x55d694f350b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f32350 0x55d694f4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 --2- 172.21.15.26:0/585787236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f30540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 -- 172.21.15.26:0/585787236 >> 172.21.15.26:0/585787236 conn(0x55d694e5a220 msgr2=0x55d694f2fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 -- 172.21.15.26:0/585787236 shutdown_connections 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 -- 172.21.15.26:0/585787236 wait complete. 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 -- start start 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.493+0000 7ff97965fb80 1 Processor -- start 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f3ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff967fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681ac050 con 0x55d694f32350 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681ad4c0 con 0x55d694f2d340 2024-05-20T21:06:06.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f3bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35662/0 (socket says 172.21.15.26:35662) 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 -- 172.21.15.26:0/2947740812 learned_addr learned my addr 172.21.15.26:0/2947740812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681ac720 con 0x55d694f2c980 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96cb65700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f3ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff97965fb80 10 client.0 _ll_register_callbacks cb 0x55d694e16ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 -- 172.21.15.26:0/2947740812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 msgr2=0x55d694f3ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f3ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 -- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 msgr2=0x55d694f3b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f3b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff96d366700 1 -- 172.21.15.26:0/2947740812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d694f3c400 con 0x55d694f32350 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.494+0000 7ff967fff700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff96d366700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f3bec0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff960005900 tx=0x7ff960009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8732229bbc62888 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff947fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff947fff700 1 -- 172.21.15.26:0/2947740812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960007db0 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff947fff700 1 -- 172.21.15.26:0/2947740812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff960022040 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c08f070 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff947fff700 1 -- 172.21.15.26:0/2947740812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff960005110 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff97965fb80 10 client.18405 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d694fb6060 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff947fff700 1 -- 172.21.15.26:0/2947740812 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff95c08f070 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff947fff700 1 -- 172.21.15.26:0/2947740812 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff96002f040 con 0x55d694f32350 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff97965fb80 10 client.18405 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff97965fb80 10 client.18405 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.495+0000 7ff97965fb80 20 client.18405 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.496+0000 7ff97965fb80 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694fb66f0 0x55d694fb8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.496+0000 7ff97965fb80 4 client.18405 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.55, "await_for": 45.0}] 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.496+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.55, "await_for": 45.0}) -- 0x55d694e8dd80 con 0x55d694fb66f0 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.496+0000 7ff96cb65700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694fb66f0 0x55d694fb8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.496+0000 7ff947fff700 10 client.18405 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.496+0000 7ff96cb65700 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694fb66f0 0x55d694fb8b10 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148e8ce5634e8194 server_cookie=b3065fc802dfd9e in_seq=0 out_seq=0 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.620+0000 7ff947fff700 1 -- 172.21.15.26:0/2947740812 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d694e8dd80 con 0x55d694fb66f0 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.620+0000 7ff947fff700 10 client.18405 handle_command_reply: tid=0 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.620+0000 7ff97965fb80 1 client.18405 shutdown 2024-05-20T21:06:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 msgr2=0x55d694f3bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f3bec0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff960005900 tx=0x7ff960009570 comp rx=0 tx=0).stop 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 shutdown_connections 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d694fb66f0 0x55d694fb8b10 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d694f32350 0x55d694f3bec0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d694f2c980 0x55d694f3ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 --2- 172.21.15.26:0/2947740812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d694f2d340 0x55d694f3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 >> 172.21.15.26:0/2947740812 conn(0x55d694e5a220 msgr2=0x55d694f306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff947fff700 0 client.18405 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:06.697 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 42, 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "a7dfde6b": { 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 42, 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 146.6, 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:06.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 shutdown_connections 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 1 -- 172.21.15.26:0/2947740812 wait complete. 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.621+0000 7ff97965fb80 20 client.18405 trim_cache size 0 max 16384 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9580b2930 msgr2=0x7ff9580b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9580b2930 0x7ff9580b4df0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff968079a60 tx=0x7ff95400a380 comp rx=0 tx=0).stop 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 msgr2=0x7ff96819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff96819f680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e9a0 tx=0x7ff95c00ecb0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 shutdown_connections 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff9580b2930 0x7ff9580b4df0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff968107a30 0x7ff968199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968102ae0 0x7ff96819f680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 --2- 172.21.15.26:0/3580774748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff968107070 0x7ff96819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 >> 172.21.15.26:0/3580774748 conn(0x7ff9680793f0 msgr2=0x7ff96806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 shutdown_connections 2024-05-20T21:06:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:06.622+0000 7ff96edc9700 1 -- 172.21.15.26:0/3580774748 wait complete. 2024-05-20T21:06:07.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:06 smithi144 ceph-mon[35660]: pgmap v689: 97 pgs: 97 active+clean; 26 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 974 KiB/s wr, 66 op/s 2024-05-20T21:06:07.416 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:06 smithi026 ceph-mon[31990]: pgmap v689: 97 pgs: 97 active+clean; 26 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 974 KiB/s wr, 66 op/s 2024-05-20T21:06:07.447 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a7dfde6b', quiesce took 10.3 seconds. Will release after: 16.25 2024-05-20T21:06:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:06 smithi055 ceph-mon[35213]: pgmap v689: 97 pgs: 97 active+clean; 26 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 974 KiB/s wr, 66 op/s 2024-05-20T21:06:07.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.23 MB/sec execute 373 sec latency 941.380 ms 2024-05-20T21:06:08.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:07 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:06:08.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:07 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:06:08.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:07 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:06:08.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.23 MB/sec execute 374 sec latency 1941.477 ms 2024-05-20T21:06:08.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 -- 172.21.15.26:0/3315613517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97841035e0 msgr2=0x7f9784103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:08.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 --2- 172.21.15.26:0/3315613517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97841035e0 0x7f9784103a60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f97780099c0 tx=0x7f9778009cd0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 -- 172.21.15.26:0/3315613517 shutdown_connections 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 --2- 172.21.15.26:0/3315613517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9784103fa0 0x7f9784110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 --2- 172.21.15.26:0/3315613517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784109680 0x7f9784109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 --2- 172.21.15.26:0/3315613517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97841035e0 0x7f9784103a60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 -- 172.21.15.26:0/3315613517 >> 172.21.15.26:0/3315613517 conn(0x7f978407a6f0 msgr2=0x7f978407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 -- 172.21.15.26:0/3315613517 shutdown_connections 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 -- 172.21.15.26:0/3315613517 wait complete. 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 Processor -- start 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.984+0000 7f978a647700 1 -- start start 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f978a647700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97841035e0 0x7f9784075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f978a647700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784103fa0 0x7f9784075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f978a647700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 0x7f97840762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f978a647700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9784113690 con 0x7f9784109680 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f978a647700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9784113510 con 0x7f97841035e0 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f978a647700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9784113810 con 0x7f9784103fa0 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 0x7f97840762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 0x7f97840762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35680/0 (socket says 172.21.15.26:35680) 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 -- 172.21.15.26:0/3789387351 learned_addr learned my addr 172.21.15.26:0/3789387351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f97837fe700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784103fa0 0x7f9784075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9783fff700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97841035e0 0x7f9784075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 -- 172.21.15.26:0/3789387351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784103fa0 msgr2=0x7f9784075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784103fa0 0x7f9784075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 -- 172.21.15.26:0/3789387351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97841035e0 msgr2=0x7f9784075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97841035e0 0x7f9784075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.985+0000 7f9788be4700 1 -- 172.21.15.26:0/3789387351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97841a8050 con 0x7f9784109680 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.986+0000 7f9788be4700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 0x7f97840762c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f977c00ada0 tx=0x7f977c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.986+0000 7f97817fa700 1 -- 172.21.15.26:0/3789387351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c017020 con 0x7f9784109680 2024-05-20T21:06:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.986+0000 7f97817fa700 1 -- 172.21.15.26:0/3789387351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f977c00cd60 con 0x7f9784109680 2024-05-20T21:06:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.986+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97841a8340 con 0x7f9784109680 2024-05-20T21:06:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.986+0000 7f97817fa700 1 -- 172.21.15.26:0/3789387351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c005280 con 0x7f9784109680 2024-05-20T21:06:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.986+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97841a8800 con 0x7f9784109680 2024-05-20T21:06:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.989+0000 7f97817fa700 1 -- 172.21.15.26:0/3789387351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f977c00c8d0 con 0x7f9784109680 2024-05-20T21:06:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.990+0000 7f97817fa700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f976c0b2880 0x7f976c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.990+0000 7f9783fff700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f976c0b2880 0x7f976c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.990+0000 7f97817fa700 1 -- 172.21.15.26:0/3789387351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f977c0e1a90 con 0x7f9784109680 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.990+0000 7f9794eddb80 1 Processor -- start 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.990+0000 7f9783fff700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f976c0b2880 0x7f976c0b4d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97740045b0 tx=0x7f977400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 -- start start 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7bfed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bff260 0x5581a7c0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5581a7b5cd70 con 0x5581a7c04320 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5581a7b4c910 con 0x5581a7bff260 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9794eddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581a7b50950 con 0x5581a7bfe8a0 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c04700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35684/0 (socket says 172.21.15.26:35684) 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 -- 172.21.15.26:0/4148733271 learned_addr learned my addr 172.21.15.26:0/4148733271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f97837fe700 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7bfed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 -- 172.21.15.26:0/4148733271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 msgr2=0x5581a7bfed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7bfed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 -- 172.21.15.26:0/4148733271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bff260 msgr2=0x5581a7c0b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bff260 0x5581a7c0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.991+0000 7f9783fff700 1 -- 172.21.15.26:0/4148733271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a7af5fb0 con 0x5581a7c04320 2024-05-20T21:06:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.992+0000 7f9783fff700 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c04700 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9774003b20 tx=0x7f9774013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99217d0f66af7d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.992+0000 7f976a7fc700 1 -- 172.21.15.26:0/4148733271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97740118d0 con 0x5581a7c04320 2024-05-20T21:06:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.992+0000 7f976a7fc700 1 -- 172.21.15.26:0/4148733271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9774004830 con 0x5581a7c04320 2024-05-20T21:06:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.992+0000 7f976a7fc700 1 -- 172.21.15.26:0/4148733271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774005480 con 0x5581a7c04320 2024-05-20T21:06:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.992+0000 7f9794eddb80 1 -- 172.21.15.26:0/4148733271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 msgr2=0x5581a7c04700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.992+0000 7f9794eddb80 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c04700 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9774003b20 tx=0x7f9774013000 comp rx=0 tx=0).stop 2024-05-20T21:06:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 -- 172.21.15.26:0/4148733271 shutdown_connections 2024-05-20T21:06:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c04700 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bff260 0x5581a7c0b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 --2- 172.21.15.26:0/4148733271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7bfed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 -- 172.21.15.26:0/4148733271 >> 172.21.15.26:0/4148733271 conn(0x5581a7b2a240 msgr2=0x5581a7bf93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 -- 172.21.15.26:0/4148733271 shutdown_connections 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 -- 172.21.15.26:0/4148733271 wait complete. 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 -- start start 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.993+0000 7f9794eddb80 1 Processor -- start 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c0ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bff260 0x5581a7c08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5581a7a52fa0 con 0x5581a7c04320 2024-05-20T21:06:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5581a7ab4da0 con 0x5581a7bfe8a0 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35692/0 (socket says 172.21.15.26:35692) 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 -- 172.21.15.26:0/1290915281 learned_addr learned my addr 172.21.15.26:0/1290915281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581a7c0b6e0 con 0x5581a7bff260 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f97837fe700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bff260 0x5581a7c08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9794eddb80 10 client.0 _ll_register_callbacks cb 0x5581a7ae7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9783fff700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c0ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 -- 172.21.15.26:0/1290915281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bff260 msgr2=0x5581a7c08090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bff260 0x5581a7c08090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 -- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 msgr2=0x5581a7c0ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c0ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9788be4700 1 -- 172.21.15.26:0/1290915281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a7a9c8d0 con 0x5581a7c04320 2024-05-20T21:06:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.994+0000 7f9783fff700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f9788be4700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c085d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f977c00ad70 tx=0x7f977c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ea4bb1c476b315 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f97637fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f97637fe700 1 -- 172.21.15.26:0/1290915281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c0c3060 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f97637fe700 1 -- 172.21.15.26:0/1290915281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f977c0ad0d0 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9774004830 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f97637fe700 1 -- 172.21.15.26:0/1290915281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f977c0a9780 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f9794eddb80 10 client.18417 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5581a7b06760 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.995+0000 7f97637fe700 1 -- 172.21.15.26:0/1290915281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f977c0a9080 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f97637fe700 1 -- 172.21.15.26:0/1290915281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f977c012070 con 0x5581a7c04320 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9794eddb80 10 client.18417 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9794eddb80 10 client.18417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:06:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9794eddb80 20 client.18417 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c18250 0x5581a7c1a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9794eddb80 4 client.18417 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9774004830 con 0x5581a7c18250 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9783fff700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c18250 0x5581a7c1a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f97637fe700 10 client.18417 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.996+0000 7f9783fff700 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c18250 0x5581a7c1a640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49b5a3914135cd1e server_cookie=7b152823e7a9cce7 in_seq=0 out_seq=0 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.997+0000 7f97637fe700 1 -- 172.21.15.26:0/1290915281 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f9774004830 con 0x5581a7c18250 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.997+0000 7f97637fe700 10 client.18417 handle_command_reply: tid=0 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.997+0000 7f9794eddb80 1 client.18417 shutdown 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.997+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 msgr2=0x5581a7c085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.997+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c085d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f977c00ad70 tx=0x7f977c004ad0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 shutdown_connections 2024-05-20T21:06:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c18250 0x5581a7c1a640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f97637fe700 0 client.18417 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7c04320 0x5581a7c085d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1290915281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bff260 0x5581a7c08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 >> 172.21.15.26:0/1290915281 conn(0x5581a7b2a240 msgr2=0x5581a7bf8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 shutdown_connections 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 1 -- 172.21.15.26:0/1290915281 wait complete. 2024-05-20T21:06:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:08.998+0000 7f9794eddb80 20 client.18417 trim_cache size 0 max 16384 2024-05-20T21:06:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.013+0000 7f9794eddb80 1 Processor -- start 2024-05-20T21:06:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.013+0000 7f9794eddb80 1 -- start start 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.013+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7c1a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.013+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c03830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7c04320 0x5581a7c07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9794eddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97841b2820 con 0x5581a7bff260 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9794eddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f97841b00b0 con 0x5581a7c04320 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9794eddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97841adf00 con 0x5581a7bfe8a0 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c03830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c03830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35702/0 (socket says 172.21.15.26:35702) 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 -- 172.21.15.26:0/422236105 learned_addr learned my addr 172.21.15.26:0/422236105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9783fff700 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7c1a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9788be4700 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7c04320 0x5581a7c07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 -- 172.21.15.26:0/422236105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 msgr2=0x5581a7c1a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7c1a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 -- 172.21.15.26:0/422236105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7c04320 msgr2=0x5581a7c07860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7c04320 0x5581a7c07860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 -- 172.21.15.26:0/422236105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a7c03f40 con 0x5581a7bff260 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9788be4700 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7c04320 0x5581a7c07860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f97837fe700 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c03830 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f97780099c0 tx=0x7f9778005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6340ba5650118ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.014+0000 7f9769ffb700 1 -- 172.21.15.26:0/422236105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97780055f0 con 0x5581a7bff260 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9769ffb700 1 -- 172.21.15.26:0/422236105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f977800bc20 con 0x5581a7bff260 2024-05-20T21:06:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9769ffb700 1 -- 172.21.15.26:0/422236105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778003d40 con 0x5581a7bff260 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 -- 172.21.15.26:0/422236105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 msgr2=0x5581a7c03830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c03830 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f97780099c0 tx=0x7f9778005fb0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 -- 172.21.15.26:0/422236105 shutdown_connections 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7c04320 0x5581a7c07860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7bfe8a0 0x5581a7c1a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 --2- 172.21.15.26:0/422236105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c03830 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 -- 172.21.15.26:0/422236105 >> 172.21.15.26:0/422236105 conn(0x5581a7b2a240 msgr2=0x5581a7c025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.015+0000 7f9794eddb80 1 -- 172.21.15.26:0/422236105 shutdown_connections 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 -- 172.21.15.26:0/422236105 wait complete. 2024-05-20T21:06:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 -- start start 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 Processor -- start 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c10b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c11080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7c04320 0x5581a7c1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97841ae570 con 0x5581a7bff260 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f97841af9e0 con 0x5581a7bfe8a0 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97841aec40 con 0x5581a7c04320 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f9794eddb80 10 client.0 _ll_register_callbacks cb 0x5581a7ae7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f97837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f97837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c11080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35716/0 (socket says 172.21.15.26:35716) 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.016+0000 7f97837fe700 1 -- 172.21.15.26:0/1347049809 learned_addr learned my addr 172.21.15.26:0/1347049809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f9783fff700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c10b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f9788be4700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7c04320 0x5581a7c1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97837fe700 1 -- 172.21.15.26:0/1347049809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7c04320 msgr2=0x5581a7c1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97837fe700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7c04320 0x5581a7c1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97837fe700 1 -- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 msgr2=0x5581a7c10b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97837fe700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c10b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97837fe700 1 -- 172.21.15.26:0/1347049809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a7c1f780 con 0x5581a7bff260 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f9788be4700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7c04320 0x5581a7c1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97837fe700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c11080 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f977800b580 tx=0x7f97780059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b051970ab7e2ca95 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97637fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97637fe700 1 -- 172.21.15.26:0/1347049809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778003940 con 0x5581a7bff260 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977800bc20 con 0x5581a7bff260 2024-05-20T21:06:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97637fe700 1 -- 172.21.15.26:0/1347049809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9778003ae0 con 0x5581a7bff260 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f97637fe700 1 -- 172.21.15.26:0/1347049809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9778003ce0 con 0x5581a7bff260 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f9794eddb80 10 client.18423 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.017+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5581a7c826b0 con 0x5581a7bff260 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f97637fe700 1 -- 172.21.15.26:0/1347049809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f977800bc20 con 0x5581a7bff260 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f97637fe700 1 -- 172.21.15.26:0/1347049809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9778024450 con 0x5581a7bff260 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9794eddb80 10 client.18423 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9794eddb80 10 client.18423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9794eddb80 20 client.18423 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c82d00 0x5581a7c850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9794eddb80 4 client.18423 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581a7b5ecc0 con 0x5581a7c82d00 2024-05-20T21:06:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.018+0000 7f9783fff700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c82d00 0x5581a7c850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9783fff700 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c82d00 0x5581a7c850f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc2ce0645bd01f5b server_cookie=cc6ee32dd651977c in_seq=0 out_seq=0 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f97637fe700 10 client.18423 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f97637fe700 1 -- 172.21.15.26:0/1347049809 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581a7b5ecc0 con 0x5581a7c82d00 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f97637fe700 10 client.18423 handle_command_reply: tid=0 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 client.18423 shutdown 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 msgr2=0x5581a7c11080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c11080 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f977800b580 tx=0x7f97780059e0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 shutdown_connections 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5581a7c82d00 0x5581a7c850f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581a7c04320 0x5581a7c1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5581a7bfe8a0 0x5581a7c10b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f97637fe700 0 client.18423 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 --2- 172.21.15.26:0/1347049809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581a7bff260 0x5581a7c11080 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 >> 172.21.15.26:0/1347049809 conn(0x5581a7b2a240 msgr2=0x5581a7bfd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 shutdown_connections 2024-05-20T21:06:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 1 -- 172.21.15.26:0/1347049809 wait complete. 2024-05-20T21:06:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.019+0000 7f9794eddb80 20 client.18423 trim_cache size 0 max 16384 2024-05-20T21:06:09.024 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f976c0b2880 msgr2=0x7f976c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f976c0b2880 0x7f976c0b4d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97740045b0 tx=0x7f977400b3c0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 msgr2=0x7f97840762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 0x7f97840762c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f977c00ada0 tx=0x7f977c00c330 comp rx=0 tx=0).stop 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 shutdown_connections 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f976c0b2880 0x7f976c0b4d40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9784109680 0x7f97840762c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97841035e0 0x7f9784075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 --2- 172.21.15.26:0/3789387351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784103fa0 0x7f9784075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 >> 172.21.15.26:0/3789387351 conn(0x7f978407a6f0 msgr2=0x7f9784102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 shutdown_connections 2024-05-20T21:06:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:09.020+0000 7f978a647700 1 -- 172.21.15.26:0/3789387351 wait complete. 2024-05-20T21:06:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:09 smithi144 ceph-mon[35660]: pgmap v690: 97 pgs: 97 active+clean; 26 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-05-20T21:06:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:09.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:09 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:09.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:09 smithi026 ceph-mon[31990]: pgmap v690: 97 pgs: 97 active+clean; 26 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-05-20T21:06:09.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:09.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:09 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:09 smithi055 ceph-mon[35213]: pgmap v690: 97 pgs: 97 active+clean; 26 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-05-20T21:06:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:09.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:09 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:09.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.22 MB/sec execute 375 sec latency 2941.569 ms 2024-05-20T21:06:09.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:06:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:10.151 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:06:10.151 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:06:10.151 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:06:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:06:10.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:10.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:06:10.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:06:10.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:06:10.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.21 MB/sec execute 376 sec latency 3941.679 ms 2024-05-20T21:06:11.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:11 smithi026 ceph-mon[31990]: pgmap v691: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-20T21:06:11.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:11 smithi144 ceph-mon[35660]: pgmap v691: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-20T21:06:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:11 smithi055 ceph-mon[35213]: pgmap v691: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-20T21:06:11.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.21 MB/sec execute 377 sec latency 4956.402 ms 2024-05-20T21:06:11.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 -- 172.21.15.26:0/3708278881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 msgr2=0x7f4fd406c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/3708278881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd406c250 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000b0d0 tx=0x7f4fd000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 -- 172.21.15.26:0/3708278881 shutdown_connections 2024-05-20T21:06:11.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/3708278881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd406c250 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/3708278881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd4108910 0x7f4fd4110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/3708278881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd406c820 0x7f4fd41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.774+0000 7f4fdbb51700 1 -- 172.21.15.26:0/3708278881 >> 172.21.15.26:0/3708278881 conn(0x7f4fd40793f0 msgr2=0x7f4fd4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 -- 172.21.15.26:0/3708278881 shutdown_connections 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 -- 172.21.15.26:0/3708278881 wait complete. 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 Processor -- start 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 -- start start 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd406c820 0x7f4fd419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd4108910 0x7f4fd4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4113610 con 0x7f4fd406be70 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4113490 con 0x7f4fd4108910 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.775+0000 7f4fdbb51700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4113790 con 0x7f4fd406c820 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42034/0 (socket says 172.21.15.26:42034) 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 -- 172.21.15.26:0/715031415 learned_addr learned my addr 172.21.15.26:0/715031415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd4108910 0x7f4fd4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd406c820 0x7f4fd419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 -- 172.21.15.26:0/715031415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd406c820 msgr2=0x7f4fd419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd406c820 0x7f4fd419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 -- 172.21.15.26:0/715031415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd4108910 msgr2=0x7f4fd4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd4108910 0x7f4fd4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 -- 172.21.15.26:0/715031415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd419a0b0 con 0x7f4fd406be70 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd4108910 0x7f4fd4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd419f790 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0007dc0 tx=0x7f4fd0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fcaffd700 1 -- 172.21.15.26:0/715031415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd001f030 con 0x7f4fd406be70 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fcaffd700 1 -- 172.21.15.26:0/715031415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fd001bb90 con 0x7f4fd406be70 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fcaffd700 1 -- 172.21.15.26:0/715031415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0012bb0 con 0x7f4fd406be70 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd419a340 con 0x7f4fd406be70 2024-05-20T21:06:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.776+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41a6660 con 0x7f4fd406be70 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.779+0000 7f4fcaffd700 1 -- 172.21.15.26:0/715031415 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4fd0022030 con 0x7f4fd406be70 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fcaffd700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4fc00b2880 0x7f4fc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4fc00b2880 0x7f4fc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fcaffd700 1 -- 172.21.15.26:0/715031415 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4fd000f020 con 0x7f4fd406be70 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 Processor -- start 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 -- start start 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f45d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f52080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4fc00b2880 0x7f4fc00b4d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005fd0 tx=0x7f4fc400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557243ea3d70 con 0x557243f4b320 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.780+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557243e93910 con 0x557243f458a0 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557243e97950 con 0x557243f46260 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42036/0 (socket says 172.21.15.26:42036) 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 -- 172.21.15.26:0/916579989 learned_addr learned my addr 172.21.15.26:0/916579989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f45d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f52080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 -- 172.21.15.26:0/916579989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 msgr2=0x557243f52080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f52080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 -- 172.21.15.26:0/916579989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 msgr2=0x557243f45d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f45d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 -- 172.21.15.26:0/916579989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557243e3cfb0 con 0x557243f4b320 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f45d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4b700 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00a6c00 tx=0x7f4fd000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5e909721572853 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fbffff700 1 -- 172.21.15.26:0/916579989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00aef30 con 0x557243f4b320 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fbffff700 1 -- 172.21.15.26:0/916579989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fd00b3950 con 0x557243f4b320 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.781+0000 7f4fbffff700 1 -- 172.21.15.26:0/916579989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00a6e10 con 0x557243f4b320 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/916579989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 msgr2=0x557243f4b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4b700 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00a6c00 tx=0x7f4fd000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/916579989 shutdown_connections 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4b700 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f52080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/916579989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f45d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/916579989 >> 172.21.15.26:0/916579989 conn(0x557243e71240 msgr2=0x557243f403e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/916579989 shutdown_connections 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.782+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/916579989 wait complete. 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 -- start start 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 Processor -- start 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f51ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f4f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557243d99fa0 con 0x557243f4b320 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557243dfbda0 con 0x557243f458a0 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557243ea3d70 con 0x557243f46260 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fe63e7b80 10 client.0 _ll_register_callbacks cb 0x557243e2e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f51ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fda0ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f51ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50146/0 (socket says 172.21.15.26:50146) 2024-05-20T21:06:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fd98ed700 1 -- 172.21.15.26:0/2994997453 learned_addr learned my addr 172.21.15.26:0/2994997453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.783+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f4f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fda0ee700 1 -- 172.21.15.26:0/2994997453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 msgr2=0x557243f4f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f4f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fda0ee700 1 -- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 msgr2=0x557243f51ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f51ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fda0ee700 1 -- 172.21.15.26:0/2994997453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557243de38d0 con 0x557243f4b320 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f51ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4f570 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00de60 tx=0x7f4fcc00f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f77b3bd59944f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/2994997453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc019030 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/2994997453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4fcc00fc30 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd00b3950 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/2994997453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc023050 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fe63e7b80 10 client.18435 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557243e4d760 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.784+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/2994997453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fd00b3950 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/2994997453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4fcc015410 con 0x557243f4b320 2024-05-20T21:06:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fe63e7b80 10 client.18435 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fe63e7b80 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fe63e7b80 20 client.18435 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243f5f060 0x557243f61470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fe63e7b80 4 client.18435 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557243ea5cc0 con 0x557243f5f060 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243f5f060 0x557243f61470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fbd7fa700 10 client.18435 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.785+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243f5f060 0x557243f61470 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02eedc4113f5c47 server_cookie=50dbca3dd8078976 in_seq=0 out_seq=0 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/2994997453 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557243ea5cc0 con 0x557243f5f060 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fbd7fa700 10 client.18435 handle_command_reply: tid=0 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 client.18435 shutdown 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 msgr2=0x557243f4f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4f570 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00de60 tx=0x7f4fcc00f4c0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 shutdown_connections 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243f5f060 0x557243f61470 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f4f570 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f51ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/2994997453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f4f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fbd7fa700 0 client.18435 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 >> 172.21.15.26:0/2994997453 conn(0x557243e71240 msgr2=0x557243f3faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.786+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 shutdown_connections 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.787+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/2994997453 wait complete. 2024-05-20T21:06:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.787+0000 7f4fe63e7b80 20 client.18435 trim_cache size 0 max 16384 2024-05-20T21:06:11.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 Processor -- start 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 -- start start 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f4a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f40270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41b04f0 con 0x557243f4b320 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41add80 con 0x557243f458a0 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41abbd0 con 0x557243f46260 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fda0ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f4a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fda0ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42048/0 (socket says 172.21.15.26:42048) 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fda0ee700 1 -- 172.21.15.26:0/339585102 learned_addr learned my addr 172.21.15.26:0/339585102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fd98ed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f4a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50156/0 (socket says 172.21.15.26:50156) 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.802+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f40270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fda0ee700 1 -- 172.21.15.26:0/339585102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 msgr2=0x557243f40270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f40270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fda0ee700 1 -- 172.21.15.26:0/339585102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 msgr2=0x557243f4a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f4a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fda0ee700 1 -- 172.21.15.26:0/339585102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557243f5c9e0 con 0x557243f4b320 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66d40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc004750 tx=0x7f4fcc006fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e068bfdd78794534 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fbf7fe700 1 -- 172.21.15.26:0/339585102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc02c070 con 0x557243f4b320 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fbf7fe700 1 -- 172.21.15.26:0/339585102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fcc00f810 con 0x557243f4b320 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.803+0000 7f4fbf7fe700 1 -- 172.21.15.26:0/339585102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc019070 con 0x557243f4b320 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/339585102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 msgr2=0x557243f66d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66d40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc004750 tx=0x7f4fcc006fd0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/339585102 shutdown_connections 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66d40 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f4a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/339585102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f40270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/339585102 >> 172.21.15.26:0/339585102 conn(0x557243e71240 msgr2=0x557243f4a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/339585102 shutdown_connections 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/339585102 wait complete. 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 -- start start 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.804+0000 7f4fe63e7b80 1 Processor -- start 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f57d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41ac240 con 0x557243f4b320 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41ad6b0 con 0x557243f458a0 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41ac910 con 0x557243f46260 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 10 client.0 _ll_register_callbacks cb 0x557243e2e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42060/0 (socket says 172.21.15.26:42060) 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 -- 172.21.15.26:0/3931760634 learned_addr learned my addr 172.21.15.26:0/3931760634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f57d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fd90ec700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 -- 172.21.15.26:0/3931760634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 msgr2=0x557243f665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 -- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 msgr2=0x557243f57d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f57d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 -- 172.21.15.26:0/3931760634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557243f67310 con 0x557243f4b320 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f57d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fda0ee700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66af0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02a040 tx=0x7f4fcc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaeb86028de81b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/3931760634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc02c030 con 0x557243f4b320 2024-05-20T21:06:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/3931760634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4fcc015040 con 0x557243f4b320 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.805+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fcc00f810 con 0x557243f4b320 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/3931760634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc019040 con 0x557243f4b320 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 10 client.18441 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557243fc9c50 con 0x557243f4b320 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/3931760634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fcc00f810 con 0x557243f4b320 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/3931760634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4fcc014c60 con 0x557243f4b320 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 10 client.18441 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 20 client.18441 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243fca290 0x557243fcc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 4 client.18441 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557243e9f4a0 con 0x557243fca290 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.806+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243fca290 0x557243fcc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fd98ed700 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243fca290 0x557243fcc6b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13a8d16d122a2b2 server_cookie=70a4f5bcf970bd2 in_seq=0 out_seq=0 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fbd7fa700 10 client.18441 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fbd7fa700 1 -- 172.21.15.26:0/3931760634 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557243e9f4a0 con 0x557243fca290 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fbd7fa700 10 client.18441 handle_command_reply: tid=0 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 client.18441 shutdown 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 msgr2=0x557243f66af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66af0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02a040 tx=0x7f4fcc00fc30 comp rx=0 tx=0).stop 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 shutdown_connections 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557243fca290 0x557243fcc6b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557243f4b320 0x557243f66af0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fbd7fa700 0 client.18441 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557243f458a0 0x557243f57d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 --2- 172.21.15.26:0/3931760634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557243f46260 0x557243f665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 >> 172.21.15.26:0/3931760634 conn(0x557243e71240 msgr2=0x557243f48d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 shutdown_connections 2024-05-20T21:06:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 1 -- 172.21.15.26:0/3931760634 wait complete. 2024-05-20T21:06:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.807+0000 7f4fe63e7b80 20 client.18441 trim_cache size 0 max 16384 2024-05-20T21:06:11.823 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4fc00b2880 msgr2=0x7f4fc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4fc00b2880 0x7f4fc00b4d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005fd0 tx=0x7f4fc400a560 comp rx=0 tx=0).stop 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 msgr2=0x7f4fd419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd419f790 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0007dc0 tx=0x7f4fd0007ea0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 shutdown_connections 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4fc00b2880 0x7f4fc00b4d40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd406be70 0x7f4fd419f790 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fd4108910 0x7f4fd4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 --2- 172.21.15.26:0/715031415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd406c820 0x7f4fd419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 >> 172.21.15.26:0/715031415 conn(0x7f4fd40793f0 msgr2=0x7f4fd4100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 shutdown_connections 2024-05-20T21:06:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:11.808+0000 7f4fdbb51700 1 -- 172.21.15.26:0/715031415 wait complete. 2024-05-20T21:06:12.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.20 MB/sec execute 378 sec latency 5956.450 ms 2024-05-20T21:06:13.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:13 smithi144 ceph-mon[35660]: pgmap v692: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-05-20T21:06:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:13 smithi055 ceph-mon[35213]: pgmap v692: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-05-20T21:06:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:13 smithi026 ceph-mon[31990]: pgmap v692: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-05-20T21:06:13.490 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:06:13.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:06:13.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.20 MB/sec execute 379 sec latency 6956.494 ms 2024-05-20T21:06:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:14.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.19 MB/sec execute 380 sec latency 7956.575 ms 2024-05-20T21:06:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:15 smithi026 ceph-mon[31990]: pgmap v693: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 989 KiB/s wr, 95 op/s 2024-05-20T21:06:15.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:15 smithi144 ceph-mon[35660]: pgmap v693: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 989 KiB/s wr, 95 op/s 2024-05-20T21:06:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:15 smithi055 ceph-mon[35213]: pgmap v693: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 989 KiB/s wr, 95 op/s 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 -- 172.21.15.26:0/873312745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 msgr2=0x7f20c01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 --2- 172.21.15.26:0/873312745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c01028c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009990 tx=0x7f20b0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 -- 172.21.15.26:0/873312745 shutdown_connections 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 --2- 172.21.15.26:0/873312745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20c0102e00 0x7f20c010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 --2- 172.21.15.26:0/873312745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20c0108520 0x7f20c0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 --2- 172.21.15.26:0/873312745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c01028c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 -- 172.21.15.26:0/873312745 >> 172.21.15.26:0/873312745 conn(0x7f20c00793f0 msgr2=0x7f20c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 -- 172.21.15.26:0/873312745 shutdown_connections 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.488+0000 7f20c81bb700 1 -- 172.21.15.26:0/873312745 wait complete. 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 Processor -- start 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 -- start start 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20c0102e00 0x7f20c019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20c0108520 0x7f20c01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0112500 con 0x7f20c0102440 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0112380 con 0x7f20c0108520 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.489+0000 7f20c81bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0112680 con 0x7f20c0102e00 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42088/0 (socket says 172.21.15.26:42088) 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 -- 172.21.15.26:0/3856747339 learned_addr learned my addr 172.21.15.26:0/3856747339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5756700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20c0102e00 0x7f20c019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c6758700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20c0108520 0x7f20c01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 -- 172.21.15.26:0/3856747339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20c0102e00 msgr2=0x7f20c019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20c0102e00 0x7f20c019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 -- 172.21.15.26:0/3856747339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20c0108520 msgr2=0x7f20c01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20c0108520 0x7f20c01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 -- 172.21.15.26:0/3856747339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c019a130 con 0x7f20c0102440 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c5f57700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c019f810 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007e20 tx=0x7f20bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20b6ffd700 1 -- 172.21.15.26:0/3856747339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc016020 con 0x7f20c0102440 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20b6ffd700 1 -- 172.21.15.26:0/3856747339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc00ad40 con 0x7f20c0102440 2024-05-20T21:06:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c019a420 con 0x7f20c0102440 2024-05-20T21:06:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20b6ffd700 1 -- 172.21.15.26:0/3856747339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0115e0 con 0x7f20c0102440 2024-05-20T21:06:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.490+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c01a66e0 con 0x7f20c0102440 2024-05-20T21:06:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.493+0000 7f20b6ffd700 1 -- 172.21.15.26:0/3856747339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f20bc019020 con 0x7f20c0102440 2024-05-20T21:06:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.494+0000 7f20b6ffd700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20ac0b2880 0x7f20ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.494+0000 7f20c5756700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20ac0b2880 0x7f20ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.494+0000 7f20b6ffd700 1 -- 172.21.15.26:0/3856747339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20bc0e1b10 con 0x7f20c0102440 2024-05-20T21:06:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.494+0000 7f20d2a51b80 1 Processor -- start 2024-05-20T21:06:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.494+0000 7f20c5756700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20ac0b2880 0x7f20ac0b4d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f20b00098e0 tx=0x7f20b0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 -- start start 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ef14dd3df0 con 0x55ef14e75920 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ef14dc3990 con 0x55ef14e7b3a0 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef14dc79d0 con 0x55ef14e762e0 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e75da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42092/0 (socket says 172.21.15.26:42092) 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 -- 172.21.15.26:0/1839471959 learned_addr learned my addr 172.21.15.26:0/1839471959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5f57700 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c6758700 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 -- 172.21.15.26:0/1839471959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 msgr2=0x55ef14e82100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e82100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 -- 172.21.15.26:0/1839471959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 msgr2=0x55ef14e7b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 -- 172.21.15.26:0/1839471959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef14d6d030 con 0x55ef14e75920 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.495+0000 7f20c5756700 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e75da0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f20b0015000 tx=0x7f20b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c98ba864bd656f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.496+0000 7f20abfff700 1 -- 172.21.15.26:0/1839471959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0034070 con 0x55ef14e75920 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.496+0000 7f20abfff700 1 -- 172.21.15.26:0/1839471959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20b000f9b0 con 0x55ef14e75920 2024-05-20T21:06:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.496+0000 7f20abfff700 1 -- 172.21.15.26:0/1839471959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b002fa50 con 0x55ef14e75920 2024-05-20T21:06:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.496+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1839471959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 msgr2=0x55ef14e75da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.496+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e75da0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f20b0015000 tx=0x7f20b001f000 comp rx=0 tx=0).stop 2024-05-20T21:06:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1839471959 shutdown_connections 2024-05-20T21:06:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e82100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1839471959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e75da0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1839471959 >> 172.21.15.26:0/1839471959 conn(0x55ef14da12c0 msgr2=0x55ef14e70460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1839471959 shutdown_connections 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1839471959 wait complete. 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 -- start start 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.497+0000 7f20d2a51b80 1 Processor -- start 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e81db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e7f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ef14cca020 con 0x55ef14e75920 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ef14d2be20 con 0x55ef14e7b3a0 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef14e82700 con 0x55ef14e762e0 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 10 client.0 _ll_register_callbacks cb 0x55ef14d5e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e81db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e81db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42094/0 (socket says 172.21.15.26:42094) 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 -- 172.21.15.26:0/1011244338 learned_addr learned my addr 172.21.15.26:0/1011244338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c6758700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5756700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e7f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 -- 172.21.15.26:0/1011244338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 msgr2=0x55ef14e7f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e7f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 -- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 msgr2=0x55ef14e7f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 -- 172.21.15.26:0/1011244338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef14d13950 con 0x55ef14e75920 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20c5f57700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e81db0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007df0 tx=0x7f20bc0a8c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd673bff538e89f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20a97fa700 1 -- 172.21.15.26:0/1011244338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0c2020 con 0x55ef14e75920 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20a97fa700 1 -- 172.21.15.26:0/1011244338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20bc0ad110 con 0x55ef14e75920 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.498+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b000f9b0 con 0x55ef14e75920 2024-05-20T21:06:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20a97fa700 1 -- 172.21.15.26:0/1011244338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f20bc0a9070 con 0x55ef14e75920 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 10 client.18453 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef14d7d7e0 con 0x55ef14e75920 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20a97fa700 1 -- 172.21.15.26:0/1011244338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20b000f9b0 con 0x55ef14e75920 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20a97fa700 1 -- 172.21.15.26:0/1011244338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f20bc012420 con 0x55ef14e75920 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 10 client.18453 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 20 client.18453 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14e8f280 0x55ef14e91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 4 client.18453 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.499+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef14dd5d40 con 0x55ef14e8f280 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.500+0000 7f20c5756700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14e8f280 0x55ef14e91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.500+0000 7f20c5756700 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14e8f280 0x55ef14e91690 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62147e53d692eb7c server_cookie=6524eabd20cc41a0 in_seq=0 out_seq=0 2024-05-20T21:06:15.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.500+0000 7f20a97fa700 10 client.18453 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20a97fa700 1 -- 172.21.15.26:0/1011244338 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ef14dd5d40 con 0x55ef14e8f280 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20a97fa700 10 client.18453 handle_command_reply: tid=0 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 client.18453 shutdown 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 msgr2=0x55ef14e81db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e81db0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007df0 tx=0x7f20bc0a8c80 comp rx=0 tx=0).stop 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 shutdown_connections 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14e8f280 0x55ef14e91690 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e81db0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e7b3a0 0x55ef14e7f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20a97fa700 0 client.18453 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/1011244338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e7f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 >> 172.21.15.26:0/1011244338 conn(0x55ef14da12c0 msgr2=0x55ef14e6fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 shutdown_connections 2024-05-20T21:06:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 1 -- 172.21.15.26:0/1011244338 wait complete. 2024-05-20T21:06:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.501+0000 7f20d2a51b80 20 client.18453 trim_cache size 0 max 16384 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 Processor -- start 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 -- start start 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e75920 0x55ef14e8b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 0x55ef14e80ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01b0520 con 0x55ef14e7b3a0 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01addb0 con 0x55ef14e75920 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01abc00 con 0x55ef14e762e0 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20c6758700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 0x55ef14e80ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20c6758700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 0x55ef14e80ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42098/0 (socket says 172.21.15.26:42098) 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.522+0000 7f20c6758700 1 -- 172.21.15.26:0/2837217688 learned_addr learned my addr 172.21.15.26:0/2837217688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:15.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c5f57700 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e75920 0x55ef14e8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c5756700 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c6758700 1 -- 172.21.15.26:0/2837217688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 msgr2=0x55ef14e702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c6758700 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c6758700 1 -- 172.21.15.26:0/2837217688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e75920 msgr2=0x55ef14e8b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c6758700 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e75920 0x55ef14e8b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c6758700 1 -- 172.21.15.26:0/2837217688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef14e7afc0 con 0x55ef14e7b3a0 2024-05-20T21:06:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20c6758700 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 0x55ef14e80ff0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b990 tx=0x7f20b800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af38c2bab21c87d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20ab7fe700 1 -- 172.21.15.26:0/2837217688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8019030 con 0x55ef14e7b3a0 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20ab7fe700 1 -- 172.21.15.26:0/2837217688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20b8012a90 con 0x55ef14e7b3a0 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.523+0000 7f20ab7fe700 1 -- 172.21.15.26:0/2837217688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8015920 con 0x55ef14e7b3a0 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 -- 172.21.15.26:0/2837217688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 msgr2=0x55ef14e80ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 0x55ef14e80ff0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b990 tx=0x7f20b800bca0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 -- 172.21.15.26:0/2837217688 shutdown_connections 2024-05-20T21:06:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e7b3a0 0x55ef14e80ff0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e75920 0x55ef14e8b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/2837217688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e762e0 0x55ef14e702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 -- 172.21.15.26:0/2837217688 >> 172.21.15.26:0/2837217688 conn(0x55ef14da12c0 msgr2=0x55ef14e808e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 -- 172.21.15.26:0/2837217688 shutdown_connections 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 -- 172.21.15.26:0/2837217688 wait complete. 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 -- start start 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.524+0000 7f20d2a51b80 1 Processor -- start 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e96530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e762e0 0x55ef14e96a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e7b3a0 0x55ef14efde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01ac270 con 0x55ef14e75920 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01ad6e0 con 0x55ef14e762e0 2024-05-20T21:06:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01ac940 con 0x55ef14e7b3a0 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20d2a51b80 10 client.0 _ll_register_callbacks cb 0x55ef14d5e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e96530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e96530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42108/0 (socket says 172.21.15.26:42108) 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 -- 172.21.15.26:0/4272311350 learned_addr learned my addr 172.21.15.26:0/4272311350 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5756700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e762e0 0x55ef14e96a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c6758700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e7b3a0 0x55ef14efde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 -- 172.21.15.26:0/4272311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e7b3a0 msgr2=0x55ef14efde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e7b3a0 0x55ef14efde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 -- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e762e0 msgr2=0x55ef14e96a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e762e0 0x55ef14e96a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 -- 172.21.15.26:0/4272311350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef14efe4b0 con 0x55ef14e75920 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5756700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e762e0 0x55ef14e96a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20c5f57700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e96530 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0a50f0 tx=0x7f20bc0a8f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b508b9bdcd76cd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20a97fa700 1 -- 172.21.15.26:0/4272311350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0c2070 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.525+0000 7f20a97fa700 1 -- 172.21.15.26:0/4272311350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20bc012420 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b8012a90 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20a97fa700 1 -- 172.21.15.26:0/4272311350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f20bc0ad420 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 10 client.18459 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef14eff7d0 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20a97fa700 1 -- 172.21.15.26:0/4272311350 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20b8012a90 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20a97fa700 1 -- 172.21.15.26:0/4272311350 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f20bc004ad0 con 0x55ef14e75920 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 10 client.18459 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 20 client.18459 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14effee0 0x55ef14f02300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 4 client.18459 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef14dcf520 con 0x55ef14effee0 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.526+0000 7f20c5756700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14effee0 0x55ef14f02300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.527+0000 7f20c5756700 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14effee0 0x55ef14f02300 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c715ab66a2fc5bd4 server_cookie=7554f9bce65da83b in_seq=0 out_seq=0 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.527+0000 7f20a97fa700 10 client.18459 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20a97fa700 1 -- 172.21.15.26:0/4272311350 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef14dcf520 con 0x55ef14effee0 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20a97fa700 10 client.18459 handle_command_reply: tid=0 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 client.18459 shutdown 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 msgr2=0x55ef14e96530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e96530 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0a50f0 tx=0x7f20bc0a8f70 comp rx=0 tx=0).stop 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 shutdown_connections 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ef14effee0 0x55ef14f02300 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef14e75920 0x55ef14e96530 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20a97fa700 0 client.18459 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef14e7b3a0 0x55ef14efde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 --2- 172.21.15.26:0/4272311350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ef14e762e0 0x55ef14e96a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 >> 172.21.15.26:0/4272311350 conn(0x55ef14da12c0 msgr2=0x55ef14e7c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.528+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 shutdown_connections 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20d2a51b80 1 -- 172.21.15.26:0/4272311350 wait complete. 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20d2a51b80 20 client.18459 trim_cache size 0 max 16384 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "92009cf5-b324-4fd5-b235-fbe7449a769f", 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:06:15.533 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:06:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20ac0b2880 msgr2=0x7f20ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20ac0b2880 0x7f20ac0b4d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f20b00098e0 tx=0x7f20b0005960 comp rx=0 tx=0).stop 2024-05-20T21:06:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 msgr2=0x7f20c019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c019f810 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007e20 tx=0x7f20bc00a340 comp rx=0 tx=0).stop 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 shutdown_connections 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20ac0b2880 0x7f20ac0b4d40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c0102440 0x7f20c019f810 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20c0108520 0x7f20c01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 --2- 172.21.15.26:0/3856747339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20c0102e00 0x7f20c019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.529+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 >> 172.21.15.26:0/3856747339 conn(0x7f20c00793f0 msgr2=0x7f20c006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.530+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 shutdown_connections 2024-05-20T21:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:15.530+0000 7f20c81bb700 1 -- 172.21.15.26:0/3856747339 wait complete. 2024-05-20T21:06:15.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.19 MB/sec execute 381 sec latency 8956.669 ms 2024-05-20T21:06:16.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:06:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:16.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.18 MB/sec execute 382 sec latency 9956.731 ms 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 -- 172.21.15.26:0/2666870717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224104660 msgr2=0x7f722410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 --2- 172.21.15.26:0/2666870717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224104660 0x7f722410aef0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f721c009ca0 tx=0x7f721c02a8c0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 -- 172.21.15.26:0/2666870717 shutdown_connections 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 --2- 172.21.15.26:0/2666870717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224104660 0x7f722410aef0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 --2- 172.21.15.26:0/2666870717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7224103ca0 0x7f7224104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 --2- 172.21.15.26:0/2666870717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224102ab0 0x7f7224102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 -- 172.21.15.26:0/2666870717 >> 172.21.15.26:0/2666870717 conn(0x7f72240fe040 msgr2=0x7f7224100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 -- 172.21.15.26:0/2666870717 shutdown_connections 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.860+0000 7f722ba59640 1 -- 172.21.15.26:0/2666870717 wait complete. 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 Processor -- start 2024-05-20T21:06:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 -- start start 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 0x7f7224076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7224103ca0 0x7f7224074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224104660 0x7f7224075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f722410e5d0 con 0x7f7224102ab0 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f722410e450 con 0x7f7224104660 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f722ba59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f722410e750 con 0x7f7224103ca0 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 0x7f7224076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 0x7f7224076760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42114/0 (socket says 172.21.15.26:42114) 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 -- 172.21.15.26:0/2323678294 learned_addr learned my addr 172.21.15.26:0/2323678294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f7229ff6640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224104660 0x7f7224075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f7228ff4640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7224103ca0 0x7f7224074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 -- 172.21.15.26:0/2323678294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7224103ca0 msgr2=0x7f7224074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7224103ca0 0x7f7224074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 -- 172.21.15.26:0/2323678294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224104660 msgr2=0x7f7224075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224104660 0x7f7224075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 -- 172.21.15.26:0/2323678294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7224075b80 con 0x7f7224102ab0 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f7229ff6640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224104660 0x7f7224075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.861+0000 7f72297f5640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 0x7f7224076760 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f722000b450 tx=0x7f722000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.862+0000 7f721a7fc640 1 -- 172.21.15.26:0/2323678294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220015020 con 0x7f7224102ab0 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.862+0000 7f721a7fc640 1 -- 172.21.15.26:0/2323678294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7220011d20 con 0x7f7224102ab0 2024-05-20T21:06:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.862+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72241a2da0 con 0x7f7224102ab0 2024-05-20T21:06:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.862+0000 7f721a7fc640 1 -- 172.21.15.26:0/2323678294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220010e10 con 0x7f7224102ab0 2024-05-20T21:06:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.862+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72241a32e0 con 0x7f7224102ab0 2024-05-20T21:06:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 Processor -- start 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 -- start start 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4dfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e5220 0x56174d4e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f721a7fc640 1 -- 172.21.15.26:0/2323678294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7220018020 con 0x7f7224102ab0 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56174d450c30 con 0x56174d4df6c0 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56174d4407d0 con 0x56174d4e5220 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f722e775740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56174d444810 con 0x56174d4e00b0 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4dfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.865+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4dfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42116/0 (socket says 172.21.15.26:42116) 2024-05-20T21:06:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 -- 172.21.15.26:0/3229289504 learned_addr learned my addr 172.21.15.26:0/3229289504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7228ff4640 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e5220 0x56174d4e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f72297f5640 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 -- 172.21.15.26:0/3229289504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 msgr2=0x56174d4e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 -- 172.21.15.26:0/3229289504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e5220 msgr2=0x56174d4e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e5220 0x56174d4e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 -- 172.21.15.26:0/3229289504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174d155210 con 0x56174d4df6c0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f7229ff6640 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4dfae0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f721c009ca0 tx=0x7f721c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d6aaafcb66c272 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f720f7fe640 1 -- 172.21.15.26:0/3229289504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c031c80 con 0x56174d4df6c0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f720f7fe640 1 -- 172.21.15.26:0/3229289504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f721c036040 con 0x56174d4df6c0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f720f7fe640 1 -- 172.21.15.26:0/3229289504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c004500 con 0x56174d4df6c0 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.866+0000 7f721a7fc640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f72100b2920 0x7f72100b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f72297f5640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f72100b2920 0x7f72100b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f721a7fc640 1 -- 172.21.15.26:0/2323678294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f72200e1d60 con 0x7f7224102ab0 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 -- 172.21.15.26:0/3229289504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 msgr2=0x56174d4dfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f72297f5640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f72100b2920 0x7f72100b4de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72200dfb70 tx=0x7f72200047d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4dfae0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f721c009ca0 tx=0x7f721c009a20 comp rx=0 tx=0).stop 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 -- 172.21.15.26:0/3229289504 shutdown_connections 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e5220 0x56174d4e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 --2- 172.21.15.26:0/3229289504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4dfae0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 -- 172.21.15.26:0/3229289504 >> 172.21.15.26:0/3229289504 conn(0x56174d4dad30 msgr2=0x56174d4db130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 -- 172.21.15.26:0/3229289504 shutdown_connections 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.867+0000 7f722e775740 1 -- 172.21.15.26:0/3229289504 wait complete. 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 -- start start 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 Processor -- start 2024-05-20T21:06:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4e9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e00b0 0x56174d4e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e5220 0x56174d4f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56174d126790 con 0x56174d4df6c0 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56174d3fc890 con 0x56174d4e00b0 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56174d4e8170 con 0x56174d4e5220 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f722e775740 10 client.0 _ll_register_callbacks cb 0x56174d154a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4e9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42126/0 (socket says 172.21.15.26:42126) 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 -- 172.21.15.26:0/4272965421 learned_addr learned my addr 172.21.15.26:0/4272965421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e5220 0x56174d4f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7228ff4640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e00b0 0x56174d4e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 -- 172.21.15.26:0/4272965421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e5220 msgr2=0x56174d4f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e5220 0x56174d4f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 -- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e00b0 msgr2=0x56174d4e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e00b0 0x56174d4e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.868+0000 7f7229ff6640 1 -- 172.21.15.26:0/4272965421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174d221620 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f7229ff6640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4e9780 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f721c02b010 tx=0x7f721c02af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab1d809d8f84d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f720cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f720cff9640 1 -- 172.21.15.26:0/4272965421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c058060 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f720cff9640 1 -- 172.21.15.26:0/4272965421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f721c0047b0 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f721c036040 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f720cff9640 1 -- 172.21.15.26:0/4272965421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f721c003970 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 10 client.18471 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56174d4f3ce0 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f720cff9640 1 -- 172.21.15.26:0/4272965421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f721c036040 con 0x56174d4df6c0 2024-05-20T21:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f720cff9640 1 -- 172.21.15.26:0/4272965421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f721c030bf0 con 0x56174d4df6c0 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 10 client.18471 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 20 client.18471 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.869+0000 7f722e775740 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4f5030 0x56174d4f7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.870+0000 7f722e775740 4 client.18471 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.870+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56174d452b80 con 0x56174d4f5030 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.870+0000 7f7228ff4640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4f5030 0x56174d4f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.870+0000 7f720cff9640 10 client.18471 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.870+0000 7f7228ff4640 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4f5030 0x56174d4f7420 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642a3d0491709fca server_cookie=3d60f6905c33f987 in_seq=0 out_seq=0 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f720cff9640 1 -- 172.21.15.26:0/4272965421 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56174d452b80 con 0x56174d4f5030 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f720cff9640 10 client.18471 handle_command_reply: tid=0 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 client.18471 shutdown 2024-05-20T21:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 msgr2=0x56174d4e9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4e9780 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f721c02b010 tx=0x7f721c02af40 comp rx=0 tx=0).stop 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 shutdown_connections 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4f5030 0x56174d4f7420 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e5220 0x56174d4f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4e00b0 0x56174d4e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 --2- 172.21.15.26:0/4272965421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4df6c0 0x56174d4e9780 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 >> 172.21.15.26:0/4272965421 conn(0x56174d4dad30 msgr2=0x56174d4d9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f720cff9640 0 client.18471 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 shutdown_connections 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 1 -- 172.21.15.26:0/4272965421 wait complete. 2024-05-20T21:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.871+0000 7f722e775740 20 client.18471 trim_cache size 0 max 16384 2024-05-20T21:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.886+0000 7f722e775740 1 Processor -- start 2024-05-20T21:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.886+0000 7f722e775740 1 -- start start 2024-05-20T21:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f722e775740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f722e775740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f722e775740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f722e775740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72241aeae0 con 0x56174d4dad30 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f722e775740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f72241ac3b0 con 0x56174d4df6c0 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f722e775740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72241aa200 con 0x56174d4e00b0 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4e9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42134/0 (socket says 172.21.15.26:42134) 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 -- 172.21.15.26:0/2901856553 learned_addr learned my addr 172.21.15.26:0/2901856553 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 -- 172.21.15.26:0/2901856553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 msgr2=0x56174d4fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7228ff4640 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 -- 172.21.15.26:0/2901856553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 msgr2=0x56174d4f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 -- 172.21.15.26:0/2901856553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174d4dab60 con 0x56174d4dad30 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7228ff4640 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f7229ff6640 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4e9aa0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f721c02ade0 tx=0x7f721c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973c88d7b18acc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f720effd640 1 -- 172.21.15.26:0/2901856553 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c0047b0 con 0x56174d4dad30 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f720effd640 1 -- 172.21.15.26:0/2901856553 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f721c036030 con 0x56174d4dad30 2024-05-20T21:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.887+0000 7f720effd640 1 -- 172.21.15.26:0/2901856553 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c003e00 con 0x56174d4dad30 2024-05-20T21:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 -- 172.21.15.26:0/2901856553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 msgr2=0x56174d4e9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4e9aa0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f721c02ade0 tx=0x7f721c009a20 comp rx=0 tx=0).stop 2024-05-20T21:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 -- 172.21.15.26:0/2901856553 shutdown_connections 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 --2- 172.21.15.26:0/2901856553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4e9aa0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 -- 172.21.15.26:0/2901856553 >> 172.21.15.26:0/2901856553 conn(0x56174d4e4cf0 msgr2=0x56174d4e9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 -- 172.21.15.26:0/2901856553 shutdown_connections 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 -- 172.21.15.26:0/2901856553 wait complete. 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 -- start start 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.888+0000 7f722e775740 1 Processor -- start 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72241aa870 con 0x56174d4dad30 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f72241abce0 con 0x56174d4df6c0 2024-05-20T21:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72241aaf40 con 0x56174d4e00b0 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f722e775740 10 client.0 _ll_register_callbacks cb 0x7f721400a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42146/0 (socket says 172.21.15.26:42146) 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 -- 172.21.15.26:0/819640771 learned_addr learned my addr 172.21.15.26:0/819640771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 -- 172.21.15.26:0/819640771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 msgr2=0x56174d4efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 -- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 msgr2=0x56174d4eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 -- 172.21.15.26:0/819640771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174d4df300 con 0x56174d4dad30 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f7229ff6640 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4ee7b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f721c02ae80 tx=0x7f721c0454d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a3705d01c1e30f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.889+0000 7f720cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f720cff9640 1 -- 172.21.15.26:0/819640771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c06b060 con 0x56174d4dad30 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f720cff9640 1 -- 172.21.15.26:0/819640771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f721c003d80 con 0x56174d4dad30 2024-05-20T21:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f721c036030 con 0x56174d4dad30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f720cff9640 1 -- 172.21.15.26:0/819640771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f721c058050 con 0x56174d4dad30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 10 client.18477 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56174d55fcd0 con 0x56174d4dad30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f720cff9640 1 -- 172.21.15.26:0/819640771 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f721c036030 con 0x56174d4dad30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f720cff9640 1 -- 172.21.15.26:0/819640771 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f721c06f030 con 0x56174d4dad30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 10 client.18477 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 20 client.18477 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4e5220 0x56174d562600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 4 client.18477 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56174d44c360 con 0x56174d4e5220 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.890+0000 7f7228ff4640 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4e5220 0x56174d562600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.891+0000 7f720cff9640 10 client.18477 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.891+0000 7f7228ff4640 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4e5220 0x56174d562600 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947ed855d560eeb9 server_cookie=340c127c38f3c62d in_seq=0 out_seq=0 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.916+0000 7f720cff9640 1 -- 172.21.15.26:0/819640771 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56174d44c360 con 0x56174d4e5220 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.916+0000 7f720cff9640 10 client.18477 handle_command_reply: tid=0 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.916+0000 7f722e775740 1 client.18477 shutdown 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.916+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 msgr2=0x56174d4ee7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4ee7b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f721c02ae80 tx=0x7f721c0454d0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 shutdown_connections 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56174d4e5220 0x56174d562600 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56174d4e00b0 0x56174d4efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f720cff9640 0 client.18477 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56174d4df6c0 0x56174d4eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 --2- 172.21.15.26:0/819640771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174d4dad30 0x56174d4ee7b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 >> 172.21.15.26:0/819640771 conn(0x56174d4e4cf0 msgr2=0x56174d4d9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 shutdown_connections 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 1 -- 172.21.15.26:0/819640771 wait complete. 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722e775740 20 client.18477 trim_cache size 0 max 16384 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:06:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f72100b2920 msgr2=0x7f72100b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722ba59640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f72100b2920 0x7f72100b4de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72200dfb70 tx=0x7f72200047d0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 msgr2=0x7f7224076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.917+0000 7f722ba59640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 0x7f7224076760 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f722000b450 tx=0x7f722000b910 comp rx=0 tx=0).stop 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 shutdown_connections 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f72100b2920 0x7f72100b4de0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7224104660 0x7f7224075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7224103ca0 0x7f7224074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 --2- 172.21.15.26:0/2323678294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7224102ab0 0x7f7224076760 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 >> 172.21.15.26:0/2323678294 conn(0x7f72240fe040 msgr2=0x7f7224106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 shutdown_connections 2024-05-20T21:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:16.918+0000 7f722ba59640 1 -- 172.21.15.26:0/2323678294 wait complete. 2024-05-20T21:06:17.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:17 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:06:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:17 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:17 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:06:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:17 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:17 smithi026 ceph-mon[31990]: pgmap v694: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 653 KiB/s wr, 69 op/s 2024-05-20T21:06:17.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:17 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:06:17.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:17 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:06:17.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:17 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:17.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:17 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:06:17.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:17 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:17.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:17 smithi144 ceph-mon[35660]: pgmap v694: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 653 KiB/s wr, 69 op/s 2024-05-20T21:06:17.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:17 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:06:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:17 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:06:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:17 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:17.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:17 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:06:17.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:17 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:17.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:17 smithi055 ceph-mon[35213]: pgmap v694: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 653 KiB/s wr, 69 op/s 2024-05-20T21:06:17.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:17 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:06:17.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.17 MB/sec execute 383 sec latency 10956.811 ms 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 -- 172.21.15.26:0/454409162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 msgr2=0x7fcbec1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/454409162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec1028c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009990 tx=0x7fcbd4009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 -- 172.21.15.26:0/454409162 shutdown_connections 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/454409162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbec102e00 0x7fcbec10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/454409162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec108520 0x7fcbec108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/454409162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec1028c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 -- 172.21.15.26:0/454409162 >> 172.21.15.26:0/454409162 conn(0x7fcbec0793f0 msgr2=0x7fcbec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 -- 172.21.15.26:0/454409162 shutdown_connections 2024-05-20T21:06:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.327+0000 7fcbf25d6700 1 -- 172.21.15.26:0/454409162 wait complete. 2024-05-20T21:06:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 Processor -- start 2024-05-20T21:06:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 -- start start 2024-05-20T21:06:18.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec102e00 0x7fcbec1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbec108520 0x7fcbec104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1124d0 con 0x7fcbec102440 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec112350 con 0x7fcbec108520 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbf25d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec112650 con 0x7fcbec102e00 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.328+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42170/0 (socket says 172.21.15.26:42170) 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 -- 172.21.15.26:0/933677668 learned_addr learned my addr 172.21.15.26:0/933677668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbf0b73700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbec108520 0x7fcbec104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec102e00 0x7fcbec1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 -- 172.21.15.26:0/933677668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec102e00 msgr2=0x7fcbec1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec102e00 0x7fcbec1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 -- 172.21.15.26:0/933677668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbec108520 msgr2=0x7fcbec104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbec108520 0x7fcbec104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 -- 172.21.15.26:0/933677668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec1a80c0 con 0x7fcbec102440 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbebfff700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec104080 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00d950 tx=0x7fcbdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbe9ffb700 1 -- 172.21.15.26:0/933677668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc016070 con 0x7fcbec102440 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbe9ffb700 1 -- 172.21.15.26:0/933677668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbdc00fd20 con 0x7fcbec102440 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbec1a83b0 con 0x7fcbec102440 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbe9ffb700 1 -- 172.21.15.26:0/933677668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00e3b0 con 0x7fcbec102440 2024-05-20T21:06:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.329+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbec1a8940 con 0x7fcbec102440 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.332+0000 7fcbe9ffb700 1 -- 172.21.15.26:0/933677668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcbdc019070 con 0x7fcbec102440 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.332+0000 7fcbe9ffb700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbd80b2900 0x7fcbd80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbe9ffb700 1 -- 172.21.15.26:0/933677668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcbdc0e1c80 con 0x7fcbec102440 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbd80b2900 0x7fcbd80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 Processor -- start 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 -- start start 2024-05-20T21:06:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbd80b2900 0x7fcbd80b4dc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcbec079c60 tx=0x7fcbd4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad2860 0x55d96cad2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cad6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad6f60 0x55d96cad73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d96ca31c90 con 0x55d96cad3210 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d96ca21830 con 0x55d96cad2860 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d96ca25870 con 0x55d96cad6f60 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cad6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cad6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42184/0 (socket says 172.21.15.26:42184) 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbe3fff700 1 -- 172.21.15.26:0/652271210 learned_addr learned my addr 172.21.15.26:0/652271210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.333+0000 7fcbebfff700 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad2860 0x55d96cad2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbf0b73700 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad6f60 0x55d96cad73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe3fff700 1 -- 172.21.15.26:0/652271210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad6f60 msgr2=0x55d96cad73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad6f60 0x55d96cad73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe3fff700 1 -- 172.21.15.26:0/652271210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad2860 msgr2=0x55d96cad2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad2860 0x55d96cad2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe3fff700 1 -- 172.21.15.26:0/652271210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96c9718d0 con 0x55d96cad3210 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cad6940 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009990 tx=0x7fcbd401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4577eb0171ed00ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe27fc700 1 -- 172.21.15.26:0/652271210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4034070 con 0x55d96cad3210 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe27fc700 1 -- 172.21.15.26:0/652271210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd400be40 con 0x55d96cad3210 2024-05-20T21:06:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.334+0000 7fcbe27fc700 1 -- 172.21.15.26:0/652271210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd402f790 con 0x55d96cad3210 2024-05-20T21:06:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/652271210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 msgr2=0x55d96cad6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cad6940 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009990 tx=0x7fcbd401f000 comp rx=0 tx=0).stop 2024-05-20T21:06:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/652271210 shutdown_connections 2024-05-20T21:06:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad6f60 0x55d96cad73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad2860 0x55d96cad2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/652271210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cad6940 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/652271210 >> 172.21.15.26:0/652271210 conn(0x55d96ca00080 msgr2=0x55d96cacce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/652271210 shutdown_connections 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/652271210 wait complete. 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.335+0000 7fcbfce6cb80 1 -- start start 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 Processor -- start 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 0x55d96cade4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad3210 0x55d96cadb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cadbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d96c927fa0 con 0x55d96cad2860 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d96c9c37c0 con 0x55d96cad6f60 2024-05-20T21:06:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d96cadf0c0 con 0x55d96cad3210 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbfce6cb80 10 client.0 _ll_register_callbacks cb 0x55d96c9bc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 0x55d96cade4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 0x55d96cade4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42188/0 (socket says 172.21.15.26:42188) 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 -- 172.21.15.26:0/1296402229 learned_addr learned my addr 172.21.15.26:0/1296402229 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad3210 0x55d96cadb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbf0b73700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cadbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 -- 172.21.15.26:0/1296402229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad3210 msgr2=0x55d96cadb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad3210 0x55d96cadb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 -- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 msgr2=0x55d96cadbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cadbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 -- 172.21.15.26:0/1296402229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96c9c0690 con 0x55d96cad2860 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbf0b73700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cadbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.336+0000 7fcbebfff700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 0x55d96cade4f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0a52c0 tx=0x7fcbdc0a9940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84614a228d1168ce server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1296402229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0c3060 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1296402229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcbdc0ad170 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd400be40 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1296402229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbdc0ca050 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 10 client.18489 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d96cae9910 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1296402229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd400be40 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1296402229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcbdc0b1c10 con 0x55d96cad2860 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 10 client.18489 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 20 client.18489 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96caebea0 0x55d96caee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 4 client.18489 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.337+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96ca9e9e0 con 0x55d96caebea0 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.338+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96caebea0 0x55d96caee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.338+0000 7fcbcb7fe700 10 client.18489 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.338+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96caebea0 0x55d96caee290 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571b897dc552985f server_cookie=69b23bc7c21e12d4 in_seq=0 out_seq=0 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1296402229 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d96ca9e9e0 con 0x55d96caebea0 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbcb7fe700 10 client.18489 handle_command_reply: tid=0 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 client.18489 shutdown 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 msgr2=0x55d96cade4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 0x55d96cade4f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0a52c0 tx=0x7fcbdc0a9940 comp rx=0 tx=0).stop 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 shutdown_connections 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96caebea0 0x55d96caee290 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad2860 0x55d96cade4f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cadbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1296402229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad3210 0x55d96cadb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 >> 172.21.15.26:0/1296402229 conn(0x55d96ca00080 msgr2=0x55d96cad8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbcb7fe700 0 client.18489 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 shutdown_connections 2024-05-20T21:06:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1296402229 wait complete. 2024-05-20T21:06:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.339+0000 7fcbfce6cb80 20 client.18489 trim_cache size 0 max 16384 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 Processor -- start 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 -- start start 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cad20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96caf3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1b27c0 con 0x55d96cad3210 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1b0050 con 0x55d96cad6f60 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1adea0 con 0x55d96cad2860 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42190/0 (socket says 172.21.15.26:42190) 2024-05-20T21:06:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.360+0000 7fcbe3fff700 1 -- 172.21.15.26:0/3967302876 learned_addr learned my addr 172.21.15.26:0/3967302876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe3fff700 1 -- 172.21.15.26:0/3967302876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 msgr2=0x55d96cad20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cad20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe3fff700 1 -- 172.21.15.26:0/3967302876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 msgr2=0x55d96caf3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96caf3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe3fff700 1 -- 172.21.15.26:0/3967302876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96cace690 con 0x55d96cad3210 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae8160 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005950 tx=0x7fcbd40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d993866564d04c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe1ffb700 1 -- 172.21.15.26:0/3967302876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4034040 con 0x55d96cad3210 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe1ffb700 1 -- 172.21.15.26:0/3967302876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd4003c50 con 0x55d96cad3210 2024-05-20T21:06:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.361+0000 7fcbe1ffb700 1 -- 172.21.15.26:0/3967302876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400f460 con 0x55d96cad3210 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/3967302876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 msgr2=0x55d96cae8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae8160 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005950 tx=0x7fcbd40058e0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/3967302876 shutdown_connections 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96caf3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cad20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/3967302876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae8160 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/3967302876 >> 172.21.15.26:0/3967302876 conn(0x55d96ca00080 msgr2=0x55d96cad19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/3967302876 shutdown_connections 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/3967302876 wait complete. 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 -- start start 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.362+0000 7fcbfce6cb80 1 Processor -- start 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cae0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cae0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1ae510 con 0x55d96cad3210 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1af980 con 0x55d96cad6f60 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1aebe0 con 0x55d96cad2860 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbfce6cb80 10 client.0 _ll_register_callbacks cb 0x55d96c9bc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42202/0 (socket says 172.21.15.26:42202) 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 -- 172.21.15.26:0/1665538509 learned_addr learned my addr 172.21.15.26:0/1665538509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbf0b73700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cae0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbebfff700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cae0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 -- 172.21.15.26:0/1665538509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 msgr2=0x55d96cae0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cae0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 -- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 msgr2=0x55d96cae0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cae0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 -- 172.21.15.26:0/1665538509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96cb55a10 con 0x55d96cad3210 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbf0b73700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cae0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbe3fff700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae05b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400ba30 tx=0x7fcbd4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21694bfd0a5313b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.363+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1665538509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4048070 con 0x55d96cad3210 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd4003c50 con 0x55d96cad3210 2024-05-20T21:06:18.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1665538509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcbd403a620 con 0x55d96cad3210 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1665538509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4034040 con 0x55d96cad3210 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 10 client.18495 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d96cb56ce0 con 0x55d96cad3210 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1665538509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd4003c50 con 0x55d96cad3210 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1665538509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcbd4004760 con 0x55d96cad3210 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 10 client.18495 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 20 client.18495 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96cb56f70 0x55d96cb59360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 4 client.18495 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d96ca33be0 con 0x55d96cb56f70 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.364+0000 7fcbebfff700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96cb56f70 0x55d96cb59360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbebfff700 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96cb56f70 0x55d96cb59360 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797a4faab4214d48 server_cookie=2583be9d42a01a29 in_seq=0 out_seq=0 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbcb7fe700 10 client.18495 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:18.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbcb7fe700 1 -- 172.21.15.26:0/1665538509 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d96ca33be0 con 0x55d96cb56f70 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbcb7fe700 10 client.18495 handle_command_reply: tid=0 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 client.18495 shutdown 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 msgr2=0x55d96cae05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae05b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400ba30 tx=0x7fcbd4003b60 comp rx=0 tx=0).stop 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 shutdown_connections 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d96cb56f70 0x55d96cb59360 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d96cad6f60 0x55d96cae0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d96cad2860 0x55d96cae0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbcb7fe700 0 client.18495 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 --2- 172.21.15.26:0/1665538509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d96cad3210 0x55d96cae05b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 >> 172.21.15.26:0/1665538509 conn(0x55d96ca00080 msgr2=0x55d96cacbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 shutdown_connections 2024-05-20T21:06:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 1 -- 172.21.15.26:0/1665538509 wait complete. 2024-05-20T21:06:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.365+0000 7fcbfce6cb80 20 client.18495 trim_cache size 0 max 16384 2024-05-20T21:06:18.371 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:06:18.371 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:06:18.371 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:06:18.371 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbd80b2900 msgr2=0x7fcbd80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbd80b2900 0x7fcbd80b4dc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcbec079c60 tx=0x7fcbd4005fd0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 msgr2=0x7fcbec104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec104080 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00d950 tx=0x7fcbdc00dc60 comp rx=0 tx=0).stop 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 shutdown_connections 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbd80b2900 0x7fcbd80b4dc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbec102440 0x7fcbec104080 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbec108520 0x7fcbec104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 --2- 172.21.15.26:0/933677668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec102e00 0x7fcbec1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:18.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 >> 172.21.15.26:0/933677668 conn(0x7fcbec0793f0 msgr2=0x7fcbec06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:18.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 shutdown_connections 2024-05-20T21:06:18.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:18.366+0000 7fcbf25d6700 1 -- 172.21.15.26:0/933677668 wait complete. 2024-05-20T21:06:18.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.17 MB/sec execute 384 sec latency 11956.910 ms 2024-05-20T21:06:19.193 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:06:19.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:06:19.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:19 smithi144 ceph-mon[35660]: pgmap v695: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 428 KiB/s wr, 61 op/s 2024-05-20T21:06:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:19.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:19 smithi055 ceph-mon[35213]: pgmap v695: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 428 KiB/s wr, 61 op/s 2024-05-20T21:06:19.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:19 smithi026 ceph-mon[31990]: pgmap v695: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 428 KiB/s wr, 61 op/s 2024-05-20T21:06:19.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.16 MB/sec execute 385 sec latency 12957.000 ms 2024-05-20T21:06:20.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.16 MB/sec execute 386 sec latency 13957.028 ms 2024-05-20T21:06:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 -- 172.21.15.26:0/4150593796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0076090 msgr2=0x7f6ab0114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/4150593796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0076090 0x7f6ab0114200 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009990 tx=0x7f6a98009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 -- 172.21.15.26:0/4150593796 shutdown_connections 2024-05-20T21:06:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/4150593796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab0114740 0x7f6ab0116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/4150593796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab0075770 0x7f6ab0075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/4150593796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0076090 0x7f6ab0114200 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 -- 172.21.15.26:0/4150593796 >> 172.21.15.26:0/4150593796 conn(0x7f6ab007a6b0 msgr2=0x7f6ab007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 -- 172.21.15.26:0/4150593796 shutdown_connections 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.180+0000 7f6ab58cf700 1 -- 172.21.15.26:0/4150593796 wait complete. 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 Processor -- start 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 -- start start 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab0075770 0x7f6ab01a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab0076090 0x7f6ab01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 0x7f6ab01a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0119c00 con 0x7f6ab0114740 2024-05-20T21:06:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0119a80 con 0x7f6ab0075770 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6ab58cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0119d80 con 0x7f6ab0076090 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6aaf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 0x7f6ab01a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6aaf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 0x7f6ab01a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56010/0 (socket says 172.21.15.26:56010) 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6aaf7fe700 1 -- 172.21.15.26:0/2610540121 learned_addr learned my addr 172.21.15.26:0/2610540121 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6aae7fc700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab0076090 0x7f6ab01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.181+0000 7f6aaeffd700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab0075770 0x7f6ab01a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6aaf7fe700 1 -- 172.21.15.26:0/2610540121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab0076090 msgr2=0x7f6ab01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6aaf7fe700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab0076090 0x7f6ab01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6aaf7fe700 1 -- 172.21.15.26:0/2610540121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab0075770 msgr2=0x7f6ab01a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6aaf7fe700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab0075770 0x7f6ab01a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6aaf7fe700 1 -- 172.21.15.26:0/2610540121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab01a5140 con 0x7f6ab0114740 2024-05-20T21:06:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6aaf7fe700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 0x7f6ab01a4a00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ada0 tx=0x7f6aa800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8017020 con 0x7f6ab0114740 2024-05-20T21:06:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab01a5430 con 0x7f6ab0114740 2024-05-20T21:06:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aa800cd60 con 0x7f6ab0114740 2024-05-20T21:06:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8005280 con 0x7f6ab0114740 2024-05-20T21:06:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.182+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01accb0 con 0x7f6ab0114740 2024-05-20T21:06:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.183+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab006a230 con 0x7f6ab0114740 2024-05-20T21:06:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.184+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6aa800c8d0 con 0x7f6ab0114740 2024-05-20T21:06:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.185+0000 7f6ab48cd700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a9c0b2880 0x7f6a9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.185+0000 7f6aaeffd700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a9c0b2880 0x7f6a9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.185+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6aa80e1900 con 0x7f6ab0114740 2024-05-20T21:06:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.186+0000 7f6aaeffd700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a9c0b2880 0x7f6a9c0b4d40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009c60 tx=0x7f6aa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.190+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6aa8021020 con 0x7f6ab0114740 2024-05-20T21:06:21.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:21 smithi144 ceph-mon[35660]: pgmap v696: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 KiB/s wr, 29 op/s 2024-05-20T21:06:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:21 smithi055 ceph-mon[35213]: pgmap v696: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 KiB/s wr, 29 op/s 2024-05-20T21:06:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:21 smithi026 ceph-mon[31990]: pgmap v696: 97 pgs: 97 active+clean; 28 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 KiB/s wr, 29 op/s 2024-05-20T21:06:21.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.454+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ab01acee0 con 0x7f6ab0114740 2024-05-20T21:06:21.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.455+0000 7f6ab48cd700 1 -- 172.21.15.26:0/2610540121 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f6aa80a95f0 con 0x7f6ab0114740 2024-05-20T21:06:21.455 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:06:21.455 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:06:21.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a9c0b2880 msgr2=0x7f6a9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:21.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a9c0b2880 0x7f6a9c0b4d40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009c60 tx=0x7f6aa0009260 comp rx=0 tx=0).stop 2024-05-20T21:06:21.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 msgr2=0x7f6ab01a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 0x7f6ab01a4a00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ada0 tx=0x7f6aa800c330 comp rx=0 tx=0).stop 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 shutdown_connections 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a9c0b2880 0x7f6a9c0b4d40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab0114740 0x7f6ab01a4a00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab0075770 0x7f6ab01a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 --2- 172.21.15.26:0/2610540121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab0076090 0x7f6ab01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 >> 172.21.15.26:0/2610540121 conn(0x7f6ab007a6b0 msgr2=0x7f6ab01005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 shutdown_connections 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:21.458+0000 7f6ab58cf700 1 -- 172.21.15.26:0/2610540121 wait complete. 2024-05-20T21:06:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:06:21.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.15 MB/sec execute 387 sec latency 14957.132 ms 2024-05-20T21:06:22.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:22 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2610540121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:22.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:22 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2610540121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:22.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:06:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:22 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2610540121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:22.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.15 MB/sec execute 388 sec latency 15957.177 ms 2024-05-20T21:06:23.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:23 smithi026 ceph-mon[31990]: pgmap v697: 97 pgs: 97 active+clean; 28 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-20T21:06:23.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:23 smithi144 ceph-mon[35660]: pgmap v697: 97 pgs: 97 active+clean; 28 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-20T21:06:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:23 smithi055 ceph-mon[35213]: pgmap v697: 97 pgs: 97 active+clean; 28 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-20T21:06:23.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.14 MB/sec execute 389 sec latency 16957.229 ms 2024-05-20T21:06:23.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:06:24.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.13 MB/sec execute 390 sec latency 17957.302 ms 2024-05-20T21:06:25.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:25 smithi144 ceph-mon[35660]: pgmap v698: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 24 op/s 2024-05-20T21:06:25.433 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:25 smithi026 ceph-mon[31990]: pgmap v698: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 24 op/s 2024-05-20T21:06:25.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:25 smithi055 ceph-mon[35213]: pgmap v698: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 24 op/s 2024-05-20T21:06:25.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 -- 172.21.15.26:0/3653137260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806bd10 msgr2=0x7f7df806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 --2- 172.21.15.26:0/3653137260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806bd10 0x7f7df806c190 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7de40099c0 tx=0x7f7de4009cd0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 -- 172.21.15.26:0/3653137260 shutdown_connections 2024-05-20T21:06:25.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 --2- 172.21.15.26:0/3653137260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df806c6d0 0x7f7df8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 --2- 172.21.15.26:0/3653137260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df8109630 0x7f7df8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 --2- 172.21.15.26:0/3653137260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806bd10 0x7f7df806c190 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 -- 172.21.15.26:0/3653137260 >> 172.21.15.26:0/3653137260 conn(0x7f7df80794e0 msgr2=0x7f7df80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:25.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 -- 172.21.15.26:0/3653137260 shutdown_connections 2024-05-20T21:06:25.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.555+0000 7f7dfec41700 1 -- 172.21.15.26:0/3653137260 wait complete. 2024-05-20T21:06:25.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 Processor -- start 2024-05-20T21:06:25.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 -- start start 2024-05-20T21:06:25.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df806bd10 0x7f7df81024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 0x7f7df81029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df8109630 0x7f7df8102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8113600 con 0x7f7df806c6d0 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8113480 con 0x7f7df8109630 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfec41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8113780 con 0x7f7df806bd10 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 0x7f7df81029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 0x7f7df81029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56026/0 (socket says 172.21.15.26:56026) 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7df7fff700 1 -- 172.21.15.26:0/639342405 learned_addr learned my addr 172.21.15.26:0/639342405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df806bd10 0x7f7df81024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.556+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df8109630 0x7f7df8102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df7fff700 1 -- 172.21.15.26:0/639342405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df806bd10 msgr2=0x7f7df81024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df7fff700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df806bd10 0x7f7df81024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df7fff700 1 -- 172.21.15.26:0/639342405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df8109630 msgr2=0x7f7df8102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df7fff700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df8109630 0x7f7df8102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df7fff700 1 -- 172.21.15.26:0/639342405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81a8000 con 0x7f7df806c6d0 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df7fff700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 0x7f7df81029f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005f50 tx=0x7f7de40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df5ffb700 1 -- 172.21.15.26:0/639342405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4027070 con 0x7f7df806c6d0 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81a8290 con 0x7f7df806c6d0 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df5ffb700 1 -- 172.21.15.26:0/639342405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de400fc80 con 0x7f7df806c6d0 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7df5ffb700 1 -- 172.21.15.26:0/639342405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4022d70 con 0x7f7df806c6d0 2024-05-20T21:06:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.557+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df81a8740 con 0x7f7df806c6d0 2024-05-20T21:06:25.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.560+0000 7f7df5ffb700 1 -- 172.21.15.26:0/639342405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7de402a070 con 0x7f7df806c6d0 2024-05-20T21:06:25.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.561+0000 7f7df5ffb700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7de80b2880 0x7f7de80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.561+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7de80b2880 0x7f7de80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.561+0000 7f7df5ffb700 1 -- 172.21.15.26:0/639342405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7de40e3090 con 0x7f7df806c6d0 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.561+0000 7f7e094d7b80 1 Processor -- start 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.561+0000 7f7e094d7b80 1 -- start start 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7de80b2880 0x7f7de80b4d40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005950 tx=0x7f7dec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901ddf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901deb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559901d34d70 con 0x559901ddf0b0 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559901d24910 con 0x559901ddfa70 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559901d28950 con 0x559901de4b30 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901ddf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901ddf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56030/0 (socket says 172.21.15.26:56030) 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 -- 172.21.15.26:0/428277149 learned_addr learned my addr 172.21.15.26:0/428277149 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901deb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 -- 172.21.15.26:0/428277149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 msgr2=0x559901de4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 -- 172.21.15.26:0/428277149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 msgr2=0x559901deb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901deb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 -- 172.21.15.26:0/428277149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559901ccdfb0 con 0x559901ddf0b0 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901deb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7df7fff700 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901ddf530 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7de40aa380 tx=0x7f7de4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d659838d4434dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.562+0000 7f7de2ffd700 1 -- 172.21.15.26:0/428277149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de40aad80 con 0x559901ddf0b0 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7de2ffd700 1 -- 172.21.15.26:0/428277149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de40aa680 con 0x559901ddf0b0 2024-05-20T21:06:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7de2ffd700 1 -- 172.21.15.26:0/428277149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de40aab40 con 0x559901ddf0b0 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 -- 172.21.15.26:0/428277149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 msgr2=0x559901ddf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901ddf530 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7de40aa380 tx=0x7f7de4009cd0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 -- 172.21.15.26:0/428277149 shutdown_connections 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901deb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/428277149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901ddf530 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 -- 172.21.15.26:0/428277149 >> 172.21.15.26:0/428277149 conn(0x559901d02240 msgr2=0x559901dd13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 -- 172.21.15.26:0/428277149 shutdown_connections 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.563+0000 7f7e094d7b80 1 -- 172.21.15.26:0/428277149 wait complete. 2024-05-20T21:06:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.564+0000 7f7e094d7b80 1 -- start start 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.564+0000 7f7e094d7b80 1 Processor -- start 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.564+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901deb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.564+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901de8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559901c2afa0 con 0x559901ddf0b0 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559901c8cda0 con 0x559901ddfa70 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559901d34d70 con 0x559901de4b30 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7e094d7b80 10 client.0 _ll_register_callbacks cb 0x559901cbf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901deb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901deb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56036/0 (socket says 172.21.15.26:56036) 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 -- 172.21.15.26:0/1294636357 learned_addr learned my addr 172.21.15.26:0/1294636357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7df7fff700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901de8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 -- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 msgr2=0x559901de8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 -- 172.21.15.26:0/1294636357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 msgr2=0x559901de8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901de8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 -- 172.21.15.26:0/1294636357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559901c748d0 con 0x559901ddf0b0 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7df7fff700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901de8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901deb540 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0116f0 tx=0x7f7dec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfd54bfc39a87f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.565+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1294636357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec010ea0 con 0x559901ddf0b0 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1294636357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7dec01e830 con 0x559901ddf0b0 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de40aa680 con 0x559901ddf0b0 2024-05-20T21:06:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1294636357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7dec01ea30 con 0x559901ddf0b0 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 10 client.18513 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559901cde760 con 0x559901ddf0b0 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1294636357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de40aa680 con 0x559901ddf0b0 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1294636357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7dec030050 con 0x559901ddf0b0 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 10 client.18513 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 10 client.18513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 20 client.18513 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901df8740 0x559901dfab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 4 client.18513 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559901d36cc0 con 0x559901df8740 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.566+0000 7f7df7fff700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901df8740 0x559901dfab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.567+0000 7f7df7fff700 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901df8740 0x559901dfab30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba579ac8318a23ca server_cookie=1dd07d49ce6175a0 in_seq=0 out_seq=0 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.567+0000 7f7dd7fff700 10 client.18513 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1294636357 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559901d36cc0 con 0x559901df8740 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7dd7fff700 10 client.18513 handle_command_reply: tid=0 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 client.18513 shutdown 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 msgr2=0x559901deb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901deb540 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0116f0 tx=0x7f7dec0047c0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 shutdown_connections 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901df8740 0x559901dfab30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901ddf0b0 0x559901deb540 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7dd7fff700 0 client.18513 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901de4b30 0x559901de8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1294636357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddfa70 0x559901de8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 >> 172.21.15.26:0/1294636357 conn(0x559901d02240 msgr2=0x559901dd0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 shutdown_connections 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1294636357 wait complete. 2024-05-20T21:06:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.568+0000 7f7e094d7b80 20 client.18513 trim_cache size 0 max 16384 2024-05-20T21:06:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 Processor -- start 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 -- start start 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901dd1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901de5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81ae320 con 0x559901de4b30 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81abbb0 con 0x559901ddf0b0 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7df819d9b0 con 0x559901ddfa70 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901de5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901de5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56046/0 (socket says 172.21.15.26:56046) 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 -- 172.21.15.26:0/4033752655 learned_addr learned my addr 172.21.15.26:0/4033752655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7df7fff700 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901dd1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 -- 172.21.15.26:0/4033752655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 msgr2=0x559901dea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 -- 172.21.15.26:0/4033752655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 msgr2=0x559901dd1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901dd1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7dfd1de700 1 -- 172.21.15.26:0/4033752655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559901dfb160 con 0x559901de4b30 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.584+0000 7f7df7fff700 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901de5820 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7df00098e0 tx=0x7f7df0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae43f7dba77db7e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7de27fc700 1 -- 172.21.15.26:0/4033752655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df001a020 con 0x559901de4b30 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7de27fc700 1 -- 172.21.15.26:0/4033752655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7df0004700 con 0x559901de4b30 2024-05-20T21:06:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7de27fc700 1 -- 172.21.15.26:0/4033752655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df00166c0 con 0x559901de4b30 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 -- 172.21.15.26:0/4033752655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 msgr2=0x559901de5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901de5820 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7df00098e0 tx=0x7f7df0009bf0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 -- 172.21.15.26:0/4033752655 shutdown_connections 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901de5820 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901dd1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/4033752655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.585+0000 7f7e094d7b80 1 -- 172.21.15.26:0/4033752655 >> 172.21.15.26:0/4033752655 conn(0x559901d02240 msgr2=0x559901dea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 -- 172.21.15.26:0/4033752655 shutdown_connections 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 -- 172.21.15.26:0/4033752655 wait complete. 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 -- start start 2024-05-20T21:06:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 Processor -- start 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901df13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901dfff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81aa070 con 0x559901de4b30 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81ab4e0 con 0x559901ddf0b0 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81aa740 con 0x559901ddfa70 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.586+0000 7f7e094d7b80 10 client.0 _ll_register_callbacks cb 0x559901cbf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901dfff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901dfff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56048/0 (socket says 172.21.15.26:56048) 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 -- 172.21.15.26:0/1176634885 learned_addr learned my addr 172.21.15.26:0/1176634885 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 -- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 msgr2=0x559901dff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 -- 172.21.15.26:0/1176634885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 msgr2=0x559901df13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901df13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 -- 172.21.15.26:0/1176634885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559901e004f0 con 0x559901de4b30 2024-05-20T21:06:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7df7fff700 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dfd1de700 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901dfff00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005ae0 tx=0x7f7df0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce18996a29d2ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:25.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.13 MB/sec execute 391 sec latency 18957.337 ms 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1176634885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df001a030 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1176634885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7df0007e00 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.587+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df0004700 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1176634885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7df0030050 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 10 client.18519 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559901defc70 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1176634885 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7df0004700 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1176634885 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7df001f590 con 0x559901de4b30 2024-05-20T21:06:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 10 client.18519 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 10 client.18519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 20 client.18519 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901e63630 0x559901e65a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 4 client.18519 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559901d304a0 con 0x559901e63630 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.588+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901e63630 0x559901e65a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7dfc9dd700 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901e63630 0x559901e65a70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d2a6163fe18af40 server_cookie=c294b86cd9602cf1 in_seq=0 out_seq=0 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7dd7fff700 10 client.18519 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7dd7fff700 1 -- 172.21.15.26:0/1176634885 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559901d304a0 con 0x559901e63630 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7dd7fff700 10 client.18519 handle_command_reply: tid=0 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 client.18519 shutdown 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 msgr2=0x559901dfff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901dfff00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005ae0 tx=0x7f7df0005b10 comp rx=0 tx=0).stop 2024-05-20T21:06:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 shutdown_connections 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x559901e63630 0x559901e65a70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559901de4b30 0x559901dfff00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559901ddf0b0 0x559901df13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 --2- 172.21.15.26:0/1176634885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559901ddfa70 0x559901dff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 >> 172.21.15.26:0/1176634885 conn(0x559901d02240 msgr2=0x559901dde930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7dd7fff700 0 client.18519 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 shutdown_connections 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 1 -- 172.21.15.26:0/1176634885 wait complete. 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.589+0000 7f7e094d7b80 20 client.18519 trim_cache size 0 max 16384 2024-05-20T21:06:25.594 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7de80b2880 msgr2=0x7f7de80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7de80b2880 0x7f7de80b4d40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005950 tx=0x7f7dec00b410 comp rx=0 tx=0).stop 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 msgr2=0x7f7df81029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 0x7f7df81029f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005f50 tx=0x7f7de40057c0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 shutdown_connections 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df8109630 0x7f7df8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df806bd10 0x7f7df81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7de80b2880 0x7f7de80b4d40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 --2- 172.21.15.26:0/639342405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df806c6d0 0x7f7df81029f0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 >> 172.21.15.26:0/639342405 conn(0x7f7df80794e0 msgr2=0x7f7df8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 shutdown_connections 2024-05-20T21:06:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:25.590+0000 7f7dfec41700 1 -- 172.21.15.26:0/639342405 wait complete. 2024-05-20T21:06:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:26.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:06:26.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.12 MB/sec execute 392 sec latency 19957.390 ms 2024-05-20T21:06:27.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:27 smithi144 ceph-mon[35660]: pgmap v699: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-05-20T21:06:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:27 smithi055 ceph-mon[35213]: pgmap v699: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-05-20T21:06:27.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:27 smithi026 ceph-mon[31990]: pgmap v699: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-05-20T21:06:27.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.12 MB/sec execute 393 sec latency 20957.475 ms 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.229+0000 7f11e079d700 1 -- 172.21.15.26:0/3341441247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806bab0 msgr2=0x7f11d806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.229+0000 7f11e079d700 1 --2- 172.21.15.26:0/3341441247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806bab0 0x7f11d806bf30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009990 tx=0x7f11c8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 -- 172.21.15.26:0/3341441247 shutdown_connections 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 --2- 172.21.15.26:0/3341441247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d806c470 0x7f11d81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 --2- 172.21.15.26:0/3341441247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d81095a0 0x7f11d8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 --2- 172.21.15.26:0/3341441247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806bab0 0x7f11d806bf30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 -- 172.21.15.26:0/3341441247 >> 172.21.15.26:0/3341441247 conn(0x7f11d80793b0 msgr2=0x7f11d80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 -- 172.21.15.26:0/3341441247 shutdown_connections 2024-05-20T21:06:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 -- 172.21.15.26:0/3341441247 wait complete. 2024-05-20T21:06:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.230+0000 7f11e079d700 1 Processor -- start 2024-05-20T21:06:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 -- start start 2024-05-20T21:06:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d806bab0 0x7f11d819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 0x7f11d819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d81095a0 0x7f11d8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8113560 con 0x7f11d806c470 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81133e0 con 0x7f11d806bab0 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11e079d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81136e0 con 0x7f11d81095a0 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11ddd38700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 0x7f11d819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11ddd38700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 0x7f11d819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56080/0 (socket says 172.21.15.26:56080) 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11ddd38700 1 -- 172.21.15.26:0/3653146231 learned_addr learned my addr 172.21.15.26:0/3653146231 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11ded3a700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d81095a0 0x7f11d8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11de539700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d806bab0 0x7f11d819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11ddd38700 1 -- 172.21.15.26:0/3653146231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d81095a0 msgr2=0x7f11d8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.231+0000 7f11ddd38700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d81095a0 0x7f11d8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11ddd38700 1 -- 172.21.15.26:0/3653146231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d806bab0 msgr2=0x7f11d819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11ddd38700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d806bab0 0x7f11d819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11ddd38700 1 -- 172.21.15.26:0/3653146231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d819a090 con 0x7f11d806c470 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11de539700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d806bab0 0x7f11d819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11ddd38700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 0x7f11d819fcb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8027070 con 0x7f11d806c470 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11c800fb80 con 0x7f11d806c470 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8022d00 con 0x7f11d806c470 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d819a320 con 0x7f11d806c470 2024-05-20T21:06:28.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.232+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81a6640 con 0x7f11d806c470 2024-05-20T21:06:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.233+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8104c60 con 0x7f11d806c470 2024-05-20T21:06:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.235+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f11c802a070 con 0x7f11d806c470 2024-05-20T21:06:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.236+0000 7f11cf7fe700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11c40b2880 0x7f11c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.236+0000 7f11de539700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11c40b2880 0x7f11c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.236+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f11c80e3560 con 0x7f11d806c470 2024-05-20T21:06:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.236+0000 7f11de539700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11c40b2880 0x7f11c40b4d40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11d819aa70 tx=0x7f11d4009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:28.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.239+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11c8031070 con 0x7f11d806c470 2024-05-20T21:06:28.261 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.518+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d819ae60 con 0x7f11d806c470 2024-05-20T21:06:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.518+0000 7f11cf7fe700 1 -- 172.21.15.26:0/3653146231 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f11c8013070 con 0x7f11d806c470 2024-05-20T21:06:28.519 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:06:28.519 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11c40b2880 msgr2=0x7f11c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11c40b2880 0x7f11c40b4d40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11d819aa70 tx=0x7f11d4009040 comp rx=0 tx=0).stop 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 msgr2=0x7f11d819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 0x7f11d819fcb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c80047b0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 shutdown_connections 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d81095a0 0x7f11d8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d806bab0 0x7f11d819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11c40b2880 0x7f11c40b4d40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 --2- 172.21.15.26:0/3653146231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d806c470 0x7f11d819fcb0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 >> 172.21.15.26:0/3653146231 conn(0x7f11d80793b0 msgr2=0x7f11d8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 shutdown_connections 2024-05-20T21:06:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:28.521+0000 7f11e079d700 1 -- 172.21.15.26:0/3653146231 wait complete. 2024-05-20T21:06:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:06:28.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.11 MB/sec execute 394 sec latency 21957.579 ms 2024-05-20T21:06:29.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:29 smithi026 ceph-mon[31990]: pgmap v700: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 23 op/s 2024-05-20T21:06:29.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:29 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3653146231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:29 smithi144 ceph-mon[35660]: pgmap v700: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 23 op/s 2024-05-20T21:06:29.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:29 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3653146231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:29.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:29 smithi055 ceph-mon[35213]: pgmap v700: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 23 op/s 2024-05-20T21:06:29.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:29 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3653146231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:29.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.11 MB/sec execute 395 sec latency 22957.634 ms 2024-05-20T21:06:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 -- 172.21.15.26:0/4242026280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64108880 msgr2=0x7fea6410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 --2- 172.21.15.26:0/4242026280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64108880 0x7fea6410ac70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009900 tx=0x7fea5c02a410 comp rx=0 tx=0).stop 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 -- 172.21.15.26:0/4242026280 shutdown_connections 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 --2- 172.21.15.26:0/4242026280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64108880 0x7fea6410ac70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 --2- 172.21.15.26:0/4242026280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea64105f50 0x7fea64108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 --2- 172.21.15.26:0/4242026280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 0x7fea64105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 -- 172.21.15.26:0/4242026280 >> 172.21.15.26:0/4242026280 conn(0x7fea640fcf50 msgr2=0x7fea640ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 -- 172.21.15.26:0/4242026280 shutdown_connections 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.014+0000 7fea6adb6640 1 -- 172.21.15.26:0/4242026280 wait complete. 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 Processor -- start 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 -- start start 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 0x7fea64102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 0x7fea641015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea64108880 0x7fea64101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea6410e540 con 0x7fea64105f50 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fea6410e3c0 con 0x7fea64103620 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea6adb6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea6410e6c0 con 0x7fea64108880 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 0x7fea641015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 0x7fea641015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33892/0 (socket says 172.21.15.26:33892) 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 -- 172.21.15.26:0/3800962268 learned_addr learned my addr 172.21.15.26:0/3800962268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea69353640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea64108880 0x7fea64101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea68b52640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 0x7fea64102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 -- 172.21.15.26:0/3800962268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea64108880 msgr2=0x7fea64101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea64108880 0x7fea64101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 -- 172.21.15.26:0/3800962268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 msgr2=0x7fea64102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 0x7fea64102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea63fff640 1 -- 172.21.15.26:0/3800962268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea641023a0 con 0x7fea64105f50 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.015+0000 7fea68b52640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 0x7fea64102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.016+0000 7fea63fff640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 0x7fea641015d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fea5400cce0 tx=0x7fea5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.016+0000 7fea61ffb640 1 -- 172.21.15.26:0/3800962268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54004b20 con 0x7fea64105f50 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.016+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea641a2c80 con 0x7fea64105f50 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.016+0000 7fea61ffb640 1 -- 172.21.15.26:0/3800962268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea54004cc0 con 0x7fea64105f50 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.016+0000 7fea61ffb640 1 -- 172.21.15.26:0/3800962268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5400f6b0 con 0x7fea64105f50 2024-05-20T21:06:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.016+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a31d0 con 0x7fea64105f50 2024-05-20T21:06:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.019+0000 7fea6dbae740 1 Processor -- start 2024-05-20T21:06:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.019+0000 7fea61ffb640 1 -- 172.21.15.26:0/3800962268 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fea5400f850 con 0x7fea64105f50 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.019+0000 7fea6dbae740 1 -- start start 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf17ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf180b0 0x55d0fdf1cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf1f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea6dbae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d0fde88c30 con 0x55d0fdf176c0 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea6dbae740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d0fde787d0 con 0x55d0fdf1d220 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea6dbae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0fde7c810 con 0x55d0fdf180b0 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf17ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf17ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33906/0 (socket says 172.21.15.26:33906) 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 -- 172.21.15.26:0/1207818779 learned_addr learned my addr 172.21.15.26:0/1207818779 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea63fff640 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf1f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea68b52640 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf180b0 0x55d0fdf1cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 -- 172.21.15.26:0/1207818779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf180b0 msgr2=0x55d0fdf1cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf180b0 0x55d0fdf1cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 -- 172.21.15.26:0/1207818779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 msgr2=0x55d0fdf1f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf1f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 -- 172.21.15.26:0/1207818779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fdb8d210 con 0x55d0fdf176c0 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea69353640 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf17ae0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0057c0 tx=0x7fea5c004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237a34c9ca57066f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea61ffb640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fea500b28a0 0x7fea500b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.020+0000 7fea5affd640 1 -- 172.21.15.26:0/1207818779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c035030 con 0x55d0fdf176c0 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea5affd640 1 -- 172.21.15.26:0/1207818779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea5c040070 con 0x55d0fdf176c0 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea68b52640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fea500b28a0 0x7fea500b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea5affd640 1 -- 172.21.15.26:0/1207818779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c031a10 con 0x55d0fdf176c0 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea61ffb640 1 -- 172.21.15.26:0/3800962268 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fea540e2350 con 0x7fea64105f50 2024-05-20T21:06:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 -- 172.21.15.26:0/1207818779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 msgr2=0x55d0fdf17ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea68b52640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fea500b28a0 0x7fea500b4d60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fea4c005fd0 tx=0x7fea4c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf17ae0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0057c0 tx=0x7fea5c004370 comp rx=0 tx=0).stop 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 -- 172.21.15.26:0/1207818779 shutdown_connections 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf1f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf180b0 0x55d0fdf1cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 --2- 172.21.15.26:0/1207818779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf17ae0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 -- 172.21.15.26:0/1207818779 >> 172.21.15.26:0/1207818779 conn(0x55d0fdf12d30 msgr2=0x55d0fdf13130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 -- 172.21.15.26:0/1207818779 shutdown_connections 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.021+0000 7fea6dbae740 1 -- 172.21.15.26:0/1207818779 wait complete. 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 -- start start 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 Processor -- start 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf21d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdb5e790 con 0x55d0fdf176c0 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d0fde34890 con 0x55d0fdf180b0 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdf20170 con 0x55d0fdf1d220 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea6dbae740 10 client.0 _ll_register_callbacks cb 0x55d0fdb8ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea68b52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea68b52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33912/0 (socket says 172.21.15.26:33912) 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea68b52640 1 -- 172.21.15.26:0/746998095 learned_addr learned my addr 172.21.15.26:0/746998095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea63fff640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf21d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.022+0000 7fea69353640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea68b52640 1 -- 172.21.15.26:0/746998095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf1d220 msgr2=0x55d0fdf28d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea68b52640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea68b52640 1 -- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 msgr2=0x55d0fdf21d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea68b52640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf21d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea68b52640 1 -- 172.21.15.26:0/746998095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fdc59620 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea69353640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea68b52640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21810 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0116c0 tx=0x7fea4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da84b0d910ff740 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea43fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea43fff640 1 -- 172.21.15.26:0/746998095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c022070 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea43fff640 1 -- 172.21.15.26:0/746998095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fea4c01e860 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c040070 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea43fff640 1 -- 172.21.15.26:0/746998095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fea4c01ea60 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea6dbae740 10 client.18537 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0fdf2be40 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.023+0000 7fea43fff640 1 -- 172.21.15.26:0/746998095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea5c040070 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea43fff640 1 -- 172.21.15.26:0/746998095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fea4c005140 con 0x55d0fdf176c0 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea6dbae740 10 client.18537 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea6dbae740 10 client.18537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea6dbae740 20 client.18537 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea6dbae740 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf2d550 0x55d0fdf2f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea6dbae740 4 client.18537 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fde8ab80 con 0x55d0fdf2d550 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea63fff640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf2d550 0x55d0fdf2f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea63fff640 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf2d550 0x55d0fdf2f9e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48e52dae58d7f3c server_cookie=f0095fe0aec960bf in_seq=0 out_seq=0 2024-05-20T21:06:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.024+0000 7fea43fff640 10 client.18537 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea43fff640 1 -- 172.21.15.26:0/746998095 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d0fde8ab80 con 0x55d0fdf2d550 2024-05-20T21:06:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea43fff640 10 client.18537 handle_command_reply: tid=0 2024-05-20T21:06:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 client.18537 shutdown 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 msgr2=0x55d0fdf21810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21810 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0116c0 tx=0x7fea4c004730 comp rx=0 tx=0).stop 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 shutdown_connections 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf2d550 0x55d0fdf2f9e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf21d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 --2- 172.21.15.26:0/746998095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21810 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea43fff640 0 client.18537 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 >> 172.21.15.26:0/746998095 conn(0x55d0fdf12d30 msgr2=0x55d0fdf11fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 shutdown_connections 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.025+0000 7fea6dbae740 1 -- 172.21.15.26:0/746998095 wait complete. 2024-05-20T21:06:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.026+0000 7fea6dbae740 20 client.18537 trim_cache size 0 max 16384 2024-05-20T21:06:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 Processor -- start 2024-05-20T21:06:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 -- start start 2024-05-20T21:06:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf28d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 0x55d0fdf34330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ae970 con 0x55d0fdf1d220 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ac240 con 0x55d0fdf180b0 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea6dbae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea641aa090 con 0x55d0fdf176c0 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea69353640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 0x55d0fdf34330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea69353640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 0x55d0fdf34330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33916/0 (socket says 172.21.15.26:33916) 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.041+0000 7fea69353640 1 -- 172.21.15.26:0/3592904496 learned_addr learned my addr 172.21.15.26:0/3592904496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea63fff640 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf28d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea68b52640 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea69353640 1 -- 172.21.15.26:0/3592904496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 msgr2=0x55d0fdf21cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea69353640 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea69353640 1 -- 172.21.15.26:0/3592904496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 msgr2=0x55d0fdf28d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea69353640 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf28d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea69353640 1 -- 172.21.15.26:0/3592904496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fdf208a0 con 0x55d0fdf1d220 2024-05-20T21:06:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea69353640 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 0x55d0fdf34330 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fea5c005ec0 tx=0x7fea5c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f1511ab64400f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea5a7fc640 1 -- 172.21.15.26:0/3592904496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c035050 con 0x55d0fdf1d220 2024-05-20T21:06:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea5a7fc640 1 -- 172.21.15.26:0/3592904496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea5c040040 con 0x55d0fdf1d220 2024-05-20T21:06:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.042+0000 7fea5a7fc640 1 -- 172.21.15.26:0/3592904496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c031d00 con 0x55d0fdf1d220 2024-05-20T21:06:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 -- 172.21.15.26:0/3592904496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 msgr2=0x55d0fdf34330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 0x55d0fdf34330 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fea5c005ec0 tx=0x7fea5c02a410 comp rx=0 tx=0).stop 2024-05-20T21:06:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 -- 172.21.15.26:0/3592904496 shutdown_connections 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf1d220 0x55d0fdf34330 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf180b0 0x55d0fdf28d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 --2- 172.21.15.26:0/3592904496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf21cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 -- 172.21.15.26:0/3592904496 >> 172.21.15.26:0/3592904496 conn(0x55d0fdf12d30 msgr2=0x55d0fdf16390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 -- 172.21.15.26:0/3592904496 shutdown_connections 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 -- 172.21.15.26:0/3592904496 wait complete. 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 -- start start 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.043+0000 7fea6dbae740 1 Processor -- start 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf26bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 0x55d0fdf27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea641aa700 con 0x55d0fdf180b0 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fea641abb70 con 0x55d0fdf1d220 2024-05-20T21:06:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea641aadd0 con 0x55d0fdf176c0 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 10 client.0 _ll_register_callbacks cb 0x7fea4c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 0x55d0fdf27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 0x55d0fdf27110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33928/0 (socket says 172.21.15.26:33928) 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 -- 172.21.15.26:0/9459521 learned_addr learned my addr 172.21.15.26:0/9459521 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea68b52640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf26bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea69353640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 -- 172.21.15.26:0/9459521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 msgr2=0x55d0fdf26bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf26bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 -- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 msgr2=0x55d0fdf28120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 -- 172.21.15.26:0/9459521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fdf28710 con 0x55d0fdf180b0 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea69353640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea68b52640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf26bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea63fff640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 0x55d0fdf27110 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fea540a6b80 tx=0x7fea54091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3858aae329148d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea43fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea43fff640 1 -- 172.21.15.26:0/9459521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540c2060 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea43fff640 1 -- 172.21.15.26:0/9459521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fea540cc0b0 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.044+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c040040 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea43fff640 1 -- 172.21.15.26:0/9459521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fea5401d040 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 10 client.18543 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0fdf989d0 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea43fff640 1 -- 172.21.15.26:0/9459521 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea5c040040 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea43fff640 1 -- 172.21.15.26:0/9459521 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fea540c8050 con 0x55d0fdf180b0 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 10 client.18543 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 10 client.18543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 20 client.18543 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf99330 0x55d0fdf9b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 4 client.18543 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:06:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0fde84360 con 0x55d0fdf99330 2024-05-20T21:06:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.045+0000 7fea68b52640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf99330 0x55d0fdf9b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.046+0000 7fea43fff640 10 client.18543 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.046+0000 7fea68b52640 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf99330 0x55d0fdf9b750 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a478b97351f4a17 server_cookie=72a4b6e36d360cc in_seq=0 out_seq=0 2024-05-20T21:06:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea43fff640 1 -- 172.21.15.26:0/9459521 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d0fde84360 con 0x55d0fdf99330 2024-05-20T21:06:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea43fff640 10 client.18543 handle_command_reply: tid=0 2024-05-20T21:06:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 client.18543 shutdown 2024-05-20T21:06:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 msgr2=0x55d0fdf27110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 0x55d0fdf27110 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fea540a6b80 tx=0x7fea54091040 comp rx=0 tx=0).stop 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 shutdown_connections 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d0fdf99330 0x55d0fdf9b750 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d0fdf1d220 0x55d0fdf28120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0fdf180b0 0x55d0fdf27110 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 --2- 172.21.15.26:0/9459521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0fdf176c0 0x55d0fdf26bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea43fff640 0 client.18543 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 >> 172.21.15.26:0/9459521 conn(0x55d0fdf12d30 msgr2=0x55d0fdf11720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 shutdown_connections 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 1 -- 172.21.15.26:0/9459521 wait complete. 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.059+0000 7fea6dbae740 20 client.18543 trim_cache size 0 max 16384 2024-05-20T21:06:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:06:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:06:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:06:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fea500b28a0 msgr2=0x7fea500b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fea500b28a0 0x7fea500b4d60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fea4c005fd0 tx=0x7fea4c005ec0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 msgr2=0x7fea641015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 0x7fea641015d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fea5400cce0 tx=0x7fea5400a620 comp rx=0 tx=0).stop 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 shutdown_connections 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fea500b28a0 0x7fea500b4d60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea64108880 0x7fea64101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea64105f50 0x7fea641015d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 --2- 172.21.15.26:0/3800962268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea64103620 0x7fea64102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 >> 172.21.15.26:0/3800962268 conn(0x7fea640fcf50 msgr2=0x7fea64103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 shutdown_connections 2024-05-20T21:06:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:30.060+0000 7fea6adb6640 1 -- 172.21.15.26:0/3800962268 wait complete. 2024-05-20T21:06:30.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.10 MB/sec execute 396 sec latency 23957.733 ms 2024-05-20T21:06:30.671 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id a7dfde6b --release --await' 2024-05-20T21:06:30.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id a7dfde6b --release --await 2024-05-20T21:06:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 -- 172.21.15.26:0/1229507904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06b220 msgr2=0x7f434c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 --2- 172.21.15.26:0/1229507904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06b220 0x7f434c06b6a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4340009990 tx=0x7f4340009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 -- 172.21.15.26:0/1229507904 shutdown_connections 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 --2- 172.21.15.26:0/1229507904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06bbe0 0x7f434c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 --2- 172.21.15.26:0/1229507904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 0x7f434c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 --2- 172.21.15.26:0/1229507904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06b220 0x7f434c06b6a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 -- 172.21.15.26:0/1229507904 >> 172.21.15.26:0/1229507904 conn(0x7f434c0793f0 msgr2=0x7f434c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 -- 172.21.15.26:0/1229507904 shutdown_connections 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.860+0000 7f4351f24700 1 -- 172.21.15.26:0/1229507904 wait complete. 2024-05-20T21:06:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 Processor -- start 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 -- start start 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06b220 0x7f434c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 0x7f434c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 0x7f434c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f434c113570 con 0x7f434c06bbe0 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1133f0 con 0x7f434c06b220 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f4351f24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1136f0 con 0x7f434c1095b0 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f434affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 0x7f434c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f434affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 0x7f434c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33944/0 (socket says 172.21.15.26:33944) 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f434affd700 1 -- 172.21.15.26:0/2406020911 learned_addr learned my addr 172.21.15.26:0/2406020911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.861+0000 7f434b7fe700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06b220 0x7f434c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434bfff700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 0x7f434c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434affd700 1 -- 172.21.15.26:0/2406020911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 msgr2=0x7f434c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434affd700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 0x7f434c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434affd700 1 -- 172.21.15.26:0/2406020911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06b220 msgr2=0x7f434c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434affd700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06b220 0x7f434c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434affd700 1 -- 172.21.15.26:0/2406020911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c19a090 con 0x7f434c06bbe0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434bfff700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 0x7f434c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f434affd700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 0x7f434c19fcb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4340000c00 tx=0x7f43400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f4348ff9700 1 -- 172.21.15.26:0/2406020911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340027070 con 0x7f434c06bbe0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f4348ff9700 1 -- 172.21.15.26:0/2406020911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f434000fb80 con 0x7f434c06bbe0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f4348ff9700 1 -- 172.21.15.26:0/2406020911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340022d00 con 0x7f434c06bbe0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434c19a320 con 0x7f434c06bbe0 2024-05-20T21:06:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.862+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434c1a6640 con 0x7f434c06bbe0 2024-05-20T21:06:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.865+0000 7f4348ff9700 1 -- 172.21.15.26:0/2406020911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f434002a070 con 0x7f434c06bbe0 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f4348ff9700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2880 0x7f43340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f434b7fe700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2880 0x7f43340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 Processor -- start 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f4348ff9700 1 -- 172.21.15.26:0/2406020911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f43400e3040 con 0x7f434c06bbe0 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 -- start start 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dbdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f65d1bd70 con 0x559f65dc3320 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559f65d0b910 con 0x559f65dbd8a0 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.866+0000 7f435c7bab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f65d0f950 con 0x559f65dbe260 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434b7fe700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2880 0x7f43340b4d40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f434c19aa70 tx=0x7f433c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33950/0 (socket says 172.21.15.26:33950) 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434b7fe700 1 -- 172.21.15.26:0/1079624439 learned_addr learned my addr 172.21.15.26:0/1079624439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434affd700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dbdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 -- 172.21.15.26:0/1079624439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 msgr2=0x559f65dbdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dbdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 -- 172.21.15.26:0/1079624439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 msgr2=0x559f65dc3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 -- 172.21.15.26:0/1079624439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f65cb4fb0 con 0x559f65dbe260 2024-05-20T21:06:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434affd700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dbdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434b7fe700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.867+0000 7f434bfff700 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dca080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4344009c60 tx=0x7f4344009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5839640f5ecc3c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f4331ffb700 1 -- 172.21.15.26:0/1079624439 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344019070 con 0x559f65dbe260 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f4331ffb700 1 -- 172.21.15.26:0/1079624439 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4344014890 con 0x559f65dbe260 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f4331ffb700 1 -- 172.21.15.26:0/1079624439 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43440137a0 con 0x559f65dbe260 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 -- 172.21.15.26:0/1079624439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 msgr2=0x559f65dca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dca080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4344009c60 tx=0x7f4344009f70 comp rx=0 tx=0).stop 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 -- 172.21.15.26:0/1079624439 shutdown_connections 2024-05-20T21:06:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dca080 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1079624439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dbdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.868+0000 7f435c7bab80 1 -- 172.21.15.26:0/1079624439 >> 172.21.15.26:0/1079624439 conn(0x559f65ce9240 msgr2=0x559f65db83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 -- 172.21.15.26:0/1079624439 shutdown_connections 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 -- 172.21.15.26:0/1079624439 wait complete. 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 -- start start 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 Processor -- start 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dc9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dc7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.869+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f435c7bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f65c11fa0 con 0x559f65dc3320 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f435c7bab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559f65c73da0 con 0x559f65dbd8a0 2024-05-20T21:06:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f435c7bab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f65dca6e0 con 0x559f65dbe260 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f435c7bab80 10 client.0 _ll_register_callbacks cb 0x559f65ca6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dc7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33964/0 (socket says 172.21.15.26:33964) 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434bfff700 1 -- 172.21.15.26:0/552671695 learned_addr learned my addr 172.21.15.26:0/552671695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dc7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38296/0 (socket says 172.21.15.26:38296) 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434b7fe700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dc9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 -- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 msgr2=0x559f65dc9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dc9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 -- 172.21.15.26:0/552671695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 msgr2=0x559f65dc7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 -- 172.21.15.26:0/552671695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f65c5b8d0 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434bfff700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f434affd700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dc7010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43400e0bd0 tx=0x7f43400aac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4522851c26191b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.870+0000 7f432affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f432affd700 1 -- 172.21.15.26:0/552671695 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400aa240 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f432affd700 1 -- 172.21.15.26:0/552671695 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f434000f6f0 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4344014890 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f432affd700 1 -- 172.21.15.26:0/552671695 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43400aae30 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 10 client.27673 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f65cc5760 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f432affd700 1 -- 172.21.15.26:0/552671695 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4344014890 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f432affd700 1 -- 172.21.15.26:0/552671695 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f43400a60a0 con 0x559f65dbe260 2024-05-20T21:06:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 10 client.27673 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 10 client.27673 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 20 client.27673 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65dd7140 0x559f65dd9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 4 client.27673 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.871+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f65d1dcc0 con 0x559f65dd7140 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.872+0000 7f434b7fe700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65dd7140 0x559f65dd9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.872+0000 7f434b7fe700 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65dd7140 0x559f65dd9560 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=232dfba5b31115b9 server_cookie=2e925f395d0aa631 in_seq=0 out_seq=0 2024-05-20T21:06:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.872+0000 7f432affd700 10 client.27673 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f432affd700 1 -- 172.21.15.26:0/552671695 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f65d1dcc0 con 0x559f65dd7140 2024-05-20T21:06:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f432affd700 10 client.27673 handle_command_reply: tid=0 2024-05-20T21:06:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 client.27673 shutdown 2024-05-20T21:06:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 msgr2=0x559f65dc7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dc7010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43400e0bd0 tx=0x7f43400aac70 comp rx=0 tx=0).stop 2024-05-20T21:06:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 shutdown_connections 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65dd7140 0x559f65dd9560 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65dc7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dc9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f432affd700 0 client.27673 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 --2- 172.21.15.26:0/552671695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dc7010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 >> 172.21.15.26:0/552671695 conn(0x559f65ce9240 msgr2=0x559f65db7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 shutdown_connections 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 1 -- 172.21.15.26:0/552671695 wait complete. 2024-05-20T21:06:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.873+0000 7f435c7bab80 20 client.27673 trim_cache size 0 max 16384 2024-05-20T21:06:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.890+0000 7f435c7bab80 1 Processor -- start 2024-05-20T21:06:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.890+0000 7f435c7bab80 1 -- start start 2024-05-20T21:06:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 0x559f65dd9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 0x559f65dc15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f434b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 0x559f65dd9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f434b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 0x559f65dd9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38310/0 (socket says 172.21.15.26:38310) 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f434affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 0x559f65dc15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f434affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 0x559f65dc15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33972/0 (socket says 172.21.15.26:33972) 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f434b7fe700 1 -- 172.21.15.26:0/1840160688 learned_addr learned my addr 172.21.15.26:0/1840160688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dc3320 0x559f65db8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1b03d0 con 0x559f65dbe260 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1adc60 con 0x559f65dc3320 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.891+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1abab0 con 0x559f65dbd8a0 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434bfff700 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dc3320 0x559f65db8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434b7fe700 1 -- 172.21.15.26:0/1840160688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dc3320 msgr2=0x559f65db8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434b7fe700 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dc3320 0x559f65db8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434b7fe700 1 -- 172.21.15.26:0/1840160688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 msgr2=0x559f65dc15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434b7fe700 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 0x559f65dc15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434b7fe700 1 -- 172.21.15.26:0/1840160688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f65dc2f40 con 0x559f65dbd8a0 2024-05-20T21:06:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434affd700 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 0x559f65dc15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f434b7fe700 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 0x559f65dd9560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f433c005bf0 tx=0x7f433c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=284ae1c8f72fcbbb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f43317fa700 1 -- 172.21.15.26:0/1840160688 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c011e40 con 0x559f65dbd8a0 2024-05-20T21:06:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f43317fa700 1 -- 172.21.15.26:0/1840160688 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f433c030070 con 0x559f65dbd8a0 2024-05-20T21:06:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.892+0000 7f43317fa700 1 -- 172.21.15.26:0/1840160688 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c01e980 con 0x559f65dbd8a0 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 msgr2=0x559f65dd9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 0x559f65dd9560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f433c005bf0 tx=0x7f433c013000 comp rx=0 tx=0).stop 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 shutdown_connections 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dc3320 0x559f65db8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbd8a0 0x559f65dd9560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 --2- 172.21.15.26:0/1840160688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dbe260 0x559f65dc15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 >> 172.21.15.26:0/1840160688 conn(0x559f65ce9240 msgr2=0x559f65dc27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 shutdown_connections 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.893+0000 7f435c7bab80 1 -- 172.21.15.26:0/1840160688 wait complete. 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.894+0000 7f435c7bab80 1 -- start start 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.894+0000 7f435c7bab80 1 Processor -- start 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.894+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dde1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.894+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dde720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.894+0000 7f435c7bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65ddec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f435c7bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1ac120 con 0x559f65dc3320 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f435c7bab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1ad590 con 0x559f65dbd8a0 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f435c7bab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1ac7f0 con 0x559f65dbe260 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65ddec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65ddec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33982/0 (socket says 172.21.15.26:33982) 2024-05-20T21:06:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 -- 172.21.15.26:0/2652605036 learned_addr learned my addr 172.21.15.26:0/2652605036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434affd700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dde720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434b7fe700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dde1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f435c7bab80 10 client.0 _ll_register_callbacks cb 0x559f65ca6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 -- 172.21.15.26:0/2652605036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 msgr2=0x559f65dde720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dde720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 -- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 msgr2=0x559f65dde1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dde1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 -- 172.21.15.26:0/2652605036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f65dcebe0 con 0x559f65dc3320 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434affd700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dde720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f434bfff700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65ddec60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f43440094f0 tx=0x7f4344004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ef13d096433e46 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f432affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.895+0000 7f432affd700 1 -- 172.21.15.26:0/2652605036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344023020 con 0x559f65dc3320 2024-05-20T21:06:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f432affd700 1 -- 172.21.15.26:0/2652605036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4344014910 con 0x559f65dc3320 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c030070 con 0x559f65dc3320 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f432affd700 1 -- 172.21.15.26:0/2652605036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f434402e050 con 0x559f65dc3320 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 10 client.18561 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f65dcfe40 con 0x559f65dc3320 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f432affd700 1 -- 172.21.15.26:0/2652605036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f433c030070 con 0x559f65dc3320 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f432affd700 1 -- 172.21.15.26:0/2652605036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f434401f960 con 0x559f65dc3320 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 10 client.18561 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 10 client.18561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 20 client.18561 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65e4a8c0 0x559f65e4ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 4 client.18561 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.896+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f65d174a0 con 0x559f65e4a8c0 2024-05-20T21:06:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f434b7fe700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65e4a8c0 0x559f65e4ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f432affd700 10 client.18561 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f434b7fe700 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65e4a8c0 0x559f65e4ccb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea54f38e3533d38c server_cookie=38102228f62597a8 in_seq=0 out_seq=0 2024-05-20T21:06:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f432affd700 1 -- 172.21.15.26:0/2652605036 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f65d174a0 con 0x559f65e4a8c0 2024-05-20T21:06:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f432affd700 10 client.18561 handle_command_reply: tid=0 2024-05-20T21:06:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f435c7bab80 1 client.18561 shutdown 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 msgr2=0x559f65ddec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.897+0000 7f435c7bab80 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65ddec60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f43440094f0 tx=0x7f4344004ee0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 shutdown_connections 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559f65e4a8c0 0x559f65e4ccb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f65dc3320 0x559f65ddec60 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559f65dbd8a0 0x559f65dde1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 --2- 172.21.15.26:0/2652605036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f65dbe260 0x559f65dde720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 >> 172.21.15.26:0/2652605036 conn(0x559f65ce9240 msgr2=0x559f65dbc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f432affd700 0 client.18561 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 shutdown_connections 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 1 -- 172.21.15.26:0/2652605036 wait complete. 2024-05-20T21:06:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.898+0000 7f435c7bab80 20 client.18561 trim_cache size 0 max 16384 2024-05-20T21:06:30.903 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2880 msgr2=0x7f43340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2880 0x7f43340b4d40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f434c19aa70 tx=0x7f433c00b410 comp rx=0 tx=0).stop 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 msgr2=0x7f434c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 0x7f434c19fcb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4340000c00 tx=0x7f43400047b0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 shutdown_connections 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f434c1095b0 0x7f434c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f434c06b220 0x7f434c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f43340b2880 0x7f43340b4d40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 --2- 172.21.15.26:0/2406020911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f434c06bbe0 0x7f434c19fcb0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 >> 172.21.15.26:0/2406020911 conn(0x7f434c0793f0 msgr2=0x7f434c101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 shutdown_connections 2024-05-20T21:06:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:30.899+0000 7f4351f24700 1 -- 172.21.15.26:0/2406020911 wait complete. 2024-05-20T21:06:31.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:31 smithi026 ceph-mon[31990]: pgmap v701: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-05-20T21:06:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:31 smithi144 ceph-mon[35660]: pgmap v701: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-05-20T21:06:31.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:31 smithi055 ceph-mon[35213]: pgmap v701: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-05-20T21:06:31.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.10 MB/sec execute 397 sec latency 24957.812 ms 2024-05-20T21:06:31.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:06:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:32.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.09 MB/sec execute 398 sec latency 25957.857 ms 2024-05-20T21:06:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:33 smithi144 ceph-mon[35660]: pgmap v702: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-20T21:06:33.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:33 smithi026 ceph-mon[31990]: pgmap v702: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-20T21:06:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:33 smithi055 ceph-mon[35213]: pgmap v702: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-20T21:06:33.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.09 MB/sec execute 399 sec latency 26957.965 ms 2024-05-20T21:06:34.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219978 2.08 MB/sec execute 400 sec latency 27958.007 ms 2024-05-20T21:06:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 -- 172.21.15.26:0/2823370322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 msgr2=0x7f22d0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 --2- 172.21.15.26:0/2823370322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0103bd0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0099c0 tx=0x7f22cc009cd0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 -- 172.21.15.26:0/2823370322 shutdown_connections 2024-05-20T21:06:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 --2- 172.21.15.26:0/2823370322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0103bd0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 --2- 172.21.15.26:0/2823370322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22d01088f0 0x7f22d0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 --2- 172.21.15.26:0/2823370322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22d01041a0 0x7f22d01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 -- 172.21.15.26:0/2823370322 >> 172.21.15.26:0/2823370322 conn(0x7f22d007a6a0 msgr2=0x7f22d007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 -- 172.21.15.26:0/2823370322 shutdown_connections 2024-05-20T21:06:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.072+0000 7f22d79c9700 1 -- 172.21.15.26:0/2823370322 wait complete. 2024-05-20T21:06:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 Processor -- start 2024-05-20T21:06:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 -- start start 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22d01041a0 0x7f22d0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22d01088f0 0x7f22d00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0113660 con 0x7f22d01037f0 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01134e0 con 0x7f22d01088f0 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d79c9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01137e0 con 0x7f22d01041a0 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d5765700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d5765700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33992/0 (socket says 172.21.15.26:33992) 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d5765700 1 -- 172.21.15.26:0/3213977459 learned_addr learned my addr 172.21.15.26:0/3213977459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d4f64700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22d01041a0 0x7f22d0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d5f66700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22d01088f0 0x7f22d00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.073+0000 7f22d5765700 1 -- 172.21.15.26:0/3213977459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22d01041a0 msgr2=0x7f22d0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d5765700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22d01041a0 0x7f22d0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d5765700 1 -- 172.21.15.26:0/3213977459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22d01088f0 msgr2=0x7f22d00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d5765700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22d01088f0 0x7f22d00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d5765700 1 -- 172.21.15.26:0/3213977459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d01a80e0 con 0x7f22d01037f0 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d5765700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0075840 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f22cc000c00 tx=0x7f22cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22c67fc700 1 -- 172.21.15.26:0/3213977459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc026070 con 0x7f22d01037f0 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01a8370 con 0x7f22d01037f0 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22c67fc700 1 -- 172.21.15.26:0/3213977459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22cc005030 con 0x7f22d01037f0 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22c67fc700 1 -- 172.21.15.26:0/3213977459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc004b00 con 0x7f22d01037f0 2024-05-20T21:06:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.074+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01a87c0 con 0x7f22d01037f0 2024-05-20T21:06:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.077+0000 7f22c67fc700 1 -- 172.21.15.26:0/3213977459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f22cc029070 con 0x7f22d01037f0 2024-05-20T21:06:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.078+0000 7f22c67fc700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22bc0b2880 0x7f22bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.078+0000 7f22d4f64700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22bc0b2880 0x7f22bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.078+0000 7f22c67fc700 1 -- 172.21.15.26:0/3213977459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22cc0e2cd0 con 0x7f22d01037f0 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22d4f64700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22bc0b2880 0x7f22bc0b4d40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f22d01031d0 tx=0x7f22c0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 Processor -- start 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 -- start start 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b48072f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480729d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b480688d90 con 0x55b4807298e0 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b480678930 con 0x55b48072f2b0 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22e225fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b48067c970 con 0x55b48072a2a0 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22d4f64700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480729d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22d4f64700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480729d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34008/0 (socket says 172.21.15.26:34008) 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22d4f64700 1 -- 172.21.15.26:0/2747526264 learned_addr learned my addr 172.21.15.26:0/2747526264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.079+0000 7f22d5765700 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b48072f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22d4f64700 1 -- 172.21.15.26:0/2747526264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 msgr2=0x55b4807360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22d4f64700 1 -- 172.21.15.26:0/2747526264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 msgr2=0x55b48072f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b48072f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22d4f64700 1 -- 172.21.15.26:0/2747526264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4805c89d0 con 0x55b4807298e0 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480729d60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009fd0 tx=0x7f22c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77594ca9320ca82a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22bb7fe700 1 -- 172.21.15.26:0/2747526264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c001b040 con 0x55b4807298e0 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22bb7fe700 1 -- 172.21.15.26:0/2747526264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22c0024070 con 0x55b4807298e0 2024-05-20T21:06:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.080+0000 7f22bb7fe700 1 -- 172.21.15.26:0/2747526264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0013650 con 0x55b4807298e0 2024-05-20T21:06:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 -- 172.21.15.26:0/2747526264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 msgr2=0x55b480729d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480729d60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009fd0 tx=0x7f22c0011000 comp rx=0 tx=0).stop 2024-05-20T21:06:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 -- 172.21.15.26:0/2747526264 shutdown_connections 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b48072f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2747526264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480729d60 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 -- 172.21.15.26:0/2747526264 >> 172.21.15.26:0/2747526264 conn(0x55b480657180 msgr2=0x55b4807241f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 -- 172.21.15.26:0/2747526264 shutdown_connections 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 -- 172.21.15.26:0/2747526264 wait complete. 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 -- start start 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.081+0000 7f22e225fb80 1 Processor -- start 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480735df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480733630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b48057f0a0 con 0x55b4807298e0 2024-05-20T21:06:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b48061a8c0 con 0x55b48072f2b0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b480736680 con 0x55b48072a2a0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22e225fb80 10 client.0 _ll_register_callbacks cb 0x55b480613a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480735df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480735df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34022/0 (socket says 172.21.15.26:34022) 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 -- 172.21.15.26:0/2278449990 learned_addr learned my addr 172.21.15.26:0/2278449990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5f66700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480733630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 -- 172.21.15.26:0/2278449990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 msgr2=0x55b4807330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807330f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 -- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 msgr2=0x55b480733630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480733630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5765700 1 -- 172.21.15.26:0/2278449990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b480617790 con 0x55b4807298e0 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.082+0000 7f22d5f66700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480733630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22d5765700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480735df0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0e0680 tx=0x7f22cc0a9dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dea8d258c095a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2278449990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0c4060 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2278449990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22cc0aa520 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c0024070 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2278449990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22cc0ae670 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 10 client.18573 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b480740e00 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2278449990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22c0024070 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2278449990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f22cc0b2c00 con 0x55b4807298e0 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 10 client.18573 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 10 client.18573 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:06:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 20 client.18573 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b480743410 0x55b480745870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 4 client.18573 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.083+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4806f5ae0 con 0x55b480743410 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.084+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b480743410 0x55b480745870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.084+0000 7f22b8ff9700 10 client.18573 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.084+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b480743410 0x55b480745870 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda9af57c7c9d93d server_cookie=1eefaf611807f812 in_seq=0 out_seq=0 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2278449990 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b4806f5ae0 con 0x55b480743410 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22b8ff9700 10 client.18573 handle_command_reply: tid=0 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 client.18573 shutdown 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 msgr2=0x55b480735df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480735df0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0e0680 tx=0x7f22cc0a9dd0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 shutdown_connections 2024-05-20T21:06:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b480743410 0x55b480745870 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4807298e0 0x55b480735df0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480733630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2278449990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b4807330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22b8ff9700 0 client.18573 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 >> 172.21.15.26:0/2278449990 conn(0x55b480657180 msgr2=0x55b48071eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 shutdown_connections 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 1 -- 172.21.15.26:0/2278449990 wait complete. 2024-05-20T21:06:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.085+0000 7f22e225fb80 20 client.18573 trim_cache size 0 max 16384 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 Processor -- start 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 -- start start 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4807298e0 0x55b48071eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b480735930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 0x55b48072bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01b2690 con 0x55b48072f2b0 2024-05-20T21:06:35.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01aff20 con 0x55b4807298e0 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22e225fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01add70 con 0x55b48072a2a0 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22d5f66700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 0x55b48072bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22d5f66700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 0x55b48072bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34034/0 (socket says 172.21.15.26:34034) 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.106+0000 7f22d5f66700 1 -- 172.21.15.26:0/3242663763 learned_addr learned my addr 172.21.15.26:0/3242663763 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d4f64700 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b480735930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5765700 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4807298e0 0x55b48071eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5f66700 1 -- 172.21.15.26:0/3242663763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 msgr2=0x55b480735930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5f66700 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b480735930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5f66700 1 -- 172.21.15.26:0/3242663763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4807298e0 msgr2=0x55b48071eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5f66700 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4807298e0 0x55b48071eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5f66700 1 -- 172.21.15.26:0/3242663763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b480731680 con 0x55b48072f2b0 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5765700 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4807298e0 0x55b48071eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22d5f66700 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 0x55b48072bd90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f22c800a530 tx=0x7f22c800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb58ac2468f409 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22baffd700 1 -- 172.21.15.26:0/3242663763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80153f0 con 0x55b48072f2b0 2024-05-20T21:06:35.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22baffd700 1 -- 172.21.15.26:0/3242663763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22c801a070 con 0x55b48072f2b0 2024-05-20T21:06:35.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.107+0000 7f22baffd700 1 -- 172.21.15.26:0/3242663763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8014770 con 0x55b48072f2b0 2024-05-20T21:06:35.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 -- 172.21.15.26:0/3242663763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 msgr2=0x55b48072bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 0x55b48072bd90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f22c800a530 tx=0x7f22c800a560 comp rx=0 tx=0).stop 2024-05-20T21:06:35.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 -- 172.21.15.26:0/3242663763 shutdown_connections 2024-05-20T21:06:35.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072f2b0 0x55b48072bd90 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4807298e0 0x55b48071eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 --2- 172.21.15.26:0/3242663763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b48072a2a0 0x55b480735930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 -- 172.21.15.26:0/3242663763 >> 172.21.15.26:0/3242663763 conn(0x55b480657180 msgr2=0x55b48071e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 -- 172.21.15.26:0/3242663763 shutdown_connections 2024-05-20T21:06:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 -- 172.21.15.26:0/3242663763 wait complete. 2024-05-20T21:06:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 -- start start 2024-05-20T21:06:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.108+0000 7f22e225fb80 1 Processor -- start 2024-05-20T21:06:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4807298e0 0x55b48073bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 0x55b4807368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480736e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01ae3e0 con 0x55b48072a2a0 2024-05-20T21:06:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01af850 con 0x55b48072f2b0 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01aeab0 con 0x55b4807298e0 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22e225fb80 10 client.0 _ll_register_callbacks cb 0x55b480613a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 0x55b4807368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 0x55b4807368d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34048/0 (socket says 172.21.15.26:34048) 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 -- 172.21.15.26:0/2935462595 learned_addr learned my addr 172.21.15.26:0/2935462595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d5765700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4807298e0 0x55b48073bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d5f66700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480736e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 -- 172.21.15.26:0/2935462595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4807298e0 msgr2=0x55b48073bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4807298e0 0x55b48073bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 -- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 msgr2=0x55b480736e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480736e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 -- 172.21.15.26:0/2935462595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b480737650 con 0x55b48072a2a0 2024-05-20T21:06:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d5f66700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480736e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22d4f64700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 0x55b4807368d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f22c000ecb0 tx=0x7f22c0009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d1e9afb1479ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.109+0000 7f22b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2935462595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0015800 con 0x55b48072a2a0 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2935462595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22c00159a0 con 0x55b48072a2a0 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c801a070 con 0x55b48072a2a0 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2935462595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22c0015ba0 con 0x55b48072a2a0 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 10 client.18579 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4807aefb0 con 0x55b48072a2a0 2024-05-20T21:06:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2935462595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22c801a070 con 0x55b48072a2a0 2024-05-20T21:06:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2935462595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f22c001e030 con 0x55b48072a2a0 2024-05-20T21:06:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 10 client.18579 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 10 client.18579 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:06:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 20 client.18579 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4807af330 0x55b4807b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 4 client.18579 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a7dfde6b", "release": true, "await": true}] 2024-05-20T21:06:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.110+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a7dfde6b", "release": true, "await": true}) -- 0x55b48068ace0 con 0x55b4807af330 2024-05-20T21:06:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.111+0000 7f22d5765700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4807af330 0x55b4807b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.111+0000 7f22d5765700 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4807af330 0x55b4807b1720 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57c1a72976b97a8 server_cookie=f96faa073e4cceaa in_seq=0 out_seq=0 2024-05-20T21:06:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.111+0000 7f22b8ff9700 10 client.18579 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22b8ff9700 1 -- 172.21.15.26:0/2935462595 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b48068ace0 con 0x55b4807af330 2024-05-20T21:06:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22b8ff9700 10 client.18579 handle_command_reply: tid=0 2024-05-20T21:06:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 client.18579 shutdown 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 msgr2=0x55b4807368d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 0x55b4807368d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f22c000ecb0 tx=0x7f22c0009200 comp rx=0 tx=0).stop 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 shutdown_connections 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4807af330 0x55b4807b1720 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b48072f2b0 0x55b480736e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4807298e0 0x55b48073bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22b8ff9700 0 client.18579 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 --2- 172.21.15.26:0/2935462595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b48072a2a0 0x55b4807368d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.123+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 >> 172.21.15.26:0/2935462595 conn(0x55b480657180 msgr2=0x55b48071eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 shutdown_connections 2024-05-20T21:06:35.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22e225fb80 1 -- 172.21.15.26:0/2935462595 wait complete. 2024-05-20T21:06:35.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22e225fb80 20 client.18579 trim_cache size 0 max 16384 2024-05-20T21:06:35.127 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:06:35.127 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:06:35.127 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:06:35.127 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 44, 2024-05-20T21:06:35.127 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:06:35.128 INFO:teuthology.orchestra.run.smithi026.stdout: "a7dfde6b": { 2024-05-20T21:06:35.128 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 44, 2024-05-20T21:06:35.128 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:06:35.128 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:06:35.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:06:35.128 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:06:35.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:06:35.129 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:06:35.129 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 146.6, 2024-05-20T21:06:35.129 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:06:35.129 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:06:35.129 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:35.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:06:35.131 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:06:35.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22bc0b2880 msgr2=0x7f22bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22d79c9700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22bc0b2880 0x7f22bc0b4d40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f22d01031d0 tx=0x7f22c0009190 comp rx=0 tx=0).stop 2024-05-20T21:06:35.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 msgr2=0x7f22d0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.124+0000 7f22d79c9700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0075840 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f22cc000c00 tx=0x7f22cc00f690 comp rx=0 tx=0).stop 2024-05-20T21:06:35.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 shutdown_connections 2024-05-20T21:06:35.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f22bc0b2880 0x7f22bc0b4d40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22d01037f0 0x7f22d0075840 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22d01088f0 0x7f22d00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 --2- 172.21.15.26:0/3213977459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22d01041a0 0x7f22d0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 >> 172.21.15.26:0/3213977459 conn(0x7f22d007a6a0 msgr2=0x7f22d0109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 shutdown_connections 2024-05-20T21:06:35.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.125+0000 7f22d79c9700 1 -- 172.21.15.26:0/3213977459 wait complete. 2024-05-20T21:06:35.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:35 smithi026 ceph-mon[31990]: pgmap v703: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-20T21:06:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:35 smithi144 ceph-mon[35660]: pgmap v703: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-20T21:06:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 -- 172.21.15.26:0/1989241203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 msgr2=0x7f656c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 --2- 172.21.15.26:0/1989241203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c103bd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f65680099c0 tx=0x7f6568009cd0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 -- 172.21.15.26:0/1989241203 shutdown_connections 2024-05-20T21:06:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 --2- 172.21.15.26:0/1989241203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c103bd0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 --2- 172.21.15.26:0/1989241203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1088f0 0x7f656c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 --2- 172.21.15.26:0/1989241203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1041a0 0x7f656c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 -- 172.21.15.26:0/1989241203 >> 172.21.15.26:0/1989241203 conn(0x7f656c07a6a0 msgr2=0x7f656c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 -- 172.21.15.26:0/1989241203 shutdown_connections 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.328+0000 7f6573880700 1 -- 172.21.15.26:0/1989241203 wait complete. 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 Processor -- start 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 -- start start 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1041a0 0x7f656c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1088f0 0x7f656c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f656c113570 con 0x7f656c1037f0 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1133f0 con 0x7f656c1088f0 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f6573880700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1136f0 con 0x7f656c1041a0 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.329+0000 7f657161c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34060/0 (socket says 172.21.15.26:34060) 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 -- 172.21.15.26:0/3350927638 learned_addr learned my addr 172.21.15.26:0/3350927638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f6571e1d700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1088f0 0x7f656c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f6570e1b700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1041a0 0x7f656c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 -- 172.21.15.26:0/3350927638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1041a0 msgr2=0x7f656c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1041a0 0x7f656c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 -- 172.21.15.26:0/3350927638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1088f0 msgr2=0x7f656c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1088f0 0x7f656c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 -- 172.21.15.26:0/3350927638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c1a8010 con 0x7f656c1037f0 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f6570e1b700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1041a0 0x7f656c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f6571e1d700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1088f0 0x7f656c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f657161c700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c075840 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6568006010 tx=0x7f65680055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f65627fc700 1 -- 172.21.15.26:0/3350927638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568026070 con 0x7f656c1037f0 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f65627fc700 1 -- 172.21.15.26:0/3350927638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f656800f460 con 0x7f656c1037f0 2024-05-20T21:06:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c1a82a0 con 0x7f656c1037f0 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f65627fc700 1 -- 172.21.15.26:0/3350927638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568009e60 con 0x7f656c1037f0 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.330+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c1a8750 con 0x7f656c1037f0 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.333+0000 7f65627fc700 1 -- 172.21.15.26:0/3350927638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6568029070 con 0x7f656c1037f0 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.334+0000 7f65627fc700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f65580b2980 0x7f65580b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 Processor -- start 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6570e1b700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f65580b2980 0x7f65580b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 -- start start 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f65627fc700 1 -- 172.21.15.26:0/3350927638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f65680e2b10 con 0x7f656c1037f0 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ff03429d70 con 0x55ff034cc260 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ff03419910 con 0x55ff034cb8a0 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657e116b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff0341d950 con 0x55ff034d1320 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6570e1b700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f65580b2980 0x7f65580b4e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f656c1031f0 tx=0x7f655c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6570e1b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6571e1d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657161c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6570e1b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:52908/0 (socket says 172.21.15.26:52908) 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6571e1d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034d8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34062/0 (socket says 172.21.15.26:34062) 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f6570e1b700 1 -- 172.21.15.26:0/2310714185 learned_addr learned my addr 172.21.15.26:0/2310714185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.335+0000 7f657161c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38402/0 (socket says 172.21.15.26:38402) 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f657161c700 1 -- 172.21.15.26:0/2310714185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 msgr2=0x55ff034cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f657161c700 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f657161c700 1 -- 172.21.15.26:0/2310714185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 msgr2=0x55ff034d8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f657161c700 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034d8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f657161c700 1 -- 172.21.15.26:0/2310714185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff033c2fb0 con 0x55ff034d1320 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f6571e1d700 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f657161c700 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034d1700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65680a7370 tx=0x7f65680a98f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa4df1e71d77413 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f65577fe700 1 -- 172.21.15.26:0/2310714185 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680aa100 con 0x55ff034d1320 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f65577fe700 1 -- 172.21.15.26:0/2310714185 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65680ae4b0 con 0x55ff034d1320 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.336+0000 7f65577fe700 1 -- 172.21.15.26:0/2310714185 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680aa360 con 0x55ff034d1320 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 -- 172.21.15.26:0/2310714185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 msgr2=0x55ff034d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034d1700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65680a7370 tx=0x7f65680a98f0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 -- 172.21.15.26:0/2310714185 shutdown_connections 2024-05-20T21:06:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034d1700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 --2- 172.21.15.26:0/2310714185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 -- 172.21.15.26:0/2310714185 >> 172.21.15.26:0/2310714185 conn(0x55ff033f7240 msgr2=0x55ff034c63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 -- 172.21.15.26:0/2310714185 shutdown_connections 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 -- 172.21.15.26:0/2310714185 wait complete. 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.337+0000 7f657e116b80 1 -- start start 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 Processor -- start 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ff0331ffa0 con 0x55ff034cb8a0 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ff03381da0 con 0x55ff034d1320 2024-05-20T21:06:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff0341d950 con 0x55ff034cc260 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657e116b80 10 client.0 _ll_register_callbacks cb 0x55ff033b4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657161c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657161c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034d7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34078/0 (socket says 172.21.15.26:34078) 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f657161c700 1 -- 172.21.15.26:0/1602770071 learned_addr learned my addr 172.21.15.26:0/1602770071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f6570e1b700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.338+0000 7f6571e1d700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657161c700 1 -- 172.21.15.26:0/1602770071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 msgr2=0x55ff034d5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657161c700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034d5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657161c700 1 -- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 msgr2=0x55ff034d5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657161c700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034d5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657161c700 1 -- 172.21.15.26:0/1602770071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff033698d0 con 0x55ff034cb8a0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657161c700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034d7d30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f6568009750 tx=0x7f65680aa930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8c4131a3eb052 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f6554ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f6554ff9700 1 -- 172.21.15.26:0/1602770071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680c9060 con 0x55ff034cb8a0 2024-05-20T21:06:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f6554ff9700 1 -- 172.21.15.26:0/1602770071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f65680a66f0 con 0x55ff034cb8a0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65680ae4b0 con 0x55ff034cb8a0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f6554ff9700 1 -- 172.21.15.26:0/1602770071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65680c3030 con 0x55ff034cb8a0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657e116b80 10 client.18591 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff033d3760 con 0x55ff034cb8a0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.339+0000 7f6554ff9700 1 -- 172.21.15.26:0/1602770071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65680ae4b0 con 0x55ff034cb8a0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f6554ff9700 1 -- 172.21.15.26:0/1602770071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f65680a5a70 con 0x55ff034cb8a0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f657e116b80 10 client.18591 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f657e116b80 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f657e116b80 20 client.18591 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f657e116b80 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff034e4da0 0x55ff034e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f657e116b80 4 client.18591 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0342bcc0 con 0x55ff034e4da0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f6570e1b700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff034e4da0 0x55ff034e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f6554ff9700 10 client.18591 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.340+0000 7f6570e1b700 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff034e4da0 0x55ff034e7190 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9373684dadbe31b1 server_cookie=2a4da4b9f5c00b0 in_seq=0 out_seq=0 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f6554ff9700 1 -- 172.21.15.26:0/1602770071 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ff0342bcc0 con 0x55ff034e4da0 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f6554ff9700 10 client.18591 handle_command_reply: tid=0 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 client.18591 shutdown 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 msgr2=0x55ff034d7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034d7d30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f6568009750 tx=0x7f65680aa930 comp rx=0 tx=0).stop 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 shutdown_connections 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff034e4da0 0x55ff034e7190 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034d7d30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f6554ff9700 0 client.18591 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 --2- 172.21.15.26:0/1602770071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034d5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 >> 172.21.15.26:0/1602770071 conn(0x55ff033f7240 msgr2=0x55ff034c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 shutdown_connections 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.341+0000 7f657e116b80 1 -- 172.21.15.26:0/1602770071 wait complete. 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.342+0000 7f657e116b80 20 client.18591 trim_cache size 0 max 16384 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.358+0000 7f657e116b80 1 Processor -- start 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 -- start start 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ae310 con 0x55ff034cc260 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1abba0 con 0x55ff034cb8a0 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657e116b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f656c19da50 con 0x55ff034d1320 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657161c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34080/0 (socket says 172.21.15.26:34080) 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f657161c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034e1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:52934/0 (socket says 172.21.15.26:52934) 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 -- 172.21.15.26:0/744013030 learned_addr learned my addr 172.21.15.26:0/744013030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6571e1d700 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 -- 172.21.15.26:0/744013030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 msgr2=0x55ff034cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 -- 172.21.15.26:0/744013030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 msgr2=0x55ff034e1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034e1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 -- 172.21.15.26:0/744013030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff034d35a0 con 0x55ff034cc260 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6571e1d700 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:35.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.359+0000 7f6570e1b700 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034c6270 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f655c009b70 tx=0x7f655c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72d9d82aacd0e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f6556ffd700 1 -- 172.21.15.26:0/744013030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c035030 con 0x55ff034cc260 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f6556ffd700 1 -- 172.21.15.26:0/744013030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c016070 con 0x55ff034cc260 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f6556ffd700 1 -- 172.21.15.26:0/744013030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0234e0 con 0x55ff034cc260 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 -- 172.21.15.26:0/744013030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 msgr2=0x55ff034c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034c6270 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f655c009b70 tx=0x7f655c014000 comp rx=0 tx=0).stop 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 -- 172.21.15.26:0/744013030 shutdown_connections 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034d1320 0x55ff034cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034cb8a0 0x55ff034e1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 --2- 172.21.15.26:0/744013030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cc260 0x55ff034c6270 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 -- 172.21.15.26:0/744013030 >> 172.21.15.26:0/744013030 conn(0x55ff033f7240 msgr2=0x55ff034ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.360+0000 7f657e116b80 1 -- 172.21.15.26:0/744013030 shutdown_connections 2024-05-20T21:06:35.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 -- 172.21.15.26:0/744013030 wait complete. 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 -- start start 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 Processor -- start 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1aa060 con 0x55ff034cb8a0 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ab4d0 con 0x55ff034d1320 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1aa730 con 0x55ff034cc260 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.361+0000 7f657e116b80 10 client.0 _ll_register_callbacks cb 0x55ff033b4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034ec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34088/0 (socket says 172.21.15.26:34088) 2024-05-20T21:06:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 -- 172.21.15.26:0/918229082 learned_addr learned my addr 172.21.15.26:0/918229082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f6570e1b700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f6571e1d700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 -- 172.21.15.26:0/918229082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 msgr2=0x55ff034ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034ecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 -- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 msgr2=0x55ff034ed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034ed140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 -- 172.21.15.26:0/918229082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff034dbcc0 con 0x55ff034cb8a0 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657161c700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034ec6c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f65680a99e0 tx=0x7f65680a92b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71ed9e6386eb928 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f6554ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f6554ff9700 1 -- 172.21.15.26:0/918229082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680c9030 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c016070 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f6554ff9700 1 -- 172.21.15.26:0/918229082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f65680a66f0 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f6554ff9700 1 -- 172.21.15.26:0/918229082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65680b2a40 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.362+0000 7f657e116b80 10 client.18597 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff034dbcc0 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f6554ff9700 1 -- 172.21.15.26:0/918229082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c016070 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f6554ff9700 1 -- 172.21.15.26:0/918229082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f65680e0a10 con 0x55ff034cb8a0 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 10 client.18597 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 10 client.18597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 20 client.18597 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff03550020 0x55ff03552440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 4 client.18597 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff034254a0 con 0x55ff03550020 2024-05-20T21:06:35.401 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.363+0000 7f6570e1b700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff03550020 0x55ff03552440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f6570e1b700 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff03550020 0x55ff03552440 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37973dfaf67017e0 server_cookie=4ebb8964b9857be8 in_seq=0 out_seq=0 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f6554ff9700 10 client.18597 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f6554ff9700 1 -- 172.21.15.26:0/918229082 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff034254a0 con 0x55ff03550020 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f6554ff9700 10 client.18597 handle_command_reply: tid=0 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 client.18597 shutdown 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 msgr2=0x55ff034ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034ec6c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f65680a99e0 tx=0x7f65680a92b0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 shutdown_connections 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff03550020 0x55ff03552440 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff034cb8a0 0x55ff034ec6c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff034d1320 0x55ff034ed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 --2- 172.21.15.26:0/918229082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff034cc260 0x55ff034ecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 >> 172.21.15.26:0/918229082 conn(0x55ff033f7240 msgr2=0x55ff034d5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f6554ff9700 0 client.18597 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 shutdown_connections 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 1 -- 172.21.15.26:0/918229082 wait complete. 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.364+0000 7f657e116b80 20 client.18597 trim_cache size 0 max 16384 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f65580b2980 msgr2=0x7f65580b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f65580b2980 0x7f65580b4e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f656c1031f0 tx=0x7f655c009260 comp rx=0 tx=0).stop 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 msgr2=0x7f656c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c075840 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6568006010 tx=0x7f65680055f0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 shutdown_connections 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f65580b2980 0x7f65580b4e40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c1037f0 0x7f656c075840 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c1088f0 0x7f656c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 --2- 172.21.15.26:0/3350927638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f656c1041a0 0x7f656c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:35.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 >> 172.21.15.26:0/3350927638 conn(0x7f656c07a6a0 msgr2=0x7f656c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:35.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 shutdown_connections 2024-05-20T21:06:35.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:35.365+0000 7f6573880700 1 -- 172.21.15.26:0/3350927638 wait complete. 2024-05-20T21:06:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:35 smithi055 ceph-mon[35213]: pgmap v703: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-20T21:06:35.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 220245 2.08 MB/sec execute 401 sec latency 28487.133 ms 2024-05-20T21:06:36.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 220713 2.08 MB/sec execute 402 sec latency 26.421 ms 2024-05-20T21:06:37.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a7dfde6b', total seconds elapsed: 40.0 2024-05-20T21:06:37.280 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:06:37.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:06:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:37 smithi144 ceph-mon[35660]: pgmap v704: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-20T21:06:37.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:37 smithi026 ceph-mon[31990]: pgmap v704: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-20T21:06:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:37 smithi055 ceph-mon[35213]: pgmap v704: 97 pgs: 97 active+clean; 28 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-20T21:06:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:37.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 221413 2.08 MB/sec execute 403 sec latency 14.378 ms 2024-05-20T21:06:38.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 222065 2.08 MB/sec execute 404 sec latency 12.493 ms 2024-05-20T21:06:39.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:39 smithi144 ceph-mon[35660]: pgmap v705: 97 pgs: 97 active+clean; 28 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-20T21:06:39.371 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:39 smithi026 ceph-mon[31990]: pgmap v705: 97 pgs: 97 active+clean; 28 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-20T21:06:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:39 smithi055 ceph-mon[35213]: pgmap v705: 97 pgs: 97 active+clean; 28 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-20T21:06:39.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.494+0000 7f167a4a5700 1 -- 172.21.15.26:0/2745281249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102e00 msgr2=0x7f167410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.494+0000 7f167a4a5700 1 --2- 172.21.15.26:0/2745281249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102e00 0x7f167410f2f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b0a0 tx=0x7f166c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 -- 172.21.15.26:0/2745281249 shutdown_connections 2024-05-20T21:06:39.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 --2- 172.21.15.26:0/2745281249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102e00 0x7f167410f2f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 --2- 172.21.15.26:0/2745281249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674108520 0x7f1674108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 --2- 172.21.15.26:0/2745281249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674102440 0x7f16741028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 -- 172.21.15.26:0/2745281249 >> 172.21.15.26:0/2745281249 conn(0x7f16740793f0 msgr2=0x7f1674079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 -- 172.21.15.26:0/2745281249 shutdown_connections 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 -- 172.21.15.26:0/2745281249 wait complete. 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 Processor -- start 2024-05-20T21:06:39.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.495+0000 7f167a4a5700 1 -- start start 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f167a4a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 0x7f167419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f167a4a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674102e00 0x7f167419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f167a4a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674108520 0x7f16741998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f167a4a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1674112480 con 0x7f1674102440 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f167a4a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1674112300 con 0x7f1674102e00 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f167a4a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1674112600 con 0x7f1674108520 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 0x7f167419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 0x7f167419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34110/0 (socket says 172.21.15.26:34110) 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 -- 172.21.15.26:0/4045053837 learned_addr learned my addr 172.21.15.26:0/4045053837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f16737fe700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674102e00 0x7f167419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1678a42700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674108520 0x7f16741998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 -- 172.21.15.26:0/4045053837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674108520 msgr2=0x7f16741998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674108520 0x7f16741998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 -- 172.21.15.26:0/4045053837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674102e00 msgr2=0x7f167419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674102e00 0x7f167419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 -- 172.21.15.26:0/4045053837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167419a040 con 0x7f1674102440 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f16737fe700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674102e00 0x7f167419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.496+0000 7f1673fff700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 0x7f167419f720 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f166400e9a0 tx=0x7f166400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.497+0000 7f16717fa700 1 -- 172.21.15.26:0/4045053837 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166400cbd0 con 0x7f1674102440 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.497+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167419a330 con 0x7f1674102440 2024-05-20T21:06:39.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.497+0000 7f16717fa700 1 -- 172.21.15.26:0/4045053837 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f166400cd70 con 0x7f1674102440 2024-05-20T21:06:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.497+0000 7f16717fa700 1 -- 172.21.15.26:0/4045053837 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664018680 con 0x7f1674102440 2024-05-20T21:06:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.497+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16741a65f0 con 0x7f1674102440 2024-05-20T21:06:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.499+0000 7f16717fa700 1 -- 172.21.15.26:0/4045053837 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1664010430 con 0x7f1674102440 2024-05-20T21:06:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.500+0000 7f16717fa700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f165c0b2900 0x7f165c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.500+0000 7f16737fe700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f165c0b2900 0x7f165c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.500+0000 7f16717fa700 1 -- 172.21.15.26:0/4045053837 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f16640e23e0 con 0x7f1674102440 2024-05-20T21:06:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 Processor -- start 2024-05-20T21:06:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f16737fe700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f165c0b2900 0x7f165c0b4dc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1674079bb0 tx=0x7f166800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 -- start start 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506f1fe0 0x5653506f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56535064cd10 con 0x5653506ed8e0 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56535063c8b0 con 0x5653506ee290 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653506408f0 con 0x5653506f1fe0 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.501+0000 7f16737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34120/0 (socket says 172.21.15.26:34120) 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f1673fff700 1 -- 172.21.15.26:0/1848473482 learned_addr learned my addr 172.21.15.26:0/1848473482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f1678a42700 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506f1fe0 0x5653506f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f16737fe700 1 -- 172.21.15.26:0/1848473482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506f1fe0 msgr2=0x5653506f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f16737fe700 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506f1fe0 0x5653506f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f16737fe700 1 -- 172.21.15.26:0/1848473482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 msgr2=0x5653506edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f16737fe700 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f16737fe700 1 -- 172.21.15.26:0/1848473482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535058c950 con 0x5653506ee290 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f16737fe700 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f19c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1668010530 tx=0x7f1668012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e4f3c83cf0d93a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f165a7fc700 1 -- 172.21.15.26:0/1848473482 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668016070 con 0x5653506ee290 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f165a7fc700 1 -- 172.21.15.26:0/1848473482 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1668010c30 con 0x5653506ee290 2024-05-20T21:06:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.502+0000 7f165a7fc700 1 -- 172.21.15.26:0/1848473482 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166801c680 con 0x5653506ee290 2024-05-20T21:06:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 -- 172.21.15.26:0/1848473482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 msgr2=0x5653506f19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f19c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1668010530 tx=0x7f1668012000 comp rx=0 tx=0).stop 2024-05-20T21:06:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 -- 172.21.15.26:0/1848473482 shutdown_connections 2024-05-20T21:06:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506f1fe0 0x5653506f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/1848473482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f19c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 -- 172.21.15.26:0/1848473482 >> 172.21.15.26:0/1848473482 conn(0x56535061b100 msgr2=0x5653506e7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 -- 172.21.15.26:0/1848473482 shutdown_connections 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.503+0000 7f1684d3bb80 1 -- 172.21.15.26:0/1848473482 wait complete. 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 -- start start 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 Processor -- start 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 0x5653506f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565350543020 con 0x5653506f1fe0 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5653505de840 con 0x5653506ee290 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653506fa1a0 con 0x5653506ed8e0 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1684d3bb80 10 client.0 _ll_register_callbacks cb 0x5653505d79a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1678a42700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 0x5653506f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f16737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1678a42700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 0x5653506f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34124/0 (socket says 172.21.15.26:34124) 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.504+0000 7f1678a42700 1 -- 172.21.15.26:0/3439206066 learned_addr learned my addr 172.21.15.26:0/3439206066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1673fff700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1678a42700 1 -- 172.21.15.26:0/3439206066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 msgr2=0x5653506f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1678a42700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1678a42700 1 -- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 msgr2=0x5653506f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1678a42700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1678a42700 1 -- 172.21.15.26:0/3439206066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653505db710 con 0x5653506f1fe0 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f16737fe700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1678a42700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 0x5653506f6ce0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b780 tx=0x7f166c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c732901074f6a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f16537fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f16537fe700 1 -- 172.21.15.26:0/3439206066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c004e60 con 0x5653506f1fe0 2024-05-20T21:06:39.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f16537fe700 1 -- 172.21.15.26:0/3439206066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f166c0074e0 con 0x5653506f1fe0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1668010c30 con 0x5653506f1fe0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f16537fe700 1 -- 172.21.15.26:0/3439206066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f166c0076e0 con 0x5653506f1fe0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1684d3bb80 10 client.18609 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.505+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565350704970 con 0x5653506f1fe0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f16537fe700 1 -- 172.21.15.26:0/3439206066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1668010c30 con 0x5653506f1fe0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f16537fe700 1 -- 172.21.15.26:0/3439206066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f166c011070 con 0x5653506f1fe0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1684d3bb80 10 client.18609 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1684d3bb80 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1684d3bb80 20 client.18609 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350709390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1684d3bb80 4 client.18609 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653506b9a60 con 0x565350706fa0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f1673fff700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350709390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.506+0000 7f16537fe700 10 client.18609 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.507+0000 7f1673fff700 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350709390 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db61a1385fbbf55a server_cookie=e10b7758eb66820e in_seq=0 out_seq=0 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.507+0000 7f16537fe700 1 -- 172.21.15.26:0/3439206066 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5653506b9a60 con 0x565350706fa0 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.507+0000 7f16537fe700 10 client.18609 handle_command_reply: tid=0 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.507+0000 7f1684d3bb80 1 client.18609 shutdown 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 msgr2=0x5653506f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 0x5653506f6ce0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b780 tx=0x7f166c00bd50 comp rx=0 tx=0).stop 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 shutdown_connections 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350709390 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506f1fe0 0x5653506f6ce0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f16537fe700 0 client.18609 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/3439206066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506ee290 0x5653506f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 >> 172.21.15.26:0/3439206066 conn(0x56535061b100 msgr2=0x5653506f33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:39.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 shutdown_connections 2024-05-20T21:06:39.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 1 -- 172.21.15.26:0/3439206066 wait complete. 2024-05-20T21:06:39.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.508+0000 7f1684d3bb80 20 client.18609 trim_cache size 0 max 16384 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 Processor -- start 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 -- start start 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 0x5653506f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x56535070ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f16741b0510 con 0x5653506ee290 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f16741adda0 con 0x5653506f1fe0 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16741abbf0 con 0x5653506ed8e0 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f16737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 0x5653506f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f16737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 0x5653506f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34138/0 (socket says 172.21.15.26:34138) 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f16737fe700 1 -- 172.21.15.26:0/608562864 learned_addr learned my addr 172.21.15.26:0/608562864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1678a42700 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x56535070ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.532+0000 7f1673fff700 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f16737fe700 1 -- 172.21.15.26:0/608562864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 msgr2=0x5653506f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f16737fe700 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f16737fe700 1 -- 172.21.15.26:0/608562864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 msgr2=0x56535070ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f16737fe700 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x56535070ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f16737fe700 1 -- 172.21.15.26:0/608562864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653506e8560 con 0x5653506ee290 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f1678a42700 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x56535070ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f16737fe700 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 0x5653506f86e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f166800aae0 tx=0x7f1668005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5b41a26ef7e2b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f1659ffb700 1 -- 172.21.15.26:0/608562864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668010a30 con 0x5653506ee290 2024-05-20T21:06:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f1659ffb700 1 -- 172.21.15.26:0/608562864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1668010bd0 con 0x5653506ee290 2024-05-20T21:06:39.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.533+0000 7f1659ffb700 1 -- 172.21.15.26:0/608562864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16680256d0 con 0x5653506ee290 2024-05-20T21:06:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 -- 172.21.15.26:0/608562864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 msgr2=0x5653506f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 0x5653506f86e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f166800aae0 tx=0x7f1668005600 comp rx=0 tx=0).stop 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 -- 172.21.15.26:0/608562864 shutdown_connections 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x56535070ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ed8e0 0x5653506f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/608562864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ee290 0x5653506f86e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 -- 172.21.15.26:0/608562864 >> 172.21.15.26:0/608562864 conn(0x56535061b100 msgr2=0x5653506f7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 -- 172.21.15.26:0/608562864 shutdown_connections 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 -- 172.21.15.26:0/608562864 wait complete. 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.534+0000 7f1684d3bb80 1 -- start start 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 Processor -- start 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506fc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ee290 0x5653506fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x5653506fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f16741ac260 con 0x5653506ed8e0 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f16741ad6d0 con 0x5653506f1fe0 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16741ac930 con 0x5653506ee290 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1684d3bb80 10 client.0 _ll_register_callbacks cb 0x5653505d79a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f16737fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ee290 0x5653506fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506fc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34140/0 (socket says 172.21.15.26:34140) 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1673fff700 1 -- 172.21.15.26:0/896448901 learned_addr learned my addr 172.21.15.26:0/896448901 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.535+0000 7f1678a42700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x5653506fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1673fff700 1 -- 172.21.15.26:0/896448901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ee290 msgr2=0x5653506fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1673fff700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ee290 0x5653506fc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1673fff700 1 -- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 msgr2=0x5653506fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1673fff700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x5653506fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1673fff700 1 -- 172.21.15.26:0/896448901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653507761d0 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f16737fe700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ee290 0x5653506fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1673fff700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506fc400 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f16640dfed0 tx=0x7f1664091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628e6244d97322a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f16537fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f16537fe700 1 -- 172.21.15.26:0/896448901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640c3070 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f16537fe700 1 -- 172.21.15.26:0/896448901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f16640a9c00 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1668010bd0 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f16537fe700 1 -- 172.21.15.26:0/896448901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16640a94b0 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1684d3bb80 10 client.18615 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565350777530 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.536+0000 7f16537fe700 1 -- 172.21.15.26:0/896448901 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1668010bd0 con 0x5653506ed8e0 2024-05-20T21:06:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f16537fe700 1 -- 172.21.15.26:0/896448901 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f16640b2370 con 0x5653506ed8e0 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f1684d3bb80 10 client.18615 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f1684d3bb80 10 client.18615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f1684d3bb80 20 client.18615 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350779c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f1684d3bb80 4 client.18615 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56535064ec60 con 0x565350706fa0 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f16737fe700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350779c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f16537fe700 10 client.18615 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.537+0000 7f16737fe700 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350779c20 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e74c23d0d25ff2c server_cookie=430a502b280278cb in_seq=0 out_seq=0 2024-05-20T21:06:39.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f16537fe700 1 -- 172.21.15.26:0/896448901 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56535064ec60 con 0x565350706fa0 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f16537fe700 10 client.18615 handle_command_reply: tid=0 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 client.18615 shutdown 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 msgr2=0x5653506fc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506fc400 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f16640dfed0 tx=0x7f1664091040 comp rx=0 tx=0).stop 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 shutdown_connections 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x565350706fa0 0x565350779c20 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653506ed8e0 0x5653506fc400 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653506f1fe0 0x5653506fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 --2- 172.21.15.26:0/896448901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653506ee290 0x5653506fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.538+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 >> 172.21.15.26:0/896448901 conn(0x56535061b100 msgr2=0x5653506f1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f16537fe700 0 client.18615 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 shutdown_connections 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f1684d3bb80 1 -- 172.21.15.26:0/896448901 wait complete. 2024-05-20T21:06:39.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f1684d3bb80 20 client.18615 trim_cache size 0 max 16384 2024-05-20T21:06:39.544 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:06:39.544 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:06:39.544 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "8ab7e103-3393-410d-8977-7fb79295f8fa", 2024-05-20T21:06:39.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:06:39.544 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:06:39.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f165c0b2900 msgr2=0x7f165c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f167a4a5700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f165c0b2900 0x7f165c0b4dc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1674079bb0 tx=0x7f166800a300 comp rx=0 tx=0).stop 2024-05-20T21:06:39.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 msgr2=0x7f167419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.539+0000 7f167a4a5700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 0x7f167419f720 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f166400e9a0 tx=0x7f166400ecb0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 shutdown_connections 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f165c0b2900 0x7f165c0b4dc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1674102440 0x7f167419f720 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674108520 0x7f16741998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 --2- 172.21.15.26:0/4045053837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1674102e00 0x7f167419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 >> 172.21.15.26:0/4045053837 conn(0x7f16740793f0 msgr2=0x7f167406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 shutdown_connections 2024-05-20T21:06:39.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:39.540+0000 7f167a4a5700 1 -- 172.21.15.26:0/4045053837 wait complete. 2024-05-20T21:06:39.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 222722 2.09 MB/sec execute 405 sec latency 39.464 ms 2024-05-20T21:06:40.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:06:40.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 223359 2.09 MB/sec execute 406 sec latency 10.899 ms 2024-05-20T21:06:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:41.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:41 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:06:41.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:41 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:41.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:41 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:06:41.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:41 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:41.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:41 smithi026 ceph-mon[31990]: pgmap v706: 97 pgs: 97 active+clean; 28 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 164 KiB/s wr, 17 op/s 2024-05-20T21:06:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:41 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:06:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:41 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:41 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:06:41.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:41 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:41.301 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:41 smithi144 ceph-mon[35660]: pgmap v706: 97 pgs: 97 active+clean; 28 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 164 KiB/s wr, 17 op/s 2024-05-20T21:06:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:41 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:06:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:41 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:41 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:06:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:41 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:06:41.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:41 smithi055 ceph-mon[35213]: pgmap v706: 97 pgs: 97 active+clean; 28 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 164 KiB/s wr, 17 op/s 2024-05-20T21:06:41.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 223931 2.10 MB/sec execute 407 sec latency 16.638 ms 2024-05-20T21:06:42.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:42 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:06:42.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:42 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:06:42.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:42 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:06:42.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 224465 2.09 MB/sec execute 408 sec latency 22.646 ms 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 -- 172.21.15.26:0/2424309728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406b250 msgr2=0x7f339406b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2424309728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406b250 0x7f339406b630 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3384009990 tx=0x7f3384009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 -- 172.21.15.26:0/2424309728 shutdown_connections 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2424309728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406b250 0x7f339406b630 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2424309728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339410fff0 0x7f33941123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2424309728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339406bb70 0x7f339410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 -- 172.21.15.26:0/2424309728 >> 172.21.15.26:0/2424309728 conn(0x7f33940793f0 msgr2=0x7f3394079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 -- 172.21.15.26:0/2424309728 shutdown_connections 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.673+0000 7f3399ed5700 1 -- 172.21.15.26:0/2424309728 wait complete. 2024-05-20T21:06:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.674+0000 7f3399ed5700 1 Processor -- start 2024-05-20T21:06:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.674+0000 7f3399ed5700 1 -- start start 2024-05-20T21:06:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.674+0000 7f3399ed5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 0x7f339419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.674+0000 7f3399ed5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 0x7f339419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3399ed5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339410fff0 0x7f33941a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3399ed5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3394115820 con 0x7f339406bb70 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3399ed5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f33941156a0 con 0x7f339406b250 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3399ed5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33941159a0 con 0x7f339410fff0 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 0x7f339419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 0x7f339419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 0x7f339419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40674/0 (socket says 172.21.15.26:40674) 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 -- 172.21.15.26:0/2622945038 learned_addr learned my addr 172.21.15.26:0/2622945038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 0x7f339419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39132/0 (socket says 172.21.15.26:39132) 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3393fff700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339410fff0 0x7f33941a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 -- 172.21.15.26:0/2622945038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339410fff0 msgr2=0x7f33941a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339410fff0 0x7f33941a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 -- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 msgr2=0x7f339419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 0x7f339419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 -- 172.21.15.26:0/2622945038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33941a0dc0 con 0x7f339406b250 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3393fff700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339410fff0 0x7f33941a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3392ffd700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 0x7f339419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f33937fe700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 0x7f339419bdb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3384006010 tx=0x7f338400bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.675+0000 7f3390ff9700 1 -- 172.21.15.26:0/2622945038 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384027070 con 0x7f339406b250 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.676+0000 7f3390ff9700 1 -- 172.21.15.26:0/2622945038 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3384022470 con 0x7f339406b250 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.676+0000 7f3390ff9700 1 -- 172.21.15.26:0/2622945038 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384017650 con 0x7f339406b250 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.676+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33941a1050 con 0x7f339406b250 2024-05-20T21:06:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.676+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33941a8930 con 0x7f339406b250 2024-05-20T21:06:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.678+0000 7f3390ff9700 1 -- 172.21.15.26:0/2622945038 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3384009d10 con 0x7f339406b250 2024-05-20T21:06:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.679+0000 7f3390ff9700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f33800b2880 0x7f33800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.679+0000 7f3392ffd700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f33800b2880 0x7f33800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.679+0000 7f3390ff9700 1 -- 172.21.15.26:0/2622945038 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f33840e2e90 con 0x7f339406b250 2024-05-20T21:06:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f3392ffd700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f33800b2880 0x7f33800b4d40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3394103ac0 tx=0x7f337c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 Processor -- start 2024-05-20T21:06:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 -- start start 2024-05-20T21:06:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 0x562d4eb65c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb69940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb6a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d4eac4c90 con 0x562d4eb69f60 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562d4eab4830 con 0x562d4eb65860 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33a476bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d4eab8870 con 0x562d4eb66210 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 0x562d4eb65c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f3393fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb6a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 0x562d4eb65c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40676/0 (socket says 172.21.15.26:40676) 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb69940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.680+0000 7f33937fe700 1 -- 172.21.15.26:0/1993205547 learned_addr learned my addr 172.21.15.26:0/1993205547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:42.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3392ffd700 1 -- 172.21.15.26:0/1993205547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 msgr2=0x562d4eb65c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3392ffd700 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 0x562d4eb65c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3392ffd700 1 -- 172.21.15.26:0/1993205547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 msgr2=0x562d4eb6a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3392ffd700 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb6a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3392ffd700 1 -- 172.21.15.26:0/1993205547 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4ea048d0 con 0x562d4eb66210 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f33937fe700 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 0x562d4eb65c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3393fff700 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb6a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3392ffd700 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb69940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f337c00ed20 tx=0x7f337c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5b7ecce256f6ab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3389ffb700 1 -- 172.21.15.26:0/1993205547 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c01b040 con 0x562d4eb66210 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3389ffb700 1 -- 172.21.15.26:0/1993205547 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337c024070 con 0x562d4eb66210 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.681+0000 7f3389ffb700 1 -- 172.21.15.26:0/1993205547 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c013980 con 0x562d4eb66210 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 -- 172.21.15.26:0/1993205547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 msgr2=0x562d4eb69940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb69940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f337c00ed20 tx=0x7f337c011000 comp rx=0 tx=0).stop 2024-05-20T21:06:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 -- 172.21.15.26:0/1993205547 shutdown_connections 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb6a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb65860 0x562d4eb65c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1993205547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb69940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 -- 172.21.15.26:0/1993205547 >> 172.21.15.26:0/1993205547 conn(0x562d4ea93080 msgr2=0x562d4eb5fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 -- 172.21.15.26:0/1993205547 shutdown_connections 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.682+0000 7f33a476bb80 1 -- 172.21.15.26:0/1993205547 wait complete. 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 -- start start 2024-05-20T21:06:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 Processor -- start 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 0x562d4eb71370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb6e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb69f60 0x562d4eb6ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d4e9bafa0 con 0x562d4eb65860 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562d4ea567c0 con 0x562d4eb69f60 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d4eb72120 con 0x562d4eb66210 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33a476bb80 10 client.0 _ll_register_callbacks cb 0x562d4ea4f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 0x562d4eb71370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 0x562d4eb71370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39150/0 (socket says 172.21.15.26:39150) 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.683+0000 7f33937fe700 1 -- 172.21.15.26:0/4264492838 learned_addr learned my addr 172.21.15.26:0/4264492838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f3393fff700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb69f60 0x562d4eb6ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f3392ffd700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb6e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33937fe700 1 -- 172.21.15.26:0/4264492838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 msgr2=0x562d4eb6e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33937fe700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb6e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33937fe700 1 -- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb69f60 msgr2=0x562d4eb6ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33937fe700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb69f60 0x562d4eb6ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33937fe700 1 -- 172.21.15.26:0/4264492838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4ea53690 con 0x562d4eb65860 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33937fe700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 0x562d4eb71370 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f33840aa3c0 tx=0x7f3384022c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a39db208fdfe2c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f3372ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f3372ffd700 1 -- 172.21.15.26:0/4264492838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33840b3250 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f3372ffd700 1 -- 172.21.15.26:0/4264492838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f33840b2fd0 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337c024070 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f3372ffd700 1 -- 172.21.15.26:0/4264492838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33840a4ba0 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33a476bb80 10 client.18627 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.684+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562d4eb7c7e0 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f3372ffd700 1 -- 172.21.15.26:0/4264492838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337c024070 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f3372ffd700 1 -- 172.21.15.26:0/4264492838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f33840a5df0 con 0x562d4eb65860 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f33a476bb80 10 client.18627 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f33a476bb80 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f33a476bb80 20 client.18627 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f33a476bb80 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4eb7eda0 0x562d4eb81190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f33a476bb80 4 client.18627 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d4eb319e0 con 0x562d4eb7eda0 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.685+0000 7f3392ffd700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4eb7eda0 0x562d4eb81190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.686+0000 7f3372ffd700 10 client.18627 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.686+0000 7f3392ffd700 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4eb7eda0 0x562d4eb81190 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88903321440e0a4 server_cookie=646f218a664aff27 in_seq=0 out_seq=0 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.686+0000 7f3372ffd700 1 -- 172.21.15.26:0/4264492838 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562d4eb319e0 con 0x562d4eb7eda0 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.686+0000 7f3372ffd700 10 client.18627 handle_command_reply: tid=0 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 client.18627 shutdown 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 msgr2=0x562d4eb71370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 0x562d4eb71370 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f33840aa3c0 tx=0x7f3384022c00 comp rx=0 tx=0).stop 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 shutdown_connections 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4eb7eda0 0x562d4eb81190 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb65860 0x562d4eb71370 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb69f60 0x562d4eb6ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 --2- 172.21.15.26:0/4264492838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb6e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 >> 172.21.15.26:0/4264492838 conn(0x562d4ea93080 msgr2=0x562d4eb5f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f3372ffd700 0 client.18627 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:42.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 shutdown_connections 2024-05-20T21:06:42.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 1 -- 172.21.15.26:0/4264492838 wait complete. 2024-05-20T21:06:42.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.687+0000 7f33a476bb80 20 client.18627 trim_cache size 0 max 16384 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 Processor -- start 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 -- start start 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb64d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb86aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb86fe0 0x562d4eb5fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33941b2730 con 0x562d4eb69f60 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f33941affc0 con 0x562d4eb86fe0 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33a476bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33941ade10 con 0x562d4eb66210 2024-05-20T21:06:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb86aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb86aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39166/0 (socket says 172.21.15.26:39166) 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f3392ffd700 1 -- 172.21.15.26:0/1150773432 learned_addr learned my addr 172.21.15.26:0/1150773432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f3393fff700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb86fe0 0x562d4eb5fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.711+0000 7f33937fe700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb64d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3392ffd700 1 -- 172.21.15.26:0/1150773432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 msgr2=0x562d4eb64d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3392ffd700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb64d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3392ffd700 1 -- 172.21.15.26:0/1150773432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb86fe0 msgr2=0x562d4eb5fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3392ffd700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb86fe0 0x562d4eb5fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3392ffd700 1 -- 172.21.15.26:0/1150773432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4eb81810 con 0x562d4eb69f60 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f33937fe700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb64d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3393fff700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb86fe0 0x562d4eb5fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f3392ffd700 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb86aa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f337c00e840 tx=0x7f337c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafa0ed441e27808 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f33897fa700 1 -- 172.21.15.26:0/1150773432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c015b90 con 0x562d4eb69f60 2024-05-20T21:06:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f33897fa700 1 -- 172.21.15.26:0/1150773432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337c024040 con 0x562d4eb69f60 2024-05-20T21:06:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.712+0000 7f33897fa700 1 -- 172.21.15.26:0/1150773432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c013e80 con 0x562d4eb69f60 2024-05-20T21:06:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 -- 172.21.15.26:0/1150773432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 msgr2=0x562d4eb86aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb86aa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f337c00e840 tx=0x7f337c015000 comp rx=0 tx=0).stop 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 -- 172.21.15.26:0/1150773432 shutdown_connections 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb86fe0 0x562d4eb5fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb66210 0x562d4eb64d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 --2- 172.21.15.26:0/1150773432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb86aa0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 -- 172.21.15.26:0/1150773432 >> 172.21.15.26:0/1150773432 conn(0x562d4ea93080 msgr2=0x562d4eb64670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 -- 172.21.15.26:0/1150773432 shutdown_connections 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 -- 172.21.15.26:0/1150773432 wait complete. 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.713+0000 7f33a476bb80 1 -- start start 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 Processor -- start 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 0x562d4eb76180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb86fe0 0x562d4eb76c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33941ae480 con 0x562d4eb69f60 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f33941af8f0 con 0x562d4eb66210 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33941aeb50 con 0x562d4eb86fe0 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33a476bb80 10 client.0 _ll_register_callbacks cb 0x562d4ea4f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 0x562d4eb76180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f3392ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb766c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39172/0 (socket says 172.21.15.26:39172) 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 0x562d4eb76180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40702/0 (socket says 172.21.15.26:40702) 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f3392ffd700 1 -- 172.21.15.26:0/2799869730 learned_addr learned my addr 172.21.15.26:0/2799869730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f3393fff700 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb86fe0 0x562d4eb76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 -- 172.21.15.26:0/2799869730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb86fe0 msgr2=0x562d4eb76c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb86fe0 0x562d4eb76c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 -- 172.21.15.26:0/2799869730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 msgr2=0x562d4eb766c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb766c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.714+0000 7f33937fe700 1 -- 172.21.15.26:0/2799869730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4ebe9a90 con 0x562d4eb66210 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f33937fe700 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 0x562d4eb76180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f33840a6500 tx=0x7f33840b32d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d9f190d79e6593 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f3372ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f3372ffd700 1 -- 172.21.15.26:0/2799869730 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33840a96e0 con 0x562d4eb66210 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f3372ffd700 1 -- 172.21.15.26:0/2799869730 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f33840a9880 con 0x562d4eb66210 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337c024040 con 0x562d4eb66210 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f3372ffd700 1 -- 172.21.15.26:0/2799869730 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33840a9150 con 0x562d4eb66210 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f33a476bb80 10 client.27935 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562d4ebeacf0 con 0x562d4eb66210 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f3372ffd700 1 -- 172.21.15.26:0/2799869730 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33840a92f0 con 0x562d4eb66210 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.715+0000 7f3372ffd700 1 -- 172.21.15.26:0/2799869730 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f33840c4ce0 con 0x562d4eb66210 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f33a476bb80 10 client.27935 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f33a476bb80 10 client.27935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f33a476bb80 20 client.27935 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f33a476bb80 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4ebeb1b0 0x562d4ebed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f33a476bb80 4 client.27935 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f337c024040 con 0x562d4ebeb1b0 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f3392ffd700 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4ebeb1b0 0x562d4ebed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.716+0000 7f3372ffd700 10 client.27935 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f3392ffd700 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4ebeb1b0 0x562d4ebed5a0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170c3e2bed5ad603 server_cookie=8410cdbfc626cadd in_seq=0 out_seq=0 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f3372ffd700 1 -- 172.21.15.26:0/2799869730 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f337c024040 con 0x562d4ebeb1b0 2024-05-20T21:06:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f3372ffd700 10 client.27935 handle_command_reply: tid=0 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 client.27935 shutdown 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 msgr2=0x562d4eb76180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 0x562d4eb76180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f33840a6500 tx=0x7f33840b32d0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 shutdown_connections 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562d4ebeb1b0 0x562d4ebed5a0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d4eb86fe0 0x562d4eb76c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f3372ffd700 0 client.27935 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562d4eb66210 0x562d4eb76180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 --2- 172.21.15.26:0/2799869730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d4eb69f60 0x562d4eb766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 >> 172.21.15.26:0/2799869730 conn(0x562d4ea93080 msgr2=0x562d4eb64f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 shutdown_connections 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.717+0000 7f33a476bb80 1 -- 172.21.15.26:0/2799869730 wait complete. 2024-05-20T21:06:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f33a476bb80 20 client.27935 trim_cache size 0 max 16384 2024-05-20T21:06:42.724 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:06:42.724 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:06:42.724 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:06:42.724 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:06:42.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f33800b2880 msgr2=0x7f33800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f33800b2880 0x7f33800b4d40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3394103ac0 tx=0x7f337c009190 comp rx=0 tx=0).stop 2024-05-20T21:06:42.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 msgr2=0x7f339419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:42.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 0x7f339419bdb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3384006010 tx=0x7f338400bfc0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 shutdown_connections 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339410fff0 0x7f33941a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f339406b250 0x7f339419bdb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f33800b2880 0x7f33800b4d40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 --2- 172.21.15.26:0/2622945038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f339406bb70 0x7f339419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 >> 172.21.15.26:0/2622945038 conn(0x7f33940793f0 msgr2=0x7f33941115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 shutdown_connections 2024-05-20T21:06:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:42.718+0000 7f3399ed5700 1 -- 172.21.15.26:0/2622945038 wait complete. 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 -- 172.21.15.26:0/2551616621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 msgr2=0x7f4580104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 --2- 172.21.15.26:0/2551616621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f4580104150 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4570009900 tx=0x7f457002a410 comp rx=0 tx=0).stop 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 -- 172.21.15.26:0/2551616621 shutdown_connections 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 --2- 172.21.15.26:0/2551616621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4580104690 0x7f458010af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 --2- 172.21.15.26:0/2551616621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f4580104150 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 --2- 172.21.15.26:0/2551616621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 0x7f4580102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 -- 172.21.15.26:0/2551616621 >> 172.21.15.26:0/2551616621 conn(0x7f45800fe070 msgr2=0x7f4580100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 -- 172.21.15.26:0/2551616621 shutdown_connections 2024-05-20T21:06:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.164+0000 7f4587be5640 1 -- 172.21.15.26:0/2551616621 wait complete. 2024-05-20T21:06:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 Processor -- start 2024-05-20T21:06:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 -- start start 2024-05-20T21:06:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 0x7f458019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f458019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4580104690 0x7f45801a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f458010e4c0 con 0x7f4580103cd0 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f458010e340 con 0x7f4580102ae0 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4587be5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f458010e640 con 0x7f4580104690 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4585180640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f458019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4585981640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 0x7f458019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4585180640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f458019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39190/0 (socket says 172.21.15.26:39190) 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.165+0000 7f4585180640 1 -- 172.21.15.26:0/1111744435 learned_addr learned my addr 172.21.15.26:0/1111744435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4586182640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4580104690 0x7f45801a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585981640 1 -- 172.21.15.26:0/1111744435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4580104690 msgr2=0x7f45801a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585981640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4580104690 0x7f45801a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585981640 1 -- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 msgr2=0x7f458019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585981640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f458019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585981640 1 -- 172.21.15.26:0/1111744435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45801a2970 con 0x7f4580102ae0 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585180640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f458019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4585981640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 0x7f458019ada0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9c0 tx=0x7f457c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4576ffd640 1 -- 172.21.15.26:0/1111744435 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00ccc0 con 0x7f4580102ae0 2024-05-20T21:06:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45801a2c60 con 0x7f4580102ae0 2024-05-20T21:06:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4576ffd640 1 -- 172.21.15.26:0/1111744435 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f457c0044b0 con 0x7f4580102ae0 2024-05-20T21:06:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4576ffd640 1 -- 172.21.15.26:0/1111744435 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c003e10 con 0x7f4580102ae0 2024-05-20T21:06:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.166+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45801a31b0 con 0x7f4580102ae0 2024-05-20T21:06:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 Processor -- start 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 -- start start 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55916b443c30 con 0x55916b4d8220 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55916b4337d0 con 0x55916b4d30b0 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.169+0000 7f458a9d6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55916b437810 con 0x55916b4d26c0 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4576ffd640 1 -- 172.21.15.26:0/1111744435 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f457c003fd0 con 0x7f4580102ae0 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4da6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39198/0 (socket says 172.21.15.26:39198) 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 -- 172.21.15.26:0/2545912719 learned_addr learned my addr 172.21.15.26:0/2545912719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4586182640 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585981640 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 -- 172.21.15.26:0/2545912719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 msgr2=0x55916b4d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 -- 172.21.15.26:0/2545912719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 msgr2=0x55916b4d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 -- 172.21.15.26:0/2545912719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916b148210 con 0x55916b4d8220 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4586182640 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f4585180640 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4da6b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4570009a20 tx=0x7f4570031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503fb0b849d8d7bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f456bfff640 1 -- 172.21.15.26:0/2545912719 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570035030 con 0x55916b4d8220 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f456bfff640 1 -- 172.21.15.26:0/2545912719 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4570040070 con 0x55916b4d8220 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.170+0000 7f456bfff640 1 -- 172.21.15.26:0/2545912719 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570003d00 con 0x55916b4d8220 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f4576ffd640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f456c0b28a0 0x7f456c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f4586182640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f456c0b28a0 0x7f456c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f4576ffd640 1 -- 172.21.15.26:0/1111744435 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f457c0e2720 con 0x7f4580102ae0 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 -- 172.21.15.26:0/2545912719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 msgr2=0x55916b4da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4da6b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4570009a20 tx=0x7f4570031930 comp rx=0 tx=0).stop 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f4586182640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f456c0b28a0 0x7f456c0b4d60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55916b4d3910 tx=0x7f4578009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 -- 172.21.15.26:0/2545912719 shutdown_connections 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4da6b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2545912719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 -- 172.21.15.26:0/2545912719 >> 172.21.15.26:0/2545912719 conn(0x55916b4cdd30 msgr2=0x55916b4ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 -- 172.21.15.26:0/2545912719 shutdown_connections 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.171+0000 7f458a9d6740 1 -- 172.21.15.26:0/2545912719 wait complete. 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 -- start start 2024-05-20T21:06:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 Processor -- start 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4dc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55916b119790 con 0x55916b4d8220 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55916b3ef890 con 0x55916b4d30b0 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55916b4db170 con 0x55916b4d26c0 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.172+0000 7f458a9d6740 10 client.0 _ll_register_callbacks cb 0x55916b147a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39214/0 (socket says 172.21.15.26:39214) 2024-05-20T21:06:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 -- 172.21.15.26:0/3885264886 learned_addr learned my addr 172.21.15.26:0/3885264886 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4585981640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4dc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4585180640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 -- 172.21.15.26:0/3885264886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 msgr2=0x55916b4dc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4dc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 -- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 msgr2=0x55916b4dccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4dccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 -- 172.21.15.26:0/3885264886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916b214620 con 0x55916b4d8220 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f4586182640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3d10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4578009ed0 tx=0x7f4578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a230f24bf0b0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f45697fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f45697fa640 1 -- 172.21.15.26:0/3885264886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578004d20 con 0x55916b4d8220 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4570040070 con 0x55916b4d8220 2024-05-20T21:06:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f45697fa640 1 -- 172.21.15.26:0/3885264886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f457801d5d0 con 0x55916b4d8220 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f45697fa640 1 -- 172.21.15.26:0/3885264886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f457801d7d0 con 0x55916b4d8220 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f458a9d6740 10 client.18645 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.173+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55916b4e6d60 con 0x55916b4d8220 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f45697fa640 1 -- 172.21.15.26:0/3885264886 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4570040070 con 0x55916b4d8220 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f45697fa640 1 -- 172.21.15.26:0/3885264886 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f457800f040 con 0x55916b4d8220 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f458a9d6740 10 client.18645 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f458a9d6740 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f458a9d6740 20 client.18645 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f458a9d6740 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b4e8560 0x55916b4ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f458a9d6740 4 client.18645 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916b445b80 con 0x55916b4e8560 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.174+0000 7f4585981640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b4e8560 0x55916b4ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f4585981640 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b4e8560 0x55916b4ea950 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d999843e75298bc server_cookie=96ac8af5d8fe7335 in_seq=0 out_seq=0 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f45697fa640 10 client.18645 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f45697fa640 1 -- 172.21.15.26:0/3885264886 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55916b445b80 con 0x55916b4e8560 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f45697fa640 10 client.18645 handle_command_reply: tid=0 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f458a9d6740 1 client.18645 shutdown 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 msgr2=0x55916b4e3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.175+0000 7f458a9d6740 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3d10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4578009ed0 tx=0x7f4578004a40 comp rx=0 tx=0).stop 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 shutdown_connections 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b4e8560 0x55916b4ea950 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3d10 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4dccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f45697fa640 0 client.18645 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 --2- 172.21.15.26:0/3885264886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 >> 172.21.15.26:0/3885264886 conn(0x55916b4cdd30 msgr2=0x55916b4ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 shutdown_connections 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 1 -- 172.21.15.26:0/3885264886 wait complete. 2024-05-20T21:06:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.176+0000 7f458a9d6740 20 client.18645 trim_cache size 0 max 16384 2024-05-20T21:06:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 Processor -- start 2024-05-20T21:06:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 -- start start 2024-05-20T21:06:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45801ae9a0 con 0x55916b4d8220 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45801ac270 con 0x55916b4d30b0 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f458a9d6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45801aa0c0 con 0x55916b4d26c0 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f4586182640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f4586182640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39228/0 (socket says 172.21.15.26:39228) 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f4586182640 1 -- 172.21.15.26:0/2097356566 learned_addr learned my addr 172.21.15.26:0/2097356566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.196+0000 7f4585180640 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4585981640 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4586182640 1 -- 172.21.15.26:0/2097356566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 msgr2=0x55916b4d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4586182640 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4586182640 1 -- 172.21.15.26:0/2097356566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 msgr2=0x55916b4d4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4586182640 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4586182640 1 -- 172.21.15.26:0/2097356566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916b4d3940 con 0x55916b4d8220 2024-05-20T21:06:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f4586182640 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3cd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4578012dd0 tx=0x7f4578012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fffa2cf90508e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f456b7fe640 1 -- 172.21.15.26:0/2097356566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578010520 con 0x55916b4d8220 2024-05-20T21:06:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f456b7fe640 1 -- 172.21.15.26:0/2097356566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4578011040 con 0x55916b4d8220 2024-05-20T21:06:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.197+0000 7f456b7fe640 1 -- 172.21.15.26:0/2097356566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578027640 con 0x55916b4d8220 2024-05-20T21:06:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 -- 172.21.15.26:0/2097356566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 msgr2=0x55916b4e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3cd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4578012dd0 tx=0x7f4578012f00 comp rx=0 tx=0).stop 2024-05-20T21:06:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 -- 172.21.15.26:0/2097356566 shutdown_connections 2024-05-20T21:06:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d8220 0x55916b4e3cd0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2097356566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d26c0 0x55916b4d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 -- 172.21.15.26:0/2097356566 >> 172.21.15.26:0/2097356566 conn(0x55916b4cdd30 msgr2=0x55916b4ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 -- 172.21.15.26:0/2097356566 shutdown_connections 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 -- 172.21.15.26:0/2097356566 wait complete. 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 -- start start 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.198+0000 7f458a9d6740 1 Processor -- start 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 0x55916b4e11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d8220 0x55916b4e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45801aa730 con 0x55916b4d26c0 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45801abba0 con 0x55916b4d30b0 2024-05-20T21:06:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45801aae00 con 0x55916b4d8220 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 10 client.0 _ll_register_callbacks cb 0x7f4578009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 0x55916b4e11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 0x55916b4e11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39242/0 (socket says 172.21.15.26:39242) 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 -- 172.21.15.26:0/2645321815 learned_addr learned my addr 172.21.15.26:0/2645321815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585180640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4586182640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d8220 0x55916b4e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 -- 172.21.15.26:0/2645321815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d8220 msgr2=0x55916b4e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d8220 0x55916b4e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 -- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 msgr2=0x55916b4e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 -- 172.21.15.26:0/2645321815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916b4e2c90 con 0x55916b4d26c0 2024-05-20T21:06:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4586182640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d8220 0x55916b4e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f4585981640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 0x55916b4e11c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f457c0e0530 tx=0x7f457c0aa4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40140159427a370d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f45697fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f45697fa640 1 -- 172.21.15.26:0/2645321815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c0ae090 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f45697fa640 1 -- 172.21.15.26:0/2645321815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f457c0a5f70 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.199+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4578011040 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f45697fa640 1 -- 172.21.15.26:0/2645321815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f457c0b2b40 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 10 client.18651 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55916b553130 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f45697fa640 1 -- 172.21.15.26:0/2645321815 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4578011040 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f45697fa640 1 -- 172.21.15.26:0/2645321815 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f457c0b26b0 con 0x55916b4d26c0 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 10 client.18651 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 10 client.18651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 20 client.18651 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b5538f0 0x55916b555d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 4 client.18651 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:06:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55916b43f360 con 0x55916b5538f0 2024-05-20T21:06:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.200+0000 7f4585180640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b5538f0 0x55916b555d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.201+0000 7f4585180640 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b5538f0 0x55916b555d10 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddcd4216266d01 server_cookie=3fd58dddbb5460f3 in_seq=0 out_seq=0 2024-05-20T21:06:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.201+0000 7f45697fa640 10 client.18651 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.249+0000 7f45697fa640 1 -- 172.21.15.26:0/2645321815 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55916b43f360 con 0x55916b5538f0 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.249+0000 7f45697fa640 10 client.18651 handle_command_reply: tid=0 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.249+0000 7f458a9d6740 1 client.18651 shutdown 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.249+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 msgr2=0x55916b4e11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.249+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 0x55916b4e11c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f457c0e0530 tx=0x7f457c0aa4a0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 shutdown_connections 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55916b5538f0 0x55916b555d10 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f45697fa640 0 client.18651 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916b4d8220 0x55916b4e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55916b4d30b0 0x55916b4e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 --2- 172.21.15.26:0/2645321815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55916b4d26c0 0x55916b4e11c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 >> 172.21.15.26:0/2645321815 conn(0x55916b4cdd30 msgr2=0x55916b4dc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 shutdown_connections 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 1 -- 172.21.15.26:0/2645321815 wait complete. 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f458a9d6740 20 client.18651 trim_cache size 0 max 16384 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:06:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:06:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:06:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f456c0b28a0 msgr2=0x7f456c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f456c0b28a0 0x7f456c0b4d60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55916b4d3910 tx=0x7f4578009600 comp rx=0 tx=0).stop 2024-05-20T21:06:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 msgr2=0x7f458019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 0x7f458019ada0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9c0 tx=0x7f457c00ee80 comp rx=0 tx=0).stop 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 shutdown_connections 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f456c0b28a0 0x7f456c0b4d60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4580104690 0x7f45801a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4580103cd0 0x7f458019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 --2- 172.21.15.26:0/1111744435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4580102ae0 0x7f458019ada0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 >> 172.21.15.26:0/1111744435 conn(0x7f45800fe070 msgr2=0x7f4580109470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 shutdown_connections 2024-05-20T21:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:43.250+0000 7f4587be5640 1 -- 172.21.15.26:0/1111744435 wait complete. 2024-05-20T21:06:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:43 smithi055 ceph-mon[35213]: pgmap v707: 97 pgs: 97 active+clean; 31 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 610 KiB/s wr, 45 op/s 2024-05-20T21:06:43.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:43 smithi026 ceph-mon[31990]: pgmap v707: 97 pgs: 97 active+clean; 31 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 610 KiB/s wr, 45 op/s 2024-05-20T21:06:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:43 smithi144 ceph-mon[35660]: pgmap v707: 97 pgs: 97 active+clean; 31 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 610 KiB/s wr, 45 op/s 2024-05-20T21:06:43.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 225135 2.10 MB/sec execute 409 sec latency 29.584 ms 2024-05-20T21:06:43.634 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:06:43.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:06:43.911 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:44.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 225868 2.10 MB/sec execute 410 sec latency 15.633 ms 2024-05-20T21:06:45.315 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:45 smithi026 ceph-mon[31990]: pgmap v708: 97 pgs: 97 active+clean; 29 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 794 KiB/s wr, 59 op/s 2024-05-20T21:06:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:45 smithi055 ceph-mon[35213]: pgmap v708: 97 pgs: 97 active+clean; 29 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 794 KiB/s wr, 59 op/s 2024-05-20T21:06:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:45 smithi144 ceph-mon[35660]: pgmap v708: 97 pgs: 97 active+clean; 29 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 794 KiB/s wr, 59 op/s 2024-05-20T21:06:45.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 226720 2.11 MB/sec execute 411 sec latency 15.980 ms 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.712+0000 7facb80ea700 1 -- 172.21.15.26:0/4263311329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb01095d0 msgr2=0x7facb01099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.712+0000 7facb80ea700 1 --2- 172.21.15.26:0/4263311329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb01095d0 0x7facb01099b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7facac009990 tx=0x7facac009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 -- 172.21.15.26:0/4263311329 shutdown_connections 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 --2- 172.21.15.26:0/4263311329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb01095d0 0x7facb01099b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 --2- 172.21.15.26:0/4263311329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb006c5d0 0x7facb01103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 --2- 172.21.15.26:0/4263311329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006bc10 0x7facb006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 -- 172.21.15.26:0/4263311329 >> 172.21.15.26:0/4263311329 conn(0x7facb00793f0 msgr2=0x7facb0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:45.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 -- 172.21.15.26:0/4263311329 shutdown_connections 2024-05-20T21:06:45.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 -- 172.21.15.26:0/4263311329 wait complete. 2024-05-20T21:06:45.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 Processor -- start 2024-05-20T21:06:45.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.713+0000 7facb80ea700 1 -- start start 2024-05-20T21:06:45.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb80ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb006bc10 0x7facb01028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:45.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb80ea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 0x7facb0102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:45.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb80ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb01095d0 0x7facb0103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:45.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb80ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facb0113590 con 0x7facb006bc10 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb80ea700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7facb0113410 con 0x7facb006c5d0 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb80ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facb0113710 con 0x7facb01095d0 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5e86700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb006bc10 0x7facb01028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 0x7facb0102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 0x7facb0102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40774/0 (socket says 172.21.15.26:40774) 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5e86700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb006bc10 0x7facb01028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39250/0 (socket says 172.21.15.26:39250) 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 -- 172.21.15.26:0/643206664 learned_addr learned my addr 172.21.15.26:0/643206664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb6687700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb01095d0 0x7facb0103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 -- 172.21.15.26:0/643206664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb01095d0 msgr2=0x7facb0103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb01095d0 0x7facb0103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 -- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb006bc10 msgr2=0x7facb01028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb006bc10 0x7facb01028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.714+0000 7facb5685700 1 -- 172.21.15.26:0/643206664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb01a7ff0 con 0x7facb006c5d0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.715+0000 7facb5685700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 0x7facb0102e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faca000e9a0 tx=0x7faca000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.715+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca000cbd0 con 0x7facb006c5d0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.715+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb01a82e0 con 0x7facb006c5d0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.715+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faca000cd70 con 0x7facb006c5d0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.715+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0018680 con 0x7facb006c5d0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.715+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb01a8870 con 0x7facb006c5d0 2024-05-20T21:06:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.716+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb010bab0 con 0x7facb006c5d0 2024-05-20T21:06:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.718+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7faca004aab0 con 0x7facb006c5d0 2024-05-20T21:06:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.718+0000 7faca6ffd700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac9c0b2900 0x7fac9c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.718+0000 7facb5e86700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac9c0b2900 0x7fac9c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.719+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faca00e2570 con 0x7facb006c5d0 2024-05-20T21:06:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.719+0000 7facb5e86700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac9c0b2900 0x7fac9c0b4dc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7facac00b2e0 tx=0x7facac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:45.722+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faca00aa230 con 0x7facb006c5d0 2024-05-20T21:06:46.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.041+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facb0104c90 con 0x7facb006c5d0 2024-05-20T21:06:46.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.042+0000 7faca6ffd700 1 -- 172.21.15.26:0/643206664 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7faca00b22b0 con 0x7facb006c5d0 2024-05-20T21:06:46.042 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:06:46.042 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:06:46.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac9c0b2900 msgr2=0x7fac9c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:46.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac9c0b2900 0x7fac9c0b4dc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7facac00b2e0 tx=0x7facac005fb0 comp rx=0 tx=0).stop 2024-05-20T21:06:46.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 msgr2=0x7facb0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 0x7facb0102e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faca000e9a0 tx=0x7faca000ecb0 comp rx=0 tx=0).stop 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 shutdown_connections 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac9c0b2900 0x7fac9c0b4dc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb006bc10 0x7facb01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb01095d0 0x7facb0103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 --2- 172.21.15.26:0/643206664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facb006c5d0 0x7facb0102e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 >> 172.21.15.26:0/643206664 conn(0x7facb00793f0 msgr2=0x7facb0101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 shutdown_connections 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:46.044+0000 7facb80ea700 1 -- 172.21.15.26:0/643206664 wait complete. 2024-05-20T21:06:46.045 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:06:46.365 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:46 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/643206664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:46.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:46 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/643206664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:46 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/643206664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:06:46.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 227247 2.12 MB/sec execute 412 sec latency 17.551 ms 2024-05-20T21:06:46.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:06:47.108 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:47.380 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:47 smithi026 ceph-mon[31990]: pgmap v709: 97 pgs: 97 active+clean; 30 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-05-20T21:06:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:47 smithi055 ceph-mon[35213]: pgmap v709: 97 pgs: 97 active+clean; 30 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-05-20T21:06:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:47 smithi144 ceph-mon[35660]: pgmap v709: 97 pgs: 97 active+clean; 30 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-05-20T21:06:47.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 227661 2.11 MB/sec execute 413 sec latency 10.895 ms 2024-05-20T21:06:48.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 228276 2.11 MB/sec execute 414 sec latency 19.766 ms 2024-05-20T21:06:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.050+0000 7f9483de1700 1 -- 172.21.15.26:0/3200108958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c102440 msgr2=0x7f947c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.050+0000 7f9483de1700 1 --2- 172.21.15.26:0/3200108958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c102440 0x7f947c1028c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f946c009990 tx=0x7f946c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 -- 172.21.15.26:0/3200108958 shutdown_connections 2024-05-20T21:06:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 --2- 172.21.15.26:0/3200108958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f947c102e00 0x7f947c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 --2- 172.21.15.26:0/3200108958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c108520 0x7f947c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 --2- 172.21.15.26:0/3200108958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c102440 0x7f947c1028c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 -- 172.21.15.26:0/3200108958 >> 172.21.15.26:0/3200108958 conn(0x7f947c0793f0 msgr2=0x7f947c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 -- 172.21.15.26:0/3200108958 shutdown_connections 2024-05-20T21:06:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.051+0000 7f9483de1700 1 -- 172.21.15.26:0/3200108958 wait complete. 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 Processor -- start 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 -- start start 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f947c102440 0x7f947c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c102e00 0x7f947c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 0x7f947c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f947c112480 con 0x7f947c108520 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f947c112300 con 0x7f947c102e00 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9483de1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f947c112600 con 0x7f947c102440 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f948237e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 0x7f947c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f948237e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 0x7f947c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39268/0 (socket says 172.21.15.26:39268) 2024-05-20T21:06:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f948237e700 1 -- 172.21.15.26:0/2055450040 learned_addr learned my addr 172.21.15.26:0/2055450040 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f948137c700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c102e00 0x7f947c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.052+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f947c102440 0x7f947c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948237e700 1 -- 172.21.15.26:0/2055450040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f947c102440 msgr2=0x7f947c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948237e700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f947c102440 0x7f947c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948237e700 1 -- 172.21.15.26:0/2055450040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c102e00 msgr2=0x7f947c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948237e700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c102e00 0x7f947c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948237e700 1 -- 172.21.15.26:0/2055450040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c19a0f0 con 0x7f947c108520 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948137c700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c102e00 0x7f947c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f948237e700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 0x7f947c199990 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9474007ee0 tx=0x7f947400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f9472ffd700 1 -- 172.21.15.26:0/2055450040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474004bb0 con 0x7f947c108520 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f9472ffd700 1 -- 172.21.15.26:0/2055450040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9474004d50 con 0x7f947c108520 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c19a3e0 con 0x7f947c108520 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f9472ffd700 1 -- 172.21.15.26:0/2055450040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740058b0 con 0x7f947c108520 2024-05-20T21:06:49.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.053+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c1a66a0 con 0x7f947c108520 2024-05-20T21:06:49.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.056+0000 7f9472ffd700 1 -- 172.21.15.26:0/2055450040 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f947401a020 con 0x7f947c108520 2024-05-20T21:06:49.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.057+0000 7f9472ffd700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94680b2900 0x7f94680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.057+0000 7f948e677b80 1 Processor -- start 2024-05-20T21:06:49.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.057+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94680b2900 0x7f94680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.057+0000 7f9472ffd700 1 -- 172.21.15.26:0/2055450040 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f94740e18d0 con 0x7f947c108520 2024-05-20T21:06:49.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.057+0000 7f948e677b80 1 -- start start 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948e677b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948e677b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948e677b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 0x55ff9d0ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948e677b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d042c90 con 0x55ff9d0ec370 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94680b2900 0x7f94680b4dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f947c19aad0 tx=0x7f9478009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948e677b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d032830 con 0x55ff9d0e7c70 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948e677b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d036870 con 0x55ff9d0e8620 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 0x55ff9d0ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 0x55ff9d0ec7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39284/0 (socket says 172.21.15.26:39284) 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 -- 172.21.15.26:0/519683736 learned_addr learned my addr 172.21.15.26:0/519683736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948137c700 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f9481b7d700 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 -- 172.21.15.26:0/519683736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e8620 msgr2=0x55ff9d0ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 -- 172.21.15.26:0/519683736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e7c70 msgr2=0x55ff9d0e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 -- 172.21.15.26:0/519683736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9cf828d0 con 0x55ff9d0ec370 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f9481b7d700 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.058+0000 7f948237e700 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 0x55ff9d0ec7b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f94740e1bf0 tx=0x7f94740a8d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa62222a97e293c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.059+0000 7f9467fff700 1 -- 172.21.15.26:0/519683736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474055040 con 0x55ff9d0ec370 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.059+0000 7f9467fff700 1 -- 172.21.15.26:0/519683736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94740a9590 con 0x55ff9d0ec370 2024-05-20T21:06:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.059+0000 7f9467fff700 1 -- 172.21.15.26:0/519683736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740a5120 con 0x55ff9d0ec370 2024-05-20T21:06:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.059+0000 7f948e677b80 1 -- 172.21.15.26:0/519683736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 msgr2=0x55ff9d0ec7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.059+0000 7f948e677b80 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 0x55ff9d0ec7b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f94740e1bf0 tx=0x7f94740a8d10 comp rx=0 tx=0).stop 2024-05-20T21:06:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 -- 172.21.15.26:0/519683736 shutdown_connections 2024-05-20T21:06:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0ec370 0x55ff9d0ec7b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 --2- 172.21.15.26:0/519683736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 -- 172.21.15.26:0/519683736 >> 172.21.15.26:0/519683736 conn(0x55ff9d011080 msgr2=0x55ff9d0dde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 -- 172.21.15.26:0/519683736 shutdown_connections 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 -- 172.21.15.26:0/519683736 wait complete. 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 -- start start 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.060+0000 7f948e677b80 1 Processor -- start 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d0f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ff9cf38fa0 con 0x55ff9d0e7c70 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ff9cfd47c0 con 0x55ff9d0e8620 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d0f4530 con 0x55ff9d0ec370 2024-05-20T21:06:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948e677b80 10 client.0 _ll_register_callbacks cb 0x55ff9cfcd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948237e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d0f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948237e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d0f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:47844/0 (socket says 172.21.15.26:47844) 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948237e700 1 -- 172.21.15.26:0/4202473892 learned_addr learned my addr 172.21.15.26:0/4202473892 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948137c700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f9481b7d700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948137c700 1 -- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 msgr2=0x55ff9d0f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948137c700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d0f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948137c700 1 -- 172.21.15.26:0/4202473892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 msgr2=0x55ff9d0f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948137c700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.061+0000 7f948137c700 1 -- 172.21.15.26:0/4202473892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9cfd1690 con 0x55ff9d0e8620 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.062+0000 7f948137c700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0f0b00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b650 tx=0x7f946c00bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab2857f31ed4ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.062+0000 7f94657fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:06:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.062+0000 7f94657fa700 1 -- 172.21.15.26:0/4202473892 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c003880 con 0x55ff9d0e8620 2024-05-20T21:06:49.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.062+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94740a9590 con 0x55ff9d0e8620 2024-05-20T21:06:49.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.062+0000 7f94657fa700 1 -- 172.21.15.26:0/4202473892 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f946c003a20 con 0x55ff9d0e8620 2024-05-20T21:06:49.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.068+0000 7f94657fa700 1 -- 172.21.15.26:0/4202473892 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f946c003c20 con 0x55ff9d0e8620 2024-05-20T21:06:49.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.068+0000 7f948e677b80 10 client.27971 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.068+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff9d0fec80 con 0x55ff9d0e8620 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.068+0000 7f94657fa700 1 -- 172.21.15.26:0/4202473892 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94740a9590 con 0x55ff9d0e8620 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f94657fa700 1 -- 172.21.15.26:0/4202473892 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f946c00f460 con 0x55ff9d0e8620 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f948e677b80 10 client.27971 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f948e677b80 10 client.27971 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f948e677b80 20 client.27971 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f948e677b80 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d1012c0 0x55ff9d1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f948e677b80 4 client.27971 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff9d0af9e0 con 0x55ff9d1012c0 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f9481b7d700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d1012c0 0x55ff9d1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f94657fa700 10 client.27971 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.069+0000 7f9481b7d700 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d1012c0 0x55ff9d1036b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e044fd0021734a0 server_cookie=1825dbec5a7bf249 in_seq=0 out_seq=0 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f94657fa700 1 -- 172.21.15.26:0/4202473892 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ff9d0af9e0 con 0x55ff9d1012c0 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f94657fa700 10 client.27971 handle_command_reply: tid=0 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f948e677b80 1 client.27971 shutdown 2024-05-20T21:06:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 msgr2=0x55ff9d0f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f948e677b80 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0f0b00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b650 tx=0x7f946c00bf30 comp rx=0 tx=0).stop 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 shutdown_connections 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f948e677b80 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d1012c0 0x55ff9d1036b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.070+0000 7f948e677b80 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f948e677b80 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d0f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f948e677b80 1 --2- 172.21.15.26:0/4202473892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0f0b00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 >> 172.21.15.26:0/4202473892 conn(0x55ff9d011080 msgr2=0x55ff9d0ed760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f94657fa700 0 client.27971 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 shutdown_connections 2024-05-20T21:06:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f948e677b80 1 -- 172.21.15.26:0/4202473892 wait complete. 2024-05-20T21:06:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.071+0000 7f948e677b80 20 client.27971 trim_cache size 0 max 16384 2024-05-20T21:06:49.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 Processor -- start 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 -- start start 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ae3a0 con 0x55ff9d0e7c70 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1abc30 con 0x55ff9d0e8620 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948e677b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1a9ac0 con 0x55ff9d0ec370 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f9481b7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f9481b7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0ef3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39312/0 (socket says 172.21.15.26:39312) 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f9481b7d700 1 -- 172.21.15.26:0/2439611638 learned_addr learned my addr 172.21.15.26:0/2439611638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948137c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948237e700 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948137c700 1 -- 172.21.15.26:0/2439611638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 msgr2=0x55ff9d108fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948137c700 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d108fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948137c700 1 -- 172.21.15.26:0/2439611638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 msgr2=0x55ff9d0ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948137c700 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f948137c700 1 -- 172.21.15.26:0/2439611638 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9d0de6b0 con 0x55ff9d0e8620 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.088+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.089+0000 7f948137c700 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ef930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f946c00f950 tx=0x7f946c005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9c0dac161bf9cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.089+0000 7f94677fe700 1 -- 172.21.15.26:0/2439611638 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00f660 con 0x55ff9d0e8620 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.089+0000 7f94677fe700 1 -- 172.21.15.26:0/2439611638 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f946c024480 con 0x55ff9d0e8620 2024-05-20T21:06:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.089+0000 7f94677fe700 1 -- 172.21.15.26:0/2439611638 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c017650 con 0x55ff9d0e8620 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 -- 172.21.15.26:0/2439611638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 msgr2=0x55ff9d0ef930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ef930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f946c00f950 tx=0x7f946c005810 comp rx=0 tx=0).stop 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 -- 172.21.15.26:0/2439611638 shutdown_connections 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0ec370 0x55ff9d108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 --2- 172.21.15.26:0/2439611638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0e8620 0x55ff9d0ef930 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 -- 172.21.15.26:0/2439611638 >> 172.21.15.26:0/2439611638 conn(0x55ff9d011080 msgr2=0x55ff9d0f0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 -- 172.21.15.26:0/2439611638 shutdown_connections 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 -- 172.21.15.26:0/2439611638 wait complete. 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 -- start start 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.090+0000 7f948e677b80 1 Processor -- start 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 0x55ff9d0e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0ec370 0x55ff9d0e9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1aa110 con 0x55ff9d0e8620 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ab560 con 0x55ff9d0ec370 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1aa7c0 con 0x55ff9d0e7c70 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948e677b80 10 client.0 _ll_register_callbacks cb 0x55ff9cfcd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 0x55ff9d0e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 0x55ff9d0e93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39328/0 (socket says 172.21.15.26:39328) 2024-05-20T21:06:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948237e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0ec370 0x55ff9d0e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 -- 172.21.15.26:0/2589198584 learned_addr learned my addr 172.21.15.26:0/2589198584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 -- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e7c70 msgr2=0x55ff9d0f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 -- 172.21.15.26:0/2589198584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0ec370 msgr2=0x55ff9d0e9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0ec370 0x55ff9d0e9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 -- 172.21.15.26:0/2589198584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9d0e9e40 con 0x55ff9d0e8620 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948237e700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0ec370 0x55ff9d0e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f948137c700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 0x55ff9d0e93c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f946c00f950 tx=0x7f946c025e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaec2b72c2b5264 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.091+0000 7f94657fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f94657fa700 1 -- 172.21.15.26:0/2589198584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00fb40 con 0x55ff9d0e8620 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f94657fa700 1 -- 172.21.15.26:0/2589198584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f946c024840 con 0x55ff9d0e8620 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946c024480 con 0x55ff9d0e8620 2024-05-20T21:06:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f94657fa700 1 -- 172.21.15.26:0/2589198584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f946c003680 con 0x55ff9d0e8620 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f948e677b80 10 client.18672 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff9d16bb60 con 0x55ff9d0e8620 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f94657fa700 1 -- 172.21.15.26:0/2589198584 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f946c003820 con 0x55ff9d0e8620 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f94657fa700 1 -- 172.21.15.26:0/2589198584 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f946c0389b0 con 0x55ff9d0e8620 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.092+0000 7f948e677b80 10 client.18672 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f948e677b80 10 client.18672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f948e677b80 20 client.18672 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f948e677b80 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d16c1a0 0x55ff9d16e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f948e677b80 4 client.18672 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f946c024480 con 0x55ff9d16c1a0 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d16c1a0 0x55ff9d16e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f9481b7d700 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d16c1a0 0x55ff9d16e5c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e6d957f7f2e2847 server_cookie=1db00e332c28d57e in_seq=0 out_seq=0 2024-05-20T21:06:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.093+0000 7f94657fa700 10 client.18672 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f94657fa700 1 -- 172.21.15.26:0/2589198584 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f946c024480 con 0x55ff9d16c1a0 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f94657fa700 10 client.18672 handle_command_reply: tid=0 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 client.18672 shutdown 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 msgr2=0x55ff9d0e93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 0x55ff9d0e93c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f946c00f950 tx=0x7f946c025e00 comp rx=0 tx=0).stop 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 shutdown_connections 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ff9d16c1a0 0x55ff9d16e5c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff9d0ec370 0x55ff9d0e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff9d0e7c70 0x55ff9d0f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f94657fa700 0 client.18672 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 --2- 172.21.15.26:0/2589198584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff9d0e8620 0x55ff9d0e93c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 >> 172.21.15.26:0/2589198584 conn(0x55ff9d011080 msgr2=0x55ff9d0eefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 shutdown_connections 2024-05-20T21:06:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 1 -- 172.21.15.26:0/2589198584 wait complete. 2024-05-20T21:06:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.094+0000 7f948e677b80 20 client.18672 trim_cache size 0 max 16384 2024-05-20T21:06:49.099 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94680b2900 msgr2=0x7f94680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94680b2900 0x7f94680b4dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f947c19aad0 tx=0x7f9478009260 comp rx=0 tx=0).stop 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 msgr2=0x7f947c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 0x7f947c199990 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9474007ee0 tx=0x7f947400d3b0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 shutdown_connections 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f94680b2900 0x7f94680b4dc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947c108520 0x7f947c199990 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f947c102440 0x7f947c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 --2- 172.21.15.26:0/2055450040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f947c102e00 0x7f947c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 >> 172.21.15.26:0/2055450040 conn(0x7f947c0793f0 msgr2=0x7f947c0ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 shutdown_connections 2024-05-20T21:06:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:49.095+0000 7f9483de1700 1 -- 172.21.15.26:0/2055450040 wait complete. 2024-05-20T21:06:49.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:49 smithi026 ceph-mon[31990]: pgmap v710: 97 pgs: 97 active+clean; 25 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-05-20T21:06:49.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:49 smithi055 ceph-mon[35213]: pgmap v710: 97 pgs: 97 active+clean; 25 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-05-20T21:06:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:49 smithi144 ceph-mon[35660]: pgmap v710: 97 pgs: 97 active+clean; 25 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-05-20T21:06:49.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 229027 2.12 MB/sec execute 415 sec latency 17.491 ms 2024-05-20T21:06:49.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:06:50.211 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:50.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 229606 2.12 MB/sec execute 416 sec latency 18.535 ms 2024-05-20T21:06:51.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:51 smithi144 ceph-mon[35660]: pgmap v711: 97 pgs: 97 active+clean; 27 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-20T21:06:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:51 smithi055 ceph-mon[35213]: pgmap v711: 97 pgs: 97 active+clean; 27 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-20T21:06:51.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:51 smithi026 ceph-mon[31990]: pgmap v711: 97 pgs: 97 active+clean; 27 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-20T21:06:51.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 230378 2.13 MB/sec execute 417 sec latency 11.826 ms 2024-05-20T21:06:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 -- 172.21.15.26:0/50214874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18102430 msgr2=0x7fec181028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 --2- 172.21.15.26:0/50214874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18102430 0x7fec181028b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009990 tx=0x7fec0c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 -- 172.21.15.26:0/50214874 shutdown_connections 2024-05-20T21:06:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 --2- 172.21.15.26:0/50214874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec18102df0 0x7fec1810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 --2- 172.21.15.26:0/50214874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18108510 0x7fec181088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 --2- 172.21.15.26:0/50214874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18102430 0x7fec181028b0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 -- 172.21.15.26:0/50214874 >> 172.21.15.26:0/50214874 conn(0x7fec180793b0 msgr2=0x7fec180797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 -- 172.21.15.26:0/50214874 shutdown_connections 2024-05-20T21:06:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.122+0000 7fec1d972700 1 -- 172.21.15.26:0/50214874 wait complete. 2024-05-20T21:06:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 Processor -- start 2024-05-20T21:06:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 -- start start 2024-05-20T21:06:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 0x7fec1819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec18102df0 0x7fec1819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 0x7fec18198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec18112490 con 0x7fec18108510 2024-05-20T21:06:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fec18112310 con 0x7fec18102430 2024-05-20T21:06:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec1d972700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec18112610 con 0x7fec18102df0 2024-05-20T21:06:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 0x7fec1819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 0x7fec18198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 0x7fec18198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35840/0 (socket says 172.21.15.26:35840) 2024-05-20T21:06:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec177fe700 1 -- 172.21.15.26:0/4086822510 learned_addr learned my addr 172.21.15.26:0/4086822510 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 0x7fec1819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42034/0 (socket says 172.21.15.26:42034) 2024-05-20T21:06:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.123+0000 7fec167fc700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec18102df0 0x7fec1819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec177fe700 1 -- 172.21.15.26:0/4086822510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec18102df0 msgr2=0x7fec1819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec177fe700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec18102df0 0x7fec1819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec177fe700 1 -- 172.21.15.26:0/4086822510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 msgr2=0x7fec1819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec177fe700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 0x7fec1819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec177fe700 1 -- 172.21.15.26:0/4086822510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec18198fd0 con 0x7fec18108510 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec16ffd700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 0x7fec1819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec177fe700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 0x7fec18198870 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fec1000ef30 tx=0x7fec1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec1c970700 1 -- 172.21.15.26:0/4086822510 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1001a070 con 0x7fec18108510 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec181992c0 con 0x7fec18108510 2024-05-20T21:06:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.124+0000 7fec1c970700 1 -- 172.21.15.26:0/4086822510 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec10004bb0 con 0x7fec18108510 2024-05-20T21:06:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.125+0000 7fec1c970700 1 -- 172.21.15.26:0/4086822510 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100157d0 con 0x7fec18108510 2024-05-20T21:06:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.125+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1806bab0 con 0x7fec18108510 2024-05-20T21:06:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.128+0000 7fec1c970700 1 -- 172.21.15.26:0/4086822510 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fec10003a00 con 0x7fec18108510 2024-05-20T21:06:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.129+0000 7fec1c970700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fec000b2880 0x7fec000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.129+0000 7fec16ffd700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fec000b2880 0x7fec000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.129+0000 7fec1c970700 1 -- 172.21.15.26:0/4086822510 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fec100e6120 con 0x7fec18108510 2024-05-20T21:06:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.129+0000 7fec16ffd700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fec000b2880 0x7fec000b4d40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fec08009680 tx=0x7fec08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 Processor -- start 2024-05-20T21:06:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 -- start start 2024-05-20T21:06:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e798c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7986d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 0x5593e7993080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5593e78e4d70 con 0x5593e7987260 2024-05-20T21:06:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5593e78d4910 con 0x5593e79868a0 2024-05-20T21:06:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec28208b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5593e78d8950 con 0x5593e798c320 2024-05-20T21:06:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 0x5593e7993080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e798c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 0x5593e7993080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35844/0 (socket says 172.21.15.26:35844) 2024-05-20T21:06:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec177fe700 1 -- 172.21.15.26:0/921898495 learned_addr learned my addr 172.21.15.26:0/921898495 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec167fc700 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7986d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec177fe700 1 -- 172.21.15.26:0/921898495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 msgr2=0x5593e798c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.130+0000 7fec177fe700 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e798c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7fec177fe700 1 -- 172.21.15.26:0/921898495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 msgr2=0x5593e7986d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7fec177fe700 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7986d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7fec177fe700 1 -- 172.21.15.26:0/921898495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e787dfb0 con 0x5593e7987260 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7fec167fc700 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7986d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7fec177fe700 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 0x5593e7993080 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fec1000ef00 tx=0x7fec100ad1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c029df86fa37a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7febfdffb700 1 -- 172.21.15.26:0/921898495 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100ad6e0 con 0x5593e7987260 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7febfdffb700 1 -- 172.21.15.26:0/921898495 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec100b1a90 con 0x5593e7987260 2024-05-20T21:06:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.131+0000 7febfdffb700 1 -- 172.21.15.26:0/921898495 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100ad940 con 0x5593e7987260 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 -- 172.21.15.26:0/921898495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 msgr2=0x5593e7993080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 0x5593e7993080 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fec1000ef00 tx=0x7fec100ad1f0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 -- 172.21.15.26:0/921898495 shutdown_connections 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e7987260 0x5593e7993080 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e798c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 --2- 172.21.15.26:0/921898495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7986d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 -- 172.21.15.26:0/921898495 >> 172.21.15.26:0/921898495 conn(0x5593e78b2240 msgr2=0x5593e79813e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 -- 172.21.15.26:0/921898495 shutdown_connections 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 -- 172.21.15.26:0/921898495 wait complete. 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.132+0000 7fec28208b80 1 -- start start 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 Processor -- start 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7992d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e7990080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 0x5593e79905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5593e77dafa0 con 0x5593e798c320 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5593e783cda0 con 0x5593e79868a0 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5593e79936e0 con 0x5593e7987260 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 0x5593e79905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec28208b80 10 client.0 _ll_register_callbacks cb 0x5593e786f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 0x5593e79905c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35846/0 (socket says 172.21.15.26:35846) 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 -- 172.21.15.26:0/3794854265 learned_addr learned my addr 172.21.15.26:0/3794854265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec16ffd700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7992d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec167fc700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e7990080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 -- 172.21.15.26:0/3794854265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 msgr2=0x5593e7990080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e7990080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 -- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 msgr2=0x5593e7992d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7992d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec177fe700 1 -- 172.21.15.26:0/3794854265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e78248d0 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.133+0000 7fec16ffd700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7992d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7fec177fe700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 0x5593e79905c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fec100a9710 tx=0x7fec10004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62963a895f9745f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7febf6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7febf6ffd700 1 -- 172.21.15.26:0/3794854265 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10072060 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec100b1a90 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7febf6ffd700 1 -- 172.21.15.26:0/3794854265 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fec100a9cd0 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7febf6ffd700 1 -- 172.21.15.26:0/3794854265 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec100952d0 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7fec28208b80 10 client.18684 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593e788e760 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7febf6ffd700 1 -- 172.21.15.26:0/3794854265 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec100b1a90 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.134+0000 7febf6ffd700 1 -- 172.21.15.26:0/3794854265 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fec100a90e0 con 0x5593e798c320 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec28208b80 10 client.18684 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec28208b80 10 client.18684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec28208b80 20 client.18684 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:52.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec28208b80 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e79a0150 0x5593e79a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec28208b80 4 client.18684 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593e78e6cc0 con 0x5593e79a0150 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec16ffd700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e79a0150 0x5593e79a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7fec16ffd700 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e79a0150 0x5593e79a2540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4899f36bc411b0a4 server_cookie=b303b9fa27417514 in_seq=0 out_seq=0 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.135+0000 7febf6ffd700 10 client.18684 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7febf6ffd700 1 -- 172.21.15.26:0/3794854265 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5593e78e6cc0 con 0x5593e79a0150 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7febf6ffd700 10 client.18684 handle_command_reply: tid=0 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 client.18684 shutdown 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 msgr2=0x5593e79905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 0x5593e79905c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fec100a9710 tx=0x7fec10004f90 comp rx=0 tx=0).stop 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 shutdown_connections 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e79a0150 0x5593e79a2540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e798c320 0x5593e79905c0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e79868a0 0x5593e7992d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 --2- 172.21.15.26:0/3794854265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e7990080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 >> 172.21.15.26:0/3794854265 conn(0x5593e78b2240 msgr2=0x5593e7980af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7febf6ffd700 0 client.18684 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.136+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 shutdown_connections 2024-05-20T21:06:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.137+0000 7fec28208b80 1 -- 172.21.15.26:0/3794854265 wait complete. 2024-05-20T21:06:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.137+0000 7fec28208b80 20 client.18684 trim_cache size 0 max 16384 2024-05-20T21:06:52.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 Processor -- start 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 -- start start 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e7981270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e798a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 0x5593e7985890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec181ae470 con 0x5593e79868a0 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fec181abd00 con 0x5593e798c320 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec28208b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec181a9b50 con 0x5593e7987260 2024-05-20T21:06:52.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.157+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e7981270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 0x5593e7985890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 0x5593e7985890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42058/0 (socket says 172.21.15.26:42058) 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e7981270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35852/0 (socket says 172.21.15.26:35852) 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 -- 172.21.15.26:0/2742041259 learned_addr learned my addr 172.21.15.26:0/2742041259 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec167fc700 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e798a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 -- 172.21.15.26:0/2742041259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 msgr2=0x5593e798a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e798a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 -- 172.21.15.26:0/2742041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 msgr2=0x5593e7981270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e7981270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 -- 172.21.15.26:0/2742041259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e798e5f0 con 0x5593e798c320 2024-05-20T21:06:52.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7fec177fe700 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 0x5593e7985890 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fec100aa950 tx=0x7fec100ad1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d49811414752e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:52.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7febfd7fa700 1 -- 172.21.15.26:0/2742041259 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10072040 con 0x5593e798c320 2024-05-20T21:06:52.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7febfd7fa700 1 -- 172.21.15.26:0/2742041259 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec100b1a90 con 0x5593e798c320 2024-05-20T21:06:52.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.158+0000 7febfd7fa700 1 -- 172.21.15.26:0/2742041259 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100ad6e0 con 0x5593e798c320 2024-05-20T21:06:52.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 -- 172.21.15.26:0/2742041259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 msgr2=0x5593e7985890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 0x5593e7985890 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fec100aa950 tx=0x7fec100ad1f0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 -- 172.21.15.26:0/2742041259 shutdown_connections 2024-05-20T21:06:52.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e7981270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e798c320 0x5593e7985890 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 --2- 172.21.15.26:0/2742041259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e7987260 0x5593e798a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 -- 172.21.15.26:0/2742041259 >> 172.21.15.26:0/2742041259 conn(0x5593e78b2240 msgr2=0x5593e7989d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:52.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 -- 172.21.15.26:0/2742041259 shutdown_connections 2024-05-20T21:06:52.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.159+0000 7fec28208b80 1 -- 172.21.15.26:0/2742041259 wait complete. 2024-05-20T21:06:52.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 -- start start 2024-05-20T21:06:52.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 Processor -- start 2024-05-20T21:06:52.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e79a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e7987260 0x5593e79a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e79a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec181aa1c0 con 0x5593e79868a0 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fec181ab630 con 0x5593e7987260 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec181aa890 con 0x5593e798c320 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec28208b80 10 client.0 _ll_register_callbacks cb 0x5593e786f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e79a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e79a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35858/0 (socket says 172.21.15.26:35858) 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec16ffd700 1 -- 172.21.15.26:0/1480759342 learned_addr learned my addr 172.21.15.26:0/1480759342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.160+0000 7fec167fc700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e7987260 0x5593e79a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec177fe700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e79a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec16ffd700 1 -- 172.21.15.26:0/1480759342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 msgr2=0x5593e79a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec16ffd700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e79a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec16ffd700 1 -- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e7987260 msgr2=0x5593e79a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec16ffd700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e7987260 0x5593e79a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec16ffd700 1 -- 172.21.15.26:0/1480759342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e79a3340 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec167fc700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e7987260 0x5593e79a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec177fe700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e79a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec16ffd700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e79a5d00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fec08005d90 tx=0x7fec080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b967ed7dc8012 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7febf6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7febf6ffd700 1 -- 172.21.15.26:0/1480759342 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0802e070 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec100b1a90 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7febf6ffd700 1 -- 172.21.15.26:0/1480759342 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fec0801ec10 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7febf6ffd700 1 -- 172.21.15.26:0/1480759342 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec0801ee10 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.161+0000 7fec28208b80 10 client.18690 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593e7a09ae0 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7febf6ffd700 1 -- 172.21.15.26:0/1480759342 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec100b1a90 con 0x5593e79868a0 2024-05-20T21:06:52.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7febf6ffd700 1 -- 172.21.15.26:0/1480759342 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fec0803c050 con 0x5593e79868a0 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 10 client.18690 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 10 client.18690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 20 client.18690 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e7a0a150 0x5593e7a0c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 4 client.18690 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593e78e04a0 con 0x5593e7a0a150 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.162+0000 7fec167fc700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e7a0a150 0x5593e7a0c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7febf6ffd700 10 client.18690 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec167fc700 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e7a0a150 0x5593e7a0c570 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=157c10d03277e576 server_cookie=7af4fcda6836ad07 in_seq=0 out_seq=0 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7febf6ffd700 1 -- 172.21.15.26:0/1480759342 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593e78e04a0 con 0x5593e7a0a150 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7febf6ffd700 10 client.18690 handle_command_reply: tid=0 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 client.18690 shutdown 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 msgr2=0x5593e79a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e79a5d00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fec08005d90 tx=0x7fec080047c0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 shutdown_connections 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5593e7a0a150 0x5593e7a0c570 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593e79868a0 0x5593e79a5d00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593e798c320 0x5593e79a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 --2- 172.21.15.26:0/1480759342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5593e7987260 0x5593e79a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 >> 172.21.15.26:0/1480759342 conn(0x5593e78b2240 msgr2=0x5593e798b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7febf6ffd700 0 client.18690 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 shutdown_connections 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.163+0000 7fec28208b80 1 -- 172.21.15.26:0/1480759342 wait complete. 2024-05-20T21:06:52.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec28208b80 20 client.18690 trim_cache size 0 max 16384 2024-05-20T21:06:52.171 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:52.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fec000b2880 msgr2=0x7fec000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fec000b2880 0x7fec000b4d40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fec08009680 tx=0x7fec08009260 comp rx=0 tx=0).stop 2024-05-20T21:06:52.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 msgr2=0x7fec18198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 0x7fec18198870 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fec1000ef30 tx=0x7fec1000c3a0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 shutdown_connections 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fec000b2880 0x7fec000b4d40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec18108510 0x7fec18198870 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec18102430 0x7fec1819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 --2- 172.21.15.26:0/4086822510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec18102df0 0x7fec1819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.164+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 >> 172.21.15.26:0/4086822510 conn(0x7fec180793b0 msgr2=0x7fec18100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.165+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 shutdown_connections 2024-05-20T21:06:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:52.165+0000 7fec1d972700 1 -- 172.21.15.26:0/4086822510 wait complete. 2024-05-20T21:06:52.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 230958 2.13 MB/sec execute 418 sec latency 24.489 ms 2024-05-20T21:06:53.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:06:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:53 smithi055 ceph-mon[35213]: pgmap v712: 97 pgs: 97 active+clean; 28 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:06:53.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:53 smithi026 ceph-mon[31990]: pgmap v712: 97 pgs: 97 active+clean; 28 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:06:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:53 smithi144 ceph-mon[35660]: pgmap v712: 97 pgs: 97 active+clean; 28 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-20T21:06:53.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 231428 2.13 MB/sec execute 419 sec latency 65.691 ms 2024-05-20T21:06:54.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 232100 2.13 MB/sec execute 420 sec latency 15.910 ms 2024-05-20T21:06:55.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 -- 172.21.15.26:0/1121280947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690406bb70 msgr2=0x7f690410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 --2- 172.21.15.26:0/1121280947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690406bb70 0x7f690410fab0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009990 tx=0x7f68f8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 -- 172.21.15.26:0/1121280947 shutdown_connections 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 --2- 172.21.15.26:0/1121280947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f690410fff0 0x7f69041123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 --2- 172.21.15.26:0/1121280947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f690406b250 0x7f690406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 --2- 172.21.15.26:0/1121280947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690406bb70 0x7f690410fab0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 -- 172.21.15.26:0/1121280947 >> 172.21.15.26:0/1121280947 conn(0x7f69040793f0 msgr2=0x7f6904079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 -- 172.21.15.26:0/1121280947 shutdown_connections 2024-05-20T21:06:55.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.221+0000 7f69096d6700 1 -- 172.21.15.26:0/1121280947 wait complete. 2024-05-20T21:06:55.222 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:55 smithi026 ceph-mon[31990]: pgmap v713: 97 pgs: 97 active+clean; 27 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T21:06:55.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 Processor -- start 2024-05-20T21:06:55.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 -- start start 2024-05-20T21:06:55.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f690406b250 0x7f690419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f690406bb70 0x7f690419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 0x7f69041a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69041157d0 con 0x7f690410fff0 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6904115650 con 0x7f690406b250 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.222+0000 7f69096d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6904115950 con 0x7f690406bb70 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 0x7f69041a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 0x7f69041a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35872/0 (socket says 172.21.15.26:35872) 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 -- 172.21.15.26:0/3640961990 learned_addr learned my addr 172.21.15.26:0/3640961990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f6902ffd700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f690406b250 0x7f690419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69027fc700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f690406bb70 0x7f690419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 -- 172.21.15.26:0/3640961990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f690406bb70 msgr2=0x7f690419c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f690406bb70 0x7f690419c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 -- 172.21.15.26:0/3640961990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f690406b250 msgr2=0x7f690419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f690406b250 0x7f690419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 -- 172.21.15.26:0/3640961990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69041a0d70 con 0x7f690410fff0 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69037fe700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 0x7f69041a0630 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00ef90 tx=0x7f68fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:55.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f68ebfff700 1 -- 172.21.15.26:0/3640961990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0092e0 con 0x7f690410fff0 2024-05-20T21:06:55.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69041a1000 con 0x7f690410fff0 2024-05-20T21:06:55.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.223+0000 7f68ebfff700 1 -- 172.21.15.26:0/3640961990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68fc004bb0 con 0x7f690410fff0 2024-05-20T21:06:55.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.224+0000 7f68ebfff700 1 -- 172.21.15.26:0/3640961990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc004020 con 0x7f690410fff0 2024-05-20T21:06:55.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.224+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a88e0 con 0x7f690410fff0 2024-05-20T21:06:55.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.226+0000 7f68ebfff700 1 -- 172.21.15.26:0/3640961990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f68fc01b030 con 0x7f690410fff0 2024-05-20T21:06:55.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.227+0000 7f68ebfff700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68ec0b2880 0x7f68ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.227+0000 7f6902ffd700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68ec0b2880 0x7f68ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.227+0000 7f6913f6cb80 1 Processor -- start 2024-05-20T21:06:55.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.227+0000 7f68ebfff700 1 -- 172.21.15.26:0/3640961990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f68fc0e1d50 con 0x7f690410fff0 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.227+0000 7f6913f6cb80 1 -- start start 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648e2718c90 con 0x5648e27b9860 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5648e2708830 con 0x5648e27bdf60 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648e270c870 con 0x5648e27ba210 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68ec0b2880 0x7f68ec0b4d40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009c60 tx=0x7f68f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f69037fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35884/0 (socket says 172.21.15.26:35884) 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 -- 172.21.15.26:0/842977811 learned_addr learned my addr 172.21.15.26:0/842977811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f69027fc700 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 -- 172.21.15.26:0/842977811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 msgr2=0x5648e27bd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 -- 172.21.15.26:0/842977811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 msgr2=0x5648e27be3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27be3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f6902ffd700 1 -- 172.21.15.26:0/842977811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e26588d0 con 0x5648e27b9860 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.228+0000 7f69037fe700 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27be3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.229+0000 7f6902ffd700 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27b9c40 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004140 tx=0x7f68f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd44d62f8fa7407 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.229+0000 7f68e97fa700 1 -- 172.21.15.26:0/842977811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003b10 con 0x5648e27b9860 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.229+0000 7f68e97fa700 1 -- 172.21.15.26:0/842977811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68f4003cb0 con 0x5648e27b9860 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.229+0000 7f68e97fa700 1 -- 172.21.15.26:0/842977811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4026660 con 0x5648e27b9860 2024-05-20T21:06:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 -- 172.21.15.26:0/842977811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 msgr2=0x5648e27b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27b9c40 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004140 tx=0x7f68f4014000 comp rx=0 tx=0).stop 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 -- 172.21.15.26:0/842977811 shutdown_connections 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27b9c40 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27be3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/842977811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 -- 172.21.15.26:0/842977811 >> 172.21.15.26:0/842977811 conn(0x5648e26e7080 msgr2=0x5648e27b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 -- 172.21.15.26:0/842977811 shutdown_connections 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 -- 172.21.15.26:0/842977811 wait complete. 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 -- start start 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.230+0000 7f6913f6cb80 1 Processor -- start 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27b9860 0x5648e27c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27bdf60 0x5648e27c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648e260efa0 con 0x5648e27bdf60 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5648e26aa7c0 con 0x5648e27ba210 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648e27c6120 con 0x5648e27b9860 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6913f6cb80 10 client.0 _ll_register_callbacks cb 0x5648e26a3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69027fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27b9860 0x5648e27c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69037fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27bdf60 0x5648e27c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69027fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42096/0 (socket says 172.21.15.26:42096) 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69027fc700 1 -- 172.21.15.26:0/3222714194 learned_addr learned my addr 172.21.15.26:0/3222714194 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69037fe700 1 -- 172.21.15.26:0/3222714194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27b9860 msgr2=0x5648e27c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69037fe700 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27b9860 0x5648e27c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69037fe700 1 -- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 msgr2=0x5648e27c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69037fe700 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.231+0000 7f69037fe700 1 -- 172.21.15.26:0/3222714194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e26a7690 con 0x5648e27bdf60 2024-05-20T21:06:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f69037fe700 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27bdf60 0x5648e27c2bd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f68fc003fd0 tx=0x7f68fc0b1fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca207e3d0e011474 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f68e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f68e6ff5700 1 -- 172.21.15.26:0/3222714194 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0c3060 con 0x5648e27bdf60 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f68e6ff5700 1 -- 172.21.15.26:0/3222714194 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f68fc003c10 con 0x5648e27bdf60 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f4003cb0 con 0x5648e27bdf60 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f68e6ff5700 1 -- 172.21.15.26:0/3222714194 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68fc06d050 con 0x5648e27bdf60 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f6913f6cb80 10 client.18702 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648e27d0860 con 0x5648e27bdf60 2024-05-20T21:06:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.232+0000 7f68e6ff5700 1 -- 172.21.15.26:0/3222714194 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68f4003cb0 con 0x5648e27bdf60 2024-05-20T21:06:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f68e6ff5700 1 -- 172.21.15.26:0/3222714194 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f68fc0a3ac0 con 0x5648e27bdf60 2024-05-20T21:06:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6913f6cb80 10 client.18702 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6913f6cb80 10 client.18702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6913f6cb80 20 client.18702 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e27d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6913f6cb80 4 client.18702 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648e27859e0 con 0x5648e27d2ea0 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6902ffd700 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e27d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f68e6ff5700 10 client.18702 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.233+0000 7f6902ffd700 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e27d5290 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388398032cd478c8 server_cookie=11fbd9f213542a0d in_seq=0 out_seq=0 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f68e6ff5700 1 -- 172.21.15.26:0/3222714194 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5648e27859e0 con 0x5648e27d2ea0 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f68e6ff5700 10 client.18702 handle_command_reply: tid=0 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 client.18702 shutdown 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27bdf60 msgr2=0x5648e27c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27bdf60 0x5648e27c2bd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f68fc003fd0 tx=0x7f68fc0b1fc0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 shutdown_connections 2024-05-20T21:06:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e27d5290 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27bdf60 0x5648e27c2bd0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27b9860 0x5648e27c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f68e6ff5700 0 client.18702 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.234+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/3222714194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.235+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 >> 172.21.15.26:0/3222714194 conn(0x5648e26e7080 msgr2=0x5648e27b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.235+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 shutdown_connections 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.235+0000 7f6913f6cb80 1 -- 172.21.15.26:0/3222714194 wait complete. 2024-05-20T21:06:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.235+0000 7f6913f6cb80 20 client.18702 trim_cache size 0 max 16384 2024-05-20T21:06:55.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.250+0000 7f6913f6cb80 1 Processor -- start 2024-05-20T21:06:55.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.250+0000 7f6913f6cb80 1 -- start start 2024-05-20T21:06:55.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27c1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69041b27d0 con 0x5648e27b9860 2024-05-20T21:06:55.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f69041b0060 con 0x5648e27bdf60 2024-05-20T21:06:55.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69041adeb0 con 0x5648e27ba210 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27c1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35892/0 (socket says 172.21.15.26:35892) 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 -- 172.21.15.26:0/1359487835 learned_addr learned my addr 172.21.15.26:0/1359487835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f69037fe700 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f69027fc700 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 -- 172.21.15.26:0/1359487835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 msgr2=0x5648e27bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 -- 172.21.15.26:0/1359487835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 msgr2=0x5648e27dab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27dab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 -- 172.21.15.26:0/1359487835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e27bd500 con 0x5648e27b9860 2024-05-20T21:06:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f69037fe700 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27dab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.251+0000 7f6902ffd700 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27c1830 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f68f4003960 tx=0x7f68f4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd6881c33e222f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.252+0000 7f68e8ff9700 1 -- 172.21.15.26:0/1359487835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4005240 con 0x5648e27b9860 2024-05-20T21:06:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.252+0000 7f68e8ff9700 1 -- 172.21.15.26:0/1359487835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68f401e420 con 0x5648e27b9860 2024-05-20T21:06:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.252+0000 7f68e8ff9700 1 -- 172.21.15.26:0/1359487835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4004730 con 0x5648e27b9860 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.252+0000 7f6913f6cb80 1 -- 172.21.15.26:0/1359487835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 msgr2=0x5648e27c1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.252+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27c1830 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f68f4003960 tx=0x7f68f4003b90 comp rx=0 tx=0).stop 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- 172.21.15.26:0/1359487835 shutdown_connections 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27c1830 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27bdf60 0x5648e27dab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/1359487835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27ba210 0x5648e27bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- 172.21.15.26:0/1359487835 >> 172.21.15.26:0/1359487835 conn(0x5648e26e7080 msgr2=0x5648e27bc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- 172.21.15.26:0/1359487835 shutdown_connections 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- 172.21.15.26:0/1359487835 wait complete. 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- start start 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 Processor -- start 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27bdf60 0x5648e27c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ae520 con 0x5648e27b9860 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f69041af990 con 0x5648e27ba210 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69041aebf0 con 0x5648e27bdf60 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.253+0000 7f6913f6cb80 10 client.0 _ll_register_callbacks cb 0x5648e26a3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f69027fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f69027fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42116/0 (socket says 172.21.15.26:42116) 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f69027fc700 1 -- 172.21.15.26:0/2210512195 learned_addr learned my addr 172.21.15.26:0/2210512195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f69037fe700 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27bdf60 0x5648e27c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 -- 172.21.15.26:0/2210512195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27bdf60 msgr2=0x5648e27c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27bdf60 0x5648e27c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 -- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 msgr2=0x5648e27c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 -- 172.21.15.26:0/2210512195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e27c9b80 con 0x5648e27b9860 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6902ffd700 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27cba60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f68f4003960 tx=0x7f68f4003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3db3bf94d131a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f68e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f68e6ff5700 1 -- 172.21.15.26:0/2210512195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f401e9e0 con 0x5648e27b9860 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f68e6ff5700 1 -- 172.21.15.26:0/2210512195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f68f4005050 con 0x5648e27b9860 2024-05-20T21:06:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.254+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f401e420 con 0x5648e27b9860 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f68e6ff5700 1 -- 172.21.15.26:0/2210512195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68f4005250 con 0x5648e27b9860 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 10 client.18708 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648e283da10 con 0x5648e27b9860 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f68e6ff5700 1 -- 172.21.15.26:0/2210512195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68f403f070 con 0x5648e27b9860 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f68e6ff5700 1 -- 172.21.15.26:0/2210512195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f68f4004ab0 con 0x5648e27b9860 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 10 client.18708 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 10 client.18708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 20 client.18708 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e283fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 4 client.18708 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.255+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f68f401e420 con 0x5648e27d2ea0 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f69027fc700 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e283fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f69027fc700 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e283fdf0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0e80e3742815d4 server_cookie=7edade5b2b0ee351 in_seq=0 out_seq=0 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f68e6ff5700 10 client.18708 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f68e6ff5700 1 -- 172.21.15.26:0/2210512195 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f68f401e420 con 0x5648e27d2ea0 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f68e6ff5700 10 client.18708 handle_command_reply: tid=0 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 client.18708 shutdown 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 msgr2=0x5648e27cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27cba60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f68f4003960 tx=0x7f68f4003b20 comp rx=0 tx=0).stop 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 shutdown_connections 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5648e27d2ea0 0x5648e283fdf0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648e27b9860 0x5648e27cba60 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648e27bdf60 0x5648e27c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 --2- 172.21.15.26:0/2210512195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648e27ba210 0x5648e27c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.256+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 >> 172.21.15.26:0/2210512195 conn(0x5648e26e7080 msgr2=0x5648e27b8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.257+0000 7f68e6ff5700 0 client.18708 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.257+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 shutdown_connections 2024-05-20T21:06:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.257+0000 7f6913f6cb80 1 -- 172.21.15.26:0/2210512195 wait complete. 2024-05-20T21:06:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.257+0000 7f6913f6cb80 20 client.18708 trim_cache size 0 max 16384 2024-05-20T21:06:55.261 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68ec0b2880 msgr2=0x7f68ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68ec0b2880 0x7f68ec0b4d40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009c60 tx=0x7f68f4009260 comp rx=0 tx=0).stop 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 msgr2=0x7f69041a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 0x7f69041a0630 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00ef90 tx=0x7f68fc00c3a0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 shutdown_connections 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f68ec0b2880 0x7f68ec0b4d40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f690410fff0 0x7f69041a0630 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f690406b250 0x7f690419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 --2- 172.21.15.26:0/3640961990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f690406bb70 0x7f690419c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.258+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 >> 172.21.15.26:0/3640961990 conn(0x7f69040793f0 msgr2=0x7f69041115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.259+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 shutdown_connections 2024-05-20T21:06:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:06:55.259+0000 7f69096d6700 1 -- 172.21.15.26:0/3640961990 wait complete. 2024-05-20T21:06:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:55 smithi055 ceph-mon[35213]: pgmap v713: 97 pgs: 97 active+clean; 27 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T21:06:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:55 smithi144 ceph-mon[35660]: pgmap v713: 97 pgs: 97 active+clean; 27 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T21:06:55.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 232790 2.13 MB/sec execute 421 sec latency 26.208 ms 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 -- 172.21.15.26:0/883519572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 msgr2=0x7f8bc8103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 --2- 172.21.15.26:0/883519572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc8103750 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb802a410 comp rx=0 tx=0).stop 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 -- 172.21.15.26:0/883519572 shutdown_connections 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 --2- 172.21.15.26:0/883519572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8106640 0x7f8bc8106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 --2- 172.21.15.26:0/883519572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc8103c90 0x7f8bc8106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 --2- 172.21.15.26:0/883519572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc8103750 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 -- 172.21.15.26:0/883519572 >> 172.21.15.26:0/883519572 conn(0x7f8bc80fcf20 msgr2=0x7f8bc80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 -- 172.21.15.26:0/883519572 shutdown_connections 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.351+0000 7f8bce558640 1 -- 172.21.15.26:0/883519572 wait complete. 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 Processor -- start 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 -- start start 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8103c90 0x7f8bc819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc8106640 0x7f8bc81a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8114ab0 con 0x7f8bc8101320 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8114930 con 0x7f8bc8103c90 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bce558640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8114c30 con 0x7f8bc8106640 2024-05-20T21:06:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35910/0 (socket says 172.21.15.26:35910) 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bc7fff640 1 -- 172.21.15.26:0/1191894909 learned_addr learned my addr 172.21.15.26:0/1191894909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8103c90 0x7f8bc819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.352+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc8106640 0x7f8bc81a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc7fff640 1 -- 172.21.15.26:0/1191894909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc8106640 msgr2=0x7f8bc81a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc8106640 0x7f8bc81a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc7fff640 1 -- 172.21.15.26:0/1191894909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8103c90 msgr2=0x7f8bc819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8103c90 0x7f8bc819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc7fff640 1 -- 172.21.15.26:0/1191894909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81a6d80 con 0x7f8bc8101320 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8103c90 0x7f8bc819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc819f160 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009a20 tx=0x7f8bb80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc57fa640 1 -- 172.21.15.26:0/1191894909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb803f030 con 0x7f8bc8101320 2024-05-20T21:06:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc57fa640 1 -- 172.21.15.26:0/1191894909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bb8003a80 con 0x7f8bc8101320 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81a7010 con 0x7f8bc8101320 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bc57fa640 1 -- 172.21.15.26:0/1191894909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8035070 con 0x7f8bc8101320 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.353+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81a7500 con 0x7f8bc8101320 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 Processor -- start 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 -- start start 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bc57fa640 1 -- 172.21.15.26:0/1191894909 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8bb8042030 con 0x7f8bc8101320 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231886c0 0x558823188ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231890b0 0x55882318dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55882318e220 0x5588231906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5588230f9c30 con 0x5588231890b0 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5588230e97d0 con 0x55882318e220 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bd1272740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5588230ed810 con 0x5588231886c0 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231890b0 0x55882318dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231890b0 0x55882318dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35912/0 (socket says 172.21.15.26:35912) 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bc7fff640 1 -- 172.21.15.26:0/1539372080 learned_addr learned my addr 172.21.15.26:0/1539372080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55882318e220 0x5588231906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.356+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231886c0 0x558823188ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc77fe640 1 -- 172.21.15.26:0/1539372080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231886c0 msgr2=0x558823188ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231886c0 0x558823188ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc77fe640 1 -- 172.21.15.26:0/1539372080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231890b0 msgr2=0x55882318dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231890b0 0x55882318dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc77fe640 1 -- 172.21.15.26:0/1539372080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558822dfe210 con 0x55882318e220 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55882318e220 0x5588231906b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00bfb0 tx=0x7f8bbc010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9be81fccda3ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bae7fc640 1 -- 172.21.15.26:0/1539372080 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc004220 con 0x55882318e220 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc57fa640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b28a0 0x7f8bb00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bae7fc640 1 -- 172.21.15.26:0/1539372080 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bbc0043c0 con 0x55882318e220 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bae7fc640 1 -- 172.21.15.26:0/1539372080 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0118b0 con 0x55882318e220 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b28a0 0x7f8bb00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.357+0000 7f8bc57fa640 1 -- 172.21.15.26:0/1191894909 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8bb81027e0 con 0x7f8bc8101320 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 -- 172.21.15.26:0/1539372080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55882318e220 msgr2=0x5588231906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b28a0 0x7f8bb00b4d60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x558823189910 tx=0x7f8bc0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55882318e220 0x5588231906b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00bfb0 tx=0x7f8bbc010860 comp rx=0 tx=0).stop 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 -- 172.21.15.26:0/1539372080 shutdown_connections 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55882318e220 0x5588231906b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231890b0 0x55882318dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1539372080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231886c0 0x558823188ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 -- 172.21.15.26:0/1539372080 >> 172.21.15.26:0/1539372080 conn(0x558823183d30 msgr2=0x558823184130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 -- 172.21.15.26:0/1539372080 shutdown_connections 2024-05-20T21:06:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 -- 172.21.15.26:0/1539372080 wait complete. 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.358+0000 7f8bd1272740 1 -- start start 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 Processor -- start 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231886c0 0x558823192820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231890b0 0x558823192d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 0x558823199da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558822dcf790 con 0x55882318e220 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5588230a5890 con 0x5588231886c0 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5588231911d0 con 0x5588231890b0 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bd1272740 10 client.0 _ll_register_callbacks cb 0x558822dfda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 0x558823199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 0x558823199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35926/0 (socket says 172.21.15.26:35926) 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 -- 172.21.15.26:0/2046136451 learned_addr learned my addr 172.21.15.26:0/2046136451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231886c0 0x558823192820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231890b0 0x558823192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 -- 172.21.15.26:0/2046136451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231890b0 msgr2=0x558823192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231890b0 0x558823192d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 -- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231886c0 msgr2=0x558823192820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231886c0 0x558823192820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 -- 172.21.15.26:0/2046136451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558822eca620 con 0x55882318e220 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231886c0 0x558823192820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.359+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 0x558823199da0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009e80 tx=0x7f8bc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b76ec70eb99951 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8ba77fe640 1 -- 172.21.15.26:0/2046136451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc002d060 con 0x55882318e220 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8ba77fe640 1 -- 172.21.15.26:0/2046136451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8bc0018030 con 0x55882318e220 2024-05-20T21:06:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bbc0043c0 con 0x55882318e220 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8ba77fe640 1 -- 172.21.15.26:0/2046136451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bc0020710 con 0x55882318e220 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8bd1272740 10 client.18720 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55882319ce50 con 0x55882318e220 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8ba77fe640 1 -- 172.21.15.26:0/2046136451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bbc0043c0 con 0x55882318e220 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8ba77fe640 1 -- 172.21.15.26:0/2046136451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8bc0031030 con 0x55882318e220 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8bd1272740 10 client.18720 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8bd1272740 10 client.18720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.360+0000 7f8bd1272740 20 client.18720 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.361+0000 7f8bd1272740 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55882319e5e0 0x5588231a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.361+0000 7f8bd1272740 4 client.18720 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.361+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588230fbb80 con 0x55882319e5e0 2024-05-20T21:06:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.361+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55882319e5e0 0x5588231a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.361+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55882319e5e0 0x5588231a0a30 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0b668930c4c123 server_cookie=69f9d4a8fa0ba8cd in_seq=0 out_seq=0 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.361+0000 7f8ba77fe640 10 client.18720 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8ba77fe640 1 -- 172.21.15.26:0/2046136451 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5588230fbb80 con 0x55882319e5e0 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8ba77fe640 10 client.18720 handle_command_reply: tid=0 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 client.18720 shutdown 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 msgr2=0x558823199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 0x558823199da0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009e80 tx=0x7f8bc0004a40 comp rx=0 tx=0).stop 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 shutdown_connections 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55882319e5e0 0x5588231a0a30 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55882318e220 0x558823199da0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588231890b0 0x558823192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8ba77fe640 0 client.18720 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 --2- 172.21.15.26:0/2046136451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231886c0 0x558823192820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 >> 172.21.15.26:0/2046136451 conn(0x558823183d30 msgr2=0x558823182fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 shutdown_connections 2024-05-20T21:06:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.362+0000 7f8bd1272740 1 -- 172.21.15.26:0/2046136451 wait complete. 2024-05-20T21:06:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.363+0000 7f8bd1272740 20 client.18720 trim_cache size 0 max 16384 2024-05-20T21:06:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 Processor -- start 2024-05-20T21:06:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 -- start start 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x55882317ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823199d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823186c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b2dd0 con 0x5588231886c0 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b06a0 con 0x5588231890b0 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bd1272740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ae4f0 con 0x55882318e220 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.379+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x55882317ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x55882317ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35938/0 (socket says 172.21.15.26:35938) 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc7fff640 1 -- 172.21.15.26:0/1669664249 learned_addr learned my addr 172.21.15.26:0/1669664249 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823186c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 -- 172.21.15.26:0/1669664249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 msgr2=0x558823186c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823186c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 -- 172.21.15.26:0/1669664249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 msgr2=0x55882317ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x55882317ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 -- 172.21.15.26:0/1669664249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558823188220 con 0x5588231890b0 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x55882317ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823199d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b5f0 tx=0x7f8bbc005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12eab5f492f50b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8badffb640 1 -- 172.21.15.26:0/1669664249 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02b070 con 0x5588231890b0 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8badffb640 1 -- 172.21.15.26:0/1669664249 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bbc0053b0 con 0x5588231890b0 2024-05-20T21:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.380+0000 7f8badffb640 1 -- 172.21.15.26:0/1669664249 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc011820 con 0x5588231890b0 2024-05-20T21:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 -- 172.21.15.26:0/1669664249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 msgr2=0x558823199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823199d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b5f0 tx=0x7f8bbc005040 comp rx=0 tx=0).stop 2024-05-20T21:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 -- 172.21.15.26:0/1669664249 shutdown_connections 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823186c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823199d60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 --2- 172.21.15.26:0/1669664249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x55882317ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 -- 172.21.15.26:0/1669664249 >> 172.21.15.26:0/1669664249 conn(0x558823183d30 msgr2=0x558823184110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 -- 172.21.15.26:0/1669664249 shutdown_connections 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.381+0000 7f8bd1272740 1 -- 172.21.15.26:0/1669664249 wait complete. 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 -- start start 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 Processor -- start 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x558823197890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823198de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81aeb60 con 0x5588231886c0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81affd0 con 0x5588231890b0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81af230 con 0x55882318e220 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bd1272740 10 client.0 _ll_register_callbacks cb 0x7f8bc0009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x558823197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x558823197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35952/0 (socket says 172.21.15.26:35952) 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bc7fff640 1 -- 172.21.15.26:0/573635765 learned_addr learned my addr 172.21.15.26:0/573635765 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.382+0000 7f8bccaf5640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc7fff640 1 -- 172.21.15.26:0/573635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 msgr2=0x558823198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc7fff640 1 -- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 msgr2=0x558823197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc7fff640 1 -- 172.21.15.26:0/573635765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588231993d0 con 0x5588231886c0 2024-05-20T21:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bc7fff640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x558823197890 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80c9b50 tx=0x7f8bb8102bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941080703e811571 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8ba77fe640 1 -- 172.21.15.26:0/573635765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8049040 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8ba77fe640 1 -- 172.21.15.26:0/573635765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8bb80e4420 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bbc0053b0 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8ba77fe640 1 -- 172.21.15.26:0/573635765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bb80c9960 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bd1272740 10 client.18726 fetch_fsmap learned FSMap version 30 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558823209630 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8ba77fe640 1 -- 172.21.15.26:0/573635765 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bbc0053b0 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.383+0000 7f8ba77fe640 1 -- 172.21.15.26:0/573635765 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8bb8003cc0 con 0x5588231886c0 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bd1272740 10 client.18726 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bd1272740 10 client.18726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bd1272740 20 client.18726 populate_metadata read hostname 'smithi026' 2024-05-20T21:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bd1272740 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558823209fe0 0x55882320c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bd1272740 4 client.18726 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588230f5360 con 0x558823209fe0 2024-05-20T21:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558823209fe0 0x55882320c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8bc77fe640 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558823209fe0 0x55882320c400 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a332487fe7bece7 server_cookie=d65471232a514b1a in_seq=0 out_seq=0 2024-05-20T21:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.384+0000 7f8ba77fe640 10 client.18726 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8ba77fe640 1 -- 172.21.15.26:0/573635765 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5588230f5360 con 0x558823209fe0 2024-05-20T21:06:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8ba77fe640 10 client.18726 handle_command_reply: tid=0 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 client.18726 shutdown 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 msgr2=0x558823197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x558823197890 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80c9b50 tx=0x7f8bb8102bd0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 shutdown_connections 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x558823209fe0 0x55882320c400 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8ba77fe640 0 client.18726 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55882318e220 0x558823198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5588231890b0 0x558823197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 --2- 172.21.15.26:0/573635765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588231886c0 0x558823197890 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 >> 172.21.15.26:0/573635765 conn(0x558823183d30 msgr2=0x55882317ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 shutdown_connections 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.418+0000 7f8bd1272740 1 -- 172.21.15.26:0/573635765 wait complete. 2024-05-20T21:06:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bd1272740 20 client.18726 trim_cache size 0 max 16384 2024-05-20T21:06:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:06:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:06:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b28a0 msgr2=0x7f8bb00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b28a0 0x7f8bb00b4d60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x558823189910 tx=0x7f8bc0009500 comp rx=0 tx=0).stop 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 msgr2=0x7f8bc819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc819f160 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009a20 tx=0x7f8bb80040c0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 shutdown_connections 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8bb00b28a0 0x7f8bb00b4d60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc8106640 0x7f8bc81a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc8103c90 0x7f8bc819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 --2- 172.21.15.26:0/1191894909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bc8101320 0x7f8bc819f160 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 >> 172.21.15.26:0/1191894909 conn(0x7f8bc80fcf20 msgr2=0x7f8bc81047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 shutdown_connections 2024-05-20T21:06:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:06:56.419+0000 7f8bce558640 1 -- 172.21.15.26:0/1191894909 wait complete. 2024-05-20T21:06:56.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 233537 2.14 MB/sec execute 422 sec latency 18.816 ms 2024-05-20T21:06:57.092 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:06:57.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:06:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:06:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:57 smithi055 ceph-mon[35213]: pgmap v714: 97 pgs: 97 active+clean; 29 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T21:06:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:57 smithi026 ceph-mon[31990]: pgmap v714: 97 pgs: 97 active+clean; 29 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T21:06:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:57 smithi144 ceph-mon[35660]: pgmap v714: 97 pgs: 97 active+clean; 29 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-05-20T21:06:57.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 234170 2.15 MB/sec execute 423 sec latency 14.788 ms 2024-05-20T21:06:58.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 234546 2.15 MB/sec execute 424 sec latency 72.042 ms 2024-05-20T21:06:59.363 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:06:59 smithi026 ceph-mon[31990]: pgmap v715: 97 pgs: 97 active+clean; 29 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:06:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:06:59 smithi055 ceph-mon[35213]: pgmap v715: 97 pgs: 97 active+clean; 29 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:06:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:06:59 smithi144 ceph-mon[35660]: pgmap v715: 97 pgs: 97 active+clean; 29 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-05-20T21:06:59.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 235060 2.14 MB/sec execute 425 sec latency 61.678 ms 2024-05-20T21:07:00.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 235763 2.15 MB/sec execute 426 sec latency 15.532 ms 2024-05-20T21:07:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 -- 172.21.15.26:0/1945939583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 msgr2=0x7f5ae01028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 --2- 172.21.15.26:0/1945939583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae01028b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00098f0 tx=0x7f5ad0009c00 comp rx=0 tx=0).stop 2024-05-20T21:07:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 -- 172.21.15.26:0/1945939583 shutdown_connections 2024-05-20T21:07:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 --2- 172.21.15.26:0/1945939583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 0x7f5ae010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 --2- 172.21.15.26:0/1945939583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae0108510 0x7f5ae01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 --2- 172.21.15.26:0/1945939583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae01028b0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 -- 172.21.15.26:0/1945939583 >> 172.21.15.26:0/1945939583 conn(0x7f5ae00793b0 msgr2=0x7f5ae00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 -- 172.21.15.26:0/1945939583 shutdown_connections 2024-05-20T21:07:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.021+0000 7f5ae7269700 1 -- 172.21.15.26:0/1945939583 wait complete. 2024-05-20T21:07:01.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 Processor -- start 2024-05-20T21:07:01.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 -- start start 2024-05-20T21:07:01.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 0x7f5ae019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae0108510 0x7f5ae01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0112490 con 0x7f5ae0102430 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0112310 con 0x7f5ae0102df0 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae7269700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0112610 con 0x7f5ae0108510 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae5005700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae4804700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 0x7f5ae019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae5005700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45846/0 (socket says 172.21.15.26:45846) 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae5005700 1 -- 172.21.15.26:0/2734015254 learned_addr learned my addr 172.21.15.26:0/2734015254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae5806700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae0108510 0x7f5ae01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae4804700 1 -- 172.21.15.26:0/2734015254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae0108510 msgr2=0x7f5ae01999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae4804700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae0108510 0x7f5ae01999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae4804700 1 -- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 msgr2=0x7f5ae019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae4804700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.022+0000 7f5ae4804700 1 -- 172.21.15.26:0/2734015254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae019a140 con 0x7f5ae0102df0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.023+0000 7f5ae4804700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 0x7f5ae019fd10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad000f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.023+0000 7f5ad67fc700 1 -- 172.21.15.26:0/2734015254 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000b560 con 0x7f5ae0102df0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.023+0000 7f5ad67fc700 1 -- 172.21.15.26:0/2734015254 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad000bb90 con 0x7f5ae0102df0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.023+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae019a3d0 con 0x7f5ae0102df0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.023+0000 7f5ad67fc700 1 -- 172.21.15.26:0/2734015254 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00038e0 con 0x7f5ae0102df0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.023+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae01a66a0 con 0x7f5ae0102df0 2024-05-20T21:07:01.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.026+0000 7f5ad67fc700 1 -- 172.21.15.26:0/2734015254 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5ad0005260 con 0x7f5ae0102df0 2024-05-20T21:07:01.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.026+0000 7f5ad67fc700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5acc0b2880 0x7f5acc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.027+0000 7f5ae5005700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5acc0b2880 0x7f5acc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.027+0000 7f5ad67fc700 1 -- 172.21.15.26:0/2734015254 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5ad00e2ff0 con 0x7f5ae0102df0 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.027+0000 7f5af1affb80 1 Processor -- start 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.027+0000 7f5ae5005700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5acc0b2880 0x7f5acc0b4d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0107af0 tx=0x7f5adc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.027+0000 7f5af1affb80 1 -- start start 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d62920 0x556c05d62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d632e0 0x556c05d6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5af1affb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556c05cc0df0 con 0x556c05d683a0 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5af1affb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556c05cb0990 con 0x556c05d62920 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5af1affb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c05cb49d0 con 0x556c05d632e0 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d68780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45850/0 (socket says 172.21.15.26:45850) 2024-05-20T21:07:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 -- 172.21.15.26:0/1841620057 learned_addr learned my addr 172.21.15.26:0/1841620057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5806700 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d632e0 0x556c05d6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae4804700 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d62920 0x556c05d62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 -- 172.21.15.26:0/1841620057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d632e0 msgr2=0x556c05d6f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d632e0 0x556c05d6f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 -- 172.21.15.26:0/1841620057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d62920 msgr2=0x556c05d62da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d62920 0x556c05d62da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 -- 172.21.15.26:0/1841620057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c05c5a030 con 0x556c05d683a0 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae4804700 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d62920 0x556c05d62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.028+0000 7f5ae5005700 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d68780 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00f550 tx=0x7f5adc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582e0a538c9917af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.029+0000 7f5acb7fe700 1 -- 172.21.15.26:0/1841620057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc026400 con 0x556c05d683a0 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.029+0000 7f5acb7fe700 1 -- 172.21.15.26:0/1841620057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5adc01e880 con 0x556c05d683a0 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.029+0000 7f5acb7fe700 1 -- 172.21.15.26:0/1841620057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc01c7f0 con 0x556c05d683a0 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 -- 172.21.15.26:0/1841620057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 msgr2=0x556c05d68780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d68780 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00f550 tx=0x7f5adc014000 comp rx=0 tx=0).stop 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 -- 172.21.15.26:0/1841620057 shutdown_connections 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d68780 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d632e0 0x556c05d6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 --2- 172.21.15.26:0/1841620057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d62920 0x556c05d62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 -- 172.21.15.26:0/1841620057 >> 172.21.15.26:0/1841620057 conn(0x556c05c8e2c0 msgr2=0x556c05d5d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 -- 172.21.15.26:0/1841620057 shutdown_connections 2024-05-20T21:07:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 -- 172.21.15.26:0/1841620057 wait complete. 2024-05-20T21:07:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 -- start start 2024-05-20T21:07:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.030+0000 7f5af1affb80 1 Processor -- start 2024-05-20T21:07:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.031+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d6edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.031+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d632e0 0x556c05d6c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.031+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d683a0 0x556c05d6c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5af1affb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556c05bb7020 con 0x556c05d632e0 2024-05-20T21:07:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5af1affb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556c05c18e20 con 0x556c05d683a0 2024-05-20T21:07:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5af1affb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c05d6f760 con 0x556c05d62920 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5af1affb80 10 client.0 _ll_register_callbacks cb 0x556c05c4b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d6edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d6edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41494/0 (socket says 172.21.15.26:41494) 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 -- 172.21.15.26:0/2188521663 learned_addr learned my addr 172.21.15.26:0/2188521663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae4804700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d632e0 0x556c05d6c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5806700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d683a0 0x556c05d6c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 -- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d683a0 msgr2=0x556c05d6c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d683a0 0x556c05d6c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 -- 172.21.15.26:0/2188521663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d632e0 msgr2=0x556c05d6c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d632e0 0x556c05d6c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 -- 172.21.15.26:0/2188521663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c05c00950 con 0x556c05d62920 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5806700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d683a0 0x556c05d6c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ae5005700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d6edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00fe20 tx=0x7f5adc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db43d45f1753a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.032+0000 7f5ac8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/2188521663 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc005240 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5adc01e880 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/2188521663 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5adc0053e0 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/2188521663 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5adc0055e0 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5af1affb80 10 client.27835 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c05c6a7e0 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.033+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/2188521663 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5adc01e880 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/2188521663 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5adc038020 con 0x556c05d62920 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5af1affb80 10 client.27835 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5af1affb80 10 client.27835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5af1affb80 20 client.27835 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5af1affb80 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d80560 0x556c05d82950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5af1affb80 4 client.27835 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c05cc2d40 con 0x556c05d80560 2024-05-20T21:07:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5ae4804700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d80560 0x556c05d82950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5ae4804700 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d80560 0x556c05d82950 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19db291e9f651c23 server_cookie=9f818455e22ab204 in_seq=0 out_seq=0 2024-05-20T21:07:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.034+0000 7f5ac8ff9700 10 client.27835 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/2188521663 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556c05cc2d40 con 0x556c05d80560 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5ac8ff9700 10 client.27835 handle_command_reply: tid=0 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 client.27835 shutdown 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 msgr2=0x556c05d6edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d6edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00fe20 tx=0x7f5adc009f90 comp rx=0 tx=0).stop 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 shutdown_connections 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d80560 0x556c05d82950 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d683a0 0x556c05d6c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d6edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 --2- 172.21.15.26:0/2188521663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d632e0 0x556c05d6c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 >> 172.21.15.26:0/2188521663 conn(0x556c05c8e2c0 msgr2=0x556c05d5cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:01.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5ac8ff9700 0 client.27835 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:01.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 shutdown_connections 2024-05-20T21:07:01.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 1 -- 172.21.15.26:0/2188521663 wait complete. 2024-05-20T21:07:01.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.036+0000 7f5af1affb80 20 client.27835 trim_cache size 0 max 16384 2024-05-20T21:07:01.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 Processor -- start 2024-05-20T21:07:01.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 -- start start 2024-05-20T21:07:01.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d5d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01ae2f0 con 0x556c05d683a0 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01abb80 con 0x556c05d632e0 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.058+0000 7f5af1affb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01a99d0 con 0x556c05d62920 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae5806700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae5806700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45876/0 (socket says 172.21.15.26:45876) 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d6dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54900/0 (socket says 172.21.15.26:54900) 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae5806700 1 -- 172.21.15.26:0/3072310891 learned_addr learned my addr 172.21.15.26:0/3072310891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae5005700 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d5d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 -- 172.21.15.26:0/3072310891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 msgr2=0x556c05d5d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d5d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 -- 172.21.15.26:0/3072310891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 msgr2=0x556c05d85590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d85590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 -- 172.21.15.26:0/3072310891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c05d6e9f0 con 0x556c05d632e0 2024-05-20T21:07:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae5806700 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d85590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5ae4804700 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d6dff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00e0e00 tx=0x7f5ad00e0960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a3471050910593 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5acaffd700 1 -- 172.21.15.26:0/3072310891 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00aad40 con 0x556c05d632e0 2024-05-20T21:07:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5acaffd700 1 -- 172.21.15.26:0/3072310891 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad00aa640 con 0x556c05d632e0 2024-05-20T21:07:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.059+0000 7f5acaffd700 1 -- 172.21.15.26:0/3072310891 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00a68d0 con 0x556c05d632e0 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 -- 172.21.15.26:0/3072310891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 msgr2=0x556c05d6dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d6dff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00e0e00 tx=0x7f5ad00e0960 comp rx=0 tx=0).stop 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 -- 172.21.15.26:0/3072310891 shutdown_connections 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d85590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d5d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 --2- 172.21.15.26:0/3072310891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d6dff0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 -- 172.21.15.26:0/3072310891 >> 172.21.15.26:0/3072310891 conn(0x556c05c8e2c0 msgr2=0x556c05d6d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 -- 172.21.15.26:0/3072310891 shutdown_connections 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.060+0000 7f5af1affb80 1 -- 172.21.15.26:0/3072310891 wait complete. 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 -- start start 2024-05-20T21:07:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 Processor -- start 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d85b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d83f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01aa040 con 0x556c05d683a0 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01ab4b0 con 0x556c05d632e0 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5af1affb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01aa710 con 0x556c05d62920 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5ae5806700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.061+0000 7f5ae4804700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d83f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d84450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45884/0 (socket says 172.21.15.26:45884) 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae4804700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d83f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54904/0 (socket says 172.21.15.26:54904) 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 -- 172.21.15.26:0/881646499 learned_addr learned my addr 172.21.15.26:0/881646499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5af1affb80 10 client.0 _ll_register_callbacks cb 0x556c05c4b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5005700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d85b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 -- 172.21.15.26:0/881646499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 msgr2=0x556c05d85b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d85b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 -- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 msgr2=0x556c05d83f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d83f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 -- 172.21.15.26:0/881646499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c05d84ba0 con 0x556c05d683a0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae4804700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d83f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ae5806700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d84450 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8007fc0 tx=0x7f5ad8009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb8b3151114108 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ac8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/881646499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8003c00 con 0x556c05d683a0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/881646499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5ad8003da0 con 0x556c05d683a0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ad00aa640 con 0x556c05d683a0 2024-05-20T21:07:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/881646499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ad8017820 con 0x556c05d683a0 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5af1affb80 10 client.18744 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.062+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c05d78ae0 con 0x556c05d683a0 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/881646499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad00aa640 con 0x556c05d683a0 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/881646499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5ad8004530 con 0x556c05d683a0 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5af1affb80 10 client.18744 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5af1affb80 10 client.18744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5af1affb80 20 client.18744 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5af1affb80 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d79220 0x556c05d7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5af1affb80 4 client.18744 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556c05cbc520 con 0x556c05d79220 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.063+0000 7f5ae5005700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d79220 0x556c05d7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.064+0000 7f5ac8ff9700 10 client.18744 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.064+0000 7f5ae5005700 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d79220 0x556c05d7b640 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad82834df0f1b82 server_cookie=5e35d7ae48f00b2d in_seq=0 out_seq=0 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5ac8ff9700 1 -- 172.21.15.26:0/881646499 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556c05cbc520 con 0x556c05d79220 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5ac8ff9700 10 client.18744 handle_command_reply: tid=0 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 client.18744 shutdown 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 msgr2=0x556c05d84450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d84450 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8007fc0 tx=0x7f5ad8009350 comp rx=0 tx=0).stop 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 shutdown_connections 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556c05d79220 0x556c05d7b640 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c05d683a0 0x556c05d84450 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c05d62920 0x556c05d85b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5ac8ff9700 0 client.18744 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 --2- 172.21.15.26:0/881646499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c05d632e0 0x556c05d83f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 >> 172.21.15.26:0/881646499 conn(0x556c05c8e2c0 msgr2=0x556c05d6b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 shutdown_connections 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.065+0000 7f5af1affb80 1 -- 172.21.15.26:0/881646499 wait complete. 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5af1affb80 20 client.18744 trim_cache size 0 max 16384 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "5db34e4b-76df-4852-ae24-0a074cee11ee", 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:07:01.070 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5acc0b2880 msgr2=0x7f5acc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5acc0b2880 0x7f5acc0b4d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0107af0 tx=0x7f5adc009260 comp rx=0 tx=0).stop 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 msgr2=0x7f5ae019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 0x7f5ae019fd10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad000f990 comp rx=0 tx=0).stop 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 shutdown_connections 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5acc0b2880 0x7f5acc0b4d40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0102430 0x7f5ae019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae0108510 0x7f5ae01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 --2- 172.21.15.26:0/2734015254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ae0102df0 0x7f5ae019fd10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.066+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 >> 172.21.15.26:0/2734015254 conn(0x7f5ae00793b0 msgr2=0x7f5ae006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.067+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 shutdown_connections 2024-05-20T21:07:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:01.067+0000 7f5ae7269700 1 -- 172.21.15.26:0/2734015254 wait complete. 2024-05-20T21:07:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:01 smithi055 ceph-mon[35213]: pgmap v716: 97 pgs: 97 active+clean; 30 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-05-20T21:07:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:01 smithi026 ceph-mon[31990]: pgmap v716: 97 pgs: 97 active+clean; 30 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-05-20T21:07:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:01 smithi144 ceph-mon[35660]: pgmap v716: 97 pgs: 97 active+clean; 30 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-05-20T21:07:01.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 236473 2.15 MB/sec execute 427 sec latency 18.149 ms 2024-05-20T21:07:01.985 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:07:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:02.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:02 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:07:02.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:02 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:02.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:02 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:07:02.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:02 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:02.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:02 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:07:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:02 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:07:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:02 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:02 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:07:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:02 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:02 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:07:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:02 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:07:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:02 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:02 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:07:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:02 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:02 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:07:02.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 237176 2.16 MB/sec execute 428 sec latency 18.163 ms 2024-05-20T21:07:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:03 smithi055 ceph-mon[35213]: pgmap v717: 97 pgs: 97 active+clean; 27 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-20T21:07:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:03 smithi026 ceph-mon[31990]: pgmap v717: 97 pgs: 97 active+clean; 27 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-20T21:07:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:03 smithi144 ceph-mon[35660]: pgmap v717: 97 pgs: 97 active+clean; 27 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-20T21:07:03.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 237816 2.16 MB/sec execute 429 sec latency 17.849 ms 2024-05-20T21:07:04.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 -- 172.21.15.26:0/3433461251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 msgr2=0x7efccc110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 --2- 172.21.15.26:0/3433461251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc110440 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b0d0 tx=0x7efcc400b3e0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 -- 172.21.15.26:0/3433461251 shutdown_connections 2024-05-20T21:07:04.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 --2- 172.21.15.26:0/3433461251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc110440 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 --2- 172.21.15.26:0/3433461251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efccc109670 0x7efccc109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 --2- 172.21.15.26:0/3433461251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc1035d0 0x7efccc103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 -- 172.21.15.26:0/3433461251 >> 172.21.15.26:0/3433461251 conn(0x7efccc07a730 msgr2=0x7efccc07ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:04.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 -- 172.21.15.26:0/3433461251 shutdown_connections 2024-05-20T21:07:04.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.187+0000 7efcd2057700 1 -- 172.21.15.26:0/3433461251 wait complete. 2024-05-20T21:07:04.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 Processor -- start 2024-05-20T21:07:04.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 -- start start 2024-05-20T21:07:04.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc1035d0 0x7efccc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efccc109670 0x7efccc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efccc113620 con 0x7efccc103f90 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1134a0 con 0x7efccc109670 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efcd2057700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1137a0 con 0x7efccc1035d0 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efccbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efccc109670 0x7efccc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.188+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45898/0 (socket says 172.21.15.26:45898) 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 -- 172.21.15.26:0/3830242308 learned_addr learned my addr 172.21.15.26:0/3830242308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc1035d0 0x7efccc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 -- 172.21.15.26:0/3830242308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc1035d0 msgr2=0x7efccc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc1035d0 0x7efccc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 -- 172.21.15.26:0/3830242308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efccc109670 msgr2=0x7efccc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efccc109670 0x7efccc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 -- 172.21.15.26:0/3830242308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc19a0a0 con 0x7efccc103f90 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efccaffd700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc19fcc0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7efcc000d950 tx=0x7efcc000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efcc8ff9700 1 -- 172.21.15.26:0/3830242308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0016070 con 0x7efccc103f90 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc19a390 con 0x7efccc103f90 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efcc8ff9700 1 -- 172.21.15.26:0/3830242308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc000fd20 con 0x7efccc103f90 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.189+0000 7efcc8ff9700 1 -- 172.21.15.26:0/3830242308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc000e3b0 con 0x7efccc103f90 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.190+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc075770 con 0x7efccc103f90 2024-05-20T21:07:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.192+0000 7efcc8ff9700 1 -- 172.21.15.26:0/3830242308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efcc0019070 con 0x7efccc103f90 2024-05-20T21:07:04.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.194+0000 7efcc8ff9700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efcb40b2900 0x7efcb40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.194+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efcb40b2900 0x7efcb40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.194+0000 7efcc8ff9700 1 -- 172.21.15.26:0/3830242308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efcc00e1d30 con 0x7efccc103f90 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.194+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efcb40b2900 0x7efcb40b4dc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efccc19aa80 tx=0x7efcbc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.194+0000 7efcdc8edb80 1 Processor -- start 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.194+0000 7efcdc8edb80 1 -- start start 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b85c860 0x56387b85cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b860940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b860f60 0x56387b8613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56387b7bbc90 con 0x56387b85d210 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56387b7ab830 con 0x56387b860f60 2024-05-20T21:07:04.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56387b7af870 con 0x56387b85c860 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b860940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b860f60 0x56387b8613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b860940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45912/0 (socket says 172.21.15.26:45912) 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccaffd700 1 -- 172.21.15.26:0/3466090023 learned_addr learned my addr 172.21.15.26:0/3466090023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b85c860 0x56387b85cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccbfff700 1 -- 172.21.15.26:0/3466090023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b85c860 msgr2=0x56387b85cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccbfff700 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b85c860 0x56387b85cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccbfff700 1 -- 172.21.15.26:0/3466090023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 msgr2=0x56387b860940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccbfff700 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b860940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccbfff700 1 -- 172.21.15.26:0/3466090023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387b6fb8d0 con 0x56387b860f60 2024-05-20T21:07:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.195+0000 7efccaffd700 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b860940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:04.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.196+0000 7efccbfff700 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b860f60 0x56387b8613a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b770 tx=0x7efcc400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7eb9b6f1aa77485 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:04.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.196+0000 7efcb1ffb700 1 -- 172.21.15.26:0/3466090023 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc40092e0 con 0x56387b860f60 2024-05-20T21:07:04.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.196+0000 7efcb1ffb700 1 -- 172.21.15.26:0/3466090023 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc4009480 con 0x56387b860f60 2024-05-20T21:07:04.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.196+0000 7efcb1ffb700 1 -- 172.21.15.26:0/3466090023 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc401cb60 con 0x56387b860f60 2024-05-20T21:07:04.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3466090023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b860f60 msgr2=0x56387b8613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b860f60 0x56387b8613a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b770 tx=0x7efcc400b040 comp rx=0 tx=0).stop 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3466090023 shutdown_connections 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b860f60 0x56387b8613a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b85c860 0x56387b85cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3466090023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b860940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3466090023 >> 172.21.15.26:0/3466090023 conn(0x56387b78a080 msgr2=0x56387b856e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3466090023 shutdown_connections 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3466090023 wait complete. 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 -- start start 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.197+0000 7efcdc8edb80 1 Processor -- start 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b868390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b865690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b865bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56387b6b1fa0 con 0x56387b85d210 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56387b74d7c0 con 0x56387b85c860 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56387b869060 con 0x56387b860f60 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efcdc8edb80 10 client.0 _ll_register_callbacks cb 0x56387b746920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b868390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b865690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b868390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54932/0 (socket says 172.21.15.26:54932) 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 -- 172.21.15.26:0/3261257284 learned_addr learned my addr 172.21.15.26:0/3261257284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccbfff700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b865bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 -- 172.21.15.26:0/3261257284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 msgr2=0x56387b865bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b865bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 -- 172.21.15.26:0/3261257284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 msgr2=0x56387b865690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b865690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccb7fe700 1 -- 172.21.15.26:0/3261257284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387b74a690 con 0x56387b85c860 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.198+0000 7efccaffd700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b865690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efccb7fe700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b868390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efcbc01dbd0 tx=0x7efcbc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de2641b7cab39b8c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcaaffd700 1 -- 172.21.15.26:0/3261257284 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc02b060 con 0x56387b85c860 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcaaffd700 1 -- 172.21.15.26:0/3261257284 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efcbc01c810 con 0x56387b85c860 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56387b8289e0 con 0x56387b85c860 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcaaffd700 1 -- 172.21.15.26:0/3261257284 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcbc01ca10 con 0x56387b85c860 2024-05-20T21:07:04.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcdc8edb80 10 client.28058 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.199+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56387b873770 con 0x56387b85c860 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcaaffd700 1 -- 172.21.15.26:0/3261257284 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcbc026410 con 0x56387b85c860 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcaaffd700 1 -- 172.21.15.26:0/3261257284 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7efcbc02f030 con 0x56387b85c860 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcdc8edb80 10 client.28058 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcdc8edb80 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcdc8edb80 20 client.28058 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b875db0 0x56387b8781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcdc8edb80 4 client.28058 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56387b8289e0 con 0x56387b875db0 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efccaffd700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b875db0 0x56387b8781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.200+0000 7efcaaffd700 10 client.28058 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.201+0000 7efccaffd700 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b875db0 0x56387b8781a0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea728432dec4125d server_cookie=2ee97c928a3dad44 in_seq=0 out_seq=0 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.201+0000 7efcaaffd700 1 -- 172.21.15.26:0/3261257284 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56387b8289e0 con 0x56387b875db0 2024-05-20T21:07:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.201+0000 7efcaaffd700 10 client.28058 handle_command_reply: tid=0 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.201+0000 7efcdc8edb80 1 client.28058 shutdown 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.201+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 msgr2=0x56387b868390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.201+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b868390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efcbc01dbd0 tx=0x7efcbc0047c0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 shutdown_connections 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b875db0 0x56387b8781a0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b865bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b868390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/3261257284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b865690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcaaffd700 0 client.28058 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 >> 172.21.15.26:0/3261257284 conn(0x56387b78a080 msgr2=0x56387b856510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 shutdown_connections 2024-05-20T21:07:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 1 -- 172.21.15.26:0/3261257284 wait complete. 2024-05-20T21:07:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.202+0000 7efcdc8edb80 20 client.28058 trim_cache size 0 max 16384 2024-05-20T21:07:04.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 Processor -- start 2024-05-20T21:07:04.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 -- start start 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b85ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b856c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b87da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1ae300 con 0x56387b85d210 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1abb90 con 0x56387b85c860 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1a99e0 con 0x56387b860f60 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b856c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efccaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b856c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45936/0 (socket says 172.21.15.26:45936) 2024-05-20T21:07:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efccaffd700 1 -- 172.21.15.26:0/4203964505 learned_addr learned my addr 172.21.15.26:0/4203964505 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.226+0000 7efccb7fe700 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b85ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccbfff700 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b87da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccaffd700 1 -- 172.21.15.26:0/4203964505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 msgr2=0x56387b87da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccaffd700 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b87da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccaffd700 1 -- 172.21.15.26:0/4203964505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 msgr2=0x56387b85ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccaffd700 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b85ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccaffd700 1 -- 172.21.15.26:0/4203964505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387b8571d0 con 0x56387b85d210 2024-05-20T21:07:04.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efccaffd700 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b856c90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7efcc00a6500 tx=0x7efcc00e1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f82093e8652be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:04.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efcb17fa700 1 -- 172.21.15.26:0/4203964505 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00c3070 con 0x56387b85d210 2024-05-20T21:07:04.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efcb17fa700 1 -- 172.21.15.26:0/4203964505 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc0010460 con 0x56387b85d210 2024-05-20T21:07:04.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.227+0000 7efcb17fa700 1 -- 172.21.15.26:0/4203964505 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0014030 con 0x56387b85d210 2024-05-20T21:07:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 -- 172.21.15.26:0/4203964505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 msgr2=0x56387b856c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b856c90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7efcc00a6500 tx=0x7efcc00e1040 comp rx=0 tx=0).stop 2024-05-20T21:07:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 -- 172.21.15.26:0/4203964505 shutdown_connections 2024-05-20T21:07:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b87da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85c860 0x56387b85ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/4203964505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85d210 0x56387b856c90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 -- 172.21.15.26:0/4203964505 >> 172.21.15.26:0/4203964505 conn(0x56387b78a080 msgr2=0x56387b85bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 -- 172.21.15.26:0/4203964505 shutdown_connections 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.228+0000 7efcdc8edb80 1 -- 172.21.15.26:0/4203964505 wait complete. 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 -- start start 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 Processor -- start 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 0x56387b86c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85d210 0x56387b86c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b86cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1aa050 con 0x56387b85c860 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1ab4c0 con 0x56387b85d210 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1aa720 con 0x56387b860f60 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efcdc8edb80 10 client.0 _ll_register_callbacks cb 0x56387b746920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efccb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 0x56387b86c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efccaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85d210 0x56387b86c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efccb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 0x56387b86c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45946/0 (socket says 172.21.15.26:45946) 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.229+0000 7efccb7fe700 1 -- 172.21.15.26:0/2116022654 learned_addr learned my addr 172.21.15.26:0/2116022654 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccbfff700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b86cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccaffd700 1 -- 172.21.15.26:0/2116022654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 msgr2=0x56387b86cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccaffd700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b86cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccaffd700 1 -- 172.21.15.26:0/2116022654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 msgr2=0x56387b86c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccaffd700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 0x56387b86c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccaffd700 1 -- 172.21.15.26:0/2116022654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387b8dfbb0 con 0x56387b85d210 2024-05-20T21:07:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccb7fe700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 0x56387b86c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efccaffd700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85d210 0x56387b86c840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efcc000b2f0 tx=0x7efcc00a5150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7192102692b2ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcaaffd700 1 -- 172.21.15.26:0/2116022654 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00c8060 con 0x56387b85d210 2024-05-20T21:07:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc0010460 con 0x56387b85d210 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcaaffd700 1 -- 172.21.15.26:0/2116022654 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efcc00ca2a0 con 0x56387b85d210 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcaaffd700 1 -- 172.21.15.26:0/2116022654 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcc0014030 con 0x56387b85d210 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcdc8edb80 10 client.28061 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.230+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56387b8e0ec0 con 0x56387b85d210 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcaaffd700 1 -- 172.21.15.26:0/2116022654 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc0010460 con 0x56387b85d210 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcaaffd700 1 -- 172.21.15.26:0/2116022654 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7efcc00c3b80 con 0x56387b85d210 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcdc8edb80 10 client.28061 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcdc8edb80 10 client.28061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcdc8edb80 20 client.28061 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:04.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b8e1470 0x56387b8e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcdc8edb80 4 client.28061 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56387b7bdbe0 con 0x56387b8e1470 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efccb7fe700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b8e1470 0x56387b8e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efcaaffd700 10 client.28061 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.231+0000 7efccb7fe700 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b8e1470 0x56387b8e3890 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f09182b00335d3 server_cookie=e480a198dfe37c1a in_seq=0 out_seq=0 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcaaffd700 1 -- 172.21.15.26:0/2116022654 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56387b7bdbe0 con 0x56387b8e1470 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcaaffd700 10 client.28061 handle_command_reply: tid=0 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 client.28061 shutdown 2024-05-20T21:07:04.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85d210 msgr2=0x56387b86c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85d210 0x56387b86c840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efcc000b2f0 tx=0x7efcc00a5150 comp rx=0 tx=0).stop 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 shutdown_connections 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56387b8e1470 0x56387b8e3890 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56387b85c860 0x56387b86c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56387b860f60 0x56387b86cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 --2- 172.21.15.26:0/2116022654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56387b85d210 0x56387b86c840 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 >> 172.21.15.26:0/2116022654 conn(0x56387b78a080 msgr2=0x56387b866ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcaaffd700 0 client.28061 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 shutdown_connections 2024-05-20T21:07:04.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 1 -- 172.21.15.26:0/2116022654 wait complete. 2024-05-20T21:07:04.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.232+0000 7efcdc8edb80 20 client.28061 trim_cache size 0 max 16384 2024-05-20T21:07:04.242 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:07:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:07:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:07:04.242 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efcb40b2900 msgr2=0x7efcb40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efcb40b2900 0x7efcb40b4dc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efccc19aa80 tx=0x7efcbc009260 comp rx=0 tx=0).stop 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 msgr2=0x7efccc19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc19fcc0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7efcc000d950 tx=0x7efcc000dc60 comp rx=0 tx=0).stop 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 shutdown_connections 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efccc109670 0x7efccc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc1035d0 0x7efccc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efcb40b2900 0x7efcb40b4dc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 --2- 172.21.15.26:0/3830242308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc103f90 0x7efccc19fcc0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:04.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 >> 172.21.15.26:0/3830242308 conn(0x7efccc07a730 msgr2=0x7efccc102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:04.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 shutdown_connections 2024-05-20T21:07:04.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:04.233+0000 7efcd2057700 1 -- 172.21.15.26:0/3830242308 wait complete. 2024-05-20T21:07:04.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 238253 2.16 MB/sec execute 430 sec latency 11.540 ms 2024-05-20T21:07:05.061 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:07:05.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:07:05.338 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:05 smithi055 ceph-mon[35213]: pgmap v718: 97 pgs: 97 active+clean; 27 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:07:05.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:05 smithi026 ceph-mon[31990]: pgmap v718: 97 pgs: 97 active+clean; 27 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:07:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:05 smithi144 ceph-mon[35660]: pgmap v718: 97 pgs: 97 active+clean; 27 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:07:05.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 238822 2.16 MB/sec execute 431 sec latency 21.113 ms 2024-05-20T21:07:06.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 239524 2.16 MB/sec execute 432 sec latency 18.935 ms 2024-05-20T21:07:07.249 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:07 smithi026 ceph-mon[31990]: pgmap v719: 97 pgs: 97 active+clean; 28 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-05-20T21:07:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 -- 172.21.15.26:0/1921655717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 msgr2=0x7f38a8103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1921655717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a8103a10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3898009990 tx=0x7f3898009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 -- 172.21.15.26:0/1921655717 shutdown_connections 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1921655717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a8103f50 0x7f38a8110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1921655717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a81096a0 0x7f38a8109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1921655717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a8103a10 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 -- 172.21.15.26:0/1921655717 >> 172.21.15.26:0/1921655717 conn(0x7f38a807a6f0 msgr2=0x7f38a807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 -- 172.21.15.26:0/1921655717 shutdown_connections 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.272+0000 7f38b07cc700 1 -- 172.21.15.26:0/1921655717 wait complete. 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 Processor -- start 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 -- start start 2024-05-20T21:07:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a8103f50 0x7f38a819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a81096a0 0x7f38a81998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8113620 con 0x7f38a8103590 2024-05-20T21:07:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81134a0 con 0x7f38a81096a0 2024-05-20T21:07:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38ae568700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38ae568700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45970/0 (socket says 172.21.15.26:45970) 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38b07cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81137a0 con 0x7f38a8103f50 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38ae568700 1 -- 172.21.15.26:0/1461366807 learned_addr learned my addr 172.21.15.26:0/1461366807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38aed69700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a81096a0 0x7f38a81998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38add67700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a8103f50 0x7f38a819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.273+0000 7f38ae568700 1 -- 172.21.15.26:0/1461366807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a8103f50 msgr2=0x7f38a819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38ae568700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a8103f50 0x7f38a819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38ae568700 1 -- 172.21.15.26:0/1461366807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a81096a0 msgr2=0x7f38a81998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38ae568700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a81096a0 0x7f38a81998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38ae568700 1 -- 172.21.15.26:0/1461366807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a819a020 con 0x7f38a8103590 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38ae568700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a819f700 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f38a400d950 tx=0x7f38a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4016070 con 0x7f38a8103590 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38a400fd20 con 0x7f38a8103590 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a400e3b0 con 0x7f38a8103590 2024-05-20T21:07:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a819a310 con 0x7f38a8103590 2024-05-20T21:07:07.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.274+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a8075770 con 0x7f38a8103590 2024-05-20T21:07:07.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.275+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a810ba80 con 0x7f38a8103590 2024-05-20T21:07:07.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.277+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f38a4019070 con 0x7f38a8103590 2024-05-20T21:07:07.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.278+0000 7f389f7fe700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38940b2880 0x7f38940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:07.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.278+0000 7f38add67700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38940b2880 0x7f38940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:07.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.278+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f38a40e1d30 con 0x7f38a8103590 2024-05-20T21:07:07.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.279+0000 7f38add67700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38940b2880 0x7f38940b4d40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f38a807aea0 tx=0x7f3898005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.283+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38a40a99f0 con 0x7f38a8103590 2024-05-20T21:07:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:07 smithi055 ceph-mon[35213]: pgmap v719: 97 pgs: 97 active+clean; 28 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-05-20T21:07:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:07 smithi144 ceph-mon[35660]: pgmap v719: 97 pgs: 97 active+clean; 28 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-05-20T21:07:07.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 240131 2.16 MB/sec execute 433 sec latency 16.846 ms 2024-05-20T21:07:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.592+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38a819adf0 con 0x7f38a8103590 2024-05-20T21:07:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.593+0000 7f389f7fe700 1 -- 172.21.15.26:0/1461366807 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f38a40a92f0 con 0x7f38a8103590 2024-05-20T21:07:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:07:07.592 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:07:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38940b2880 msgr2=0x7f38940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38940b2880 0x7f38940b4d40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f38a807aea0 tx=0x7f3898005fb0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 msgr2=0x7f38a819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a819f700 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f38a400d950 tx=0x7f38a400dc60 comp rx=0 tx=0).stop 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 shutdown_connections 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f38940b2880 0x7f38940b4d40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8103590 0x7f38a819f700 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a81096a0 0x7f38a81998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 --2- 172.21.15.26:0/1461366807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a8103f50 0x7f38a819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 >> 172.21.15.26:0/1461366807 conn(0x7f38a807a6f0 msgr2=0x7f38a8102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 shutdown_connections 2024-05-20T21:07:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:07.596+0000 7f38b07cc700 1 -- 172.21.15.26:0/1461366807 wait complete. 2024-05-20T21:07:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:07:08.348 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:08 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1461366807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:08.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:07:08.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:08 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1461366807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:08.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:08 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1461366807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:08.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 240983 2.18 MB/sec execute 434 sec latency 17.515 ms 2024-05-20T21:07:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:09.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:09 smithi026 ceph-mon[31990]: pgmap v720: 97 pgs: 97 active+clean; 28 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-05-20T21:07:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:09 smithi055 ceph-mon[35213]: pgmap v720: 97 pgs: 97 active+clean; 28 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-05-20T21:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 -- 172.21.15.26:0/3301825813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 msgr2=0x7f7984105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 --2- 172.21.15.26:0/3301825813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984105920 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7974009860 tx=0x7f797402a340 comp rx=0 tx=0).stop 2024-05-20T21:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 -- 172.21.15.26:0/3301825813 shutdown_connections 2024-05-20T21:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 --2- 172.21.15.26:0/3301825813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984108790 0x7f798410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 --2- 172.21.15.26:0/3301825813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984105e60 0x7f7984108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 --2- 172.21.15.26:0/3301825813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984105920 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 -- 172.21.15.26:0/3301825813 >> 172.21.15.26:0/3301825813 conn(0x7f79840fcf20 msgr2=0x7f79840ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 -- 172.21.15.26:0/3301825813 shutdown_connections 2024-05-20T21:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.528+0000 7f79899f7640 1 -- 172.21.15.26:0/3301825813 wait complete. 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 Processor -- start 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 -- start start 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984105e60 0x7f79841014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984108790 0x7f7984101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f798410e4b0 con 0x7f7984103530 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f798410e330 con 0x7f7984108790 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f79899f7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f798410e630 con 0x7f7984105e60 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45978/0 (socket says 172.21.15.26:45978) 2024-05-20T21:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.529+0000 7f7982ffd640 1 -- 172.21.15.26:0/2092734151 learned_addr learned my addr 172.21.15.26:0/2092734151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79837fe640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984108790 0x7f7984101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79827fc640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984105e60 0x7f79841014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f7982ffd640 1 -- 172.21.15.26:0/2092734151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984105e60 msgr2=0x7f79841014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f7982ffd640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984105e60 0x7f79841014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f7982ffd640 1 -- 172.21.15.26:0/2092734151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984108790 msgr2=0x7f7984101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f7982ffd640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984108790 0x7f7984101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f7982ffd640 1 -- 172.21.15.26:0/2092734151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79841022b0 con 0x7f7984103530 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79837fe640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984108790 0x7f7984101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f7982ffd640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984102ea0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7974005cd0 tx=0x7f797402ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79889f5640 1 -- 172.21.15.26:0/2092734151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974042070 con 0x7f7984103530 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79889f5640 1 -- 172.21.15.26:0/2092734151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7974004050 con 0x7f7984103530 2024-05-20T21:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841a2c40 con 0x7f7984103530 2024-05-20T21:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79889f5640 1 -- 172.21.15.26:0/2092734151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974037040 con 0x7f7984103530 2024-05-20T21:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.530+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a3190 con 0x7f7984103530 2024-05-20T21:07:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 Processor -- start 2024-05-20T21:07:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 -- start start 2024-05-20T21:07:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a6cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a6f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556a89d8c30 con 0x5556a8a680b0 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5556a89c87d0 con 0x5556a8a676c0 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f798c7ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556a89cc810 con 0x5556a8a6d220 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f79889f5640 1 -- 172.21.15.26:0/2092734151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7974045070 con 0x7f7984103530 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f79837fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a6cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f79827fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a6f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f79837fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a67ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55004/0 (socket says 172.21.15.26:55004) 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.533+0000 7f79837fe640 1 -- 172.21.15.26:0/2130692640 learned_addr learned my addr 172.21.15.26:0/2130692640 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79827fc640 1 -- 172.21.15.26:0/2130692640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 msgr2=0x5556a8a67ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79827fc640 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a67ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79827fc640 1 -- 172.21.15.26:0/2130692640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 msgr2=0x5556a8a6cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79827fc640 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a6cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79827fc640 1 -- 172.21.15.26:0/2130692640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a86dd210 con 0x5556a8a6d220 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79827fc640 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a6f6b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7978009530 tx=0x7f7978011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457084a8a669e008 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f7969ffb640 1 -- 172.21.15.26:0/2130692640 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978004020 con 0x5556a8a6d220 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f7969ffb640 1 -- 172.21.15.26:0/2130692640 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79780041c0 con 0x5556a8a6d220 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f7969ffb640 1 -- 172.21.15.26:0/2130692640 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978019690 con 0x5556a8a6d220 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79889f5640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f796c0b28a0 0x7f796c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79837fe640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f796c0b28a0 0x7f796c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.534+0000 7f79889f5640 1 -- 172.21.15.26:0/2092734151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7974101b60 con 0x7f7984103530 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 -- 172.21.15.26:0/2130692640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 msgr2=0x5556a8a6f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a6f6b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7978009530 tx=0x7f7978011810 comp rx=0 tx=0).stop 2024-05-20T21:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f79837fe640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f796c0b28a0 0x7f796c0b4d60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5556a8a68910 tx=0x7f797c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 -- 172.21.15.26:0/2130692640 shutdown_connections 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a6f6b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a6cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2130692640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 -- 172.21.15.26:0/2130692640 >> 172.21.15.26:0/2130692640 conn(0x5556a8a62d30 msgr2=0x5556a8a63130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 -- 172.21.15.26:0/2130692640 shutdown_connections 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 -- 172.21.15.26:0/2130692640 wait complete. 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.535+0000 7f798c7ec740 1 -- start start 2024-05-20T21:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 Processor -- start 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 0x5556a8a71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a680b0 0x5556a8a71cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556a86ae790 con 0x5556a8a676c0 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5556a8984890 con 0x5556a8a680b0 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556a8a701d0 con 0x5556a8a6d220 2024-05-20T21:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f798c7ec740 10 client.0 _ll_register_callbacks cb 0x5556a86dca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 0x5556a8a71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 0x5556a8a71780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46000/0 (socket says 172.21.15.26:46000) 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 -- 172.21.15.26:0/2651230269 learned_addr learned my addr 172.21.15.26:0/2651230269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f79827fc640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a680b0 0x5556a8a71cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f79837fe640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 -- 172.21.15.26:0/2651230269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 msgr2=0x5556a8a78d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a78d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 -- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a680b0 msgr2=0x5556a8a71cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a680b0 0x5556a8a71cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f7982ffd640 1 -- 172.21.15.26:0/2651230269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a87a9620 con 0x5556a8a676c0 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.536+0000 7f79837fe640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7982ffd640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 0x5556a8a71780 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f79740c9100 tx=0x7f79740045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51755c671a36c4ed server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7962ffd640 1 -- 172.21.15.26:0/2651230269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79740c8ac0 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7962ffd640 1 -- 172.21.15.26:0/2651230269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f79740c5760 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79780041c0 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7962ffd640 1 -- 172.21.15.26:0/2651230269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79740c88e0 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 10 client.18780 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556a8a7bce0 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7962ffd640 1 -- 172.21.15.26:0/2651230269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79780041c0 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f7962ffd640 1 -- 172.21.15.26:0/2651230269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f79740054d0 con 0x5556a8a676c0 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 10 client.18780 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 10 client.18780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 20 client.18780 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8a7d6c0 0x5556a8a7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 4 client.18780 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.537+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556a89dab80 con 0x5556a8a7d6c0 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.538+0000 7f79827fc640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8a7d6c0 0x5556a8a7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.538+0000 7f7962ffd640 10 client.18780 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.538+0000 7f79827fc640 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8a7d6c0 0x5556a8a7fb30 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91b6cb9aa93b8e5 server_cookie=92398498b37cbf1d in_seq=0 out_seq=0 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f7962ffd640 1 -- 172.21.15.26:0/2651230269 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5556a89dab80 con 0x5556a8a7d6c0 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f7962ffd640 10 client.18780 handle_command_reply: tid=0 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 client.18780 shutdown 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 msgr2=0x5556a8a71780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 0x5556a8a71780 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f79740c9100 tx=0x7f79740045c0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 shutdown_connections 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8a7d6c0 0x5556a8a7fb30 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a6d220 0x5556a8a78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a680b0 0x5556a8a71cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 --2- 172.21.15.26:0/2651230269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a676c0 0x5556a8a71780 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f7962ffd640 0 client.18780 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 >> 172.21.15.26:0/2651230269 conn(0x5556a8a62d30 msgr2=0x5556a8a61fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 shutdown_connections 2024-05-20T21:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 1 -- 172.21.15.26:0/2651230269 wait complete. 2024-05-20T21:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.539+0000 7f798c7ec740 20 client.18780 trim_cache size 0 max 16384 2024-05-20T21:07:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:09 smithi144 ceph-mon[35660]: pgmap v720: 97 pgs: 97 active+clean; 28 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-05-20T21:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 Processor -- start 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 -- start start 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a676c0 0x5556a8a66350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a6d220 0x5556a8a5dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ae910 con 0x5556a8a680b0 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ac1e0 con 0x5556a8a6d220 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f798c7ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79841aa030 con 0x5556a8a676c0 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.556+0000 7f79827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a78cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46014/0 (socket says 172.21.15.26:46014) 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 -- 172.21.15.26:0/712389816 learned_addr learned my addr 172.21.15.26:0/712389816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79837fe640 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a6d220 0x5556a8a5dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f7982ffd640 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a676c0 0x5556a8a66350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 -- 172.21.15.26:0/712389816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a676c0 msgr2=0x5556a8a66350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a676c0 0x5556a8a66350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 -- 172.21.15.26:0/712389816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a6d220 msgr2=0x5556a8a5dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a6d220 0x5556a8a5dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 -- 172.21.15.26:0/712389816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a8a68490 con 0x5556a8a680b0 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79827fc640 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a78cc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x5556a8a63c00 tx=0x7f7978009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4686f2a1712c1620 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79697fa640 1 -- 172.21.15.26:0/712389816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978005cb0 con 0x5556a8a680b0 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79697fa640 1 -- 172.21.15.26:0/712389816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7978032070 con 0x5556a8a680b0 2024-05-20T21:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.557+0000 7f79697fa640 1 -- 172.21.15.26:0/712389816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79780054d0 con 0x5556a8a680b0 2024-05-20T21:07:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 -- 172.21.15.26:0/712389816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 msgr2=0x5556a8a78cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a78cc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x5556a8a63c00 tx=0x7f7978009790 comp rx=0 tx=0).stop 2024-05-20T21:07:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 -- 172.21.15.26:0/712389816 shutdown_connections 2024-05-20T21:07:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a6d220 0x5556a8a5dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a680b0 0x5556a8a78cc0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 --2- 172.21.15.26:0/712389816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a676c0 0x5556a8a66350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 -- 172.21.15.26:0/712389816 >> 172.21.15.26:0/712389816 conn(0x5556a8a62d30 msgr2=0x5556a8a65c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 -- 172.21.15.26:0/712389816 shutdown_connections 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 -- 172.21.15.26:0/712389816 wait complete. 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.558+0000 7f798c7ec740 1 -- start start 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 Processor -- start 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a680b0 0x5556a8a766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 0x5556a8a776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79841aa6a0 con 0x5556a8a6d220 2024-05-20T21:07:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f79841abb10 con 0x5556a8a676c0 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79841aad70 con 0x5556a8a680b0 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f798c7ec740 10 client.0 _ll_register_callbacks cb 0x7f79740097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 0x5556a8a776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 0x5556a8a776a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46026/0 (socket says 172.21.15.26:46026) 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 -- 172.21.15.26:0/4157377575 learned_addr learned my addr 172.21.15.26:0/4157377575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79827fc640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a680b0 0x5556a8a766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 -- 172.21.15.26:0/4157377575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a680b0 msgr2=0x5556a8a766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a680b0 0x5556a8a766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 -- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 msgr2=0x5556a8a761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f79837fe640 1 -- 172.21.15.26:0/4157377575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a8a77c90 con 0x5556a8a6d220 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.559+0000 7f7982ffd640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f79837fe640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 0x5556a8a776a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f797c011d00 tx=0x7f797c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ea1d9cecd64cba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7962ffd640 1 -- 172.21.15.26:0/4157377575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c01cba0 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7962ffd640 1 -- 172.21.15.26:0/4157377575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f797c01cd40 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7978032070 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7962ffd640 1 -- 172.21.15.26:0/4157377575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f797c029440 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 10 client.18786 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556a8a79070 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7962ffd640 1 -- 172.21.15.26:0/4157377575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7978032070 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7962ffd640 1 -- 172.21.15.26:0/4157377575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f797c033530 con 0x5556a8a6d220 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 10 client.18786 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 10 client.18786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 20 client.18786 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8ae8b40 0x5556a8aeaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 4 client.18786 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:07:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556a89d4360 con 0x5556a8ae8b40 2024-05-20T21:07:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.560+0000 7f7982ffd640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8ae8b40 0x5556a8aeaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.561+0000 7f7962ffd640 10 client.18786 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.561+0000 7f7982ffd640 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8ae8b40 0x5556a8aeaf60 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92af136e7d8f8b72 server_cookie=91db2f5dd29b2c54 in_seq=0 out_seq=0 2024-05-20T21:07:09.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 241486 2.18 MB/sec execute 435 sec latency 16.264 ms 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.599+0000 7f7962ffd640 1 -- 172.21.15.26:0/4157377575 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5556a89d4360 con 0x5556a8ae8b40 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.599+0000 7f7962ffd640 10 client.18786 handle_command_reply: tid=0 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.599+0000 7f798c7ec740 1 client.18786 shutdown 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 msgr2=0x5556a8a776a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 0x5556a8a776a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f797c011d00 tx=0x7f797c004ab0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 shutdown_connections 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556a8ae8b40 0x5556a8aeaf60 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556a8a6d220 0x5556a8a776a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556a8a680b0 0x5556a8a766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f7962ffd640 0 client.18786 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 --2- 172.21.15.26:0/4157377575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556a8a676c0 0x5556a8a761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 >> 172.21.15.26:0/4157377575 conn(0x5556a8a62d30 msgr2=0x5556a8a71300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 shutdown_connections 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 1 -- 172.21.15.26:0/4157377575 wait complete. 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.600+0000 7f798c7ec740 20 client.18786 trim_cache size 0 max 16384 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:07:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:07:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f796c0b28a0 msgr2=0x7f796c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f796c0b28a0 0x7f796c0b4d60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5556a8a68910 tx=0x7f797c009500 comp rx=0 tx=0).stop 2024-05-20T21:07:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 msgr2=0x7f7984102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984102ea0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7974005cd0 tx=0x7f797402ae50 comp rx=0 tx=0).stop 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 shutdown_connections 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f796c0b28a0 0x7f796c0b4d60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7984108790 0x7f7984101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984105e60 0x7f79841014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 --2- 172.21.15.26:0/2092734151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984103530 0x7f7984102ea0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 >> 172.21.15.26:0/2092734151 conn(0x7f79840fcf20 msgr2=0x7f7984104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 shutdown_connections 2024-05-20T21:07:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:09.601+0000 7f79899f7640 1 -- 172.21.15.26:0/2092734151 wait complete. 2024-05-20T21:07:10.288 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:10 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:07:10.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:10 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.537+0000 7f4a109e0700 1 -- 172.21.15.26:0/1165311431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 msgr2=0x7f4a0c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.537+0000 7f4a109e0700 1 --2- 172.21.15.26:0/1165311431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 0x7f4a0c108900 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009990 tx=0x7f49fc009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 -- 172.21.15.26:0/1165311431 shutdown_connections 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 --2- 172.21.15.26:0/1165311431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 0x7f4a0c108900 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 --2- 172.21.15.26:0/1165311431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102e00 0x7f4a0c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 --2- 172.21.15.26:0/1165311431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102440 0x7f4a0c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 -- 172.21.15.26:0/1165311431 >> 172.21.15.26:0/1165311431 conn(0x7f4a0c0793f0 msgr2=0x7f4a0c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 -- 172.21.15.26:0/1165311431 shutdown_connections 2024-05-20T21:07:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.538+0000 7f4a109e0700 1 -- 172.21.15.26:0/1165311431 wait complete. 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 Processor -- start 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 -- start start 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102440 0x7f4a0c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 0x7f4a0c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 0x7f4a0c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c112480 con 0x7f4a0c108520 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c112300 con 0x7f4a0c102e00 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a109e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c112600 con 0x7f4a0c102440 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102440 0x7f4a0c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a09d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 0x7f4a0c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102440 0x7f4a0c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49330/0 (socket says 172.21.15.26:49330) 2024-05-20T21:07:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a09d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 0x7f4a0c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55446/0 (socket says 172.21.15.26:55446) 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4175202994 learned_addr learned my addr 172.21.15.26:0/4175202994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.539+0000 7f4a0ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 0x7f4a0c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4175202994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102440 msgr2=0x7f4a0c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102440 0x7f4a0c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 msgr2=0x7f4a0c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 0x7f4a0c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4175202994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c198fc0 con 0x7f4a0c102e00 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 0x7f4a0c19ebe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000eb30 tx=0x7f4a0000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f49fb7fe700 1 -- 172.21.15.26:0/4175202994 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000cbd0 con 0x7f4a0c102e00 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f49fb7fe700 1 -- 172.21.15.26:0/4175202994 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a000044b0 con 0x7f4a0c102e00 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.540+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1992b0 con 0x7f4a0c102e00 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.541+0000 7f49fb7fe700 1 -- 172.21.15.26:0/4175202994 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00003cb0 con 0x7f4a0c102e00 2024-05-20T21:07:10.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.541+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c06b220 con 0x7f4a0c102e00 2024-05-20T21:07:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.544+0000 7f49fb7fe700 1 -- 172.21.15.26:0/4175202994 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4a00003f10 con 0x7f4a0c102e00 2024-05-20T21:07:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.544+0000 7f4a1b276b80 1 Processor -- start 2024-05-20T21:07:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 -- start start 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75fa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5598c754bd70 con 0x5598c75ed8a0 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5598c753b910 con 0x5598c75ee260 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5598c753f950 con 0x5598c75f3320 2024-05-20T21:07:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39082/0 (socket says 172.21.15.26:39082) 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a0a59c700 1 -- 172.21.15.26:0/4086699300 learned_addr learned my addr 172.21.15.26:0/4086699300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f49fb7fe700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49f40b2880 0x7f49f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.545+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0a59c700 1 -- 172.21.15.26:0/4086699300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 msgr2=0x5598c75f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0a59c700 1 -- 172.21.15.26:0/4086699300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 msgr2=0x5598c75fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0a59c700 1 -- 172.21.15.26:0/4086699300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c74e4fb0 con 0x5598c75ed8a0 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f49fb7fe700 1 -- 172.21.15.26:0/4175202994 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4a000e24f0 con 0x7f4a0c102e00 2024-05-20T21:07:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49f40b2880 0x7f49f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75edd20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009990 tx=0x7f49fc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da6d3bb1092972e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f49f8ff9700 1 -- 172.21.15.26:0/4086699300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003680 con 0x5598c75ed8a0 2024-05-20T21:07:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f49f8ff9700 1 -- 172.21.15.26:0/4086699300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49fc027070 con 0x5598c75ed8a0 2024-05-20T21:07:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.546+0000 7f49f8ff9700 1 -- 172.21.15.26:0/4086699300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00b9d0 con 0x5598c75ed8a0 2024-05-20T21:07:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4086699300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 msgr2=0x5598c75edd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49f40b2880 0x7f49f40b4d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c090 tx=0x7f4a04004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75edd20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009990 tx=0x7f49fc005fb0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4086699300 shutdown_connections 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4086699300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75edd20 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4086699300 >> 172.21.15.26:0/4086699300 conn(0x5598c7519240 msgr2=0x5598c75e83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4086699300 shutdown_connections 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.547+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4086699300 wait complete. 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 -- start start 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 Processor -- start 2024-05-20T21:07:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ee260 0x5598c75f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5598c7441fa0 con 0x5598c75f3320 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5598c74a3da0 con 0x5598c75ed8a0 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5598c75fa680 con 0x5598c75ee260 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a1b276b80 10 client.0 _ll_register_callbacks cb 0x5598c74d6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a0ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a09d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ee260 0x5598c75f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55462/0 (socket says 172.21.15.26:55462) 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a0a59c700 1 -- 172.21.15.26:0/4249995144 learned_addr learned my addr 172.21.15.26:0/4249995144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.548+0000 7f4a0ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39084/0 (socket says 172.21.15.26:39084) 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4249995144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ed8a0 msgr2=0x5598c75f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4249995144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 msgr2=0x5598c75f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a09d9b700 1 -- 172.21.15.26:0/4249995144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c748b8d0 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ee260 0x5598c75f6fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a000dffe0 tx=0x7f4a00009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753c168c22e9e8f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f49f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f49f1ffb700 1 -- 172.21.15.26:0/4249995144 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000a9400 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f49f1ffb700 1 -- 172.21.15.26:0/4249995144 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4a000a60a0 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49fc027070 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f49f1ffb700 1 -- 172.21.15.26:0/4249995144 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a00004200 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a1b276b80 10 client.27892 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598c74f5760 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.549+0000 7f49f1ffb700 1 -- 172.21.15.26:0/4249995144 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49fc027070 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f49f1ffb700 1 -- 172.21.15.26:0/4249995144 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4a000e20b0 con 0x5598c75ee260 2024-05-20T21:07:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a1b276b80 10 client.27892 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a1b276b80 10 client.27892 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a1b276b80 20 client.27892 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7607070 0x5598c7609460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a1b276b80 4 client.27892 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598c754dcc0 con 0x5598c7607070 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7607070 0x5598c7609460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7607070 0x5598c7609460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=972176cccf79a4d3 server_cookie=f4d65e84f8592eb9 in_seq=0 out_seq=0 2024-05-20T21:07:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.550+0000 7f49f1ffb700 10 client.27892 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:10.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:10 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f49f1ffb700 1 -- 172.21.15.26:0/4249995144 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5598c754dcc0 con 0x5598c7607070 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f49f1ffb700 10 client.27892 handle_command_reply: tid=0 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 client.27892 shutdown 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ee260 msgr2=0x5598c75f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ee260 0x5598c75f6fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a000dffe0 tx=0x7f4a00009f60 comp rx=0 tx=0).stop 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 shutdown_connections 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7607070 0x5598c7609460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/4249995144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ee260 0x5598c75f6fb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 >> 172.21.15.26:0/4249995144 conn(0x5598c7519240 msgr2=0x5598c75e7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f49f1ffb700 0 client.27892 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 shutdown_connections 2024-05-20T21:07:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.551+0000 7f4a1b276b80 1 -- 172.21.15.26:0/4249995144 wait complete. 2024-05-20T21:07:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.552+0000 7f4a1b276b80 20 client.27892 trim_cache size 0 max 16384 2024-05-20T21:07:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 Processor -- start 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 -- start start 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ae310 con 0x5598c75ed8a0 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1abba0 con 0x5598c75ee260 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.568+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1a99f0 con 0x5598c75f3320 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39086/0 (socket says 172.21.15.26:39086) 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a09d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 -- 172.21.15.26:0/2334369173 learned_addr learned my addr 172.21.15.26:0/2334369173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 -- 172.21.15.26:0/2334369173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 msgr2=0x5598c75f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 -- 172.21.15.26:0/2334369173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 msgr2=0x5598c75f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 -- 172.21.15.26:0/2334369173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c75f5540 con 0x5598c75ed8a0 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009990 tx=0x7f49fc003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894cc9bffb71e9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f49f3fff700 1 -- 172.21.15.26:0/2334369173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc005040 con 0x5598c75ed8a0 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f49f3fff700 1 -- 172.21.15.26:0/2334369173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49fc032030 con 0x5598c75ed8a0 2024-05-20T21:07:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.569+0000 7f49f3fff700 1 -- 172.21.15.26:0/2334369173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00bc10 con 0x5598c75ed8a0 2024-05-20T21:07:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 -- 172.21.15.26:0/2334369173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 msgr2=0x5598c75f9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009990 tx=0x7f49fc003f60 comp rx=0 tx=0).stop 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 -- 172.21.15.26:0/2334369173 shutdown_connections 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75ed8a0 0x5598c75f9cb0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75f3320 0x5598c75f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/2334369173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 -- 172.21.15.26:0/2334369173 >> 172.21.15.26:0/2334369173 conn(0x5598c7519240 msgr2=0x5598c75ed120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 -- 172.21.15.26:0/2334369173 shutdown_connections 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.570+0000 7f4a1b276b80 1 -- 172.21.15.26:0/2334369173 wait complete. 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 -- start start 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 Processor -- start 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ed8a0 0x5598c75efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1aa060 con 0x5598c75f3320 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ab4d0 con 0x5598c75ee260 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1aa730 con 0x5598c75ed8a0 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a1b276b80 10 client.0 _ll_register_callbacks cb 0x5598c74d6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a0ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a09d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a0ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39098/0 (socket says 172.21.15.26:39098) 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a0ad9d700 1 -- 172.21.15.26:0/1616493464 learned_addr learned my addr 172.21.15.26:0/1616493464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.571+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ed8a0 0x5598c75efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a0ad9d700 1 -- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ed8a0 msgr2=0x5598c75efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ed8a0 0x5598c75efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a0ad9d700 1 -- 172.21.15.26:0/1616493464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 msgr2=0x5598c75f02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a0ad9d700 1 -- 172.21.15.26:0/1616493464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c75feb80 con 0x5598c75f3320 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a09d9b700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a0ad9d700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f0810 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4a0401cef0 tx=0x7f4a0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3d2e9d8c8ab458 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f49f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f49f1ffb700 1 -- 172.21.15.26:0/1616493464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04007d60 con 0x5598c75f3320 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f49f1ffb700 1 -- 172.21.15.26:0/1616493464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4a0401c050 con 0x5598c75f3320 2024-05-20T21:07:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49fc032030 con 0x5598c75f3320 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f49f1ffb700 1 -- 172.21.15.26:0/1616493464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a0401c250 con 0x5598c75f3320 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a1b276b80 10 client.18804 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598c75ffe80 con 0x5598c75f3320 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.572+0000 7f49f1ffb700 1 -- 172.21.15.26:0/1616493464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49fc032030 con 0x5598c75f3320 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f49f1ffb700 1 -- 172.21.15.26:0/1616493464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4a0402a440 con 0x5598c75f3320 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a1b276b80 10 client.18804 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a1b276b80 10 client.18804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a1b276b80 20 client.18804 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7671be0 0x5598c7673fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a1b276b80 4 client.18804 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598c75474a0 con 0x5598c7671be0 2024-05-20T21:07:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7671be0 0x5598c7673fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f49f1ffb700 10 client.18804 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.573+0000 7f4a0a59c700 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7671be0 0x5598c7673fd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e273421ef02a72bb server_cookie=8f3ccd6584b0ae4f in_seq=0 out_seq=0 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f49f1ffb700 1 -- 172.21.15.26:0/1616493464 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598c75474a0 con 0x5598c7671be0 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f49f1ffb700 10 client.18804 handle_command_reply: tid=0 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 client.18804 shutdown 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 msgr2=0x5598c75f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f0810 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4a0401cef0 tx=0x7f4a0400b040 comp rx=0 tx=0).stop 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 shutdown_connections 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5598c7671be0 0x5598c7673fd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598c75f3320 0x5598c75f0810 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598c75ed8a0 0x5598c75efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f49f1ffb700 0 client.18804 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 --2- 172.21.15.26:0/1616493464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598c75ee260 0x5598c75f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 >> 172.21.15.26:0/1616493464 conn(0x5598c7519240 msgr2=0x5598c75ed120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 shutdown_connections 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 1 -- 172.21.15.26:0/1616493464 wait complete. 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.574+0000 7f4a1b276b80 20 client.18804 trim_cache size 0 max 16384 2024-05-20T21:07:10.580 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49f40b2880 msgr2=0x7f49f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49f40b2880 0x7f49f40b4d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c090 tx=0x7f4a04004ad0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 msgr2=0x7f4a0c19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 0x7f4a0c19ebe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000eb30 tx=0x7f4a0000ee40 comp rx=0 tx=0).stop 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 shutdown_connections 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49f40b2880 0x7f49f40b4d40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a0c108520 0x7f4a0c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a0c102440 0x7f4a0c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 --2- 172.21.15.26:0/4175202994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a0c102e00 0x7f4a0c19ebe0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 >> 172.21.15.26:0/4175202994 conn(0x7f4a0c0793f0 msgr2=0x7f4a0c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 shutdown_connections 2024-05-20T21:07:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:10.575+0000 7f4a109e0700 1 -- 172.21.15.26:0/4175202994 wait complete. 2024-05-20T21:07:10.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 241851 2.18 MB/sec execute 436 sec latency 42.720 ms 2024-05-20T21:07:11.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: pgmap v721: 97 pgs: 97 active+clean; 25 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:07:11.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:07:11.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:07:11.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: pgmap v721: 97 pgs: 97 active+clean; 25 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:07:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 242460 2.18 MB/sec execute 437 sec latency 19.900 ms 2024-05-20T21:07:11.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: pgmap v721: 97 pgs: 97 active+clean; 25 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:07:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:07:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:12.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 243144 2.18 MB/sec execute 438 sec latency 17.682 ms 2024-05-20T21:07:13.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:13 smithi026 ceph-mon[31990]: pgmap v722: 97 pgs: 97 active+clean; 24 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-05-20T21:07:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:13 smithi144 ceph-mon[35660]: pgmap v722: 97 pgs: 97 active+clean; 24 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-05-20T21:07:13.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 243793 2.18 MB/sec execute 439 sec latency 17.584 ms 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 -- 172.21.15.26:0/703954997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4108520 msgr2=0x7f8eb4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 --2- 172.21.15.26:0/703954997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4108520 0x7f8eb4108900 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400faf0 tx=0x7f8ea400fe00 comp rx=0 tx=0).stop 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 -- 172.21.15.26:0/703954997 shutdown_connections 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 --2- 172.21.15.26:0/703954997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4108520 0x7f8eb4108900 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 --2- 172.21.15.26:0/703954997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 0x7f8eb410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 --2- 172.21.15.26:0/703954997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb4102440 0x7f8eb41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 -- 172.21.15.26:0/703954997 >> 172.21.15.26:0/703954997 conn(0x7f8eb40793f0 msgr2=0x7f8eb4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 -- 172.21.15.26:0/703954997 shutdown_connections 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.671+0000 7f8eb981c700 1 -- 172.21.15.26:0/703954997 wait complete. 2024-05-20T21:07:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 Processor -- start 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 -- start start 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 0x7f8eb419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 0x7f8eb419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb4108520 0x7f8eb41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41113b0 con 0x7f8eb4102440 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111230 con 0x7f8eb4102e00 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb981c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111530 con 0x7f8eb4108520 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 0x7f8eb419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 0x7f8eb419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 0x7f8eb419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39114/0 (socket says 172.21.15.26:39114) 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb2ffd700 1 -- 172.21.15.26:0/15741293 learned_addr learned my addr 172.21.15.26:0/15741293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.672+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb4108520 0x7f8eb41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb27fc700 1 -- 172.21.15.26:0/15741293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb4108520 msgr2=0x7f8eb41976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb4108520 0x7f8eb41976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb27fc700 1 -- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 msgr2=0x7f8eb419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 0x7f8eb419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb27fc700 1 -- 172.21.15.26:0/15741293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb4197fd0 con 0x7f8eb4102e00 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 0x7f8eb419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 0x7f8eb419db60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800e9f0 tx=0x7f8ea800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb881a700 1 -- 172.21.15.26:0/15741293 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800c9c0 con 0x7f8eb4102e00 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb881a700 1 -- 172.21.15.26:0/15741293 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ea8010430 con 0x7f8eb4102e00 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb41982c0 con 0x7f8eb4102e00 2024-05-20T21:07:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb881a700 1 -- 172.21.15.26:0/15741293 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8018680 con 0x7f8eb4102e00 2024-05-20T21:07:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.673+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41a45a0 con 0x7f8eb4102e00 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.676+0000 7f8eb881a700 1 -- 172.21.15.26:0/15741293 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8ea800cb60 con 0x7f8eb4102e00 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.677+0000 7f8eb881a700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e9c0b2880 0x7f8e9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.677+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e9c0b2880 0x7f8e9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.677+0000 7f8eb881a700 1 -- 172.21.15.26:0/15741293 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8ea80e2380 con 0x7f8eb4102e00 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.677+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e9c0b2880 0x7f8e9c0b4d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009870 tx=0x7f8ea400ff90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 Processor -- start 2024-05-20T21:07:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 -- start start 2024-05-20T21:07:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cd210 0x55f50a3d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f50a32bc90 con 0x55f50a3cc860 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f50a31b830 con 0x55f50a3d0f60 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f50a31f870 con 0x55f50a3cd210 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8eb2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8eb2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39124/0 (socket says 172.21.15.26:39124) 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55508/0 (socket says 172.21.15.26:55508) 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8eb2ffd700 1 -- 172.21.15.26:0/2907782230 learned_addr learned my addr 172.21.15.26:0/2907782230 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.678+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cd210 0x55f50a3d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb37fe700 1 -- 172.21.15.26:0/2907782230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cd210 msgr2=0x55f50a3d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cd210 0x55f50a3d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb37fe700 1 -- 172.21.15.26:0/2907782230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 msgr2=0x55f50a3ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb37fe700 1 -- 172.21.15.26:0/2907782230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50a26b8d0 con 0x55f50a3d0f60 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cd210 0x55f50a3d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3d13a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b3b0 tx=0x7f8eac009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299e1b536fb04ea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8e99ffb700 1 -- 172.21.15.26:0/2907782230 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac004bb0 con 0x55f50a3d0f60 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8e99ffb700 1 -- 172.21.15.26:0/2907782230 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8eac004020 con 0x55f50a3d0f60 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.679+0000 7f8e99ffb700 1 -- 172.21.15.26:0/2907782230 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0186e0 con 0x55f50a3d0f60 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/2907782230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 msgr2=0x55f50a3d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3d13a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b3b0 tx=0x7f8eac009460 comp rx=0 tx=0).stop 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/2907782230 shutdown_connections 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3d13a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/2907782230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cd210 0x55f50a3d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/2907782230 >> 172.21.15.26:0/2907782230 conn(0x55f50a2fa080 msgr2=0x55f50a3c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/2907782230 shutdown_connections 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/2907782230 wait complete. 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.680+0000 7f8ec40b2b80 1 -- start start 2024-05-20T21:07:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 Processor -- start 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3d55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3d5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f50a221fa0 con 0x55f50a3cc860 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f50a2bd7c0 con 0x55f50a3cd210 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f50a3d9120 con 0x55f50a3d0f60 2024-05-20T21:07:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8ec40b2b80 10 client.0 _ll_register_callbacks cb 0x55f50a2b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3d55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3d5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49386/0 (socket says 172.21.15.26:49386) 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 -- 172.21.15.26:0/3709347734 learned_addr learned my addr 172.21.15.26:0/3709347734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 -- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 msgr2=0x55f50a3d55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3d55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 -- 172.21.15.26:0/3709347734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 msgr2=0x55f50a3d82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3d82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb37fe700 1 -- 172.21.15.26:0/3709347734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50a2ba690 con 0x55f50a3d0f60 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.681+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3d82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3d5b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0046b0 tx=0x7f8eac004d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1e31f8364cace4b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8e92ffd700 1 -- 172.21.15.26:0/3709347734 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac026020 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eac004020 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8e92ffd700 1 -- 172.21.15.26:0/3709347734 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8eac003c10 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8e92ffd700 1 -- 172.21.15.26:0/3709347734 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8eac031050 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8ec40b2b80 10 client.27910 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.682+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f50a3e3790 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8e92ffd700 1 -- 172.21.15.26:0/3709347734 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8eac004020 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8e92ffd700 1 -- 172.21.15.26:0/3709347734 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8eac018c00 con 0x55f50a3d0f60 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8ec40b2b80 10 client.27910 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8ec40b2b80 10 client.27910 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8ec40b2b80 20 client.27910 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a3e5d20 0x55f50a3e8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8ec40b2b80 4 client.27910 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50a3989e0 con 0x55f50a3e5d20 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.683+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a3e5d20 0x55f50a3e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.684+0000 7f8e92ffd700 10 client.27910 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.684+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a3e5d20 0x55f50a3e8110 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93c69efc5ac78d9d server_cookie=127201462b9b84ce in_seq=0 out_seq=0 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.684+0000 7f8e92ffd700 1 -- 172.21.15.26:0/3709347734 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f50a3989e0 con 0x55f50a3e5d20 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.684+0000 7f8e92ffd700 10 client.27910 handle_command_reply: tid=0 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.684+0000 7f8ec40b2b80 1 client.27910 shutdown 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 msgr2=0x55f50a3d5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3d5b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0046b0 tx=0x7f8eac004d80 comp rx=0 tx=0).stop 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 shutdown_connections 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a3e5d20 0x55f50a3e8110 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3d82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3d5b20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/3709347734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3d55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8e92ffd700 0 client.27910 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 >> 172.21.15.26:0/3709347734 conn(0x55f50a2fa080 msgr2=0x55f50a3c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 shutdown_connections 2024-05-20T21:07:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/3709347734 wait complete. 2024-05-20T21:07:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.685+0000 7f8ec40b2b80 20 client.27910 trim_cache size 0 max 16384 2024-05-20T21:07:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.701+0000 7f8ec40b2b80 1 Processor -- start 2024-05-20T21:07:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.701+0000 7f8ec40b2b80 1 -- start start 2024-05-20T21:07:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cc860 0x55f50a3ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 0x55f50a3cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41ae430 con 0x55f50a3cd210 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41abcc0 con 0x55f50a3d0f60 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41a9b10 con 0x55f50a3cc860 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 0x55f50a3cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 0x55f50a3cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39148/0 (socket says 172.21.15.26:39148) 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 -- 172.21.15.26:0/1831291092 learned_addr learned my addr 172.21.15.26:0/1831291092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cc860 0x55f50a3ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 -- 172.21.15.26:0/1831291092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cc860 msgr2=0x55f50a3ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cc860 0x55f50a3ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 -- 172.21.15.26:0/1831291092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 msgr2=0x55f50a3ed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3ed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 -- 172.21.15.26:0/1831291092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50a3d53b0 con 0x55f50a3cd210 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb37fe700 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 0x55f50a3cedf0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80e0190 tx=0x7f8ea80cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf42518e3c42a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8e997fa700 1 -- 172.21.15.26:0/1831291092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80c3070 con 0x55f50a3cd210 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8e997fa700 1 -- 172.21.15.26:0/1831291092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ea80a9450 con 0x55f50a3cd210 2024-05-20T21:07:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.702+0000 7f8e997fa700 1 -- 172.21.15.26:0/1831291092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80108e0 con 0x55f50a3cd210 2024-05-20T21:07:13.704 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:13 smithi055 ceph-mon[35213]: pgmap v722: 97 pgs: 97 active+clean; 24 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/1831291092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 msgr2=0x55f50a3cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 0x55f50a3cedf0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80e0190 tx=0x7f8ea80cc040 comp rx=0 tx=0).stop 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/1831291092 shutdown_connections 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3d0f60 0x55f50a3ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3cc860 0x55f50a3ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/1831291092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cd210 0x55f50a3cedf0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/1831291092 >> 172.21.15.26:0/1831291092 conn(0x55f50a2fa080 msgr2=0x55f50a3d76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/1831291092 shutdown_connections 2024-05-20T21:07:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.703+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/1831291092 wait complete. 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 -- start start 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 Processor -- start 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41aa180 con 0x55f50a3cc860 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41ab5f0 con 0x55f50a3cd210 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41aa850 con 0x55f50a3d0f60 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8ec40b2b80 10 client.0 _ll_register_callbacks cb 0x55f50a2b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8eb2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3dec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55538/0 (socket says 172.21.15.26:55538) 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8eb27fc700 1 -- 172.21.15.26:0/53858309 learned_addr learned my addr 172.21.15.26:0/53858309 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.704+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39162/0 (socket says 172.21.15.26:39162) 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8eb27fc700 1 -- 172.21.15.26:0/53858309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 msgr2=0x55f50a3c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8eb27fc700 1 -- 172.21.15.26:0/53858309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 msgr2=0x55f50a3de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3de700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8eb27fc700 1 -- 172.21.15.26:0/53858309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50a3c6c50 con 0x55f50a3cd210 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8eb27fc700 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3dec40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80a59c0 tx=0x7f8ea8010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04c549b655e776b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8e92ffd700 1 -- 172.21.15.26:0/53858309 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80c3040 con 0x55f50a3cd210 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8e92ffd700 1 -- 172.21.15.26:0/53858309 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8ea80a9ba0 con 0x55f50a3cd210 2024-05-20T21:07:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea80a9450 con 0x55f50a3cd210 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8e92ffd700 1 -- 172.21.15.26:0/53858309 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ea80b20e0 con 0x55f50a3cd210 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8ec40b2b80 10 client.28121 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.705+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f50a450180 con 0x55f50a3cd210 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8e92ffd700 1 -- 172.21.15.26:0/53858309 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ea80a9450 con 0x55f50a3cd210 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8e92ffd700 1 -- 172.21.15.26:0/53858309 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f8ea80e2040 con 0x55f50a3cd210 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8ec40b2b80 10 client.28121 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8ec40b2b80 10 client.28121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8ec40b2b80 20 client.28121 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a450750 0x55f50a452b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8ec40b2b80 4 client.28121 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f50a32dbe0 con 0x55f50a450750 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a450750 0x55f50a452b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8e92ffd700 10 client.28121 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.706+0000 7f8eb2ffd700 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a450750 0x55f50a452b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c598e35fda8ef347 server_cookie=e2d3eb9682d3d7cf in_seq=0 out_seq=0 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8e92ffd700 1 -- 172.21.15.26:0/53858309 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f50a32dbe0 con 0x55f50a450750 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8e92ffd700 10 client.28121 handle_command_reply: tid=0 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 client.28121 shutdown 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 msgr2=0x55f50a3dec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3dec40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80a59c0 tx=0x7f8ea8010b10 comp rx=0 tx=0).stop 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 shutdown_connections 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f50a450750 0x55f50a452b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f50a3cc860 0x55f50a3de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f50a3d0f60 0x55f50a3c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 --2- 172.21.15.26:0/53858309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f50a3cd210 0x55f50a3dec40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 >> 172.21.15.26:0/53858309 conn(0x55f50a2fa080 msgr2=0x55f50a3d6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8e92ffd700 0 client.28121 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 shutdown_connections 2024-05-20T21:07:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 1 -- 172.21.15.26:0/53858309 wait complete. 2024-05-20T21:07:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.707+0000 7f8ec40b2b80 20 client.28121 trim_cache size 0 max 16384 2024-05-20T21:07:13.713 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e9c0b2880 msgr2=0x7f8e9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e9c0b2880 0x7f8e9c0b4d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009870 tx=0x7f8ea400ff90 comp rx=0 tx=0).stop 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 msgr2=0x7f8eb419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 0x7f8eb419db60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800e9f0 tx=0x7f8ea800ed00 comp rx=0 tx=0).stop 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 shutdown_connections 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e9c0b2880 0x7f8e9c0b4d40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb4102440 0x7f8eb419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb4108520 0x7f8eb41976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 --2- 172.21.15.26:0/15741293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8eb4102e00 0x7f8eb419db60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 >> 172.21.15.26:0/15741293 conn(0x7f8eb40793f0 msgr2=0x7f8eb4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 shutdown_connections 2024-05-20T21:07:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:13.708+0000 7f8eb981c700 1 -- 172.21.15.26:0/15741293 wait complete. 2024-05-20T21:07:14.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:07:14.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 244562 2.19 MB/sec execute 440 sec latency 13.534 ms 2024-05-20T21:07:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:15.259 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:15 smithi026 ceph-mon[31990]: pgmap v723: 97 pgs: 97 active+clean; 24 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T21:07:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:15 smithi144 ceph-mon[35660]: pgmap v723: 97 pgs: 97 active+clean; 24 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T21:07:15.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 245141 2.20 MB/sec execute 441 sec latency 16.483 ms 2024-05-20T21:07:15.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:15 smithi055 ceph-mon[35213]: pgmap v723: 97 pgs: 97 active+clean; 24 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-05-20T21:07:16.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 245620 2.19 MB/sec execute 442 sec latency 21.056 ms 2024-05-20T21:07:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 -- 172.21.15.26:0/380640909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 msgr2=0x7efc54109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 --2- 172.21.15.26:0/380640909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54109990 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7efc44009990 tx=0x7efc44009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 -- 172.21.15.26:0/380640909 shutdown_connections 2024-05-20T21:07:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 --2- 172.21.15.26:0/380640909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54109990 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 --2- 172.21.15.26:0/380640909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406bbe0 0x7efc541103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 --2- 172.21.15.26:0/380640909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc5406b220 0x7efc5406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 -- 172.21.15.26:0/380640909 >> 172.21.15.26:0/380640909 conn(0x7efc540793f0 msgr2=0x7efc54079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:16.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 -- 172.21.15.26:0/380640909 shutdown_connections 2024-05-20T21:07:16.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.643+0000 7efc58b10700 1 -- 172.21.15.26:0/380640909 wait complete. 2024-05-20T21:07:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 Processor -- start 2024-05-20T21:07:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 -- start start 2024-05-20T21:07:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 0x7efc5419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc5406bbe0 0x7efc5419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc54113590 con 0x7efc541095b0 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efc54113410 con 0x7efc5406bbe0 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc58b10700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc54113710 con 0x7efc5406b220 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.644+0000 7efc5259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 0x7efc5419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc52d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc52d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39180/0 (socket says 172.21.15.26:39180) 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc52d9d700 1 -- 172.21.15.26:0/1587166010 learned_addr learned my addr 172.21.15.26:0/1587166010 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 0x7efc5419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49410/0 (socket says 172.21.15.26:49410) 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc5406bbe0 0x7efc5419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 -- 172.21.15.26:0/1587166010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc5406bbe0 msgr2=0x7efc5419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc5406bbe0 0x7efc5419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 -- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 msgr2=0x7efc54199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 -- 172.21.15.26:0/1587166010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5419a090 con 0x7efc5406b220 2024-05-20T21:07:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc5259c700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 0x7efc5419f770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc44009990 tx=0x7efc440057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc437fe700 1 -- 172.21.15.26:0/1587166010 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44026070 con 0x7efc5406b220 2024-05-20T21:07:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5419a320 con 0x7efc5406b220 2024-05-20T21:07:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc437fe700 1 -- 172.21.15.26:0/1587166010 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc44021410 con 0x7efc5406b220 2024-05-20T21:07:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.645+0000 7efc437fe700 1 -- 172.21.15.26:0/1587166010 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44017650 con 0x7efc5406b220 2024-05-20T21:07:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.646+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc541a6640 con 0x7efc5406b220 2024-05-20T21:07:16.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.648+0000 7efc437fe700 1 -- 172.21.15.26:0/1587166010 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efc44029070 con 0x7efc5406b220 2024-05-20T21:07:16.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.649+0000 7efc437fe700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc3c0b2880 0x7efc3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.649+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc3c0b2880 0x7efc3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.649+0000 7efc437fe700 1 -- 172.21.15.26:0/1587166010 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efc440e2fe0 con 0x7efc5406b220 2024-05-20T21:07:16.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 Processor -- start 2024-05-20T21:07:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc3c0b2880 0x7efc3c0b4d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc54079b60 tx=0x7efc48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 -- start start 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ebe4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebf1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b82eb42d70 con 0x55b82ebe5260 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b82eb32910 con 0x55b82ebe48a0 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc633a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b82eb36950 con 0x55b82ebea320 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc52d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebf1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc52d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebf1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39184/0 (socket says 172.21.15.26:39184) 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc5259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc52d9d700 1 -- 172.21.15.26:0/1189396534 learned_addr learned my addr 172.21.15.26:0/1189396534 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.650+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ebe4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc52d9d700 1 -- 172.21.15.26:0/1189396534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 msgr2=0x55b82ebea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc52d9d700 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc52d9d700 1 -- 172.21.15.26:0/1189396534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 msgr2=0x55b82ebe4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc52d9d700 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ebe4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc52d9d700 1 -- 172.21.15.26:0/1189396534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82eadbfb0 con 0x55b82ebe5260 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ebe4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc5259c700 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.651+0000 7efc52d9d700 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebf1080 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e6a0 tx=0x7efc4c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812f14e9ef51d80c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.652+0000 7efc40ff9700 1 -- 172.21.15.26:0/1189396534 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c019030 con 0x55b82ebe5260 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.652+0000 7efc40ff9700 1 -- 172.21.15.26:0/1189396534 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc4c0092e0 con 0x55b82ebe5260 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.652+0000 7efc40ff9700 1 -- 172.21.15.26:0/1189396534 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c007500 con 0x55b82ebe5260 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.652+0000 7efc633a6b80 1 -- 172.21.15.26:0/1189396534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 msgr2=0x55b82ebf1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.652+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebf1080 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e6a0 tx=0x7efc4c009920 comp rx=0 tx=0).stop 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 -- 172.21.15.26:0/1189396534 shutdown_connections 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebf1080 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1189396534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ebe4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 -- 172.21.15.26:0/1189396534 >> 172.21.15.26:0/1189396534 conn(0x55b82eb10240 msgr2=0x55b82ebdf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:16.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 -- 172.21.15.26:0/1189396534 shutdown_connections 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 -- 172.21.15.26:0/1189396534 wait complete. 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 -- start start 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.653+0000 7efc633a6b80 1 Processor -- start 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b82ea38fa0 con 0x55b82ebe48a0 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b82ea9ada0 con 0x55b82ebe5260 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b82ebf16e0 con 0x55b82ebea320 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc633a6b80 10 client.0 _ll_register_callbacks cb 0x55b82eacd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc51d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc52d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39194/0 (socket says 172.21.15.26:39194) 2024-05-20T21:07:16.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 -- 172.21.15.26:0/2746882471 learned_addr learned my addr 172.21.15.26:0/2746882471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:16.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc51d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebee020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:55566/0 (socket says 172.21.15.26:55566) 2024-05-20T21:07:16.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 -- 172.21.15.26:0/2746882471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 msgr2=0x55b82ebee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebee560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 -- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 msgr2=0x55b82ebee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 -- 172.21.15.26:0/2746882471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82ea828d0 con 0x55b82ebe48a0 2024-05-20T21:07:16.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc52d9d700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc51d9b700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc5259c700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf0d20 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efc440aa330 tx=0x7efc44021ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484f4be17ac1a474 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.654+0000 7efc39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc39ffb700 1 -- 172.21.15.26:0/2746882471 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440aa630 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc39ffb700 1 -- 172.21.15.26:0/2746882471 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efc440aaaa0 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc4c0092e0 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc39ffb700 1 -- 172.21.15.26:0/2746882471 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc440ae9e0 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 10 client.18834 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b82eaec760 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc39ffb700 1 -- 172.21.15.26:0/2746882471 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc4c0092e0 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc39ffb700 1 -- 172.21.15.26:0/2746882471 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7efc44090ba0 con 0x55b82ebe48a0 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 10 client.18834 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 20 client.18834 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ebfe150 0x55b82ec00540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 4 client.18834 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.655+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82eb44cc0 con 0x55b82ebfe150 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.656+0000 7efc51d9b700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ebfe150 0x55b82ec00540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.656+0000 7efc39ffb700 10 client.18834 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.656+0000 7efc51d9b700 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ebfe150 0x55b82ec00540 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3f1ee02d0d3297 server_cookie=458b5a30e9703173 in_seq=0 out_seq=0 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc39ffb700 1 -- 172.21.15.26:0/2746882471 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b82eb44cc0 con 0x55b82ebfe150 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc39ffb700 10 client.18834 handle_command_reply: tid=0 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 client.18834 shutdown 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 msgr2=0x55b82ebf0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf0d20 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efc440aa330 tx=0x7efc44021ba0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 shutdown_connections 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ebfe150 0x55b82ec00540 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf0d20 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc39ffb700 0 client.18834 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 --2- 172.21.15.26:0/2746882471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 >> 172.21.15.26:0/2746882471 conn(0x55b82eb10240 msgr2=0x55b82ebdeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 shutdown_connections 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 1 -- 172.21.15.26:0/2746882471 wait complete. 2024-05-20T21:07:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.657+0000 7efc633a6b80 20 client.18834 trim_cache size 0 max 16384 2024-05-20T21:07:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 Processor -- start 2024-05-20T21:07:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 -- start start 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ec00540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebdf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebe8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc541b03c0 con 0x55b82ebe5260 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efc541adc50 con 0x55b82ebe48a0 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc633a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc541abaa0 con 0x55b82ebea320 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc51d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebdf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc51d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebdf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39206/0 (socket says 172.21.15.26:39206) 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc51d9b700 1 -- 172.21.15.26:0/1093976985 learned_addr learned my addr 172.21.15.26:0/1093976985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc52d9d700 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebe8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.676+0000 7efc5259c700 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ec00540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc51d9b700 1 -- 172.21.15.26:0/1093976985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 msgr2=0x55b82ebe8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebe8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc51d9b700 1 -- 172.21.15.26:0/1093976985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 msgr2=0x55b82ec00540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ec00540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc51d9b700 1 -- 172.21.15.26:0/1093976985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82ec00ba0 con 0x55b82ebe5260 2024-05-20T21:07:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc51d9b700 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebdf270 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7efc480097c0 tx=0x7efc48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965b6ec3401e74c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc3bfff700 1 -- 172.21.15.26:0/1093976985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48029450 con 0x55b82ebe5260 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc3bfff700 1 -- 172.21.15.26:0/1093976985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc48029a80 con 0x55b82ebe5260 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.677+0000 7efc3bfff700 1 -- 172.21.15.26:0/1093976985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4801c7c0 con 0x55b82ebe5260 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 -- 172.21.15.26:0/1093976985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 msgr2=0x55b82ebdf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebdf270 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7efc480097c0 tx=0x7efc48014000 comp rx=0 tx=0).stop 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 -- 172.21.15.26:0/1093976985 shutdown_connections 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ebe8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe48a0 0x55b82ec00540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 --2- 172.21.15.26:0/1093976985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe5260 0x55b82ebdf270 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 -- 172.21.15.26:0/1093976985 >> 172.21.15.26:0/1093976985 conn(0x55b82eb10240 msgr2=0x55b82ebe7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 -- 172.21.15.26:0/1093976985 shutdown_connections 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 -- 172.21.15.26:0/1093976985 wait complete. 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 -- start start 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.678+0000 7efc633a6b80 1 Processor -- start 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebf7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ec05240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc541ac110 con 0x55b82ebe48a0 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efc541ad580 con 0x55b82ebe5260 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc541ac7e0 con 0x55b82ebea320 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc633a6b80 10 client.0 _ll_register_callbacks cb 0x55b82eacd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc5259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc51d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebf7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc5259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39216/0 (socket says 172.21.15.26:39216) 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc5259c700 1 -- 172.21.15.26:0/4108603284 learned_addr learned my addr 172.21.15.26:0/4108603284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc52d9d700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ec05240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc51d9b700 1 -- 172.21.15.26:0/4108603284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 msgr2=0x55b82ec05240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc51d9b700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ec05240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc51d9b700 1 -- 172.21.15.26:0/4108603284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 msgr2=0x55b82ebf6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc51d9b700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc51d9b700 1 -- 172.21.15.26:0/4108603284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82ec05a80 con 0x55b82ebe5260 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.679+0000 7efc5259c700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc51d9b700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebf7050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55b82ebef780 tx=0x7efc48009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515a0ef919b31905 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc39ffb700 1 -- 172.21.15.26:0/4108603284 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48029450 con 0x55b82ebe5260 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc48029a80 con 0x55b82ebe5260 2024-05-20T21:07:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc39ffb700 1 -- 172.21.15.26:0/4108603284 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efc480295f0 con 0x55b82ebe5260 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc39ffb700 1 -- 172.21.15.26:0/4108603284 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc48045050 con 0x55b82ebe5260 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc633a6b80 10 client.28139 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b82ec6cc70 con 0x55b82ebe5260 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.680+0000 7efc39ffb700 1 -- 172.21.15.26:0/4108603284 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc48029a80 con 0x55b82ebe5260 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc39ffb700 1 -- 172.21.15.26:0/4108603284 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7efc48033410 con 0x55b82ebe5260 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 10 client.28139 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 10 client.28139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 20 client.28139 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ec6d260 0x55b82ec6f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 4 client.28139 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b82eb3e4a0 con 0x55b82ec6d260 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc5259c700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ec6d260 0x55b82ec6f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc5259c700 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ec6d260 0x55b82ec6f680 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ec41f10a0c2555 server_cookie=b25f319345e5934b in_seq=0 out_seq=0 2024-05-20T21:07:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc39ffb700 10 client.28139 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc39ffb700 1 -- 172.21.15.26:0/4108603284 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b82eb3e4a0 con 0x55b82ec6d260 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc39ffb700 10 client.28139 handle_command_reply: tid=0 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.681+0000 7efc633a6b80 1 client.28139 shutdown 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 msgr2=0x55b82ebf7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebf7050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55b82ebef780 tx=0x7efc48009490 comp rx=0 tx=0).stop 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 shutdown_connections 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b82ec6d260 0x55b82ec6f680 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82ebe48a0 0x55b82ebf6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b82ebea320 0x55b82ec05240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 --2- 172.21.15.26:0/4108603284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b82ebe5260 0x55b82ebf7050 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 >> 172.21.15.26:0/4108603284 conn(0x55b82eb10240 msgr2=0x55b82ebe85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc39ffb700 0 client.28139 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 shutdown_connections 2024-05-20T21:07:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 1 -- 172.21.15.26:0/4108603284 wait complete. 2024-05-20T21:07:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.682+0000 7efc633a6b80 20 client.28139 trim_cache size 0 max 16384 2024-05-20T21:07:16.686 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc3c0b2880 msgr2=0x7efc3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc3c0b2880 0x7efc3c0b4d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc54079b60 tx=0x7efc48009260 comp rx=0 tx=0).stop 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 msgr2=0x7efc5419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 0x7efc5419f770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc44009990 tx=0x7efc440057c0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 shutdown_connections 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc3c0b2880 0x7efc3c0b4d40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541095b0 0x7efc54199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc5406b220 0x7efc5419f770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 --2- 172.21.15.26:0/1587166010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc5406bbe0 0x7efc5419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 >> 172.21.15.26:0/1587166010 conn(0x7efc540793f0 msgr2=0x7efc54101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 shutdown_connections 2024-05-20T21:07:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:16.683+0000 7efc58b10700 1 -- 172.21.15.26:0/1587166010 wait complete. 2024-05-20T21:07:17.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:17 smithi026 ceph-mon[31990]: pgmap v724: 97 pgs: 97 active+clean; 26 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-20T21:07:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:17 smithi144 ceph-mon[35660]: pgmap v724: 97 pgs: 97 active+clean; 26 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-20T21:07:17.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 246319 2.20 MB/sec execute 443 sec latency 15.969 ms 2024-05-20T21:07:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:17 smithi055 ceph-mon[35213]: pgmap v724: 97 pgs: 97 active+clean; 26 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-20T21:07:18.540 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:07:18.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:07:18.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 247014 2.19 MB/sec execute 444 sec latency 20.777 ms 2024-05-20T21:07:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:19.502 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:19 smithi055 ceph-mon[35213]: pgmap v725: 97 pgs: 97 active+clean; 26 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-05-20T21:07:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:19 smithi144 ceph-mon[35660]: pgmap v725: 97 pgs: 97 active+clean; 26 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-05-20T21:07:19.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 247792 2.20 MB/sec execute 445 sec latency 12.143 ms 2024-05-20T21:07:19.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:19 smithi026 ceph-mon[31990]: pgmap v725: 97 pgs: 97 active+clean; 26 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-05-20T21:07:20.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 248440 2.21 MB/sec execute 446 sec latency 40.501 ms 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.682+0000 7f18b86f3700 1 -- 172.21.15.26:0/171652791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 msgr2=0x7f18b01088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.682+0000 7f18b86f3700 1 --2- 172.21.15.26:0/171652791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01088a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009990 tx=0x7f18ac009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 -- 172.21.15.26:0/171652791 shutdown_connections 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 --2- 172.21.15.26:0/171652791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01088a0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 --2- 172.21.15.26:0/171652791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 0x7f18b010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 --2- 172.21.15.26:0/171652791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 0x7f18b01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 -- 172.21.15.26:0/171652791 >> 172.21.15.26:0/171652791 conn(0x7f18b0078ab0 msgr2=0x7f18b00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 -- 172.21.15.26:0/171652791 shutdown_connections 2024-05-20T21:07:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 -- 172.21.15.26:0/171652791 wait complete. 2024-05-20T21:07:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.683+0000 7f18b86f3700 1 Processor -- start 2024-05-20T21:07:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 -- start start 2024-05-20T21:07:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 0x7f18b019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 0x7f18b019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112480 con 0x7f18b01084c0 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112300 con 0x7f18b0102c40 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b86f3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112600 con 0x7f18b0103600 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b5c8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 0x7f18b019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b648f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 0x7f18b019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b648f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 0x7f18b019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42850/0 (socket says 172.21.15.26:42850) 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b5c8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 0x7f18b019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:37912/0 (socket says 172.21.15.26:37912) 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b648f700 1 -- 172.21.15.26:0/2014384884 learned_addr learned my addr 172.21.15.26:0/2014384884 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56724/0 (socket says 172.21.15.26:56724) 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 -- 172.21.15.26:0/2014384884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 msgr2=0x7f18b019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 0x7f18b019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 -- 172.21.15.26:0/2014384884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 msgr2=0x7f18b019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 0x7f18b019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.684+0000 7f18b6c90700 1 -- 172.21.15.26:0/2014384884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b01a5ea0 con 0x7f18b01084c0 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.685+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01a5960 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f18a8007f10 tx=0x7f18a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.685+0000 7f18a77fe700 1 -- 172.21.15.26:0/2014384884 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004db0 con 0x7f18b01084c0 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.685+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b01a6070 con 0x7f18b01084c0 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.685+0000 7f18a77fe700 1 -- 172.21.15.26:0/2014384884 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a8009e20 con 0x7f18b01084c0 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.685+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a6560 con 0x7f18b01084c0 2024-05-20T21:07:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.685+0000 7f18a77fe700 1 -- 172.21.15.26:0/2014384884 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8015400 con 0x7f18b01084c0 2024-05-20T21:07:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.688+0000 7f18a77fe700 1 -- 172.21.15.26:0/2014384884 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f18a800db20 con 0x7f18b01084c0 2024-05-20T21:07:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.689+0000 7f18a77fe700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f189c0b2880 0x7f189c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.689+0000 7f18b648f700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f189c0b2880 0x7f189c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.689+0000 7f18a77fe700 1 -- 172.21.15.26:0/2014384884 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f18a80e1ab0 con 0x7f18b01084c0 2024-05-20T21:07:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.689+0000 7f18c2f89b80 1 Processor -- start 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.689+0000 7f18c2f89b80 1 -- start start 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b648f700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f189c0b2880 0x7f189c0b4d40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00b2e0 tx=0x7f18ac005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 0x55ec9e0d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec9e035df0 con 0x55ec9e0d7920 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ec9e025990 con 0x55ec9e0d82e0 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ec9e0299d0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b5c8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 0x55ec9e0d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b5c8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 0x55ec9e0d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56730/0 (socket says 172.21.15.26:56730) 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b5c8e700 1 -- 172.21.15.26:0/2878702449 learned_addr learned my addr 172.21.15.26:0/2878702449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b648f700 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b6c90700 1 -- 172.21.15.26:0/2878702449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 msgr2=0x55ec9e0dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b6c90700 1 -- 172.21.15.26:0/2878702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 msgr2=0x55ec9e0d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 0x55ec9e0d7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b6c90700 1 -- 172.21.15.26:0/2878702449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9dfcf030 con 0x55ec9e0d82e0 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.690+0000 7f18b5c8e700 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 0x55ec9e0d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.691+0000 7f18b6c90700 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e4100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f18a80df8c0 tx=0x7f18a80a88b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b1f702c0d85936 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.691+0000 7f18a4ff9700 1 -- 172.21.15.26:0/2878702449 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80a9070 con 0x55ec9e0d82e0 2024-05-20T21:07:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.691+0000 7f18a4ff9700 1 -- 172.21.15.26:0/2878702449 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a80ad420 con 0x55ec9e0d82e0 2024-05-20T21:07:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.691+0000 7f18a4ff9700 1 -- 172.21.15.26:0/2878702449 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80b1a40 con 0x55ec9e0d82e0 2024-05-20T21:07:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.691+0000 7f18c2f89b80 1 -- 172.21.15.26:0/2878702449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 msgr2=0x55ec9e0e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.691+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e4100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f18a80df8c0 tx=0x7f18a80a88b0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 -- 172.21.15.26:0/2878702449 shutdown_connections 2024-05-20T21:07:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e4100 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/2878702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d7920 0x55ec9e0d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 -- 172.21.15.26:0/2878702449 >> 172.21.15.26:0/2878702449 conn(0x55ec9e0032c0 msgr2=0x55ec9e0d2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 -- 172.21.15.26:0/2878702449 shutdown_connections 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 -- 172.21.15.26:0/2878702449 wait complete. 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 -- start start 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.692+0000 7f18c2f89b80 1 Processor -- start 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec9df2c020 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ec9df8de20 con 0x55ec9e0d82e0 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ec9e0e4760 con 0x55ec9e0d7920 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 10 client.0 _ll_register_callbacks cb 0x55ec9dfc09d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56744/0 (socket says 172.21.15.26:56744) 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 -- 172.21.15.26:0/4228536111 learned_addr learned my addr 172.21.15.26:0/4228536111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b648f700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b5c8e700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 -- 172.21.15.26:0/4228536111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0d7920 msgr2=0x55ec9e0e3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 -- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 msgr2=0x55ec9e0e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 -- 172.21.15.26:0/4228536111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9df75950 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18b6c90700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e1530 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009fd0 tx=0x7f18a80ad340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a884578a6fafab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f1899ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f1899ffb700 1 -- 172.21.15.26:0/4228536111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80b1ed0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f1899ffb700 1 -- 172.21.15.26:0/4228536111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f18a80b1c50 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.693+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a80ad420 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f1899ffb700 1 -- 172.21.15.26:0/4228536111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18a80a9070 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 10 client.18852 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec9dfdf7e0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f1899ffb700 1 -- 172.21.15.26:0/4228536111 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a80ad420 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f1899ffb700 1 -- 172.21.15.26:0/4228536111 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f18a80a7d40 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 10 client.18852 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 10 client.18852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 20 client.18852 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e0f10d0 0x55ec9e0f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 4 client.18852 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec9e037d40 con 0x55ec9e0f10d0 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.694+0000 7f18b648f700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e0f10d0 0x55ec9e0f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.695+0000 7f18b648f700 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e0f10d0 0x55ec9e0f34c0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f786c5920412a23d server_cookie=3216ad9415619f75 in_seq=0 out_seq=0 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.695+0000 7f1899ffb700 10 client.18852 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.695+0000 7f1899ffb700 1 -- 172.21.15.26:0/4228536111 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ec9e037d40 con 0x55ec9e0f10d0 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.695+0000 7f1899ffb700 10 client.18852 handle_command_reply: tid=0 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 client.18852 shutdown 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 msgr2=0x55ec9e0e1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e1530 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009fd0 tx=0x7f18a80ad340 comp rx=0 tx=0).stop 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 shutdown_connections 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e0f10d0 0x55ec9e0f34c0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e1530 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/4228536111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 >> 172.21.15.26:0/4228536111 conn(0x55ec9e0032c0 msgr2=0x55ec9e0d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f1899ffb700 0 client.18852 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 shutdown_connections 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 1 -- 172.21.15.26:0/4228536111 wait complete. 2024-05-20T21:07:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.696+0000 7f18c2f89b80 20 client.18852 trim_cache size 0 max 16384 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 Processor -- start 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 -- start start 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0db660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f189400fde0 con 0x55ec9e0d82e0 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f189400d670 con 0x55ec9e0d7920 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f189400b4c0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18b648f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18b5c8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18b5c8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56752/0 (socket says 172.21.15.26:56752) 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18b648f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0db660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:42886/0 (socket says 172.21.15.26:42886) 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18b5c8e700 1 -- 172.21.15.26:0/1354615382 learned_addr learned my addr 172.21.15.26:0/1354615382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.722+0000 7f18b6c90700 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b5c8e700 1 -- 172.21.15.26:0/1354615382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 msgr2=0x55ec9e0d6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b5c8e700 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0d6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b5c8e700 1 -- 172.21.15.26:0/1354615382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 msgr2=0x55ec9e0db660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b5c8e700 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0db660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b5c8e700 1 -- 172.21.15.26:0/1354615382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9e0d3550 con 0x55ec9e0d82e0 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b648f700 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0db660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b6c90700 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0d6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f18b5c8e700 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0d22f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009f50 tx=0x7f18a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2595fcff1dd24890 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f189bfff700 1 -- 172.21.15.26:0/1354615382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0012c40 con 0x55ec9e0d82e0 2024-05-20T21:07:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f189bfff700 1 -- 172.21.15.26:0/1354615382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a0012de0 con 0x55ec9e0d82e0 2024-05-20T21:07:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.723+0000 7f189bfff700 1 -- 172.21.15.26:0/1354615382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0003d40 con 0x55ec9e0d82e0 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1354615382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 msgr2=0x55ec9e0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0d22f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009f50 tx=0x7f18a0009260 comp rx=0 tx=0).stop 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1354615382 shutdown_connections 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0d6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0db660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1354615382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0d22f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1354615382 >> 172.21.15.26:0/1354615382 conn(0x55ec9e0032c0 msgr2=0x55ec9e0dc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1354615382 shutdown_connections 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.724+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1354615382 wait complete. 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 -- start start 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 Processor -- start 2024-05-20T21:07:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f189400bb30 con 0x55ec9e0d82e0 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f189400cfa0 con 0x55ec9e0d7920 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f189400c200 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.725+0000 7f18c2f89b80 10 client.0 _ll_register_callbacks cb 0x55ec9dfc09d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:37938/0 (socket says 172.21.15.26:37938) 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 -- 172.21.15.26:0/1931966590 learned_addr learned my addr 172.21.15.26:0/1931966590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b5c8e700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b648f700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 -- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 msgr2=0x55ec9e0e99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 -- 172.21.15.26:0/1931966590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 msgr2=0x55ec9e0e9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 -- 172.21.15.26:0/1931966590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9e157920 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f18b6c90700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e69b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18a80a80e0 tx=0x7f18a80a95e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a95a5f4035c7b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.726+0000 7f1899ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f1899ffb700 1 -- 172.21.15.26:0/1931966590 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80a87e0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f1899ffb700 1 -- 172.21.15.26:0/1931966590 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f18a80a49e0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a0012de0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f1899ffb700 1 -- 172.21.15.26:0/1931966590 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18a80a4be0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 10 client.27952 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec9e164880 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f1899ffb700 1 -- 172.21.15.26:0/1931966590 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a0012de0 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f1899ffb700 1 -- 172.21.15.26:0/1931966590 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f18a808f130 con 0x55ec9e0dd3a0 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 10 client.27952 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 10 client.27952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 20 client.27952 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e164c70 0x55ec9e167090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 4 client.27952 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec9e031520 con 0x55ec9e164c70 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.727+0000 7f18b648f700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e164c70 0x55ec9e167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.728+0000 7f1899ffb700 10 client.27952 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.728+0000 7f18b648f700 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e164c70 0x55ec9e167090 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27fd924e51525ec server_cookie=8dce295a7cc3a0a9 in_seq=0 out_seq=0 2024-05-20T21:07:20.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f1899ffb700 1 -- 172.21.15.26:0/1931966590 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec9e031520 con 0x55ec9e164c70 2024-05-20T21:07:20.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f1899ffb700 10 client.27952 handle_command_reply: tid=0 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 client.27952 shutdown 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 msgr2=0x55ec9e0e69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e69b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18a80a80e0 tx=0x7f18a80a95e0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 shutdown_connections 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ec9e164c70 0x55ec9e167090 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec9e0dd3a0 0x55ec9e0e69b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ec9e0d7920 0x55ec9e0e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f1899ffb700 0 client.27952 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 --2- 172.21.15.26:0/1931966590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec9e0d82e0 0x55ec9e0e9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 >> 172.21.15.26:0/1931966590 conn(0x55ec9e0032c0 msgr2=0x55ec9e0d6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 shutdown_connections 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 1 -- 172.21.15.26:0/1931966590 wait complete. 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.729+0000 7f18c2f89b80 20 client.27952 trim_cache size 0 max 16384 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "c0541828-9697-4da8-8d30-5cd94d679735", 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:07:20.734 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:07:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f189c0b2880 msgr2=0x7f189c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f189c0b2880 0x7f189c0b4d40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00b2e0 tx=0x7f18ac005960 comp rx=0 tx=0).stop 2024-05-20T21:07:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 msgr2=0x7f18b01a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01a5960 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f18a8007f10 tx=0x7f18a800d3b0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 shutdown_connections 2024-05-20T21:07:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f189c0b2880 0x7f189c0b4d40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b01084c0 0x7f18b01a5960 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18b0102c40 0x7f18b019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 --2- 172.21.15.26:0/2014384884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0103600 0x7f18b019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 >> 172.21.15.26:0/2014384884 conn(0x7f18b0078ab0 msgr2=0x7f18b0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 shutdown_connections 2024-05-20T21:07:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:20.730+0000 7f18b86f3700 1 -- 172.21.15.26:0/2014384884 wait complete. 2024-05-20T21:07:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:21 smithi026 ceph-mon[31990]: pgmap v726: 97 pgs: 97 active+clean; 26 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-05-20T21:07:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:21 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:07:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:21 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:21 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:07:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:21 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:21.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:21 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:07:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:21 smithi144 ceph-mon[35660]: pgmap v726: 97 pgs: 97 active+clean; 26 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-05-20T21:07:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:21 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:07:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:21 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:21 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:07:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:21 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:21 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:07:21.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:07:21.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 248878 2.21 MB/sec execute 447 sec latency 13.618 ms 2024-05-20T21:07:21.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:21 smithi055 ceph-mon[35213]: pgmap v726: 97 pgs: 97 active+clean; 26 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-05-20T21:07:21.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:21 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:07:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:21 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:21 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:07:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:21 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:21 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:07:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:22.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 249505 2.21 MB/sec execute 448 sec latency 26.230 ms 2024-05-20T21:07:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 -- 172.21.15.26:0/3737741941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 msgr2=0x7f15c81016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3737741941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c81016a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f15b800aa10 tx=0x7f15b802b720 comp rx=0 tx=0).stop 2024-05-20T21:07:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 -- 172.21.15.26:0/3737741941 shutdown_connections 2024-05-20T21:07:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3737741941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 0x7f15c8109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3737741941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c8102330 0x7f15c8102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3737741941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c81016a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 -- 172.21.15.26:0/3737741941 >> 172.21.15.26:0/3737741941 conn(0x7f15c80fc270 msgr2=0x7f15c80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 -- 172.21.15.26:0/3737741941 shutdown_connections 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.705+0000 7f15cd56a640 1 -- 172.21.15.26:0/3737741941 wait complete. 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 Processor -- start 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 -- start start 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c8102330 0x7f15c8196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 0x7f15c819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810c270 con 0x7f15c8101280 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810c0f0 con 0x7f15c8102330 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15cd56a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810c3f0 con 0x7f15c8102cd0 2024-05-20T21:07:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c8196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56782/0 (socket says 172.21.15.26:56782) 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 -- 172.21.15.26:0/3267415106 learned_addr learned my addr 172.21.15.26:0/3267415106 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15bffff640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c8102330 0x7f15c8196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c77fe640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 0x7f15c819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 -- 172.21.15.26:0/3267415106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 msgr2=0x7f15c819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 0x7f15c819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 -- 172.21.15.26:0/3267415106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c8102330 msgr2=0x7f15c8196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c8102330 0x7f15c8196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.706+0000 7f15c6ffd640 1 -- 172.21.15.26:0/3267415106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c819e540 con 0x7f15c8101280 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15c77fe640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 0x7f15c819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c8196920 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f15b800ab30 tx=0x7f15b80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15c4ff9640 1 -- 172.21.15.26:0/3267415106 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8040070 con 0x7f15c8101280 2024-05-20T21:07:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15c4ff9640 1 -- 172.21.15.26:0/3267415106 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b8003c80 con 0x7f15c8101280 2024-05-20T21:07:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c819e7d0 con 0x7f15c8101280 2024-05-20T21:07:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15c4ff9640 1 -- 172.21.15.26:0/3267415106 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8036030 con 0x7f15c8101280 2024-05-20T21:07:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.707+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c819ed10 con 0x7f15c8101280 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 Processor -- start 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15c4ff9640 1 -- 172.21.15.26:0/3267415106 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f15b8043070 con 0x7f15c8101280 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 -- start start 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bacf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bad01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55985ba45c30 con 0x55985bade9a0 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55985ba357d0 con 0x55985bad10a0 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15d0282740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55985ba39810 con 0x55985bacfd30 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bacf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56788/0 (socket says 172.21.15.26:56788) 2024-05-20T21:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15bffff640 1 -- 172.21.15.26:0/3877023406 learned_addr learned my addr 172.21.15.26:0/3877023406 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.710+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bad01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bffff640 1 -- 172.21.15.26:0/3877023406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 msgr2=0x55985bad01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bffff640 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bad01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bffff640 1 -- 172.21.15.26:0/3877023406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 msgr2=0x55985bacf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bffff640 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bacf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bffff640 1 -- 172.21.15.26:0/3877023406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985b74a210 con 0x55985bade9a0 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15c77fe640 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bacf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15c4ff9640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b40b2920 0x7f15b40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bffff640 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae0d90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009c90 tx=0x7f15b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88433e1f4c3c2946 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15c77fe640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b40b2920 0x7f15b40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15c4ff9640 1 -- 172.21.15.26:0/3267415106 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f15b8102cb0 con 0x7f15c8101280 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bd7fa640 1 -- 172.21.15.26:0/3877023406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b001e070 con 0x55985bade9a0 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bd7fa640 1 -- 172.21.15.26:0/3877023406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b0019ac0 con 0x55985bade9a0 2024-05-20T21:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.711+0000 7f15bd7fa640 1 -- 172.21.15.26:0/3877023406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b000e930 con 0x55985bade9a0 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 -- 172.21.15.26:0/3877023406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 msgr2=0x55985bae0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15c77fe640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b40b2920 0x7f15b40b4de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55985bad0c30 tx=0x7f15c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae0d90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009c90 tx=0x7f15b0011810 comp rx=0 tx=0).stop 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 -- 172.21.15.26:0/3877023406 shutdown_connections 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae0d90 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bad01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 --2- 172.21.15.26:0/3877023406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bacf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 -- 172.21.15.26:0/3877023406 >> 172.21.15.26:0/3877023406 conn(0x55985bad1f40 msgr2=0x55985bad2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 -- 172.21.15.26:0/3877023406 shutdown_connections 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.712+0000 7f15d0282740 1 -- 172.21.15.26:0/3877023406 wait complete. 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 -- start start 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 Processor -- start 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985bae3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bade9a0 0x55985baea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55985b71b790 con 0x55985bad10a0 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55985b9f1890 con 0x55985bade9a0 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55985bae15d0 con 0x55985bacfd30 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15d0282740 10 client.0 _ll_register_callbacks cb 0x55985b749a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985bae3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985bae3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56800/0 (socket says 172.21.15.26:56800) 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 -- 172.21.15.26:0/3299944868 learned_addr learned my addr 172.21.15.26:0/3299944868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15c77fe640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bade9a0 0x55985baea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 -- 172.21.15.26:0/3299944868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 msgr2=0x55985bae2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 -- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bade9a0 msgr2=0x55985baea160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bade9a0 0x55985baea160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.713+0000 7f15bffff640 1 -- 172.21.15.26:0/3299944868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985b816620 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15bffff640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985bae3120 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f15b0019660 tx=0x7f15b0019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9016ed9562368d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15ae7fc640 1 -- 172.21.15.26:0/3299944868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0029030 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15ae7fc640 1 -- 172.21.15.26:0/3299944868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f15b00198b0 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b0019ac0 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15ae7fc640 1 -- 172.21.15.26:0/3299944868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15b0033050 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 10 client.18870 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55985baed140 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15ae7fc640 1 -- 172.21.15.26:0/3299944868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b0019ac0 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15ae7fc640 1 -- 172.21.15.26:0/3299944868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f15b0004a60 con 0x55985bad10a0 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 10 client.18870 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 10 client.18870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 20 client.18870 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.714+0000 7f15d0282740 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985baee650 0x55985baf0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.715+0000 7f15d0282740 4 client.18870 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.715+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55985ba47b80 con 0x55985baee650 2024-05-20T21:07:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.715+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985baee650 0x55985baf0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.715+0000 7f15ae7fc640 10 client.18870 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.715+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985baee650 0x55985baf0a40 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0531d15bfec108a server_cookie=ef1b031cf06a84c8 in_seq=0 out_seq=0 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15ae7fc640 1 -- 172.21.15.26:0/3299944868 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55985ba47b80 con 0x55985baee650 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15ae7fc640 10 client.18870 handle_command_reply: tid=0 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 client.18870 shutdown 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 msgr2=0x55985bae3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985bae3120 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f15b0019660 tx=0x7f15b0019690 comp rx=0 tx=0).stop 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 shutdown_connections 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985baee650 0x55985baf0a40 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bade9a0 0x55985baea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985bae3120 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15ae7fc640 0 client.18870 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 --2- 172.21.15.26:0/3299944868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 >> 172.21.15.26:0/3299944868 conn(0x55985bad1f40 msgr2=0x55985bacefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 shutdown_connections 2024-05-20T21:07:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 1 -- 172.21.15.26:0/3299944868 wait complete. 2024-05-20T21:07:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.716+0000 7f15d0282740 20 client.18870 trim_cache size 0 max 16384 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 Processor -- start 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 -- start start 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bacfd30 0x55985baf5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985baea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bade9a0 0x55985baf5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81aa550 con 0x55985bad10a0 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81a7e20 con 0x55985bacfd30 2024-05-20T21:07:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15d0282740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81a5c70 con 0x55985bade9a0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985baea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985baea120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56802/0 (socket says 172.21.15.26:56802) 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15bffff640 1 -- 172.21.15.26:0/2691837215 learned_addr learned my addr 172.21.15.26:0/2691837215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bacfd30 0x55985baf5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15c77fe640 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bade9a0 0x55985baf5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15bffff640 1 -- 172.21.15.26:0/2691837215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bade9a0 msgr2=0x55985baf5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.734+0000 7f15bffff640 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bade9a0 0x55985baf5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bffff640 1 -- 172.21.15.26:0/2691837215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bacfd30 msgr2=0x55985baf5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bffff640 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bacfd30 0x55985baf5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bffff640 1 -- 172.21.15.26:0/2691837215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985bad1d50 con 0x55985bad10a0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bffff640 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985baea120 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f15b0011850 tx=0x7f15b0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c85ab59c5571fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bcff9640 1 -- 172.21.15.26:0/2691837215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0029070 con 0x55985bad10a0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bcff9640 1 -- 172.21.15.26:0/2691837215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b000e8d0 con 0x55985bad10a0 2024-05-20T21:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.735+0000 7f15bcff9640 1 -- 172.21.15.26:0/2691837215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b001e040 con 0x55985bad10a0 2024-05-20T21:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 -- 172.21.15.26:0/2691837215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 msgr2=0x55985baea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985baea120 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f15b0011850 tx=0x7f15b0005f90 comp rx=0 tx=0).stop 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 -- 172.21.15.26:0/2691837215 shutdown_connections 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bade9a0 0x55985baf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bad10a0 0x55985baea120 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 --2- 172.21.15.26:0/2691837215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bacfd30 0x55985baf5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 -- 172.21.15.26:0/2691837215 >> 172.21.15.26:0/2691837215 conn(0x55985bad1f40 msgr2=0x55985bacefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 -- 172.21.15.26:0/2691837215 shutdown_connections 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 -- 172.21.15.26:0/2691837215 wait complete. 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 -- start start 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.736+0000 7f15d0282740 1 Processor -- start 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bae7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81a62e0 con 0x55985bade9a0 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81a7750 con 0x55985bad10a0 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81a69b0 con 0x55985bacfd30 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15d0282740 10 client.0 _ll_register_callbacks cb 0x7f15b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38002/0 (socket says 172.21.15.26:38002) 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 -- 172.21.15.26:0/1723519049 learned_addr learned my addr 172.21.15.26:0/1723519049 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15bffff640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bae7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 -- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 msgr2=0x55985bae7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bae7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 -- 172.21.15.26:0/1723519049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 msgr2=0x55985bae8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 -- 172.21.15.26:0/1723519049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985bae9190 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c77fe640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15c6ffd640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae7650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15b80cb300 tx=0x7f15b80c72f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d3554b3fbf207 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.737+0000 7f15ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15ae7fc640 1 -- 172.21.15.26:0/1723519049 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b804a040 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15ae7fc640 1 -- 172.21.15.26:0/1723519049 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f15b80037f0 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b000e8d0 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15ae7fc640 1 -- 172.21.15.26:0/1723519049 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15b80caae0 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 10 client.27970 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55985bb591e0 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15ae7fc640 1 -- 172.21.15.26:0/1723519049 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b000e8d0 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15ae7fc640 1 -- 172.21.15.26:0/1723519049 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f15b800b040 con 0x55985bacfd30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 10 client.27970 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 10 client.27970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 20 client.27970 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985bb599a0 0x55985bb5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 4 client.27970 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55985ba41360 con 0x55985bb599a0 2024-05-20T21:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.738+0000 7f15bffff640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985bb599a0 0x55985bb5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.739+0000 7f15ae7fc640 10 client.27970 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.739+0000 7f15bffff640 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985bb599a0 0x55985bb5bdc0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3706e65b6fe024b1 server_cookie=b655b4148fef53a5 in_seq=0 out_seq=0 2024-05-20T21:07:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.912+0000 7f15ae7fc640 1 -- 172.21.15.26:0/1723519049 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55985ba41360 con 0x55985bb599a0 2024-05-20T21:07:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.912+0000 7f15ae7fc640 10 client.27970 handle_command_reply: tid=0 2024-05-20T21:07:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 client.27970 shutdown 2024-05-20T21:07:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 msgr2=0x55985bae7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae7650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15b80cb300 tx=0x7f15b80c72f0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 shutdown_connections 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55985bb599a0 0x55985bb5bdc0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bade9a0 0x55985bae8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15ae7fc640 0 client.27970 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55985bad10a0 0x55985bae7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 --2- 172.21.15.26:0/1723519049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55985bacfd30 0x55985bae7650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 >> 172.21.15.26:0/1723519049 conn(0x55985bad1f40 msgr2=0x55985bace720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 shutdown_connections 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 1 -- 172.21.15.26:0/1723519049 wait complete. 2024-05-20T21:07:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.913+0000 7f15d0282740 20 client.27970 trim_cache size 0 max 16384 2024-05-20T21:07:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:07:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:07:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b40b2920 msgr2=0x7f15b40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b40b2920 0x7f15b40b4de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55985bad0c30 tx=0x7f15c0009500 comp rx=0 tx=0).stop 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 msgr2=0x7f15c8196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c8196920 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f15b800ab30 tx=0x7f15b80040c0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 shutdown_connections 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f15b40b2920 0x7f15b40b4de0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15c8102cd0 0x7f15c819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15c8102330 0x7f15c8196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 --2- 172.21.15.26:0/3267415106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c8101280 0x7f15c8196920 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 >> 172.21.15.26:0/3267415106 conn(0x7f15c80fc270 msgr2=0x7f15c81056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 shutdown_connections 2024-05-20T21:07:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:22.914+0000 7f15cd56a640 1 -- 172.21.15.26:0/3267415106 wait complete. 2024-05-20T21:07:23.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:23 smithi026 ceph-mon[31990]: pgmap v727: 97 pgs: 97 active+clean; 28 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T21:07:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:23 smithi144 ceph-mon[35660]: pgmap v727: 97 pgs: 97 active+clean; 28 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T21:07:23.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 250175 2.21 MB/sec execute 449 sec latency 37.572 ms 2024-05-20T21:07:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 -- 172.21.15.26:0/3269428019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168101e60 msgr2=0x7fe168102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 --2- 172.21.15.26:0/3269428019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168101e60 0x7fe168102240 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe1640099c0 tx=0x7fe164009cd0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 -- 172.21.15.26:0/3269428019 shutdown_connections 2024-05-20T21:07:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 --2- 172.21.15.26:0/3269428019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168101e60 0x7fe168102240 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 --2- 172.21.15.26:0/3269428019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168106ef0 0x7fe16810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 --2- 172.21.15.26:0/3269428019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe168102810 0x7fe168106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 -- 172.21.15.26:0/3269428019 >> 172.21.15.26:0/3269428019 conn(0x7fe168078ab0 msgr2=0x7fe1680ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 -- 172.21.15.26:0/3269428019 shutdown_connections 2024-05-20T21:07:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.607+0000 7fe1702eb700 1 -- 172.21.15.26:0/3269428019 wait complete. 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 Processor -- start 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 -- start start 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168101e60 0x7fe16819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe168102810 0x7fe16819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 0x7fe168199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111400 con 0x7fe168106ef0 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111280 con 0x7fe168101e60 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.608+0000 7fe1702eb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111580 con 0x7fe168102810 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 0x7fe168199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 0x7fe168199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56814/0 (socket says 172.21.15.26:56814) 2024-05-20T21:07:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e087700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168101e60 0x7fe16819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 -- 172.21.15.26:0/2531354032 learned_addr learned my addr 172.21.15.26:0/2531354032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16d886700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe168102810 0x7fe16819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 -- 172.21.15.26:0/2531354032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe168102810 msgr2=0x7fe16819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe168102810 0x7fe16819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 -- 172.21.15.26:0/2531354032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168101e60 msgr2=0x7fe16819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168101e60 0x7fe16819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 -- 172.21.15.26:0/2531354032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16819a0e0 con 0x7fe168106ef0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e087700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168101e60 0x7fe16819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe16e888700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 0x7fe168199980 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe16000ef30 tx=0x7fe16000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe15f7fe700 1 -- 172.21.15.26:0/2531354032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600092e0 con 0x7fe168106ef0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe15f7fe700 1 -- 172.21.15.26:0/2531354032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe160004bb0 con 0x7fe168106ef0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe15f7fe700 1 -- 172.21.15.26:0/2531354032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160004020 con 0x7fe168106ef0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.609+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16819a370 con 0x7fe168106ef0 2024-05-20T21:07:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.610+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a6690 con 0x7fe168106ef0 2024-05-20T21:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.612+0000 7fe15f7fe700 1 -- 172.21.15.26:0/2531354032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe16001b030 con 0x7fe168106ef0 2024-05-20T21:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.613+0000 7fe15f7fe700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1540b2900 0x7fe1540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.613+0000 7fe16e087700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1540b2900 0x7fe1540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.613+0000 7fe15f7fe700 1 -- 172.21.15.26:0/2531354032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe1600e1c70 con 0x7fe168106ef0 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.613+0000 7fe17ab81b80 1 Processor -- start 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 -- start start 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16e087700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1540b2900 0x7fe1540b4dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe164000c00 tx=0x7fe164005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2dc70 0x56037df2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56037de88c90 con 0x56037df32370 2024-05-20T21:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56037de78830 con 0x56037df2e620 2024-05-20T21:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56037de7c870 con 0x56037df2dc70 2024-05-20T21:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16e087700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2dc70 0x56037df2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16d886700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16e087700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2dc70 0x56037df2e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38020/0 (socket says 172.21.15.26:38020) 2024-05-20T21:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16e888700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16e087700 1 -- 172.21.15.26:0/3529342463 learned_addr learned my addr 172.21.15.26:0/3529342463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16d886700 1 -- 172.21.15.26:0/3529342463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2dc70 msgr2=0x56037df2e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16d886700 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2dc70 0x56037df2e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16d886700 1 -- 172.21.15.26:0/3529342463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 msgr2=0x56037df327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16d886700 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df327b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.614+0000 7fe16d886700 1 -- 172.21.15.26:0/3529342463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037ddc88d0 con 0x56037df2e620 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.615+0000 7fe16d886700 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31d50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe168075dd0 tx=0x7fe15800a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67be9ff888f819fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.615+0000 7fe15cd70700 1 -- 172.21.15.26:0/3529342463 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15800ed20 con 0x56037df2e620 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.615+0000 7fe15cd70700 1 -- 172.21.15.26:0/3529342463 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1580044c0 con 0x56037df2e620 2024-05-20T21:07:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.615+0000 7fe15cd70700 1 -- 172.21.15.26:0/3529342463 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158018430 con 0x56037df2e620 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 -- 172.21.15.26:0/3529342463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 msgr2=0x56037df31d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31d50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe168075dd0 tx=0x7fe15800a370 comp rx=0 tx=0).stop 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 -- 172.21.15.26:0/3529342463 shutdown_connections 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2dc70 0x56037df2e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/3529342463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31d50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 -- 172.21.15.26:0/3529342463 >> 172.21.15.26:0/3529342463 conn(0x56037de57080 msgr2=0x56037df23e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 -- 172.21.15.26:0/3529342463 shutdown_connections 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 -- 172.21.15.26:0/3529342463 wait complete. 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 -- start start 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.616+0000 7fe17ab81b80 1 Processor -- start 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df39780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df36a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df32370 0x56037df36fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56037dd7efa0 con 0x56037df2dc70 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56037de1a7c0 con 0x56037df32370 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56037df3a530 con 0x56037df2e620 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e087700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df39780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e087700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df39780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56842/0 (socket says 172.21.15.26:56842) 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e087700 1 -- 172.21.15.26:0/1866820095 learned_addr learned my addr 172.21.15.26:0/1866820095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e888700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df32370 0x56037df36fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16d886700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df36a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe17ab81b80 10 client.0 _ll_register_callbacks cb 0x56037de13920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e888700 1 -- 172.21.15.26:0/1866820095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 msgr2=0x56037df36a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e888700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df36a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e888700 1 -- 172.21.15.26:0/1866820095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 msgr2=0x56037df39780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e888700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df39780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e888700 1 -- 172.21.15.26:0/1866820095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037de17690 con 0x56037df32370 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.617+0000 7fe16e087700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df39780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe16e888700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df32370 0x56037df36fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe1600a64d0 tx=0x7fe160003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d42169fa6caad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe151ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe151ffb700 1 -- 172.21.15.26:0/1866820095 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600c3060 con 0x56037df32370 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe151ffb700 1 -- 172.21.15.26:0/1866820095 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe1600b1db0 con 0x56037df32370 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1580044c0 con 0x56037df32370 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe151ffb700 1 -- 172.21.15.26:0/1866820095 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1600ca050 con 0x56037df32370 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 10 client.28187 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56037df44c60 con 0x56037df32370 2024-05-20T21:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe151ffb700 1 -- 172.21.15.26:0/1866820095 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1580044c0 con 0x56037df32370 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe151ffb700 1 -- 172.21.15.26:0/1866820095 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fe1600ad170 con 0x56037df32370 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 10 client.28187 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 10 client.28187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 20 client.28187 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df471f0 0x56037df49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 4 client.28187 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.618+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56037def59e0 con 0x56037df471f0 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.619+0000 7fe16e087700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df471f0 0x56037df49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.619+0000 7fe16e087700 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df471f0 0x56037df49660 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5316358864de07 server_cookie=b717d1d01d42df1f in_seq=0 out_seq=0 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.619+0000 7fe151ffb700 10 client.28187 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.620+0000 7fe151ffb700 1 -- 172.21.15.26:0/1866820095 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56037def59e0 con 0x56037df471f0 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.620+0000 7fe151ffb700 10 client.28187 handle_command_reply: tid=0 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.620+0000 7fe17ab81b80 1 client.28187 shutdown 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df32370 msgr2=0x56037df36fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df32370 0x56037df36fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe1600a64d0 tx=0x7fe160003ce0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 shutdown_connections 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df471f0 0x56037df49660 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df39780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df32370 0x56037df36fc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/1866820095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df36a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe151ffb700 0 client.28187 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 >> 172.21.15.26:0/1866820095 conn(0x56037de57080 msgr2=0x56037df23510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 shutdown_connections 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 1 -- 172.21.15.26:0/1866820095 wait complete. 2024-05-20T21:07:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.621+0000 7fe17ab81b80 20 client.28187 trim_cache size 0 max 16384 2024-05-20T21:07:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.643+0000 7fe17ab81b80 1 Processor -- start 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.643+0000 7fe17ab81b80 1 -- start start 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df32370 0x56037df28d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681ae350 con 0x56037df2dc70 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681abbe0 con 0x56037df2e620 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681a9a30 con 0x56037df32370 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16e087700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16e087700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56852/0 (socket says 172.21.15.26:56852) 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16e087700 1 -- 172.21.15.26:0/205775270 learned_addr learned my addr 172.21.15.26:0/205775270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16e888700 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df32370 0x56037df28d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 -- 172.21.15.26:0/205775270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df32370 msgr2=0x56037df28d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df32370 0x56037df28d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 -- 172.21.15.26:0/205775270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 msgr2=0x56037df356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 -- 172.21.15.26:0/205775270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037df20790 con 0x56037df2e620 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16e087700 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.644+0000 7fe16d886700 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31550 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a040 tx=0x7fe158003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e36f4895b08c26 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe153fff700 1 -- 172.21.15.26:0/205775270 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158028070 con 0x56037df2e620 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe153fff700 1 -- 172.21.15.26:0/205775270 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe15800eb20 con 0x56037df2e620 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe153fff700 1 -- 172.21.15.26:0/205775270 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158005660 con 0x56037df2e620 2024-05-20T21:07:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 -- 172.21.15.26:0/205775270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 msgr2=0x56037df31550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31550 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a040 tx=0x7fe158003680 comp rx=0 tx=0).stop 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 -- 172.21.15.26:0/205775270 shutdown_connections 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df2dc70 0x56037df356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df32370 0x56037df28d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/205775270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2e620 0x56037df31550 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 -- 172.21.15.26:0/205775270 >> 172.21.15.26:0/205775270 conn(0x56037de57080 msgr2=0x56037df28670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 -- 172.21.15.26:0/205775270 shutdown_connections 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.645+0000 7fe17ab81b80 1 -- 172.21.15.26:0/205775270 wait complete. 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 -- start start 2024-05-20T21:07:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 Processor -- start 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2dc70 0x56037df3fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681aa0a0 con 0x56037df32370 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681ab510 con 0x56037df2dc70 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681aa770 con 0x56037df2e620 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe17ab81b80 10 client.0 _ll_register_callbacks cb 0x56037de13920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe16e888700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe16d886700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe16e888700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df3d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56858/0 (socket says 172.21.15.26:56858) 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe16e888700 1 -- 172.21.15.26:0/784370062 learned_addr learned my addr 172.21.15.26:0/784370062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.646+0000 7fe16e087700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2dc70 0x56037df3fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16d886700 1 -- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2dc70 msgr2=0x56037df3fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16d886700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2dc70 0x56037df3fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16d886700 1 -- 172.21.15.26:0/784370062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 msgr2=0x56037df3d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16d886700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df3d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16d886700 1 -- 172.21.15.26:0/784370062 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037df3dc70 con 0x56037df2e620 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16e888700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df3d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe16d886700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df3d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a040 tx=0x7fe15800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f732c1886dbec24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe151ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe151ffb700 1 -- 172.21.15.26:0/784370062 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158035070 con 0x56037df2e620 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe151ffb700 1 -- 172.21.15.26:0/784370062 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe158005070 con 0x56037df2e620 2024-05-20T21:07:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15800eb20 con 0x56037df2e620 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe151ffb700 1 -- 172.21.15.26:0/784370062 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe158028070 con 0x56037df2e620 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe17ab81b80 10 client.27988 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.647+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56037df2a520 con 0x56037df2e620 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe151ffb700 1 -- 172.21.15.26:0/784370062 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe15800eb20 con 0x56037df2e620 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe151ffb700 1 -- 172.21.15.26:0/784370062 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fe158038030 con 0x56037df2e620 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe17ab81b80 10 client.27988 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe17ab81b80 10 client.27988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe17ab81b80 20 client.27988 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df2a9c0 0x56037df2cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe17ab81b80 4 client.27988 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56037de8abe0 con 0x56037df2a9c0 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe16e087700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df2a9c0 0x56037df2cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe151ffb700 10 client.27988 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.648+0000 7fe16e087700 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df2a9c0 0x56037df2cde0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72ae21b441be577 server_cookie=3d0fe55ec8f8ac65 in_seq=0 out_seq=0 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe151ffb700 1 -- 172.21.15.26:0/784370062 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56037de8abe0 con 0x56037df2a9c0 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe151ffb700 10 client.27988 handle_command_reply: tid=0 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 client.27988 shutdown 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 msgr2=0x56037df3d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df3d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a040 tx=0x7fe15800ef50 comp rx=0 tx=0).stop 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 shutdown_connections 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56037df2a9c0 0x56037df2cde0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56037df32370 0x56037df3d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56037df2dc70 0x56037df3fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe151ffb700 0 client.27988 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 --2- 172.21.15.26:0/784370062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56037df2e620 0x56037df3d0c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 >> 172.21.15.26:0/784370062 conn(0x56037de57080 msgr2=0x56037df28f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 shutdown_connections 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 1 -- 172.21.15.26:0/784370062 wait complete. 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.649+0000 7fe17ab81b80 20 client.27988 trim_cache size 0 max 16384 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:07:23.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1540b2900 msgr2=0x7fe1540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1540b2900 0x7fe1540b4dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe164000c00 tx=0x7fe164005fb0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 msgr2=0x7fe168199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 0x7fe168199980 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe16000ef30 tx=0x7fe16000c3a0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 shutdown_connections 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe1540b2900 0x7fe1540b4dc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe168106ef0 0x7fe168199980 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe168101e60 0x7fe16819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 --2- 172.21.15.26:0/2531354032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe168102810 0x7fe16819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 >> 172.21.15.26:0/2531354032 conn(0x7fe168078ab0 msgr2=0x7fe168100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 shutdown_connections 2024-05-20T21:07:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:23.650+0000 7fe1702eb700 1 -- 172.21.15.26:0/2531354032 wait complete. 2024-05-20T21:07:23.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:23 smithi055 ceph-mon[35213]: pgmap v727: 97 pgs: 97 active+clean; 28 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-05-20T21:07:24.400 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:07:24.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:07:24.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 250797 2.21 MB/sec execute 450 sec latency 20.293 ms 2024-05-20T21:07:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:25.401 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:25 smithi026 ceph-mon[31990]: pgmap v728: 97 pgs: 97 active+clean; 28 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:07:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:25 smithi144 ceph-mon[35660]: pgmap v728: 97 pgs: 97 active+clean; 28 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:07:25.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 251629 2.22 MB/sec execute 451 sec latency 17.765 ms 2024-05-20T21:07:25.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:25 smithi055 ceph-mon[35213]: pgmap v728: 97 pgs: 97 active+clean; 28 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-20T21:07:26.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 252158 2.23 MB/sec execute 452 sec latency 18.528 ms 2024-05-20T21:07:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3566382833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81037f0 msgr2=0x7fcbc8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3566382833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81037f0 0x7fcbc8103bd0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800b0d0 tx=0x7fcbb800b3e0 comp rx=0 tx=0).stop 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3566382833 shutdown_connections 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3566382833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81037f0 0x7fcbc8103bd0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3566382833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81088f0 0x7fcbc8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3566382833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc81041a0 0x7fcbc81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3566382833 >> 172.21.15.26:0/3566382833 conn(0x7fcbc807a6a0 msgr2=0x7fcbc807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3566382833 shutdown_connections 2024-05-20T21:07:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.805+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3566382833 wait complete. 2024-05-20T21:07:26.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 Processor -- start 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 -- start start 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81037f0 0x7fcbc819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 0x7fcbc819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc81088f0 0x7fcbc81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcd27d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 0x7fcbc819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcda7e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81037f0 0x7fcbc819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8113570 con 0x7fcbc81041a0 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcfce2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81133f0 con 0x7fcbc81037f0 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.806+0000 7fcbcd27d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 0x7fcbc819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56870/0 (socket says 172.21.15.26:56870) 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcd27d700 1 -- 172.21.15.26:0/3339246457 learned_addr learned my addr 172.21.15.26:0/3339246457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81136f0 con 0x7fcbc81088f0 2024-05-20T21:07:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbce27f700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc81088f0 0x7fcbc81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcda7e700 1 -- 172.21.15.26:0/3339246457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc81088f0 msgr2=0x7fcbc81998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcda7e700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc81088f0 0x7fcbc81998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcda7e700 1 -- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 msgr2=0x7fcbc819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcda7e700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 0x7fcbc819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcda7e700 1 -- 172.21.15.26:0/3339246457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc819a010 con 0x7fcbc81037f0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcd27d700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 0x7fcbc819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcda7e700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81037f0 0x7fcbc819f6f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800b770 tx=0x7fcbb8007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb801f070 con 0x7fcbc81037f0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc819a2a0 con 0x7fcbc81037f0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbb8004c50 con 0x7fcbc81037f0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.807+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb801b690 con 0x7fcbc81037f0 2024-05-20T21:07:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.808+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81a65c0 con 0x7fcbc81037f0 2024-05-20T21:07:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.808+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc806a230 con 0x7fcbc81037f0 2024-05-20T21:07:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.810+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcbb8022070 con 0x7fcbc81037f0 2024-05-20T21:07:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.811+0000 7fcbc6ffd700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbb00b2900 0x7fcbb00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.811+0000 7fcbcd27d700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbb00b2900 0x7fcbb00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.811+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcbb800f020 con 0x7fcbc81037f0 2024-05-20T21:07:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.811+0000 7fcbcd27d700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbb00b2900 0x7fcbb00b4dc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8103130 tx=0x7fcbbc00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:26.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:26.816+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbb8029020 con 0x7fcbc81037f0 2024-05-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.106+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbc819ade0 con 0x7fcbc81037f0 2024-05-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.107+0000 7fcbc6ffd700 1 -- 172.21.15.26:0/3339246457 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fcbb80ab460 con 0x7fcbc81037f0 2024-05-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbb00b2900 msgr2=0x7fcbb00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbb00b2900 0x7fcbb00b4dc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8103130 tx=0x7fcbbc00b500 comp rx=0 tx=0).stop 2024-05-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81037f0 msgr2=0x7fcbc819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81037f0 0x7fcbc819f6f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800b770 tx=0x7fcbb8007a20 comp rx=0 tx=0).stop 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 shutdown_connections 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc81088f0 0x7fcbc81998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbc81037f0 0x7fcbc819f6f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcbb00b2900 0x7fcbb00b4dc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 --2- 172.21.15.26:0/3339246457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc81041a0 0x7fcbc819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 >> 172.21.15.26:0/3339246457 conn(0x7fcbc807a6a0 msgr2=0x7fcbc8109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 shutdown_connections 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:27.110+0000 7fcbcfce2700 1 -- 172.21.15.26:0/3339246457 wait complete. 2024-05-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:07:27.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:27 smithi026 ceph-mon[31990]: pgmap v729: 97 pgs: 97 active+clean; 28 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-20T21:07:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:27 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3339246457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:27 smithi144 ceph-mon[35660]: pgmap v729: 97 pgs: 97 active+clean; 28 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-20T21:07:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:27 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3339246457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:27.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 252565 2.22 MB/sec execute 453 sec latency 37.548 ms 2024-05-20T21:07:27.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:27 smithi055 ceph-mon[35213]: pgmap v729: 97 pgs: 97 active+clean; 28 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-20T21:07:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:27 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3339246457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:27.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:07:28.239 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:28.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 253270 2.23 MB/sec execute 454 sec latency 17.554 ms 2024-05-20T21:07:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:29 smithi144 ceph-mon[35660]: pgmap v730: 97 pgs: 97 active+clean; 27 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-05-20T21:07:29.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 253944 2.22 MB/sec execute 455 sec latency 17.351 ms 2024-05-20T21:07:29.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:29 smithi026 ceph-mon[31990]: pgmap v730: 97 pgs: 97 active+clean; 27 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-05-20T21:07:29.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:29 smithi055 ceph-mon[35213]: pgmap v730: 97 pgs: 97 active+clean; 27 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-05-20T21:07:30.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 254623 2.23 MB/sec execute 456 sec latency 66.231 ms 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 -- 172.21.15.26:0/1425450823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102440 msgr2=0x7f7c9c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/1425450823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102440 0x7f7c9c1028c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009990 tx=0x7f7c84009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 -- 172.21.15.26:0/1425450823 shutdown_connections 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/1425450823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c9c102e00 0x7f7c9c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/1425450823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c108520 0x7f7c9c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/1425450823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102440 0x7f7c9c1028c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 -- 172.21.15.26:0/1425450823 >> 172.21.15.26:0/1425450823 conn(0x7f7c9c0793f0 msgr2=0x7f7c9c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 -- 172.21.15.26:0/1425450823 shutdown_connections 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.552+0000 7f7ca1e47700 1 -- 172.21.15.26:0/1425450823 wait complete. 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 Processor -- start 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 -- start start 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c102440 0x7f7c9c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102e00 0x7f7c9c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c9c108520 0x7f7c9c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1124a0 con 0x7f7c9c102e00 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c112320 con 0x7f7c9c102440 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7ca1e47700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c112620 con 0x7f7c9c108520 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102e00 0x7f7c9c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7c937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c102440 0x7f7c9c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102e00 0x7f7c9c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58408/0 (socket says 172.21.15.26:58408) 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.553+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/2807281563 learned_addr learned my addr 172.21.15.26:0/2807281563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c9bfff700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c9c108520 0x7f7c9c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c937fe700 1 -- 172.21.15.26:0/2807281563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c9c108520 msgr2=0x7f7c9c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c937fe700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c9c108520 0x7f7c9c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c937fe700 1 -- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102e00 msgr2=0x7f7c9c19c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c937fe700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102e00 0x7f7c9c19c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c937fe700 1 -- 172.21.15.26:0/2807281563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c1a0d70 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c937fe700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c102440 0x7f7c9c19bd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00e9f0 tx=0x7f7c8c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c997fa700 1 -- 172.21.15.26:0/2807281563 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00c9c0 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c997fa700 1 -- 172.21.15.26:0/2807281563 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c8c010430 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c1a1060 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c1a88e0 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.554+0000 7f7c997fa700 1 -- 172.21.15.26:0/2807281563 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c018620 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.557+0000 7f7c997fa700 1 -- 172.21.15.26:0/2807281563 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7c8c00cb60 con 0x7f7c9c102440 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.558+0000 7f7c997fa700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c880b2880 0x7f7c880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.558+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c880b2880 0x7f7c880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.558+0000 7f7c997fa700 1 -- 172.21.15.26:0/2807281563 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7c8c0e22e0 con 0x7f7c9c102440 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 Processor -- start 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c880b2880 0x7f7c880b4d40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c102290 tx=0x7f7c84005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 -- start start 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e571860 0x55ba7e571c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 0x55ba7e575940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e575f60 0x55ba7e5763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba7e4d0c90 con 0x55ba7e575f60 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ba7e4c0830 con 0x55ba7e572210 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ba7e4c4870 con 0x55ba7e571860 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7c9bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e575f60 0x55ba7e5763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 0x55ba7e575940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 0x55ba7e575940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50522/0 (socket says 172.21.15.26:50522) 2024-05-20T21:07:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/3392977599 learned_addr learned my addr 172.21.15.26:0/3392977599 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.559+0000 7f7c937fe700 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e571860 0x55ba7e571c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/3392977599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e571860 msgr2=0x55ba7e571c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e571860 0x55ba7e571c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/3392977599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e575f60 msgr2=0x55ba7e5763a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e575f60 0x55ba7e5763a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/3392977599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba7e4108d0 con 0x55ba7e572210 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 0x55ba7e575940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c84000c00 tx=0x7f7c8401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3965240758b0d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c91ffb700 1 -- 172.21.15.26:0/3392977599 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84033070 con 0x55ba7e572210 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c91ffb700 1 -- 172.21.15.26:0/3392977599 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c8400bb40 con 0x55ba7e572210 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.560+0000 7f7c91ffb700 1 -- 172.21.15.26:0/3392977599 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400fbc0 con 0x55ba7e572210 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/3392977599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 msgr2=0x55ba7e575940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 0x55ba7e575940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c84000c00 tx=0x7f7c8401f000 comp rx=0 tx=0).stop 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/3392977599 shutdown_connections 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e575f60 0x55ba7e5763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e571860 0x55ba7e571c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/3392977599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e572210 0x55ba7e575940 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/3392977599 >> 172.21.15.26:0/3392977599 conn(0x55ba7e49f080 msgr2=0x55ba7e56be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/3392977599 shutdown_connections 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.561+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/3392977599 wait complete. 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 -- start start 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 Processor -- start 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e57d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e57ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba7e3c6fa0 con 0x55ba7e572210 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ba7e4627c0 con 0x55ba7e571860 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ba7e57e0c0 con 0x55ba7e575f60 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7cac6ddb80 10 client.0 _ll_register_callbacks cb 0x55ba7e45b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7c937fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e57d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.562+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58430/0 (socket says 172.21.15.26:58430) 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/1724944969 learned_addr learned my addr 172.21.15.26:0/1724944969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c9bfff700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e57ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c937fe700 1 -- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 msgr2=0x55ba7e57ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c937fe700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e57ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c937fe700 1 -- 172.21.15.26:0/1724944969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 msgr2=0x55ba7e57a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c937fe700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c937fe700 1 -- 172.21.15.26:0/1724944969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba7e45f690 con 0x55ba7e571860 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c9bfff700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e57ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c937fe700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e57d410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0a9540 tx=0x7f7c8c010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c2fad09eb68b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c7affd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c7affd700 1 -- 172.21.15.26:0/1724944969 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0a9840 con 0x55ba7e571860 2024-05-20T21:07:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c7affd700 1 -- 172.21.15.26:0/1724944969 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7c8c0cc0b0 con 0x55ba7e571860 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c8400bb40 con 0x55ba7e571860 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7c7affd700 1 -- 172.21.15.26:0/1724944969 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c0adbf0 con 0x55ba7e571860 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7cac6ddb80 10 client.28211 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.563+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba7e588930 con 0x55ba7e571860 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7c7affd700 1 -- 172.21.15.26:0/1724944969 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c8400bb40 con 0x55ba7e571860 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7c7affd700 1 -- 172.21.15.26:0/1724944969 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7c8c0a8aa0 con 0x55ba7e571860 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7cac6ddb80 10 client.28211 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7cac6ddb80 10 client.28211 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7cac6ddb80 20 client.28211 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e58d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7cac6ddb80 4 client.28211 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba7e53d9e0 con 0x55ba7e58af70 2024-05-20T21:07:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.564+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e58d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.565+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e58d360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19caaff2aca0915a server_cookie=b51f4e0de0008106 in_seq=0 out_seq=0 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.565+0000 7f7c7affd700 10 client.28211 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7c7affd700 1 -- 172.21.15.26:0/1724944969 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ba7e53d9e0 con 0x55ba7e58af70 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7c7affd700 10 client.28211 handle_command_reply: tid=0 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 client.28211 shutdown 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 msgr2=0x55ba7e57d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e57d410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0a9540 tx=0x7f7c8c010b10 comp rx=0 tx=0).stop 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 shutdown_connections 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e58d360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7c7affd700 0 client.28211 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e57ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e57d410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724944969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 >> 172.21.15.26:0/1724944969 conn(0x55ba7e49f080 msgr2=0x55ba7e56b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 shutdown_connections 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724944969 wait complete. 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.566+0000 7f7cac6ddb80 20 client.28211 trim_cache size 0 max 16384 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 Processor -- start 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 -- start start 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e5871c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e56bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1b2780 con 0x55ba7e572210 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1b0010 con 0x55ba7e571860 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.584+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1ade60 con 0x55ba7e575f60 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58438/0 (socket says 172.21.15.26:58438) 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/1724709909 learned_addr learned my addr 172.21.15.26:0/1724709909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c937fe700 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e5871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/1724709909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 msgr2=0x55ba7e56bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e56bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.413 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:31.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/1724709909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 msgr2=0x55ba7e5871c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e5871c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/1724709909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba7e578c80 con 0x55ba7e572210 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57c650 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f7c84000c00 tx=0x7f7c84009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8789894a8f05259b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c917fa700 1 -- 172.21.15.26:0/1724709909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84033040 con 0x55ba7e572210 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c917fa700 1 -- 172.21.15.26:0/1724709909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c84041040 con 0x55ba7e572210 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.585+0000 7f7c917fa700 1 -- 172.21.15.26:0/1724709909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84025680 con 0x55ba7e572210 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724709909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 msgr2=0x55ba7e57c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57c650 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f7c84000c00 tx=0x7f7c84009310 comp rx=0 tx=0).stop 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724709909 shutdown_connections 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e56bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e5871c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/1724709909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e57c650 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724709909 >> 172.21.15.26:0/1724709909 conn(0x55ba7e49f080 msgr2=0x55ba7e57bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724709909 shutdown_connections 2024-05-20T21:07:31.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.586+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/1724709909 wait complete. 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 -- start start 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 Processor -- start 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e583710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e583c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e580c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1ae4d0 con 0x55ba7e572210 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1af940 con 0x55ba7e571860 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1aeba0 con 0x55ba7e575f60 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7cac6ddb80 10 client.0 _ll_register_callbacks cb 0x55ba7e45b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e583c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7c9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e583c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58450/0 (socket says 172.21.15.26:58450) 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/464061499 learned_addr learned my addr 172.21.15.26:0/464061499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7c937fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e583710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.587+0000 7f7c9bfff700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e580c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 msgr2=0x55ba7e580c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e580c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/464061499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 msgr2=0x55ba7e583710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e583710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9b7fe700 1 -- 172.21.15.26:0/464061499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba7e5814a0 con 0x55ba7e572210 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9bfff700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e580c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c937fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e583710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c9b7fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e583c50 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f7c84005070 tx=0x7f7c8400bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f312e841f054b48c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c7affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c7affd700 1 -- 172.21.15.26:0/464061499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8403e070 con 0x55ba7e572210 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c7affd700 1 -- 172.21.15.26:0/464061499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7c8400f6e0 con 0x55ba7e572210 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c84041040 con 0x55ba7e572210 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7c7affd700 1 -- 172.21.15.26:0/464061499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c84033040 con 0x55ba7e572210 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7cac6ddb80 10 client.18918 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.588+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba7e5f5780 con 0x55ba7e572210 2024-05-20T21:07:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7c7affd700 1 -- 172.21.15.26:0/464061499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c84041040 con 0x55ba7e572210 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7c7affd700 1 -- 172.21.15.26:0/464061499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7c8402d030 con 0x55ba7e572210 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7cac6ddb80 10 client.18918 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7cac6ddb80 10 client.18918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7cac6ddb80 20 client.18918 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e5f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7cac6ddb80 4 client.18918 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba7e4d2be0 con 0x55ba7e58af70 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.589+0000 7f7c937fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e5f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7c7affd700 10 client.18918 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7c937fe700 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e5f7a70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f041cec41fd7d93c server_cookie=67b6a723f2556f09 in_seq=0 out_seq=0 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7c7affd700 1 -- 172.21.15.26:0/464061499 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba7e4d2be0 con 0x55ba7e58af70 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7c7affd700 10 client.18918 handle_command_reply: tid=0 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 client.18918 shutdown 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 msgr2=0x55ba7e583c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e583c50 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f7c84005070 tx=0x7f7c8400bad0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 shutdown_connections 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ba7e58af70 0x55ba7e5f7a70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba7e575f60 0x55ba7e580c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba7e571860 0x55ba7e583710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7c7affd700 0 client.18918 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 --2- 172.21.15.26:0/464061499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba7e572210 0x55ba7e583c50 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 >> 172.21.15.26:0/464061499 conn(0x55ba7e49f080 msgr2=0x55ba7e575140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 shutdown_connections 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 1 -- 172.21.15.26:0/464061499 wait complete. 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.590+0000 7f7cac6ddb80 20 client.18918 trim_cache size 0 max 16384 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c880b2880 msgr2=0x7f7c880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c880b2880 0x7f7c880b4d40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c102290 tx=0x7f7c84005960 comp rx=0 tx=0).stop 2024-05-20T21:07:31.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c102440 msgr2=0x7f7c9c19bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c102440 0x7f7c9c19bd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00e9f0 tx=0x7f7c8c00ed00 comp rx=0 tx=0).stop 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 shutdown_connections 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c9c108520 0x7f7c9c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c9c102440 0x7f7c9c19bd60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7c880b2880 0x7f7c880b4d40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 --2- 172.21.15.26:0/2807281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c9c102e00 0x7f7c9c19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 >> 172.21.15.26:0/2807281563 conn(0x7f7c9c0793f0 msgr2=0x7f7c9c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 shutdown_connections 2024-05-20T21:07:31.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:30.591+0000 7f7ca1e47700 1 -- 172.21.15.26:0/2807281563 wait complete. 2024-05-20T21:07:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:31 smithi144 ceph-mon[35660]: pgmap v731: 97 pgs: 97 active+clean; 29 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-20T21:07:31.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 255202 2.23 MB/sec execute 457 sec latency 52.275 ms 2024-05-20T21:07:31.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:31 smithi055 ceph-mon[35213]: pgmap v731: 97 pgs: 97 active+clean; 29 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-20T21:07:31.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:31 smithi026 ceph-mon[31990]: pgmap v731: 97 pgs: 97 active+clean; 29 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-20T21:07:32.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 255742 2.24 MB/sec execute 458 sec latency 77.287 ms 2024-05-20T21:07:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:33 smithi144 ceph-mon[35660]: pgmap v732: 97 pgs: 97 active+clean; 29 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-20T21:07:33.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 256144 2.23 MB/sec execute 459 sec latency 39.990 ms 2024-05-20T21:07:33.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:33 smithi055 ceph-mon[35213]: pgmap v732: 97 pgs: 97 active+clean; 29 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-20T21:07:33.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:33 smithi026 ceph-mon[31990]: pgmap v732: 97 pgs: 97 active+clean; 29 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-20T21:07:34.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 256705 2.23 MB/sec execute 460 sec latency 85.548 ms 2024-05-20T21:07:35.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:07:35.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:35 smithi026 ceph-mon[31990]: pgmap v733: 97 pgs: 97 active+clean; 29 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-05-20T21:07:35.467 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:35 smithi144 ceph-mon[35660]: pgmap v733: 97 pgs: 97 active+clean; 29 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-05-20T21:07:35.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 257347 2.24 MB/sec execute 461 sec latency 113.694 ms 2024-05-20T21:07:35.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:35 smithi055 ceph-mon[35213]: pgmap v733: 97 pgs: 97 active+clean; 29 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-05-20T21:07:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 -- 172.21.15.26:0/4157991031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 msgr2=0x7f1044103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 --2- 172.21.15.26:0/4157991031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044103040 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f102c009900 tx=0x7f102c02a410 comp rx=0 tx=0).stop 2024-05-20T21:07:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 -- 172.21.15.26:0/4157991031 shutdown_connections 2024-05-20T21:07:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 --2- 172.21.15.26:0/4157991031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1044103580 0x7f1044109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 --2- 172.21.15.26:0/4157991031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044103040 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 --2- 172.21.15.26:0/4157991031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10441019d0 0x7f1044101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 -- 172.21.15.26:0/4157991031 >> 172.21.15.26:0/4157991031 conn(0x7f10440fcf60 msgr2=0x7f10440ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 -- 172.21.15.26:0/4157991031 shutdown_connections 2024-05-20T21:07:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.019+0000 7f10492e8640 1 -- 172.21.15.26:0/4157991031 wait complete. 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 Processor -- start 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 -- start start 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10441019d0 0x7f1044198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1044103580 0x7f10441a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f104410d3b0 con 0x7f1044102bc0 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f104410d230 con 0x7f1044103580 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10492e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f104410d530 con 0x7f10441019d0 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58472/0 (socket says 172.21.15.26:58472) 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 -- 172.21.15.26:0/1230783517 learned_addr learned my addr 172.21.15.26:0/1230783517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f1042ffd640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10441019d0 0x7f1044198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f10437fe640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1044103580 0x7f10441a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 -- 172.21.15.26:0/1230783517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10441019d0 msgr2=0x7f1044198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10441019d0 0x7f1044198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 -- 172.21.15.26:0/1230783517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1044103580 msgr2=0x7f10441a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1044103580 0x7f10441a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.020+0000 7f103bfff640 1 -- 172.21.15.26:0/1230783517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10441a0790 con 0x7f1044102bc0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.021+0000 7f103bfff640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044199170 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f102c009230 tx=0x7f102c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.021+0000 7f1040ff9640 1 -- 172.21.15.26:0/1230783517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c03f030 con 0x7f1044102bc0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.021+0000 7f1040ff9640 1 -- 172.21.15.26:0/1230783517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f102c003c90 con 0x7f1044102bc0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.021+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10441a0a20 con 0x7f1044102bc0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.021+0000 7f1040ff9640 1 -- 172.21.15.26:0/1230783517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c035070 con 0x7f1044102bc0 2024-05-20T21:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.021+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10441a0f10 con 0x7f1044102bc0 2024-05-20T21:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.024+0000 7f1040ff9640 1 -- 172.21.15.26:0/1230783517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f102c042030 con 0x7f1044102bc0 2024-05-20T21:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.024+0000 7f104c008740 1 Processor -- start 2024-05-20T21:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.024+0000 7f104c008740 1 -- start start 2024-05-20T21:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.024+0000 7f104c008740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 0x56292a0e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.024+0000 7f104c008740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e10b0 0x56292a0e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.024+0000 7f104c008740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f104c008740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56292a051c30 con 0x56292a0e6220 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f104c008740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56292a0417d0 con 0x56292a0e06c0 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f104c008740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56292a045810 con 0x56292a0e10b0 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 0x56292a0e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 0x56292a0e0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50594/0 (socket says 172.21.15.26:50594) 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 -- 172.21.15.26:0/4077385748 learned_addr learned my addr 172.21.15.26:0/4077385748 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f1042ffd640 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e10b0 0x56292a0e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f1040ff9640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10300b28a0 0x7f10300b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 -- 172.21.15.26:0/4077385748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e10b0 msgr2=0x56292a0e5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e10b0 0x56292a0e5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 -- 172.21.15.26:0/4077385748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 msgr2=0x56292a0e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 -- 172.21.15.26:0/4077385748 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562929d56210 con 0x56292a0e06c0 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10300b28a0 0x7f10300b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f1040ff9640 1 -- 172.21.15.26:0/1230783517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f102c102930 con 0x7f1044102bc0 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10437fe640 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 0x56292a0e0ae0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f103c009f70 tx=0x7f103c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee13840fce3c22f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.025+0000 7f10397fa640 1 -- 172.21.15.26:0/4077385748 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c016070 con 0x56292a0e06c0 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.026+0000 7f10397fa640 1 -- 172.21.15.26:0/4077385748 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f103c01c440 con 0x56292a0e06c0 2024-05-20T21:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.026+0000 7f10397fa640 1 -- 172.21.15.26:0/4077385748 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c01b5e0 con 0x56292a0e06c0 2024-05-20T21:07:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.026+0000 7f104c008740 1 -- 172.21.15.26:0/4077385748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 msgr2=0x56292a0e0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.026+0000 7f10437fe640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10300b28a0 0x7f10300b4d60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f103c011010 tx=0x7f103c01bc90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.026+0000 7f104c008740 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 0x56292a0e0ae0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f103c009f70 tx=0x7f103c010800 comp rx=0 tx=0).stop 2024-05-20T21:07:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- 172.21.15.26:0/4077385748 shutdown_connections 2024-05-20T21:07:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e10b0 0x56292a0e5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 --2- 172.21.15.26:0/4077385748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e06c0 0x56292a0e0ae0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- 172.21.15.26:0/4077385748 >> 172.21.15.26:0/4077385748 conn(0x56292a0dbd30 msgr2=0x56292a0dc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- 172.21.15.26:0/4077385748 shutdown_connections 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- 172.21.15.26:0/4077385748 wait complete. 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- start start 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 Processor -- start 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 0x56292a0ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562929d27790 con 0x56292a0e6220 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562929ffd890 con 0x56292a0e10b0 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56292a0e91d0 con 0x56292a0e06c0 2024-05-20T21:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f104c008740 10 client.0 _ll_register_callbacks cb 0x562929d55a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.027+0000 7f10437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 0x56292a0ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f10437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58496/0 (socket says 172.21.15.26:58496) 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 0x56292a0ead80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50610/0 (socket says 172.21.15.26:50610) 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f10437fe640 1 -- 172.21.15.26:0/2689727666 learned_addr learned my addr 172.21.15.26:0/2689727666 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f1042ffd640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 -- 172.21.15.26:0/2689727666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 msgr2=0x56292a0ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 -- 172.21.15.26:0/2689727666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 msgr2=0x56292a0f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 -- 172.21.15.26:0/2689727666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562929e22620 con 0x56292a0e10b0 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f10437fe640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f103bfff640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 0x56292a0ead80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f102c100420 tx=0x7f102c0c5fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6301ba0880b3cdc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f102a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f102a7fc640 1 -- 172.21.15.26:0/2689727666 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c049070 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f102a7fc640 1 -- 172.21.15.26:0/2689727666 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f102c0ca2c0 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103c01c440 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f102a7fc640 1 -- 172.21.15.26:0/2689727666 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f102c0c97d0 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f104c008740 10 client.28229 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.028+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56292a0f4d30 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f102a7fc640 1 -- 172.21.15.26:0/2689727666 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f103c01c440 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f102a7fc640 1 -- 172.21.15.26:0/2689727666 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f102c0d2980 con 0x56292a0e10b0 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f104c008740 10 client.28229 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f104c008740 10 client.28229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f104c008740 20 client.28229 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f104c008740 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a0f6510 0x56292a0f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f104c008740 4 client.28229 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56292a053b80 con 0x56292a0f6510 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.029+0000 7f1042ffd640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a0f6510 0x56292a0f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f1042ffd640 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a0f6510 0x56292a0f8970 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faca0d2ce329a964 server_cookie=965979923f15819d in_seq=0 out_seq=0 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f102a7fc640 10 client.28229 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f102a7fc640 1 -- 172.21.15.26:0/2689727666 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56292a053b80 con 0x56292a0f6510 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f102a7fc640 10 client.28229 handle_command_reply: tid=0 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f104c008740 1 client.28229 shutdown 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 msgr2=0x56292a0ead80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.030+0000 7f104c008740 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 0x56292a0ead80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f102c100420 tx=0x7f102c0c5fe0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 shutdown_connections 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a0f6510 0x56292a0f8970 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e6220 0x56292a0f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e10b0 0x56292a0ead80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 --2- 172.21.15.26:0/2689727666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f102a7fc640 0 client.28229 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 >> 172.21.15.26:0/2689727666 conn(0x56292a0dbd30 msgr2=0x56292a0dafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 shutdown_connections 2024-05-20T21:07:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 1 -- 172.21.15.26:0/2689727666 wait complete. 2024-05-20T21:07:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.031+0000 7f104c008740 20 client.28229 trim_cache size 0 max 16384 2024-05-20T21:07:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 Processor -- start 2024-05-20T21:07:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 -- start start 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10441ac710 con 0x56292a0e10b0 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10441a9fe0 con 0x56292a0e6220 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.048+0000 7f104c008740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10441a7e30 con 0x56292a0e06c0 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58506/0 (socket says 172.21.15.26:58506) 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 -- 172.21.15.26:0/1422500541 learned_addr learned my addr 172.21.15.26:0/1422500541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f10437fe640 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f1042ffd640 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 -- 172.21.15.26:0/1422500541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 msgr2=0x56292a0df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 -- 172.21.15.26:0/1422500541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 msgr2=0x56292a0fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 -- 172.21.15.26:0/1422500541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292a0d76f0 con 0x56292a0e10b0 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f1042ffd640 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f103bfff640 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0f1d10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f102c0c9d60 tx=0x7f102c102cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4e4bb3df62c28f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f1038ff9640 1 -- 172.21.15.26:0/1422500541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c049030 con 0x56292a0e10b0 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f1038ff9640 1 -- 172.21.15.26:0/1422500541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f102c03b460 con 0x56292a0e10b0 2024-05-20T21:07:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.049+0000 7f1038ff9640 1 -- 172.21.15.26:0/1422500541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c0044e0 con 0x56292a0e10b0 2024-05-20T21:07:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 -- 172.21.15.26:0/1422500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 msgr2=0x56292a0f1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0f1d10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f102c0c9d60 tx=0x7f102c102cc0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 -- 172.21.15.26:0/1422500541 shutdown_connections 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0f1d10 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 --2- 172.21.15.26:0/1422500541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 -- 172.21.15.26:0/1422500541 >> 172.21.15.26:0/1422500541 conn(0x56292a0dbd30 msgr2=0x56292a0ded20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 -- 172.21.15.26:0/1422500541 shutdown_connections 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.050+0000 7f104c008740 1 -- 172.21.15.26:0/1422500541 wait complete. 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 -- start start 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 Processor -- start 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0ef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10441a84a0 con 0x56292a0e10b0 2024-05-20T21:07:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10441a9910 con 0x56292a0e6220 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10441a8b70 con 0x56292a0e06c0 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f104c008740 10 client.0 _ll_register_callbacks cb 0x7f102c108240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0ef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58508/0 (socket says 172.21.15.26:58508) 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 -- 172.21.15.26:0/2825394466 learned_addr learned my addr 172.21.15.26:0/2825394466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f10437fe640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f1042ffd640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 -- 172.21.15.26:0/2825394466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 msgr2=0x56292a0ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 -- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 msgr2=0x56292a0f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.051+0000 7f103bfff640 1 -- 172.21.15.26:0/2825394466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292a0f0b90 con 0x56292a0e10b0 2024-05-20T21:07:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f103bfff640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0ef590 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f102c0092a0 tx=0x7f102c0ca510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b398ed832f79e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f102a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f102a7fc640 1 -- 172.21.15.26:0/2825394466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c049070 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f102a7fc640 1 -- 172.21.15.26:0/2825394466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f102c108040 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56292a04d360 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f102a7fc640 1 -- 172.21.15.26:0/2825394466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f102c0044e0 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 10 client.18936 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56292a0f0b90 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f102a7fc640 1 -- 172.21.15.26:0/2825394466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56292a04d360 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f102a7fc640 1 -- 172.21.15.26:0/2825394466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f102c0e4020 con 0x56292a0e10b0 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 10 client.18936 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 10 client.18936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 20 client.18936 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a161340 0x56292a163730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 4 client.18936 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:07:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.052+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562929fc75f0 con 0x56292a161340 2024-05-20T21:07:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.053+0000 7f1042ffd640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a161340 0x56292a163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.053+0000 7f1042ffd640 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a161340 0x56292a163730 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ff177b06c3e1e server_cookie=eb5db242fb5193ee in_seq=0 out_seq=0 2024-05-20T21:07:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.053+0000 7f102a7fc640 10 client.18936 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f102a7fc640 1 -- 172.21.15.26:0/2825394466 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562929fc75f0 con 0x56292a161340 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f102a7fc640 10 client.18936 handle_command_reply: tid=0 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 client.18936 shutdown 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 msgr2=0x56292a0ef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0ef590 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f102c0092a0 tx=0x7f102c0ca510 comp rx=0 tx=0).stop 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 shutdown_connections 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56292a161340 0x56292a163730 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56292a0e6220 0x56292a0f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f102a7fc640 0 client.18936 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56292a0e10b0 0x56292a0ef590 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 --2- 172.21.15.26:0/2825394466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292a0e06c0 0x56292a0ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 >> 172.21.15.26:0/2825394466 conn(0x56292a0dbd30 msgr2=0x56292a0ded20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 shutdown_connections 2024-05-20T21:07:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.102+0000 7f104c008740 1 -- 172.21.15.26:0/2825394466 wait complete. 2024-05-20T21:07:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f104c008740 20 client.18936 trim_cache size 0 max 16384 2024-05-20T21:07:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:07:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:07:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10300b28a0 msgr2=0x7f10300b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10300b28a0 0x7f10300b4d60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f103c011010 tx=0x7f103c01bc90 comp rx=0 tx=0).stop 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 msgr2=0x7f1044199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044199170 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f102c009230 tx=0x7f102c0042f0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 shutdown_connections 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10300b28a0 0x7f10300b4d60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1044103580 0x7f10441a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1044102bc0 0x7f1044199170 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 --2- 172.21.15.26:0/1230783517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10441019d0 0x7f1044198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 >> 172.21.15.26:0/1230783517 conn(0x7f10440fcf60 msgr2=0x7f10440fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 shutdown_connections 2024-05-20T21:07:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:36.103+0000 7f10492e8640 1 -- 172.21.15.26:0/1230783517 wait complete. 2024-05-20T21:07:36.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 257980 2.24 MB/sec execute 462 sec latency 50.543 ms 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 -- 172.21.15.26:0/2612857885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460108520 msgr2=0x7f2460108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 --2- 172.21.15.26:0/2612857885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460108520 0x7f2460108900 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 -- 172.21.15.26:0/2612857885 shutdown_connections 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 --2- 172.21.15.26:0/2612857885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460108520 0x7f2460108900 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 --2- 172.21.15.26:0/2612857885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 --2- 172.21.15.26:0/2612857885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460102440 0x7f24601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.308+0000 7f24668fe700 1 -- 172.21.15.26:0/2612857885 >> 172.21.15.26:0/2612857885 conn(0x7f24600793f0 msgr2=0x7f2460079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.309+0000 7f24668fe700 1 -- 172.21.15.26:0/2612857885 shutdown_connections 2024-05-20T21:07:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.309+0000 7f24668fe700 1 -- 172.21.15.26:0/2612857885 wait complete. 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.309+0000 7f24668fe700 1 Processor -- start 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.309+0000 7f24668fe700 1 -- start start 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f24668fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 0x7f246019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f24668fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f24668fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460108520 0x7f24601a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f24668fe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24601124a0 con 0x7f2460102440 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f24668fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2460112320 con 0x7f2460102e00 2024-05-20T21:07:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f24668fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2460112620 con 0x7f2460108520 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 0x7f246019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50644/0 (socket says 172.21.15.26:50644) 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 0x7f246019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58540/0 (socket says 172.21.15.26:58540) 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 -- 172.21.15.26:0/886229314 learned_addr learned my addr 172.21.15.26:0/886229314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f2464e9b700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460108520 0x7f24601a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 -- 172.21.15.26:0/886229314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460108520 msgr2=0x7f24601a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460108520 0x7f24601a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 -- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 msgr2=0x7f246019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 0x7f246019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245f7fe700 1 -- 172.21.15.26:0/886229314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24601a5fa0 con 0x7f2460102e00 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.310+0000 7f245ffff700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 0x7f246019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.311+0000 7f245f7fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246019af20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f245400d980 tx=0x7f245400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.311+0000 7f245d7fa700 1 -- 172.21.15.26:0/886229314 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454017070 con 0x7f2460102e00 2024-05-20T21:07:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.311+0000 7f245d7fa700 1 -- 172.21.15.26:0/886229314 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2454005020 con 0x7f2460102e00 2024-05-20T21:07:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.311+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24601a61d0 con 0x7f2460102e00 2024-05-20T21:07:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.311+0000 7f245d7fa700 1 -- 172.21.15.26:0/886229314 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245400f980 con 0x7f2460102e00 2024-05-20T21:07:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.311+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24601a6690 con 0x7f2460102e00 2024-05-20T21:07:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.314+0000 7f245d7fa700 1 -- 172.21.15.26:0/886229314 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2454004a30 con 0x7f2460102e00 2024-05-20T21:07:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.314+0000 7f245d7fa700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24480b2880 0x7f24480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 Processor -- start 2024-05-20T21:07:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245ffff700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24480b2880 0x7f24480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245d7fa700 1 -- 172.21.15.26:0/886229314 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f24540e20d0 con 0x7f2460102e00 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 -- start start 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31f300 0x561fcf31f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf31bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31c2b0 0x561fcf330510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561fcf279d70 con 0x561fcf31f300 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561fcf269910 con 0x561fcf31c2b0 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2471194b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561fcf26d950 con 0x561fcf31b8a0 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245ffff700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24480b2880 0x7f24480b4d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f245000b2e0 tx=0x7f2450005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31f300 0x561fcf31f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31f300 0x561fcf31f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58548/0 (socket says 172.21.15.26:58548) 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245ffff700 1 -- 172.21.15.26:0/1227957394 learned_addr learned my addr 172.21.15.26:0/1227957394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf31bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.315+0000 7f2464e9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31c2b0 0x561fcf330510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f245f7fe700 1 -- 172.21.15.26:0/1227957394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31c2b0 msgr2=0x561fcf330510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31c2b0 0x561fcf330510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f245f7fe700 1 -- 172.21.15.26:0/1227957394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31f300 msgr2=0x561fcf31f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31f300 0x561fcf31f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f245f7fe700 1 -- 172.21.15.26:0/1227957394 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fcf212fb0 con 0x561fcf31b8a0 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf31bd20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24540dfb70 tx=0x7f24540a8f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc6cd57c28cc82d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f24467fc700 1 -- 172.21.15.26:0/1227957394 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540a9690 con 0x561fcf31b8a0 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f24467fc700 1 -- 172.21.15.26:0/1227957394 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24540ada40 con 0x561fcf31b8a0 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.316+0000 7f24467fc700 1 -- 172.21.15.26:0/1227957394 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540b2060 con 0x561fcf31b8a0 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 -- 172.21.15.26:0/1227957394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 msgr2=0x561fcf31bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf31bd20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24540dfb70 tx=0x7f24540a8f30 comp rx=0 tx=0).stop 2024-05-20T21:07:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 -- 172.21.15.26:0/1227957394 shutdown_connections 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31f300 0x561fcf31f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31c2b0 0x561fcf330510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 --2- 172.21.15.26:0/1227957394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf31bd20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 -- 172.21.15.26:0/1227957394 >> 172.21.15.26:0/1227957394 conn(0x561fcf247240 msgr2=0x561fcf3163e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 -- 172.21.15.26:0/1227957394 shutdown_connections 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 -- 172.21.15.26:0/1227957394 wait complete. 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.317+0000 7f2471194b80 1 -- start start 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 Processor -- start 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf330240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf32d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf32da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561fcf16ffa0 con 0x561fcf31b8a0 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561fcf1d1da0 con 0x561fcf31f300 2024-05-20T21:07:37.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561fcf330e20 con 0x561fcf31c2b0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2471194b80 10 client.0 _ll_register_callbacks cb 0x561fcf204950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf330240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf330240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58554/0 (socket says 172.21.15.26:58554) 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f245ffff700 1 -- 172.21.15.26:0/1430914500 learned_addr learned my addr 172.21.15.26:0/1430914500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf32d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f2464e9b700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf32da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.318+0000 7f245f7fe700 1 -- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 msgr2=0x561fcf32da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf32da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f245f7fe700 1 -- 172.21.15.26:0/1430914500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 msgr2=0x561fcf330240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf330240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f245f7fe700 1 -- 172.21.15.26:0/1430914500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fcf1b98d0 con 0x561fcf31c2b0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f245ffff700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf330240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf32d540 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24540b26b0 tx=0x7f24540cc120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba637965c7e315dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f243f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f243f7fe700 1 -- 172.21.15.26:0/1430914500 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540a9690 con 0x561fcf31c2b0 2024-05-20T21:07:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f243f7fe700 1 -- 172.21.15.26:0/1430914500 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f24540b2060 con 0x561fcf31c2b0 2024-05-20T21:07:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24540ada40 con 0x561fcf31c2b0 2024-05-20T21:07:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f243f7fe700 1 -- 172.21.15.26:0/1430914500 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24540b2260 con 0x561fcf31c2b0 2024-05-20T21:07:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f2471194b80 10 client.28036 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.319+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561fcf223760 con 0x561fcf31c2b0 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f243f7fe700 1 -- 172.21.15.26:0/1430914500 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24540beae0 con 0x561fcf31c2b0 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f243f7fe700 1 -- 172.21.15.26:0/1430914500 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f24540a5150 con 0x561fcf31c2b0 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f2471194b80 10 client.28036 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f2471194b80 10 client.28036 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f2471194b80 20 client.28036 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f2471194b80 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf33d8a0 0x561fcf33fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f2471194b80 4 client.28036 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f24540ada40 con 0x561fcf33d8a0 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f245ffff700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf33d8a0 0x561fcf33fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f243f7fe700 10 client.28036 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.320+0000 7f245ffff700 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf33d8a0 0x561fcf33fc90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e303e0069ae9b73 server_cookie=bfd7f534f6e9d24b in_seq=0 out_seq=0 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f243f7fe700 1 -- 172.21.15.26:0/1430914500 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f24540ada40 con 0x561fcf33d8a0 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f243f7fe700 10 client.28036 handle_command_reply: tid=0 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 client.28036 shutdown 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 msgr2=0x561fcf32d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf32d540 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24540b26b0 tx=0x7f24540cc120 comp rx=0 tx=0).stop 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 shutdown_connections 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf33d8a0 0x561fcf33fc90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf330240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf32da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 --2- 172.21.15.26:0/1430914500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf32d540 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 >> 172.21.15.26:0/1430914500 conn(0x561fcf247240 msgr2=0x561fcf315af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f243f7fe700 0 client.28036 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:37.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.321+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 shutdown_connections 2024-05-20T21:07:37.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.322+0000 7f2471194b80 1 -- 172.21.15.26:0/1430914500 wait complete. 2024-05-20T21:07:37.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.322+0000 7f2471194b80 20 client.28036 trim_cache size 0 max 16384 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 Processor -- start 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 -- start start 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf316270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf3284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf328a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24601ae470 con 0x561fcf31b8a0 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f24601abd00 con 0x561fcf31f300 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2471194b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24601a9b50 con 0x561fcf31c2b0 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf316270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf316270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58564/0 (socket says 172.21.15.26:58564) 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 -- 172.21.15.26:0/1061686559 learned_addr learned my addr 172.21.15.26:0/1061686559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2464e9b700 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf328a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245f7fe700 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf3284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 -- 172.21.15.26:0/1061686559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 msgr2=0x561fcf3284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf3284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 -- 172.21.15.26:0/1061686559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 msgr2=0x561fcf328a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf328a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f245ffff700 1 -- 172.21.15.26:0/1061686559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fcf3174d0 con 0x561fcf31b8a0 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.340+0000 7f2464e9b700 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf328a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.341+0000 7f245ffff700 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf316270 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713ae151dc53d418 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.341+0000 7f2445ffb700 1 -- 172.21.15.26:0/1061686559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450003e60 con 0x561fcf31b8a0 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.341+0000 7f2445ffb700 1 -- 172.21.15.26:0/1061686559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24500056a0 con 0x561fcf31b8a0 2024-05-20T21:07:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.341+0000 7f2445ffb700 1 -- 172.21.15.26:0/1061686559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450031680 con 0x561fcf31b8a0 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 -- 172.21.15.26:0/1061686559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 msgr2=0x561fcf316270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf316270 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450005000 comp rx=0 tx=0).stop 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 -- 172.21.15.26:0/1061686559 shutdown_connections 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31b8a0 0x561fcf316270 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf328a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 --2- 172.21.15.26:0/1061686559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31c2b0 0x561fcf3284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 -- 172.21.15.26:0/1061686559 >> 172.21.15.26:0/1061686559 conn(0x561fcf247240 msgr2=0x561fcf312760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 -- 172.21.15.26:0/1061686559 shutdown_connections 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 -- 172.21.15.26:0/1061686559 wait complete. 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 -- start start 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.342+0000 7f2471194b80 1 Processor -- start 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf3431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31c2b0 0x561fcf340570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf340ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24601aa1c0 con 0x561fcf31c2b0 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f24601ab630 con 0x561fcf31f300 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24601aa890 con 0x561fcf31b8a0 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2471194b80 10 client.0 _ll_register_callbacks cb 0x561fcf204950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31c2b0 0x561fcf340570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2464e9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf340ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31c2b0 0x561fcf340570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58572/0 (socket says 172.21.15.26:58572) 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f245f7fe700 1 -- 172.21.15.26:0/3087566723 learned_addr learned my addr 172.21.15.26:0/3087566723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f245ffff700 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf3431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2464e9b700 1 -- 172.21.15.26:0/3087566723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 msgr2=0x561fcf3431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2464e9b700 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf3431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2464e9b700 1 -- 172.21.15.26:0/3087566723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31c2b0 msgr2=0x561fcf340570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2464e9b700 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31c2b0 0x561fcf340570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.343+0000 7f2464e9b700 1 -- 172.21.15.26:0/3087566723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fcf3410f0 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2464e9b700 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf340ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2458009510 tx=0x7f2458018750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1f3cfc1d4048c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f243f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f243f7fe700 1 -- 172.21.15.26:0/3087566723 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458003a00 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f243f7fe700 1 -- 172.21.15.26:0/3087566723 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f2458003ba0 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24500056a0 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f243f7fe700 1 -- 172.21.15.26:0/3087566723 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2458003da0 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 10 client.28253 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561fcf3a7fb0 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f243f7fe700 1 -- 172.21.15.26:0/3087566723 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24500056a0 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f243f7fe700 1 -- 172.21.15.26:0/3087566723 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f245802b050 con 0x561fcf31f300 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 10 client.28253 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:37.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 10 client.28253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:07:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 20 client.28253 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.344+0000 7f2471194b80 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf3a86a0 0x561fcf3aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f2471194b80 4 client.28253 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fcf27bcc0 con 0x561fcf3a86a0 2024-05-20T21:07:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f245ffff700 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf3a86a0 0x561fcf3aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f245ffff700 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf3a86a0 0x561fcf3aaac0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8a7f092d0aa94b0 server_cookie=f3856b05aa588c04 in_seq=0 out_seq=0 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f243f7fe700 10 client.28253 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f243f7fe700 1 -- 172.21.15.26:0/3087566723 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fcf27bcc0 con 0x561fcf3a86a0 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f243f7fe700 10 client.28253 handle_command_reply: tid=0 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.345+0000 7f2471194b80 1 client.28253 shutdown 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 msgr2=0x561fcf340ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf340ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2458009510 tx=0x7f2458018750 comp rx=0 tx=0).stop 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 shutdown_connections 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x561fcf3a86a0 0x561fcf3aaac0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fcf31f300 0x561fcf340ab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fcf31b8a0 0x561fcf3431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 --2- 172.21.15.26:0/3087566723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fcf31c2b0 0x561fcf340570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 >> 172.21.15.26:0/3087566723 conn(0x561fcf247240 msgr2=0x561fcf3284c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f243f7fe700 0 client.28253 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 shutdown_connections 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 1 -- 172.21.15.26:0/3087566723 wait complete. 2024-05-20T21:07:37.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f2471194b80 20 client.28253 trim_cache size 0 max 16384 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24480b2880 msgr2=0x7f24480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f24668fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24480b2880 0x7f24480b4d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f245000b2e0 tx=0x7f2450005fb0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 msgr2=0x7f246019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.346+0000 7f24668fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246019af20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f245400d980 tx=0x7f245400dc90 comp rx=0 tx=0).stop 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 shutdown_connections 2024-05-20T21:07:37.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24480b2880 0x7f24480b4d40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2460102440 0x7f246019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460108520 0x7f24601a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 --2- 172.21.15.26:0/886229314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2460102e00 0x7f246019af20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 >> 172.21.15.26:0/886229314 conn(0x7f24600793f0 msgr2=0x7f2460101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 shutdown_connections 2024-05-20T21:07:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:37.347+0000 7f24668fe700 1 -- 172.21.15.26:0/886229314 wait complete. 2024-05-20T21:07:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:37 smithi026 ceph-mon[31990]: pgmap v734: 97 pgs: 97 active+clean; 31 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-20T21:07:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:37 smithi144 ceph-mon[35660]: pgmap v734: 97 pgs: 97 active+clean; 31 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-20T21:07:37.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 258738 2.25 MB/sec execute 463 sec latency 19.021 ms 2024-05-20T21:07:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:37 smithi055 ceph-mon[35213]: pgmap v734: 97 pgs: 97 active+clean; 31 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-20T21:07:38.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:07:38.492 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:38.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 259274 2.25 MB/sec execute 464 sec latency 16.463 ms 2024-05-20T21:07:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:39 smithi144 ceph-mon[35660]: pgmap v735: 97 pgs: 97 active+clean; 31 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-05-20T21:07:39.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 259737 2.25 MB/sec execute 465 sec latency 18.849 ms 2024-05-20T21:07:39.655 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:39 smithi026 ceph-mon[31990]: pgmap v735: 97 pgs: 97 active+clean; 31 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-05-20T21:07:39.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:39 smithi055 ceph-mon[35213]: pgmap v735: 97 pgs: 97 active+clean; 31 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-05-20T21:07:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 -- 172.21.15.26:0/4091073370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 msgr2=0x7f9c4410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 --2- 172.21.15.26:0/4091073370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4410f2f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b0a0 tx=0x7f9c3c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 -- 172.21.15.26:0/4091073370 shutdown_connections 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 --2- 172.21.15.26:0/4091073370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4410f2f0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 --2- 172.21.15.26:0/4091073370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44108520 0x7f9c44108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 --2- 172.21.15.26:0/4091073370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44102440 0x7f9c441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 -- 172.21.15.26:0/4091073370 >> 172.21.15.26:0/4091073370 conn(0x7f9c440793f0 msgr2=0x7f9c44079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 -- 172.21.15.26:0/4091073370 shutdown_connections 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.241+0000 7f9c4b525700 1 -- 172.21.15.26:0/4091073370 wait complete. 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 Processor -- start 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 -- start start 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 0x7f9c4419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44108520 0x7f9c44199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441124d0 con 0x7f9c44102e00 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44112350 con 0x7f9c44102440 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c4b525700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44112650 con 0x7f9c44108520 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c48ac0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 0x7f9c4419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c48ac0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51598/0 (socket says 172.21.15.26:51598) 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c48ac0700 1 -- 172.21.15.26:0/1755627598 learned_addr learned my addr 172.21.15.26:0/1755627598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 0x7f9c4419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40926/0 (socket says 172.21.15.26:40926) 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.242+0000 7f9c49ac2700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44108520 0x7f9c44199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c492c1700 1 -- 172.21.15.26:0/1755627598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44108520 msgr2=0x7f9c44199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c492c1700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44108520 0x7f9c44199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c492c1700 1 -- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 msgr2=0x7f9c4419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c492c1700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c492c1700 1 -- 172.21.15.26:0/1755627598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4419a0a0 con 0x7f9c44102440 2024-05-20T21:07:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c492c1700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 0x7f9c4419f730 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e9f0 tx=0x7f9c4000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c3a7fc700 1 -- 172.21.15.26:0/1755627598 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000c9c0 con 0x7f9c44102440 2024-05-20T21:07:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c3a7fc700 1 -- 172.21.15.26:0/1755627598 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c40010430 con 0x7f9c44102440 2024-05-20T21:07:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4419a330 con 0x7f9c44102440 2024-05-20T21:07:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.243+0000 7f9c3a7fc700 1 -- 172.21.15.26:0/1755627598 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40018680 con 0x7f9c44102440 2024-05-20T21:07:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.244+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c441a6600 con 0x7f9c44102440 2024-05-20T21:07:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.247+0000 7f9c3a7fc700 1 -- 172.21.15.26:0/1755627598 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9c4000cb60 con 0x7f9c44102440 2024-05-20T21:07:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.248+0000 7f9c3a7fc700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c300b2900 0x7f9c300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.248+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c300b2900 0x7f9c300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.248+0000 7f9c3a7fc700 1 -- 172.21.15.26:0/1755627598 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9c400e22a0 con 0x7f9c44102440 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 Processor -- start 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 -- start start 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c300b2900 0x7f9c300b4dc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9c44079b90 tx=0x7f9c3400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba647700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba641d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba642260 0x5616ba64e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5616ba59fd70 con 0x5616ba642260 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5616ba58f910 con 0x5616ba647320 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616ba593950 con 0x5616ba6418a0 2024-05-20T21:07:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba647700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c49ac2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba642260 0x5616ba64e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba647700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40936/0 (socket says 172.21.15.26:40936) 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 -- 172.21.15.26:0/4046860352 learned_addr learned my addr 172.21.15.26:0/4046860352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba641d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 -- 172.21.15.26:0/4046860352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 msgr2=0x5616ba641d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba641d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 -- 172.21.15.26:0/4046860352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba642260 msgr2=0x5616ba64e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba642260 0x5616ba64e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.249+0000 7f9c492c1700 1 -- 172.21.15.26:0/4046860352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ba538fb0 con 0x5616ba647320 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.250+0000 7f9c492c1700 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba647700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a9560 tx=0x7f9c4001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e222944b0fe9d82 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.250+0000 7f9c2f7fe700 1 -- 172.21.15.26:0/4046860352 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400a9370 con 0x5616ba647320 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.250+0000 7f9c2f7fe700 1 -- 172.21.15.26:0/4046860352 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c400a9f60 con 0x5616ba647320 2024-05-20T21:07:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.250+0000 7f9c2f7fe700 1 -- 172.21.15.26:0/4046860352 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400a9ac0 con 0x5616ba647320 2024-05-20T21:07:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/4046860352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 msgr2=0x5616ba647700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba647700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a9560 tx=0x7f9c4001f000 comp rx=0 tx=0).stop 2024-05-20T21:07:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/4046860352 shutdown_connections 2024-05-20T21:07:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba642260 0x5616ba64e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba647700 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/4046860352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba641d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/4046860352 >> 172.21.15.26:0/4046860352 conn(0x5616ba56d240 msgr2=0x5616ba63c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/4046860352 shutdown_connections 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/4046860352 wait complete. 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 -- start start 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.251+0000 7f9c55dbbb80 1 Processor -- start 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba64dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba64b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba64b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5616ba495fa0 con 0x5616ba6418a0 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5616ba4f7da0 con 0x5616ba647320 2024-05-20T21:07:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616ba64e6e0 con 0x5616ba642260 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c55dbbb80 10 client.0 _ll_register_callbacks cb 0x5616ba52a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba64dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba64dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51614/0 (socket says 172.21.15.26:51614) 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c48ac0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba64b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 -- 172.21.15.26:0/2327875965 learned_addr learned my addr 172.21.15.26:0/2327875965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c49ac2700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba64b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 -- 172.21.15.26:0/2327875965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 msgr2=0x5616ba64b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba64b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 -- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 msgr2=0x5616ba64b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba64b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c492c1700 1 -- 172.21.15.26:0/2327875965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ba4df8d0 con 0x5616ba6418a0 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.252+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba64b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c492c1700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba64dd70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9c400dfd90 tx=0x7f9c400a93d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f8c3a43e077834 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c2cff9700 1 -- 172.21.15.26:0/2327875965 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400c8070 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c2cff9700 1 -- 172.21.15.26:0/2327875965 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9c400b2230 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c400a9f60 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c2cff9700 1 -- 172.21.15.26:0/2327875965 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c400adc10 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c55dbbb80 10 client.18966 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616ba549760 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c2cff9700 1 -- 172.21.15.26:0/2327875965 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c400a9f60 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.253+0000 7f9c2cff9700 1 -- 172.21.15.26:0/2327875965 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9c400cab10 con 0x5616ba6418a0 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c55dbbb80 10 client.18966 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c55dbbb80 10 client.18966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c55dbbb80 20 client.18966 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba65b100 0x5616ba65d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c55dbbb80 4 client.18966 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616ba5a1cc0 con 0x5616ba65b100 2024-05-20T21:07:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba65b100 0x5616ba65d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba65b100 0x5616ba65d4f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bbc9e434990605 server_cookie=df4c0e4ef752055f in_seq=0 out_seq=0 2024-05-20T21:07:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.254+0000 7f9c2cff9700 10 client.18966 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c2cff9700 1 -- 172.21.15.26:0/2327875965 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5616ba5a1cc0 con 0x5616ba65b100 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c2cff9700 10 client.18966 handle_command_reply: tid=0 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 client.18966 shutdown 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 msgr2=0x5616ba64dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba64dd70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9c400dfd90 tx=0x7f9c400a93d0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 shutdown_connections 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba65b100 0x5616ba65d4f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba64dd70 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba64b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c2cff9700 0 client.18966 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/2327875965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba64b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 >> 172.21.15.26:0/2327875965 conn(0x5616ba56d240 msgr2=0x5616ba63baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 shutdown_connections 2024-05-20T21:07:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.255+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/2327875965 wait complete. 2024-05-20T21:07:40.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.256+0000 7f9c55dbbb80 20 client.18966 trim_cache size 0 max 16384 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 Processor -- start 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 -- start start 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba63c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 0x5616ba640fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba647320 0x5616ba648010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441ae3b0 con 0x5616ba647320 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441abc40 con 0x5616ba642260 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a9a90 con 0x5616ba6418a0 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c48ac0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 0x5616ba640fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c48ac0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 0x5616ba640fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40954/0 (socket says 172.21.15.26:40954) 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c48ac0700 1 -- 172.21.15.26:0/493532681 learned_addr learned my addr 172.21.15.26:0/493532681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c492c1700 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba63c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.273+0000 7f9c49ac2700 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba647320 0x5616ba648010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c48ac0700 1 -- 172.21.15.26:0/493532681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 msgr2=0x5616ba63c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba63c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c48ac0700 1 -- 172.21.15.26:0/493532681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba647320 msgr2=0x5616ba648010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba647320 0x5616ba648010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c48ac0700 1 -- 172.21.15.26:0/493532681 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ba63dba0 con 0x5616ba642260 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 0x5616ba640fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5616ba64c640 tx=0x7f9c34013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16383a85b233d52e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c2effd700 1 -- 172.21.15.26:0/493532681 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34011dc0 con 0x5616ba642260 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c2effd700 1 -- 172.21.15.26:0/493532681 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c340044b0 con 0x5616ba642260 2024-05-20T21:07:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.274+0000 7f9c2effd700 1 -- 172.21.15.26:0/493532681 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3402a470 con 0x5616ba642260 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/493532681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 msgr2=0x5616ba640fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 0x5616ba640fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5616ba64c640 tx=0x7f9c34013000 comp rx=0 tx=0).stop 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/493532681 shutdown_connections 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba647320 0x5616ba648010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba6418a0 0x5616ba63c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/493532681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba642260 0x5616ba640fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/493532681 >> 172.21.15.26:0/493532681 conn(0x5616ba56d240 msgr2=0x5616ba640890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/493532681 shutdown_connections 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/493532681 wait complete. 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 -- start start 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.275+0000 7f9c55dbbb80 1 Processor -- start 2024-05-20T21:07:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba660ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba65be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba65c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441aa100 con 0x5616ba6418a0 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441ab570 con 0x5616ba647320 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441aa7d0 con 0x5616ba642260 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c55dbbb80 10 client.0 _ll_register_callbacks cb 0x5616ba52a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba660ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba660ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51624/0 (socket says 172.21.15.26:51624) 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 -- 172.21.15.26:0/1899943933 learned_addr learned my addr 172.21.15.26:0/1899943933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c49ac2700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba65c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba65be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 -- 172.21.15.26:0/1899943933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 msgr2=0x5616ba65be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba65be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 -- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 msgr2=0x5616ba65c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba65c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c492c1700 1 -- 172.21.15.26:0/1899943933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ba65ca90 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.276+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba65be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c492c1700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba660ad0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a51d0 tx=0x7f9c400b24c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744ff0298702680c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c2cff9700 1 -- 172.21.15.26:0/1899943933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400c8040 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c2cff9700 1 -- 172.21.15.26:0/1899943933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9c400aada0 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c340044b0 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c2cff9700 1 -- 172.21.15.26:0/1899943933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c400a5530 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 10 client.18972 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616ba65ca90 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c2cff9700 1 -- 172.21.15.26:0/1899943933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c340044b0 con 0x5616ba6418a0 2024-05-20T21:07:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c2cff9700 1 -- 172.21.15.26:0/1899943933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9c400910d0 con 0x5616ba6418a0 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 10 client.18972 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 10 client.18972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 20 client.18972 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba6c5b10 0x5616ba6c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 4 client.18972 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.277+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616ba59b4a0 con 0x5616ba6c5b10 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.278+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba6c5b10 0x5616ba6c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.278+0000 7f9c48ac0700 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba6c5b10 0x5616ba6c7f30 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f8de3e5fae05f5 server_cookie=7de0c38e670fd33 in_seq=0 out_seq=0 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.278+0000 7f9c2cff9700 10 client.18972 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c2cff9700 1 -- 172.21.15.26:0/1899943933 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616ba59b4a0 con 0x5616ba6c5b10 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c2cff9700 10 client.18972 handle_command_reply: tid=0 2024-05-20T21:07:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 client.18972 shutdown 2024-05-20T21:07:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 msgr2=0x5616ba660ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba660ad0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a51d0 tx=0x7f9c400b24c0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 shutdown_connections 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5616ba6c5b10 0x5616ba6c7f30 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616ba6418a0 0x5616ba660ad0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5616ba647320 0x5616ba65c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 --2- 172.21.15.26:0/1899943933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616ba642260 0x5616ba65be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 >> 172.21.15.26:0/1899943933 conn(0x5616ba56d240 msgr2=0x5616ba641120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.279+0000 7f9c2cff9700 0 client.18972 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 shutdown_connections 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c55dbbb80 1 -- 172.21.15.26:0/1899943933 wait complete. 2024-05-20T21:07:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c55dbbb80 20 client.18972 trim_cache size 0 max 16384 2024-05-20T21:07:40.285 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c300b2900 msgr2=0x7f9c300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c300b2900 0x7f9c300b4dc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9c44079b90 tx=0x7f9c3400b500 comp rx=0 tx=0).stop 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 msgr2=0x7f9c4419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 0x7f9c4419f730 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e9f0 tx=0x7f9c4000ed00 comp rx=0 tx=0).stop 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 shutdown_connections 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44108520 0x7f9c44199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c44102440 0x7f9c4419f730 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c300b2900 0x7f9c300b4dc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 --2- 172.21.15.26:0/1755627598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102e00 0x7f9c4419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.280+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 >> 172.21.15.26:0/1755627598 conn(0x7f9c440793f0 msgr2=0x7f9c4406b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.281+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 shutdown_connections 2024-05-20T21:07:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:40.281+0000 7f9c4b525700 1 -- 172.21.15.26:0/1755627598 wait complete. 2024-05-20T21:07:40.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 260421 2.25 MB/sec execute 466 sec latency 18.287 ms 2024-05-20T21:07:41.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:41 smithi026 ceph-mon[31990]: pgmap v736: 97 pgs: 97 active+clean; 32 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-05-20T21:07:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:41 smithi144 ceph-mon[35660]: pgmap v736: 97 pgs: 97 active+clean; 32 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-05-20T21:07:41.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 261118 2.25 MB/sec execute 467 sec latency 16.315 ms 2024-05-20T21:07:41.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:41 smithi055 ceph-mon[35213]: pgmap v736: 97 pgs: 97 active+clean; 32 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-05-20T21:07:42.052 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:07:42.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:07:42.325 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:42.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 261909 2.26 MB/sec execute 468 sec latency 12.586 ms 2024-05-20T21:07:43.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:43 smithi026 ceph-mon[31990]: pgmap v737: 97 pgs: 97 active+clean; 29 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-05-20T21:07:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:43 smithi144 ceph-mon[35660]: pgmap v737: 97 pgs: 97 active+clean; 29 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-05-20T21:07:43.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 262534 2.26 MB/sec execute 469 sec latency 16.461 ms 2024-05-20T21:07:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:43 smithi055 ceph-mon[35213]: pgmap v737: 97 pgs: 97 active+clean; 29 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-05-20T21:07:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 -- 172.21.15.26:0/1760837493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1095b0 msgr2=0x7fda2c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 --2- 172.21.15.26:0/1760837493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1095b0 0x7fda2c109990 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fda20009990 tx=0x7fda20009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 -- 172.21.15.26:0/1760837493 shutdown_connections 2024-05-20T21:07:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 --2- 172.21.15.26:0/1760837493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1095b0 0x7fda2c109990 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 --2- 172.21.15.26:0/1760837493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2c06bbe0 0x7fda2c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 --2- 172.21.15.26:0/1760837493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda2c06b220 0x7fda2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 -- 172.21.15.26:0/1760837493 >> 172.21.15.26:0/1760837493 conn(0x7fda2c0793f0 msgr2=0x7fda2c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 -- 172.21.15.26:0/1760837493 shutdown_connections 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 -- 172.21.15.26:0/1760837493 wait complete. 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.120+0000 7fda32ef0700 1 Processor -- start 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 -- start start 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 0x7fda2c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2c06bbe0 0x7fda2c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda2c1095b0 0x7fda2c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c113570 con 0x7fda2c06b220 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1133f0 con 0x7fda2c06bbe0 2024-05-20T21:07:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda32ef0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1136f0 con 0x7fda2c1095b0 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 0x7fda2c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 0x7fda2c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51648/0 (socket says 172.21.15.26:51648) 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 -- 172.21.15.26:0/4218825254 learned_addr learned my addr 172.21.15.26:0/4218825254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda2bfff700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2c06bbe0 0x7fda2c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 -- 172.21.15.26:0/4218825254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda2c1095b0 msgr2=0x7fda2c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda2c1095b0 0x7fda2c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 -- 172.21.15.26:0/4218825254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2c06bbe0 msgr2=0x7fda2c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2c06bbe0 0x7fda2c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.121+0000 7fda30c8c700 1 -- 172.21.15.26:0/4218825254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c19a070 con 0x7fda2c06b220 2024-05-20T21:07:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.122+0000 7fda30c8c700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 0x7fda2c19f750 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fda20006010 tx=0x7fda2000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.122+0000 7fda29ffb700 1 -- 172.21.15.26:0/4218825254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20027070 con 0x7fda2c06b220 2024-05-20T21:07:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.122+0000 7fda29ffb700 1 -- 172.21.15.26:0/4218825254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda20022470 con 0x7fda2c06b220 2024-05-20T21:07:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.122+0000 7fda29ffb700 1 -- 172.21.15.26:0/4218825254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20017650 con 0x7fda2c06b220 2024-05-20T21:07:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.122+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c19a300 con 0x7fda2c06b220 2024-05-20T21:07:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.122+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1a6620 con 0x7fda2c06b220 2024-05-20T21:07:44.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.125+0000 7fda29ffb700 1 -- 172.21.15.26:0/4218825254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fda20009d10 con 0x7fda2c06b220 2024-05-20T21:07:44.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.126+0000 7fda29ffb700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda1c0b2880 0x7fda1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.126+0000 7fda2bfff700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda1c0b2880 0x7fda1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.126+0000 7fda29ffb700 1 -- 172.21.15.26:0/4218825254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fda200e2d40 con 0x7fda2c06b220 2024-05-20T21:07:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 Processor -- start 2024-05-20T21:07:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 -- start start 2024-05-20T21:07:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda2bfff700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda1c0b2880 0x7fda1c0b4d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda2c1085b0 tx=0x7fda1800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 0x561b8cfa3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfa6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfa7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b8cf02d10 con 0x561b8cfa38e0 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561b8cef28b0 con 0x561b8cfa4290 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3d786b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b8cef68f0 con 0x561b8cfa7110 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 0x561b8cfa3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 0x561b8cfa3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51654/0 (socket says 172.21.15.26:51654) 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 -- 172.21.15.26:0/3303902378 learned_addr learned my addr 172.21.15.26:0/3303902378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda2bfff700 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfa6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda3148d700 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfa7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 -- 172.21.15.26:0/3303902378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 msgr2=0x561b8cfa7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfa7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 -- 172.21.15.26:0/3303902378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 msgr2=0x561b8cfa6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfa6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.127+0000 7fda30c8c700 1 -- 172.21.15.26:0/3303902378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8ce42950 con 0x561b8cfa38e0 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.128+0000 7fda30c8c700 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 0x561b8cfa3cc0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fda200a6410 tx=0x7fda200a9ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc859d539e24e8f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.128+0000 7fda16ffd700 1 -- 172.21.15.26:0/3303902378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200ae740 con 0x561b8cfa38e0 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.128+0000 7fda16ffd700 1 -- 172.21.15.26:0/3303902378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda200b3160 con 0x561b8cfa38e0 2024-05-20T21:07:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.128+0000 7fda16ffd700 1 -- 172.21.15.26:0/3303902378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200b2f30 con 0x561b8cfa38e0 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 -- 172.21.15.26:0/3303902378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 msgr2=0x561b8cfa3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 0x561b8cfa3cc0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fda200a6410 tx=0x7fda200a9ea0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 -- 172.21.15.26:0/3303902378 shutdown_connections 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa38e0 0x561b8cfa3cc0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfa7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 --2- 172.21.15.26:0/3303902378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfa6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 -- 172.21.15.26:0/3303902378 >> 172.21.15.26:0/3303902378 conn(0x561b8ced1100 msgr2=0x561b8cf9df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 -- 172.21.15.26:0/3303902378 shutdown_connections 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 -- 172.21.15.26:0/3303902378 wait complete. 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 -- start start 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.129+0000 7fda3d786b80 1 Processor -- start 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa38e0 0x561b8cfb7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfb55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b8cdf9020 con 0x561b8cfa7110 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561b8ce94840 con 0x561b8cfa4290 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b8cfb8800 con 0x561b8cfa38e0 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3d786b80 10 client.0 _ll_register_callbacks cb 0x561b8ce8d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfb55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfb55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51666/0 (socket says 172.21.15.26:51666) 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 -- 172.21.15.26:0/587494066 learned_addr learned my addr 172.21.15.26:0/587494066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda30c8c700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa38e0 0x561b8cfb7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 -- 172.21.15.26:0/587494066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa38e0 msgr2=0x561b8cfb7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa38e0 0x561b8cfb7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 -- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 msgr2=0x561b8cfb50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfb50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda3148d700 1 -- 172.21.15.26:0/587494066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8ce91710 con 0x561b8cfa7110 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.130+0000 7fda2bfff700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfb50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda3148d700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfb55e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fda24011b60 tx=0x7fda24011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea9d208c392b94d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda0bfff700 1 -- 172.21.15.26:0/587494066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2401a020 con 0x561b8cfa7110 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda0bfff700 1 -- 172.21.15.26:0/587494066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fda2400f660 con 0x561b8cfa7110 2024-05-20T21:07:44.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda200b3160 con 0x561b8cfa7110 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda0bfff700 1 -- 172.21.15.26:0/587494066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda24025050 con 0x561b8cfa7110 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda3d786b80 10 client.18984 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b8cfc3020 con 0x561b8cfa7110 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.131+0000 7fda0bfff700 1 -- 172.21.15.26:0/587494066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda200b3160 con 0x561b8cfa7110 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda0bfff700 1 -- 172.21.15.26:0/587494066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fda24007a00 con 0x561b8cfa7110 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda3d786b80 10 client.18984 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda3d786b80 10 client.18984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda3d786b80 20 client.18984 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda3d786b80 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8cfc5700 0x561b8cfc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda3d786b80 4 client.18984 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b8cf6fa60 con 0x561b8cfc5700 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda30c8c700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8cfc5700 0x561b8cfc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda0bfff700 10 client.18984 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.132+0000 7fda30c8c700 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8cfc5700 0x561b8cfc7b20 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7439d341afd3968b server_cookie=2e7b51fc8a9a9b77 in_seq=0 out_seq=0 2024-05-20T21:07:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda0bfff700 1 -- 172.21.15.26:0/587494066 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561b8cf6fa60 con 0x561b8cfc5700 2024-05-20T21:07:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda0bfff700 10 client.18984 handle_command_reply: tid=0 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 client.18984 shutdown 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 msgr2=0x561b8cfb55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfb55e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fda24011b60 tx=0x7fda24011e70 comp rx=0 tx=0).stop 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 shutdown_connections 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8cfc5700 0x561b8cfc7b20 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfb55e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda0bfff700 0 client.18984 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa38e0 0x561b8cfb7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 --2- 172.21.15.26:0/587494066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa4290 0x561b8cfb50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 >> 172.21.15.26:0/587494066 conn(0x561b8ced1100 msgr2=0x561b8cf9d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 shutdown_connections 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.133+0000 7fda3d786b80 1 -- 172.21.15.26:0/587494066 wait complete. 2024-05-20T21:07:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.134+0000 7fda3d786b80 20 client.18984 trim_cache size 0 max 16384 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 Processor -- start 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 -- start start 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfb78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 0x561b8cfcd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfcd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1ae1f0 con 0x561b8cfa4290 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1aba80 con 0x561b8cfa38e0 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3d786b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1a98d0 con 0x561b8cfa7110 2024-05-20T21:07:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 0x561b8cfcd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfb78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 0x561b8cfcd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51678/0 (socket says 172.21.15.26:51678) 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda2bfff700 1 -- 172.21.15.26:0/2587139975 learned_addr learned my addr 172.21.15.26:0/2587139975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda3148d700 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfcd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda30c8c700 1 -- 172.21.15.26:0/2587139975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 msgr2=0x561b8cfcd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda30c8c700 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfcd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda30c8c700 1 -- 172.21.15.26:0/2587139975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 msgr2=0x561b8cfcd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.160+0000 7fda30c8c700 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 0x561b8cfcd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.161+0000 7fda30c8c700 1 -- 172.21.15.26:0/2587139975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8cfa3f10 con 0x561b8cfa38e0 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.161+0000 7fda2bfff700 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 0x561b8cfcd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.161+0000 7fda30c8c700 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfb78e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b360 tx=0x7fda200aa510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fa58eb29e0ae6c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.161+0000 7fda167fc700 1 -- 172.21.15.26:0/2587139975 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200aa5f0 con 0x561b8cfa38e0 2024-05-20T21:07:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.161+0000 7fda167fc700 1 -- 172.21.15.26:0/2587139975 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda200aa790 con 0x561b8cfa38e0 2024-05-20T21:07:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.161+0000 7fda167fc700 1 -- 172.21.15.26:0/2587139975 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200a5d00 con 0x561b8cfa38e0 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 -- 172.21.15.26:0/2587139975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 msgr2=0x561b8cfb78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfb78e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b360 tx=0x7fda200aa510 comp rx=0 tx=0).stop 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 -- 172.21.15.26:0/2587139975 shutdown_connections 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa7110 0x561b8cfcd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfb78e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 --2- 172.21.15.26:0/2587139975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa4290 0x561b8cfcd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 -- 172.21.15.26:0/2587139975 >> 172.21.15.26:0/2587139975 conn(0x561b8ced1100 msgr2=0x561b8cfc0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 -- 172.21.15.26:0/2587139975 shutdown_connections 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 -- 172.21.15.26:0/2587139975 wait complete. 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 -- start start 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.162+0000 7fda3d786b80 1 Processor -- start 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfbdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa4290 0x561b8cfbaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfbaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1a9f40 con 0x561b8cfa7110 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1ab3b0 con 0x561b8cfa38e0 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1aa610 con 0x561b8cfa4290 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3d786b80 10 client.0 _ll_register_callbacks cb 0x561b8ce8d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfbdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfbaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda30c8c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfbdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41020/0 (socket says 172.21.15.26:41020) 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfbaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51694/0 (socket says 172.21.15.26:51694) 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda30c8c700 1 -- 172.21.15.26:0/1755507729 learned_addr learned my addr 172.21.15.26:0/1755507729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda2bfff700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa4290 0x561b8cfbaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 -- 172.21.15.26:0/1755507729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa4290 msgr2=0x561b8cfbaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa4290 0x561b8cfbaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 -- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 msgr2=0x561b8cfbdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfbdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda3148d700 1 -- 172.21.15.26:0/1755507729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8cfbb5d0 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.163+0000 7fda30c8c700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfbdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda3148d700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfbaf60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fda24011fd0 tx=0x7fda24009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685ae4d4ad61199 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda0bfff700 1 -- 172.21.15.26:0/1755507729 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2402d020 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda0bfff700 1 -- 172.21.15.26:0/1755507729 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fda24004db0 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda200aa790 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda0bfff700 1 -- 172.21.15.26:0/1755507729 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda2401a040 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda3d786b80 10 client.18990 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b8d031230 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.164+0000 7fda0bfff700 1 -- 172.21.15.26:0/1755507729 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda200aa790 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda0bfff700 1 -- 172.21.15.26:0/1755507729 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fda24031020 con 0x561b8cfa7110 2024-05-20T21:07:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda3d786b80 10 client.18990 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda3d786b80 10 client.18990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda3d786b80 20 client.18990 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda3d786b80 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8d031940 0x561b8d033d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda3d786b80 4 client.18990 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b8cf04c60 con 0x561b8d031940 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda30c8c700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8d031940 0x561b8d033d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda0bfff700 10 client.18990 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.165+0000 7fda30c8c700 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8d031940 0x561b8d033d60 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ece78f60f96829a server_cookie=90f4826f056ee151 in_seq=0 out_seq=0 2024-05-20T21:07:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda0bfff700 1 -- 172.21.15.26:0/1755507729 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b8cf04c60 con 0x561b8d031940 2024-05-20T21:07:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda0bfff700 10 client.18990 handle_command_reply: tid=0 2024-05-20T21:07:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 client.18990 shutdown 2024-05-20T21:07:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 msgr2=0x561b8cfbaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfbaf60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fda24011fd0 tx=0x7fda24009510 comp rx=0 tx=0).stop 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 shutdown_connections 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561b8d031940 0x561b8d033d60 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b8cfa7110 0x561b8cfbaf60 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561b8cfa38e0 0x561b8cfbdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 --2- 172.21.15.26:0/1755507729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b8cfa4290 0x561b8cfbaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 >> 172.21.15.26:0/1755507729 conn(0x561b8ced1100 msgr2=0x561b8cfc10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda0bfff700 0 client.18990 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 shutdown_connections 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 1 -- 172.21.15.26:0/1755507729 wait complete. 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.167+0000 7fda3d786b80 20 client.18990 trim_cache size 0 max 16384 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:07:44.171 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:07:44.172 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "f68394bc-3f12-44af-baeb-8d4ec57007ef", 2024-05-20T21:07:44.172 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:07:44.172 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:07:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda1c0b2880 msgr2=0x7fda1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda1c0b2880 0x7fda1c0b4d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda2c1085b0 tx=0x7fda1800b410 comp rx=0 tx=0).stop 2024-05-20T21:07:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 msgr2=0x7fda2c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 0x7fda2c19f750 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fda20006010 tx=0x7fda2000bfc0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 shutdown_connections 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda1c0b2880 0x7fda1c0b4d40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c06b220 0x7fda2c19f750 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda2c1095b0 0x7fda2c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 --2- 172.21.15.26:0/4218825254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2c06bbe0 0x7fda2c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 >> 172.21.15.26:0/4218825254 conn(0x7fda2c0793f0 msgr2=0x7fda2c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 shutdown_connections 2024-05-20T21:07:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:44.168+0000 7fda32ef0700 1 -- 172.21.15.26:0/4218825254 wait complete. 2024-05-20T21:07:44.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:44 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:07:44.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:44 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:44.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:44 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:07:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:44 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:07:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:44 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:44.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:44 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:07:44.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 262996 2.26 MB/sec execute 470 sec latency 17.013 ms 2024-05-20T21:07:44.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:44 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:07:44.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:44 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:44.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:44 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:07:44.982 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:07:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:45.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:45 smithi026 ceph-mon[31990]: pgmap v738: 97 pgs: 97 active+clean; 28 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-05-20T21:07:45.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:45 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:45 smithi144 ceph-mon[35660]: pgmap v738: 97 pgs: 97 active+clean; 28 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-05-20T21:07:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:45 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:45.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 263616 2.26 MB/sec execute 471 sec latency 19.744 ms 2024-05-20T21:07:45.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:45 smithi055 ceph-mon[35213]: pgmap v738: 97 pgs: 97 active+clean; 28 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-05-20T21:07:45.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:45 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:07:46.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:46 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:07:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:46 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:07:46.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 264357 2.27 MB/sec execute 472 sec latency 17.609 ms 2024-05-20T21:07:46.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:46 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 -- 172.21.15.26:0/2229031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 msgr2=0x7fcfdc1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/2229031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1103c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b0a0 tx=0x7fcfd400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 -- 172.21.15.26:0/2229031736 shutdown_connections 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/2229031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1103c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/2229031736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfdc1095a0 0x7fcfdc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/2229031736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc06bab0 0x7fcfdc06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 -- 172.21.15.26:0/2229031736 >> 172.21.15.26:0/2229031736 conn(0x7fcfdc0793b0 msgr2=0x7fcfdc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 -- 172.21.15.26:0/2229031736 shutdown_connections 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.043+0000 7fcfe42a0700 1 -- 172.21.15.26:0/2229031736 wait complete. 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 Processor -- start 2024-05-20T21:07:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 -- start start 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc06bab0 0x7fcfdc1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfdc1095a0 0x7fcfdc19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc113560 con 0x7fcfdc06c470 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1133e0 con 0x7fcfdc06bab0 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe42a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1136e0 con 0x7fcfdc1095a0 2024-05-20T21:07:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51706/0 (socket says 172.21.15.26:51706) 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe183b700 1 -- 172.21.15.26:0/3188497257 learned_addr learned my addr 172.21.15.26:0/3188497257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe203c700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc06bab0 0x7fcfdc1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.044+0000 7fcfe283d700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfdc1095a0 0x7fcfdc19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe183b700 1 -- 172.21.15.26:0/3188497257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfdc1095a0 msgr2=0x7fcfdc19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe183b700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfdc1095a0 0x7fcfdc19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe183b700 1 -- 172.21.15.26:0/3188497257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc06bab0 msgr2=0x7fcfdc1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe183b700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc06bab0 0x7fcfdc1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe183b700 1 -- 172.21.15.26:0/3188497257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc19c220 con 0x7fcfdc06c470 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe183b700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1a1e40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e9a0 tx=0x7fcfcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfd37fe700 1 -- 172.21.15.26:0/3188497257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00cbd0 con 0x7fcfdc06c470 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc19c510 con 0x7fcfdc06c470 2024-05-20T21:07:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfd37fe700 1 -- 172.21.15.26:0/3188497257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfcc00cd70 con 0x7fcfdc06c470 2024-05-20T21:07:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfd37fe700 1 -- 172.21.15.26:0/3188497257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc018680 con 0x7fcfdc06c470 2024-05-20T21:07:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.045+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a87d0 con 0x7fcfdc06c470 2024-05-20T21:07:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.048+0000 7fcfd37fe700 1 -- 172.21.15.26:0/3188497257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcfcc010430 con 0x7fcfdc06c470 2024-05-20T21:07:47.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.049+0000 7fcfd37fe700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfc80b2900 0x7fcfc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.049+0000 7fcfe203c700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfc80b2900 0x7fcfc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.049+0000 7fcfd37fe700 1 -- 172.21.15.26:0/3188497257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcfcc0e2470 con 0x7fcfdc06c470 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 Processor -- start 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfe203c700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfc80b2900 0x7fcfc80b4dc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc19cc00 tx=0x7fcfd80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 -- start start 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbe7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbe1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a7bb3fd70 con 0x563a7bbe7320 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563a7bb2f910 con 0x563a7bbe2260 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a7bb33950 con 0x563a7bbe18a0 2024-05-20T21:07:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfe203c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbe7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfe203c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbe7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51720/0 (socket says 172.21.15.26:51720) 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfe203c700 1 -- 172.21.15.26:0/542835913 learned_addr learned my addr 172.21.15.26:0/542835913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfe183b700 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbe1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.050+0000 7fcfe283d700 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfe203c700 1 -- 172.21.15.26:0/542835913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 msgr2=0x563a7bbe1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfe203c700 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbe1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfe203c700 1 -- 172.21.15.26:0/542835913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 msgr2=0x563a7bbee080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfe203c700 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbee080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfe203c700 1 -- 172.21.15.26:0/542835913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a7bad8fb0 con 0x563a7bbe7320 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfe203c700 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbe7700 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8004570 tx=0x7fcfd8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3dbf8a031a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfd0d70700 1 -- 172.21.15.26:0/542835913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8024070 con 0x563a7bbe7320 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfd0d70700 1 -- 172.21.15.26:0/542835913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfd8015070 con 0x563a7bbe7320 2024-05-20T21:07:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.051+0000 7fcfd0d70700 1 -- 172.21.15.26:0/542835913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8011770 con 0x563a7bbe7320 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/542835913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 msgr2=0x563a7bbe7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbe7700 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8004570 tx=0x7fcfd8013000 comp rx=0 tx=0).stop 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/542835913 shutdown_connections 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbe7700 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/542835913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbe1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/542835913 >> 172.21.15.26:0/542835913 conn(0x563a7bb0d240 msgr2=0x563a7bbdc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/542835913 shutdown_connections 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/542835913 wait complete. 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.052+0000 7fcfeeb36b80 1 -- start start 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 Processor -- start 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbeb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbeb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a7ba35fa0 con 0x563a7bbe7320 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563a7ba97da0 con 0x563a7bbe2260 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a7bbee6e0 con 0x563a7bbe18a0 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbeb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfe283d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbeb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbeb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41056/0 (socket says 172.21.15.26:41056) 2024-05-20T21:07:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfe183b700 1 -- 172.21.15.26:0/2975009614 learned_addr learned my addr 172.21.15.26:0/2975009614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfeeb36b80 10 client.0 _ll_register_callbacks cb 0x563a7baca950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.053+0000 7fcfe203c700 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfe183b700 1 -- 172.21.15.26:0/2975009614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 msgr2=0x563a7bbedd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfe183b700 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfe183b700 1 -- 172.21.15.26:0/2975009614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 msgr2=0x563a7bbeb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfe183b700 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbeb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfe183b700 1 -- 172.21.15.26:0/2975009614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a7ba7f8d0 con 0x563a7bbe2260 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfe183b700 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbeb040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0e0280 tx=0x7fcfcc0b26e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61cb2f70be03f493 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/2975009614 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0b21d0 con 0x563a7bbe2260 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/2975009614 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcfcc0cc2d0 con 0x563a7bbe2260 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd8015070 con 0x563a7bbe2260 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.054+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/2975009614 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfcc0a9540 con 0x563a7bbe2260 2024-05-20T21:07:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 10 client.28301 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a7bae9760 con 0x563a7bbe2260 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/2975009614 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfd8015070 con 0x563a7bbe2260 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/2975009614 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcfcc0cace0 con 0x563a7bbe2260 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 10 client.28301 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 10 client.28301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 20 client.28301 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bbfb2c0 0x563a7bbfd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 4 client.28301 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a7bb41cc0 con 0x563a7bbfb2c0 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.055+0000 7fcfe203c700 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bbfb2c0 0x563a7bbfd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.056+0000 7fcfe203c700 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bbfb2c0 0x563a7bbfd6b0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94585582157a051 server_cookie=84486431c8349e0a in_seq=0 out_seq=0 2024-05-20T21:07:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.056+0000 7fcfc5ffb700 10 client.28301 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/2975009614 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a7bb41cc0 con 0x563a7bbfb2c0 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfc5ffb700 10 client.28301 handle_command_reply: tid=0 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 client.28301 shutdown 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 msgr2=0x563a7bbeb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbeb040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0e0280 tx=0x7fcfcc0b26e0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 shutdown_connections 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bbfb2c0 0x563a7bbfd6b0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bbeb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfc5ffb700 0 client.28301 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:47.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/2975009614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe2260 0x563a7bbeb040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 >> 172.21.15.26:0/2975009614 conn(0x563a7bb0d240 msgr2=0x563a7bbdbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:47.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 shutdown_connections 2024-05-20T21:07:47.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/2975009614 wait complete. 2024-05-20T21:07:47.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.057+0000 7fcfeeb36b80 20 client.28301 trim_cache size 0 max 16384 2024-05-20T21:07:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 Processor -- start 2024-05-20T21:07:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 -- start start 2024-05-20T21:07:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 0x563a7bbdc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe7320 0x563a7bbe8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b2770 con 0x563a7bbe2260 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b0000 con 0x563a7bbe18a0 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.082+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1ade50 con 0x563a7bbe7320 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 0x563a7bbdc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 0x563a7bbdc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51742/0 (socket says 172.21.15.26:51742) 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 -- 172.21.15.26:0/420314524 learned_addr learned my addr 172.21.15.26:0/420314524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe203c700 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe283d700 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe7320 0x563a7bbe8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 -- 172.21.15.26:0/420314524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe7320 msgr2=0x563a7bbe8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe7320 0x563a7bbe8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 -- 172.21.15.26:0/420314524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe18a0 msgr2=0x563a7bbedd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 -- 172.21.15.26:0/420314524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a7bbfdce0 con 0x563a7bbe2260 2024-05-20T21:07:47.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfe183b700 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 0x563a7bbdc270 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0a6ca0 tx=0x7fcfcc0a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8286a01c9edb2f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:47.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfc7fff700 1 -- 172.21.15.26:0/420314524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0ca070 con 0x563a7bbe2260 2024-05-20T21:07:47.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfc7fff700 1 -- 172.21.15.26:0/420314524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfcc0b21d0 con 0x563a7bbe2260 2024-05-20T21:07:47.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.083+0000 7fcfc7fff700 1 -- 172.21.15.26:0/420314524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0a9a30 con 0x563a7bbe2260 2024-05-20T21:07:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/420314524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 msgr2=0x563a7bbdc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 0x563a7bbdc270 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0a6ca0 tx=0x7fcfcc0a5a20 comp rx=0 tx=0).stop 2024-05-20T21:07:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/420314524 shutdown_connections 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe7320 0x563a7bbe8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bbe18a0 0x563a7bbedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/420314524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe2260 0x563a7bbdc270 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/420314524 >> 172.21.15.26:0/420314524 conn(0x563a7bb0d240 msgr2=0x563a7bbe1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/420314524 shutdown_connections 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.084+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/420314524 wait complete. 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 -- start start 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 Processor -- start 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bc02c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bc031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bc03700 0x563a7bbf20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1ae4c0 con 0x563a7bbe7320 2024-05-20T21:07:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1af930 con 0x563a7bc03700 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1aeb90 con 0x563a7bbe18a0 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfeeb36b80 10 client.0 _ll_register_callbacks cb 0x563a7baca950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bc031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfe203c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bc02c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfe183b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bc031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51758/0 (socket says 172.21.15.26:51758) 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfe183b700 1 -- 172.21.15.26:0/4195031233 learned_addr learned my addr 172.21.15.26:0/4195031233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.085+0000 7fcfe283d700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bc03700 0x563a7bbf20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe283d700 1 -- 172.21.15.26:0/4195031233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 msgr2=0x563a7bc02c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe283d700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bc02c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe283d700 1 -- 172.21.15.26:0/4195031233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 msgr2=0x563a7bc031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe283d700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bc031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe283d700 1 -- 172.21.15.26:0/4195031233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a7bbf27a0 con 0x563a7bc03700 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe183b700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bc031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfe283d700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bc03700 0x563a7bbf20a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b780 tx=0x7fcfd400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef37dcfebc87b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/4195031233 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4004c60 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/4195031233 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcfd4004e00 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a7bb3b4a0 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/4195031233 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfd40074e0 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfeeb36b80 10 client.28307 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.086+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a7bc6f000 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/4195031233 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563a7bb3b4a0 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/4195031233 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcfd402d020 con 0x563a7bc03700 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfeeb36b80 10 client.28307 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfeeb36b80 10 client.28307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfeeb36b80 20 client.28307 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bc6f6e0 0x563a7bc71b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfeeb36b80 4 client.28307 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a7b9b2250 con 0x563a7bc6f6e0 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfe203c700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bc6f6e0 0x563a7bc71b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfe203c700 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bc6f6e0 0x563a7bc71b00 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85616af3680d572 server_cookie=6a87aa3337510142 in_seq=0 out_seq=0 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.087+0000 7fcfc5ffb700 10 client.28307 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfc5ffb700 1 -- 172.21.15.26:0/4195031233 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a7b9b2250 con 0x563a7bc6f6e0 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfc5ffb700 10 client.28307 handle_command_reply: tid=0 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 client.28307 shutdown 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bc03700 msgr2=0x563a7bbf20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bc03700 0x563a7bbf20a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b780 tx=0x7fcfd400be20 comp rx=0 tx=0).stop 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 shutdown_connections 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563a7bc6f6e0 0x563a7bc71b00 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563a7bc03700 0x563a7bbf20a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a7bbe18a0 0x563a7bc02c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 --2- 172.21.15.26:0/4195031233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7bbe7320 0x563a7bc031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 >> 172.21.15.26:0/4195031233 conn(0x563a7bb0d240 msgr2=0x563a7bbe4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfc5ffb700 0 client.28307 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 shutdown_connections 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 1 -- 172.21.15.26:0/4195031233 wait complete. 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.088+0000 7fcfeeb36b80 20 client.28307 trim_cache size 0 max 16384 2024-05-20T21:07:47.093 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:07:47.094 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:07:47.094 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:07:47.094 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:07:47.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfc80b2900 msgr2=0x7fcfc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfc80b2900 0x7fcfc80b4dc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc19cc00 tx=0x7fcfd80073d0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 msgr2=0x7fcfdc1a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1a1e40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e9a0 tx=0x7fcfcc00ecb0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 shutdown_connections 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfdc1095a0 0x7fcfdc19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc06bab0 0x7fcfdc1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfc80b2900 0x7fcfc80b4dc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 --2- 172.21.15.26:0/3188497257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc06c470 0x7fcfdc1a1e40 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 >> 172.21.15.26:0/3188497257 conn(0x7fcfdc0793b0 msgr2=0x7fcfdc10e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 shutdown_connections 2024-05-20T21:07:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:47.089+0000 7fcfe42a0700 1 -- 172.21.15.26:0/3188497257 wait complete. 2024-05-20T21:07:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:47 smithi144 ceph-mon[35660]: pgmap v739: 97 pgs: 97 active+clean; 29 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.6 MiB/s wr, 97 op/s 2024-05-20T21:07:47.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:47 smithi026 ceph-mon[31990]: pgmap v739: 97 pgs: 97 active+clean; 29 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.6 MiB/s wr, 97 op/s 2024-05-20T21:07:47.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 264980 2.27 MB/sec execute 473 sec latency 43.284 ms 2024-05-20T21:07:47.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:47 smithi055 ceph-mon[35213]: pgmap v739: 97 pgs: 97 active+clean; 29 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.6 MiB/s wr, 97 op/s 2024-05-20T21:07:47.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:07:47.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:07:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:48.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 265823 2.27 MB/sec execute 474 sec latency 11.668 ms 2024-05-20T21:07:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 -- 172.21.15.26:0/1894984505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 msgr2=0x7fccc8101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1894984505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8101dd0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009900 tx=0x7fccb802a410 comp rx=0 tx=0).stop 2024-05-20T21:07:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 -- 172.21.15.26:0/1894984505 shutdown_connections 2024-05-20T21:07:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1894984505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8103560 0x7fccc8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1894984505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc8102ba0 0x7fccc8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1894984505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8101dd0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 -- 172.21.15.26:0/1894984505 >> 172.21.15.26:0/1894984505 conn(0x7fccc80fcf20 msgr2=0x7fccc80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 -- 172.21.15.26:0/1894984505 shutdown_connections 2024-05-20T21:07:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.201+0000 7fccccd0a640 1 -- 172.21.15.26:0/1894984505 wait complete. 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 Processor -- start 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 -- start start 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8102ba0 0x7fccc8103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc8103560 0x7fccc81044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810d380 con 0x7fccc81019b0 2024-05-20T21:07:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810d200 con 0x7fccc8102ba0 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccccd0a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810d500 con 0x7fccc8103560 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51780/0 (socket says 172.21.15.26:51780) 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 -- 172.21.15.26:0/1502197535 learned_addr learned my addr 172.21.15.26:0/1502197535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8102ba0 0x7fccc8103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc8103560 0x7fccc81044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 -- 172.21.15.26:0/1502197535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc8103560 msgr2=0x7fccc81044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc8103560 0x7fccc81044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 -- 172.21.15.26:0/1502197535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8102ba0 msgr2=0x7fccc8103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.202+0000 7fccc659c640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8102ba0 0x7fccc8103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccc659c640 1 -- 172.21.15.26:0/1502197535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc8104d50 con 0x7fccc81019b0 2024-05-20T21:07:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8102ba0 0x7fccc8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccc659c640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8105940 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009230 tx=0x7fccb80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccb77fe640 1 -- 172.21.15.26:0/1502197535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb803f030 con 0x7fccc81019b0 2024-05-20T21:07:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccb77fe640 1 -- 172.21.15.26:0/1502197535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccb80044c0 con 0x7fccc81019b0 2024-05-20T21:07:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccb77fe640 1 -- 172.21.15.26:0/1502197535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8035070 con 0x7fccc81019b0 2024-05-20T21:07:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a2c90 con 0x7fccc81019b0 2024-05-20T21:07:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.203+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a3190 con 0x7fccc81019b0 2024-05-20T21:07:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 Processor -- start 2024-05-20T21:07:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fccb77fe640 1 -- 172.21.15.26:0/1502197535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fccb8042030 con 0x7fccc81019b0 2024-05-20T21:07:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 -- start start 2024-05-20T21:07:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cd0b0 0x5568b92d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5568b923dc30 con 0x5568b92cd0b0 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5568b922d7d0 con 0x5568b92cc6c0 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fcccfa27740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5568b9231810 con 0x5568b92d2220 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fccc659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cd0b0 0x5568b92d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.206+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cd0b0 0x5568b92d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51790/0 (socket says 172.21.15.26:51790) 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc659c640 1 -- 172.21.15.26:0/4255409379 learned_addr learned my addr 172.21.15.26:0/4255409379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4255409379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 msgr2=0x5568b92d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92d46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4255409379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cd0b0 msgr2=0x5568b92d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cd0b0 0x5568b92d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4255409379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b8f42210 con 0x5568b92cc6c0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccb77fe640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccb00b28a0 0x7fccb00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92ccae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009ed0 tx=0x7fccc00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83003e1e7ddde23c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccc659c640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccb00b28a0 0x7fccb00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccb4ff9640 1 -- 172.21.15.26:0/4255409379 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003b70 con 0x5568b92cc6c0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccb4ff9640 1 -- 172.21.15.26:0/4255409379 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccc0015070 con 0x5568b92cc6c0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccb4ff9640 1 -- 172.21.15.26:0/4255409379 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00116e0 con 0x5568b92cc6c0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.207+0000 7fccb77fe640 1 -- 172.21.15.26:0/1502197535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fccb81027f0 con 0x7fccc81019b0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 -- 172.21.15.26:0/4255409379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 msgr2=0x5568b92ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fccc659c640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccb00b28a0 0x7fccb00b4d60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fccb80c5e30 tx=0x7fccb8108550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92ccae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009ed0 tx=0x7fccc00107a0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 -- 172.21.15.26:0/4255409379 shutdown_connections 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92d46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cd0b0 0x5568b92d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4255409379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92ccae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 -- 172.21.15.26:0/4255409379 >> 172.21.15.26:0/4255409379 conn(0x5568b92c7d30 msgr2=0x5568b92c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 -- 172.21.15.26:0/4255409379 shutdown_connections 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.208+0000 7fcccfa27740 1 -- 172.21.15.26:0/4255409379 wait complete. 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 -- start start 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 Processor -- start 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92d6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5568b8f13790 con 0x5568b92d2220 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5568b91e9890 con 0x5568b92cc6c0 2024-05-20T21:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5568b92d51d0 con 0x5568b92cd0b0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fcccfa27740 10 client.0 _ll_register_callbacks cb 0x5568b8f41a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92d6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92d6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41114/0 (socket says 172.21.15.26:41114) 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4114913745 learned_addr learned my addr 172.21.15.26:0/4114913745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4114913745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 msgr2=0x5568b92d6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92d6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4114913745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 msgr2=0x5568b92ddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92ddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.209+0000 7fccc6d9d640 1 -- 172.21.15.26:0/4114913745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b900e620 con 0x5568b92cc6c0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92d6820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011890 tx=0x7fccc0010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53bb8d77ad9c6787 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccadffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccadffb640 1 -- 172.21.15.26:0/4114913745 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0025040 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc0015070 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccadffb640 1 -- 172.21.15.26:0/4114913745 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fccc0004c40 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccadffb640 1 -- 172.21.15.26:0/4114913745 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccc0035050 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 10 client.28319 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568b92e0dd0 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccadffb640 1 -- 172.21.15.26:0/4114913745 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccc0015070 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fccadffb640 1 -- 172.21.15.26:0/4114913745 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fccc001c470 con 0x5568b92cc6c0 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 10 client.28319 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 10 client.28319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 20 client.28319 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b92e22a0 0x5568b92e4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 4 client.28319 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.210+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b923fb80 con 0x5568b92e22a0 2024-05-20T21:07:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.211+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b92e22a0 0x5568b92e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.211+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b92e22a0 0x5568b92e4690 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ed0c9f90007a1 server_cookie=fbefc64605c8f6cb in_seq=0 out_seq=0 2024-05-20T21:07:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.211+0000 7fccadffb640 10 client.28319 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fccadffb640 1 -- 172.21.15.26:0/4114913745 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5568b923fb80 con 0x5568b92e22a0 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fccadffb640 10 client.28319 handle_command_reply: tid=0 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 client.28319 shutdown 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 msgr2=0x5568b92d6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92d6820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011890 tx=0x7fccc0010fd0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 shutdown_connections 2024-05-20T21:07:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b92e22a0 0x5568b92e4690 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92ddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92d6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fccadffb640 0 client.28319 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 --2- 172.21.15.26:0/4114913745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92d6820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 >> 172.21.15.26:0/4114913745 conn(0x5568b92c7d30 msgr2=0x5568b92c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 shutdown_connections 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 1 -- 172.21.15.26:0/4114913745 wait complete. 2024-05-20T21:07:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.212+0000 7fcccfa27740 20 client.28319 trim_cache size 0 max 16384 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 Processor -- start 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 -- start start 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 0x5568b92e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cd0b0 0x5568b92ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81aea20 con 0x5568b92cc6c0 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81ac2f0 con 0x5568b92cd0b0 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fcccfa27740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81aa140 con 0x5568b92d2220 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 0x5568b92e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 0x5568b92e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51804/0 (socket says 172.21.15.26:51804) 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fccc6d9d640 1 -- 172.21.15.26:0/108776966 learned_addr learned my addr 172.21.15.26:0/108776966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.229+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cd0b0 0x5568b92ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccc6d9d640 1 -- 172.21.15.26:0/108776966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 msgr2=0x5568b92ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccc6d9d640 1 -- 172.21.15.26:0/108776966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cd0b0 msgr2=0x5568b92ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cd0b0 0x5568b92ddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccc6d9d640 1 -- 172.21.15.26:0/108776966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b92c7b60 con 0x5568b92cc6c0 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 0x5568b92e9f30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fccc00042a0 tx=0x7fccc00116d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8391c2a19ef0002 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccaffff640 1 -- 172.21.15.26:0/108776966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0032040 con 0x5568b92cc6c0 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccaffff640 1 -- 172.21.15.26:0/108776966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccc0015030 con 0x5568b92cc6c0 2024-05-20T21:07:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.230+0000 7fccaffff640 1 -- 172.21.15.26:0/108776966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003bd0 con 0x5568b92cc6c0 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 -- 172.21.15.26:0/108776966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 msgr2=0x5568b92e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 0x5568b92e9f30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fccc00042a0 tx=0x7fccc00116d0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 -- 172.21.15.26:0/108776966 shutdown_connections 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92d2220 0x5568b92ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cd0b0 0x5568b92ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 --2- 172.21.15.26:0/108776966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92cc6c0 0x5568b92e9f30 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 -- 172.21.15.26:0/108776966 >> 172.21.15.26:0/108776966 conn(0x5568b92c7d30 msgr2=0x5568b92c2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 -- 172.21.15.26:0/108776966 shutdown_connections 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 -- 172.21.15.26:0/108776966 wait complete. 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 -- start start 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.231+0000 7fcccfa27740 1 Processor -- start 2024-05-20T21:07:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81aa7b0 con 0x5568b92d2220 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81abc20 con 0x5568b92cc6c0 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81aae80 con 0x5568b92cd0b0 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fcccfa27740 10 client.0 _ll_register_callbacks cb 0x7fccb8108420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41140/0 (socket says 172.21.15.26:41140) 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 -- 172.21.15.26:0/3572986662 learned_addr learned my addr 172.21.15.26:0/3572986662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 -- 172.21.15.26:0/3572986662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 msgr2=0x5568b92db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92db830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 -- 172.21.15.26:0/3572986662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 msgr2=0x5568b92dc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92dc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 -- 172.21.15.26:0/3572986662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b92dce30 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccc6d9d640 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92db2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fccc00107e0 tx=0x7fccc0005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5bec68fa8c4a230 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.232+0000 7fccadffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fccadffb640 1 -- 172.21.15.26:0/3572986662 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003b70 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fccadffb640 1 -- 172.21.15.26:0/3572986662 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fccc00044e0 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc0015030 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fccadffb640 1 -- 172.21.15.26:0/3572986662 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccc0003eb0 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 10 client.28325 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568b92de140 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fccadffb640 1 -- 172.21.15.26:0/3572986662 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccc0015030 con 0x5568b92cc6c0 2024-05-20T21:07:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fccadffb640 1 -- 172.21.15.26:0/3572986662 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fccc0045070 con 0x5568b92cc6c0 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 10 client.28325 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 10 client.28325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 20 client.28325 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b934d510 0x5568b934f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 4 client.28325 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.233+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568b9239360 con 0x5568b934d510 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.234+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b934d510 0x5568b934f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.234+0000 7fccadffb640 10 client.28325 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.234+0000 7fccc5d9b640 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b934d510 0x5568b934f930 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e485026e9eb4fbfb server_cookie=b5cfa9b6d1e27b95 in_seq=0 out_seq=0 2024-05-20T21:07:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fccadffb640 1 -- 172.21.15.26:0/3572986662 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5568b9239360 con 0x5568b934d510 2024-05-20T21:07:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fccadffb640 10 client.28325 handle_command_reply: tid=0 2024-05-20T21:07:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 client.28325 shutdown 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 msgr2=0x5568b92db2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92db2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fccc00107e0 tx=0x7fccc0005a70 comp rx=0 tx=0).stop 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 shutdown_connections 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5568b934d510 0x5568b934f930 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568b92d2220 0x5568b92dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568b92cd0b0 0x5568b92db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fccadffb640 0 client.28325 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 --2- 172.21.15.26:0/3572986662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568b92cc6c0 0x5568b92db2f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 >> 172.21.15.26:0/3572986662 conn(0x5568b92c7d30 msgr2=0x5568b92d6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 shutdown_connections 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 1 -- 172.21.15.26:0/3572986662 wait complete. 2024-05-20T21:07:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.271+0000 7fcccfa27740 20 client.28325 trim_cache size 0 max 16384 2024-05-20T21:07:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:07:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:07:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccb00b28a0 msgr2=0x7fccb00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccb00b28a0 0x7fccb00b4d60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fccb80c5e30 tx=0x7fccb8108550 comp rx=0 tx=0).stop 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 msgr2=0x7fccc8105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8105940 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009230 tx=0x7fccb80040c0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 shutdown_connections 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fccb00b28a0 0x7fccb00b4d60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccc8103560 0x7fccc81044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc8102ba0 0x7fccc8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 --2- 172.21.15.26:0/1502197535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc81019b0 0x7fccc8105940 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 >> 172.21.15.26:0/1502197535 conn(0x7fccc80fcf20 msgr2=0x7fccc806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 shutdown_connections 2024-05-20T21:07:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:07:49.272+0000 7fccccd0a640 1 -- 172.21.15.26:0/1502197535 wait complete. 2024-05-20T21:07:49.368 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:49 smithi026 ceph-mon[31990]: pgmap v740: 97 pgs: 97 active+clean; 30 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T21:07:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:49 smithi144 ceph-mon[35660]: pgmap v740: 97 pgs: 97 active+clean; 30 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T21:07:49.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 266378 2.28 MB/sec execute 475 sec latency 25.399 ms 2024-05-20T21:07:49.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:49 smithi055 ceph-mon[35213]: pgmap v740: 97 pgs: 97 active+clean; 30 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T21:07:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 -- 172.21.15.26:0/2655242427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 msgr2=0x7f28dc109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 --2- 172.21.15.26:0/2655242427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc109a60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f28d80099c0 tx=0x7f28d8009cd0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 -- 172.21.15.26:0/2655242427 shutdown_connections 2024-05-20T21:07:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 --2- 172.21.15.26:0/2655242427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc109a60 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 --2- 172.21.15.26:0/2655242427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28dc103fa0 0x7f28dc110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 --2- 172.21.15.26:0/2655242427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 0x7f28dc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 -- 172.21.15.26:0/2655242427 >> 172.21.15.26:0/2655242427 conn(0x7f28dc07a6f0 msgr2=0x7f28dc07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 -- 172.21.15.26:0/2655242427 shutdown_connections 2024-05-20T21:07:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.966+0000 7f28e3675700 1 -- 172.21.15.26:0/2655242427 wait complete. 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 Processor -- start 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 -- start start 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 0x7f28dc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28dc103fa0 0x7f28dc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc113650 con 0x7f28dc109680 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1134d0 con 0x7f28dc1035e0 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e3675700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1137d0 con 0x7f28dc103fa0 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e1c12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e1c12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40866/0 (socket says 172.21.15.26:40866) 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e1c12700 1 -- 172.21.15.26:0/55029407 learned_addr learned my addr 172.21.15.26:0/55029407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.967+0000 7f28e1411700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 0x7f28dc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e0c10700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28dc103fa0 0x7f28dc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1411700 1 -- 172.21.15.26:0/55029407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28dc103fa0 msgr2=0x7f28dc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1411700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28dc103fa0 0x7f28dc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1411700 1 -- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 msgr2=0x7f28dc1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1411700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1411700 1 -- 172.21.15.26:0/55029407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc19a110 con 0x7f28dc1035e0 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1c12700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e1411700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 0x7f28dc19f7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28d80098e0 tx=0x7f28d80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8026070 con 0x7f28dc1035e0 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc19a3a0 con 0x7f28dc1035e0 2024-05-20T21:07:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28d8003d50 con 0x7f28dc1035e0 2024-05-20T21:07:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d800fb80 con 0x7f28dc1035e0 2024-05-20T21:07:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.968+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc1a6670 con 0x7f28dc1035e0 2024-05-20T21:07:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.969+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28dc10bb10 con 0x7f28dc1035e0 2024-05-20T21:07:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.971+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f28d8029070 con 0x7f28dc1035e0 2024-05-20T21:07:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.971+0000 7f28d27fc700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28c80b2880 0x7f28c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.972+0000 7f28e0c10700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28c80b2880 0x7f28c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.972+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f28d80e2ee0 con 0x7f28dc1035e0 2024-05-20T21:07:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.972+0000 7f28e0c10700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28c80b2880 0x7f28c80b4d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28dc07ad60 tx=0x7f28cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:49.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:49.975+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28d8030080 con 0x7f28dc1035e0 2024-05-20T21:07:50.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.282+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28dc19ae90 con 0x7f28dc1035e0 2024-05-20T21:07:50.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.283+0000 7f28d27fc700 1 -- 172.21.15.26:0/55029407 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f28d8013070 con 0x7f28dc1035e0 2024-05-20T21:07:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:07:50.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:07:50.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28c80b2880 msgr2=0x7f28c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:50.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28c80b2880 0x7f28c80b4d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28dc07ad60 tx=0x7f28cc009260 comp rx=0 tx=0).stop 2024-05-20T21:07:50.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 msgr2=0x7f28dc19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:50.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 0x7f28dc19f7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28d80098e0 tx=0x7f28d80045e0 comp rx=0 tx=0).stop 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 shutdown_connections 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f28c80b2880 0x7f28c80b4d40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28dc109680 0x7f28dc1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28dc1035e0 0x7f28dc19f7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 --2- 172.21.15.26:0/55029407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28dc103fa0 0x7f28dc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.285+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 >> 172.21.15.26:0/55029407 conn(0x7f28dc07a6f0 msgr2=0x7f28dc102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.286+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 shutdown_connections 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:50.286+0000 7f28e3675700 1 -- 172.21.15.26:0/55029407 wait complete. 2024-05-20T21:07:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:07:50.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 266850 2.28 MB/sec execute 476 sec latency 36.270 ms 2024-05-20T21:07:51.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:07:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:51 smithi144 ceph-mon[35660]: pgmap v741: 97 pgs: 97 active+clean; 26 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-20T21:07:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:51 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/55029407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:51.582 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:51 smithi026 ceph-mon[31990]: pgmap v741: 97 pgs: 97 active+clean; 26 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-20T21:07:51.582 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:51 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/55029407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:51.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 267515 2.28 MB/sec execute 477 sec latency 17.381 ms 2024-05-20T21:07:51.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:51 smithi055 ceph-mon[35213]: pgmap v741: 97 pgs: 97 active+clean; 26 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-20T21:07:51.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:51 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/55029407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:07:52.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 268238 2.28 MB/sec execute 478 sec latency 14.358 ms 2024-05-20T21:07:53.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 -- 172.21.15.26:0/697691479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc101b30 msgr2=0x7f7fbc101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 --2- 172.21.15.26:0/697691479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc101b30 0x7f7fbc101fb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009990 tx=0x7f7fac009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 -- 172.21.15.26:0/697691479 shutdown_connections 2024-05-20T21:07:53.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 --2- 172.21.15.26:0/697691479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 0x7f7fbc1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 --2- 172.21.15.26:0/697691479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc104450 0x7f7fbc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 --2- 172.21.15.26:0/697691479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc101b30 0x7f7fbc101fb0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 -- 172.21.15.26:0/697691479 >> 172.21.15.26:0/697691479 conn(0x7f7fbc078ab0 msgr2=0x7f7fbc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:53.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.105+0000 7f7fc4637700 1 -- 172.21.15.26:0/697691479 shutdown_connections 2024-05-20T21:07:53.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.106+0000 7f7fc4637700 1 -- 172.21.15.26:0/697691479 wait complete. 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.106+0000 7f7fc4637700 1 Processor -- start 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.106+0000 7f7fc4637700 1 -- start start 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc4637700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 0x7f7fbc19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc4637700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 0x7f7fbc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc4637700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc104450 0x7f7fbc1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc4637700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc115890 con 0x7f7fbc104450 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc4637700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc115710 con 0x7f7fbc101b30 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc4637700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc115a10 con 0x7f7fbc1025d0 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 0x7f7fbc19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 0x7f7fbc19bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46244/0 (socket says 172.21.15.26:46244) 2024-05-20T21:07:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 -- 172.21.15.26:0/534492208 learned_addr learned my addr 172.21.15.26:0/534492208 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc104450 0x7f7fbc1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 0x7f7fbc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 -- 172.21.15.26:0/534492208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 msgr2=0x7f7fbc19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 0x7f7fbc19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 -- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc104450 msgr2=0x7f7fbc1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc104450 0x7f7fbc1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc23d3700 1 -- 172.21.15.26:0/534492208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc1a0ea0 con 0x7f7fbc101b30 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.107+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 0x7f7fbc19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.108+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 0x7f7fbc19bee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800b530 tx=0x7f7fb800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.108+0000 7f7fb37fe700 1 -- 172.21.15.26:0/534492208 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8015020 con 0x7f7fbc101b30 2024-05-20T21:07:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.108+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc1a1190 con 0x7f7fbc101b30 2024-05-20T21:07:53.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.108+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1a8a10 con 0x7f7fbc101b30 2024-05-20T21:07:53.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.108+0000 7f7fb37fe700 1 -- 172.21.15.26:0/534492208 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fb8011cc0 con 0x7f7fbc101b30 2024-05-20T21:07:53.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.108+0000 7f7fb37fe700 1 -- 172.21.15.26:0/534492208 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8010dc0 con 0x7f7fbc101b30 2024-05-20T21:07:53.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.111+0000 7f7fb37fe700 1 -- 172.21.15.26:0/534492208 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7fb8018020 con 0x7f7fbc101b30 2024-05-20T21:07:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.112+0000 7f7fb37fe700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fa80b2880 0x7f7fa80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.112+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fa80b2880 0x7f7fa80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fb37fe700 1 -- 172.21.15.26:0/534492208 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7fb80e1a90 con 0x7f7fbc101b30 2024-05-20T21:07:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 Processor -- start 2024-05-20T21:07:53.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 -- start start 2024-05-20T21:07:53.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fa80b2880 0x7f7fa80b4d40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009750 tx=0x7f7fac01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:53.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a34bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 0x56468a34f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34ff60 0x56468a3503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56468a2aac90 con 0x56468a34c210 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56468a29a830 con 0x56468a34ff60 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56468a29e870 con 0x56468a34b860 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fc1bd2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 0x56468a34f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fc1bd2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 0x56468a34f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40902/0 (socket says 172.21.15.26:40902) 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fc1bd2700 1 -- 172.21.15.26:0/1917925482 learned_addr learned my addr 172.21.15.26:0/1917925482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.113+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34ff60 0x56468a3503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a34bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1917925482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 msgr2=0x56468a34bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a34bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1917925482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 msgr2=0x56468a34f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 0x56468a34f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1917925482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468a1ea8d0 con 0x56468a34ff60 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 0x56468a34f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34ff60 0x56468a3503a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009f70 tx=0x7f7fb4009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5f58daa808d49f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fb0ff9700 1 -- 172.21.15.26:0/1917925482 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4014420 con 0x56468a34ff60 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fb0ff9700 1 -- 172.21.15.26:0/1917925482 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fb4019070 con 0x56468a34ff60 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.114+0000 7f7fb0ff9700 1 -- 172.21.15.26:0/1917925482 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4013830 con 0x56468a34ff60 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1917925482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34ff60 msgr2=0x56468a3503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34ff60 0x56468a3503a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009f70 tx=0x7f7fb4009650 comp rx=0 tx=0).stop 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1917925482 shutdown_connections 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34ff60 0x56468a3503a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a34bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1917925482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34c210 0x56468a34f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1917925482 >> 172.21.15.26:0/1917925482 conn(0x56468a279080 msgr2=0x56468a345e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1917925482 shutdown_connections 2024-05-20T21:07:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.115+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1917925482 wait complete. 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 -- start start 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 Processor -- start 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 0x56468a357330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a354630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34ff60 0x56468a354b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56468a1a0fa0 con 0x56468a34b860 2024-05-20T21:07:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56468a23c7c0 con 0x56468a34c210 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56468a358120 con 0x56468a34ff60 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fceecdb80 10 client.0 _ll_register_callbacks cb 0x56468a235920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fc23d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 0x56468a357330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fc23d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 0x56468a357330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40916/0 (socket says 172.21.15.26:40916) 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.116+0000 7f7fc23d3700 1 -- 172.21.15.26:0/1093656704 learned_addr learned my addr 172.21.15.26:0/1093656704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a354630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34ff60 0x56468a354b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc23d3700 1 -- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34ff60 msgr2=0x56468a354b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34ff60 0x56468a354b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc23d3700 1 -- 172.21.15.26:0/1093656704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 msgr2=0x56468a354630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a354630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc23d3700 1 -- 172.21.15.26:0/1093656704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468a239690 con 0x56468a34b860 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34ff60 0x56468a354b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 0x56468a357330 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80df580 tx=0x7f7fb8010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24334107b1184b19 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1093656704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80a9050 con 0x56468a34b860 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1093656704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7fb80a5640 con 0x56468a34b860 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb4019070 con 0x56468a34b860 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1093656704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fb80ad400 con 0x56468a34b860 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fceecdb80 10 client.19044 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.117+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56468a362800 con 0x56468a34b860 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1093656704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fb4019070 con 0x56468a34b860 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1093656704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7fb80a82b0 con 0x56468a34b860 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fceecdb80 10 client.19044 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fceecdb80 10 client.19044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fceecdb80 20 client.19044 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a367180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fceecdb80 4 client.19044 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468a3179e0 con 0x56468a364d90 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.118+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a367180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.119+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a367180 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf7f75ff943ced9f server_cookie=ebdd386ee1d6e129 in_seq=0 out_seq=0 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.119+0000 7f7fa5ffb700 10 client.19044 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1093656704 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56468a3179e0 con 0x56468a364d90 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fa5ffb700 10 client.19044 handle_command_reply: tid=0 2024-05-20T21:07:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 client.19044 shutdown 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 msgr2=0x56468a357330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 0x56468a357330 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80df580 tx=0x7f7fb8010760 comp rx=0 tx=0).stop 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 shutdown_connections 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a367180 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fa5ffb700 0 client.19044 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34b860 0x56468a357330 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34ff60 0x56468a354b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1093656704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a354630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 >> 172.21.15.26:0/1093656704 conn(0x56468a279080 msgr2=0x56468a345510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 shutdown_connections 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1093656704 wait complete. 2024-05-20T21:07:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.120+0000 7f7fceecdb80 20 client.19044 trim_cache size 0 max 16384 2024-05-20T21:07:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.135+0000 7f7fceecdb80 1 Processor -- start 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 -- start start 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a356570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a345c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a36c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1b2890 con 0x56468a34ff60 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1b0120 con 0x56468a34c210 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1adf70 con 0x56468a34b860 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc2bd4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a36c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a345c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc2bd4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a36c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40924/0 (socket says 172.21.15.26:40924) 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a345c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46278/0 (socket says 172.21.15.26:46278) 2024-05-20T21:07:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1159010046 learned_addr learned my addr 172.21.15.26:0/1159010046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a356570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 -- 172.21.15.26:0/1159010046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 msgr2=0x56468a356570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a356570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 -- 172.21.15.26:0/1159010046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 msgr2=0x56468a36c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a36c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc1bd2700 1 -- 172.21.15.26:0/1159010046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468a34fc80 con 0x56468a34c210 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.136+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a36c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.137+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a345c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7fac005f50 tx=0x7f7fac01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c419be34dbc93250 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.137+0000 7f7fa7fff700 1 -- 172.21.15.26:0/1159010046 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00bb10 con 0x56468a34c210 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.137+0000 7f7fa7fff700 1 -- 172.21.15.26:0/1159010046 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fac00bcb0 con 0x56468a34c210 2024-05-20T21:07:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.137+0000 7f7fa7fff700 1 -- 172.21.15.26:0/1159010046 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac023660 con 0x56468a34c210 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.137+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1159010046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 msgr2=0x56468a345c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.137+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a345c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7fac005f50 tx=0x7f7fac01e000 comp rx=0 tx=0).stop 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1159010046 shutdown_connections 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a36c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34b860 0x56468a356570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1159010046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34c210 0x56468a345c90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1159010046 >> 172.21.15.26:0/1159010046 conn(0x56468a279080 msgr2=0x56468a355e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1159010046 shutdown_connections 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1159010046 wait complete. 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 -- start start 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 Processor -- start 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34b860 0x56468a35b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34c210 0x56468a35b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.138+0000 7f7fceecdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a35bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fceecdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ae5e0 con 0x56468a34ff60 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a35bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a35bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40932/0 (socket says 172.21.15.26:40932) 2024-05-20T21:07:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1983097515 learned_addr learned my addr 172.21.15.26:0/1983097515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1afa50 con 0x56468a34b860 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1aecb0 con 0x56468a34c210 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fceecdb80 10 client.0 _ll_register_callbacks cb 0x56468a235920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc1bd2700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34c210 0x56468a35b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34b860 0x56468a35b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34c210 msgr2=0x56468a35b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34c210 0x56468a35b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1983097515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34b860 msgr2=0x56468a35b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34b860 0x56468a35b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 -- 172.21.15.26:0/1983097515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468a3cea30 con 0x56468a34ff60 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34b860 0x56468a35b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fc2bd4700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a35bcd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4000f80 tx=0x7f7fb4014650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2214c7967c4b978 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1983097515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4014a70 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fac00bcb0 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.139+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1983097515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7fb4014c10 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1983097515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fb4014e10 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 10 client.19050 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56468a3cfd20 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1983097515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fac00bcb0 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1983097515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7fb4030020 con 0x56468a34ff60 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 10 client.19050 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 10 client.19050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 20 client.19050 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a3d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 4 client.19050 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.140+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56468a2acbe0 con 0x56468a364d90 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a3d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fc23d3700 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a3d2280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a791b91f01dd1909 server_cookie=df2ffe0612e285b in_seq=0 out_seq=0 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fa5ffb700 10 client.19050 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fa5ffb700 1 -- 172.21.15.26:0/1983097515 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56468a2acbe0 con 0x56468a364d90 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fa5ffb700 10 client.19050 handle_command_reply: tid=0 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 client.19050 shutdown 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 msgr2=0x56468a35bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a35bcd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4000f80 tx=0x7f7fb4014650 comp rx=0 tx=0).stop 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 shutdown_connections 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56468a364d90 0x56468a3d2280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56468a34ff60 0x56468a35bcd0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56468a34b860 0x56468a35b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 --2- 172.21.15.26:0/1983097515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56468a34c210 0x56468a35b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.141+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 >> 172.21.15.26:0/1983097515 conn(0x56468a279080 msgr2=0x56468a355e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fa5ffb700 0 client.19050 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 shutdown_connections 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fceecdb80 1 -- 172.21.15.26:0/1983097515 wait complete. 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fceecdb80 20 client.19050 trim_cache size 0 max 16384 2024-05-20T21:07:53.147 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fa80b2880 msgr2=0x7f7fa80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fa80b2880 0x7f7fa80b4d40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009750 tx=0x7f7fac01a040 comp rx=0 tx=0).stop 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 msgr2=0x7f7fbc19bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 0x7f7fbc19bee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800b530 tx=0x7f7fb800b840 comp rx=0 tx=0).stop 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 shutdown_connections 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7fa80b2880 0x7f7fa80b4d40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fbc104450 0x7f7fbc1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fbc101b30 0x7f7fbc19bee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 --2- 172.21.15.26:0/534492208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fbc1025d0 0x7f7fbc19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.142+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 >> 172.21.15.26:0/534492208 conn(0x7f7fbc078ab0 msgr2=0x7f7fbc0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.143+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 shutdown_connections 2024-05-20T21:07:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:53.143+0000 7f7fc4637700 1 -- 172.21.15.26:0/534492208 wait complete. 2024-05-20T21:07:53.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:53 smithi026 ceph-mon[31990]: pgmap v742: 97 pgs: 97 active+clean; 27 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2024-05-20T21:07:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:53 smithi144 ceph-mon[35660]: pgmap v742: 97 pgs: 97 active+clean; 27 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2024-05-20T21:07:53.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 269024 2.29 MB/sec execute 479 sec latency 15.633 ms 2024-05-20T21:07:53.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:53 smithi055 ceph-mon[35213]: pgmap v742: 97 pgs: 97 active+clean; 27 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2024-05-20T21:07:53.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:07:54.248 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:54.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 269637 2.29 MB/sec execute 480 sec latency 16.505 ms 2024-05-20T21:07:55.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:55 smithi026 ceph-mon[31990]: pgmap v743: 97 pgs: 97 active+clean; 27 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:07:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:55 smithi144 ceph-mon[35660]: pgmap v743: 97 pgs: 97 active+clean; 27 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:07:55.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 270068 2.29 MB/sec execute 481 sec latency 62.142 ms 2024-05-20T21:07:55.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:55 smithi055 ceph-mon[35213]: pgmap v743: 97 pgs: 97 active+clean; 27 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-05-20T21:07:56.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 -- 172.21.15.26:0/1329579024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 msgr2=0x7f523c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 --2- 172.21.15.26:0/1329579024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c1028c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5230009990 tx=0x7f5230009ca0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 -- 172.21.15.26:0/1329579024 shutdown_connections 2024-05-20T21:07:56.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 --2- 172.21.15.26:0/1329579024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c102e00 0x7f523c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 --2- 172.21.15.26:0/1329579024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c108520 0x7f523c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 --2- 172.21.15.26:0/1329579024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c1028c0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 -- 172.21.15.26:0/1329579024 >> 172.21.15.26:0/1329579024 conn(0x7f523c0793f0 msgr2=0x7f523c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 -- 172.21.15.26:0/1329579024 shutdown_connections 2024-05-20T21:07:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.310+0000 7f5240d13700 1 -- 172.21.15.26:0/1329579024 wait complete. 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 Processor -- start 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 -- start start 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c102e00 0x7f523c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c108520 0x7f523c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1124a0 con 0x7f523c102440 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f523c112320 con 0x7f523c102e00 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5240d13700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f523c112620 con 0x7f523c108520 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f523a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f523a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40946/0 (socket says 172.21.15.26:40946) 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f523a59c700 1 -- 172.21.15.26:0/1077219193 learned_addr learned my addr 172.21.15.26:0/1077219193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.311+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c102e00 0x7f523c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f523ad9d700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c108520 0x7f523c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5239d9b700 1 -- 172.21.15.26:0/1077219193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c108520 msgr2=0x7f523c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c108520 0x7f523c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5239d9b700 1 -- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 msgr2=0x7f523c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5239d9b700 1 -- 172.21.15.26:0/1077219193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c1ac4c0 con 0x7f523c102e00 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f523ad9d700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c108520 0x7f523c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f523a59c700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c102e00 0x7f523c1045c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f52300047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f522b7fe700 1 -- 172.21.15.26:0/1077219193 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230026070 con 0x7f523c102e00 2024-05-20T21:07:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c1ac6f0 con 0x7f523c102e00 2024-05-20T21:07:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f522b7fe700 1 -- 172.21.15.26:0/1077219193 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5230021410 con 0x7f523c102e00 2024-05-20T21:07:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f522b7fe700 1 -- 172.21.15.26:0/1077219193 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230017650 con 0x7f523c102e00 2024-05-20T21:07:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.312+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1acbf0 con 0x7f523c102e00 2024-05-20T21:07:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.315+0000 7f522b7fe700 1 -- 172.21.15.26:0/1077219193 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5230029070 con 0x7f523c102e00 2024-05-20T21:07:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.316+0000 7f522b7fe700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52240b2880 0x7f52240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.316+0000 7f523a59c700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52240b2880 0x7f52240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.316+0000 7f522b7fe700 1 -- 172.21.15.26:0/1077219193 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f52300e3080 con 0x7f523c102e00 2024-05-20T21:07:56.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.316+0000 7f523a59c700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52240b2880 0x7f52240b4d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f522c009680 tx=0x7f522c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:56.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.316+0000 7f524b5a9b80 1 Processor -- start 2024-05-20T21:07:56.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 -- start start 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x557917391700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x55791738bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x557917398080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5579172e9d70 con 0x55791738c260 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5579172d9910 con 0x557917391320 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579172dd950 con 0x55791738b8a0 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x557917391700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x557917398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x557917391700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46310/0 (socket says 172.21.15.26:46310) 2024-05-20T21:07:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 -- 172.21.15.26:0/2666702171 learned_addr learned my addr 172.21.15.26:0/2666702171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f5239d9b700 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x55791738bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 -- 172.21.15.26:0/2666702171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 msgr2=0x55791738bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x55791738bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 -- 172.21.15.26:0/2666702171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 msgr2=0x557917398080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x557917398080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.317+0000 7f523a59c700 1 -- 172.21.15.26:0/2666702171 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557917282fb0 con 0x557917391320 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.318+0000 7f523a59c700 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x557917391700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f522c00f7c0 tx=0x7f522c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a3bb8c004fa0f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.318+0000 7f5228ff9700 1 -- 172.21.15.26:0/2666702171 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c01e420 con 0x557917391320 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.318+0000 7f5228ff9700 1 -- 172.21.15.26:0/2666702171 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522c011e50 con 0x557917391320 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.318+0000 7f5228ff9700 1 -- 172.21.15.26:0/2666702171 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0267c0 con 0x557917391320 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 -- 172.21.15.26:0/2666702171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 msgr2=0x557917391700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x557917391700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f522c00f7c0 tx=0x7f522c014000 comp rx=0 tx=0).stop 2024-05-20T21:07:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 -- 172.21.15.26:0/2666702171 shutdown_connections 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x557917398080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x557917391700 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/2666702171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x55791738bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 -- 172.21.15.26:0/2666702171 >> 172.21.15.26:0/2666702171 conn(0x5579172b7240 msgr2=0x5579173863e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 -- 172.21.15.26:0/2666702171 shutdown_connections 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 -- 172.21.15.26:0/2666702171 wait complete. 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 -- start start 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.319+0000 7f524b5a9b80 1 Processor -- start 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 0x557917395030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557917391320 0x557917395570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5579171dffa0 con 0x557917391320 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557917241da0 con 0x55791738c260 2024-05-20T21:07:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579173986e0 con 0x55791738b8a0 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f524b5a9b80 10 client.0 _ll_register_callbacks cb 0x557917274950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 0x557917395030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 0x557917395030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46326/0 (socket says 172.21.15.26:46326) 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 -- 172.21.15.26:0/3972487468 learned_addr learned my addr 172.21.15.26:0/3972487468 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557917391320 0x557917395570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f523a59c700 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 -- 172.21.15.26:0/3972487468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 msgr2=0x557917397d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 -- 172.21.15.26:0/3972487468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557917391320 msgr2=0x557917395570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557917391320 0x557917395570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 -- 172.21.15.26:0/3972487468 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579172298d0 con 0x55791738c260 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.320+0000 7f5239d9b700 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 0x557917395030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f52300a7940 tx=0x7f52300aa0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b73f8af87c9781b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:56.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f5221ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f5221ffb700 1 -- 172.21.15.26:0/3972487468 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300aadd0 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f5221ffb700 1 -- 172.21.15.26:0/3972487468 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f52300920b0 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c011e50 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f5221ffb700 1 -- 172.21.15.26:0/3972487468 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52300922b0 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 10 client.28358 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557917293760 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f5221ffb700 1 -- 172.21.15.26:0/3972487468 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522c011e50 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f5221ffb700 1 -- 172.21.15.26:0/3972487468 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f52300b3010 con 0x55791738c260 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 10 client.28358 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 10 client.28358 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 20 client.28358 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.321+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5579173a53a0 0x5579173a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.322+0000 7f524b5a9b80 4 client.28358 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:56.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.322+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579172ebcc0 con 0x5579173a53a0 2024-05-20T21:07:56.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.322+0000 7f523a59c700 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5579173a53a0 0x5579173a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.322+0000 7f523a59c700 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5579173a53a0 0x5579173a77c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d3cbfbfc9af6f9d server_cookie=2ce1a2326f396b6f in_seq=0 out_seq=0 2024-05-20T21:07:56.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.322+0000 7f5221ffb700 10 client.28358 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f5221ffb700 1 -- 172.21.15.26:0/3972487468 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5579172ebcc0 con 0x5579173a53a0 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f5221ffb700 10 client.28358 handle_command_reply: tid=0 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 client.28358 shutdown 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 msgr2=0x557917395030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 0x557917395030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f52300a7940 tx=0x7f52300aa0a0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 shutdown_connections 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x5579173a53a0 0x5579173a77c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557917391320 0x557917395570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f5221ffb700 0 client.28358 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/3972487468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738c260 0x557917395030 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 >> 172.21.15.26:0/3972487468 conn(0x5579172b7240 msgr2=0x557917385af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 shutdown_connections 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 1 -- 172.21.15.26:0/3972487468 wait complete. 2024-05-20T21:07:56.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.323+0000 7f524b5a9b80 20 client.28358 trim_cache size 0 max 16384 2024-05-20T21:07:56.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.339+0000 7f524b5a9b80 1 Processor -- start 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.339+0000 7f524b5a9b80 1 -- start start 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x5579173937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557917391320 0x557917393d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b6ad0 con 0x55791738c260 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b4360 con 0x55791738b8a0 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b21b0 con 0x557917391320 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x5579173937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x5579173937f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40970/0 (socket says 172.21.15.26:40970) 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 -- 172.21.15.26:0/4047221758 learned_addr learned my addr 172.21.15.26:0/4047221758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f523a59c700 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f523ad9d700 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557917391320 0x557917393d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 -- 172.21.15.26:0/4047221758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557917391320 msgr2=0x557917393d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557917391320 0x557917393d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 -- 172.21.15.26:0/4047221758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738b8a0 msgr2=0x557917397d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 -- 172.21.15.26:0/4047221758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557917390c50 con 0x55791738c260 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f523a59c700 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.340+0000 7f5239d9b700 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x5579173937f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f52300213e0 tx=0x7f5230009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d7d4d0f64b2395 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f5223fff700 1 -- 172.21.15.26:0/4047221758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000fe60 con 0x55791738c260 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f5223fff700 1 -- 172.21.15.26:0/4047221758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52300c8020 con 0x55791738c260 2024-05-20T21:07:56.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f5223fff700 1 -- 172.21.15.26:0/4047221758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230092070 con 0x55791738c260 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 -- 172.21.15.26:0/4047221758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 msgr2=0x5579173937f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x5579173937f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f52300213e0 tx=0x7f5230009310 comp rx=0 tx=0).stop 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 -- 172.21.15.26:0/4047221758 shutdown_connections 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557917391320 0x557917393d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55791738b8a0 0x557917397d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/4047221758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738c260 0x5579173937f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 -- 172.21.15.26:0/4047221758 >> 172.21.15.26:0/4047221758 conn(0x5579172b7240 msgr2=0x5579173970f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.341+0000 7f524b5a9b80 1 -- 172.21.15.26:0/4047221758 shutdown_connections 2024-05-20T21:07:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 -- 172.21.15.26:0/4047221758 wait complete. 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 -- start start 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 Processor -- start 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 0x55791738e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738c260 0x55791738e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x55791739c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b2820 con 0x55791738b8a0 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b3c90 con 0x557917391320 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b2ef0 con 0x55791738c260 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f524b5a9b80 10 client.0 _ll_register_callbacks cb 0x557917274950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f523a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 0x55791738e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f523a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 0x55791738e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40976/0 (socket says 172.21.15.26:40976) 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.342+0000 7f523a59c700 1 -- 172.21.15.26:0/1019436735 learned_addr learned my addr 172.21.15.26:0/1019436735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523ad9d700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x55791739c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738c260 0x55791738e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523a59c700 1 -- 172.21.15.26:0/1019436735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738c260 msgr2=0x55791738e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523a59c700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738c260 0x55791738e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523a59c700 1 -- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 msgr2=0x55791739c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523a59c700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x55791739c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523a59c700 1 -- 172.21.15.26:0/1019436735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791739cf70 con 0x55791738b8a0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f523a59c700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 0x55791738e3f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f522c00fdb0 tx=0x7f522c01ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e6c74204b02958 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f5221ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f5221ffb700 1 -- 172.21.15.26:0/1019436735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c035020 con 0x55791738b8a0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f5221ffb700 1 -- 172.21.15.26:0/1019436735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f522c0052d0 con 0x55791738b8a0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52300c8020 con 0x55791738b8a0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f5221ffb700 1 -- 172.21.15.26:0/1019436735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f522c040050 con 0x55791738b8a0 2024-05-20T21:07:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f524b5a9b80 10 client.19068 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55791739e270 con 0x55791738b8a0 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.343+0000 7f5221ffb700 1 -- 172.21.15.26:0/1019436735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52300c8020 con 0x55791738b8a0 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f5221ffb700 1 -- 172.21.15.26:0/1019436735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f522c01e420 con 0x55791738b8a0 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f524b5a9b80 10 client.19068 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f524b5a9b80 10 client.19068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f524b5a9b80 20 client.19068 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55791740fd80 0x557917412170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f524b5a9b80 4 client.19068 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579172e54a0 con 0x55791740fd80 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55791740fd80 0x557917412170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f5221ffb700 10 client.19068 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.344+0000 7f5239d9b700 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55791740fd80 0x557917412170 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e2853715a5b93e5 server_cookie=a74bfb85bdaebde2 in_seq=0 out_seq=0 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5221ffb700 1 -- 172.21.15.26:0/1019436735 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579172e54a0 con 0x55791740fd80 2024-05-20T21:07:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5221ffb700 10 client.19068 handle_command_reply: tid=0 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 client.19068 shutdown 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 msgr2=0x55791738e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 0x55791738e3f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f522c00fdb0 tx=0x7f522c01ef50 comp rx=0 tx=0).stop 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 shutdown_connections 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55791740fd80 0x557917412170 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55791738b8a0 0x55791738e3f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557917391320 0x55791739c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 --2- 172.21.15.26:0/1019436735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55791738c260 0x55791738e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5221ffb700 0 client.19068 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 >> 172.21.15.26:0/1019436735 conn(0x5579172b7240 msgr2=0x55791738a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 shutdown_connections 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 1 -- 172.21.15.26:0/1019436735 wait complete. 2024-05-20T21:07:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f524b5a9b80 20 client.19068 trim_cache size 0 max 16384 2024-05-20T21:07:56.351 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:56.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52240b2880 msgr2=0x7f52240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5240d13700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52240b2880 0x7f52240b4d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f522c009680 tx=0x7f522c009260 comp rx=0 tx=0).stop 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c102e00 msgr2=0x7f523c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.345+0000 7f5240d13700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c102e00 0x7f523c1045c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f52300047b0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 shutdown_connections 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f52240b2880 0x7f52240b4d40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c102440 0x7f523c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f523c108520 0x7f523c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 --2- 172.21.15.26:0/1077219193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c102e00 0x7f523c1045c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 >> 172.21.15.26:0/1077219193 conn(0x7f523c0793f0 msgr2=0x7f523c06b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 shutdown_connections 2024-05-20T21:07:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:56.346+0000 7f5240d13700 1 -- 172.21.15.26:0/1077219193 wait complete. 2024-05-20T21:07:56.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 270639 2.29 MB/sec execute 482 sec latency 26.238 ms 2024-05-20T21:07:57.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:07:57.447 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:07:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:57 smithi026 ceph-mon[31990]: pgmap v744: 97 pgs: 97 active+clean; 27 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T21:07:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:57 smithi144 ceph-mon[35660]: pgmap v744: 97 pgs: 97 active+clean; 27 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T21:07:57.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 271309 2.29 MB/sec execute 483 sec latency 15.030 ms 2024-05-20T21:07:57.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:57 smithi055 ceph-mon[35213]: pgmap v744: 97 pgs: 97 active+clean; 27 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-05-20T21:07:58.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 271992 2.29 MB/sec execute 484 sec latency 22.095 ms 2024-05-20T21:07:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 -- 172.21.15.26:0/487586291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 msgr2=0x7f869410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 --2- 172.21.15.26:0/487586291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f869410f2f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b0a0 tx=0x7f868c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 -- 172.21.15.26:0/487586291 shutdown_connections 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 --2- 172.21.15.26:0/487586291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f869410f2f0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 --2- 172.21.15.26:0/487586291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694108520 0x7f8694108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 --2- 172.21.15.26:0/487586291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694102440 0x7f86941028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 -- 172.21.15.26:0/487586291 >> 172.21.15.26:0/487586291 conn(0x7f86940793f0 msgr2=0x7f8694079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 -- 172.21.15.26:0/487586291 shutdown_connections 2024-05-20T21:07:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.342+0000 7f869a7e9700 1 -- 172.21.15.26:0/487586291 wait complete. 2024-05-20T21:07:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 Processor -- start 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 -- start start 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694102440 0x7f8694198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f8694198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694108520 0x7f86941991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8694112480 con 0x7f8694102e00 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8694112300 con 0x7f8694108520 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f869a7e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8694112600 con 0x7f8694102440 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.343+0000 7f868bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f8694198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694108520 0x7f86941991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f868bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f8694198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40996/0 (socket says 172.21.15.26:40996) 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f868bfff700 1 -- 172.21.15.26:0/1617765814 learned_addr learned my addr 172.21.15.26:0/1617765814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8693fff700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694102440 0x7f8694198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 -- 172.21.15.26:0/1617765814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694102440 msgr2=0x7f8694198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694102440 0x7f8694198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 -- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 msgr2=0x7f8694198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f8694198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 -- 172.21.15.26:0/1617765814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86941a7fa0 con 0x7f8694108520 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8693fff700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694102440 0x7f8694198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8698d86700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694108520 0x7f86941991b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b0a0 tx=0x7f868c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8691ffb700 1 -- 172.21.15.26:0/1617765814 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c021070 con 0x7f8694108520 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8691ffb700 1 -- 172.21.15.26:0/1617765814 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f868c007b80 con 0x7f8694108520 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86941a8230 con 0x7f8694108520 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f8691ffb700 1 -- 172.21.15.26:0/1617765814 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c005040 con 0x7f8694108520 2024-05-20T21:07:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.344+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941a8680 con 0x7f8694108520 2024-05-20T21:07:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.347+0000 7f8691ffb700 1 -- 172.21.15.26:0/1617765814 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f868c024070 con 0x7f8694108520 2024-05-20T21:07:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.348+0000 7f8691ffb700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86800b2880 0x7f86800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.348+0000 7f8693fff700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86800b2880 0x7f86800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.348+0000 7f8691ffb700 1 -- 172.21.15.26:0/1617765814 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f868c00f020 con 0x7f8694108520 2024-05-20T21:07:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.348+0000 7f86a507fb80 1 Processor -- start 2024-05-20T21:07:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 -- start start 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86800b2880 0x7f86800b4d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f868400bb90 tx=0x7f8684005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 0x55c6f2649e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f26440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f26446d0 0x55c6f2644b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c6f2599d70 con 0x55c6f2649a70 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c6f2589910 con 0x55c6f26446d0 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f86a507fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c6f258d950 con 0x55c6f264a420 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 0x55c6f2649e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 0x55c6f2649e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41008/0 (socket says 172.21.15.26:41008) 2024-05-20T21:07:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 -- 172.21.15.26:0/2548286826 learned_addr learned my addr 172.21.15.26:0/2548286826 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8698d86700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f26446d0 0x55c6f2644b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f868bfff700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f26440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 -- 172.21.15.26:0/2548286826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 msgr2=0x55c6f26440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f26440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 -- 172.21.15.26:0/2548286826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f26446d0 msgr2=0x55c6f2644b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f26446d0 0x55c6f2644b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8693fff700 1 -- 172.21.15.26:0/2548286826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f2532fb0 con 0x55c6f2649a70 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f868bfff700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f26440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.349+0000 7f8698d86700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f26446d0 0x55c6f2644b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.350+0000 7f8693fff700 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 0x55c6f2649e50 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f868400b970 tx=0x7f8684012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aed4bcc3662f27b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.350+0000 7f868a7fc700 1 -- 172.21.15.26:0/2548286826 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684022070 con 0x55c6f2649a70 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.350+0000 7f868a7fc700 1 -- 172.21.15.26:0/2548286826 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f868401e450 con 0x55c6f2649a70 2024-05-20T21:07:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.350+0000 7f868a7fc700 1 -- 172.21.15.26:0/2548286826 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868401d610 con 0x55c6f2649a70 2024-05-20T21:07:59.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.350+0000 7f86a507fb80 1 -- 172.21.15.26:0/2548286826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 msgr2=0x55c6f2649e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.350+0000 7f86a507fb80 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 0x55c6f2649e50 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f868400b970 tx=0x7f8684012000 comp rx=0 tx=0).stop 2024-05-20T21:07:59.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 -- 172.21.15.26:0/2548286826 shutdown_connections 2024-05-20T21:07:59.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f2649a70 0x55c6f2649e50 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f26446d0 0x55c6f2644b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 --2- 172.21.15.26:0/2548286826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f26440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 -- 172.21.15.26:0/2548286826 >> 172.21.15.26:0/2548286826 conn(0x55c6f2567240 msgr2=0x55c6f26363e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 -- 172.21.15.26:0/2548286826 shutdown_connections 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 -- 172.21.15.26:0/2548286826 wait complete. 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 -- start start 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 Processor -- start 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2650290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.351+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f264dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86a507fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c6f248ffa0 con 0x55c6f26446d0 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86a507fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c6f24f1da0 con 0x55c6f264a420 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86a507fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c6f2650ea0 con 0x55c6f2649a70 2024-05-20T21:07:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86a507fb80 10 client.0 _ll_register_callbacks cb 0x55c6f2524950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2650290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2650290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41020/0 (socket says 172.21.15.26:41020) 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 -- 172.21.15.26:0/4066110044 learned_addr learned my addr 172.21.15.26:0/4066110044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f868bfff700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8698d86700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f264dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 -- 172.21.15.26:0/4066110044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 msgr2=0x55c6f264d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 -- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 msgr2=0x55c6f264dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f264dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 -- 172.21.15.26:0/4066110044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f24d98d0 con 0x55c6f26446d0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f8693fff700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2650290 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f868401e840 tx=0x7f868401e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb0d96eaa64c14c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86737fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:07:59.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86737fe700 1 -- 172.21.15.26:0/4066110044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684036060 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f868401e450 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.352+0000 7f86737fe700 1 -- 172.21.15.26:0/4066110044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8684011ae0 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86737fe700 1 -- 172.21.15.26:0/4066110044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8684022050 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 10 client.19080 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6f2543760 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86737fe700 1 -- 172.21.15.26:0/4066110044 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f868401e450 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86737fe700 1 -- 172.21.15.26:0/4066110044 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f868403a030 con 0x55c6f26446d0 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 10 client.19080 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 10 client.19080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 20 client.19080 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f265fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 4 client.19080 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:07:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f259bcc0 con 0x55c6f265d950 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.353+0000 7f868bfff700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f265fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.354+0000 7f86737fe700 10 client.19080 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.354+0000 7f868bfff700 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f265fd40 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5bd6fcef930152 server_cookie=a11e273300c92033 in_seq=0 out_seq=0 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.354+0000 7f86737fe700 1 -- 172.21.15.26:0/4066110044 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c6f259bcc0 con 0x55c6f265d950 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.354+0000 7f86737fe700 10 client.19080 handle_command_reply: tid=0 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.354+0000 7f86a507fb80 1 client.19080 shutdown 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 msgr2=0x55c6f2650290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2650290 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f868401e840 tx=0x7f868401e870 comp rx=0 tx=0).stop 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 shutdown_connections 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f265fd40 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2650290 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f264dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 --2- 172.21.15.26:0/4066110044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86737fe700 0 client.19080 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 >> 172.21.15.26:0/4066110044 conn(0x55c6f2567240 msgr2=0x55c6f2635af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 shutdown_connections 2024-05-20T21:07:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 1 -- 172.21.15.26:0/4066110044 wait complete. 2024-05-20T21:07:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.355+0000 7f86a507fb80 20 client.19080 trim_cache size 0 max 16384 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 Processor -- start 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 -- start start 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f264af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f2636270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86941ae330 con 0x55c6f26446d0 2024-05-20T21:07:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86941abbc0 con 0x55c6f264a420 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f86a507fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f869419ebb0 con 0x55c6f2649a70 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.371+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f264af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f264af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41030/0 (socket says 172.21.15.26:41030) 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 -- 172.21.15.26:0/3313569209 learned_addr learned my addr 172.21.15.26:0/3313569209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8698d86700 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f2636270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f868bfff700 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 -- 172.21.15.26:0/3313569209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 msgr2=0x55c6f264c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 -- 172.21.15.26:0/3313569209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 msgr2=0x55c6f2636270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f2636270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 -- 172.21.15.26:0/3313569209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f264fdd0 con 0x55c6f26446d0 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8693fff700 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f264af20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8684011720 tx=0x7f8684005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31fe0be773b2715 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8689ffb700 1 -- 172.21.15.26:0/3313569209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684015040 con 0x55c6f26446d0 2024-05-20T21:07:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8689ffb700 1 -- 172.21.15.26:0/3313569209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86840103d0 con 0x55c6f26446d0 2024-05-20T21:07:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.372+0000 7f8689ffb700 1 -- 172.21.15.26:0/3313569209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684036040 con 0x55c6f26446d0 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 -- 172.21.15.26:0/3313569209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 msgr2=0x55c6f264af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f264af20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8684011720 tx=0x7f8684005e50 comp rx=0 tx=0).stop 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 -- 172.21.15.26:0/3313569209 shutdown_connections 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f264af20 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f264a420 0x55c6f2636270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 --2- 172.21.15.26:0/3313569209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f2649a70 0x55c6f264c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 -- 172.21.15.26:0/3313569209 >> 172.21.15.26:0/3313569209 conn(0x55c6f2567240 msgr2=0x55c6f2632760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 -- 172.21.15.26:0/3313569209 shutdown_connections 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 -- 172.21.15.26:0/3313569209 wait complete. 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.373+0000 7f86a507fb80 1 -- start start 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 Processor -- start 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2662cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f2649a70 0x55c6f26631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f2660340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86941aa080 con 0x55c6f26446d0 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86941ab4f0 con 0x55c6f2649a70 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86941aa750 con 0x55c6f264a420 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f86a507fb80 10 client.0 _ll_register_callbacks cb 0x55c6f2524950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2662cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f8693fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2662cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41038/0 (socket says 172.21.15.26:41038) 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f8698d86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f2660340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f8693fff700 1 -- 172.21.15.26:0/1272420917 learned_addr learned my addr 172.21.15.26:0/1272420917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.374+0000 7f868bfff700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f2649a70 0x55c6f26631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8698d86700 1 -- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f2649a70 msgr2=0x55c6f26631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8698d86700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f2649a70 0x55c6f26631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8698d86700 1 -- 172.21.15.26:0/1272420917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 msgr2=0x55c6f2662cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8698d86700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2662cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8698d86700 1 -- 172.21.15.26:0/1272420917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f2660a40 con 0x55c6f264a420 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8693fff700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2662cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f8698d86700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f2660340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f868c006130 tx=0x7f868c0060a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34f5f6292c452cd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86737fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86737fe700 1 -- 172.21.15.26:0/1272420917 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0c53a0 con 0x55c6f264a420 2024-05-20T21:07:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86840103d0 con 0x55c6f264a420 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86737fe700 1 -- 172.21.15.26:0/1272420917 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f868c0aafb0 con 0x55c6f264a420 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86737fe700 1 -- 172.21.15.26:0/1272420917 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f868c0af100 con 0x55c6f264a420 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86a507fb80 10 client.28153 fetch_fsmap learned FSMap version 30 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.375+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6f26c7cd0 con 0x55c6f264a420 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86737fe700 1 -- 172.21.15.26:0/1272420917 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86840103d0 con 0x55c6f264a420 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86737fe700 1 -- 172.21.15.26:0/1272420917 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f868c011070 con 0x55c6f264a420 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86a507fb80 10 client.28153 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86a507fb80 10 client.28153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86a507fb80 20 client.28153 populate_metadata read hostname 'smithi026' 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86a507fb80 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f26ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86a507fb80 4 client.28153 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6f25954a0 con 0x55c6f265d950 2024-05-20T21:07:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f8693fff700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f26ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f8693fff700 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f26ca3f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8493ee6a05c57c2d server_cookie=76f701b349471aa9 in_seq=0 out_seq=0 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.376+0000 7f86737fe700 10 client.28153 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86737fe700 1 -- 172.21.15.26:0/1272420917 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6f25954a0 con 0x55c6f265d950 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86737fe700 10 client.28153 handle_command_reply: tid=0 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 client.28153 shutdown 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 msgr2=0x55c6f2660340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f2660340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f868c006130 tx=0x7f868c0060a0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 shutdown_connections 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c6f265d950 0x55c6f26ca3f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6f26446d0 0x55c6f2662cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6f264a420 0x55c6f2660340 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86737fe700 0 client.28153 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:07:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 --2- 172.21.15.26:0/1272420917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6f2649a70 0x55c6f26631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 >> 172.21.15.26:0/1272420917 conn(0x55c6f2567240 msgr2=0x55c6f263a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 shutdown_connections 2024-05-20T21:07:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 1 -- 172.21.15.26:0/1272420917 wait complete. 2024-05-20T21:07:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.377+0000 7f86a507fb80 20 client.28153 trim_cache size 0 max 16384 2024-05-20T21:07:59.382 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86800b2880 msgr2=0x7f86800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86800b2880 0x7f86800b4d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f868400bb90 tx=0x7f8684005d80 comp rx=0 tx=0).stop 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694108520 msgr2=0x7f86941991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694108520 0x7f86941991b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b0a0 tx=0x7f868c004d00 comp rx=0 tx=0).stop 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 shutdown_connections 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8694108520 0x7f86941991b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8694102440 0x7f8694198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f86800b2880 0x7f86800b4d40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 --2- 172.21.15.26:0/1617765814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694102e00 0x7f8694198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 >> 172.21.15.26:0/1617765814 conn(0x7f86940793f0 msgr2=0x7f8694100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 shutdown_connections 2024-05-20T21:07:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:07:59.378+0000 7f869a7e9700 1 -- 172.21.15.26:0/1617765814 wait complete. 2024-05-20T21:07:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:07:59 smithi144 ceph-mon[35660]: pgmap v745: 97 pgs: 97 active+clean; 27 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-05-20T21:07:59.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 272891 2.30 MB/sec execute 485 sec latency 12.638 ms 2024-05-20T21:07:59.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:07:59 smithi055 ceph-mon[35213]: pgmap v745: 97 pgs: 97 active+clean; 27 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-05-20T21:07:59.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:07:59 smithi026 ceph-mon[31990]: pgmap v745: 97 pgs: 97 active+clean; 27 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-05-20T21:08:00.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 273423 2.31 MB/sec execute 486 sec latency 15.422 ms 2024-05-20T21:08:01.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:08:01.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:08:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:01.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:01 smithi026 ceph-mon[31990]: pgmap v746: 97 pgs: 97 active+clean; 28 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:08:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:01 smithi144 ceph-mon[35660]: pgmap v746: 97 pgs: 97 active+clean; 28 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:08:01.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 273819 2.30 MB/sec execute 487 sec latency 21.611 ms 2024-05-20T21:08:01.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:01 smithi055 ceph-mon[35213]: pgmap v746: 97 pgs: 97 active+clean; 28 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-05-20T21:08:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 -- 172.21.15.26:0/731883876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 msgr2=0x7f0470105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 --2- 172.21.15.26:0/731883876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470105920 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0460009900 tx=0x7f046002a410 comp rx=0 tx=0).stop 2024-05-20T21:08:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 -- 172.21.15.26:0/731883876 shutdown_connections 2024-05-20T21:08:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 --2- 172.21.15.26:0/731883876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470108790 0x7f047010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 --2- 172.21.15.26:0/731883876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470105e60 0x7f0470108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 --2- 172.21.15.26:0/731883876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470105920 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 -- 172.21.15.26:0/731883876 >> 172.21.15.26:0/731883876 conn(0x7f04700fcf20 msgr2=0x7f04700ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 -- 172.21.15.26:0/731883876 shutdown_connections 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 -- 172.21.15.26:0/731883876 wait complete. 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 Processor -- start 2024-05-20T21:08:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.376+0000 7f047652b640 1 -- start start 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f047652b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f047652b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470105e60 0x7f0470199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f047652b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470108790 0x7f04701a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f047652b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f047010e490 con 0x7f0470103530 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f047652b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f047010e310 con 0x7f0470108790 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f047652b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f047010e610 con 0x7f0470105e60 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38712/0 (socket says 172.21.15.26:38712) 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 -- 172.21.15.26:0/383807789 learned_addr learned my addr 172.21.15.26:0/383807789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f0474ac8640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470108790 0x7f04701a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046f7fe640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470105e60 0x7f0470199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 -- 172.21.15.26:0/383807789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470105e60 msgr2=0x7f0470199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470105e60 0x7f0470199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 -- 172.21.15.26:0/383807789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470108790 msgr2=0x7f04701a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470108790 0x7f04701a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046ffff640 1 -- 172.21.15.26:0/383807789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04701a0760 con 0x7f0470103530 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f0474ac8640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470108790 0x7f04701a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.377+0000 7f046f7fe640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470105e60 0x7f0470199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.378+0000 7f046ffff640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470198b40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0460009230 tx=0x7f04600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.378+0000 7f046d7fa640 1 -- 172.21.15.26:0/383807789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046003f030 con 0x7f0470103530 2024-05-20T21:08:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.378+0000 7f046d7fa640 1 -- 172.21.15.26:0/383807789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0460032e10 con 0x7f0470103530 2024-05-20T21:08:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.378+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04701a09f0 con 0x7f0470103530 2024-05-20T21:08:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.378+0000 7f046d7fa640 1 -- 172.21.15.26:0/383807789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460035070 con 0x7f0470103530 2024-05-20T21:08:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.378+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04701a0ee0 con 0x7f0470103530 2024-05-20T21:08:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 Processor -- start 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 -- start start 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f046d7fa640 1 -- 172.21.15.26:0/383807789 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0460042030 con 0x7f0470103530 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe76c0 0x555f19fe7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19fecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19fef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555f19f58c30 con 0x555f19fe80b0 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555f19f487d0 con 0x555f19fed220 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.381+0000 7f047924a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f19f4c810 con 0x555f19fe76c0 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19fecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19fecbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38714/0 (socket says 172.21.15.26:38714) 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046ffff640 1 -- 172.21.15.26:0/1548182668 learned_addr learned my addr 172.21.15.26:0/1548182668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19fef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f0474ac8640 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe76c0 0x555f19fe7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 -- 172.21.15.26:0/1548182668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe76c0 msgr2=0x555f19fe7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe76c0 0x555f19fe7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 -- 172.21.15.26:0/1548182668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 msgr2=0x555f19fecbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19fecbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 -- 172.21.15.26:0/1548182668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f19c5d210 con 0x555f19fed220 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19fef6b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f047006ac60 tx=0x7f0464010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cac755fea1f71b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f046d7fa640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04580b28a0 0x7f04580b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f04567fc640 1 -- 172.21.15.26:0/1548182668 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464004030 con 0x555f19fed220 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.382+0000 7f04567fc640 1 -- 172.21.15.26:0/1548182668 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04640035f0 con 0x555f19fed220 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f04567fc640 1 -- 172.21.15.26:0/1548182668 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04640126b0 con 0x555f19fed220 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f0474ac8640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04580b28a0 0x7f04580b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f046d7fa640 1 -- 172.21.15.26:0/383807789 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f04601028e0 con 0x7f0470103530 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f0474ac8640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04580b28a0 0x7f04580b4d60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x555f19fe8910 tx=0x7f0468009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 -- 172.21.15.26:0/1548182668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 msgr2=0x555f19fef6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19fef6b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f047006ac60 tx=0x7f0464010740 comp rx=0 tx=0).stop 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 -- 172.21.15.26:0/1548182668 shutdown_connections 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19fef6b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19fecbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 --2- 172.21.15.26:0/1548182668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe76c0 0x555f19fe7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 -- 172.21.15.26:0/1548182668 >> 172.21.15.26:0/1548182668 conn(0x555f19fe2d30 msgr2=0x555f19fe3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.383+0000 7f047924a740 1 -- 172.21.15.26:0/1548182668 shutdown_connections 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 -- 172.21.15.26:0/1548182668 wait complete. 2024-05-20T21:08:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 -- start start 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 Processor -- start 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19ff1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 0x555f19ff8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555f19c2e790 con 0x555f19fed220 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555f19f04890 con 0x555f19fe76c0 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.384+0000 7f047924a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f19ff01d0 con 0x555f19fe80b0 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f047924a740 10 client.0 _ll_register_callbacks cb 0x555f19c5ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 0x555f19ff8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 0x555f19ff8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38718/0 (socket says 172.21.15.26:38718) 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 -- 172.21.15.26:0/3838314708 learned_addr learned my addr 172.21.15.26:0/3838314708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f046f7fe640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19ff1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f046ffff640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 -- 172.21.15.26:0/3838314708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 msgr2=0x555f19ff1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19ff1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 -- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 msgr2=0x555f19ff1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 -- 172.21.15.26:0/3838314708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f19d29620 con 0x555f19fed220 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f046f7fe640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19ff1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f0474ac8640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 0x555f19ff8da0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f04680127c0 tx=0x7f0468004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef50cc6905f898f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f044f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f044f7fe640 1 -- 172.21.15.26:0/3838314708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468011ba0 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f044f7fe640 1 -- 172.21.15.26:0/3838314708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f0468011d40 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.385+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04640035f0 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f044f7fe640 1 -- 172.21.15.26:0/3838314708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0468029440 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 10 client.19098 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f19ffbe50 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f044f7fe640 1 -- 172.21.15.26:0/3838314708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04640035f0 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f044f7fe640 1 -- 172.21.15.26:0/3838314708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f0468031020 con 0x555f19fed220 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 10 client.19098 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 10 client.19098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 20 client.19098 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f19ffd780 0x555f19fffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 4 client.19098 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f19f5ab80 con 0x555f19ffd780 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.386+0000 7f046ffff640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f19ffd780 0x555f19fffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.387+0000 7f046ffff640 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f19ffd780 0x555f19fffbe0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a8b2454bff7c9f server_cookie=5f84b256039f3915 in_seq=0 out_seq=0 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.387+0000 7f044f7fe640 10 client.19098 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.387+0000 7f044f7fe640 1 -- 172.21.15.26:0/3838314708 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555f19f5ab80 con 0x555f19ffd780 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.387+0000 7f044f7fe640 10 client.19098 handle_command_reply: tid=0 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.387+0000 7f047924a740 1 client.19098 shutdown 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 msgr2=0x555f19ff8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 0x555f19ff8da0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f04680127c0 tx=0x7f0468004ab0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 shutdown_connections 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f19ffd780 0x555f19fffbe0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fed220 0x555f19ff8da0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19ff1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 --2- 172.21.15.26:0/3838314708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f044f7fe640 0 client.19098 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 >> 172.21.15.26:0/3838314708 conn(0x555f19fe2d30 msgr2=0x555f19fe1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 shutdown_connections 2024-05-20T21:08:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 1 -- 172.21.15.26:0/3838314708 wait complete. 2024-05-20T21:08:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.388+0000 7f047924a740 20 client.19098 trim_cache size 0 max 16384 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 Processor -- start 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 -- start start 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 0x555f19fe9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19fe9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19ff8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04701ac7b0 con 0x555f19fe76c0 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04701aa080 con 0x555f19fed220 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f047924a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04701a7ed0 con 0x555f19fe80b0 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 0x555f19fe9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 0x555f19fe9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38726/0 (socket says 172.21.15.26:38726) 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 -- 172.21.15.26:0/671338472 learned_addr learned my addr 172.21.15.26:0/671338472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f0474ac8640 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19ff8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 -- 172.21.15.26:0/671338472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 msgr2=0x555f19fe9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19fe9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 -- 172.21.15.26:0/671338472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 msgr2=0x555f19ff8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19ff8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.406+0000 7f046ffff640 1 -- 172.21.15.26:0/671338472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f19fe7220 con 0x555f19fe76c0 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.407+0000 7f046ffff640 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 0x555f19fe9330 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f04600c5f20 tx=0x7f0460108550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e050019bcb50ce server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.407+0000 7f0455ffb640 1 -- 172.21.15.26:0/671338472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460049040 con 0x555f19fe76c0 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.407+0000 7f0455ffb640 1 -- 172.21.15.26:0/671338472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04600c99b0 con 0x555f19fe76c0 2024-05-20T21:08:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.407+0000 7f0455ffb640 1 -- 172.21.15.26:0/671338472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600d2d00 con 0x555f19fe76c0 2024-05-20T21:08:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 -- 172.21.15.26:0/671338472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 msgr2=0x555f19fe9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 0x555f19fe9330 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f04600c5f20 tx=0x7f0460108550 comp rx=0 tx=0).stop 2024-05-20T21:08:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 -- 172.21.15.26:0/671338472 shutdown_connections 2024-05-20T21:08:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fed220 0x555f19ff8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fe80b0 0x555f19fe9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- 172.21.15.26:0/671338472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe76c0 0x555f19fe9330 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 -- 172.21.15.26:0/671338472 >> 172.21.15.26:0/671338472 conn(0x555f19fe2d30 msgr2=0x555f19ff1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 -- 172.21.15.26:0/671338472 shutdown_connections 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 -- 172.21.15.26:0/671338472 wait complete. 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 -- start start 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 Processor -- start 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19ff5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.408+0000 7f047924a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fed220 0x555f1a066370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f047924a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04701a8540 con 0x555f19fe80b0 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f047924a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04701a99b0 con 0x555f19fe76c0 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f047924a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04701a8c10 con 0x555f19fed220 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f047924a740 10 client.0 _ll_register_callbacks cb 0x7f0468012710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19ff5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19ff5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38728/0 (socket says 172.21.15.26:38728) 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 -- 172.21.15.26:0/1989151080 learned_addr learned my addr 172.21.15.26:0/1989151080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046ffff640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f0474ac8640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fed220 0x555f1a066370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 -- 172.21.15.26:0/1989151080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fed220 msgr2=0x555f1a066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fed220 0x555f1a066370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 -- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 msgr2=0x555f19ff5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 -- 172.21.15.26:0/1989151080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f19ff8f30 con 0x555f19fe80b0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046ffff640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f046f7fe640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19ff5e60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0464010c40 tx=0x7f0464005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400367e6bf284463 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f044f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f044f7fe640 1 -- 172.21.15.26:0/1989151080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04640037f0 con 0x555f19fe80b0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f044f7fe640 1 -- 172.21.15.26:0/1989151080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f0464003990 con 0x555f19fe80b0 2024-05-20T21:08:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.409+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04600c99b0 con 0x555f19fe80b0 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f044f7fe640 1 -- 172.21.15.26:0/1989151080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0464003b90 con 0x555f19fe80b0 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 10 client.19104 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f1a067850 con 0x555f19fe80b0 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f044f7fe640 1 -- 172.21.15.26:0/1989151080 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04600c99b0 con 0x555f19fe80b0 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f044f7fe640 1 -- 172.21.15.26:0/1989151080 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f046400f040 con 0x555f19fe80b0 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 10 client.19104 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 10 client.19104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 20 client.19104 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1a067f90 0x555f1a06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 4 client.19104 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:08:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f19f54360 con 0x555f1a067f90 2024-05-20T21:08:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.410+0000 7f046ffff640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1a067f90 0x555f1a06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.411+0000 7f044f7fe640 10 client.19104 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.411+0000 7f046ffff640 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1a067f90 0x555f1a06a3b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05f93a2c4308cc2 server_cookie=d4beb974180b623e in_seq=0 out_seq=0 2024-05-20T21:08:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.456+0000 7f044f7fe640 1 -- 172.21.15.26:0/1989151080 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555f19f54360 con 0x555f1a067f90 2024-05-20T21:08:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.456+0000 7f044f7fe640 10 client.19104 handle_command_reply: tid=0 2024-05-20T21:08:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.456+0000 7f047924a740 1 client.19104 shutdown 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 msgr2=0x555f19ff5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19ff5e60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0464010c40 tx=0x7f0464005500 comp rx=0 tx=0).stop 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 shutdown_connections 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x555f1a067f90 0x555f1a06a3b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f19fed220 0x555f1a066370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f19fe80b0 0x555f19ff5e60 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 --2- 172.21.15.26:0/1989151080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f19fe76c0 0x555f19ff5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 >> 172.21.15.26:0/1989151080 conn(0x555f19fe2d30 msgr2=0x555f19fe9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f044f7fe640 0 client.19104 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 shutdown_connections 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 1 -- 172.21.15.26:0/1989151080 wait complete. 2024-05-20T21:08:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.457+0000 7f047924a740 20 client.19104 trim_cache size 0 max 16384 2024-05-20T21:08:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:08:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:08:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04580b28a0 msgr2=0x7f04580b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04580b28a0 0x7f04580b4d60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x555f19fe8910 tx=0x7f0468009500 comp rx=0 tx=0).stop 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 msgr2=0x7f0470198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470198b40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0460009230 tx=0x7f04600042f0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 shutdown_connections 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f04580b28a0 0x7f04580b4d60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0470108790 0x7f04701a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0470105e60 0x7f0470199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 --2- 172.21.15.26:0/383807789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0470103530 0x7f0470198b40 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 >> 172.21.15.26:0/383807789 conn(0x7f04700fcf20 msgr2=0x7f04701040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.458+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 shutdown_connections 2024-05-20T21:08:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:02.459+0000 7f047652b640 1 -- 172.21.15.26:0/383807789 wait complete. 2024-05-20T21:08:02.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 274535 2.31 MB/sec execute 488 sec latency 40.281 ms 2024-05-20T21:08:03.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 -- 172.21.15.26:0/1232908214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 msgr2=0x7fedd810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 --2- 172.21.15.26:0/1232908214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd810f2f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b0a0 tx=0x7fedd000b3b0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 -- 172.21.15.26:0/1232908214 shutdown_connections 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 --2- 172.21.15.26:0/1232908214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd810f2f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 --2- 172.21.15.26:0/1232908214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8108520 0x7fedd8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 --2- 172.21.15.26:0/1232908214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8102440 0x7fedd81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 -- 172.21.15.26:0/1232908214 >> 172.21.15.26:0/1232908214 conn(0x7fedd80793f0 msgr2=0x7fedd8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 -- 172.21.15.26:0/1232908214 shutdown_connections 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 -- 172.21.15.26:0/1232908214 wait complete. 2024-05-20T21:08:03.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.252+0000 7feddd3f5700 1 Processor -- start 2024-05-20T21:08:03.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 -- start start 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 0x7fedd819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8108520 0x7fedd8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112480 con 0x7fedd8102e00 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112300 con 0x7fedd8108520 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7feddd3f5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112600 con 0x7fedd8102440 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 0x7fedd819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7fedd67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 0x7fedd819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:40950/0 (socket says 172.21.15.26:40950) 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7fedd6ffd700 1 -- 172.21.15.26:0/2119857824 learned_addr learned my addr 172.21.15.26:0/2119857824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.253+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8108520 0x7fedd8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd77fe700 1 -- 172.21.15.26:0/2119857824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 msgr2=0x7fedd819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 0x7fedd819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd77fe700 1 -- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 msgr2=0x7fedd819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd77fe700 1 -- 172.21.15.26:0/2119857824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd819a090 con 0x7fedd8108520 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 0x7fedd819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd67fc700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8108520 0x7fedd8199930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b0a0 tx=0x7fedd0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedbffff700 1 -- 172.21.15.26:0/2119857824 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0021070 con 0x7fedd8108520 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd819a320 con 0x7fedd8108520 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedbffff700 1 -- 172.21.15.26:0/2119857824 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd0007b80 con 0x7fedd8108520 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7fedbffff700 1 -- 172.21.15.26:0/2119857824 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0005040 con 0x7fedd8108520 2024-05-20T21:08:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.254+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a65f0 con 0x7fedd8108520 2024-05-20T21:08:03.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.257+0000 7fedbffff700 1 -- 172.21.15.26:0/2119857824 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fedd0024070 con 0x7fedd8108520 2024-05-20T21:08:03.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedbffff700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fedc00b2880 0x7fedc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 Processor -- start 2024-05-20T21:08:03.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 -- start start 2024-05-20T21:08:03.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedbffff700 1 -- 172.21.15.26:0/2119857824 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fedd000f020 con 0x7fedd8108520 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fedc00b2880 0x7fedc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862acf0 0x55efa862b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa862edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa862f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55efa8585d10 con 0x55efa862f3f0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55efa85758b0 con 0x55efa862b6a0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55efa85798f0 con 0x55efa862acf0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa862f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa862f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38752/0 (socket says 172.21.15.26:38752) 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedd77fe700 1 -- 172.21.15.26:0/1118694689 learned_addr learned my addr 172.21.15.26:0/1118694689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedd67fc700 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa862edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.258+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862acf0 0x55efa862b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fedc00b2880 0x7fedc00b4d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fedd81079f0 tx=0x7fedc8010870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd67fc700 1 -- 172.21.15.26:0/1118694689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862acf0 msgr2=0x55efa862b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd67fc700 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862acf0 0x55efa862b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd67fc700 1 -- 172.21.15.26:0/1118694689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 msgr2=0x55efa862f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd67fc700 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa862f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd67fc700 1 -- 172.21.15.26:0/1118694689 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa84c5950 con 0x55efa862b6a0 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862acf0 0x55efa862b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedd67fc700 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa862edd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0114c0 tx=0x7fedcc0117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ab4bcf335298a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedbd7fa700 1 -- 172.21.15.26:0/1118694689 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc004230 con 0x55efa862b6a0 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedbd7fa700 1 -- 172.21.15.26:0/1118694689 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedcc011e50 con 0x55efa862b6a0 2024-05-20T21:08:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.259+0000 7fedbd7fa700 1 -- 172.21.15.26:0/1118694689 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc005450 con 0x55efa862b6a0 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 -- 172.21.15.26:0/1118694689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 msgr2=0x55efa862edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa862edd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0114c0 tx=0x7fedcc0117d0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 -- 172.21.15.26:0/1118694689 shutdown_connections 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa862f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862acf0 0x55efa862b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/1118694689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa862edd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 -- 172.21.15.26:0/1118694689 >> 172.21.15.26:0/1118694689 conn(0x55efa8554100 msgr2=0x55efa8620f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 -- 172.21.15.26:0/1118694689 shutdown_connections 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.260+0000 7fede7c8bb80 1 -- 172.21.15.26:0/1118694689 wait complete. 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 -- start start 2024-05-20T21:08:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 Processor -- start 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8636850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 0x55efa8633b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa8634090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55efa847c020 con 0x55efa862b6a0 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55efa8517840 con 0x55efa862acf0 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55efa86375b0 con 0x55efa862f3f0 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fede7c8bb80 10 client.0 _ll_register_callbacks cb 0x55efa85109a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fedd67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 0x55efa8633b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fedd67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 0x55efa8633b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38764/0 (socket says 172.21.15.26:38764) 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fedd67fc700 1 -- 172.21.15.26:0/2928446737 learned_addr learned my addr 172.21.15.26:0/2928446737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa8634090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.261+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8636850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd77fe700 1 -- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 msgr2=0x55efa8636850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8636850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd77fe700 1 -- 172.21.15.26:0/2928446737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 msgr2=0x55efa8633b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 0x55efa8633b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd77fe700 1 -- 172.21.15.26:0/2928446737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa8514710 con 0x55efa862f3f0 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8636850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd67fc700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 0x55efa8633b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedd77fe700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa8634090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedd00a7f80 tx=0x7fedd00aa480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8f61492c3e3cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedbaff5700 1 -- 172.21.15.26:0/2928446737 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00b3420 con 0x55efa862f3f0 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedcc011e50 con 0x55efa862f3f0 2024-05-20T21:08:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedbaff5700 1 -- 172.21.15.26:0/2928446737 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fedd00c5300 con 0x55efa862f3f0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fedbaff5700 1 -- 172.21.15.26:0/2928446737 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fedd00aa820 con 0x55efa862f3f0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fede7c8bb80 10 client.28177 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.262+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55efa8641cc0 con 0x55efa862f3f0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fedbaff5700 1 -- 172.21.15.26:0/2928446737 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedcc011e50 con 0x55efa862f3f0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fedbaff5700 1 -- 172.21.15.26:0/2928446737 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fedd0011070 con 0x55efa862f3f0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fede7c8bb80 10 client.28177 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fede7c8bb80 10 client.28177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fede7c8bb80 20 client.28177 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86442d0 0x55efa86466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fede7c8bb80 4 client.28177 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efa85f2a60 con 0x55efa86442d0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.263+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86442d0 0x55efa86466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.264+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86442d0 0x55efa86466f0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90650ce3730d8c4 server_cookie=34f0a2fd70f78f61 in_seq=0 out_seq=0 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.264+0000 7fedbaff5700 10 client.28177 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.264+0000 7fedbaff5700 1 -- 172.21.15.26:0/2928446737 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55efa85f2a60 con 0x55efa86442d0 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.264+0000 7fedbaff5700 10 client.28177 handle_command_reply: tid=0 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.264+0000 7fede7c8bb80 1 client.28177 shutdown 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 msgr2=0x55efa8634090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa8634090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedd00a7f80 tx=0x7fedd00aa480 comp rx=0 tx=0).stop 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 shutdown_connections 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86442d0 0x55efa86466f0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fedbaff5700 0 client.28177 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa8634090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8636850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/2928446737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862b6a0 0x55efa8633b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 >> 172.21.15.26:0/2928446737 conn(0x55efa8554100 msgr2=0x55efa86307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 shutdown_connections 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 1 -- 172.21.15.26:0/2928446737 wait complete. 2024-05-20T21:08:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.265+0000 7fede7c8bb80 20 client.28177 trim_cache size 0 max 16384 2024-05-20T21:08:03.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 Processor -- start 2024-05-20T21:08:03.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 -- start start 2024-05-20T21:08:03.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8635a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862b6a0 0x55efa864bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa864c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81ae230 con 0x55efa862f3f0 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81abac0 con 0x55efa862acf0 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.293+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81a9930 con 0x55efa862b6a0 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa864c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8635a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8635a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54032/0 (socket says 172.21.15.26:54032) 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa864c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38774/0 (socket says 172.21.15.26:38774) 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd6ffd700 1 -- 172.21.15.26:0/403951478 learned_addr learned my addr 172.21.15.26:0/403951478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd67fc700 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862b6a0 0x55efa864bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 -- 172.21.15.26:0/403951478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862b6a0 msgr2=0x55efa864bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862b6a0 0x55efa864bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 -- 172.21.15.26:0/403951478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 msgr2=0x55efa8635a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8635a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 -- 172.21.15.26:0/403951478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa8621dc0 con 0x55efa862f3f0 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8635a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedd77fe700 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa864c4e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009ca0 tx=0x7fedd000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a13cb3dcabaed54 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedbcff9700 1 -- 172.21.15.26:0/403951478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00b3420 con 0x55efa862f3f0 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedbcff9700 1 -- 172.21.15.26:0/403951478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd00c3070 con 0x55efa862f3f0 2024-05-20T21:08:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.294+0000 7fedbcff9700 1 -- 172.21.15.26:0/403951478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00aad10 con 0x55efa862f3f0 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 -- 172.21.15.26:0/403951478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 msgr2=0x55efa864c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa864c4e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009ca0 tx=0x7fedd000b930 comp rx=0 tx=0).stop 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 -- 172.21.15.26:0/403951478 shutdown_connections 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862f3f0 0x55efa864c4e0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862acf0 0x55efa8635a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/403951478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862b6a0 0x55efa864bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 -- 172.21.15.26:0/403951478 >> 172.21.15.26:0/403951478 conn(0x55efa8554100 msgr2=0x55efa8635380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 -- 172.21.15.26:0/403951478 shutdown_connections 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.295+0000 7fede7c8bb80 1 -- 172.21.15.26:0/403951478 wait complete. 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 -- start start 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 Processor -- start 2024-05-20T21:08:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 0x55efa8639810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa8639d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa863a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81a9f80 con 0x55efa862acf0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81ab3f0 con 0x55efa862b6a0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81aa650 con 0x55efa862f3f0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fede7c8bb80 10 client.0 _ll_register_callbacks cb 0x55efa85109a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 0x55efa8639810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 0x55efa8639810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38778/0 (socket says 172.21.15.26:38778) 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd6ffd700 1 -- 172.21.15.26:0/277662453 learned_addr learned my addr 172.21.15.26:0/277662453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd67fc700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa8639d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd77fe700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa863a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd6ffd700 1 -- 172.21.15.26:0/277662453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 msgr2=0x55efa863a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa863a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.296+0000 7fedd6ffd700 1 -- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 msgr2=0x55efa8639d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa8639d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedd6ffd700 1 -- 172.21.15.26:0/277662453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa86af050 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedd67fc700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa8639d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedd6ffd700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 0x55efa8639810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fedc80049f0 tx=0x7fedc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b6466a44bbefd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedbaff5700 1 -- 172.21.15.26:0/277662453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8005670 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedbaff5700 1 -- 172.21.15.26:0/277662453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fedc802a410 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd00c3070 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedbaff5700 1 -- 172.21.15.26:0/277662453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fedc8004020 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fede7c8bb80 10 client.19122 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55efa86b03d0 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.297+0000 7fedbaff5700 1 -- 172.21.15.26:0/277662453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd00c3070 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fedbaff5700 1 -- 172.21.15.26:0/277662453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fedc8003680 con 0x55efa862acf0 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fede7c8bb80 10 client.19122 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fede7c8bb80 10 client.19122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fede7c8bb80 20 client.19122 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86b0ac0 0x55efa86b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fede7c8bb80 4 client.19122 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55efa8587c60 con 0x55efa86b0ac0 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fedd67fc700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86b0ac0 0x55efa86b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fedd67fc700 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86b0ac0 0x55efa86b2ee0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1b052e7d7a20b3 server_cookie=296b197ec701c7d9 in_seq=0 out_seq=0 2024-05-20T21:08:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.298+0000 7fedbaff5700 10 client.19122 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fedbaff5700 1 -- 172.21.15.26:0/277662453 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55efa8587c60 con 0x55efa86b0ac0 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fedbaff5700 10 client.19122 handle_command_reply: tid=0 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 client.19122 shutdown 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 msgr2=0x55efa8639810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 0x55efa8639810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fedc80049f0 tx=0x7fedc80047c0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 shutdown_connections 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55efa86b0ac0 0x55efa86b2ee0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55efa862acf0 0x55efa8639810 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55efa862f3f0 0x55efa863a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 --2- 172.21.15.26:0/277662453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55efa862b6a0 0x55efa8639d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 >> 172.21.15.26:0/277662453 conn(0x55efa8554100 msgr2=0x55efa8635380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fedbaff5700 0 client.19122 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 shutdown_connections 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 1 -- 172.21.15.26:0/277662453 wait complete. 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.300+0000 7fede7c8bb80 20 client.19122 trim_cache size 0 max 16384 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "a9adcd89-cb26-4caf-9e68-bf3c376ca39a", 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:08:03.304 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fedc00b2880 msgr2=0x7fedc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fedc00b2880 0x7fedc00b4d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fedd81079f0 tx=0x7fedc8010870 comp rx=0 tx=0).stop 2024-05-20T21:08:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8108520 msgr2=0x7fedd8199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8108520 0x7fedd8199930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b0a0 tx=0x7fedd0004d00 comp rx=0 tx=0).stop 2024-05-20T21:08:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 shutdown_connections 2024-05-20T21:08:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedd8108520 0x7fedd8199930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd8102440 0x7fedd819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fedc00b2880 0x7fedc00b4d40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 --2- 172.21.15.26:0/2119857824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedd8102e00 0x7fedd819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 >> 172.21.15.26:0/2119857824 conn(0x7fedd80793f0 msgr2=0x7fedd806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 shutdown_connections 2024-05-20T21:08:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:03.301+0000 7feddd3f5700 1 -- 172.21.15.26:0/2119857824 wait complete. 2024-05-20T21:08:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:03 smithi026 ceph-mon[31990]: pgmap v747: 97 pgs: 97 active+clean; 28 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-05-20T21:08:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:03 smithi144 ceph-mon[35660]: pgmap v747: 97 pgs: 97 active+clean; 28 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-05-20T21:08:03.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 275166 2.30 MB/sec execute 489 sec latency 18.844 ms 2024-05-20T21:08:03.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:03 smithi055 ceph-mon[35213]: pgmap v747: 97 pgs: 97 active+clean; 28 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-05-20T21:08:03.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:08:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:04.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:04 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:08:04.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:04 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:04.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:04 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:08:04.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:04 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:04 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:08:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:04 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:04 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:08:04.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:04 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:04.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 275772 2.30 MB/sec execute 490 sec latency 16.363 ms 2024-05-20T21:08:04.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:04 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:08:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:04 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:04 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:08:04.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:04 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:05 smithi026 ceph-mon[31990]: pgmap v748: 97 pgs: 97 active+clean; 30 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T21:08:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:05 smithi144 ceph-mon[35660]: pgmap v748: 97 pgs: 97 active+clean; 30 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T21:08:05.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 276485 2.31 MB/sec execute 491 sec latency 72.268 ms 2024-05-20T21:08:05.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:05 smithi055 ceph-mon[35213]: pgmap v748: 97 pgs: 97 active+clean; 30 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-05-20T21:08:06.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:06 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:08:06.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 277057 2.32 MB/sec execute 492 sec latency 70.451 ms 2024-05-20T21:08:06.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:06 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:08:06.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:06 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:08:07.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:07 smithi026 ceph-mon[31990]: pgmap v749: 97 pgs: 97 active+clean; 29 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:08:07.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 277558 2.31 MB/sec execute 493 sec latency 25.228 ms 2024-05-20T21:08:07.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:07 smithi055 ceph-mon[35213]: pgmap v749: 97 pgs: 97 active+clean; 29 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:08:07.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:07 smithi144 ceph-mon[35660]: pgmap v749: 97 pgs: 97 active+clean; 29 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:08:08.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 278119 2.32 MB/sec execute 494 sec latency 92.416 ms 2024-05-20T21:08:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:09 smithi026 ceph-mon[31990]: pgmap v750: 97 pgs: 97 active+clean; 26 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-20T21:08:09.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 278674 2.31 MB/sec execute 495 sec latency 91.724 ms 2024-05-20T21:08:09.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:09 smithi055 ceph-mon[35213]: pgmap v750: 97 pgs: 97 active+clean; 26 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-20T21:08:09.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:09 smithi144 ceph-mon[35660]: pgmap v750: 97 pgs: 97 active+clean; 26 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-20T21:08:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 -- 172.21.15.26:0/1191462927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 msgr2=0x7fef2c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 --2- 172.21.15.26:0/1191462927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c109990 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009990 tx=0x7fef1c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 -- 172.21.15.26:0/1191462927 shutdown_connections 2024-05-20T21:08:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 --2- 172.21.15.26:0/1191462927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c109990 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 --2- 172.21.15.26:0/1191462927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2c06bbe0 0x7fef2c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 --2- 172.21.15.26:0/1191462927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06b220 0x7fef2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 -- 172.21.15.26:0/1191462927 >> 172.21.15.26:0/1191462927 conn(0x7fef2c0793f0 msgr2=0x7fef2c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 -- 172.21.15.26:0/1191462927 shutdown_connections 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 -- 172.21.15.26:0/1191462927 wait complete. 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.198+0000 7fef32276700 1 Processor -- start 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 -- start start 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2c06b220 0x7fef2c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06bbe0 0x7fef2c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113590 con 0x7fef2c1095b0 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113410 con 0x7fef2c06bbe0 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef32276700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113710 con 0x7fef2c06b220 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38796/0 (socket says 172.21.15.26:38796) 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 -- 172.21.15.26:0/3280525679 learned_addr learned my addr 172.21.15.26:0/3280525679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06bbe0 0x7fef2c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef2bfff700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2c06b220 0x7fef2c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 -- 172.21.15.26:0/3280525679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2c06b220 msgr2=0x7fef2c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2c06b220 0x7fef2c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 -- 172.21.15.26:0/3280525679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06bbe0 msgr2=0x7fef2c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06bbe0 0x7fef2c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.199+0000 7fef30813700 1 -- 172.21.15.26:0/3280525679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2c19a090 con 0x7fef2c1095b0 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06bbe0 0x7fef2c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef30813700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c199930 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fef2400ada0 tx=0x7fef2400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef297fa700 1 -- 172.21.15.26:0/3280525679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24017020 con 0x7fef2c1095b0 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef297fa700 1 -- 172.21.15.26:0/3280525679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef2400ce10 con 0x7fef2c1095b0 2024-05-20T21:08:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2c19a380 con 0x7fef2c1095b0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef297fa700 1 -- 172.21.15.26:0/3280525679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24005440 con 0x7fef2c1095b0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.200+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2c1a6640 con 0x7fef2c1095b0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.203+0000 7fef297fa700 1 -- 172.21.15.26:0/3280525679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fef24004ad0 con 0x7fef2c1095b0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.204+0000 7fef297fa700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef140b2880 0x7fef140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.204+0000 7fef2bfff700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef140b2880 0x7fef140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.204+0000 7fef297fa700 1 -- 172.21.15.26:0/3280525679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fef240e19b0 con 0x7fef2c1095b0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.204+0000 7fef3cb0cb80 1 Processor -- start 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.204+0000 7fef2bfff700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef140b2880 0x7fef140b4d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00b650 tx=0x7fef1c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.204+0000 7fef3cb0cb80 1 -- start start 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f76c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f7707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55575f6c6c90 con 0x55575f770370 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55575f6b6830 con 0x55575f76c620 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55575f6ba870 con 0x55575f76bc70 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f7707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f7707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38810/0 (socket says 172.21.15.26:38810) 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 -- 172.21.15.26:0/339053795 learned_addr learned my addr 172.21.15.26:0/339053795 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef2bfff700 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f76c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 -- 172.21.15.26:0/339053795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 msgr2=0x55575f76c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f76c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 -- 172.21.15.26:0/339053795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 msgr2=0x55575f76fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 -- 172.21.15.26:0/339053795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575f6068d0 con 0x55575f770370 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef30813700 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f7707b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fef2400a0a0 tx=0x7fef240a4ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51165c8e61d50677 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.205+0000 7fef127fc700 1 -- 172.21.15.26:0/339053795 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef240a9000 con 0x55575f770370 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef127fc700 1 -- 172.21.15.26:0/339053795 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef240ad3b0 con 0x55575f770370 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef127fc700 1 -- 172.21.15.26:0/339053795 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2400c920 con 0x55575f770370 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/339053795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 msgr2=0x55575f7707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f7707b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fef2400a0a0 tx=0x7fef240a4ff0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/339053795 shutdown_connections 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f7707b0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f76c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/339053795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/339053795 >> 172.21.15.26:0/339053795 conn(0x55575f695080 msgr2=0x55575f761e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/339053795 shutdown_connections 2024-05-20T21:08:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.206+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/339053795 wait complete. 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 -- start start 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 Processor -- start 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f777740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 0x55575f774a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f770370 0x55575f774f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55575f5bcfa0 con 0x55575f76c620 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55575f6587c0 con 0x55575f770370 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55575f778530 con 0x55575f76bc70 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef3cb0cb80 10 client.0 _ll_register_callbacks cb 0x55575f651920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef2b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 0x55575f774a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef2b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 0x55575f774a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38826/0 (socket says 172.21.15.26:38826) 2024-05-20T21:08:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef2b7fe700 1 -- 172.21.15.26:0/407065317 learned_addr learned my addr 172.21.15.26:0/407065317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef30813700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f770370 0x55575f774f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.207+0000 7fef2bfff700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f777740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef2b7fe700 1 -- 172.21.15.26:0/407065317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 msgr2=0x55575f777740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f777740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef2b7fe700 1 -- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f770370 msgr2=0x55575f774f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f770370 0x55575f774f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef2b7fe700 1 -- 172.21.15.26:0/407065317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575f655690 con 0x55575f76c620 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 0x55575f774a40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fef20012840 tx=0x7fef20012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab12b5dd867ba732 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef0b7fe700 1 -- 172.21.15.26:0/407065317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200117d0 con 0x55575f76c620 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef0b7fe700 1 -- 172.21.15.26:0/407065317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fef20011970 con 0x55575f76c620 2024-05-20T21:08:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef240ad3b0 con 0x55575f76c620 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef0b7fe700 1 -- 172.21.15.26:0/407065317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef20011b70 con 0x55575f76c620 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef3cb0cb80 10 client.19134 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55575f782c60 con 0x55575f76c620 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.208+0000 7fef0b7fe700 1 -- 172.21.15.26:0/407065317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef240ad3b0 con 0x55575f76c620 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef0b7fe700 1 -- 172.21.15.26:0/407065317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fef20024020 con 0x55575f76c620 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef3cb0cb80 10 client.19134 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef3cb0cb80 10 client.19134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef3cb0cb80 20 client.19134 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f785240 0x55575f787630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef3cb0cb80 4 client.19134 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575f7339e0 con 0x55575f785240 2024-05-20T21:08:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef2bfff700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f785240 0x55575f787630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef2bfff700 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f785240 0x55575f787630 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4a0abc5ae5efaf server_cookie=7a5a893c326766bd in_seq=0 out_seq=0 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.209+0000 7fef0b7fe700 10 client.19134 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef0b7fe700 1 -- 172.21.15.26:0/407065317 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55575f7339e0 con 0x55575f785240 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef0b7fe700 10 client.19134 handle_command_reply: tid=0 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 client.19134 shutdown 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 msgr2=0x55575f774a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 0x55575f774a40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fef20012840 tx=0x7fef20012b50 comp rx=0 tx=0).stop 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 shutdown_connections 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f785240 0x55575f787630 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f770370 0x55575f774f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef0b7fe700 0 client.19134 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f777740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/407065317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76c620 0x55575f774a40 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 >> 172.21.15.26:0/407065317 conn(0x55575f695080 msgr2=0x55575f761510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 shutdown_connections 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.210+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/407065317 wait complete. 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.211+0000 7fef3cb0cb80 20 client.19134 trim_cache size 0 max 16384 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.238+0000 7fef3cb0cb80 1 Processor -- start 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.238+0000 7fef3cb0cb80 1 -- start start 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f776980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f76e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1ae3d0 con 0x55575f770370 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1abc60 con 0x55575f76c620 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1a9ab0 con 0x55575f76bc70 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef30813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f76e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef30813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f76e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38830/0 (socket says 172.21.15.26:38830) 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef30813700 1 -- 172.21.15.26:0/490016526 learned_addr learned my addr 172.21.15.26:0/490016526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2bfff700 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f776980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 -- 172.21.15.26:0/490016526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 msgr2=0x55575f776980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f776980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 -- 172.21.15.26:0/490016526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 msgr2=0x55575f76e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f76e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 -- 172.21.15.26:0/490016526 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575f762940 con 0x55575f76c620 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2bfff700 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f776980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef30813700 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f76e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.239+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76dcc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fef20000c00 tx=0x7fef20009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683e603a0d4731f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.240+0000 7fef11ffb700 1 -- 172.21.15.26:0/490016526 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20011d90 con 0x55575f76c620 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.240+0000 7fef11ffb700 1 -- 172.21.15.26:0/490016526 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef2000fde0 con 0x55575f76c620 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.240+0000 7fef11ffb700 1 -- 172.21.15.26:0/490016526 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200266c0 con 0x55575f76c620 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/490016526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 msgr2=0x55575f76dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76dcc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fef20000c00 tx=0x7fef20009870 comp rx=0 tx=0).stop 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/490016526 shutdown_connections 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f770370 0x55575f76e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f76bc70 0x55575f776980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/490016526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f76dcc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/490016526 >> 172.21.15.26:0/490016526 conn(0x55575f695080 msgr2=0x55575f776270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/490016526 shutdown_connections 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/490016526 wait complete. 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 -- start start 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.241+0000 7fef3cb0cb80 1 Processor -- start 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 0x55575f779cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f77a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f770370 0x55575f7eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1aa120 con 0x55575f76bc70 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1ab590 con 0x55575f76c620 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1aa7f0 con 0x55575f770370 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 0x55575f779cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef3cb0cb80 10 client.0 _ll_register_callbacks cb 0x55575f651920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 0x55575f779cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38836/0 (socket says 172.21.15.26:38836) 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 -- 172.21.15.26:0/1069254094 learned_addr learned my addr 172.21.15.26:0/1069254094 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f77a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef30813700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f770370 0x55575f7eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 -- 172.21.15.26:0/1069254094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f770370 msgr2=0x55575f7eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f770370 0x55575f7eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 -- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 msgr2=0x55575f77a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f77a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 -- 172.21.15.26:0/1069254094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575f7ef370 con 0x55575f76bc70 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef30813700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f770370 0x55575f7eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef2bfff700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 0x55575f779cb0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009560 tx=0x7fef1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d51984d3aae0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.242+0000 7fef0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef0b7fe700 1 -- 172.21.15.26:0/1069254094 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c037070 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef0b7fe700 1 -- 172.21.15.26:0/1069254094 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fef1c004a30 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2000fde0 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef0b7fe700 1 -- 172.21.15.26:0/1069254094 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef1c042050 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 10 client.19140 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55575f7f0660 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef0b7fe700 1 -- 172.21.15.26:0/1069254094 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef2000fde0 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef0b7fe700 1 -- 172.21.15.26:0/1069254094 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fef1c022070 con 0x55575f76bc70 2024-05-20T21:08:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 10 client.19140 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 10 client.19140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 20 client.19140 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f7f0da0 0x55575f7f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 4 client.19140 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.243+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55575f6c8be0 con 0x55575f7f0da0 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f7f0da0 0x55575f7f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef0b7fe700 10 client.19140 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef2b7fe700 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f7f0da0 0x55575f7f31c0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f288ce721d54e29 server_cookie=e37123f924208c69 in_seq=0 out_seq=0 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef0b7fe700 1 -- 172.21.15.26:0/1069254094 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55575f6c8be0 con 0x55575f7f0da0 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef0b7fe700 10 client.19140 handle_command_reply: tid=0 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef3cb0cb80 1 client.19140 shutdown 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 msgr2=0x55575f779cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 0x55575f779cb0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009560 tx=0x7fef1c009310 comp rx=0 tx=0).stop 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 shutdown_connections 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55575f7f0da0 0x55575f7f31c0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef0b7fe700 0 client.19140 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.244+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55575f76bc70 0x55575f779cb0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575f770370 0x55575f7eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef3cb0cb80 1 --2- 172.21.15.26:0/1069254094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55575f76c620 0x55575f77a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 >> 172.21.15.26:0/1069254094 conn(0x55575f695080 msgr2=0x55575f776b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:08:09.828 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 shutdown_connections 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef3cb0cb80 1 -- 172.21.15.26:0/1069254094 wait complete. 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef3cb0cb80 20 client.19140 trim_cache size 0 max 16384 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef140b2880 msgr2=0x7fef140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef32276700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef140b2880 0x7fef140b4d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00b650 tx=0x7fef1c01a040 comp rx=0 tx=0).stop 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 msgr2=0x7fef2c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.245+0000 7fef32276700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c199930 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fef2400ada0 tx=0x7fef2400c330 comp rx=0 tx=0).stop 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 shutdown_connections 2024-05-20T21:08:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fef140b2880 0x7fef140b4d40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef2c1095b0 0x7fef2c199930 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2c06b220 0x7fef2c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 --2- 172.21.15.26:0/3280525679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef2c06bbe0 0x7fef2c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 >> 172.21.15.26:0/3280525679 conn(0x7fef2c0793f0 msgr2=0x7fef2c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 shutdown_connections 2024-05-20T21:08:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:09.246+0000 7fef32276700 1 -- 172.21.15.26:0/3280525679 wait complete. 2024-05-20T21:08:10.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 279308 2.32 MB/sec execute 496 sec latency 17.545 ms 2024-05-20T21:08:10.674 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:08:10.675 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:11.406 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:11 smithi026 ceph-mon[31990]: pgmap v751: 97 pgs: 97 active+clean; 27 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T21:08:11.407 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:11 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:08:11.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 280023 2.32 MB/sec execute 497 sec latency 18.342 ms 2024-05-20T21:08:11.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:11 smithi055 ceph-mon[35213]: pgmap v751: 97 pgs: 97 active+clean; 27 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T21:08:11.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:11 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:08:11.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:11 smithi144 ceph-mon[35660]: pgmap v751: 97 pgs: 97 active+clean; 27 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-20T21:08:11.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:11 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:08:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:08:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:08:12.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:12 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:08:12.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 280614 2.33 MB/sec execute 498 sec latency 18.596 ms 2024-05-20T21:08:12.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:08:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:08:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:12 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:08:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:08:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:08:12.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:12 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:08:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 -- 172.21.15.26:0/4078246858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 msgr2=0x7f955006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 --2- 172.21.15.26:0/4078246858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955006b6a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9540009990 tx=0x7f9540009ca0 comp rx=0 tx=0).stop 2024-05-20T21:08:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 -- 172.21.15.26:0/4078246858 shutdown_connections 2024-05-20T21:08:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 --2- 172.21.15.26:0/4078246858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f955006bbe0 0x7f95501103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 --2- 172.21.15.26:0/4078246858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95501095b0 0x7f9550109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 --2- 172.21.15.26:0/4078246858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955006b6a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 -- 172.21.15.26:0/4078246858 >> 172.21.15.26:0/4078246858 conn(0x7f95500793f0 msgr2=0x7f9550079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 -- 172.21.15.26:0/4078246858 shutdown_connections 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.909+0000 7f955875e700 1 -- 172.21.15.26:0/4078246858 wait complete. 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 Processor -- start 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 -- start start 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f955006bbe0 0x7f955019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95501095b0 0x7f9550199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95501135f0 con 0x7f955006b220 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9550113470 con 0x7f95501095b0 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f955875e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9550113770 con 0x7f955006bbe0 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f95564fa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:12.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f95564fa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44424/0 (socket says 172.21.15.26:44424) 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.910+0000 7f95564fa700 1 -- 172.21.15.26:0/2282148174 learned_addr learned my addr 172.21.15.26:0/2282148174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f9556cfb700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95501095b0 0x7f9550199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f9555cf9700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f955006bbe0 0x7f955019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95564fa700 1 -- 172.21.15.26:0/2282148174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f955006bbe0 msgr2=0x7f955019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95564fa700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f955006bbe0 0x7f955019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95564fa700 1 -- 172.21.15.26:0/2282148174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95501095b0 msgr2=0x7f9550199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95564fa700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95501095b0 0x7f9550199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95564fa700 1 -- 172.21.15.26:0/2282148174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955019a0f0 con 0x7f955006b220 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f9556cfb700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95501095b0 0x7f9550199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95564fa700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955019f7d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f954c00d980 tx=0x7f954c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c017070 con 0x7f955006b220 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955019a3e0 con 0x7f955006b220 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.911+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95501a66a0 con 0x7f955006b220 2024-05-20T21:08:12.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.912+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f954c005020 con 0x7f955006b220 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.912+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c00f980 con 0x7f955006b220 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.912+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955006a230 con 0x7f955006b220 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.914+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f954c004a30 con 0x7f955006b220 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.915+0000 7f95477fe700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f953c0b2900 0x7f953c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.915+0000 7f9555cf9700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f953c0b2900 0x7f953c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.915+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f954c0e1fb0 con 0x7f955006b220 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.915+0000 7f9555cf9700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f953c0b2900 0x7f953c0b4dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9550100a40 tx=0x7f9540005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:12.920+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f954c020020 con 0x7f955006b220 2024-05-20T21:08:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.235+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f955002e560 con 0x7f955006b220 2024-05-20T21:08:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.236+0000 7f95477fe700 1 -- 172.21.15.26:0/2282148174 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f954c0a9c70 con 0x7f955006b220 2024-05-20T21:08:13.235 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:08:13.235 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:08:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f953c0b2900 msgr2=0x7f953c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f953c0b2900 0x7f953c0b4dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9550100a40 tx=0x7f9540005fb0 comp rx=0 tx=0).stop 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 msgr2=0x7f955019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955019f7d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f954c00d980 tx=0x7f954c00dc90 comp rx=0 tx=0).stop 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 shutdown_connections 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f953c0b2900 0x7f953c0b4dc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955006b220 0x7f955019f7d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95501095b0 0x7f9550199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 --2- 172.21.15.26:0/2282148174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f955006bbe0 0x7f955019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.239+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 >> 172.21.15.26:0/2282148174 conn(0x7f95500793f0 msgr2=0x7f9550101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.240+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 shutdown_connections 2024-05-20T21:08:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:13.240+0000 7f955875e700 1 -- 172.21.15.26:0/2282148174 wait complete. 2024-05-20T21:08:13.240 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:08:13.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 281133 2.32 MB/sec execute 499 sec latency 20.622 ms 2024-05-20T21:08:13.615 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:13 smithi026 ceph-mon[31990]: pgmap v752: 97 pgs: 97 active+clean; 26 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-05-20T21:08:13.615 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:13 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2282148174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:13.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:13 smithi055 ceph-mon[35213]: pgmap v752: 97 pgs: 97 active+clean; 26 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-05-20T21:08:13.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:13 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2282148174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:13.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:13 smithi144 ceph-mon[35660]: pgmap v752: 97 pgs: 97 active+clean; 26 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-05-20T21:08:13.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:13 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2282148174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:14.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:08:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:14.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 281836 2.33 MB/sec execute 500 sec latency 15.612 ms 2024-05-20T21:08:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 -- 172.21.15.26:0/1952267772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c103380 msgr2=0x7f098c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 --2- 172.21.15.26:0/1952267772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c103380 0x7f098c109c10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0984009ca0 tx=0x7f098402a8c0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 -- 172.21.15.26:0/1952267772 shutdown_connections 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 --2- 172.21.15.26:0/1952267772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c103380 0x7f098c109c10 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 --2- 172.21.15.26:0/1952267772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c1029c0 0x7f098c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 --2- 172.21.15.26:0/1952267772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1017d0 0x7f098c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 -- 172.21.15.26:0/1952267772 >> 172.21.15.26:0/1952267772 conn(0x7f098c0fcd40 msgr2=0x7f098c0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 -- 172.21.15.26:0/1952267772 shutdown_connections 2024-05-20T21:08:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.561+0000 7f099179e640 1 -- 172.21.15.26:0/1952267772 wait complete. 2024-05-20T21:08:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 Processor -- start 2024-05-20T21:08:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 -- start start 2024-05-20T21:08:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 0x7f098c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1029c0 0x7f098c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c103380 0x7f098c1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10d260 con 0x7f098c1017d0 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10d0e0 con 0x7f098c1029c0 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f099179e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10d3e0 con 0x7f098c103380 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f098bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 0x7f098c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.562+0000 7f098bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 0x7f098c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44448/0 (socket says 172.21.15.26:44448) 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098bfff640 1 -- 172.21.15.26:0/1539339985 learned_addr learned my addr 172.21.15.26:0/1539339985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1029c0 0x7f098c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f0990f9d640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c103380 0x7f098c1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 -- 172.21.15.26:0/1539339985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c103380 msgr2=0x7f098c1a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c103380 0x7f098c1a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 -- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 msgr2=0x7f098c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 0x7f098c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 -- 172.21.15.26:0/1539339985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c1a0680 con 0x7f098c1029c0 2024-05-20T21:08:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098bfff640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 0x7f098c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1029c0 0x7f098c198fa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f098000b3f0 tx=0x7f098000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f09897fa640 1 -- 172.21.15.26:0/1539339985 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980015020 con 0x7f098c1029c0 2024-05-20T21:08:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c1a0970 con 0x7f098c1029c0 2024-05-20T21:08:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f09897fa640 1 -- 172.21.15.26:0/1539339985 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0980011cc0 con 0x7f098c1029c0 2024-05-20T21:08:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f09897fa640 1 -- 172.21.15.26:0/1539339985 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980010cc0 con 0x7f098c1029c0 2024-05-20T21:08:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.563+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c1a0f10 con 0x7f098c1029c0 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.566+0000 7f09897fa640 1 -- 172.21.15.26:0/1539339985 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0980018020 con 0x7f098c1029c0 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.566+0000 7f09957f0740 1 Processor -- start 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.566+0000 7f09957f0740 1 -- start start 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09957f0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09957f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e9290 0x559e867edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09957f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09957f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559e86759c30 con 0x559e867ee400 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09957f0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559e867497d0 con 0x559e867e88a0 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09957f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e8674d810 con 0x559e867e9290 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f0990f9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f0990f9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867e8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33908/0 (socket says 172.21.15.26:33908) 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f0990f9d640 1 -- 172.21.15.26:0/3655662480 learned_addr learned my addr 172.21.15.26:0/3655662480 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098bfff640 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e9290 0x559e867edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09897fa640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f09740b27e0 0x7f09740b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098b7fe640 1 -- 172.21.15.26:0/3655662480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e9290 msgr2=0x559e867edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098b7fe640 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e9290 0x559e867edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098b7fe640 1 -- 172.21.15.26:0/3655662480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 msgr2=0x559e867e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098b7fe640 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098b7fe640 1 -- 172.21.15.26:0/3655662480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e8645e210 con 0x559e867ee400 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f0990f9d640 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f098bfff640 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e9290 0x559e867edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f0990f9d640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f09740b27e0 0x7f09740b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.567+0000 7f09897fa640 1 -- 172.21.15.26:0/1539339985 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f09800e19d0 con 0x7f098c1029c0 2024-05-20T21:08:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.568+0000 7f098b7fe640 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f0890 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f09800df440 tx=0x7f0980021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1434173e7f74c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.568+0000 7f09727fc640 1 -- 172.21.15.26:0/3655662480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a8d60 con 0x559e867ee400 2024-05-20T21:08:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.568+0000 7f09727fc640 1 -- 172.21.15.26:0/3655662480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09800a8aa0 con 0x559e867ee400 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.568+0000 7f09727fc640 1 -- 172.21.15.26:0/3655662480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980015420 con 0x559e867ee400 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.568+0000 7f0990f9d640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f09740b27e0 0x7f09740b4ca0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0984005fe0 tx=0x7f09840095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 -- 172.21.15.26:0/3655662480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 msgr2=0x559e867f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f0890 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f09800df440 tx=0x7f0980021000 comp rx=0 tx=0).stop 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 -- 172.21.15.26:0/3655662480 shutdown_connections 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f0890 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e9290 0x559e867edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 --2- 172.21.15.26:0/3655662480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 -- 172.21.15.26:0/3655662480 >> 172.21.15.26:0/3655662480 conn(0x559e867e3f10 msgr2=0x559e867e4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 -- 172.21.15.26:0/3655662480 shutdown_connections 2024-05-20T21:08:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 -- 172.21.15.26:0/3655662480 wait complete. 2024-05-20T21:08:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 -- start start 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.569+0000 7f09957f0740 1 Processor -- start 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e88a0 0x559e867f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e9290 0x559e867f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559e8642f790 con 0x559e867ee400 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559e86705890 con 0x559e867e9290 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e867f1350 con 0x559e867e88a0 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f09957f0740 10 client.0 _ll_register_callbacks cb 0x559e8645da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44476/0 (socket says 172.21.15.26:44476) 2024-05-20T21:08:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 -- 172.21.15.26:0/3752635246 learned_addr learned my addr 172.21.15.26:0/3752635246 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f098bfff640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e88a0 0x559e867f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f098b7fe640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e9290 0x559e867f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 -- 172.21.15.26:0/3752635246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e88a0 msgr2=0x559e867f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e88a0 0x559e867f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 -- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e9290 msgr2=0x559e867f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e9290 0x559e867f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 -- 172.21.15.26:0/3752635246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e8652a620 con 0x559e867ee400 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f0990f9d640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f9eb0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f098402aeb0 tx=0x7f098402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf34a5d8964247b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.570+0000 7f096b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f096b7fe640 1 -- 172.21.15.26:0/3752635246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984004250 con 0x559e867ee400 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f096b7fe640 1 -- 172.21.15.26:0/3752635246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f09840043f0 con 0x559e867ee400 2024-05-20T21:08:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09800a8aa0 con 0x559e867ee400 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f096b7fe640 1 -- 172.21.15.26:0/3752635246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09840045f0 con 0x559e867ee400 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 10 client.19158 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e867fcf00 con 0x559e867ee400 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f096b7fe640 1 -- 172.21.15.26:0/3752635246 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09800a8aa0 con 0x559e867ee400 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f096b7fe640 1 -- 172.21.15.26:0/3752635246 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f098405b020 con 0x559e867ee400 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 10 client.19158 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 10 client.19158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 20 client.19158 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e867fe6b0 0x559e86800aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 4 client.19158 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e8675bb80 con 0x559e867fe6b0 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.571+0000 7f098bfff640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e867fe6b0 0x559e86800aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.572+0000 7f096b7fe640 10 client.19158 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.572+0000 7f098bfff640 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e867fe6b0 0x559e86800aa0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c360f9df19f0e52 server_cookie=a5df3a98ad85b0ea in_seq=0 out_seq=0 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.572+0000 7f096b7fe640 1 -- 172.21.15.26:0/3752635246 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e8675bb80 con 0x559e867fe6b0 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.572+0000 7f096b7fe640 10 client.19158 handle_command_reply: tid=0 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.572+0000 7f09957f0740 1 client.19158 shutdown 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 msgr2=0x559e867f9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f9eb0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f098402aeb0 tx=0x7f098402a8c0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 shutdown_connections 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e867fe6b0 0x559e86800aa0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867ee400 0x559e867f9eb0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e9290 0x559e867f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 --2- 172.21.15.26:0/3752635246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867e88a0 0x559e867f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 >> 172.21.15.26:0/3752635246 conn(0x559e867e3f10 msgr2=0x559e867e3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f096b7fe640 0 client.19158 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 shutdown_connections 2024-05-20T21:08:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 1 -- 172.21.15.26:0/3752635246 wait complete. 2024-05-20T21:08:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.573+0000 7f09957f0740 20 client.19158 trim_cache size 0 max 16384 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 Processor -- start 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 -- start start 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e868006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867e7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1ac660 con 0x559e867e9290 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1a9f30 con 0x559e867e88a0 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f09957f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1a7d80 con 0x559e867ee400 2024-05-20T21:08:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f098b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f098bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e868006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f098b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44478/0 (socket says 172.21.15.26:44478) 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f098b7fe640 1 -- 172.21.15.26:0/1241706301 learned_addr learned my addr 172.21.15.26:0/1241706301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.591+0000 7f0990f9d640 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098b7fe640 1 -- 172.21.15.26:0/1241706301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 msgr2=0x559e867e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098b7fe640 1 -- 172.21.15.26:0/1241706301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 msgr2=0x559e868006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e868006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098b7fe640 1 -- 172.21.15.26:0/1241706301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e867ed720 con 0x559e867e9290 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098bfff640 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e868006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f098b7fe640 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f9e70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f09800a6200 tx=0x7f09800047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6862dc5cc358bb8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f0971ffb640 1 -- 172.21.15.26:0/1241706301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a8700 con 0x559e867e9290 2024-05-20T21:08:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f0971ffb640 1 -- 172.21.15.26:0/1241706301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09800a8aa0 con 0x559e867e9290 2024-05-20T21:08:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.592+0000 7f0971ffb640 1 -- 172.21.15.26:0/1241706301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a88e0 con 0x559e867e9290 2024-05-20T21:08:15.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 282578 2.33 MB/sec execute 501 sec latency 17.035 ms 2024-05-20T21:08:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 -- 172.21.15.26:0/1241706301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 msgr2=0x559e867f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f9e70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f09800a6200 tx=0x7f09800047d0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 -- 172.21.15.26:0/1241706301 shutdown_connections 2024-05-20T21:08:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f9e70 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 --2- 172.21.15.26:0/1241706301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e868006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 -- 172.21.15.26:0/1241706301 >> 172.21.15.26:0/1241706301 conn(0x559e867e3f10 msgr2=0x559e867e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 -- 172.21.15.26:0/1241706301 shutdown_connections 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 -- 172.21.15.26:0/1241706301 wait complete. 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 -- start start 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.593+0000 7f09957f0740 1 Processor -- start 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1a83f0 con 0x559e867e9290 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1a9860 con 0x559e867e88a0 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1a8ac0 con 0x559e867ee400 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f09957f0740 10 client.0 _ll_register_callbacks cb 0x7f09840315d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44482/0 (socket says 172.21.15.26:44482) 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098b7fe640 1 -- 172.21.15.26:0/2141507787 learned_addr learned my addr 172.21.15.26:0/2141507787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f0990f9d640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 -- 172.21.15.26:0/2141507787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 msgr2=0x559e867f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 -- 172.21.15.26:0/2141507787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 msgr2=0x559e867f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 -- 172.21.15.26:0/2141507787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e867f8de0 con 0x559e867e88a0 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098b7fe640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f098bfff640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867f72a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x559e868012e0 tx=0x7f097c012e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af999100579cbf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f096b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:08:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.594+0000 7f096b7fe640 1 -- 172.21.15.26:0/2141507787 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c004110 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09800a8aa0 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f096b7fe640 1 -- 172.21.15.26:0/2141507787 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f097c0042b0 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f096b7fe640 1 -- 172.21.15.26:0/2141507787 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f097c0044b0 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 10 client.28460 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e867fa160 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f096b7fe640 1 -- 172.21.15.26:0/2141507787 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09800a8aa0 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f096b7fe640 1 -- 172.21.15.26:0/2141507787 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f097c031050 con 0x559e867e88a0 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 10 client.28460 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 10 client.28460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 20 client.28460 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e86869880 0x559e8686bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 4 client.28460 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e86755360 con 0x559e86869880 2024-05-20T21:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.595+0000 7f098b7fe640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e86869880 0x559e8686bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.596+0000 7f096b7fe640 10 client.28460 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.596+0000 7f098b7fe640 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e86869880 0x559e8686bca0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fa4777e4ec7bc9 server_cookie=68cec5be9af75851 in_seq=0 out_seq=0 2024-05-20T21:08:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f096b7fe640 1 -- 172.21.15.26:0/2141507787 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e86755360 con 0x559e86869880 2024-05-20T21:08:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f096b7fe640 10 client.28460 handle_command_reply: tid=0 2024-05-20T21:08:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f09957f0740 1 client.28460 shutdown 2024-05-20T21:08:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 msgr2=0x559e867f72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f09957f0740 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867f72a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x559e868012e0 tx=0x7f097c012e00 comp rx=0 tx=0).stop 2024-05-20T21:08:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 shutdown_connections 2024-05-20T21:08:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.637+0000 7f09957f0740 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559e86869880 0x559e8686bca0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.637 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:15 smithi026 ceph-mon[31990]: pgmap v753: 97 pgs: 97 active+clean; 27 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f096b7fe640 0 client.28460 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e867ee400 0x559e867f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559e867e9290 0x559e867f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 1 --2- 172.21.15.26:0/2141507787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559e867e88a0 0x559e867f72a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 >> 172.21.15.26:0/2141507787 conn(0x559e867e3f10 msgr2=0x559e867e2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 shutdown_connections 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 1 -- 172.21.15.26:0/2141507787 wait complete. 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.638+0000 7f09957f0740 20 client.28460 trim_cache size 0 max 16384 2024-05-20T21:08:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:08:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:08:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:08:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f09740b27e0 msgr2=0x7f09740b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f09740b27e0 0x7f09740b4ca0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0984005fe0 tx=0x7f09840095c0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1029c0 msgr2=0x7f098c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1029c0 0x7f098c198fa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f098000b3f0 tx=0x7f098000b8b0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 shutdown_connections 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f09740b27e0 0x7f09740b4ca0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c103380 0x7f098c1a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f098c1029c0 0x7f098c198fa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 --2- 172.21.15.26:0/1539339985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c1017d0 0x7f098c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 >> 172.21.15.26:0/1539339985 conn(0x7f098c0fcd40 msgr2=0x7f098c06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 shutdown_connections 2024-05-20T21:08:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:15.639+0000 7f099179e640 1 -- 172.21.15.26:0/1539339985 wait complete. 2024-05-20T21:08:15.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:15 smithi055 ceph-mon[35213]: pgmap v753: 97 pgs: 97 active+clean; 27 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-20T21:08:15.736 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-20T21:08:15.736 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-20T21:08:15.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:08:15.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:15 smithi144 ceph-mon[35660]: pgmap v753: 97 pgs: 97 active+clean; 27 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-20T21:08:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 -- 172.21.15.26:0/3168772954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281088f0 msgr2=0x7f0628110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 --2- 172.21.15.26:0/3168772954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281088f0 0x7f0628110470 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f062000b0d0 tx=0x7f062000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 -- 172.21.15.26:0/3168772954 shutdown_connections 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 --2- 172.21.15.26:0/3168772954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281088f0 0x7f0628110470 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 --2- 172.21.15.26:0/3168772954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281037f0 0x7f0628103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 --2- 172.21.15.26:0/3168772954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281041a0 0x7f06281081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 -- 172.21.15.26:0/3168772954 >> 172.21.15.26:0/3168772954 conn(0x7f062807a6a0 msgr2=0x7f062807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 -- 172.21.15.26:0/3168772954 shutdown_connections 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.181+0000 7f062e531700 1 -- 172.21.15.26:0/3168772954 wait complete. 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 Processor -- start 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 -- start start 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281037f0 0x7f062819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 0x7f062819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281088f0 0x7f06281997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0628113570 con 0x7f06281041a0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f06281133f0 con 0x7f06281088f0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062e531700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06281136f0 con 0x7f06281037f0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 0x7f062819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 0x7f062819fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44500/0 (socket says 172.21.15.26:44500) 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 -- 172.21.15.26:0/3871512168 learned_addr learned my addr 172.21.15.26:0/3871512168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062cace700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281088f0 0x7f06281997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f0627fff700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281037f0 0x7f062819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 -- 172.21.15.26:0/3871512168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281037f0 msgr2=0x7f062819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281037f0 0x7f062819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 -- 172.21.15.26:0/3871512168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281088f0 msgr2=0x7f06281997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281088f0 0x7f06281997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f061ffff700 1 -- 172.21.15.26:0/3871512168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0628199fb0 con 0x7f06281041a0 2024-05-20T21:08:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.182+0000 7f062cace700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281088f0 0x7f06281997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.183+0000 7f061ffff700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 0x7f062819fb40 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f061000c9b0 tx=0x7f061000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.183+0000 7f0625ffb700 1 -- 172.21.15.26:0/3871512168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610007b00 con 0x7f06281041a0 2024-05-20T21:08:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.183+0000 7f0625ffb700 1 -- 172.21.15.26:0/3871512168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f061000ce50 con 0x7f06281041a0 2024-05-20T21:08:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.183+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062819a2a0 con 0x7f06281041a0 2024-05-20T21:08:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.183+0000 7f0625ffb700 1 -- 172.21.15.26:0/3871512168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610018660 con 0x7f06281041a0 2024-05-20T21:08:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.183+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a6460 con 0x7f06281041a0 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.186+0000 7f0625ffb700 1 -- 172.21.15.26:0/3871512168 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0610004830 con 0x7f06281041a0 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.187+0000 7f0625ffb700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06140b2880 0x7f06140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.187+0000 7f0638dc7b80 1 Processor -- start 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.187+0000 7f0627fff700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06140b2880 0x7f06140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.187+0000 7f0625ffb700 1 -- 172.21.15.26:0/3871512168 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f06100e21f0 con 0x7f06281041a0 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.187+0000 7f0638dc7b80 1 -- start start 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc5320 0x5637acfc5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfbfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc0260 0x5637acfcc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5637acf1dd70 con 0x5637acfbf8a0 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5637acf0d910 con 0x5637acfc0260 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637acf11950 con 0x5637acfc5320 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfbfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f062cace700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc0260 0x5637acfcc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0627fff700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06140b2880 0x7f06140b4d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0618009c60 tx=0x7f0618009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfbfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44514/0 (socket says 172.21.15.26:44514) 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 -- 172.21.15.26:0/3643053093 learned_addr learned my addr 172.21.15.26:0/3643053093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0627fff700 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc5320 0x5637acfc5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 -- 172.21.15.26:0/3643053093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc5320 msgr2=0x5637acfc5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc5320 0x5637acfc5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 -- 172.21.15.26:0/3643053093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc0260 msgr2=0x5637acfcc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc0260 0x5637acfcc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 -- 172.21.15.26:0/3643053093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637aceb6fb0 con 0x5637acfbf8a0 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f062cace700 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc0260 0x5637acfcc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f0627fff700 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc5320 0x5637acfc5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.188+0000 7f061ffff700 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfbfd20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f06100a5830 tx=0x7f06100a92c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7543b6338c03091 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.189+0000 7f061e7fc700 1 -- 172.21.15.26:0/3643053093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100adb60 con 0x5637acfbf8a0 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.189+0000 7f061e7fc700 1 -- 172.21.15.26:0/3643053093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06100b2610 con 0x5637acfbf8a0 2024-05-20T21:08:16.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.189+0000 7f061e7fc700 1 -- 172.21.15.26:0/3643053093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610091110 con 0x5637acfbf8a0 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.189+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3643053093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 msgr2=0x5637acfbfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.189+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfbfd20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f06100a5830 tx=0x7f06100a92c0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.189+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3643053093 shutdown_connections 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc0260 0x5637acfcc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc5320 0x5637acfc5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3643053093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfbfd20 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3643053093 >> 172.21.15.26:0/3643053093 conn(0x5637aceeb240 msgr2=0x5637acfba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3643053093 shutdown_connections 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3643053093 wait complete. 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 -- start start 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 Processor -- start 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfcbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc0260 0x5637acfc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.190+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfc95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5637ace13fa0 con 0x5637acfbf8a0 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5637ace75da0 con 0x5637acfc5320 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637acfcc6e0 con 0x5637acfc0260 2024-05-20T21:08:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0638dc7b80 10 client.0 _ll_register_callbacks cb 0x5637acea8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0627fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfcbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfc95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfc95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33968/0 (socket says 172.21.15.26:33968) 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0627fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfcbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44524/0 (socket says 172.21.15.26:44524) 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 -- 172.21.15.26:0/3957944182 learned_addr learned my addr 172.21.15.26:0/3957944182 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f061ffff700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc0260 0x5637acfc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 -- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc0260 msgr2=0x5637acfc9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc0260 0x5637acfc9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 -- 172.21.15.26:0/3957944182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 msgr2=0x5637acfcbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfcbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 -- 172.21.15.26:0/3957944182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ace5d8d0 con 0x5637acfc5320 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f0627fff700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfcbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f062cace700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfc95c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0620007760 tx=0x7f0620007790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb18bba63d638f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f06077fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.191+0000 7f06077fe700 1 -- 172.21.15.26:0/3957944182 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620007b40 con 0x5637acfc5320 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f06077fe700 1 -- 172.21.15.26:0/3957944182 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f0620007ce0 con 0x5637acfc5320 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06100b2610 con 0x5637acfc5320 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f06077fe700 1 -- 172.21.15.26:0/3957944182 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0620003a00 con 0x5637acfc5320 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 10 client.28472 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637acec7760 con 0x5637acfc5320 2024-05-20T21:08:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f06077fe700 1 -- 172.21.15.26:0/3957944182 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06100b2610 con 0x5637acfc5320 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f06077fe700 1 -- 172.21.15.26:0/3957944182 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f062002f050 con 0x5637acfc5320 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 10 client.28472 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 10 client.28472 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 20 client.28472 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637acfd9370 0x5637acfdb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 4 client.28472 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.192+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637acf1fcc0 con 0x5637acfd9370 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.193+0000 7f0627fff700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637acfd9370 0x5637acfdb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.193+0000 7f0627fff700 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637acfd9370 0x5637acfdb760 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9ae97bfe421d8be server_cookie=eb4755da83916422 in_seq=0 out_seq=0 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.193+0000 7f06077fe700 10 client.28472 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:16.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f06077fe700 1 -- 172.21.15.26:0/3957944182 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5637acf1fcc0 con 0x5637acfd9370 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f06077fe700 10 client.28472 handle_command_reply: tid=0 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 client.28472 shutdown 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 msgr2=0x5637acfc95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfc95c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0620007760 tx=0x7f0620007790 comp rx=0 tx=0).stop 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 shutdown_connections 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637acfd9370 0x5637acfdb760 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfbf8a0 0x5637acfcbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfc95c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/3957944182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfc0260 0x5637acfc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 >> 172.21.15.26:0/3957944182 conn(0x5637aceeb240 msgr2=0x5637acfb9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.194+0000 7f06077fe700 0 client.28472 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.195+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 shutdown_connections 2024-05-20T21:08:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.195+0000 7f0638dc7b80 1 -- 172.21.15.26:0/3957944182 wait complete. 2024-05-20T21:08:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.195+0000 7f0638dc7b80 20 client.28472 trim_cache size 0 max 16384 2024-05-20T21:08:16.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.215+0000 7f0638dc7b80 1 Processor -- start 2024-05-20T21:08:16.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.215+0000 7f0638dc7b80 1 -- start start 2024-05-20T21:08:16.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfc48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b0420 con 0x5637acfc0260 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f06281adcb0 con 0x5637acfc5320 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06281abb00 con 0x5637acfbf8a0 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfc48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfba270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:33970/0 (socket says 172.21.15.26:33970) 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 -- 172.21.15.26:0/722489302 learned_addr learned my addr 172.21.15.26:0/722489302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f0627fff700 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 -- 172.21.15.26:0/722489302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 msgr2=0x5637acfc7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfc7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 -- 172.21.15.26:0/722489302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 msgr2=0x5637acfc48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfc48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.216+0000 7f062cace700 1 -- 172.21.15.26:0/722489302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637acfbbe10 con 0x5637acfc5320 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.217+0000 7f062cace700 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfba270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f062000b0d0 tx=0x7f062000b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ad36790ef519bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.217+0000 7f061dffb700 1 -- 172.21.15.26:0/722489302 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200265f0 con 0x5637acfc5320 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.217+0000 7f061dffb700 1 -- 172.21.15.26:0/722489302 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0620007b40 con 0x5637acfc5320 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.217+0000 7f061dffb700 1 -- 172.21.15.26:0/722489302 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620031440 con 0x5637acfc5320 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 -- 172.21.15.26:0/722489302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 msgr2=0x5637acfba270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfba270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f062000b0d0 tx=0x7f062000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 -- 172.21.15.26:0/722489302 shutdown_connections 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfba270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfc7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/722489302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfc48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 -- 172.21.15.26:0/722489302 >> 172.21.15.26:0/722489302 conn(0x5637aceeb240 msgr2=0x5637acfc41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 -- 172.21.15.26:0/722489302 shutdown_connections 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 -- 172.21.15.26:0/722489302 wait complete. 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.218+0000 7f0638dc7b80 1 -- start start 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 Processor -- start 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfd1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfd20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfe0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06281ac170 con 0x5637acfc0260 2024-05-20T21:08:16.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f06281ad5e0 con 0x5637acfc5320 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06281ac840 con 0x5637acfbf8a0 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0638dc7b80 10 client.0 _ll_register_callbacks cb 0x5637acea8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfd20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f061ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfd20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44550/0 (socket says 172.21.15.26:44550) 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f061ffff700 1 -- 172.21.15.26:0/2810978838 learned_addr learned my addr 172.21.15.26:0/2810978838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.219+0000 7f0627fff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfd1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0627fff700 1 -- 172.21.15.26:0/2810978838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 msgr2=0x5637acfe0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f062cace700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfe0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0627fff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfe0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0627fff700 1 -- 172.21.15.26:0/2810978838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 msgr2=0x5637acfd20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0627fff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfd20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0627fff700 1 -- 172.21.15.26:0/2810978838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637acfe0a60 con 0x5637acfbf8a0 2024-05-20T21:08:16.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f061ffff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfd20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0627fff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfd1b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0618029fd0 tx=0x7f0618013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c15660671672c9d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f06077fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f06077fe700 1 -- 172.21.15.26:0/2810978838 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061802f060 con 0x5637acfbf8a0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f06077fe700 1 -- 172.21.15.26:0/2810978838 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f0618029630 con 0x5637acfbf8a0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.220+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0620007b40 con 0x5637acfbf8a0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.221+0000 7f06077fe700 1 -- 172.21.15.26:0/2810978838 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0618003e70 con 0x5637acfbf8a0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.221+0000 7f0638dc7b80 10 client.28234 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.221+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637acfd1550 con 0x5637acfbf8a0 2024-05-20T21:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.221+0000 7f06077fe700 1 -- 172.21.15.26:0/2810978838 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0620007b40 con 0x5637acfbf8a0 2024-05-20T21:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f06077fe700 1 -- 172.21.15.26:0/2810978838 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f0618033030 con 0x5637acfbf8a0 2024-05-20T21:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f0638dc7b80 10 client.28234 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f0638dc7b80 10 client.28234 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f0638dc7b80 20 client.28234 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637ad043f70 0x5637ad0463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f0638dc7b80 4 client.28234 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637acf194a0 con 0x5637ad043f70 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.222+0000 7f061ffff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637ad043f70 0x5637ad0463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f061ffff700 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637ad043f70 0x5637ad0463b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14fd7bec47b02f0c server_cookie=721168f961c1c1df in_seq=0 out_seq=0 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f06077fe700 10 client.28234 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f06077fe700 1 -- 172.21.15.26:0/2810978838 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637acf194a0 con 0x5637ad043f70 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f06077fe700 10 client.28234 handle_command_reply: tid=0 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 client.28234 shutdown 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 msgr2=0x5637acfd1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfd1b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0618029fd0 tx=0x7f0618013040 comp rx=0 tx=0).stop 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 shutdown_connections 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5637ad043f70 0x5637ad0463b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5637acfc5320 0x5637acfe0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637acfbf8a0 0x5637acfd1b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 --2- 172.21.15.26:0/2810978838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637acfc0260 0x5637acfd20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 >> 172.21.15.26:0/2810978838 conn(0x5637aceeb240 msgr2=0x5637acfbe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.223+0000 7f06077fe700 0 client.28234 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 shutdown_connections 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f0638dc7b80 1 -- 172.21.15.26:0/2810978838 wait complete. 2024-05-20T21:08:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f0638dc7b80 20 client.28234 trim_cache size 0 max 16384 2024-05-20T21:08:16.227 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:08:16.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06140b2880 msgr2=0x7f06140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06140b2880 0x7f06140b4d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0618009c60 tx=0x7f0618009260 comp rx=0 tx=0).stop 2024-05-20T21:08:16.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 msgr2=0x7f062819fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:16.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 0x7f062819fb40 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f061000c9b0 tx=0x7f061000ccc0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 shutdown_connections 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06281088f0 0x7f06281997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281037f0 0x7f062819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f06140b2880 0x7f06140b4d40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 --2- 172.21.15.26:0/3871512168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06281041a0 0x7f062819fb40 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.224+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 >> 172.21.15.26:0/3871512168 conn(0x7f062807a6a0 msgr2=0x7f06281015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.225+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 shutdown_connections 2024-05-20T21:08:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:16.225+0000 7f062e531700 1 -- 172.21.15.26:0/3871512168 wait complete. 2024-05-20T21:08:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:16.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 283208 2.33 MB/sec execute 502 sec latency 54.325 ms 2024-05-20T21:08:17.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:08:17.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 283805 2.34 MB/sec execute 503 sec latency 19.494 ms 2024-05-20T21:08:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:17 smithi055 ceph-mon[35213]: pgmap v754: 97 pgs: 97 active+clean; 28 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:08:17.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:17 smithi026 ceph-mon[31990]: pgmap v754: 97 pgs: 97 active+clean; 28 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:08:17.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:17 smithi144 ceph-mon[35660]: pgmap v754: 97 pgs: 97 active+clean; 28 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:08:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:18.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 284274 2.34 MB/sec execute 504 sec latency 20.111 ms 2024-05-20T21:08:19.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 284841 2.34 MB/sec execute 505 sec latency 21.265 ms 2024-05-20T21:08:19.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:19 smithi026 ceph-mon[31990]: pgmap v755: 97 pgs: 97 active+clean; 29 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:08:19.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:19 smithi055 ceph-mon[35213]: pgmap v755: 97 pgs: 97 active+clean; 29 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:08:19.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:19 smithi144 ceph-mon[35660]: pgmap v755: 97 pgs: 97 active+clean; 29 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 -- 172.21.15.26:0/3147989574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 msgr2=0x7fdde0109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 --2- 172.21.15.26:0/3147989574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde0109a10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fddd00099c0 tx=0x7fddd0009cd0 comp rx=0 tx=0).stop 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 -- 172.21.15.26:0/3147989574 shutdown_connections 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 --2- 172.21.15.26:0/3147989574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde0109a10 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 --2- 172.21.15.26:0/3147989574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006c6d0 0x7fdde0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 --2- 172.21.15.26:0/3147989574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006bd10 0x7fdde006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 -- 172.21.15.26:0/3147989574 >> 172.21.15.26:0/3147989574 conn(0x7fdde00794e0 msgr2=0x7fdde00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 -- 172.21.15.26:0/3147989574 shutdown_connections 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.762+0000 7fdde5502700 1 -- 172.21.15.26:0/3147989574 wait complete. 2024-05-20T21:08:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.763+0000 7fdde5502700 1 Processor -- start 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.763+0000 7fdde5502700 1 -- start start 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdde5502700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 0x7fdde019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdde5502700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006c6d0 0x7fdde019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdde5502700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdde5502700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdde01135f0 con 0x7fdde0109630 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdde5502700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0113470 con 0x7fdde006bd10 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdde5502700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0113770 con 0x7fdde006c6d0 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 0x7fdde019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 0x7fdde019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57380/0 (socket says 172.21.15.26:57380) 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 -- 172.21.15.26:0/3807106630 learned_addr learned my addr 172.21.15.26:0/3807106630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fddde7fc700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006c6d0 0x7fdde019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddf7fe700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 -- 172.21.15.26:0/3807106630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006c6d0 msgr2=0x7fdde019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006c6d0 0x7fdde019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 -- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 msgr2=0x7fdde01998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde01998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 -- 172.21.15.26:0/3807106630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde019a010 con 0x7fdde006bd10 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddf7fe700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fddde7fc700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006c6d0 0x7fdde019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.764+0000 7fdddeffd700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 0x7fdde019f6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddd0006010 tx=0x7fddd0004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.765+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0026070 con 0x7fdde006bd10 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.765+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde019a2a0 con 0x7fdde006bd10 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.765+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddd000f460 con 0x7fdde006bd10 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.765+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0009e60 con 0x7fdde006bd10 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.765+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01a65c0 con 0x7fdde006bd10 2024-05-20T21:08:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.766+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde010bac0 con 0x7fdde006bd10 2024-05-20T21:08:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.767+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fddd0029070 con 0x7fdde006bd10 2024-05-20T21:08:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.768+0000 7fddc7fff700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fddc80b2880 0x7fddc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.768+0000 7fddde7fc700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fddc80b2880 0x7fddc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.768+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fddd00e2e70 con 0x7fdde006bd10 2024-05-20T21:08:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.769+0000 7fddde7fc700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fddc80b2880 0x7fddc80b4d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005fd0 tx=0x7fddd4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:19.773+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fddd0030080 con 0x7fdde006bd10 2024-05-20T21:08:20.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.087+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdde019ade0 con 0x7fdde006bd10 2024-05-20T21:08:20.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.088+0000 7fddc7fff700 1 -- 172.21.15.26:0/3807106630 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fddd0013070 con 0x7fdde006bd10 2024-05-20T21:08:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:08:20.088 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:08:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fddc80b2880 msgr2=0x7fddc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fddc80b2880 0x7fddc80b4d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005fd0 tx=0x7fddd4005e30 comp rx=0 tx=0).stop 2024-05-20T21:08:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 msgr2=0x7fdde019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 0x7fdde019f6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddd0006010 tx=0x7fddd0004690 comp rx=0 tx=0).stop 2024-05-20T21:08:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 shutdown_connections 2024-05-20T21:08:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fddc80b2880 0x7fddc80b4d40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdde0109630 0x7fdde01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde006bd10 0x7fdde019f6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 --2- 172.21.15.26:0/3807106630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdde006c6d0 0x7fdde019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 >> 172.21.15.26:0/3807106630 conn(0x7fdde00794e0 msgr2=0x7fdde0101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 shutdown_connections 2024-05-20T21:08:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:20.091+0000 7fdde5502700 1 -- 172.21.15.26:0/3807106630 wait complete. 2024-05-20T21:08:20.092 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:08:20.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 285560 2.34 MB/sec execute 506 sec latency 15.739 ms 2024-05-20T21:08:20.621 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:20 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3807106630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:20.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:20 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3807106630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:20.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:20 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3807106630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:21.346 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-20T21:08:21.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-20T21:08:21.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 286215 2.34 MB/sec execute 507 sec latency 16.301 ms 2024-05-20T21:08:21.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:21 smithi026 ceph-mon[31990]: pgmap v756: 97 pgs: 97 active+clean; 25 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:08:21.700 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:21 smithi144 ceph-mon[35660]: pgmap v756: 97 pgs: 97 active+clean; 25 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:08:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:21 smithi055 ceph-mon[35213]: pgmap v756: 97 pgs: 97 active+clean; 25 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-05-20T21:08:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 -- 172.21.15.26:0/618493017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 msgr2=0x7f9a4c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/618493017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1088f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0098f0 tx=0x7f9a3c009c00 comp rx=0 tx=0).stop 2024-05-20T21:08:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 -- 172.21.15.26:0/618493017 shutdown_connections 2024-05-20T21:08:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/618493017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102df0 0x7f9a4c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/618493017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1088f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/618493017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a4c102430 0x7f9a4c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 -- 172.21.15.26:0/618493017 >> 172.21.15.26:0/618493017 conn(0x7f9a4c0793b0 msgr2=0x7f9a4c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 -- 172.21.15.26:0/618493017 shutdown_connections 2024-05-20T21:08:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.859+0000 7f9a51cd8700 1 -- 172.21.15.26:0/618493017 wait complete. 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.860+0000 7f9a51cd8700 1 Processor -- start 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.860+0000 7f9a51cd8700 1 -- start start 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a51cd8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 0x7f9a4c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a51cd8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a4c102df0 0x7f9a4c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a51cd8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a51cd8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c112470 con 0x7f9a4c102430 2024-05-20T21:08:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a51cd8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1122f0 con 0x7f9a4c108510 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a51cd8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1125f0 con 0x7f9a4c102df0 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a4b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 0x7f9a4c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a4b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 0x7f9a4c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55874/0 (socket says 172.21.15.26:55874) 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57404/0 (socket says 172.21.15.26:57404) 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1499234454 learned_addr learned my addr 172.21.15.26:0/1499234454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.861+0000 7f9a4affd700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a4c102df0 0x7f9a4c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1499234454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a4c102df0 msgr2=0x7f9a4c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a4c102df0 0x7f9a4c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 msgr2=0x7f9a4c19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 0x7f9a4c19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1499234454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c1a0da0 con 0x7f9a4c108510 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1a0660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e3c0 tx=0x7f9a4400e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 0x7f9a4c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a48ff9700 1 -- 172.21.15.26:0/1499234454 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440092d0 con 0x7f9a4c108510 2024-05-20T21:08:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a48ff9700 1 -- 172.21.15.26:0/1499234454 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a4400ee70 con 0x7f9a4c108510 2024-05-20T21:08:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a48ff9700 1 -- 172.21.15.26:0/1499234454 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44007d80 con 0x7f9a4c108510 2024-05-20T21:08:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c1a1090 con 0x7f9a4c108510 2024-05-20T21:08:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.862+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a8910 con 0x7f9a4c108510 2024-05-20T21:08:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.865+0000 7f9a48ff9700 1 -- 172.21.15.26:0/1499234454 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9a44012050 con 0x7f9a4c108510 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.866+0000 7f9a48ff9700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9a340b2980 0x7f9a340b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.866+0000 7f9a48ff9700 1 -- 172.21.15.26:0/1499234454 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9a440e1cd0 con 0x7f9a4c108510 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.866+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9a340b2980 0x7f9a340b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.866+0000 7f9a5c56eb80 1 Processor -- start 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.866+0000 7f9a5c56eb80 1 -- start start 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89c3320 0x55baa89c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 0x55baa89ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55baa891bd70 con 0x55baa89c3320 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55baa890b910 con 0x55baa89be260 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55baa890f950 con 0x55baa89bd8a0 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 0x55baa89ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 0x55baa89ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57412/0 (socket says 172.21.15.26:57412) 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1015280658 learned_addr learned my addr 172.21.15.26:0/1015280658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4affd700 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1015280658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 msgr2=0x55baa89bdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89bdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1015280658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89c3320 msgr2=0x55baa89c3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89c3320 0x55baa89c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89c3320 0x55baa89c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4bfff700 1 -- 172.21.15.26:0/1015280658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa88b4fb0 con 0x55baa89be260 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.867+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89c3320 0x55baa89c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.868+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9a340b2980 0x7f9a340b4e40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009fd0 tx=0x7f9a3c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.868+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 0x55baa89ca080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a9160 tx=0x7f9a440a9190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fbc0f75684ca59 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:21.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.868+0000 7f9a31ffb700 1 -- 172.21.15.26:0/1015280658 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440a99c0 con 0x55baa89be260 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.868+0000 7f9a31ffb700 1 -- 172.21.15.26:0/1015280658 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a440a92c0 con 0x55baa89be260 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.868+0000 7f9a31ffb700 1 -- 172.21.15.26:0/1015280658 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440dfbd0 con 0x55baa89be260 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/1015280658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 msgr2=0x55baa89ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 0x55baa89ca080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a9160 tx=0x7f9a440a9190 comp rx=0 tx=0).stop 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/1015280658 shutdown_connections 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89c3320 0x55baa89c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89be260 0x55baa89ca080 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/1015280658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/1015280658 >> 172.21.15.26:0/1015280658 conn(0x55baa88e9240 msgr2=0x55baa89b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/1015280658 shutdown_connections 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.869+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/1015280658 wait complete. 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 -- start start 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 Processor -- start 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89bd8a0 0x55baa89c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89be260 0x55baa89c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55baa8811fa0 con 0x55baa89bd8a0 2024-05-20T21:08:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55baa8873da0 con 0x55baa89c3320 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.870+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55baa89ca6e0 con 0x55baa89be260 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a5c56eb80 10 client.0 _ll_register_callbacks cb 0x55baa88a6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89bd8a0 0x55baa89c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89bd8a0 0x55baa89c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55894/0 (socket says 172.21.15.26:55894) 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57414/0 (socket says 172.21.15.26:57414) 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4b7fe700 1 -- 172.21.15.26:0/419327752 learned_addr learned my addr 172.21.15.26:0/419327752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4affd700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89be260 0x55baa89c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 -- 172.21.15.26:0/419327752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89be260 msgr2=0x55baa89c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89be260 0x55baa89c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 -- 172.21.15.26:0/419327752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89bd8a0 msgr2=0x55baa89c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89bd8a0 0x55baa89c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 -- 172.21.15.26:0/419327752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa885b8d0 con 0x55baa89c3320 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4affd700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89be260 0x55baa89c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89c7570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a9160 tx=0x7f9a44007f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a82682be500fd90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.871+0000 7f9a2affd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a2affd700 1 -- 172.21.15.26:0/419327752 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440a99c0 con 0x55baa89c3320 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a2affd700 1 -- 172.21.15.26:0/419327752 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9a440a9730 con 0x55baa89c3320 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a440a92c0 con 0x55baa89c3320 2024-05-20T21:08:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a2affd700 1 -- 172.21.15.26:0/419327752 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a440ca1a0 con 0x55baa89c3320 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 10 client.28493 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55baa88c5760 con 0x55baa89c3320 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a2affd700 1 -- 172.21.15.26:0/419327752 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a440a92c0 con 0x55baa89c3320 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a2affd700 1 -- 172.21.15.26:0/419327752 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9a440c3b20 con 0x55baa89c3320 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 10 client.28493 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 10 client.28493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 20 client.28493 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa89d7090 0x55baa89d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 4 client.28493 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.872+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa891dcc0 con 0x55baa89d7090 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.873+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa89d7090 0x55baa89d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.873+0000 7f9a2affd700 10 client.28493 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.873+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa89d7090 0x55baa89d9480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2fbec4226bad5dc server_cookie=fa1c26b933f7f0cf in_seq=0 out_seq=0 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a2affd700 1 -- 172.21.15.26:0/419327752 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55baa891dcc0 con 0x55baa89d7090 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a2affd700 10 client.28493 handle_command_reply: tid=0 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 client.28493 shutdown 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 msgr2=0x55baa89c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89c7570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a9160 tx=0x7f9a44007f20 comp rx=0 tx=0).stop 2024-05-20T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 shutdown_connections 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa89d7090 0x55baa89d9480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89bd8a0 0x55baa89c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89c7570 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/419327752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89be260 0x55baa89c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a2affd700 0 client.28493 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 >> 172.21.15.26:0/419327752 conn(0x55baa88e9240 msgr2=0x55baa89b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 shutdown_connections 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/419327752 wait complete. 2024-05-20T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.874+0000 7f9a5c56eb80 20 client.28493 trim_cache size 0 max 16384 2024-05-20T21:08:21.886 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.894+0000 7f9a5c56eb80 1 Processor -- start 2024-05-20T21:08:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.894+0000 7f9a5c56eb80 1 -- start start 2024-05-20T21:08:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b2870 con 0x55baa89be260 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b0100 con 0x55baa89c3320 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1adf50 con 0x55baa89bd8a0 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55896/0 (socket says 172.21.15.26:55896) 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4affd700 1 -- 172.21.15.26:0/2861328265 learned_addr learned my addr 172.21.15.26:0/2861328265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4b7fe700 1 -- 172.21.15.26:0/2861328265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 msgr2=0x55baa89d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89d2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4b7fe700 1 -- 172.21.15.26:0/2861328265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 msgr2=0x55baa89c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.895+0000 7f9a4b7fe700 1 -- 172.21.15.26:0/2861328265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa89c3a70 con 0x55baa89bd8a0 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.896+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89b8270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55baa89ca770 tx=0x7f9a3c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d8e413c36bf712 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.896+0000 7f9a317fa700 1 -- 172.21.15.26:0/2861328265 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0050b0 con 0x55baa89bd8a0 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.896+0000 7f9a317fa700 1 -- 172.21.15.26:0/2861328265 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a3c035070 con 0x55baa89bd8a0 2024-05-20T21:08:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.896+0000 7f9a317fa700 1 -- 172.21.15.26:0/2861328265 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00b560 con 0x55baa89bd8a0 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/2861328265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 msgr2=0x55baa89b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89b8270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55baa89ca770 tx=0x7f9a3c01d000 comp rx=0 tx=0).stop 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/2861328265 shutdown_connections 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89c3320 0x55baa89d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89bd8a0 0x55baa89b8270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/2861328265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/2861328265 >> 172.21.15.26:0/2861328265 conn(0x55baa88e9240 msgr2=0x55baa89c57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/2861328265 shutdown_connections 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/2861328265 wait complete. 2024-05-20T21:08:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.897+0000 7f9a5c56eb80 1 -- start start 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 Processor -- start 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89bd8a0 0x55baa89d7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89c3320 0x55baa8a3f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1ae5c0 con 0x55baa89be260 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1afa30 con 0x55baa89bd8a0 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1aec90 con 0x55baa89c3320 2024-05-20T21:08:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a5c56eb80 10 client.0 _ll_register_callbacks cb 0x55baa88a6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a4b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89bd8a0 0x55baa89d7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89c3320 0x55baa8a3f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a4b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89bd8a0 0x55baa89d7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57442/0 (socket says 172.21.15.26:57442) 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a4b7fe700 1 -- 172.21.15.26:0/3299045642 learned_addr learned my addr 172.21.15.26:0/3299045642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.898+0000 7f9a4affd700 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a4bfff700 1 -- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89bd8a0 msgr2=0x55baa89d7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89bd8a0 0x55baa89d7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a4bfff700 1 -- 172.21.15.26:0/3299045642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 msgr2=0x55baa89d7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89d7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a4bfff700 1 -- 172.21.15.26:0/3299045642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa8a3fdf0 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a4bfff700 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89c3320 0x55baa8a3f8b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a5340 tx=0x7f9a440a8d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17da7a043197ecec server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a2affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a2affd700 1 -- 172.21.15.26:0/3299045642 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440a99c0 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a2affd700 1 -- 172.21.15.26:0/3299045642 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9a440a58b0 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.899+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3c035070 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a2affd700 1 -- 172.21.15.26:0/3299045642 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a440a92c0 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 10 client.28255 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55baa8a410d0 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a2affd700 1 -- 172.21.15.26:0/3299045642 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a44087020 con 0x55baa89c3320 2024-05-20T21:08:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a2affd700 1 -- 172.21.15.26:0/3299045642 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9a440dfbd0 con 0x55baa89c3320 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 10 client.28255 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 10 client.28255 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 20 client.28255 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa8a415e0 0x55baa8a439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 4 client.28255 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.900+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9a3c035070 con 0x55baa8a415e0 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.901+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa8a415e0 0x55baa8a439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:21.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.901+0000 7f9a2affd700 10 client.28255 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.901+0000 7f9a4b7fe700 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa8a415e0 0x55baa8a439d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53e5797a2d7fe7eb server_cookie=3b6a2ef382b8ed21 in_seq=0 out_seq=0 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.901+0000 7f9a2affd700 1 -- 172.21.15.26:0/3299045642 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9a3c035070 con 0x55baa8a415e0 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.901+0000 7f9a2affd700 10 client.28255 handle_command_reply: tid=0 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.901+0000 7f9a5c56eb80 1 client.28255 shutdown 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89c3320 msgr2=0x55baa8a3f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89c3320 0x55baa8a3f8b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a5340 tx=0x7f9a440a8d10 comp rx=0 tx=0).stop 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 shutdown_connections 2024-05-20T21:08:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55baa8a415e0 0x55baa8a439d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a2affd700 0 client.28255 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55baa89c3320 0x55baa8a3f8b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55baa89bd8a0 0x55baa89d7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 --2- 172.21.15.26:0/3299045642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55baa89be260 0x55baa89d7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 >> 172.21.15.26:0/3299045642 conn(0x55baa88e9240 msgr2=0x55baa89c6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 shutdown_connections 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 1 -- 172.21.15.26:0/3299045642 wait complete. 2024-05-20T21:08:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.902+0000 7f9a5c56eb80 20 client.28255 trim_cache size 0 max 16384 2024-05-20T21:08:21.909 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:08:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9a340b2980 msgr2=0x7f9a340b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9a340b2980 0x7f9a340b4e40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009fd0 tx=0x7f9a3c019040 comp rx=0 tx=0).stop 2024-05-20T21:08:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 msgr2=0x7f9a4c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1a0660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e3c0 tx=0x7f9a4400e6d0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 shutdown_connections 2024-05-20T21:08:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9a340b2980 0x7f9a340b4e40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c102430 0x7f9a4c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c108510 0x7f9a4c1a0660 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 --2- 172.21.15.26:0/1499234454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a4c102df0 0x7f9a4c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 >> 172.21.15.26:0/1499234454 conn(0x7f9a4c0793b0 msgr2=0x7f9a4c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 shutdown_connections 2024-05-20T21:08:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:21.903+0000 7f9a51cd8700 1 -- 172.21.15.26:0/1499234454 wait complete. 2024-05-20T21:08:22.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 287076 2.35 MB/sec execute 508 sec latency 15.420 ms 2024-05-20T21:08:22.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:08:23.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:23 smithi026 ceph-mon[31990]: pgmap v757: 97 pgs: 97 active+clean; 25 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-05-20T21:08:23.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 287601 2.35 MB/sec execute 509 sec latency 15.273 ms 2024-05-20T21:08:23.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:23 smithi055 ceph-mon[35213]: pgmap v757: 97 pgs: 97 active+clean; 25 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-05-20T21:08:23.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:23 smithi144 ceph-mon[35660]: pgmap v757: 97 pgs: 97 active+clean; 25 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-05-20T21:08:24.015 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:24.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288055 2.35 MB/sec execute 510 sec latency 22.891 ms 2024-05-20T21:08:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1672002010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 msgr2=0x7fa0d0101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1672002010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d0101fb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009990 tx=0x7fa0c0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1672002010 shutdown_connections 2024-05-20T21:08:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1672002010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d01024f0 0x7fa0d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1672002010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 0x7fa0d0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1672002010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d0101fb0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1672002010 >> 172.21.15.26:0/1672002010 conn(0x7fa0d0078ae0 msgr2=0x7fa0d00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1672002010 shutdown_connections 2024-05-20T21:08:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.224+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1672002010 wait complete. 2024-05-20T21:08:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 Processor -- start 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 -- start start 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d01024f0 0x7fa0d019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 0x7fa0d0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01135b0 con 0x7fa0d0101b30 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0113430 con 0x7fa0d01095b0 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d7c39700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0113730 con 0x7fa0d01024f0 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.225+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55942/0 (socket says 172.21.15.26:55942) 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d59d5700 1 -- 172.21.15.26:0/1328628197 learned_addr learned my addr 172.21.15.26:0/1328628197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 0x7fa0d0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d01024f0 0x7fa0d019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 -- 172.21.15.26:0/1328628197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d01024f0 msgr2=0x7fa0d019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d01024f0 0x7fa0d019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 -- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 msgr2=0x7fa0d019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 -- 172.21.15.26:0/1328628197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d0199fd0 con 0x7fa0d01095b0 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 0x7fa0d0199870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8007f10 tx=0x7fa0c800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0c6ffd700 1 -- 172.21.15.26:0/1328628197 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8004db0 con 0x7fa0d01095b0 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.226+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d019a2c0 con 0x7fa0d01095b0 2024-05-20T21:08:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.227+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d01a6580 con 0x7fa0d01095b0 2024-05-20T21:08:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.227+0000 7fa0c6ffd700 1 -- 172.21.15.26:0/1328628197 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0c800d910 con 0x7fa0d01095b0 2024-05-20T21:08:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.227+0000 7fa0c6ffd700 1 -- 172.21.15.26:0/1328628197 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8004440 con 0x7fa0d01095b0 2024-05-20T21:08:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.229+0000 7fa0c6ffd700 1 -- 172.21.15.26:0/1328628197 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa0c801c020 con 0x7fa0d01095b0 2024-05-20T21:08:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.230+0000 7fa0c6ffd700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0bc0b29b0 0x7fa0bc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.230+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0bc0b29b0 0x7fa0bc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.230+0000 7fa0c6ffd700 1 -- 172.21.15.26:0/1328628197 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa0c80e1870 con 0x7fa0d01095b0 2024-05-20T21:08:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 Processor -- start 2024-05-20T21:08:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0bc0b29b0 0x7fa0bc0b4e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009680 tx=0x7fa0cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 -- start start 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d097c7b0 0x55f9d097cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0978100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d09846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f9d08d5f00 con 0x55f9d0978640 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f9d08c5aa0 con 0x55f9d097c7b0 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.231+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9d08c9ae0 con 0x55f9d0977c80 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d61d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d09846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d097c7b0 0x55f9d097cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d61d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d09846b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55956/0 (socket says 172.21.15.26:55956) 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d61d6700 1 -- 172.21.15.26:0/3077712326 learned_addr learned my addr 172.21.15.26:0/3077712326 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0978100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3077712326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 msgr2=0x55f9d0978100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0978100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3077712326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 msgr2=0x55f9d09846b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d09846b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3077712326 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d086f140 con 0x55f9d097c7b0 2024-05-20T21:08:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d097c7b0 0x55f9d097cb90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa0d019a9b0 tx=0x7fa0cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd3fb9899076c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0bbfff700 1 -- 172.21.15.26:0/3077712326 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc01e420 con 0x55f9d097c7b0 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0bbfff700 1 -- 172.21.15.26:0/3077712326 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc01ea50 con 0x55f9d097c7b0 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.232+0000 7fa0bbfff700 1 -- 172.21.15.26:0/3077712326 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc026810 con 0x55f9d097c7b0 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3077712326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d097c7b0 msgr2=0x55f9d097cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d097c7b0 0x55f9d097cb90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa0d019a9b0 tx=0x7fa0cc014000 comp rx=0 tx=0).stop 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3077712326 shutdown_connections 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d09846b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d097c7b0 0x55f9d097cb90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3077712326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0978100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3077712326 >> 172.21.15.26:0/3077712326 conn(0x55f9d08a33d0 msgr2=0x55f9d0972970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3077712326 shutdown_connections 2024-05-20T21:08:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.233+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3077712326 wait complete. 2024-05-20T21:08:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 -- start start 2024-05-20T21:08:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 Processor -- start 2024-05-20T21:08:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d0984390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d0981690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d0981bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f9d07cc130 con 0x55f9d0978640 2024-05-20T21:08:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f9d082df30 con 0x55f9d0977c80 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.234+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9d0984d10 con 0x55f9d097c7b0 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0e24cfb80 10 client.0 _ll_register_callbacks cb 0x55f9d0860ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d0984390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d0984390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57466/0 (socket says 172.21.15.26:57466) 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3052901173 learned_addr learned my addr 172.21.15.26:0/3052901173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d51d4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d0981690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d0981bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3052901173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 msgr2=0x55f9d0981bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d0981bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3052901173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 msgr2=0x55f9d0981690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d0981690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 -- 172.21.15.26:0/3052901173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d0815a60 con 0x55f9d0977c80 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d0981bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d0984390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc01e650 tx=0x7fa0cc01e680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f85686680fe5bc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0b97fa700 1 -- 172.21.15.26:0/3052901173 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc034060 con 0x55f9d0977c80 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0b97fa700 1 -- 172.21.15.26:0/3052901173 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa0cc009040 con 0x55f9d0977c80 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.235+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc01ea50 con 0x55f9d0977c80 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0b97fa700 1 -- 172.21.15.26:0/3052901173 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc026640 con 0x55f9d0977c80 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 10 client.28511 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9d087f8f0 con 0x55f9d0977c80 2024-05-20T21:08:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0b97fa700 1 -- 172.21.15.26:0/3052901173 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc01ea50 con 0x55f9d0977c80 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0b97fa700 1 -- 172.21.15.26:0/3052901173 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fa0cc038030 con 0x55f9d0977c80 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 10 client.28511 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 10 client.28511 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 20 client.28511 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09918a0 0x55f9d0993c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 4 client.28511 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d08d7e50 con 0x55f9d09918a0 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.236+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09918a0 0x55f9d0993c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.237+0000 7fa0b97fa700 10 client.28511 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.237+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09918a0 0x55f9d0993c90 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36fceb92d1a90d32 server_cookie=d04bc609fc41c30a in_seq=0 out_seq=0 2024-05-20T21:08:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0b97fa700 1 -- 172.21.15.26:0/3052901173 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f9d08d7e50 con 0x55f9d09918a0 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0b97fa700 10 client.28511 handle_command_reply: tid=0 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 client.28511 shutdown 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 msgr2=0x55f9d0984390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d0984390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc01e650 tx=0x7fa0cc01e680 comp rx=0 tx=0).stop 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 shutdown_connections 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09918a0 0x55f9d0993c90 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d0981bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d0984390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/3052901173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d0981690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0b97fa700 0 client.28511 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 >> 172.21.15.26:0/3052901173 conn(0x55f9d08a33d0 msgr2=0x55f9d0976c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.238+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 shutdown_connections 2024-05-20T21:08:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.239+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/3052901173 wait complete. 2024-05-20T21:08:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.239+0000 7fa0e24cfb80 20 client.28511 trim_cache size 0 max 16384 2024-05-20T21:08:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 Processor -- start 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 -- start start 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d096cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d097d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d097d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01ae2d0 con 0x55f9d0978640 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01abb60 con 0x55f9d0977c80 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a99b0 con 0x55f9d097c7b0 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d096cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d096cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57476/0 (socket says 172.21.15.26:57476) 2024-05-20T21:08:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0d59d5700 1 -- 172.21.15.26:0/2405568326 learned_addr learned my addr 172.21.15.26:0/2405568326 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d097d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d097d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.263+0000 7fa0d59d5700 1 -- 172.21.15.26:0/2405568326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 msgr2=0x55f9d097d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d097d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0d59d5700 1 -- 172.21.15.26:0/2405568326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 msgr2=0x55f9d097d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d097d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0d59d5700 1 -- 172.21.15.26:0/2405568326 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d0984b90 con 0x55f9d0977c80 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0d61d6700 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d097d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:25.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d096cf00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc039dd0 tx=0x7fa0cc0057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820423b9337c4f62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0bb7fe700 1 -- 172.21.15.26:0/2405568326 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc034040 con 0x55f9d0977c80 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0bb7fe700 1 -- 172.21.15.26:0/2405568326 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc0092d0 con 0x55f9d0977c80 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.264+0000 7fa0bb7fe700 1 -- 172.21.15.26:0/2405568326 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0052c0 con 0x55f9d0977c80 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2405568326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 msgr2=0x55f9d096cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d096cf00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc039dd0 tx=0x7fa0cc0057d0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2405568326 shutdown_connections 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d097c7b0 0x55f9d097d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0977c80 0x55f9d096cf00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2405568326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d0978640 0x55f9d097d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2405568326 >> 172.21.15.26:0/2405568326 conn(0x55f9d08a33d0 msgr2=0x55f9d0983750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2405568326 shutdown_connections 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2405568326 wait complete. 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.265+0000 7fa0e24cfb80 1 -- start start 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 Processor -- start 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0996fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 0x55f9d09974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d097c7b0 0x55f9d0995560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01aa020 con 0x55f9d097c7b0 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01ab490 con 0x55f9d0978640 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01aa6f0 con 0x55f9d0977c80 2024-05-20T21:08:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0d61d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d097c7b0 0x55f9d0995560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0e24cfb80 10 client.0 _ll_register_callbacks cb 0x55f9d0860ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0d51d4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 0x55f9d09974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0996fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0d59d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0996fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46696/0 (socket says 172.21.15.26:46696) 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0d51d4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 0x55f9d09974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57488/0 (socket says 172.21.15.26:57488) 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.266+0000 7fa0d59d5700 1 -- 172.21.15.26:0/2690009014 learned_addr learned my addr 172.21.15.26:0/2690009014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0d51d4700 1 -- 172.21.15.26:0/2690009014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 msgr2=0x55f9d0996fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0996fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0d51d4700 1 -- 172.21.15.26:0/2690009014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d097c7b0 msgr2=0x55f9d0995560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d097c7b0 0x55f9d0995560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0d51d4700 1 -- 172.21.15.26:0/2690009014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d0995da0 con 0x55f9d0978640 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0d51d4700 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 0x55f9d09974f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000b650 tx=0x7fa0c0005330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9da470d6b7844d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0b97fa700 1 -- 172.21.15.26:0/2690009014 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003a90 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0b97fa700 1 -- 172.21.15.26:0/2690009014 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa0c0003c30 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc0092d0 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0b97fa700 1 -- 172.21.15.26:0/2690009014 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0c0003e50 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0e24cfb80 10 client.28517 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.267+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9d09fcd70 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0b97fa700 1 -- 172.21.15.26:0/2690009014 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc0092d0 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0b97fa700 1 -- 172.21.15.26:0/2690009014 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fa0c000f740 con 0x55f9d0978640 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0e24cfb80 10 client.28517 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0e24cfb80 10 client.28517 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0e24cfb80 20 client.28517 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09fd520 0x55f9d09ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0e24cfb80 4 client.28517 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-20T21:08:25.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55f9d08d1630 con 0x55f9d09fd520 2024-05-20T21:08:25.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09fd520 0x55f9d09ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:25.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0b97fa700 10 client.28517 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:25.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.268+0000 7fa0d59d5700 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09fd520 0x55f9d09ff940 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8d4e7a1c04dffe server_cookie=8b69b0dae20f8ada in_seq=0 out_seq=0 2024-05-20T21:08:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0b97fa700 1 -- 172.21.15.26:0/2690009014 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f9d08d1630 con 0x55f9d09fd520 2024-05-20T21:08:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0b97fa700 10 client.28517 handle_command_reply: tid=0 2024-05-20T21:08:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 client.28517 shutdown 2024-05-20T21:08:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 msgr2=0x55f9d09974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 0x55f9d09974f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000b650 tx=0x7fa0c0005330 comp rx=0 tx=0).stop 2024-05-20T21:08:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 shutdown_connections 2024-05-20T21:08:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f9d09fd520 0x55f9d09ff940 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.368 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:25 smithi026 ceph-mon[31990]: pgmap v758: 97 pgs: 97 active+clean; 26 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-05-20T21:08:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9d097c7b0 0x55f9d0995560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0977c80 0x55f9d0996fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 --2- 172.21.15.26:0/2690009014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f9d0978640 0x55f9d09974f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 >> 172.21.15.26:0/2690009014 conn(0x55f9d08a33d0 msgr2=0x55f9d097fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.367+0000 7fa0b97fa700 0 client.28517 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.368+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 shutdown_connections 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.368+0000 7fa0e24cfb80 1 -- 172.21.15.26:0/2690009014 wait complete. 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.368+0000 7fa0e24cfb80 20 client.28517 trim_cache size 0 max 16384 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 46, 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "ec03d4bd": { 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 46, 2024-05-20T21:08:25.370 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:08:25.371 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:25.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:25.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:25.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:25.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:25.372 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0bc0b29b0 msgr2=0x7fa0bc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0bc0b29b0 0x7fa0bc0b4e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009680 tx=0x7fa0cc009260 comp rx=0 tx=0).stop 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 msgr2=0x7fa0d0199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 0x7fa0d0199870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8007f10 tx=0x7fa0c800d3b0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 shutdown_connections 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0bc0b29b0 0x7fa0bc0b4e70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0d0101b30 0x7fa0d019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0d01095b0 0x7fa0d0199870 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 --2- 172.21.15.26:0/1328628197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d01024f0 0x7fa0d019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 >> 172.21.15.26:0/1328628197 conn(0x7fa0d0078ae0 msgr2=0x7fa0d01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 shutdown_connections 2024-05-20T21:08:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:25.369+0000 7fa0d7c39700 1 -- 172.21.15.26:0/1328628197 wait complete. 2024-05-20T21:08:25.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.35 MB/sec execute 511 sec latency 224.335 ms 2024-05-20T21:08:25.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:25 smithi055 ceph-mon[35213]: pgmap v758: 97 pgs: 97 active+clean; 26 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-05-20T21:08:25.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:25 smithi144 ceph-mon[35660]: pgmap v758: 97 pgs: 97 active+clean; 26 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-05-20T21:08:26.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.34 MB/sec execute 512 sec latency 1224.429 ms 2024-05-20T21:08:26.667 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ec03d4bd', quiesce took 10.9 seconds. Will release after: 19.1 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 -- 172.21.15.26:0/477619007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658108910 msgr2=0x7fb658110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/477619007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658108910 0x7fb658110490 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b0d0 tx=0x7fb65000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 -- 172.21.15.26:0/477619007 shutdown_connections 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/477619007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658108910 0x7fb658110490 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/477619007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb658103820 0x7fb658103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/477619007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6581041d0 0x7fb6581081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 -- 172.21.15.26:0/477619007 >> 172.21.15.26:0/477619007 conn(0x7fb65807a6b0 msgr2=0x7fb65807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 -- 172.21.15.26:0/477619007 shutdown_connections 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.115+0000 7fb65e7b8700 1 -- 172.21.15.26:0/477619007 wait complete. 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 Processor -- start 2024-05-20T21:08:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 -- start start 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 0x7fb65819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6581041d0 0x7fb65819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb658108910 0x7fb658199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113610 con 0x7fb658103820 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113490 con 0x7fb658108910 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65e7b8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113790 con 0x7fb6581041d0 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb657fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 0x7fb65819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb657fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 0x7fb65819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55990/0 (socket says 172.21.15.26:55990) 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb657fff700 1 -- 172.21.15.26:0/1868866833 learned_addr learned my addr 172.21.15.26:0/1868866833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb64ffff700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6581041d0 0x7fb65819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.116+0000 7fb65cd55700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb658108910 0x7fb658199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb657fff700 1 -- 172.21.15.26:0/1868866833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6581041d0 msgr2=0x7fb65819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb657fff700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6581041d0 0x7fb65819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb657fff700 1 -- 172.21.15.26:0/1868866833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb658108910 msgr2=0x7fb658199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb657fff700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb658108910 0x7fb658199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb657fff700 1 -- 172.21.15.26:0/1868866833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb658199ff0 con 0x7fb658103820 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb657fff700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 0x7fb65819f6d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b530 tx=0x7fb64800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb655ffb700 1 -- 172.21.15.26:0/1868866833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648015020 con 0x7fb658103820 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb655ffb700 1 -- 172.21.15.26:0/1868866833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb648011cc0 con 0x7fb658103820 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65819a2e0 con 0x7fb658103820 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb655ffb700 1 -- 172.21.15.26:0/1868866833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648010c80 con 0x7fb658103820 2024-05-20T21:08:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.117+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a65a0 con 0x7fb658103820 2024-05-20T21:08:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.120+0000 7fb655ffb700 1 -- 172.21.15.26:0/1868866833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb648018020 con 0x7fb658103820 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.120+0000 7fb655ffb700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6440b2880 0x7fb6440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.120+0000 7fb64ffff700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6440b2880 0x7fb6440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb655ffb700 1 -- 172.21.15.26:0/1868866833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb6480e1950 con 0x7fb658103820 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 Processor -- start 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 -- start start 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb64ffff700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6440b2880 0x7fb6440b4d40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb658108340 tx=0x7fb64000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d52aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d4d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b4e2ca6d70 con 0x55b4e2d4d670 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b4e2c96910 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb66904eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b4e2c9a950 con 0x55b4e2d526c0 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb65cd55700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb65cd55700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d59460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56002/0 (socket says 172.21.15.26:56002) 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb65cd55700 1 -- 172.21.15.26:0/2503913639 learned_addr learned my addr 172.21.15.26:0/2503913639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb64ffff700 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d4d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb657fff700 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d52aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb65cd55700 1 -- 172.21.15.26:0/2503913639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 msgr2=0x55b4e2d52aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.121+0000 7fb65cd55700 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d52aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb65cd55700 1 -- 172.21.15.26:0/2503913639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4ccb0 msgr2=0x55b4e2d4d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb65cd55700 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d4d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb65cd55700 1 -- 172.21.15.26:0/2503913639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e2c3ffb0 con 0x55b4e2d4d670 2024-05-20T21:08:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb65cd55700 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d59460 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb650005ba0 tx=0x7fb6500090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6ced515ef00d24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb64e7fc700 1 -- 172.21.15.26:0/2503913639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650003a00 con 0x55b4e2d4d670 2024-05-20T21:08:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb64e7fc700 1 -- 172.21.15.26:0/2503913639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb650020070 con 0x55b4e2d4d670 2024-05-20T21:08:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.122+0000 7fb64e7fc700 1 -- 172.21.15.26:0/2503913639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650005070 con 0x55b4e2d4d670 2024-05-20T21:08:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 -- 172.21.15.26:0/2503913639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 msgr2=0x55b4e2d59460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d59460 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb650005ba0 tx=0x7fb6500090d0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 -- 172.21.15.26:0/2503913639 shutdown_connections 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d59460 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d52aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 --2- 172.21.15.26:0/2503913639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d4d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 -- 172.21.15.26:0/2503913639 >> 172.21.15.26:0/2503913639 conn(0x55b4e2c74240 msgr2=0x55b4e2d433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 -- 172.21.15.26:0/2503913639 shutdown_connections 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 -- 172.21.15.26:0/2503913639 wait complete. 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 -- start start 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.123+0000 7fb66904eb80 1 Processor -- start 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b4e2b9cfa0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b4e2bfeda0 con 0x55b4e2d4d670 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b4e2d59af0 con 0x55b4e2d526c0 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 10 client.0 _ll_register_callbacks cb 0x55b4e2c31950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56018/0 (socket says 172.21.15.26:56018) 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 -- 172.21.15.26:0/3556009295 learned_addr learned my addr 172.21.15.26:0/3556009295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 -- 172.21.15.26:0/3556009295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 msgr2=0x55b4e2d568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 -- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 msgr2=0x55b4e2d563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 -- 172.21.15.26:0/3556009295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e2be68d0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb657fff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d590a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb6480df760 tx=0x7fb6480a8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07cef5cb29cec95 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb6377fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb6377fe700 1 -- 172.21.15.26:0/3556009295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480ad2c0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb6377fe700 1 -- 172.21.15.26:0/3556009295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb6480a9380 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.124+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb650020070 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb6377fe700 1 -- 172.21.15.26:0/3556009295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6480b1d70 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 10 client.19233 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4e2c50760 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb6377fe700 1 -- 172.21.15.26:0/3556009295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb650020070 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb6377fe700 1 -- 172.21.15.26:0/3556009295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fb6480a4880 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 10 client.19233 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 10 client.19233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 20 client.19233 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2d68ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 4 client.19233 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e2ca8cc0 con 0x55b4e2d666d0 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.125+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2d68ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.126+0000 7fb6377fe700 10 client.19233 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.126+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2d68ac0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e4d7bcae8fe1ff server_cookie=37d13c9db4811c3a in_seq=0 out_seq=0 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb6377fe700 1 -- 172.21.15.26:0/3556009295 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b4e2ca8cc0 con 0x55b4e2d666d0 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb6377fe700 10 client.19233 handle_command_reply: tid=0 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 client.19233 shutdown 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 msgr2=0x55b4e2d590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d590a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb6480df760 tx=0x7fb6480a8b50 comp rx=0 tx=0).stop 2024-05-20T21:08:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 shutdown_connections 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2d68ac0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d590a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb6377fe700 0 client.19233 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3556009295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 >> 172.21.15.26:0/3556009295 conn(0x55b4e2c74240 msgr2=0x55b4e2d42af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 shutdown_connections 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 1 -- 172.21.15.26:0/3556009295 wait complete. 2024-05-20T21:08:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.127+0000 7fb66904eb80 20 client.19233 trim_cache size 0 max 16384 2024-05-20T21:08:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.142+0000 7fb66904eb80 1 Processor -- start 2024-05-20T21:08:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.142+0000 7fb66904eb80 1 -- start start 2024-05-20T21:08:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d51d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb66904eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ae2b0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb66904eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581abb40 con 0x55b4e2d4d670 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb66904eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581a9990 con 0x55b4e2d526c0 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d51d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d51d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56026/0 (socket says 172.21.15.26:56026) 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 -- 172.21.15.26:0/3218076082 learned_addr learned my addr 172.21.15.26:0/3218076082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb65cd55700 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 -- 172.21.15.26:0/3218076082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 msgr2=0x55b4e2d500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 -- 172.21.15.26:0/3218076082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 msgr2=0x55b4e2d43270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d43270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 -- 172.21.15.26:0/3218076082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e2d43eb0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb65cd55700 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb657fff700 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d51d30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb6480a8ea0 tx=0x7fb648000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796389bc0642c91a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb64dffb700 1 -- 172.21.15.26:0/3218076082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480a4880 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb64dffb700 1 -- 172.21.15.26:0/3218076082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6480b18e0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.143+0000 7fb64dffb700 1 -- 172.21.15.26:0/3218076082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480a7e40 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 -- 172.21.15.26:0/3218076082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 msgr2=0x55b4e2d51d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d51d30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb6480a8ea0 tx=0x7fb648000bc0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 -- 172.21.15.26:0/3218076082 shutdown_connections 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d51d30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d526c0 0x55b4e2d500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3218076082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d4d670 0x55b4e2d43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 -- 172.21.15.26:0/3218076082 >> 172.21.15.26:0/3218076082 conn(0x55b4e2c74240 msgr2=0x55b4e2d48120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 -- 172.21.15.26:0/3218076082 shutdown_connections 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.144+0000 7fb66904eb80 1 -- 172.21.15.26:0/3218076082 wait complete. 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 -- start start 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 Processor -- start 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d6e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d526c0 0x55b4e2d6eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aa000 con 0x55b4e2d4d670 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ab470 con 0x55b4e2d526c0 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aa6d0 con 0x55b4e2d4ccb0 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb66904eb80 10 client.0 _ll_register_callbacks cb 0x55b4e2c31950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb64ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d6e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb64ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d6e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56042/0 (socket says 172.21.15.26:56042) 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb64ffff700 1 -- 172.21.15.26:0/3742844087 learned_addr learned my addr 172.21.15.26:0/3742844087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb657fff700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.145+0000 7fb65cd55700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d526c0 0x55b4e2d6eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb64ffff700 1 -- 172.21.15.26:0/3742844087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d4ccb0 msgr2=0x55b4e2d6e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d6e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb64ffff700 1 -- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d526c0 msgr2=0x55b4e2d6eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d526c0 0x55b4e2d6eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb64ffff700 1 -- 172.21.15.26:0/3742844087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e2dd04b0 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb64ffff700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d6e660 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb64000aa30 tx=0x7fb6400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d123c589c2701ee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb6377fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb6377fe700 1 -- 172.21.15.26:0/3742844087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640016070 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb6377fe700 1 -- 172.21.15.26:0/3742844087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb6400262d0 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6480b18e0 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb6377fe700 1 -- 172.21.15.26:0/3742844087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb640026510 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb66904eb80 10 client.19239 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4e2dd1770 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb6377fe700 1 -- 172.21.15.26:0/3742844087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6480b18e0 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb6377fe700 1 -- 172.21.15.26:0/3742844087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fb640010680 con 0x55b4e2d4d670 2024-05-20T21:08:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb66904eb80 10 client.19239 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb66904eb80 10 client.19239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.146+0000 7fb66904eb80 20 client.19239 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2dd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb66904eb80 4 client.19239 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4e2ca24a0 con 0x55b4e2d666d0 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb657fff700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2dd3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb657fff700 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2dd3f20 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b1c1b0f3836ab server_cookie=44ed3cc0d2ab5e71 in_seq=0 out_seq=0 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb6377fe700 10 client.19239 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb6377fe700 1 -- 172.21.15.26:0/3742844087 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4e2ca24a0 con 0x55b4e2d666d0 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb6377fe700 10 client.19239 handle_command_reply: tid=0 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.147+0000 7fb66904eb80 1 client.19239 shutdown 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 msgr2=0x55b4e2d6e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d6e660 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb64000aa30 tx=0x7fb6400047c0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 shutdown_connections 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55b4e2d666d0 0x55b4e2dd3f20 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4e2d526c0 0x55b4e2d6eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4e2d4ccb0 0x55b4e2d6e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 --2- 172.21.15.26:0/3742844087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4e2d4d670 0x55b4e2d6e660 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 >> 172.21.15.26:0/3742844087 conn(0x55b4e2c74240 msgr2=0x55b4e2d48120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb6377fe700 0 client.19239 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 shutdown_connections 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 1 -- 172.21.15.26:0/3742844087 wait complete. 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb66904eb80 20 client.19239 trim_cache size 0 max 16384 2024-05-20T21:08:27.153 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:08:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6440b2880 msgr2=0x7fb6440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6440b2880 0x7fb6440b4d40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb658108340 tx=0x7fb64000a380 comp rx=0 tx=0).stop 2024-05-20T21:08:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 msgr2=0x7fb65819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 0x7fb65819f6d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b530 tx=0x7fb64800b840 comp rx=0 tx=0).stop 2024-05-20T21:08:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 shutdown_connections 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb6440b2880 0x7fb6440b4d40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658103820 0x7fb65819f6d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb658108910 0x7fb658199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 --2- 172.21.15.26:0/1868866833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6581041d0 0x7fb65819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.148+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 >> 172.21.15.26:0/1868866833 conn(0x7fb65807a6b0 msgr2=0x7fb658109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.149+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 shutdown_connections 2024-05-20T21:08:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:27.149+0000 7fb65e7b8700 1 -- 172.21.15.26:0/1868866833 wait complete. 2024-05-20T21:08:27.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.34 MB/sec execute 513 sec latency 2224.511 ms 2024-05-20T21:08:27.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:27 smithi055 ceph-mon[35213]: pgmap v759: 97 pgs: 97 active+clean; 26 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-20T21:08:27.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:27 smithi026 ceph-mon[31990]: pgmap v759: 97 pgs: 97 active+clean; 26 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-20T21:08:27.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:27 smithi144 ceph-mon[35660]: pgmap v759: 97 pgs: 97 active+clean; 26 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-20T21:08:28.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.33 MB/sec execute 514 sec latency 3224.645 ms 2024-05-20T21:08:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 -- 172.21.15.26:0/1340098583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3504108840 msgr2=0x7f350410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 --2- 172.21.15.26:0/1340098583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3504108840 0x7f350410ac90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ca0 tx=0x7f34fc02a8c0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 -- 172.21.15.26:0/1340098583 shutdown_connections 2024-05-20T21:08:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 --2- 172.21.15.26:0/1340098583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3504108840 0x7f350410ac90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 --2- 172.21.15.26:0/1340098583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504105f10 0x7f3504108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 --2- 172.21.15.26:0/1340098583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35041035e0 0x7f35041059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 -- 172.21.15.26:0/1340098583 >> 172.21.15.26:0/1340098583 conn(0x7f35040fcfd0 msgr2=0x7f35040ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 -- 172.21.15.26:0/1340098583 shutdown_connections 2024-05-20T21:08:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 -- 172.21.15.26:0/1340098583 wait complete. 2024-05-20T21:08:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.742+0000 7f3509664640 1 Processor -- start 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 -- start start 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 0x7f3504198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3504105f10 0x7f3504199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504108840 0x7f35041a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f350410e540 con 0x7f35041035e0 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f350410e3c0 con 0x7f3504108840 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3509664640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f350410e6c0 con 0x7f3504105f10 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 0x7f3504198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 0x7f3504198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56046/0 (socket says 172.21.15.26:56046) 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 -- 172.21.15.26:0/578025151 learned_addr learned my addr 172.21.15.26:0/578025151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f35037fe640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504108840 0x7f35041a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f35027fc640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3504105f10 0x7f3504199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 -- 172.21.15.26:0/578025151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3504105f10 msgr2=0x7f3504199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3504105f10 0x7f3504199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 -- 172.21.15.26:0/578025151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504108840 msgr2=0x7f35041a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504108840 0x7f35041a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f3502ffd640 1 -- 172.21.15.26:0/578025151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35041a0750 con 0x7f35041035e0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.743+0000 7f35037fe640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504108840 0x7f35041a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.744+0000 7f3502ffd640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 0x7f3504198b30 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f34f400e9f0 tx=0x7f34f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.744+0000 7f34ebfff640 1 -- 172.21.15.26:0/578025151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f400cd80 con 0x7f35041035e0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.744+0000 7f34ebfff640 1 -- 172.21.15.26:0/578025151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34f40054b0 con 0x7f35041035e0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.744+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041a0a40 con 0x7f35041035e0 2024-05-20T21:08:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.744+0000 7f34ebfff640 1 -- 172.21.15.26:0/578025151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4010430 con 0x7f35041035e0 2024-05-20T21:08:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.744+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041a0f40 con 0x7f35041035e0 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.746+0000 7f34ebfff640 1 -- 172.21.15.26:0/578025151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f34f40046c0 con 0x7f35041035e0 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.746+0000 7f350c37d740 1 Processor -- start 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 -- start start 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37ead0 0x55c70a37eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 0x55c70a383fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a384630 0x55c70a386ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c70a2ebc30 con 0x55c70a37f4c0 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c70a2db7d0 con 0x55c70a384630 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f350c37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c70a2df810 con 0x55c70a37ead0 2024-05-20T21:08:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 0x55c70a383fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 0x55c70a383fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56062/0 (socket says 172.21.15.26:56062) 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 -- 172.21.15.26:0/1727497525 learned_addr learned my addr 172.21.15.26:0/1727497525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f35037fe640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37ead0 0x55c70a37eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f35027fc640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a384630 0x55c70a386ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 -- 172.21.15.26:0/1727497525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37ead0 msgr2=0x55c70a37eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37ead0 0x55c70a37eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f34ebfff640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34ec0b28a0 0x7f34ec0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 -- 172.21.15.26:0/1727497525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a384630 msgr2=0x55c70a386ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a384630 0x55c70a386ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f3502ffd640 1 -- 172.21.15.26:0/1727497525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c709ff0210 con 0x55c70a37f4c0 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f35027fc640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a384630 0x55c70a386ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.747+0000 7f35037fe640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37ead0 0x55c70a37eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.748+0000 7f35037fe640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34ec0b28a0 0x7f34ec0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.748+0000 7f34ebfff640 1 -- 172.21.15.26:0/578025151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f34f40e2740 con 0x7f35041035e0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.748+0000 7f3502ffd640 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 0x55c70a383fc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f34f40bfc10 tx=0x7f34f40a5bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9259185ee53c0a1f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.748+0000 7f34e97fa640 1 -- 172.21.15.26:0/1727497525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40a9ad0 con 0x55c70a37f4c0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.748+0000 7f34e97fa640 1 -- 172.21.15.26:0/1727497525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34f40bfa50 con 0x55c70a37f4c0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.748+0000 7f34e97fa640 1 -- 172.21.15.26:0/1727497525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40a9470 con 0x55c70a37f4c0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 -- 172.21.15.26:0/1727497525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 msgr2=0x55c70a383fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f35037fe640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34ec0b28a0 0x7f34ec0b4d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f34fc004af0 tx=0x7f34fc005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 0x55c70a383fc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f34f40bfc10 tx=0x7f34f40a5bc0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 -- 172.21.15.26:0/1727497525 shutdown_connections 2024-05-20T21:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a384630 0x55c70a386ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37f4c0 0x55c70a383fc0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 --2- 172.21.15.26:0/1727497525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37ead0 0x55c70a37eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 -- 172.21.15.26:0/1727497525 >> 172.21.15.26:0/1727497525 conn(0x55c70a375d30 msgr2=0x55c70a376130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 -- 172.21.15.26:0/1727497525 shutdown_connections 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 -- 172.21.15.26:0/1727497525 wait complete. 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 -- start start 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.749+0000 7f350c37d740 1 Processor -- start 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a388b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a3890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a390110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c709fc1790 con 0x55c70a37ead0 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c70a297890 con 0x55c70a37f4c0 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c70a3875e0 con 0x55c70a384630 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 10 client.0 _ll_register_callbacks cb 0x55c709fefa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a388b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a388b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56074/0 (socket says 172.21.15.26:56074) 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 -- 172.21.15.26:0/3654870069 learned_addr learned my addr 172.21.15.26:0/3654870069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f35027fc640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a3890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f35037fe640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a390110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 -- 172.21.15.26:0/3654870069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 msgr2=0x55c70a390110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a390110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 -- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 msgr2=0x55c70a3890d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a3890d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 -- 172.21.15.26:0/3654870069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70a0bc620 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f3502ffd640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a388b90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f34f40e0230 tx=0x7f34f40097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb941963a1f75b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f34e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f34e6ff5640 1 -- 172.21.15.26:0/3654870069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40a9650 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f34e6ff5640 1 -- 172.21.15.26:0/3654870069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f34f4091be0 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.750+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c70a2edb80 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f34e6ff5640 1 -- 172.21.15.26:0/3654870069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34f40a9470 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 10 client.19251 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c70a3930f0 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f34e6ff5640 1 -- 172.21.15.26:0/3654870069 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c70a2edb80 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f34e6ff5640 1 -- 172.21.15.26:0/3654870069 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f34f40c4170 con 0x55c70a37ead0 2024-05-20T21:08:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 10 client.19251 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 10 client.19251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 20 client.19251 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a394440 0x55c70a396830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 4 client.19251 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70a2e7360 con 0x55c70a394440 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.751+0000 7f35027fc640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a394440 0x55c70a396830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.752+0000 7f35027fc640 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a394440 0x55c70a396830 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d0bbc0aa6a6f2e server_cookie=a50984ac13a71432 in_seq=0 out_seq=0 2024-05-20T21:08:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.752+0000 7f34e6ff5640 10 client.19251 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.752+0000 7f34e6ff5640 1 -- 172.21.15.26:0/3654870069 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c70a2e7360 con 0x55c70a394440 2024-05-20T21:08:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.752+0000 7f34e6ff5640 10 client.19251 handle_command_reply: tid=0 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.752+0000 7f350c37d740 1 client.19251 shutdown 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 msgr2=0x55c70a388b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a388b90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f34f40e0230 tx=0x7f34f40097b0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 shutdown_connections 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a394440 0x55c70a396830 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a390110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a3890d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f34e6ff5640 0 client.19251 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 --2- 172.21.15.26:0/3654870069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a388b90 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 >> 172.21.15.26:0/3654870069 conn(0x55c70a375d30 msgr2=0x55c70a374fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 shutdown_connections 2024-05-20T21:08:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 1 -- 172.21.15.26:0/3654870069 wait complete. 2024-05-20T21:08:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.753+0000 7f350c37d740 20 client.19251 trim_cache size 0 max 16384 2024-05-20T21:08:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 Processor -- start 2024-05-20T21:08:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 -- start start 2024-05-20T21:08:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37ead0 0x55c70a379590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37f4c0 0x55c70a3900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 0x55c70a39c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35041aa590 con 0x55c70a384630 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35041a7e60 con 0x55c70a37ead0 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.768+0000 7f350c37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35041a5cb0 con 0x55c70a37f4c0 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 0x55c70a39c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 0x55c70a39c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56082/0 (socket says 172.21.15.26:56082) 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 -- 172.21.15.26:0/3867429838 learned_addr learned my addr 172.21.15.26:0/3867429838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35027fc640 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37f4c0 0x55c70a3900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 -- 172.21.15.26:0/3867429838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37f4c0 msgr2=0x55c70a3900a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f3502ffd640 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37ead0 0x55c70a379590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37f4c0 0x55c70a3900a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 -- 172.21.15.26:0/3867429838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37ead0 msgr2=0x55c70a379590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37ead0 0x55c70a379590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 -- 172.21.15.26:0/3867429838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70a396de0 con 0x55c70a384630 2024-05-20T21:08:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f3502ffd640 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37ead0 0x55c70a379590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f35037fe640 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 0x55c70a39c070 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ca0 tx=0x7f34fc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4546741d13f653e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f34e8ff9640 1 -- 172.21.15.26:0/3867429838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0305c0 con 0x55c70a384630 2024-05-20T21:08:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f34e8ff9640 1 -- 172.21.15.26:0/3867429838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34fc036040 con 0x55c70a384630 2024-05-20T21:08:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.769+0000 7f34e8ff9640 1 -- 172.21.15.26:0/3867429838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc049820 con 0x55c70a384630 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- 172.21.15.26:0/3867429838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 msgr2=0x55c70a39c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 0x55c70a39c070 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ca0 tx=0x7f34fc041000 comp rx=0 tx=0).stop 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- 172.21.15.26:0/3867429838 shutdown_connections 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a384630 0x55c70a39c070 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a37f4c0 0x55c70a3900a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- 172.21.15.26:0/3867429838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37ead0 0x55c70a379590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- 172.21.15.26:0/3867429838 >> 172.21.15.26:0/3867429838 conn(0x55c70a375d30 msgr2=0x55c70a370e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- 172.21.15.26:0/3867429838 shutdown_connections 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- 172.21.15.26:0/3867429838 wait complete. 2024-05-20T21:08:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- start start 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 Processor -- start 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a38dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a38e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a38f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35041a6320 con 0x55c70a37ead0 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35041a7790 con 0x55c70a37f4c0 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35041a69f0 con 0x55c70a384630 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.770+0000 7f350c37d740 10 client.0 _ll_register_callbacks cb 0x7f34f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a38dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a38dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56098/0 (socket says 172.21.15.26:56098) 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 -- 172.21.15.26:0/1688467342 learned_addr learned my addr 172.21.15.26:0/1688467342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f35027fc640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a38e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f35037fe640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a38f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 -- 172.21.15.26:0/1688467342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 msgr2=0x55c70a38f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a38f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 -- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 msgr2=0x55c70a38e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a38e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 -- 172.21.15.26:0/1688467342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70a38f7f0 con 0x55c70a37ead0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f35027fc640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a38e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f3502ffd640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a38dcb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005780 tx=0x7f34f40e8170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32109c54f8ffa96 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f34e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f34e6ff5640 1 -- 172.21.15.26:0/1688467342 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40a9ad0 con 0x55c70a37ead0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f34e6ff5640 1 -- 172.21.15.26:0/1688467342 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f34f40a9f60 con 0x55c70a37ead0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34fc036040 con 0x55c70a37ead0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f34e6ff5640 1 -- 172.21.15.26:0/1688467342 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34f40a9650 con 0x55c70a37ead0 2024-05-20T21:08:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f350c37d740 10 client.19257 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.771+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c70a3ff3a0 con 0x55c70a37ead0 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f34e6ff5640 1 -- 172.21.15.26:0/1688467342 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34fc036040 con 0x55c70a37ead0 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f34e6ff5640 1 -- 172.21.15.26:0/1688467342 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f34f4091be0 con 0x55c70a37ead0 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f350c37d740 10 client.19257 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f350c37d740 10 client.19257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f350c37d740 20 client.19257 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f350c37d740 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a3ff9c0 0x55c70a401de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f350c37d740 4 client.19257 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c70a2615f0 con 0x55c70a3ff9c0 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f35027fc640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a3ff9c0 0x55c70a401de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f34e6ff5640 10 client.19257 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.772+0000 7f35027fc640 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a3ff9c0 0x55c70a401de0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c779a9caeb2d04d server_cookie=20bbaf2c00e4cb98 in_seq=0 out_seq=0 2024-05-20T21:08:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f34e6ff5640 1 -- 172.21.15.26:0/1688467342 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c70a2615f0 con 0x55c70a3ff9c0 2024-05-20T21:08:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f34e6ff5640 10 client.19257 handle_command_reply: tid=0 2024-05-20T21:08:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 client.19257 shutdown 2024-05-20T21:08:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 msgr2=0x55c70a38dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a38dcb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005780 tx=0x7f34f40e8170 comp rx=0 tx=0).stop 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 shutdown_connections 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c70a3ff9c0 0x55c70a401de0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c70a384630 0x55c70a38f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c70a37f4c0 0x55c70a38e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 --2- 172.21.15.26:0/1688467342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70a37ead0 0x55c70a38dcb0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f34e6ff5640 0 client.19257 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 >> 172.21.15.26:0/1688467342 conn(0x55c70a375d30 msgr2=0x55c70a3887a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 shutdown_connections 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 1 -- 172.21.15.26:0/1688467342 wait complete. 2024-05-20T21:08:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.801+0000 7f350c37d740 20 client.19257 trim_cache size 0 max 16384 2024-05-20T21:08:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:08:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:08:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34ec0b28a0 msgr2=0x7f34ec0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34ec0b28a0 0x7f34ec0b4d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f34fc004af0 tx=0x7f34fc005c50 comp rx=0 tx=0).stop 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 msgr2=0x7f3504198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 0x7f3504198b30 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f34f400e9f0 tx=0x7f34f400edf0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 shutdown_connections 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34ec0b28a0 0x7f34ec0b4d60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3504108840 0x7f35041a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3504105f10 0x7f3504199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 --2- 172.21.15.26:0/578025151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35041035e0 0x7f3504198b30 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 >> 172.21.15.26:0/578025151 conn(0x7f35040fcfd0 msgr2=0x7f3504103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 shutdown_connections 2024-05-20T21:08:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:28.802+0000 7f3509664640 1 -- 172.21.15.26:0/578025151 wait complete. 2024-05-20T21:08:28.978 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:08:28.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:08:29.231 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:29.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.33 MB/sec execute 515 sec latency 4224.701 ms 2024-05-20T21:08:29.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:29 smithi026 ceph-mon[31990]: pgmap v760: 97 pgs: 97 active+clean; 28 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:08:29.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:29 smithi055 ceph-mon[35213]: pgmap v760: 97 pgs: 97 active+clean; 28 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:08:29.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:29 smithi144 ceph-mon[35660]: pgmap v760: 97 pgs: 97 active+clean; 28 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-20T21:08:30.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.32 MB/sec execute 516 sec latency 5224.801 ms 2024-05-20T21:08:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 -- 172.21.15.26:0/2839501246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb928101e20 msgr2=0x7fb928102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2839501246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb928101e20 0x7fb928102200 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb9240099c0 tx=0x7fb924009cd0 comp rx=0 tx=0).stop 2024-05-20T21:08:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 -- 172.21.15.26:0/2839501246 shutdown_connections 2024-05-20T21:08:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2839501246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb928101e20 0x7fb928102200 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2839501246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb928106f00 0x7fb92810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2839501246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9281027d0 0x7fb928106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 -- 172.21.15.26:0/2839501246 >> 172.21.15.26:0/2839501246 conn(0x7fb928078a40 msgr2=0x7fb9280ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 -- 172.21.15.26:0/2839501246 shutdown_connections 2024-05-20T21:08:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.995+0000 7fb92f94a700 1 -- 172.21.15.26:0/2839501246 wait complete. 2024-05-20T21:08:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 Processor -- start 2024-05-20T21:08:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 -- start start 2024-05-20T21:08:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb928101e20 0x7fb92819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 0x7fb92819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb928106f00 0x7fb9281a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb928111d10 con 0x7fb9281027d0 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb928111b90 con 0x7fb928106f00 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.996+0000 7fb92f94a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb928111e90 con 0x7fb928101e20 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 0x7fb92819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 0x7fb92819af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43728/0 (socket says 172.21.15.26:43728) 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 -- 172.21.15.26:0/2841972075 learned_addr learned my addr 172.21.15.26:0/2841972075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb928101e20 0x7fb92819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92dee7700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb928106f00 0x7fb9281a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 -- 172.21.15.26:0/2841972075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb928101e20 msgr2=0x7fb92819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb928101e20 0x7fb92819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 -- 172.21.15.26:0/2841972075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb928106f00 msgr2=0x7fb9281a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb928106f00 0x7fb9281a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 -- 172.21.15.26:0/2841972075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9281a5fd0 con 0x7fb9281027d0 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92cee5700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 0x7fb92819af50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b500 tx=0x7fb91800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb91e7fc700 1 -- 172.21.15.26:0/2841972075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918015020 con 0x7fb9281027d0 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb91e7fc700 1 -- 172.21.15.26:0/2841972075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb918011cc0 con 0x7fb9281027d0 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9281a6200 con 0x7fb9281027d0 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb91e7fc700 1 -- 172.21.15.26:0/2841972075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918010c80 con 0x7fb9281027d0 2024-05-20T21:08:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:30.997+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281a6790 con 0x7fb9281027d0 2024-05-20T21:08:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.000+0000 7fb91e7fc700 1 -- 172.21.15.26:0/2841972075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb918018020 con 0x7fb9281027d0 2024-05-20T21:08:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.001+0000 7fb91e7fc700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb9140b2900 0x7fb9140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.001+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb9140b2900 0x7fb9140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.001+0000 7fb91e7fc700 1 -- 172.21.15.26:0/2841972075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb9180e1930 con 0x7fb9281027d0 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.001+0000 7fb93a1e0b80 1 Processor -- start 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.001+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb9140b2900 0x7fb9140b4dc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb924000c00 tx=0x7fb924005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.001+0000 7fb93a1e0b80 1 -- start start 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb868f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb868f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb8684fd10 con 0x55cb868f4fe0 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cb8683f8b0 con 0x55cb868f1290 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cb868438f0 con 0x55cb868f08e0 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43732/0 (socket says 172.21.15.26:43732) 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 -- 172.21.15.26:0/1094373253 learned_addr learned my addr 172.21.15.26:0/1094373253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92cee5700 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb868f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb868f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 -- 172.21.15.26:0/1094373253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 msgr2=0x55cb868f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb868f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 -- 172.21.15.26:0/1094373253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 msgr2=0x55cb868f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb868f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 -- 172.21.15.26:0/1094373253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8678f950 con 0x55cb868f4fe0 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.002+0000 7fb92dee7700 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f5420 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e190 tx=0x7fb92000e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1044d859f06833 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb9137fe700 1 -- 172.21.15.26:0/1094373253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92000ede0 con 0x55cb868f4fe0 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb9137fe700 1 -- 172.21.15.26:0/1094373253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb920004bb0 con 0x55cb868f4fe0 2024-05-20T21:08:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb9137fe700 1 -- 172.21.15.26:0/1094373253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920014dc0 con 0x55cb868f4fe0 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1094373253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 msgr2=0x55cb868f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f5420 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e190 tx=0x7fb92000e4a0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1094373253 shutdown_connections 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f5420 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb868f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1094373253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb868f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1094373253 >> 172.21.15.26:0/1094373253 conn(0x55cb8681e100 msgr2=0x55cb868eaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.003+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1094373253 shutdown_connections 2024-05-20T21:08:31.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1094373253 wait complete. 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 -- start start 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 Processor -- start 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f08e0 0x55cb868fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f1290 0x55cb868f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb86746020 con 0x55cb868f4fe0 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cb867e1840 con 0x55cb868f08e0 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cb868fd1a0 con 0x55cb868f1290 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.004+0000 7fb93a1e0b80 10 client.0 _ll_register_callbacks cb 0x55cb867da9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43744/0 (socket says 172.21.15.26:43744) 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 -- 172.21.15.26:0/3986195881 learned_addr learned my addr 172.21.15.26:0/3986195881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f08e0 0x55cb868fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92cee5700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f1290 0x55cb868f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 -- 172.21.15.26:0/3986195881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f1290 msgr2=0x55cb868f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f1290 0x55cb868f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 -- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f08e0 msgr2=0x55cb868fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f08e0 0x55cb868fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 -- 172.21.15.26:0/3986195881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb867de710 con 0x55cb868f4fe0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb92dee7700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f9bf0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb920007d20 tx=0x7fb920004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785baf51b5396768 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb910ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb910ff9700 1 -- 172.21.15.26:0/3986195881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920022070 con 0x55cb868f4fe0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb910ff9700 1 -- 172.21.15.26:0/3986195881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb920019040 con 0x55cb868f4fe0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb920004bb0 con 0x55cb868f4fe0 2024-05-20T21:08:31.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb910ff9700 1 -- 172.21.15.26:0/3986195881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb92002e050 con 0x55cb868f4fe0 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb93a1e0b80 10 client.19269 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.005+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb86907850 con 0x55cb868f4fe0 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb910ff9700 1 -- 172.21.15.26:0/3986195881 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb920004bb0 con 0x55cb868f4fe0 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb910ff9700 1 -- 172.21.15.26:0/3986195881 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fb920014480 con 0x55cb868f4fe0 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb93a1e0b80 10 client.19269 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb93a1e0b80 10 client.19269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb93a1e0b80 20 client.19269 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb86909e30 0x55cb8690c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb93a1e0b80 4 client.19269 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb868bca60 con 0x55cb86909e30 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.006+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb86909e30 0x55cb8690c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.007+0000 7fb910ff9700 10 client.19269 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:31.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.007+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb86909e30 0x55cb8690c220 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6036a6b3c71bb552 server_cookie=8ab0750a62e88e57 in_seq=0 out_seq=0 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.007+0000 7fb910ff9700 1 -- 172.21.15.26:0/3986195881 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cb868bca60 con 0x55cb86909e30 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.007+0000 7fb910ff9700 10 client.19269 handle_command_reply: tid=0 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.007+0000 7fb93a1e0b80 1 client.19269 shutdown 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 msgr2=0x55cb868f9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f9bf0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb920007d20 tx=0x7fb920004d60 comp rx=0 tx=0).stop 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 shutdown_connections 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb86909e30 0x55cb8690c220 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f4fe0 0x55cb868f9bf0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f08e0 0x55cb868fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/3986195881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f1290 0x55cb868f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb910ff9700 0 client.19269 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:31.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 >> 172.21.15.26:0/3986195881 conn(0x55cb8681e100 msgr2=0x55cb868ea590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:31.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 shutdown_connections 2024-05-20T21:08:31.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/3986195881 wait complete. 2024-05-20T21:08:31.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.008+0000 7fb93a1e0b80 20 client.19269 trim_cache size 0 max 16384 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 Processor -- start 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 -- start start 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb8681e100 0x55cb868ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb869117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 0x55cb86911d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281b0680 con 0x55cb868f1290 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281adf10 con 0x55cb8681e100 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281abd60 con 0x55cb868f08e0 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb92dee7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 0x55cb86911d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb92dee7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 0x55cb86911d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43756/0 (socket says 172.21.15.26:43756) 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb92dee7700 1 -- 172.21.15.26:0/2237678283 learned_addr learned my addr 172.21.15.26:0/2237678283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.029+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb8681e100 0x55cb868ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92cee5700 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb869117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92dee7700 1 -- 172.21.15.26:0/2237678283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 msgr2=0x55cb869117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92dee7700 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb869117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92dee7700 1 -- 172.21.15.26:0/2237678283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb8681e100 msgr2=0x55cb868ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92dee7700 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb8681e100 0x55cb868ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92dee7700 1 -- 172.21.15.26:0/2237678283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb868f9480 con 0x55cb868f1290 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92cee5700 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb869117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb92dee7700 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 0x55cb86911d30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e8a0 tx=0x7fb92000e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6d658e3098bf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:31.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb912ffd700 1 -- 172.21.15.26:0/2237678283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92002b070 con 0x55cb868f1290 2024-05-20T21:08:31.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb912ffd700 1 -- 172.21.15.26:0/2237678283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9200145c0 con 0x55cb868f1290 2024-05-20T21:08:31.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.030+0000 7fb912ffd700 1 -- 172.21.15.26:0/2237678283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920022070 con 0x55cb868f1290 2024-05-20T21:08:31.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/2237678283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 msgr2=0x55cb86911d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 0x55cb86911d30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e8a0 tx=0x7fb92000e4a0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/2237678283 shutdown_connections 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb868f1290 0x55cb86911d30 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb8681e100 0x55cb868ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/2237678283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb869117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/2237678283 >> 172.21.15.26:0/2237678283 conn(0x55cb868f8b50 msgr2=0x55cb868eff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/2237678283 shutdown_connections 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/2237678283 wait complete. 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 -- start start 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.031+0000 7fb93a1e0b80 1 Processor -- start 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 0x55cb86901350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb86901890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb86901dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281ac3d0 con 0x55cb8681e100 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281ad840 con 0x55cb868f1290 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281acaa0 con 0x55cb868f08e0 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 10 client.0 _ll_register_callbacks cb 0x55cb867da9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 0x55cb86901350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 0x55cb86901350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43760/0 (socket says 172.21.15.26:43760) 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 -- 172.21.15.26:0/1515707891 learned_addr learned my addr 172.21.15.26:0/1515707891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92dee7700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb86901dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92cee5700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb86901890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 -- 172.21.15.26:0/1515707891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 msgr2=0x55cb86901890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb86901890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 -- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 msgr2=0x55cb86901dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb86901dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 -- 172.21.15.26:0/1515707891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb869748f0 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb92d6e6700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 0x55cb86901350 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb924005050 tx=0x7fb924009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f8f3f8b6cf557a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb910ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb910ff9700 1 -- 172.21.15.26:0/1515707891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924038070 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb910ff9700 1 -- 172.21.15.26:0/1515707891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb924003880 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.032+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9200145c0 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb910ff9700 1 -- 172.21.15.26:0/1515707891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb924003ac0 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 10 client.19275 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb86975c00 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb910ff9700 1 -- 172.21.15.26:0/1515707891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9200145c0 con 0x55cb8681e100 2024-05-20T21:08:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb910ff9700 1 -- 172.21.15.26:0/1515707891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fb92403b030 con 0x55cb8681e100 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 10 client.19275 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 10 client.19275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 20 client.19275 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb868f4fe0 0x55cb869783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 4 client.19275 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb86851c60 con 0x55cb868f4fe0 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.033+0000 7fb92cee5700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb868f4fe0 0x55cb869783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.034+0000 7fb910ff9700 10 client.19275 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.034+0000 7fb92cee5700 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb868f4fe0 0x55cb869783a0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae959e3107358d6f server_cookie=7405fc3599cc0738 in_seq=0 out_seq=0 2024-05-20T21:08:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb910ff9700 1 -- 172.21.15.26:0/1515707891 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb86851c60 con 0x55cb868f4fe0 2024-05-20T21:08:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb910ff9700 10 client.19275 handle_command_reply: tid=0 2024-05-20T21:08:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 client.19275 shutdown 2024-05-20T21:08:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 msgr2=0x55cb86901350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 0x55cb86901350 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb924005050 tx=0x7fb924009cd0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 shutdown_connections 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cb868f4fe0 0x55cb869783a0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8681e100 0x55cb86901350 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cb868f1290 0x55cb86901dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb910ff9700 0 client.19275 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 --2- 172.21.15.26:0/1515707891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb868f08e0 0x55cb86901890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 >> 172.21.15.26:0/1515707891 conn(0x55cb868f8b50 msgr2=0x55cb868faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 shutdown_connections 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.035+0000 7fb93a1e0b80 1 -- 172.21.15.26:0/1515707891 wait complete. 2024-05-20T21:08:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb93a1e0b80 20 client.19275 trim_cache size 0 max 16384 2024-05-20T21:08:31.041 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:31.041 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:08:31.041 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "b05f3d72-9b65-491f-ab86-20c931a840ba", 2024-05-20T21:08:31.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:08:31.041 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb9140b2900 msgr2=0x7fb9140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb9140b2900 0x7fb9140b4dc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb924000c00 tx=0x7fb924005fb0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 msgr2=0x7fb92819af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 0x7fb92819af50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b500 tx=0x7fb91800b810 comp rx=0 tx=0).stop 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 shutdown_connections 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb928106f00 0x7fb9281a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb928101e20 0x7fb92819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb9140b2900 0x7fb9140b4dc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 --2- 172.21.15.26:0/2841972075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9281027d0 0x7fb92819af50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 >> 172.21.15.26:0/2841972075 conn(0x7fb928078a40 msgr2=0x7fb9280ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 shutdown_connections 2024-05-20T21:08:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:31.036+0000 7fb92f94a700 1 -- 172.21.15.26:0/2841972075 wait complete. 2024-05-20T21:08:31.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.32 MB/sec execute 517 sec latency 6224.861 ms 2024-05-20T21:08:31.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:31 smithi055 ceph-mon[35213]: pgmap v761: 97 pgs: 97 active+clean; 28 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-05-20T21:08:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:31 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:08:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:31 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:31 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:08:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:31 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:31 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:08:31.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:31 smithi026 ceph-mon[31990]: pgmap v761: 97 pgs: 97 active+clean; 28 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-05-20T21:08:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:31 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:08:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:31 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:31 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:08:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:31 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:31.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:31 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:08:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:31 smithi144 ceph-mon[35660]: pgmap v761: 97 pgs: 97 active+clean; 28 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-05-20T21:08:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:31 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:08:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:31 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:31 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:08:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:31 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:31.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:31 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:08:31.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:08:32.150 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:32.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.32 MB/sec execute 518 sec latency 7224.917 ms 2024-05-20T21:08:33.431 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:33 smithi026 ceph-mon[31990]: pgmap v762: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-20T21:08:33.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.31 MB/sec execute 519 sec latency 8225.239 ms 2024-05-20T21:08:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:33 smithi055 ceph-mon[35213]: pgmap v762: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-20T21:08:33.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:33 smithi144 ceph-mon[35660]: pgmap v762: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 -- 172.21.15.26:0/133337124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 msgr2=0x7ff038103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 --2- 172.21.15.26:0/133337124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038103bd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0099c0 tx=0x7ff02c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 -- 172.21.15.26:0/133337124 shutdown_connections 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 --2- 172.21.15.26:0/133337124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038103bd0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 --2- 172.21.15.26:0/133337124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381088f0 0x7ff038110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 --2- 172.21.15.26:0/133337124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381041a0 0x7ff0381081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 -- 172.21.15.26:0/133337124 >> 172.21.15.26:0/133337124 conn(0x7ff03807a6a0 msgr2=0x7ff03807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 -- 172.21.15.26:0/133337124 shutdown_connections 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.037+0000 7ff03ef34700 1 -- 172.21.15.26:0/133337124 wait complete. 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 Processor -- start 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 -- start start 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381041a0 0x7ff038075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381088f0 0x7ff0380762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381135e0 con 0x7ff0381037f0 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff038113460 con 0x7ff0381041a0 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ef34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff038113760 con 0x7ff0381088f0 2024-05-20T21:08:34.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ccd0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ccd0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43770/0 (socket says 172.21.15.26:43770) 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3736903302 learned_addr learned my addr 172.21.15.26:0/3736903302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff037fff700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381041a0 0x7ff038075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.038+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381088f0 0x7ff0380762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3736903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381088f0 msgr2=0x7ff0380762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381088f0 0x7ff0380762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3736903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381041a0 msgr2=0x7ff038075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381041a0 0x7ff038075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3736903302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0381a8080 con 0x7ff0381037f0 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff037fff700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381041a0 0x7ff038075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381088f0 0x7ff0380762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038075840 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b580 tx=0x7ff02c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:34.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff035ffb700 1 -- 172.21.15.26:0/3736903302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c027070 con 0x7ff0381037f0 2024-05-20T21:08:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff035ffb700 1 -- 172.21.15.26:0/3736903302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff02c00fd30 con 0x7ff0381037f0 2024-05-20T21:08:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0381a8310 con 0x7ff0381037f0 2024-05-20T21:08:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff035ffb700 1 -- 172.21.15.26:0/3736903302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c022e50 con 0x7ff0381037f0 2024-05-20T21:08:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.039+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0381a8760 con 0x7ff0381037f0 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.042+0000 7ff035ffb700 1 -- 172.21.15.26:0/3736903302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff02c02a070 con 0x7ff0381037f0 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.043+0000 7ff035ffb700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0280b2a20 0x7ff0280b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.043+0000 7ff037fff700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0280b2a20 0x7ff0280b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.043+0000 7ff035ffb700 1 -- 172.21.15.26:0/3736903302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff02c0e2f20 con 0x7ff0381037f0 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.043+0000 7ff037fff700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0280b2a20 0x7ff0280b4ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff0381031f0 tx=0x7ff02400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.043+0000 7ff0497cab80 1 Processor -- start 2024-05-20T21:08:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.043+0000 7ff0497cab80 1 -- start start 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a3260 0x5601ad8af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff0497cab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5601ad800d70 con 0x5601ad8a8320 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff0497cab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5601ad7f0910 con 0x5601ad8a28a0 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff0497cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5601ad7f4950 con 0x5601ad8a3260 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43776/0 (socket says 172.21.15.26:43776) 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3416072085 learned_addr learned my addr 172.21.15.26:0/3416072085 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff037fff700 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3416072085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a3260 msgr2=0x5601ad8af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a3260 0x5601ad8af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a3260 0x5601ad8af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3416072085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 msgr2=0x5601ad8a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 -- 172.21.15.26:0/3416072085 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601ad799fb0 con 0x5601ad8a8320 2024-05-20T21:08:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff037fff700 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.044+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a8700 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0a7780 tx=0x7ff02c0a9e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee5794e1b690002 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff022ffd700 1 -- 172.21.15.26:0/3416072085 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0aac10 con 0x5601ad8a8320 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff022ffd700 1 -- 172.21.15.26:0/3416072085 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff02c0aa510 con 0x5601ad8a8320 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff022ffd700 1 -- 172.21.15.26:0/3416072085 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0b2e50 con 0x5601ad8a8320 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 -- 172.21.15.26:0/3416072085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 msgr2=0x5601ad8a8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a8700 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0a7780 tx=0x7ff02c0a9e60 comp rx=0 tx=0).stop 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 -- 172.21.15.26:0/3416072085 shutdown_connections 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a8700 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a3260 0x5601ad8af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3416072085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 -- 172.21.15.26:0/3416072085 >> 172.21.15.26:0/3416072085 conn(0x5601ad7ce240 msgr2=0x5601ad89d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 -- 172.21.15.26:0/3416072085 shutdown_connections 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.045+0000 7ff0497cab80 1 -- 172.21.15.26:0/3416072085 wait complete. 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 -- start start 2024-05-20T21:08:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 Processor -- start 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5601ad6f6fa0 con 0x5601ad8a3260 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5601ad758da0 con 0x5601ad8a28a0 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5601ad800d70 con 0x5601ad8a8320 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff0497cab80 10 client.0 _ll_register_callbacks cb 0x5601ad78b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff037fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.046+0000 7ff037fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43782/0 (socket says 172.21.15.26:43782) 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 -- 172.21.15.26:0/1992815879 learned_addr learned my addr 172.21.15.26:0/1992815879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 -- 172.21.15.26:0/1992815879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 msgr2=0x5601ad8ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 -- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 msgr2=0x5601ad8aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 -- 172.21.15.26:0/1992815879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601ad7408d0 con 0x5601ad8a3260 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff037fff700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8ac040 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7ff024011810 tx=0x7ff0240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a306eb3c2a6267c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff017fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff017fff700 1 -- 172.21.15.26:0/1992815879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02401e830 con 0x5601ad8a3260 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff017fff700 1 -- 172.21.15.26:0/1992815879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff02401e9d0 con 0x5601ad8a3260 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02c0aa510 con 0x5601ad8a3260 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff017fff700 1 -- 172.21.15.26:0/1992815879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff02401ebd0 con 0x5601ad8a3260 2024-05-20T21:08:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.047+0000 7ff0497cab80 10 client.19287 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601ad7aa760 con 0x5601ad8a3260 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff017fff700 1 -- 172.21.15.26:0/1992815879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff02c0aa510 con 0x5601ad8a3260 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff017fff700 1 -- 172.21.15.26:0/1992815879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff024030050 con 0x5601ad8a3260 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 10 client.19287 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 10 client.19287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 20 client.19287 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad8be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 4 client.19287 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601ad802cc0 con 0x5601ad8bbf30 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.048+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad8be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.049+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad8be320 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43709e121faba44 server_cookie=16de5b0e64df16cd in_seq=0 out_seq=0 2024-05-20T21:08:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.049+0000 7ff017fff700 10 client.19287 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.049+0000 7ff017fff700 1 -- 172.21.15.26:0/1992815879 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5601ad802cc0 con 0x5601ad8bbf30 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.049+0000 7ff017fff700 10 client.19287 handle_command_reply: tid=0 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 client.19287 shutdown 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 msgr2=0x5601ad8ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8ac040 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7ff024011810 tx=0x7ff0240047c0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 shutdown_connections 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad8be320 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff017fff700 0 client.19287 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 --2- 172.21.15.26:0/1992815879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8ac040 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 >> 172.21.15.26:0/1992815879 conn(0x5601ad7ce240 msgr2=0x5601ad89caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 shutdown_connections 2024-05-20T21:08:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 1 -- 172.21.15.26:0/1992815879 wait complete. 2024-05-20T21:08:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.050+0000 7ff0497cab80 20 client.19287 trim_cache size 0 max 16384 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 Processor -- start 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 -- start start 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a28a0 0x5601ad8ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a3260 0x5601ad89d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381b2780 con 0x5601ad8a8320 2024-05-20T21:08:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381b0010 con 0x5601ad8a3260 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff0497cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381ade60 con 0x5601ad8a28a0 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff03d4d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff03d4d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43792/0 (socket says 172.21.15.26:43792) 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.071+0000 7ff03d4d1700 1 -- 172.21.15.26:0/3755302173 learned_addr learned my addr 172.21.15.26:0/3755302173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a28a0 0x5601ad8ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff037fff700 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a3260 0x5601ad89d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03d4d1700 1 -- 172.21.15.26:0/3755302173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a28a0 msgr2=0x5601ad8ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a28a0 0x5601ad8ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03d4d1700 1 -- 172.21.15.26:0/3755302173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a3260 msgr2=0x5601ad89d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a3260 0x5601ad89d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03d4d1700 1 -- 172.21.15.26:0/3755302173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601ad89ec50 con 0x5601ad8a8320 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff037fff700 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a3260 0x5601ad89d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a6460 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff030009e60 tx=0x7ff030009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bafcbd7f620d27d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff0227fc700 1 -- 172.21.15.26:0/3755302173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03001a020 con 0x5601ad8a8320 2024-05-20T21:08:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff0227fc700 1 -- 172.21.15.26:0/3755302173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff030016af0 con 0x5601ad8a8320 2024-05-20T21:08:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.072+0000 7ff0227fc700 1 -- 172.21.15.26:0/3755302173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030015b00 con 0x5601ad8a8320 2024-05-20T21:08:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 -- 172.21.15.26:0/3755302173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 msgr2=0x5601ad8a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a6460 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff030009e60 tx=0x7ff030009040 comp rx=0 tx=0).stop 2024-05-20T21:08:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 -- 172.21.15.26:0/3755302173 shutdown_connections 2024-05-20T21:08:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a8320 0x5601ad8a6460 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a28a0 0x5601ad8ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 --2- 172.21.15.26:0/3755302173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a3260 0x5601ad89d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 -- 172.21.15.26:0/3755302173 >> 172.21.15.26:0/3755302173 conn(0x5601ad7ce240 msgr2=0x5601ad8a5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 -- 172.21.15.26:0/3755302173 shutdown_connections 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 -- 172.21.15.26:0/3755302173 wait complete. 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 -- start start 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.073+0000 7ff0497cab80 1 Processor -- start 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381ae4d0 con 0x5601ad8a3260 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381af940 con 0x5601ad8a28a0 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0381aeba0 con 0x5601ad8a8320 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff0497cab80 10 client.0 _ll_register_callbacks cb 0x5601ad78b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8c3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43804/0 (socket says 172.21.15.26:43804) 2024-05-20T21:08:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 -- 172.21.15.26:0/648775314 learned_addr learned my addr 172.21.15.26:0/648775314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 -- 172.21.15.26:0/648775314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 msgr2=0x5601ad8c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 -- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 msgr2=0x5601ad8c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8c3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 -- 172.21.15.26:0/648775314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601ad925cd0 con 0x5601ad8a3260 2024-05-20T21:08:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff03d4d1700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff037fff700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8c3e80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff024005fb0 tx=0x7ff02400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ee886a64c4a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff017fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff017fff700 1 -- 172.21.15.26:0/648775314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024004550 con 0x5601ad8a3260 2024-05-20T21:08:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.074+0000 7ff017fff700 1 -- 172.21.15.26:0/648775314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff024003680 con 0x5601ad8a3260 2024-05-20T21:08:34.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff030016af0 con 0x5601ad8a3260 2024-05-20T21:08:34.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff017fff700 1 -- 172.21.15.26:0/648775314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff024003880 con 0x5601ad8a3260 2024-05-20T21:08:34.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 10 client.19293 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:34.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601ad926fa0 con 0x5601ad8a3260 2024-05-20T21:08:34.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff017fff700 1 -- 172.21.15.26:0/648775314 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff030016af0 con 0x5601ad8a3260 2024-05-20T21:08:34.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff017fff700 1 -- 172.21.15.26:0/648775314 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff024030070 con 0x5601ad8a3260 2024-05-20T21:08:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 10 client.19293 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 10 client.19293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 20 client.19293 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad929770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 4 client.19293 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:08:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601ad7fc4a0 con 0x5601ad8bbf30 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.075+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad929770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff03ccd0700 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad929770 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8484a8d83340f8 server_cookie=47d91210d901ccc in_seq=0 out_seq=0 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff017fff700 10 client.19293 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff017fff700 1 -- 172.21.15.26:0/648775314 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5601ad7fc4a0 con 0x5601ad8bbf30 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff017fff700 10 client.19293 handle_command_reply: tid=0 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 client.19293 shutdown 2024-05-20T21:08:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 msgr2=0x5601ad8c3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8c3e80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff024005fb0 tx=0x7ff02400bee0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 shutdown_connections 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5601ad8bbf30 0x5601ad929770 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601ad8a8320 0x5601ad8c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5601ad8a28a0 0x5601ad8c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 --2- 172.21.15.26:0/648775314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601ad8a3260 0x5601ad8c3e80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 >> 172.21.15.26:0/648775314 conn(0x5601ad7ce240 msgr2=0x5601ad8a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff017fff700 0 client.19293 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 shutdown_connections 2024-05-20T21:08:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 1 -- 172.21.15.26:0/648775314 wait complete. 2024-05-20T21:08:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.076+0000 7ff0497cab80 20 client.19293 trim_cache size 0 max 16384 2024-05-20T21:08:34.086 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:08:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:08:34.087 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0280b2a20 msgr2=0x7ff0280b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0280b2a20 0x7ff0280b4ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff0381031f0 tx=0x7ff02400b3c0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 msgr2=0x7ff038075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038075840 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b580 tx=0x7ff02c0047b0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 shutdown_connections 2024-05-20T21:08:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff0280b2a20 0x7ff0280b4ee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0381037f0 0x7ff038075840 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0381088f0 0x7ff0380762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 --2- 172.21.15.26:0/3736903302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0381041a0 0x7ff038075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 >> 172.21.15.26:0/3736903302 conn(0x7ff03807a6a0 msgr2=0x7ff0381018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 shutdown_connections 2024-05-20T21:08:34.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:34.077+0000 7ff03ef34700 1 -- 172.21.15.26:0/3736903302 wait complete. 2024-05-20T21:08:34.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.31 MB/sec execute 520 sec latency 9225.326 ms 2024-05-20T21:08:34.916 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:08:34.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:08:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:35.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.30 MB/sec execute 521 sec latency 10225.433 ms 2024-05-20T21:08:35.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:35 smithi055 ceph-mon[35213]: pgmap v763: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 876 KiB/s wr, 74 op/s 2024-05-20T21:08:35.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:35 smithi026 ceph-mon[31990]: pgmap v763: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 876 KiB/s wr, 74 op/s 2024-05-20T21:08:35.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:35 smithi144 ceph-mon[35660]: pgmap v763: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 876 KiB/s wr, 74 op/s 2024-05-20T21:08:36.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.30 MB/sec execute 522 sec latency 11225.512 ms 2024-05-20T21:08:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 -- 172.21.15.26:0/1244780790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f8108930 msgr2=0x7f80f81104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 --2- 172.21.15.26:0/1244780790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f8108930 0x7f80f81104b0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b0d0 tx=0x7f80f000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 -- 172.21.15.26:0/1244780790 shutdown_connections 2024-05-20T21:08:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 --2- 172.21.15.26:0/1244780790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f8108930 0x7f80f81104b0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 --2- 172.21.15.26:0/1244780790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f8101cf0 0x7f80f81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 --2- 172.21.15.26:0/1244780790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f81026a0 0x7f80f8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 -- 172.21.15.26:0/1244780790 >> 172.21.15.26:0/1244780790 conn(0x7f80f8078a40 msgr2=0x7f80f80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 -- 172.21.15.26:0/1244780790 shutdown_connections 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.173+0000 7f80fe16c700 1 -- 172.21.15.26:0/1244780790 wait complete. 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 Processor -- start 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 -- start start 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f8101cf0 0x7f80f81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 0x7f80f81a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f8108930 0x7f80f819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81135b0 con 0x7f80f81026a0 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8113430 con 0x7f80f8108930 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80fe16c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8113730 con 0x7f80f8101cf0 2024-05-20T21:08:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.174+0000 7f80f6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 0x7f80f81a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 0x7f80f81a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43828/0 (socket says 172.21.15.26:43828) 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 -- 172.21.15.26:0/813740291 learned_addr learned my addr 172.21.15.26:0/813740291 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f77fe700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f8101cf0 0x7f80f81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f7fff700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f8108930 0x7f80f819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 -- 172.21.15.26:0/813740291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f8101cf0 msgr2=0x7f80f81a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f8101cf0 0x7f80f81a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 -- 172.21.15.26:0/813740291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f8108930 msgr2=0x7f80f819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f8108930 0x7f80f819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 -- 172.21.15.26:0/813740291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f819c1f0 con 0x7f80f81026a0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f6ffd700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 0x7f80f81a1e10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c920 tx=0x7f80ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec004a30 con 0x7f80f81026a0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f819c4e0 con 0x7f80f81026a0 2024-05-20T21:08:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80ec0054b0 con 0x7f80f81026a0 2024-05-20T21:08:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec007b70 con 0x7f80f81026a0 2024-05-20T21:08:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.175+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81a87a0 con 0x7f80f81026a0 2024-05-20T21:08:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.176+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f806a230 con 0x7f80f81026a0 2024-05-20T21:08:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.178+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f80ec004bd0 con 0x7f80f81026a0 2024-05-20T21:08:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.178+0000 7f80f4ff9700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80e00b2880 0x7f80e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:37.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.179+0000 7f80f77fe700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80e00b2880 0x7f80e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:37.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.179+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f80ec0e1d00 con 0x7f80f81026a0 2024-05-20T21:08:37.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.179+0000 7f80f77fe700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80e00b2880 0x7f80e00b4d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f80f819cbd0 tx=0x7f80e80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:37.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.183+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80ec014030 con 0x7f80f81026a0 2024-05-20T21:08:37.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:37 smithi026 ceph-mon[31990]: pgmap v764: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 586 KiB/s wr, 51 op/s 2024-05-20T21:08:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.454+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80f819cfc0 con 0x7f80f81026a0 2024-05-20T21:08:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.455+0000 7f80f4ff9700 1 -- 172.21.15.26:0/813740291 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f80ec005660 con 0x7f80f81026a0 2024-05-20T21:08:37.455 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:08:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80e00b2880 msgr2=0x7f80e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80e00b2880 0x7f80e00b4d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f80f819cbd0 tx=0x7f80e80073d0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 msgr2=0x7f80f81a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 0x7f80f81a1e10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c920 tx=0x7f80ec00cc30 comp rx=0 tx=0).stop 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 shutdown_connections 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f8108930 0x7f80f819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f8101cf0 0x7f80f81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f80e00b2880 0x7f80e00b4d40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 --2- 172.21.15.26:0/813740291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f81026a0 0x7f80f81a1e10 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 >> 172.21.15.26:0/813740291 conn(0x7f80f8078a40 msgr2=0x7f80f8105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 shutdown_connections 2024-05-20T21:08:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:37.458+0000 7f80fe16c700 1 -- 172.21.15.26:0/813740291 wait complete. 2024-05-20T21:08:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:08:37.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.29 MB/sec execute 523 sec latency 12225.559 ms 2024-05-20T21:08:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:37 smithi055 ceph-mon[35213]: pgmap v764: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 586 KiB/s wr, 51 op/s 2024-05-20T21:08:37.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:37 smithi144 ceph-mon[35660]: pgmap v764: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 586 KiB/s wr, 51 op/s 2024-05-20T21:08:38.183 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:08:38.437 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:38.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:38 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/813740291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:38.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.29 MB/sec execute 524 sec latency 13225.608 ms 2024-05-20T21:08:38.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:38 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/813740291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:38.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:38 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/813740291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:39.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.28 MB/sec execute 525 sec latency 14225.637 ms 2024-05-20T21:08:39.608 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:39 smithi026 ceph-mon[31990]: pgmap v765: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 306 KiB/s wr, 48 op/s 2024-05-20T21:08:39.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:39 smithi055 ceph-mon[35213]: pgmap v765: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 306 KiB/s wr, 48 op/s 2024-05-20T21:08:39.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:39 smithi144 ceph-mon[35660]: pgmap v765: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 306 KiB/s wr, 48 op/s 2024-05-20T21:08:40.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 -- 172.21.15.26:0/3239777054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac5006c6d0 msgr2=0x7fac50110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 --2- 172.21.15.26:0/3239777054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac5006c6d0 0x7fac50110400 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fac4800b0d0 tx=0x7fac4800b3e0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 -- 172.21.15.26:0/3239777054 shutdown_connections 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 --2- 172.21.15.26:0/3239777054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac5006c6d0 0x7fac50110400 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 --2- 172.21.15.26:0/3239777054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac50109630 0x7fac50109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 --2- 172.21.15.26:0/3239777054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac5006bd10 0x7fac5006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 -- 172.21.15.26:0/3239777054 >> 172.21.15.26:0/3239777054 conn(0x7fac500794e0 msgr2=0x7fac500798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 -- 172.21.15.26:0/3239777054 shutdown_connections 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 -- 172.21.15.26:0/3239777054 wait complete. 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.374+0000 7fac55832700 1 Processor -- start 2024-05-20T21:08:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 -- start start 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac5006bd10 0x7fac5019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac5006c6d0 0x7fac5019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 0x7fac50199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac501136c0 con 0x7fac50109630 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fac50113540 con 0x7fac5006c6d0 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac55832700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fac50113840 con 0x7fac5006bd10 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 0x7fac50199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 0x7fac50199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34328/0 (socket says 172.21.15.26:34328) 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 -- 172.21.15.26:0/346916370 learned_addr learned my addr 172.21.15.26:0/346916370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4effd700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac5006bd10 0x7fac5019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac5006c6d0 0x7fac5019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 -- 172.21.15.26:0/346916370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac5006bd10 msgr2=0x7fac5019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac5006bd10 0x7fac5019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 -- 172.21.15.26:0/346916370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac5006c6d0 msgr2=0x7fac5019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac5006c6d0 0x7fac5019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4f7fe700 1 -- 172.21.15.26:0/346916370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5019a160 con 0x7fac50109630 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.375+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac5006c6d0 0x7fac5019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.376+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 0x7fac50199a00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fac48007d90 tx=0x7fac48007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.376+0000 7fac54830700 1 -- 172.21.15.26:0/346916370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4801f030 con 0x7fac50109630 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.376+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5019a3f0 con 0x7fac50109630 2024-05-20T21:08:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.376+0000 7fac54830700 1 -- 172.21.15.26:0/346916370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac4801bb90 con 0x7fac50109630 2024-05-20T21:08:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.376+0000 7fac54830700 1 -- 172.21.15.26:0/346916370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48012bd0 con 0x7fac50109630 2024-05-20T21:08:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.376+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501a6710 con 0x7fac50109630 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.378+0000 7fac54830700 1 -- 172.21.15.26:0/346916370 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fac48022030 con 0x7fac50109630 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.379+0000 7fac54830700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac380b2880 0x7fac380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.379+0000 7fac4effd700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac380b2880 0x7fac380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.379+0000 7fac54830700 1 -- 172.21.15.26:0/346916370 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fac480e3380 con 0x7fac50109630 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.379+0000 7fac600c8b80 1 Processor -- start 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 -- start start 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac380b2880 0x7fac380b4d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fac5019ab40 tx=0x7fac40009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd347aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd341cb0 0x55ccdd342130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd34e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd29bd70 con 0x55ccdd3476c0 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd28b910 con 0x55ccdd342670 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac600c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd28f950 con 0x55ccdd341cb0 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd347aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd347aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34334/0 (socket says 172.21.15.26:34334) 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 -- 172.21.15.26:0/3718060564 learned_addr learned my addr 172.21.15.26:0/3718060564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd341cb0 0x55ccdd342130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd34e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 -- 172.21.15.26:0/3718060564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd341cb0 msgr2=0x55ccdd342130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd341cb0 0x55ccdd342130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 -- 172.21.15.26:0/3718060564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 msgr2=0x55ccdd34e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd34e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 -- 172.21.15.26:0/3718060564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdd234fb0 con 0x55ccdd3476c0 2024-05-20T21:08:40.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd34e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:40.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.380+0000 7fac4effd700 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd347aa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fac40009fd0 tx=0x7fac40011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feef4e2f2632bfde server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:40.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.381+0000 7fac35ffb700 1 -- 172.21.15.26:0/3718060564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4001b040 con 0x55ccdd3476c0 2024-05-20T21:08:40.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.381+0000 7fac35ffb700 1 -- 172.21.15.26:0/3718060564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac40024070 con 0x55ccdd3476c0 2024-05-20T21:08:40.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.381+0000 7fac35ffb700 1 -- 172.21.15.26:0/3718060564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac400136b0 con 0x55ccdd3476c0 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 -- 172.21.15.26:0/3718060564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 msgr2=0x55ccdd347aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd347aa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fac40009fd0 tx=0x7fac40011000 comp rx=0 tx=0).stop 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 -- 172.21.15.26:0/3718060564 shutdown_connections 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd347aa0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd34e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 --2- 172.21.15.26:0/3718060564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd341cb0 0x55ccdd342130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 -- 172.21.15.26:0/3718060564 >> 172.21.15.26:0/3718060564 conn(0x55ccdd269240 msgr2=0x55ccdd3383e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:40.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 -- 172.21.15.26:0/3718060564 shutdown_connections 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 -- 172.21.15.26:0/3718060564 wait complete. 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.382+0000 7fac600c8b80 1 -- start start 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 Processor -- start 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd34b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd3476c0 0x55ccdd34b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd191fa0 con 0x55ccdd341cb0 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd1f3da0 con 0x55ccdd3476c0 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd34eb50 con 0x55ccdd342670 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac600c8b80 10 client.0 _ll_register_callbacks cb 0x55ccdd226950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34336/0 (socket says 172.21.15.26:34336) 2024-05-20T21:08:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 -- 172.21.15.26:0/476533568 learned_addr learned my addr 172.21.15.26:0/476533568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd34b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd3476c0 0x55ccdd34b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 -- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 msgr2=0x55ccdd34b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd34b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 -- 172.21.15.26:0/476533568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd3476c0 msgr2=0x55ccdd34b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd3476c0 0x55ccdd34b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.383+0000 7fac4effd700 1 -- 172.21.15.26:0/476533568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdd1db8d0 con 0x55ccdd341cb0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac4effd700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fac40015a40 tx=0x7fac40015a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9e3b751e2c6a9a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac2effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac2effd700 1 -- 172.21.15.26:0/476533568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4002e070 con 0x55ccdd341cb0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac2effd700 1 -- 172.21.15.26:0/476533568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fac40028830 con 0x55ccdd341cb0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac40024070 con 0x55ccdd341cb0 2024-05-20T21:08:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac2effd700 1 -- 172.21.15.26:0/476533568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac4003a050 con 0x55ccdd341cb0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac600c8b80 10 client.19311 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccdd245760 con 0x55ccdd341cb0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.384+0000 7fac2effd700 1 -- 172.21.15.26:0/476533568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac40024070 con 0x55ccdd341cb0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac2effd700 1 -- 172.21.15.26:0/476533568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fac4001e030 con 0x55ccdd341cb0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac600c8b80 10 client.19311 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac600c8b80 10 client.19311 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac600c8b80 20 client.19311 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac600c8b80 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd35d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac600c8b80 4 client.19311 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccdd29dcc0 con 0x55ccdd35b4c0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd35d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd35d8b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2735c77fdaa294df server_cookie=85a00b4206085bb0 in_seq=0 out_seq=0 2024-05-20T21:08:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.385+0000 7fac2effd700 10 client.19311 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:40.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.386+0000 7fac2effd700 1 -- 172.21.15.26:0/476533568 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ccdd29dcc0 con 0x55ccdd35b4c0 2024-05-20T21:08:40.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.386+0000 7fac2effd700 10 client.19311 handle_command_reply: tid=0 2024-05-20T21:08:40.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.386+0000 7fac600c8b80 1 client.19311 shutdown 2024-05-20T21:08:40.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.386+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 msgr2=0x55ccdd34e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.386+0000 7fac600c8b80 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fac40015a40 tx=0x7fac40015a70 comp rx=0 tx=0).stop 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 shutdown_connections 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd35d8b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd3476c0 0x55ccdd34b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac2effd700 0 client.19311 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 --2- 172.21.15.26:0/476533568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd34b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 >> 172.21.15.26:0/476533568 conn(0x55ccdd269240 msgr2=0x55ccdd337af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 shutdown_connections 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 1 -- 172.21.15.26:0/476533568 wait complete. 2024-05-20T21:08:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.387+0000 7fac600c8b80 20 client.19311 trim_cache size 0 max 16384 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 Processor -- start 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 -- start start 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd338270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd3476c0 0x55ccdd34d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ae360 con 0x55ccdd341cb0 2024-05-20T21:08:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fac501abbf0 con 0x55ccdd342670 2024-05-20T21:08:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.402+0000 7fac600c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fac501a9a40 con 0x55ccdd3476c0 2024-05-20T21:08:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34340/0 (socket says 172.21.15.26:34340) 2024-05-20T21:08:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 -- 172.21.15.26:0/4071954397 learned_addr learned my addr 172.21.15.26:0/4071954397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd338270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 -- 172.21.15.26:0/4071954397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd3476c0 msgr2=0x55ccdd34d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd3476c0 0x55ccdd34d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 -- 172.21.15.26:0/4071954397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 msgr2=0x55ccdd338270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd338270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 -- 172.21.15.26:0/4071954397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdd347cf0 con 0x55ccdd341cb0 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd338270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac4effd700 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fac400048a0 tx=0x7fac40013ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e256ac0e471ca059 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac357fa700 1 -- 172.21.15.26:0/4071954397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4002e040 con 0x55ccdd341cb0 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac357fa700 1 -- 172.21.15.26:0/4071954397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac40024070 con 0x55ccdd341cb0 2024-05-20T21:08:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.403+0000 7fac357fa700 1 -- 172.21.15.26:0/4071954397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40028ac0 con 0x55ccdd341cb0 2024-05-20T21:08:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 -- 172.21.15.26:0/4071954397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 msgr2=0x55ccdd34e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fac400048a0 tx=0x7fac40013ef0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 -- 172.21.15.26:0/4071954397 shutdown_connections 2024-05-20T21:08:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd341cb0 0x55ccdd34e080 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd3476c0 0x55ccdd34d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 --2- 172.21.15.26:0/4071954397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd342670 0x55ccdd338270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 -- 172.21.15.26:0/4071954397 >> 172.21.15.26:0/4071954397 conn(0x55ccdd269240 msgr2=0x55ccdd34cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 -- 172.21.15.26:0/4071954397 shutdown_connections 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 -- 172.21.15.26:0/4071954397 wait complete. 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 -- start start 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.404+0000 7fac600c8b80 1 Processor -- start 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd341cb0 0x55ccdd353de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd354320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd361f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac501aa0b0 con 0x55ccdd3476c0 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ab520 con 0x55ccdd341cb0 2024-05-20T21:08:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fac501aa780 con 0x55ccdd342670 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac600c8b80 10 client.0 _ll_register_callbacks cb 0x55ccdd226950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd361f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd361f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34348/0 (socket says 172.21.15.26:34348) 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 -- 172.21.15.26:0/2075579033 learned_addr learned my addr 172.21.15.26:0/2075579033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4effd700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd341cb0 0x55ccdd353de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4e7fc700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd354320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 -- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 msgr2=0x55ccdd354320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd354320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 -- 172.21.15.26:0/2075579033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd341cb0 msgr2=0x55ccdd353de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd341cb0 0x55ccdd353de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 -- 172.21.15.26:0/2075579033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdd3638e0 con 0x55ccdd3476c0 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4effd700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd341cb0 0x55ccdd353de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac4f7fe700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd361f00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fac4800ba60 tx=0x7fac480a6290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b20ed78315553 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.405+0000 7fac2effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac2effd700 1 -- 172.21.15.26:0/2075579033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48029070 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac2effd700 1 -- 172.21.15.26:0/2075579033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fac480a6c60 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac40024070 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac2effd700 1 -- 172.21.15.26:0/2075579033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac480ab0d0 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 10 client.19317 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccdd3c5190 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac2effd700 1 -- 172.21.15.26:0/2075579033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac40024070 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac2effd700 1 -- 172.21.15.26:0/2075579033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fac48011070 con 0x55ccdd3476c0 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 10 client.19317 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 10 client.19317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 20 client.19317 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd3c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 4 client.19317 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccdd2974a0 con 0x55ccdd35b4c0 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.406+0000 7fac4effd700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd3c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac4effd700 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd3c7830 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=940b41210d9719f0 server_cookie=8fe2879b82e22495 in_seq=0 out_seq=0 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac2effd700 10 client.19317 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac2effd700 1 -- 172.21.15.26:0/2075579033 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccdd2974a0 con 0x55ccdd35b4c0 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac2effd700 10 client.19317 handle_command_reply: tid=0 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 client.19317 shutdown 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 msgr2=0x55ccdd361f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd361f00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fac4800ba60 tx=0x7fac480a6290 comp rx=0 tx=0).stop 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 shutdown_connections 2024-05-20T21:08:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55ccdd35b4c0 0x55ccdd3c7830 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccdd3476c0 0x55ccdd361f00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccdd341cb0 0x55ccdd353de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac2effd700 0 client.19317 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 --2- 172.21.15.26:0/2075579033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccdd342670 0x55ccdd354320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 >> 172.21.15.26:0/2075579033 conn(0x55ccdd269240 msgr2=0x55ccdd34d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 shutdown_connections 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 1 -- 172.21.15.26:0/2075579033 wait complete. 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.407+0000 7fac600c8b80 20 client.19317 trim_cache size 0 max 16384 2024-05-20T21:08:40.415 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:08:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac380b2880 msgr2=0x7fac380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac380b2880 0x7fac380b4d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fac5019ab40 tx=0x7fac40009190 comp rx=0 tx=0).stop 2024-05-20T21:08:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 msgr2=0x7fac50199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 0x7fac50199a00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fac48007d90 tx=0x7fac48007dc0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 shutdown_connections 2024-05-20T21:08:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fac380b2880 0x7fac380b4d40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac50109630 0x7fac50199a00 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac5006bd10 0x7fac5019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 --2- 172.21.15.26:0/346916370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac5006c6d0 0x7fac5019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 >> 172.21.15.26:0/346916370 conn(0x7fac500794e0 msgr2=0x7fac50101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 shutdown_connections 2024-05-20T21:08:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:40.408+0000 7fac55832700 1 -- 172.21.15.26:0/346916370 wait complete. 2024-05-20T21:08:40.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.28 MB/sec execute 526 sec latency 15225.665 ms 2024-05-20T21:08:41.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.28 MB/sec execute 527 sec latency 16225.737 ms 2024-05-20T21:08:41.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:41 smithi055 ceph-mon[35213]: pgmap v766: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 125 KiB/s wr, 27 op/s 2024-05-20T21:08:41.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:41 smithi026 ceph-mon[31990]: pgmap v766: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 125 KiB/s wr, 27 op/s 2024-05-20T21:08:41.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:41 smithi144 ceph-mon[35660]: pgmap v766: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 125 KiB/s wr, 27 op/s 2024-05-20T21:08:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.903+0000 7f81b59b6640 1 -- 172.21.15.26:0/1299853170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 msgr2=0x7f81b0109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.903+0000 7f81b59b6640 1 --2- 172.21.15.26:0/1299853170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b0109ea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009ca0 tx=0x7f81a802a8c0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- 172.21.15.26:0/1299853170 shutdown_connections 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 --2- 172.21.15.26:0/1299853170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b0109ea0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 --2- 172.21.15.26:0/1299853170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0102c50 0x7f81b01030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 --2- 172.21.15.26:0/1299853170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 0x7f81b0101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- 172.21.15.26:0/1299853170 >> 172.21.15.26:0/1299853170 conn(0x7f81b00fcfd0 msgr2=0x7f81b00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- 172.21.15.26:0/1299853170 shutdown_connections 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- 172.21.15.26:0/1299853170 wait complete. 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 Processor -- start 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- start start 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 0x7f81b0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0102c50 0x7f81b0199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b01a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010d400 con 0x7f81b0103610 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010d280 con 0x7f81b0101a60 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.904+0000 7f81b59b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010d580 con 0x7f81b0102c50 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b01a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b01a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34368/0 (socket says 172.21.15.26:34368) 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 -- 172.21.15.26:0/26686254 learned_addr learned my addr 172.21.15.26:0/26686254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81aeffd640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 0x7f81b0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 -- 172.21.15.26:0/26686254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0102c50 msgr2=0x7f81b0199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81a7fff640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0102c50 0x7f81b0199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0102c50 0x7f81b0199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 -- 172.21.15.26:0/26686254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 msgr2=0x7f81b0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 0x7f81b0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 -- 172.21.15.26:0/26686254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a0710 con 0x7f81b0103610 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81aeffd640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 0x7f81b0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81af7fe640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b01a00a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f81a802aeb0 tx=0x7f81a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81b49b4640 1 -- 172.21.15.26:0/26686254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8032ed0 con 0x7f81b0103610 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81b49b4640 1 -- 172.21.15.26:0/26686254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81a8003e10 con 0x7f81b0103610 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a09a0 con 0x7f81b0103610 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81b49b4640 1 -- 172.21.15.26:0/26686254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8030cd0 con 0x7f81b0103610 2024-05-20T21:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.905+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a0f40 con 0x7f81b0103610 2024-05-20T21:08:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b49b4640 1 -- 172.21.15.26:0/26686254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f81a8004240 con 0x7f81b0103610 2024-05-20T21:08:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 Processor -- start 2024-05-20T21:08:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 -- start start 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d95def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d962fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d965ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f42d8cac30 con 0x55f42d95dad0 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f42d8ba7d0 con 0x55f42d95e4c0 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.908+0000 7f81b87ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f42d8be810 con 0x55f42d963630 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d95def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d95def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34378/0 (socket says 172.21.15.26:34378) 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 -- 172.21.15.26:0/2568769203 learned_addr learned my addr 172.21.15.26:0/2568769203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81aeffd640 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d962fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 -- 172.21.15.26:0/2568769203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 msgr2=0x55f42d965ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d965ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 -- 172.21.15.26:0/2568769203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 msgr2=0x55f42d962fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d962fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 -- 172.21.15.26:0/2568769203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42d5cf210 con 0x55f42d95dad0 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81aeffd640 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d962fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81b49b4640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f819c0b2920 0x7f819c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a7fff640 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d95def0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8198009b60 tx=0x7f81980106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecfe4a5aa1a1fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81aeffd640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f819c0b2920 0x7f819c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a5ffb640 1 -- 172.21.15.26:0/2568769203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198016070 con 0x55f42d95dad0 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a5ffb640 1 -- 172.21.15.26:0/2568769203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8198009e50 con 0x55f42d95dad0 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81a5ffb640 1 -- 172.21.15.26:0/2568769203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980115e0 con 0x55f42d95dad0 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.909+0000 7f81b49b4640 1 -- 172.21.15.26:0/26686254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f81a8103b10 con 0x7f81b0103610 2024-05-20T21:08:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 -- 172.21.15.26:0/2568769203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 msgr2=0x55f42d95def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81aeffd640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f819c0b2920 0x7f819c0b4de0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55f42d95e8c0 tx=0x7f81a00097d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d95def0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8198009b60 tx=0x7f81980106e0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 -- 172.21.15.26:0/2568769203 shutdown_connections 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d965ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d962fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 --2- 172.21.15.26:0/2568769203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d95def0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 -- 172.21.15.26:0/2568769203 >> 172.21.15.26:0/2568769203 conn(0x55f42d954d30 msgr2=0x55f42d955130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 -- 172.21.15.26:0/2568769203 shutdown_connections 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 -- 172.21.15.26:0/2568769203 wait complete. 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.910+0000 7f81b87ac740 1 -- start start 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 Processor -- start 2024-05-20T21:08:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d967cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d968200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f42d5a0790 con 0x55f42d963630 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f42d876890 con 0x55f42d95dad0 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f42d9665e0 con 0x55f42d95e4c0 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81b87ac740 10 client.0 _ll_register_callbacks cb 0x55f42d5cea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34386/0 (socket says 172.21.15.26:34386) 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 -- 172.21.15.26:0/1501132322 learned_addr learned my addr 172.21.15.26:0/1501132322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81a7fff640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d968200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81aeffd640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d967cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 -- 172.21.15.26:0/1501132322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 msgr2=0x55f42d968200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d968200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 -- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 msgr2=0x55f42d967cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d967cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.911+0000 7f81af7fe640 1 -- 172.21.15.26:0/1501132322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42d69b620 con 0x55f42d963630 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81aeffd640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d967cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81af7fe640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96f240 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f81a80c7150 tx=0x7f81a80caca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61513d68b41db2d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f818effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f818effd640 1 -- 172.21.15.26:0/1501132322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a80e5060 con 0x55f42d963630 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f818effd640 1 -- 172.21.15.26:0/1501132322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f81a80cf170 con 0x55f42d963630 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8198009e50 con 0x55f42d963630 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f818effd640 1 -- 172.21.15.26:0/1501132322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81a80ec050 con 0x55f42d963630 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 10 client.19329 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42d972220 con 0x55f42d963630 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f818effd640 1 -- 172.21.15.26:0/1501132322 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8198009e50 con 0x55f42d963630 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f818effd640 1 -- 172.21.15.26:0/1501132322 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f81a80d3aa0 con 0x55f42d963630 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 10 client.19329 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 10 client.19329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 20 client.19329 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d975da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 4 client.19329 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.912+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f42d8ccb80 con 0x55f42d973950 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.913+0000 7f81aeffd640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d975da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.913+0000 7f81aeffd640 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d975da0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab283e1d11784d6 server_cookie=7e49a5d0b125ab33 in_seq=0 out_seq=0 2024-05-20T21:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.913+0000 7f818effd640 10 client.19329 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f818effd640 1 -- 172.21.15.26:0/1501132322 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f42d8ccb80 con 0x55f42d973950 2024-05-20T21:08:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f818effd640 10 client.19329 handle_command_reply: tid=0 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 client.19329 shutdown 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 msgr2=0x55f42d96f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96f240 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f81a80c7150 tx=0x7f81a80caca0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 shutdown_connections 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d975da0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96f240 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d968200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f818effd640 0 client.19329 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 --2- 172.21.15.26:0/1501132322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d967cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 >> 172.21.15.26:0/1501132322 conn(0x55f42d954d30 msgr2=0x55f42d953fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 shutdown_connections 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 1 -- 172.21.15.26:0/1501132322 wait complete. 2024-05-20T21:08:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.914+0000 7f81b87ac740 20 client.19329 trim_cache size 0 max 16384 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 Processor -- start 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 -- start start 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d9672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d96f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d955110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01aa590 con 0x55f42d95dad0 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01a7e60 con 0x55f42d95e4c0 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81b87ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01a5cb0 con 0x55f42d963630 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d9672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d9672e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34402/0 (socket says 172.21.15.26:34402) 2024-05-20T21:08:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 -- 172.21.15.26:0/3195688749 learned_addr learned my addr 172.21.15.26:0/3195688749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81af7fe640 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d955110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81a7fff640 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d96f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 -- 172.21.15.26:0/3195688749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 msgr2=0x55f42d955110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d955110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 -- 172.21.15.26:0/3195688749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 msgr2=0x55f42d96f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d96f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81aeffd640 1 -- 172.21.15.26:0/3195688749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42d954b60 con 0x55f42d95dad0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.930+0000 7f81a7fff640 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d96f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.931+0000 7f81aeffd640 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d9672e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f81a0015040 tx=0x7f81a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f288e1082814d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.931+0000 7f81a57fa640 1 -- 172.21.15.26:0/3195688749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0005020 con 0x55f42d95dad0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.931+0000 7f81a57fa640 1 -- 172.21.15.26:0/3195688749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81a00047a0 con 0x55f42d95dad0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.931+0000 7f81a57fa640 1 -- 172.21.15.26:0/3195688749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0005650 con 0x55f42d95dad0 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 -- 172.21.15.26:0/3195688749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 msgr2=0x55f42d9672e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d9672e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f81a0015040 tx=0x7f81a0016000 comp rx=0 tx=0).stop 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 -- 172.21.15.26:0/3195688749 shutdown_connections 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d963630 0x55f42d955110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95e4c0 0x55f42d96f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3195688749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d95dad0 0x55f42d9672e0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 -- 172.21.15.26:0/3195688749 >> 172.21.15.26:0/3195688749 conn(0x55f42d954d30 msgr2=0x55f42d962e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 -- 172.21.15.26:0/3195688749 shutdown_connections 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 -- 172.21.15.26:0/3195688749 wait complete. 2024-05-20T21:08:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 -- start start 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 Processor -- start 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d96c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d96cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.932+0000 7f81b87ac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81b87ac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01a6320 con 0x55f42d963630 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81b87ac740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01a7790 con 0x55f42d95dad0 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81b87ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01a69f0 con 0x55f42d95e4c0 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81b87ac740 10 client.0 _ll_register_callbacks cb 0x7f81a8030ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34414/0 (socket says 172.21.15.26:34414) 2024-05-20T21:08:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 -- 172.21.15.26:0/3245984083 learned_addr learned my addr 172.21.15.26:0/3245984083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81aeffd640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d96c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81a7fff640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d96cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 -- 172.21.15.26:0/3245984083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 msgr2=0x55f42d96cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d96cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 -- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 msgr2=0x55f42d96c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d96c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 -- 172.21.15.26:0/3245984083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42d96e180 con 0x55f42d963630 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81a7fff640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d96cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81af7fe640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96db90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f81a802ae80 tx=0x7f81a80cf380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53127d6f7989ac8a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f818effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f818effd640 1 -- 172.21.15.26:0/3245984083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a80a9060 con 0x55f42d963630 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f818effd640 1 -- 172.21.15.26:0/3245984083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f81a8004740 con 0x55f42d963630 2024-05-20T21:08:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.933+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a00047a0 con 0x55f42d963630 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f818effd640 1 -- 172.21.15.26:0/3245984083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81a80caa20 con 0x55f42d963630 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 10 client.19335 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42d96f470 con 0x55f42d963630 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f818effd640 1 -- 172.21.15.26:0/3245984083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81a00047a0 con 0x55f42d963630 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f818effd640 1 -- 172.21.15.26:0/3245984083 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f81a80b10d0 con 0x55f42d963630 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 10 client.19335 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 10 client.19335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 20 client.19335 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d9e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 4 client.19335 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f42d8c6360 con 0x55f42d973950 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.934+0000 7f81aeffd640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d9e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.935+0000 7f81aeffd640 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d9e0a70 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3f125a9aa0f37e server_cookie=75b4133b2be6c2ae in_seq=0 out_seq=0 2024-05-20T21:08:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.935+0000 7f818effd640 10 client.19335 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.951+0000 7f818effd640 1 -- 172.21.15.26:0/3245984083 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f42d8c6360 con 0x55f42d973950 2024-05-20T21:08:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f818effd640 10 client.19335 handle_command_reply: tid=0 2024-05-20T21:08:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 client.19335 shutdown 2024-05-20T21:08:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 msgr2=0x55f42d96db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96db90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f81a802ae80 tx=0x7f81a80cf380 comp rx=0 tx=0).stop 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 shutdown_connections 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55f42d973950 0x55f42d9e0a70 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f42d963630 0x55f42d96db90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f42d95e4c0 0x55f42d96cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 --2- 172.21.15.26:0/3245984083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f42d95dad0 0x55f42d96c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 >> 172.21.15.26:0/3245984083 conn(0x55f42d954d30 msgr2=0x55f42d962d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f818effd640 0 client.19335 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 shutdown_connections 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 1 -- 172.21.15.26:0/3245984083 wait complete. 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.952+0000 7f81b87ac740 20 client.19335 trim_cache size 0 max 16384 2024-05-20T21:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:08:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:08:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:08:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f819c0b2920 msgr2=0x7f819c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f819c0b2920 0x7f819c0b4de0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55f42d95e8c0 tx=0x7f81a00097d0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 msgr2=0x7f81b01a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b01a00a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f81a802aeb0 tx=0x7f81a8003c20 comp rx=0 tx=0).stop 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 shutdown_connections 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f819c0b2920 0x7f819c0b4de0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81b0103610 0x7f81b01a00a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0102c50 0x7f81b0199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 --2- 172.21.15.26:0/26686254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81b0101a60 0x7f81b0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 >> 172.21.15.26:0/26686254 conn(0x7f81b00fcfd0 msgr2=0x7f81b00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 shutdown_connections 2024-05-20T21:08:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:41.953+0000 7f81b59b6640 1 -- 172.21.15.26:0/26686254 wait complete. 2024-05-20T21:08:42.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.27 MB/sec execute 528 sec latency 17225.798 ms 2024-05-20T21:08:43.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:08:43.388 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:43.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.27 MB/sec execute 529 sec latency 18225.857 ms 2024-05-20T21:08:43.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:43 smithi026 ceph-mon[31990]: pgmap v767: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 51 KiB/s wr, 23 op/s 2024-05-20T21:08:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:43 smithi055 ceph-mon[35213]: pgmap v767: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 51 KiB/s wr, 23 op/s 2024-05-20T21:08:43.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:43 smithi144 ceph-mon[35660]: pgmap v767: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 51 KiB/s wr, 23 op/s 2024-05-20T21:08:44.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.26 MB/sec execute 530 sec latency 19225.904 ms 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 -- 172.21.15.26:0/2987612551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9101028c0 msgr2=0x7ff910110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 --2- 172.21.15.26:0/2987612551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9101028c0 0x7ff910110330 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b0a0 tx=0x7ff90800b3b0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 -- 172.21.15.26:0/2987612551 shutdown_connections 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 --2- 172.21.15.26:0/2987612551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9101028c0 0x7ff910110330 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 --2- 172.21.15.26:0/2987612551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff910108620 0x7ff910108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 --2- 172.21.15.26:0/2987612551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff910101f00 0x7ff910102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 -- 172.21.15.26:0/2987612551 >> 172.21.15.26:0/2987612551 conn(0x7ff910077780 msgr2=0x7ff910077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 -- 172.21.15.26:0/2987612551 shutdown_connections 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.290+0000 7ff917ba6700 1 -- 172.21.15.26:0/2987612551 wait complete. 2024-05-20T21:08:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 Processor -- start 2024-05-20T21:08:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 -- start start 2024-05-20T21:08:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff910101f00 0x7ff91019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9101028c0 0x7ff91019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 0x7ff910199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff910113620 con 0x7ff910108620 2024-05-20T21:08:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101134a0 con 0x7ff910101f00 2024-05-20T21:08:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff917ba6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101137a0 con 0x7ff9101028c0 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff916143700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 0x7ff910199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff916143700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 0x7ff910199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34434/0 (socket says 172.21.15.26:34434) 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.291+0000 7ff916143700 1 -- 172.21.15.26:0/1414436263 learned_addr learned my addr 172.21.15.26:0/1414436263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff915942700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff910101f00 0x7ff91019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff915141700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9101028c0 0x7ff91019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff916143700 1 -- 172.21.15.26:0/1414436263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9101028c0 msgr2=0x7ff91019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff916143700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9101028c0 0x7ff91019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff916143700 1 -- 172.21.15.26:0/1414436263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff910101f00 msgr2=0x7ff91019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff916143700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff910101f00 0x7ff91019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff916143700 1 -- 172.21.15.26:0/1414436263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91019a080 con 0x7ff910108620 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff916143700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 0x7ff910199920 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b780 tx=0x7ff908004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff906ffd700 1 -- 172.21.15.26:0/1414436263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908021070 con 0x7ff910108620 2024-05-20T21:08:45.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff906ffd700 1 -- 172.21.15.26:0/1414436263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff908007b80 con 0x7ff910108620 2024-05-20T21:08:45.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91019a310 con 0x7ff910108620 2024-05-20T21:08:45.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff906ffd700 1 -- 172.21.15.26:0/1414436263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080052a0 con 0x7ff910108620 2024-05-20T21:08:45.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.292+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a6630 con 0x7ff910108620 2024-05-20T21:08:45.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.295+0000 7ff906ffd700 1 -- 172.21.15.26:0/1414436263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff908024070 con 0x7ff910108620 2024-05-20T21:08:45.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.295+0000 7ff906ffd700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8fc0b2880 0x7ff8fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff915942700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8fc0b2880 0x7ff8fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff906ffd700 1 -- 172.21.15.26:0/1414436263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff90800f020 con 0x7ff910108620 2024-05-20T21:08:45.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 Processor -- start 2024-05-20T21:08:45.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff915942700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8fc0b2880 0x7ff8fc0b4d40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff91019aa60 tx=0x7ff90c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:45.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 -- start start 2024-05-20T21:08:45.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 0x562a06875c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a06879940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06879f60 0x562a0687a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a067d4c90 con 0x562a06875860 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a067c4830 con 0x562a06879f60 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.296+0000 7ff92243cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a067c8870 con 0x562a06876210 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 0x562a06875c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 0x562a06875c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34436/0 (socket says 172.21.15.26:34436) 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 -- 172.21.15.26:0/161298142 learned_addr learned my addr 172.21.15.26:0/161298142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff916143700 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06879f60 0x562a0687a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915141700 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a06879940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 -- 172.21.15.26:0/161298142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 msgr2=0x562a06879940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a06879940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 -- 172.21.15.26:0/161298142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06879f60 msgr2=0x562a0687a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06879f60 0x562a0687a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 -- 172.21.15.26:0/161298142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a067148d0 con 0x562a06875860 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915141700 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a06879940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff915942700 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 0x562a06875c40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff90c011d30 tx=0x7ff90c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4449f18613f8af61 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff8fbfff700 1 -- 172.21.15.26:0/161298142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c003680 con 0x562a06875860 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff8fbfff700 1 -- 172.21.15.26:0/161298142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff90c003cb0 con 0x562a06875860 2024-05-20T21:08:45.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.297+0000 7ff8fbfff700 1 -- 172.21.15.26:0/161298142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c01ebc0 con 0x562a06875860 2024-05-20T21:08:45.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 -- 172.21.15.26:0/161298142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 msgr2=0x562a06875c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 0x562a06875c40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff90c011d30 tx=0x7ff90c014000 comp rx=0 tx=0).stop 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 -- 172.21.15.26:0/161298142 shutdown_connections 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06875860 0x562a06875c40 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06879f60 0x562a0687a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 --2- 172.21.15.26:0/161298142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a06879940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 -- 172.21.15.26:0/161298142 >> 172.21.15.26:0/161298142 conn(0x562a067a3080 msgr2=0x562a0686fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 -- 172.21.15.26:0/161298142 shutdown_connections 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.298+0000 7ff92243cb80 1 -- 172.21.15.26:0/161298142 wait complete. 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 -- start start 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 Processor -- start 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06881350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a0687e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a0687eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a066cafa0 con 0x562a06879f60 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562a067667c0 con 0x562a06875860 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a06882120 con 0x562a06876210 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff92243cb80 10 client.0 _ll_register_callbacks cb 0x562a0675f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff916143700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a0687eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff915942700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06881350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff916143700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a0687eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34444/0 (socket says 172.21.15.26:34444) 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff916143700 1 -- 172.21.15.26:0/3720517644 learned_addr learned my addr 172.21.15.26:0/3720517644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.299+0000 7ff915141700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a0687e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff916143700 1 -- 172.21.15.26:0/3720517644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 msgr2=0x562a0687e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff916143700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a0687e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff916143700 1 -- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 msgr2=0x562a06881350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff916143700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06881350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff916143700 1 -- 172.21.15.26:0/3720517644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a06763690 con 0x562a06879f60 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff915942700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06881350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff916143700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a0687eb90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff9080062a0 tx=0x7ff9080aacb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637d212d4a274c4d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff8f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff8f97fa700 1 -- 172.21.15.26:0/3720517644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080c1060 con 0x562a06879f60 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff8f97fa700 1 -- 172.21.15.26:0/3720517644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff908007d30 con 0x562a06879f60 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90c003cb0 con 0x562a06879f60 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff8f97fa700 1 -- 172.21.15.26:0/3720517644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9080aadb0 con 0x562a06879f60 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff92243cb80 10 client.19347 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a0688c760 con 0x562a06879f60 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.300+0000 7ff8f97fa700 1 -- 172.21.15.26:0/3720517644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff90c003cb0 con 0x562a06879f60 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff8f97fa700 1 -- 172.21.15.26:0/3720517644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff9080c5030 con 0x562a06879f60 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff92243cb80 10 client.19347 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff92243cb80 10 client.19347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff92243cb80 20 client.19347 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff92243cb80 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a0688ed70 0x562a06891190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff92243cb80 4 client.19347 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a068419e0 con 0x562a0688ed70 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff915942700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a0688ed70 0x562a06891190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff915942700 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a0688ed70 0x562a06891190 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47df4cd40ade019a server_cookie=623a787fd091c05b in_seq=0 out_seq=0 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.301+0000 7ff8f97fa700 10 client.19347 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff8f97fa700 1 -- 172.21.15.26:0/3720517644 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562a068419e0 con 0x562a0688ed70 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff8f97fa700 10 client.19347 handle_command_reply: tid=0 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 client.19347 shutdown 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 msgr2=0x562a0687eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a0687eb90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff9080062a0 tx=0x7ff9080aacb0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 shutdown_connections 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a0688ed70 0x562a06891190 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a0687eb90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06881350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff8f97fa700 0 client.19347 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 --2- 172.21.15.26:0/3720517644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a0687e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 >> 172.21.15.26:0/3720517644 conn(0x562a067a3080 msgr2=0x562a0686f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.302+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 shutdown_connections 2024-05-20T21:08:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.303+0000 7ff92243cb80 1 -- 172.21.15.26:0/3720517644 wait complete. 2024-05-20T21:08:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.303+0000 7ff92243cb80 20 client.19347 trim_cache size 0 max 16384 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 Processor -- start 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 -- start start 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06879140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 0x562a0687d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06879f60 0x562a0686fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101b04e0 con 0x562a06876210 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101add70 con 0x562a06875860 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff92243cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101abbc0 con 0x562a06879f60 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 0x562a0687d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 0x562a0687d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34450/0 (socket says 172.21.15.26:34450) 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 -- 172.21.15.26:0/2323756317 learned_addr learned my addr 172.21.15.26:0/2323756317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915942700 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06879140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff916143700 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06879f60 0x562a0686fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 -- 172.21.15.26:0/2323756317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06879f60 msgr2=0x562a0686fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06879f60 0x562a0686fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 -- 172.21.15.26:0/2323756317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 msgr2=0x562a06879140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06879140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 -- 172.21.15.26:0/2323756317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0687e420 con 0x562a06876210 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.318+0000 7ff915141700 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 0x562a0687d650 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff90000ac20 tx=0x7ff90000af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd61e9f6d909d5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff8fb7fe700 1 -- 172.21.15.26:0/2323756317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900012b10 con 0x562a06876210 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff8fb7fe700 1 -- 172.21.15.26:0/2323756317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff900012cb0 con 0x562a06876210 2024-05-20T21:08:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff8fb7fe700 1 -- 172.21.15.26:0/2323756317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9000116a0 con 0x562a06876210 2024-05-20T21:08:45.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 -- 172.21.15.26:0/2323756317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 msgr2=0x562a0687d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 0x562a0687d650 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff90000ac20 tx=0x7ff90000af30 comp rx=0 tx=0).stop 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 -- 172.21.15.26:0/2323756317 shutdown_connections 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06879f60 0x562a0686fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a06879140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2323756317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06876210 0x562a0687d650 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 -- 172.21.15.26:0/2323756317 >> 172.21.15.26:0/2323756317 conn(0x562a067a3080 msgr2=0x562a0687cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 -- 172.21.15.26:0/2323756317 shutdown_connections 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.319+0000 7ff92243cb80 1 -- 172.21.15.26:0/2323756317 wait complete. 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 -- start start 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 Processor -- start 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a068f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a068f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a06884a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101ac230 con 0x562a06879f60 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101ad6a0 con 0x562a06875860 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101ac900 con 0x562a06876210 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff92243cb80 10 client.0 _ll_register_callbacks cb 0x562a0675f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff916143700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a06884a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff916143700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a06884a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34452/0 (socket says 172.21.15.26:34452) 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.320+0000 7ff916143700 1 -- 172.21.15.26:0/2787155092 learned_addr learned my addr 172.21.15.26:0/2787155092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff915141700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a068f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff915942700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a068f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff916143700 1 -- 172.21.15.26:0/2787155092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 msgr2=0x562a068f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff916143700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a068f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff916143700 1 -- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 msgr2=0x562a068f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff916143700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a068f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff916143700 1 -- 172.21.15.26:0/2787155092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a06885170 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff915942700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a068f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff916143700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a06884a70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff9080a7350 tx=0x7ff9080b3980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ea1343e787c400 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff8f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff8f97fa700 1 -- 172.21.15.26:0/2787155092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080aa520 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff8f97fa700 1 -- 172.21.15.26:0/2787155092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff9080a73a0 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff900012cb0 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff8f97fa700 1 -- 172.21.15.26:0/2787155092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9080a6c70 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff92243cb80 10 client.19353 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.321+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a068f9540 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff8f97fa700 1 -- 172.21.15.26:0/2787155092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff900012cb0 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff8f97fa700 1 -- 172.21.15.26:0/2787155092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff9080c5040 con 0x562a06879f60 2024-05-20T21:08:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff92243cb80 10 client.19353 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff92243cb80 10 client.19353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff92243cb80 20 client.19353 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a068f9a00 0x562a068fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff92243cb80 4 client.19353 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a067d6be0 con 0x562a068f9a00 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.322+0000 7ff915942700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a068f9a00 0x562a068fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff915942700 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a068f9a00 0x562a068fbdf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52a2f54c46a268fa server_cookie=bbae4ccbf1b164ac in_seq=0 out_seq=0 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff8f97fa700 10 client.19353 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff8f97fa700 1 -- 172.21.15.26:0/2787155092 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a067d6be0 con 0x562a068f9a00 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff8f97fa700 10 client.19353 handle_command_reply: tid=0 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 client.19353 shutdown 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 msgr2=0x562a06884a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a06884a70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff9080a7350 tx=0x7ff9080b3980 comp rx=0 tx=0).stop 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 shutdown_connections 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x562a068f9a00 0x562a068fbdf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a06879f60 0x562a06884a70 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562a06875860 0x562a068f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 --2- 172.21.15.26:0/2787155092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a06876210 0x562a068f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff8f97fa700 0 client.19353 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 >> 172.21.15.26:0/2787155092 conn(0x562a067a3080 msgr2=0x562a0687d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 shutdown_connections 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 1 -- 172.21.15.26:0/2787155092 wait complete. 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.323+0000 7ff92243cb80 20 client.19353 trim_cache size 0 max 16384 2024-05-20T21:08:45.328 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:08:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8fc0b2880 msgr2=0x7ff8fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8fc0b2880 0x7ff8fc0b4d40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff91019aa60 tx=0x7ff90c009260 comp rx=0 tx=0).stop 2024-05-20T21:08:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 msgr2=0x7ff910199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 0x7ff910199920 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b780 tx=0x7ff908004d00 comp rx=0 tx=0).stop 2024-05-20T21:08:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 shutdown_connections 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff8fc0b2880 0x7ff8fc0b4d40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff910108620 0x7ff910199920 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff910101f00 0x7ff91019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 --2- 172.21.15.26:0/1414436263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9101028c0 0x7ff91019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 >> 172.21.15.26:0/1414436263 conn(0x7ff910077780 msgr2=0x7ff910100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 shutdown_connections 2024-05-20T21:08:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:45.324+0000 7ff917ba6700 1 -- 172.21.15.26:0/1414436263 wait complete. 2024-05-20T21:08:45.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:45 smithi026 ceph-mon[31990]: pgmap v768: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-05-20T21:08:45.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.26 MB/sec execute 531 sec latency 20225.937 ms 2024-05-20T21:08:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:45 smithi055 ceph-mon[35213]: pgmap v768: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-05-20T21:08:45.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:08:45.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:45 smithi144 ceph-mon[35660]: pgmap v768: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-05-20T21:08:46.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:08:46.361 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:46.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.25 MB/sec execute 532 sec latency 21226.028 ms 2024-05-20T21:08:47.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.25 MB/sec execute 533 sec latency 22226.123 ms 2024-05-20T21:08:47.602 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:47 smithi026 ceph-mon[31990]: pgmap v769: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-05-20T21:08:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:47.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:47 smithi055 ceph-mon[35213]: pgmap v769: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-05-20T21:08:47.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:47 smithi144 ceph-mon[35660]: pgmap v769: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 -- 172.21.15.26:0/1605990980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 msgr2=0x7f735810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 --2- 172.21.15.26:0/1605990980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f735810f2f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f735000b0a0 tx=0x7f735000b3b0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 -- 172.21.15.26:0/1605990980 shutdown_connections 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 --2- 172.21.15.26:0/1605990980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f735810f2f0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 --2- 172.21.15.26:0/1605990980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7358108520 0x7f7358108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 --2- 172.21.15.26:0/1605990980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358102440 0x7f73581028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 -- 172.21.15.26:0/1605990980 >> 172.21.15.26:0/1605990980 conn(0x7f73580793f0 msgr2=0x7f7358079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 -- 172.21.15.26:0/1605990980 shutdown_connections 2024-05-20T21:08:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 -- 172.21.15.26:0/1605990980 wait complete. 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.419+0000 7f735e756700 1 Processor -- start 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 -- start start 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7358102440 0x7f73581986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f7358198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358108520 0x7f7358199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7358112480 con 0x7f7358102e00 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7358112300 con 0x7f7358108520 2024-05-20T21:08:48.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735e756700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7358112600 con 0x7f7358102440 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f7358198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f7358198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34466/0 (socket says 172.21.15.26:34466) 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 -- 172.21.15.26:0/3924698822 learned_addr learned my addr 172.21.15.26:0/3924698822 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f7357fff700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7358102440 0x7f73581986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735ccf3700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358108520 0x7f7358199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 -- 172.21.15.26:0/3924698822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7358102440 msgr2=0x7f73581986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7358102440 0x7f73581986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 -- 172.21.15.26:0/3924698822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358108520 msgr2=0x7f7358199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358108520 0x7f7358199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 -- 172.21.15.26:0/3924698822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73581a7f70 con 0x7f7358102e00 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f735ccf3700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358108520 0x7f7358199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.420+0000 7f734ffff700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f7358198c30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f734000c9b0 tx=0x7f734000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:48.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.421+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340007b00 con 0x7f7358102e00 2024-05-20T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.421+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73581a8200 con 0x7f7358102e00 2024-05-20T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.421+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7340007ca0 con 0x7f7358102e00 2024-05-20T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.421+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340018660 con 0x7f7358102e00 2024-05-20T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.421+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73581a8740 con 0x7f7358102e00 2024-05-20T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.422+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f735806a230 con 0x7f7358102e00 2024-05-20T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.423+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f734000f450 con 0x7f7358102e00 2024-05-20T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.424+0000 7f7355ffb700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73440b2880 0x7f73440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.424+0000 7f7357fff700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73440b2880 0x7f73440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.424+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f73400e22b0 con 0x7f7358102e00 2024-05-20T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.425+0000 7f7357fff700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73440b2880 0x7f73440b4d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7348007b60 tx=0x7f73480073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:48.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.429+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7340014030 con 0x7f7358102e00 2024-05-20T21:08:48.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.25 MB/sec execute 534 sec latency 23226.175 ms 2024-05-20T21:08:48.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.710+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f735806b810 con 0x7f7358102e00 2024-05-20T21:08:48.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.710+0000 7f7355ffb700 1 -- 172.21.15.26:0/3924698822 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f73400a9f10 con 0x7f7358102e00 2024-05-20T21:08:48.711 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:08:48.711 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73440b2880 msgr2=0x7f73440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73440b2880 0x7f73440b4d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7348007b60 tx=0x7f73480073d0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 msgr2=0x7f7358198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f7358198c30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f734000c9b0 tx=0x7f734000ccc0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 shutdown_connections 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7358108520 0x7f7358199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7358102440 0x7f73581986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f73440b2880 0x7f73440b4d40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 --2- 172.21.15.26:0/3924698822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7358102e00 0x7f7358198c30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 >> 172.21.15.26:0/3924698822 conn(0x7f73580793f0 msgr2=0x7f73581004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 shutdown_connections 2024-05-20T21:08:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:48.713+0000 7f735e756700 1 -- 172.21.15.26:0/3924698822 wait complete. 2024-05-20T21:08:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:08:49.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.24 MB/sec execute 535 sec latency 24226.269 ms 2024-05-20T21:08:49.602 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:49 smithi026 ceph-mon[31990]: pgmap v770: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-20T21:08:49.602 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:49 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3924698822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:49.637 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id ec03d4bd --release --await' 2024-05-20T21:08:49.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id ec03d4bd --release --await 2024-05-20T21:08:49.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:49 smithi055 ceph-mon[35213]: pgmap v770: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-20T21:08:49.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:49 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3924698822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:49.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:49 smithi144 ceph-mon[35660]: pgmap v770: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-20T21:08:49.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:49 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3924698822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:50.132 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:50.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.24 MB/sec execute 536 sec latency 25226.329 ms 2024-05-20T21:08:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/3803492270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41088f0 msgr2=0x7fc8d4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/3803492270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41088f0 0x7fc8d4110470 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00b0d0 tx=0x7fc8cc00b3e0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/3803492270 shutdown_connections 2024-05-20T21:08:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/3803492270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41088f0 0x7fc8d4110470 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/3803492270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41037f0 0x7fc8d4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/3803492270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d41041a0 0x7fc8d41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/3803492270 >> 172.21.15.26:0/3803492270 conn(0x7fc8d407a6a0 msgr2=0x7fc8d407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/3803492270 shutdown_connections 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/3803492270 wait complete. 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 Processor -- start 2024-05-20T21:08:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.813+0000 7fc8dc7f4700 1 -- start start 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dc7f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d41037f0 0x7fc8d419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dc7f4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 0x7fc8d419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dc7f4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41088f0 0x7fc8d4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dc7f4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4113570 con 0x7fc8d41041a0 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dc7f4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41133f0 con 0x7fc8d41088f0 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dc7f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41136f0 con 0x7fc8d41037f0 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 0x7fc8d419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 0x7fc8d419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56786/0 (socket says 172.21.15.26:56786) 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/203636715 learned_addr learned my addr 172.21.15.26:0/203636715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8da590700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d41037f0 0x7fc8d419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dad91700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41088f0 0x7fc8d4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/203636715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d41037f0 msgr2=0x7fc8d419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d41037f0 0x7fc8d419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/203636715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41088f0 msgr2=0x7fc8d4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41088f0 0x7fc8d4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/203636715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d419a030 con 0x7fc8d41041a0 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8dad91700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41088f0 0x7fc8d4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.814+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 0x7fc8d419fc10 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400d950 tx=0x7fc8c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.815+0000 7fc8cb7fe700 1 -- 172.21.15.26:0/203636715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4016070 con 0x7fc8d41041a0 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.815+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d419a320 con 0x7fc8d41041a0 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.815+0000 7fc8cb7fe700 1 -- 172.21.15.26:0/203636715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8c4009dd0 con 0x7fc8d41041a0 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.815+0000 7fc8cb7fe700 1 -- 172.21.15.26:0/203636715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c400f3c0 con 0x7fc8d41041a0 2024-05-20T21:08:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.815+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41a6530 con 0x7fc8d41041a0 2024-05-20T21:08:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.817+0000 7fc8cb7fe700 1 -- 172.21.15.26:0/203636715 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc8c4019070 con 0x7fc8d41041a0 2024-05-20T21:08:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.818+0000 7fc8cb7fe700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc8c00b2880 0x7fc8c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.818+0000 7fc8da590700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc8c00b2880 0x7fc8c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.818+0000 7fc8cb7fe700 1 -- 172.21.15.26:0/203636715 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fc8c40e1ee0 con 0x7fc8d41041a0 2024-05-20T21:08:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.818+0000 7fc8e708ab80 1 Processor -- start 2024-05-20T21:08:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8da590700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc8c00b2880 0x7fc8c00b4d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0005950 tx=0x7fc8d000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 -- start start 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff207050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff20b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560cff161c90 con 0x560cff207620 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560cff151830 con 0x560cff20b370 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560cff155870 con 0x560cff206c70 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56800/0 (socket says 172.21.15.26:56800) 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/1110694368 learned_addr learned my addr 172.21.15.26:0/1110694368 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8dad91700 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff20b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8da590700 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff207050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/1110694368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 msgr2=0x560cff207050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff207050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/1110694368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 msgr2=0x560cff20b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff20b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/1110694368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cff0a18d0 con 0x560cff207620 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8dad91700 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff20b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.819+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20ad50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a5520 tx=0x7fc8c40a9ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97579e0a95357551 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8c8ff9700 1 -- 172.21.15.26:0/1110694368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4020070 con 0x560cff207620 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8c8ff9700 1 -- 172.21.15.26:0/1110694368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8c40ad850 con 0x560cff207620 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8c8ff9700 1 -- 172.21.15.26:0/1110694368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40b0dd0 con 0x560cff207620 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1110694368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 msgr2=0x560cff20ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20ad50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a5520 tx=0x7fc8c40a9ba0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1110694368 shutdown_connections 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff20b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff207050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1110694368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20ad50 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.820+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1110694368 >> 172.21.15.26:0/1110694368 conn(0x560cff130080 msgr2=0x560cff1fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1110694368 shutdown_connections 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1110694368 wait complete. 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 -- start start 2024-05-20T21:08:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 Processor -- start 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff2127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff210030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560cff057fa0 con 0x560cff207620 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560cff0f37c0 con 0x560cff20b370 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560cff2134d0 con 0x560cff206c70 2024-05-20T21:08:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.821+0000 7fc8e708ab80 10 client.0 _ll_register_callbacks cb 0x560cff0ec920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56806/0 (socket says 172.21.15.26:56806) 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/3873427130 learned_addr learned my addr 172.21.15.26:0/3873427130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8da590700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff2127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/3873427130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 msgr2=0x560cff2127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff2127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8dad91700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff210030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 msgr2=0x560cff210030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff210030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/3873427130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cff0f0690 con 0x560cff207620 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20faf0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a9b00 tx=0x7fc8c40109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3682ac4b1ff0e9e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8bdffb700 1 -- 172.21.15.26:0/3873427130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40c2020 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8bdffb700 1 -- 172.21.15.26:0/3873427130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fc8c40b0de0 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c40ad850 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8bdffb700 1 -- 172.21.15.26:0/3873427130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8c4020070 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8e708ab80 10 client.19371 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560cff21dc00 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.822+0000 7fc8bdffb700 1 -- 172.21.15.26:0/3873427130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8c40ad850 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8bdffb700 1 -- 172.21.15.26:0/3873427130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fc8c40a8700 con 0x560cff207620 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8e708ab80 10 client.19371 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8e708ab80 10 client.19371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8e708ab80 20 client.19371 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff2200d0 0x560cff2224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8e708ab80 4 client.19371 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cff1ce9e0 con 0x560cff2200d0 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8da590700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff2200d0 0x560cff2224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8bdffb700 10 client.19371 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.823+0000 7fc8da590700 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff2200d0 0x560cff2224c0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca96701e8adf649c server_cookie=9cd3ec5d473897f0 in_seq=0 out_seq=0 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8bdffb700 1 -- 172.21.15.26:0/3873427130 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560cff1ce9e0 con 0x560cff2200d0 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8bdffb700 10 client.19371 handle_command_reply: tid=0 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 client.19371 shutdown 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 msgr2=0x560cff20faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20faf0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a9b00 tx=0x7fc8c40109e0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 shutdown_connections 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff2200d0 0x560cff2224c0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff20b370 0x560cff210030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff2127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/3873427130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff20faf0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 >> 172.21.15.26:0/3873427130 conn(0x560cff130080 msgr2=0x560cff1fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:50.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8bdffb700 0 client.19371 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:50.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.824+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 shutdown_connections 2024-05-20T21:08:50.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.825+0000 7fc8e708ab80 1 -- 172.21.15.26:0/3873427130 wait complete. 2024-05-20T21:08:50.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.825+0000 7fc8e708ab80 20 client.19371 trim_cache size 0 max 16384 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 Processor -- start 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 -- start start 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff1fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff207620 0x560cff225100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 0x560cff225640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41ae410 con 0x560cff20b370 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41abca0 con 0x560cff207620 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41a9af0 con 0x560cff206c70 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 0x560cff225640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 0x560cff225640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56808/0 (socket says 172.21.15.26:56808) 2024-05-20T21:08:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 -- 172.21.15.26:0/1951431122 learned_addr learned my addr 172.21.15.26:0/1951431122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8da590700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff1fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff207620 0x560cff225100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 -- 172.21.15.26:0/1951431122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 msgr2=0x560cff1fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff1fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 -- 172.21.15.26:0/1951431122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff207620 msgr2=0x560cff225100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff207620 0x560cff225100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.840+0000 7fc8dad91700 1 -- 172.21.15.26:0/1951431122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cff212b60 con 0x560cff20b370 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.841+0000 7fc8da590700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff1fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.841+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff207620 0x560cff225100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.841+0000 7fc8dad91700 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 0x560cff225640 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009d40 tx=0x7fc8cc009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1874f184e99cdff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.841+0000 7fc8bffff700 1 -- 172.21.15.26:0/1951431122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc021070 con 0x560cff20b370 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.841+0000 7fc8bffff700 1 -- 172.21.15.26:0/1951431122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8cc009450 con 0x560cff20b370 2024-05-20T21:08:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.841+0000 7fc8bffff700 1 -- 172.21.15.26:0/1951431122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc005030 con 0x560cff20b370 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1951431122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 msgr2=0x560cff225640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 0x560cff225640 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009d40 tx=0x7fc8cc009d70 comp rx=0 tx=0).stop 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1951431122 shutdown_connections 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff20b370 0x560cff225640 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff206c70 0x560cff1fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/1951431122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff207620 0x560cff225100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1951431122 >> 172.21.15.26:0/1951431122 conn(0x560cff130080 msgr2=0x560cff20a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1951431122 shutdown_connections 2024-05-20T21:08:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- 172.21.15.26:0/1951431122 wait complete. 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- start start 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 Processor -- start 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff206c70 0x560cff224120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff224660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff20b370 0x560cff224ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41aa160 con 0x560cff207620 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41ab5d0 con 0x560cff206c70 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41aa830 con 0x560cff20b370 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8e708ab80 10 client.0 _ll_register_callbacks cb 0x560cff0ec920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.842+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff224660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff224660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56820/0 (socket says 172.21.15.26:56820) 2024-05-20T21:08:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/887789556 learned_addr learned my addr 172.21.15.26:0/887789556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8da590700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff206c70 0x560cff224120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8dad91700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff20b370 0x560cff224ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/887789556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff20b370 msgr2=0x560cff224ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff20b370 0x560cff224ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff206c70 msgr2=0x560cff224120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff206c70 0x560cff224120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 -- 172.21.15.26:0/887789556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cff288bc0 con 0x560cff207620 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8d9d8f700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff224660 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a8ba0 tx=0x7fc8c4010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e54e56ed619cc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8bdffb700 1 -- 172.21.15.26:0/887789556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40c2070 con 0x560cff207620 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8bdffb700 1 -- 172.21.15.26:0/887789556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fc8c40a5a90 con 0x560cff207620 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8cc009450 con 0x560cff207620 2024-05-20T21:08:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8bdffb700 1 -- 172.21.15.26:0/887789556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8c40a5360 con 0x560cff207620 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8e708ab80 10 client.19377 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.843+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560cff289e30 con 0x560cff207620 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8bdffb700 1 -- 172.21.15.26:0/887789556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8cc009450 con 0x560cff207620 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8bdffb700 1 -- 172.21.15.26:0/887789556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fc8c40ca350 con 0x560cff207620 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8e708ab80 10 client.19377 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8e708ab80 10 client.19377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8e708ab80 20 client.19377 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff28a380 0x560cff28c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8e708ab80 4 client.19377 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cff163be0 con 0x560cff28a380 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.844+0000 7fc8da590700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff28a380 0x560cff28c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8bdffb700 10 client.19377 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:50.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8da590700 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff28a380 0x560cff28c7a0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17d970052964f05 server_cookie=a24b86400f8a422a in_seq=0 out_seq=0 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8bdffb700 1 -- 172.21.15.26:0/887789556 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cff163be0 con 0x560cff28a380 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8bdffb700 10 client.19377 handle_command_reply: tid=0 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 client.19377 shutdown 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 msgr2=0x560cff224660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff224660 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a8ba0 tx=0x7fc8c4010f30 comp rx=0 tx=0).stop 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 shutdown_connections 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560cff28a380 0x560cff28c7a0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cff20b370 0x560cff224ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cff206c70 0x560cff224120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 --2- 172.21.15.26:0/887789556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cff207620 0x560cff224660 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8bdffb700 0 client.19377 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 >> 172.21.15.26:0/887789556 conn(0x560cff130080 msgr2=0x560cff201670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:50.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 shutdown_connections 2024-05-20T21:08:50.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 1 -- 172.21.15.26:0/887789556 wait complete. 2024-05-20T21:08:50.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.845+0000 7fc8e708ab80 20 client.19377 trim_cache size 0 max 16384 2024-05-20T21:08:50.851 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc8c00b2880 msgr2=0x7fc8c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc8c00b2880 0x7fc8c00b4d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0005950 tx=0x7fc8d000b410 comp rx=0 tx=0).stop 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 msgr2=0x7fc8d419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 0x7fc8d419fc10 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400d950 tx=0x7fc8c400dc60 comp rx=0 tx=0).stop 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 shutdown_connections 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d41088f0 0x7fc8d4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d41037f0 0x7fc8d419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fc8c00b2880 0x7fc8c00b4d40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 --2- 172.21.15.26:0/203636715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8d41041a0 0x7fc8d419fc10 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 >> 172.21.15.26:0/203636715 conn(0x7fc8d407a6a0 msgr2=0x7fc8d4101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:50.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 shutdown_connections 2024-05-20T21:08:50.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:50.846+0000 7fc8dc7f4700 1 -- 172.21.15.26:0/203636715 wait complete. 2024-05-20T21:08:51.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.23 MB/sec execute 537 sec latency 26226.409 ms 2024-05-20T21:08:51.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:51 smithi055 ceph-mon[35213]: pgmap v771: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T21:08:51.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:51 smithi026 ceph-mon[31990]: pgmap v771: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T21:08:51.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:51 smithi144 ceph-mon[35660]: pgmap v771: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T21:08:52.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288487 2.23 MB/sec execute 538 sec latency 27226.461 ms 2024-05-20T21:08:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 -- 172.21.15.26:0/3164812842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806bc70 msgr2=0x7fd7d806c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 --2- 172.21.15.26:0/3164812842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806bc70 0x7fd7d806c0f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009990 tx=0x7fd7c8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 -- 172.21.15.26:0/3164812842 shutdown_connections 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 --2- 172.21.15.26:0/3164812842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7d806c630 0x7fd7d81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 --2- 172.21.15.26:0/3164812842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d81095b0 0x7fd7d8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 --2- 172.21.15.26:0/3164812842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806bc70 0x7fd7d806c0f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 -- 172.21.15.26:0/3164812842 >> 172.21.15.26:0/3164812842 conn(0x7fd7d80793f0 msgr2=0x7fd7d8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 -- 172.21.15.26:0/3164812842 shutdown_connections 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.122+0000 7fd7df33b700 1 -- 172.21.15.26:0/3164812842 wait complete. 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 Processor -- start 2024-05-20T21:08:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 -- start start 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7d806bc70 0x7fd7d8102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 0x7fd7d8102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d81095b0 0x7fd7d8102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8113590 con 0x7fd7d806c630 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8113410 con 0x7fd7d806bc70 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7df33b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8113710 con 0x7fd7d81095b0 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 0x7fd7d8102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 0x7fd7d8102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56840/0 (socket says 172.21.15.26:56840) 2024-05-20T21:08:53.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 -- 172.21.15.26:0/1883668578 learned_addr learned my addr 172.21.15.26:0/1883668578 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7d806bc70 0x7fd7d8102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d81095b0 0x7fd7d8102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 -- 172.21.15.26:0/1883668578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d81095b0 msgr2=0x7fd7d8102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d81095b0 0x7fd7d8102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 -- 172.21.15.26:0/1883668578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7d806bc70 msgr2=0x7fd7d8102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7d806bc70 0x7fd7d8102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.123+0000 7fd7dc8d6700 1 -- 172.21.15.26:0/1883668578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d81a8090 con 0x7fd7d806c630 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.124+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 0x7fd7d8102a80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8006010 tx=0x7fd7c800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.124+0000 7fd7ce7fc700 1 -- 172.21.15.26:0/1883668578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8027070 con 0x7fd7d806c630 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.124+0000 7fd7ce7fc700 1 -- 172.21.15.26:0/1883668578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7c8022470 con 0x7fd7d806c630 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.124+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d81a8320 con 0x7fd7d806c630 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.124+0000 7fd7ce7fc700 1 -- 172.21.15.26:0/1883668578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8017650 con 0x7fd7d806c630 2024-05-20T21:08:53.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.124+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81a8770 con 0x7fd7d806c630 2024-05-20T21:08:53.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.127+0000 7fd7ce7fc700 1 -- 172.21.15.26:0/1883668578 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd7c8009d10 con 0x7fd7d806c630 2024-05-20T21:08:53.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.127+0000 7fd7ce7fc700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd7c40b2880 0x7fd7c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.127+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd7c40b2880 0x7fd7c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.127+0000 7fd7ce7fc700 1 -- 172.21.15.26:0/1883668578 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd7c80e2e30 con 0x7fd7d806c630 2024-05-20T21:08:53.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd7c40b2880 0x7fd7c40b4d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40045b0 tx=0x7fd7d400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:53.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 Processor -- start 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 -- start start 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 0x5560987e6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556098737e70 con 0x5560987da5b0 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556098727a10 con 0x5560987d9bf0 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.128+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55609872ba50 con 0x5560987de720 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 0x5560987e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 0x5560987e6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56850/0 (socket says 172.21.15.26:56850) 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/756399738 learned_addr learned my addr 172.21.15.26:0/756399738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/756399738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 msgr2=0x5560987deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/756399738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 msgr2=0x5560987da070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987da070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/756399738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560986d10b0 con 0x5560987da5b0 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 0x5560987e6620 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009440 tx=0x7fd7d0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f709ff53068f1687 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7c37fe700 1 -- 172.21.15.26:0/756399738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0013750 con 0x5560987da5b0 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7c37fe700 1 -- 172.21.15.26:0/756399738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d0013d80 con 0x5560987da5b0 2024-05-20T21:08:53.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.129+0000 7fd7c37fe700 1 -- 172.21.15.26:0/756399738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0015dd0 con 0x5560987da5b0 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/756399738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 msgr2=0x5560987e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 0x5560987e6620 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009440 tx=0x7fd7d0009920 comp rx=0 tx=0).stop 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/756399738 shutdown_connections 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987da5b0 0x5560987e6620 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/756399738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987da070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/756399738 >> 172.21.15.26:0/756399738 conn(0x556098705340 msgr2=0x5560987d48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/756399738 shutdown_connections 2024-05-20T21:08:53.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.130+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/756399738 wait complete. 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 -- start start 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 Processor -- start 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 0x5560987e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987e3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55609862e0a0 con 0x5560987d9bf0 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55609868fea0 con 0x5560987da5b0 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560987e6c80 con 0x5560987de720 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7dd0d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 0x5560987e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7dd0d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 0x5560987e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56854/0 (socket says 172.21.15.26:56854) 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7dd0d7700 1 -- 172.21.15.26:0/1563398436 learned_addr learned my addr 172.21.15.26:0/1563398436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7e9bd1b80 10 client.0 _ll_register_callbacks cb 0x5560986c2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.131+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7dd0d7700 1 -- 172.21.15.26:0/1563398436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 msgr2=0x5560987e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7dd0d7700 1 -- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 msgr2=0x5560987e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7dd0d7700 1 -- 172.21.15.26:0/1563398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560986779d0 con 0x5560987d9bf0 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 0x5560987e6230 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4003e90 tx=0x7fd7d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d415dc8a84fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/1563398436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4011db0 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/1563398436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd7d4028470 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d0013d80 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/1563398436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7d4028670 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7e9bd1b80 10 client.19389 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560986e1860 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.132+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/1563398436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d0013d80 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/1563398436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fd7d4031050 con 0x5560987d9bf0 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7e9bd1b80 10 client.19389 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7e9bd1b80 10 client.19389 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:08:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7e9bd1b80 20 client.19389 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x5560987f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7e9bd1b80 4 client.19389 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556098739dc0 con 0x5560987f3720 2024-05-20T21:08:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.133+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x5560987f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.134+0000 7fd7c0ff9700 10 client.19389 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.134+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x5560987f5b10 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac80848720fa2228 server_cookie=cff372a5b03915 in_seq=0 out_seq=0 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.134+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/1563398436 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556098739dc0 con 0x5560987f3720 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.134+0000 7fd7c0ff9700 10 client.19389 handle_command_reply: tid=0 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.134+0000 7fd7e9bd1b80 1 client.19389 shutdown 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 msgr2=0x5560987e6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 0x5560987e6230 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4003e90 tx=0x7fd7d40047c0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 shutdown_connections 2024-05-20T21:08:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x5560987f5b10 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987d9bf0 0x5560987e6230 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987de720 0x5560987e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7c0ff9700 0 client.19389 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/1563398436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 >> 172.21.15.26:0/1563398436 conn(0x556098705340 msgr2=0x5560987d8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 shutdown_connections 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/1563398436 wait complete. 2024-05-20T21:08:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.135+0000 7fd7e9bd1b80 20 client.19389 trim_cache size 0 max 16384 2024-05-20T21:08:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.155+0000 7fd7e9bd1b80 1 Processor -- start 2024-05-20T21:08:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 -- start start 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987da5b0 0x5560987cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b2710 con 0x5560987de720 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81affa0 con 0x5560987d9bf0 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81addf0 con 0x5560987da5b0 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987dfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56866/0 (socket says 172.21.15.26:56866) 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/3476585334 learned_addr learned my addr 172.21.15.26:0/3476585334 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987da5b0 0x5560987cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/3476585334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987da5b0 msgr2=0x5560987cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987da5b0 0x5560987cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/3476585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 msgr2=0x5560987dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/3476585334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560987e2490 con 0x5560987de720 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987dfb90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0007ac0 tx=0x7fd7d0006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e649882c320864a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.156+0000 7fd7c2ffd700 1 -- 172.21.15.26:0/3476585334 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0013550 con 0x5560987de720 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7c2ffd700 1 -- 172.21.15.26:0/3476585334 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d0013b80 con 0x5560987de720 2024-05-20T21:08:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7c2ffd700 1 -- 172.21.15.26:0/3476585334 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0015b30 con 0x5560987de720 2024-05-20T21:08:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/3476585334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 msgr2=0x5560987dfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987dfb90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0007ac0 tx=0x7fd7d0006040 comp rx=0 tx=0).stop 2024-05-20T21:08:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/3476585334 shutdown_connections 2024-05-20T21:08:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987dfb90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987d9bf0 0x5560987dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/3476585334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987da5b0 0x5560987cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/3476585334 >> 172.21.15.26:0/3476585334 conn(0x556098705340 msgr2=0x5560987df480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/3476585334 shutdown_connections 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.157+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/3476585334 wait complete. 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 -- start start 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 Processor -- start 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987d9bf0 0x5560987dc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987ec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81ae460 con 0x5560987de720 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81af8d0 con 0x5560987da5b0 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81aeb30 con 0x5560987d9bf0 2024-05-20T21:08:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7e9bd1b80 10 client.0 _ll_register_callbacks cb 0x5560986c2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7dd8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7dd8d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987ec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56874/0 (socket says 172.21.15.26:56874) 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.158+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/2509938603 learned_addr learned my addr 172.21.15.26:0/2509938603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987d9bf0 0x5560987dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/2509938603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987d9bf0 msgr2=0x5560987dc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987d9bf0 0x5560987dc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 msgr2=0x5560987ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd8d8700 1 -- 172.21.15.26:0/2509938603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560987eca80 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987d9bf0 0x5560987dc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dc8d6700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7dd8d8700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987ec410 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009d70 tx=0x7fd7d0009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4179502c02a800 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/2509938603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0013750 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/2509938603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd7d00138f0 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d0013b80 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/2509938603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7d003b050 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7e9bd1b80 10 client.19395 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.159+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55609885f310 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/2509938603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d0013b80 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/2509938603 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fd7d00053b0 con 0x5560987de720 2024-05-20T21:08:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7e9bd1b80 10 client.19395 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7e9bd1b80 10 client.19395 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7e9bd1b80 20 client.19395 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x556098861770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7e9bd1b80 4 client.19395 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ec03d4bd", "release": true, "await": true}] 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ec03d4bd", "release": true, "await": true}) -- 0x5560987335a0 con 0x5560987f3720 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.160+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x556098861770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.161+0000 7fd7dd0d7700 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x556098861770 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659822c2b7c5442d server_cookie=4fa8c17561c864c6 in_seq=0 out_seq=0 2024-05-20T21:08:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.161+0000 7fd7c0ff9700 10 client.19395 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7c0ff9700 1 -- 172.21.15.26:0/2509938603 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5560987335a0 con 0x5560987f3720 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7c0ff9700 10 client.19395 handle_command_reply: tid=0 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 client.19395 shutdown 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 msgr2=0x5560987ec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987ec410 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009d70 tx=0x7fd7d0009720 comp rx=0 tx=0).stop 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 shutdown_connections 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5560987f3720 0x556098861770 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560987de720 0x5560987ec410 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560987d9bf0 0x5560987dc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7c0ff9700 0 client.19395 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 --2- 172.21.15.26:0/2509938603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560987da5b0 0x5560987ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 >> 172.21.15.26:0/2509938603 conn(0x556098705340 msgr2=0x5560987e55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.172+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 shutdown_connections 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7e9bd1b80 1 -- 172.21.15.26:0/2509938603 wait complete. 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7e9bd1b80 20 client.19395 trim_cache size 0 max 16384 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 48, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "ec03d4bd": { 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 48, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:08:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:08:53.176 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd7c40b2880 msgr2=0x7fd7c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd7c40b2880 0x7fd7c40b4d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40045b0 tx=0x7fd7d400b3c0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 msgr2=0x7fd7d8102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 0x7fd7d8102a80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8006010 tx=0x7fd7c800bfc0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 shutdown_connections 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d81095b0 0x7fd7d8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7d806bc70 0x7fd7d8102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd7c40b2880 0x7fd7c40b4d40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 --2- 172.21.15.26:0/1883668578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d806c630 0x7fd7d8102a80 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 >> 172.21.15.26:0/1883668578 conn(0x7fd7d80793f0 msgr2=0x7fd7d81013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 shutdown_connections 2024-05-20T21:08:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:53.173+0000 7fd7df33b700 1 -- 172.21.15.26:0/1883668578 wait complete. 2024-05-20T21:08:53.571 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:08:53.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:08:53.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288785 2.23 MB/sec execute 539 sec latency 27805.001 ms 2024-05-20T21:08:53.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:53 smithi055 ceph-mon[35213]: pgmap v772: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T21:08:53.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:53 smithi026 ceph-mon[31990]: pgmap v772: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T21:08:53.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:53 smithi144 ceph-mon[35660]: pgmap v772: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-20T21:08:53.969 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ec03d4bd', total seconds elapsed: 38.2 2024-05-20T21:08:53.970 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-20T21:08:53.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:08:54.037 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:54.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 289414 2.23 MB/sec execute 540 sec latency 18.520 ms 2024-05-20T21:08:54.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:54 smithi055 ceph-mon[35213]: pgmap v773: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-20T21:08:54.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:54 smithi026 ceph-mon[31990]: pgmap v773: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-20T21:08:54.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:54 smithi144 ceph-mon[35660]: pgmap v773: 97 pgs: 97 active+clean; 25 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-20T21:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 -- 172.21.15.26:0/3963827081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c0105ec0 msgr2=0x7f30c01082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 --2- 172.21.15.26:0/3963827081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c0105ec0 0x7f30c01082b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009900 tx=0x7f30a802a410 comp rx=0 tx=0).stop 2024-05-20T21:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 -- 172.21.15.26:0/3963827081 shutdown_connections 2024-05-20T21:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 --2- 172.21.15.26:0/3963827081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c01087f0 0x7f30c010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 --2- 172.21.15.26:0/3963827081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c0105ec0 0x7f30c01082b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 --2- 172.21.15.26:0/3963827081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30c0103590 0x7f30c0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 -- 172.21.15.26:0/3963827081 >> 172.21.15.26:0/3963827081 conn(0x7f30c00fcf20 msgr2=0x7f30c00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 -- 172.21.15.26:0/3963827081 shutdown_connections 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.054+0000 7f30c5f38640 1 -- 172.21.15.26:0/3963827081 wait complete. 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 Processor -- start 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 -- start start 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c0103590 0x7f30c0198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30c0105ec0 0x7f30c01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 0x7f30c01a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30c010e4d0 con 0x7f30c01087f0 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30c010e350 con 0x7f30c0103590 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30c5f38640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30c010e650 con 0x7f30c0105ec0 2024-05-20T21:08:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 0x7f30c01a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c0103590 0x7f30c0198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 0x7f30c01a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56902/0 (socket says 172.21.15.26:56902) 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bffff640 1 -- 172.21.15.26:0/709363754 learned_addr learned my addr 172.21.15.26:0/709363754 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30b7fff640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30c0105ec0 0x7f30c01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bf7fe640 1 -- 172.21.15.26:0/709363754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30c0105ec0 msgr2=0x7f30c01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30c0105ec0 0x7f30c01990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bf7fe640 1 -- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 msgr2=0x7f30c01a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 0x7f30c01a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bf7fe640 1 -- 172.21.15.26:0/709363754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c01a07c0 con 0x7f30c0103590 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.055+0000 7f30bffff640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 0x7f30c01a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.056+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c0103590 0x7f30c0198ba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f30b000d720 tx=0x7f30b000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.056+0000 7f30bd7fa640 1 -- 172.21.15.26:0/709363754 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0017070 con 0x7f30c0103590 2024-05-20T21:08:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.056+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c01a0ab0 con 0x7f30c0103590 2024-05-20T21:08:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.056+0000 7f30bd7fa640 1 -- 172.21.15.26:0/709363754 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b000de30 con 0x7f30c0103590 2024-05-20T21:08:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.056+0000 7f30bd7fa640 1 -- 172.21.15.26:0/709363754 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00047a0 con 0x7f30c0103590 2024-05-20T21:08:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.056+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c01a0f80 con 0x7f30c0103590 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.058+0000 7f30bd7fa640 1 -- 172.21.15.26:0/709363754 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f30b00099d0 con 0x7f30c0103590 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.058+0000 7f30c8c52740 1 Processor -- start 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 -- start start 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde51fad0 0x55dcde51fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 0x55dcde524fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde525630 0x55dcde527ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcde48cc30 con 0x55dcde5204c0 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dcde47c7d0 con 0x55dcde525630 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30c8c52740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dcde480810 con 0x55dcde51fad0 2024-05-20T21:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 0x55dcde524fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 0x55dcde524fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56912/0 (socket says 172.21.15.26:56912) 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 -- 172.21.15.26:0/895900947 learned_addr learned my addr 172.21.15.26:0/895900947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30b7fff640 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde525630 0x55dcde527ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bffff640 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde51fad0 0x55dcde51fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 -- 172.21.15.26:0/895900947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde51fad0 msgr2=0x55dcde51fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde51fad0 0x55dcde51fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 -- 172.21.15.26:0/895900947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde525630 msgr2=0x55dcde527ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde525630 0x55dcde527ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bf7fe640 1 -- 172.21.15.26:0/895900947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcde191210 con 0x55dcde5204c0 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30b7fff640 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde525630 0x55dcde527ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.059+0000 7f30bd7fa640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30ac0b28a0 0x7f30ac0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.060+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 0x55dcde524fc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f30b00e5f50 tx=0x7f30b00ea570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d529584bc8024c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.060+0000 7f30bffff640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30ac0b28a0 0x7f30ac0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.060+0000 7f30bd7fa640 1 -- 172.21.15.26:0/709363754 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f30b00e1e40 con 0x7f30c0103590 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.060+0000 7f30b5ffb640 1 -- 172.21.15.26:0/895900947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00a52c0 con 0x55dcde5204c0 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.060+0000 7f30b5ffb640 1 -- 172.21.15.26:0/895900947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b00a8f10 con 0x55dcde5204c0 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.060+0000 7f30b5ffb640 1 -- 172.21.15.26:0/895900947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00a8b70 con 0x55dcde5204c0 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 -- 172.21.15.26:0/895900947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 msgr2=0x55dcde524fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30bffff640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30ac0b28a0 0x7f30ac0b4d60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55dcde520d20 tx=0x7f30b8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 0x55dcde524fc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f30b00e5f50 tx=0x7f30b00ea570 comp rx=0 tx=0).stop 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 -- 172.21.15.26:0/895900947 shutdown_connections 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde525630 0x55dcde527ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5204c0 0x55dcde524fc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 --2- 172.21.15.26:0/895900947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde51fad0 0x55dcde51fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 -- 172.21.15.26:0/895900947 >> 172.21.15.26:0/895900947 conn(0x55dcde516d30 msgr2=0x55dcde517130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 -- 172.21.15.26:0/895900947 shutdown_connections 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 -- 172.21.15.26:0/895900947 wait complete. 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 -- start start 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.061+0000 7f30c8c52740 1 Processor -- start 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde529bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde531150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcde162790 con 0x55dcde525630 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dcde438890 con 0x55dcde51fad0 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dcde5285e0 con 0x55dcde5204c0 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 10 client.0 _ll_register_callbacks cb 0x55dcde190a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde529bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde529bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:38310/0 (socket says 172.21.15.26:38310) 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2580329901 learned_addr learned my addr 172.21.15.26:0/2580329901 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bffff640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde531150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30b7fff640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2580329901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 msgr2=0x55dcde52a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2580329901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 msgr2=0x55dcde531150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde531150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2580329901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcde25d620 con 0x55dcde51fad0 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde529bd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30b00df930 tx=0x7f30b00e5710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8ef45c145f7c8d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f309effd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:08:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f309effd640 1 -- 172.21.15.26:0/2580329901 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00a8d50 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30b00a8f10 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.062+0000 7f309effd640 1 -- 172.21.15.26:0/2580329901 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f30b00e5ad0 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f309effd640 1 -- 172.21.15.26:0/2580329901 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30b00a91d0 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 10 client.28649 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dcde534130 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f309effd640 1 -- 172.21.15.26:0/2580329901 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b00a8f10 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f309effd640 1 -- 172.21.15.26:0/2580329901 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f30b00e3030 con 0x55dcde51fad0 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 10 client.28649 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 10 client.28649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 20 client.28649 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde535480 0x55dcde537870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 4 client.28649 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcde48eb80 con 0x55dcde535480 2024-05-20T21:08:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.063+0000 7f30b7fff640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde535480 0x55dcde537870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.064+0000 7f30b7fff640 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde535480 0x55dcde537870 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69082368867a69b0 server_cookie=a9c8645d9f4b96df in_seq=0 out_seq=0 2024-05-20T21:08:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.064+0000 7f309effd640 10 client.28649 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f309effd640 1 -- 172.21.15.26:0/2580329901 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dcde48eb80 con 0x55dcde535480 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f309effd640 10 client.28649 handle_command_reply: tid=0 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 client.28649 shutdown 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 msgr2=0x55dcde529bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde529bd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30b00df930 tx=0x7f30b00e5710 comp rx=0 tx=0).stop 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 shutdown_connections 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde535480 0x55dcde537870 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde531150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2580329901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde529bd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 >> 172.21.15.26:0/2580329901 conn(0x55dcde516d30 msgr2=0x55dcde515fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f309effd640 0 client.28649 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 shutdown_connections 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 1 -- 172.21.15.26:0/2580329901 wait complete. 2024-05-20T21:08:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.065+0000 7f30c8c52740 20 client.28649 trim_cache size 0 max 16384 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.082+0000 7f30c8c52740 1 Processor -- start 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.082+0000 7f30c8c52740 1 -- start start 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.082+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 0x55dcde515720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.082+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde5204c0 0x55dcde531110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.082+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde525630 0x55dcde515c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30c8c52740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01aa630 con 0x55dcde51fad0 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30c8c52740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01a7f00 con 0x55dcde5204c0 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30c8c52740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01a5d50 con 0x55dcde525630 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 0x55dcde515720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 0x55dcde515720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56938/0 (socket says 172.21.15.26:56938) 2024-05-20T21:08:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2035314619 learned_addr learned my addr 172.21.15.26:0/2035314619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30b7fff640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde5204c0 0x55dcde531110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bffff640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde525630 0x55dcde515c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2035314619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde525630 msgr2=0x55dcde515c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde525630 0x55dcde515c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2035314619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde5204c0 msgr2=0x55dcde531110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde5204c0 0x55dcde531110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 -- 172.21.15.26:0/2035314619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcde524a00 con 0x55dcde51fad0 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bffff640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde525630 0x55dcde515c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30b7fff640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde5204c0 0x55dcde531110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 0x55dcde515720 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f30b00dfc50 tx=0x7f30b00eafb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d43874523d5f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30b57fa640 1 -- 172.21.15.26:0/2035314619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00a8d50 con 0x55dcde51fad0 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30b57fa640 1 -- 172.21.15.26:0/2035314619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b00a8f10 con 0x55dcde51fad0 2024-05-20T21:08:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.083+0000 7f30b57fa640 1 -- 172.21.15.26:0/2035314619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00a8b70 con 0x55dcde51fad0 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 -- 172.21.15.26:0/2035314619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 msgr2=0x55dcde515720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 0x55dcde515720 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f30b00dfc50 tx=0x7f30b00eafb0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 -- 172.21.15.26:0/2035314619 shutdown_connections 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde525630 0x55dcde515c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde5204c0 0x55dcde531110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 --2- 172.21.15.26:0/2035314619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde51fad0 0x55dcde515720 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 -- 172.21.15.26:0/2035314619 >> 172.21.15.26:0/2035314619 conn(0x55dcde516d30 msgr2=0x55dcde51a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 -- 172.21.15.26:0/2035314619 shutdown_connections 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 -- 172.21.15.26:0/2035314619 wait complete. 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.084+0000 7f30c8c52740 1 -- start start 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 Processor -- start 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde52e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde59de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01a63c0 con 0x55dcde525630 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01a7830 con 0x55dcde51fad0 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01a6a90 con 0x55dcde5204c0 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30c8c52740 10 client.0 _ll_register_callbacks cb 0x7f30a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde59de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde59de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56954/0 (socket says 172.21.15.26:56954) 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 -- 172.21.15.26:0/4132025617 learned_addr learned my addr 172.21.15.26:0/4132025617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde52e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30b7fff640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 -- 172.21.15.26:0/4132025617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 msgr2=0x55dcde52e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 -- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 msgr2=0x55dcde52e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde52e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bffff640 1 -- 172.21.15.26:0/4132025617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcde59e360 con 0x55dcde525630 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde52e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.085+0000 7f30b7fff640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30bffff640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde59de20 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f30b800fef0 tx=0x7f30b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f3ca118314dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f309effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f309effd640 1 -- 172.21.15.26:0/4132025617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8004b20 con 0x55dcde525630 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f309effd640 1 -- 172.21.15.26:0/4132025617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f30b8004cc0 con 0x55dcde525630 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30b00a8f10 con 0x55dcde525630 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f309effd640 1 -- 172.21.15.26:0/4132025617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30b8005020 con 0x55dcde525630 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 10 client.19413 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dcde59f5c0 con 0x55dcde525630 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f309effd640 1 -- 172.21.15.26:0/4132025617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b00a8f10 con 0x55dcde525630 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f309effd640 1 -- 172.21.15.26:0/4132025617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f30b8027070 con 0x55dcde525630 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 10 client.19413 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 10 client.19413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 20 client.19413 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde59ff90 0x55dcde5a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 4 client.19413 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.086+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcde488360 con 0x55dcde59ff90 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.087+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde59ff90 0x55dcde5a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.087+0000 7f309effd640 10 client.19413 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.087+0000 7f30bf7fe640 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde59ff90 0x55dcde5a23b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3bb30f21e30d8b server_cookie=604b05b1399ae32e in_seq=0 out_seq=0 2024-05-20T21:08:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f309effd640 1 -- 172.21.15.26:0/4132025617 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dcde488360 con 0x55dcde59ff90 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f309effd640 10 client.19413 handle_command_reply: tid=0 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 client.19413 shutdown 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 msgr2=0x55dcde59de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde59de20 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f30b800fef0 tx=0x7f30b8004ab0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 shutdown_connections 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55dcde59ff90 0x55dcde5a23b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde525630 0x55dcde59de20 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcde5204c0 0x55dcde52e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f30c8c52740 1 --2- 172.21.15.26:0/4132025617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcde51fad0 0x55dcde52e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.112+0000 7f309effd640 0 client.19413 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 >> 172.21.15.26:0/4132025617 conn(0x55dcde516d30 msgr2=0x55dcde5291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 shutdown_connections 2024-05-20T21:08:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c8c52740 1 -- 172.21.15.26:0/4132025617 wait complete. 2024-05-20T21:08:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c8c52740 20 client.19413 trim_cache size 0 max 16384 2024-05-20T21:08:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:08:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:08:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30ac0b28a0 msgr2=0x7f30ac0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30ac0b28a0 0x7f30ac0b4d60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55dcde520d20 tx=0x7f30b8009500 comp rx=0 tx=0).stop 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c0103590 msgr2=0x7f30c0198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c0103590 0x7f30c0198ba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f30b000d720 tx=0x7f30b000dbe0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 shutdown_connections 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f30ac0b28a0 0x7f30ac0b4d60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c01087f0 0x7f30c01a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30c0105ec0 0x7f30c01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 --2- 172.21.15.26:0/709363754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c0103590 0x7f30c0198ba0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 >> 172.21.15.26:0/709363754 conn(0x7f30c00fcf20 msgr2=0x7f30c0104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 shutdown_connections 2024-05-20T21:08:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:08:55.113+0000 7f30c5f38640 1 -- 172.21.15.26:0/709363754 wait complete. 2024-05-20T21:08:55.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 290101 2.23 MB/sec execute 541 sec latency 16.925 ms 2024-05-20T21:08:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 -- 172.21.15.26:0/3670285163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 msgr2=0x7fcfcc109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/3670285163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 0x7fcfcc109990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00098f0 tx=0x7fcfc0009c00 comp rx=0 tx=0).stop 2024-05-20T21:08:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 -- 172.21.15.26:0/3670285163 shutdown_connections 2024-05-20T21:08:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/3670285163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 0x7fcfcc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/3670285163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 0x7fcfcc109990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/3670285163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 -- 172.21.15.26:0/3670285163 >> 172.21.15.26:0/3670285163 conn(0x7fcfcc0793f0 msgr2=0x7fcfcc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 -- 172.21.15.26:0/3670285163 shutdown_connections 2024-05-20T21:08:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.953+0000 7fcfd2f54700 1 -- 172.21.15.26:0/3670285163 wait complete. 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 Processor -- start 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 -- start start 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 0x7fcfcc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 0x7fcfcc1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc113590 con 0x7fcfcc06b220 2024-05-20T21:08:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc113410 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd2f54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc113710 con 0x7fcfcc1095b0 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56972/0 (socket says 172.21.15.26:56972) 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2031398407 learned_addr learned my addr 172.21.15.26:0/2031398407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.954+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 0x7fcfcc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfd14f1700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 0x7fcfcc1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfcbfff700 1 -- 172.21.15.26:0/2031398407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 msgr2=0x7fcfcc1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 0x7fcfcc1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfcbfff700 1 -- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 msgr2=0x7fcfcc19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfcbfff700 1 -- 172.21.15.26:0/2031398407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc19a110 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 0x7fcfcc19fce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800eb90 tx=0x7fcfb800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfc9ffb700 1 -- 172.21.15.26:0/2031398407 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800cc90 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfc9ffb700 1 -- 172.21.15.26:0/2031398407 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfb800ce30 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc19a400 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.955+0000 7fcfc9ffb700 1 -- 172.21.15.26:0/2031398407 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8003e30 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.956+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc1a6670 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.958+0000 7fcfc9ffb700 1 -- 172.21.15.26:0/2031398407 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcfb8004000 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.959+0000 7fcfc9ffb700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfbc0b2880 0x7fcfbc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.959+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfbc0b2880 0x7fcfbc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfc9ffb700 1 -- 172.21.15.26:0/2031398407 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcfb80e25a0 con 0x7fcfcc06bbe0 2024-05-20T21:08:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfbc0b2880 0x7fcfbc0b4d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009680 tx=0x7fcfc0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfdd7eab80 1 Processor -- start 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfdd7eab80 1 -- start start 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.960+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653f6346d10 con 0x5653f63e8290 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5653f63368b0 con 0x5653f63eb110 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653f633a8f0 con 0x5653f63e78e0 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfcbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:38352/0 (socket says 172.21.15.26:38352) 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 -- 172.21.15.26:0/1455363317 learned_addr learned my addr 172.21.15.26:0/1455363317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 -- 172.21.15.26:0/1455363317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 msgr2=0x5653f63e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 -- 172.21.15.26:0/1455363317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 msgr2=0x5653f63eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63eaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 -- 172.21.15.26:0/1455363317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f6286950 con 0x5653f63eb110 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.961+0000 7fcfd14f1700 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63eb590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011680 tx=0x7fcfc4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b6007106d6b122 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.962+0000 7fcfb6ffd700 1 -- 172.21.15.26:0/1455363317 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4019030 con 0x5653f63eb110 2024-05-20T21:08:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.962+0000 7fcfb6ffd700 1 -- 172.21.15.26:0/1455363317 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc40092e0 con 0x5653f63eb110 2024-05-20T21:08:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.962+0000 7fcfb6ffd700 1 -- 172.21.15.26:0/1455363317 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4007500 con 0x5653f63eb110 2024-05-20T21:08:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.962+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1455363317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 msgr2=0x5653f63eb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.962+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63eb590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011680 tx=0x7fcfc4009920 comp rx=0 tx=0).stop 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1455363317 shutdown_connections 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63eb590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1455363317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1455363317 >> 172.21.15.26:0/1455363317 conn(0x5653f6315100 msgr2=0x5653f63e1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1455363317 shutdown_connections 2024-05-20T21:08:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1455363317 wait complete. 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 -- start start 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.963+0000 7fcfdd7eab80 1 Processor -- start 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653f623d020 con 0x5653f63e8290 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfcbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5653f62d8840 con 0x5653f63eb110 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653f63fca40 con 0x5653f63e78e0 2024-05-20T21:08:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:33056/0 (socket says 172.21.15.26:33056) 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/1114512548 learned_addr learned my addr 172.21.15.26:0/1114512548 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfd14f1700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfdd7eab80 10 client.0 _ll_register_callbacks cb 0x5653f62d19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfcbfff700 1 -- 172.21.15.26:0/1114512548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 msgr2=0x5653f63fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfcbfff700 1 -- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 msgr2=0x5653f63f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfcbfff700 1 -- 172.21.15.26:0/1114512548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f62d5710 con 0x5653f63e8290 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.964+0000 7fcfd14f1700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63f9040 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80e0090 tx=0x7fcfb80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c48ed1dfe9e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfabfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfabfff700 1 -- 172.21.15.26:0/1114512548 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a9670 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc40092e0 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfabfff700 1 -- 172.21.15.26:0/1114512548 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcfb80b2530 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfabfff700 1 -- 172.21.15.26:0/1114512548 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfb80aa260 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfdd7eab80 10 client.19425 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653f64072b0 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.965+0000 7fcfabfff700 1 -- 172.21.15.26:0/1114512548 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc40092e0 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfabfff700 1 -- 172.21.15.26:0/1114512548 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcfb80a8dc0 con 0x5653f63e8290 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfdd7eab80 10 client.19425 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfdd7eab80 10 client.19425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:55.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfdd7eab80 20 client.19425 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f64098f0 0x5653f640bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfdd7eab80 4 client.19425 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:08:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f63b3a60 con 0x5653f64098f0 2024-05-20T21:08:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f64098f0 0x5653f640bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfabfff700 10 client.19425 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.966+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f64098f0 0x5653f640bce0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee55466b07b2e289 server_cookie=f37f60b1915093b6 in_seq=0 out_seq=0 2024-05-20T21:08:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfabfff700 1 -- 172.21.15.26:0/1114512548 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5653f63b3a60 con 0x5653f64098f0 2024-05-20T21:08:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfabfff700 10 client.19425 handle_command_reply: tid=0 2024-05-20T21:08:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 client.19425 shutdown 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 msgr2=0x5653f63f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63f9040 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80e0090 tx=0x7fcfb80038b0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 shutdown_connections 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f64098f0 0x5653f640bce0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfabfff700 0 client.19425 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63eb110 0x5653f63f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.967+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/1114512548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63f9040 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.968+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 >> 172.21.15.26:0/1114512548 conn(0x5653f6315100 msgr2=0x5653f63e1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.968+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 shutdown_connections 2024-05-20T21:08:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.968+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/1114512548 wait complete. 2024-05-20T21:08:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.968+0000 7fcfdd7eab80 20 client.19425 trim_cache size 0 max 16384 2024-05-20T21:08:55.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 Processor -- start 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 -- start start 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 0x5653f63f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e78e0 0x5653f63e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1ae430 con 0x5653f63e8290 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1abcc0 con 0x5653f63e78e0 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1a9b10 con 0x5653f6315100 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfcbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e78e0 0x5653f63e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfd14f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 0x5653f63f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 0x5653f63f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:33064/0 (socket says 172.21.15.26:33064) 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfd14f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63e2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57008/0 (socket says 172.21.15.26:57008) 2024-05-20T21:08:55.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.995+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2981384851 learned_addr learned my addr 172.21.15.26:0/2981384851 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2981384851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e78e0 msgr2=0x5653f63e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e78e0 0x5653f63e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2981384851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 msgr2=0x5653f63e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2981384851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f63e30f0 con 0x5653f6315100 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 0x5653f63f7fa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00052a0 tx=0x7fcfc001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f169e966fb232ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfb67fc700 1 -- 172.21.15.26:0/2981384851 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc003c070 con 0x5653f6315100 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfb67fc700 1 -- 172.21.15.26:0/2981384851 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc0003b10 con 0x5653f6315100 2024-05-20T21:08:55.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.996+0000 7fcfb67fc700 1 -- 172.21.15.26:0/2981384851 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0029040 con 0x5653f6315100 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.997+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2981384851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 msgr2=0x5653f63f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.997+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 0x5653f63f7fa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00052a0 tx=0x7fcfc001f000 comp rx=0 tx=0).stop 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.997+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2981384851 shutdown_connections 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.997+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f63e8290 0x5653f63e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.997+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f6315100 0x5653f63f7fa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.998+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2981384851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e78e0 0x5653f63e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.998+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2981384851 >> 172.21.15.26:0/2981384851 conn(0x5653f63deb80 msgr2=0x5653f63e6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:56.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.998+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2981384851 shutdown_connections 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.998+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2981384851 wait complete. 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.998+0000 7fcfdd7eab80 1 -- start start 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.998+0000 7fcfdd7eab80 1 Processor -- start 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 0x5653f63fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e8290 0x5653f6473430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1aa180 con 0x5653f6315100 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1ab5f0 con 0x5653f63e8290 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1aa850 con 0x5653f63e78e0 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfdd7eab80 10 client.0 _ll_register_callbacks cb 0x5653f62d19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 0x5653f63fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 0x5653f63fe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57022/0 (socket says 172.21.15.26:57022) 2024-05-20T21:08:56.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2729495610 learned_addr learned my addr 172.21.15.26:0/2729495610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2729495610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 msgr2=0x5653f63fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e8290 msgr2=0x5653f6473430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e8290 0x5653f6473430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:55.999+0000 7fcfd0cf0700 1 -- 172.21.15.26:0/2729495610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f64739f0 con 0x5653f6315100 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfd0cf0700 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 0x5653f63fe1c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0005920 tx=0x7fcfc00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6674f0f976d713 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfabfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfabfff700 1 -- 172.21.15.26:0/2729495610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc003c030 con 0x5653f6315100 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfabfff700 1 -- 172.21.15.26:0/2729495610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcfc0003680 con 0x5653f6315100 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc0003b10 con 0x5653f6315100 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfabfff700 1 -- 172.21.15.26:0/2729495610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfc0029040 con 0x5653f6315100 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfdd7eab80 10 client.19431 fetch_fsmap learned FSMap version 30 2024-05-20T21:08:56.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.000+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653f6474d10 con 0x5653f6315100 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfabfff700 1 -- 172.21.15.26:0/2729495610 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc0003b10 con 0x5653f6315100 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfabfff700 1 -- 172.21.15.26:0/2729495610 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcfc0048020 con 0x5653f6315100 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfdd7eab80 10 client.19431 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfdd7eab80 10 client.19431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfdd7eab80 20 client.19431 populate_metadata read hostname 'smithi026' 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f63eb110 0x5653f64772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfdd7eab80 4 client.19431 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:08:56.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5653f6348c60 con 0x5653f63eb110 2024-05-20T21:08:56.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.001+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f63eb110 0x5653f64772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:56.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.002+0000 7fcfabfff700 10 client.19431 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.002+0000 7fcfcbfff700 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f63eb110 0x5653f64772f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92017e81782d373e server_cookie=d5932d240ca7e3f8 in_seq=0 out_seq=0 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfabfff700 1 -- 172.21.15.26:0/2729495610 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5653f6348c60 con 0x5653f63eb110 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfabfff700 10 client.19431 handle_command_reply: tid=0 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 client.19431 shutdown 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 msgr2=0x5653f63fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 0x5653f63fe1c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0005920 tx=0x7fcfc00056e0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 shutdown_connections 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5653f63eb110 0x5653f64772f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653f6315100 0x5653f63fe1c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5653f63e8290 0x5653f6473430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 --2- 172.21.15.26:0/2729495610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653f63e78e0 0x5653f63fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfabfff700 0 client.19431 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:08:56.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 >> 172.21.15.26:0/2729495610 conn(0x5653f63deb80 msgr2=0x5653f63fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 shutdown_connections 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.003+0000 7fcfdd7eab80 1 -- 172.21.15.26:0/2729495610 wait complete. 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfdd7eab80 20 client.19431 trim_cache size 0 max 16384 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "2d6d174b-9d5f-423d-a287-c735f2d009c3", 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:08:56.007 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfbc0b2880 msgr2=0x7fcfbc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfbc0b2880 0x7fcfbc0b4d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009680 tx=0x7fcfc0019040 comp rx=0 tx=0).stop 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 msgr2=0x7fcfcc19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 0x7fcfcc19fce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800eb90 tx=0x7fcfb800eea0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 shutdown_connections 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcfbc0b2880 0x7fcfbc0b4d40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfcc06b220 0x7fcfcc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfcc1095b0 0x7fcfcc1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 --2- 172.21.15.26:0/2031398407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfcc06bbe0 0x7fcfcc19fce0 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800eb90 tx=0x7fcfb800eea0 comp rx=0 tx=0).stop 2024-05-20T21:08:56.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.004+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 >> 172.21.15.26:0/2031398407 conn(0x7fcfcc0793f0 msgr2=0x7fcfcc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:56.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.005+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 shutdown_connections 2024-05-20T21:08:56.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:56.005+0000 7fcfd2f54700 1 -- 172.21.15.26:0/2031398407 wait complete. 2024-05-20T21:08:56.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 290703 2.24 MB/sec execute 542 sec latency 19.469 ms 2024-05-20T21:08:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:57 smithi026 ceph-mon[31990]: pgmap v774: 97 pgs: 97 active+clean; 26 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 107 KiB/s wr, 9 op/s 2024-05-20T21:08:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:57 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:08:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:57 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:57 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:08:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:57 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:57.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:57 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:08:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:57 smithi144 ceph-mon[35660]: pgmap v774: 97 pgs: 97 active+clean; 26 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 107 KiB/s wr, 9 op/s 2024-05-20T21:08:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:57 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:08:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:57 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:57 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:08:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:57 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:57.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:57 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:08:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:57 smithi055 ceph-mon[35213]: pgmap v774: 97 pgs: 97 active+clean; 26 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 107 KiB/s wr, 9 op/s 2024-05-20T21:08:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:57 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:08:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:57 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:57 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:08:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:57 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:08:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:57 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:08:57.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 291298 2.24 MB/sec execute 543 sec latency 16.214 ms 2024-05-20T21:08:57.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:08:58.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 -- 172.21.15.26:0/430919825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec075770 msgr2=0x7fcdec075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 --2- 172.21.15.26:0/430919825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec075770 0x7fcdec075b50 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009990 tx=0x7fcddc009ca0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 -- 172.21.15.26:0/430919825 shutdown_connections 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 --2- 172.21.15.26:0/430919825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec075770 0x7fcdec075b50 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 --2- 172.21.15.26:0/430919825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdec10ff40 0x7fcdec112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 --2- 172.21.15.26:0/430919825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec076090 0x7fcdec10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 -- 172.21.15.26:0/430919825 >> 172.21.15.26:0/430919825 conn(0x7fcdec07a6f0 msgr2=0x7fcdec07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 -- 172.21.15.26:0/430919825 shutdown_connections 2024-05-20T21:08:58.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.555+0000 7fcdf1871700 1 -- 172.21.15.26:0/430919825 wait complete. 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.556+0000 7fcdf1871700 1 Processor -- start 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.556+0000 7fcdf1871700 1 -- start start 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdf1871700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 0x7fcdec19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdf1871700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec076090 0x7fcdec19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdf1871700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdec10ff40 0x7fcdec1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdf1871700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1157f0 con 0x7fcdec076090 2024-05-20T21:08:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdf1871700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec115670 con 0x7fcdec075770 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdf1871700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec115970 con 0x7fcdec10ff40 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 0x7fcdec19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdec10ff40 0x7fcdec1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 0x7fcdec19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:38386/0 (socket says 172.21.15.26:38386) 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 -- 172.21.15.26:0/878115596 learned_addr learned my addr 172.21.15.26:0/878115596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 -- 172.21.15.26:0/878115596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdec10ff40 msgr2=0x7fcdec1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdec10ff40 0x7fcdec1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 -- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec076090 msgr2=0x7fcdec19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec076090 0x7fcdec19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.557+0000 7fcdeaffd700 1 -- 172.21.15.26:0/878115596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec1a0df0 con 0x7fcdec075770 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.558+0000 7fcdeaffd700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 0x7fcdec19bde0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.558+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc027070 con 0x7fcdec075770 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.558+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec1a1080 con 0x7fcdec075770 2024-05-20T21:08:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.558+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcddc00fb90 con 0x7fcdec075770 2024-05-20T21:08:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.558+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0173f0 con 0x7fcdec075770 2024-05-20T21:08:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.558+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a8960 con 0x7fcdec075770 2024-05-20T21:08:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.559+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec076e90 con 0x7fcdec075770 2024-05-20T21:08:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.561+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcddc02a070 con 0x7fcdec075770 2024-05-20T21:08:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.562+0000 7fcdf086f700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdd40b2880 0x7fcdd40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:08:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.562+0000 7fcdea7fc700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdd40b2880 0x7fcdd40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:08:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.562+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcddc0e3290 con 0x7fcdec075770 2024-05-20T21:08:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.563+0000 7fcdea7fc700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdd40b2880 0x7fcdd40b4d40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcde0005950 tx=0x7fcde00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:08:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.565+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcddc013070 con 0x7fcdec075770 2024-05-20T21:08:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:08:58.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 291814 2.24 MB/sec execute 544 sec latency 26.381 ms 2024-05-20T21:08:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.880+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdec19d140 con 0x7fcdec075770 2024-05-20T21:08:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.881+0000 7fcdf086f700 1 -- 172.21.15.26:0/878115596 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7fcddc0aaf80 con 0x7fcdec075770 2024-05-20T21:08:58.881 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:08:58.881 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:08:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdd40b2880 msgr2=0x7fcdd40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdd40b2880 0x7fcdd40b4d40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcde0005950 tx=0x7fcde00058e0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 msgr2=0x7fcdec19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 0x7fcdec19bde0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc0057c0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 shutdown_connections 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdec10ff40 0x7fcdec1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdec075770 0x7fcdec19bde0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcdd40b2880 0x7fcdd40b4d40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 --2- 172.21.15.26:0/878115596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdec076090 0x7fcdec19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.884+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 >> 172.21.15.26:0/878115596 conn(0x7fcdec07a6f0 msgr2=0x7fcdec101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.885+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 shutdown_connections 2024-05-20T21:08:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:08:58.885+0000 7fcdf1871700 1 -- 172.21.15.26:0/878115596 wait complete. 2024-05-20T21:08:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:08:59.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:59 smithi026 ceph-mon[31990]: pgmap v775: 97 pgs: 97 active+clean; 27 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 269 KiB/s wr, 37 op/s 2024-05-20T21:08:59.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:08:59 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/878115596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:59 smithi144 ceph-mon[35660]: pgmap v775: 97 pgs: 97 active+clean; 27 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 269 KiB/s wr, 37 op/s 2024-05-20T21:08:59.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:08:59 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/878115596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:59 smithi055 ceph-mon[35213]: pgmap v775: 97 pgs: 97 active+clean; 27 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 269 KiB/s wr, 37 op/s 2024-05-20T21:08:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:08:59 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/878115596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:08:59.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 292461 2.24 MB/sec execute 545 sec latency 20.198 ms 2024-05-20T21:08:59.656 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-20T21:08:59.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-20T21:09:00.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 293183 2.24 MB/sec execute 546 sec latency 20.132 ms 2024-05-20T21:09:00.651 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:01.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:01 smithi026 ceph-mon[31990]: pgmap v776: 97 pgs: 97 active+clean; 28 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 480 KiB/s wr, 54 op/s 2024-05-20T21:09:01.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:01 smithi144 ceph-mon[35660]: pgmap v776: 97 pgs: 97 active+clean; 28 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 480 KiB/s wr, 54 op/s 2024-05-20T21:09:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:01 smithi055 ceph-mon[35213]: pgmap v776: 97 pgs: 97 active+clean; 28 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 480 KiB/s wr, 54 op/s 2024-05-20T21:09:01.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 293868 2.25 MB/sec execute 547 sec latency 18.484 ms 2024-05-20T21:09:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1652399862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 msgr2=0x7fcdbc103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1652399862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc103a10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009990 tx=0x7fcdb0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1652399862 shutdown_connections 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1652399862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc103f50 0x7fcdbc110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1652399862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 0x7fcdbc109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1652399862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc103a10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1652399862 >> 172.21.15.26:0/1652399862 conn(0x7fcdbc07a6f0 msgr2=0x7fcdbc07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1652399862 shutdown_connections 2024-05-20T21:09:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.837+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1652399862 wait complete. 2024-05-20T21:09:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 Processor -- start 2024-05-20T21:09:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 -- start start 2024-05-20T21:09:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc103f50 0x7fcdbc1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 0x7fcdbc10f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc113670 con 0x7fcdbc103590 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1134f0 con 0x7fcdbc1096a0 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdc0cf1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1137f0 con 0x7fcdbc103f50 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdbad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 0x7fcdbc10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41346/0 (socket says 172.21.15.26:41346) 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.838+0000 7fcdba59c700 1 -- 172.21.15.26:0/1691054487 learned_addr learned my addr 172.21.15.26:0/1691054487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc103f50 0x7fcdbc1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1691054487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc103f50 msgr2=0x7fcdbc1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc103f50 0x7fcdbc1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 msgr2=0x7fcdbc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1691054487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc10fce0 con 0x7fcdbc1096a0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 0x7fcdbc10f460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400aa90 tx=0x7fcdb400ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdab7fe700 1 -- 172.21.15.26:0/1691054487 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4017020 con 0x7fcdbc1096a0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc10ffd0 con 0x7fcdbc1096a0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdab7fe700 1 -- 172.21.15.26:0/1691054487 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb4004bb0 con 0x7fcdbc1096a0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdab7fe700 1 -- 172.21.15.26:0/1691054487 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005020 con 0x7fcdbc1096a0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.839+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc19c370 con 0x7fcdbc1096a0 2024-05-20T21:09:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.842+0000 7fcdab7fe700 1 -- 172.21.15.26:0/1691054487 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcdb400c580 con 0x7fcdbc1096a0 2024-05-20T21:09:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.843+0000 7fcdab7fe700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcda40b2880 0x7fcda40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.844+0000 7fcdba59c700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcda40b2880 0x7fcda40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.844+0000 7fcdab7fe700 1 -- 172.21.15.26:0/1691054487 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcdb40e16e0 con 0x7fcdbc1096a0 2024-05-20T21:09:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.844+0000 7fcdcb587b80 1 Processor -- start 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.844+0000 7fcdba59c700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcda40b2880 0x7fcda40b4d40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009d20 tx=0x7fcdac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.844+0000 7fcdcb587b80 1 -- start start 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade552860 0x557ade552c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 0x557ade556940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade5573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ade4b1c90 con 0x557ade552860 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557ade4a1830 con 0x557ade556f60 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ade4a5870 con 0x557ade553210 2024-05-20T21:09:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 0x557ade556940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade552860 0x557ade552c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 0x557ade556940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45838/0 (socket says 172.21.15.26:45838) 2024-05-20T21:09:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 -- 172.21.15.26:0/3179794874 learned_addr learned my addr 172.21.15.26:0/3179794874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade5573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 -- 172.21.15.26:0/3179794874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 msgr2=0x557ade5573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade5573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 -- 172.21.15.26:0/3179794874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade552860 msgr2=0x557ade552c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade552860 0x557ade552c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 -- 172.21.15.26:0/3179794874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ade3f18d0 con 0x557ade553210 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade5573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdba59c700 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade552860 0x557ade552c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.845+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 0x557ade556940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009990 tx=0x7fcdb0005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a4bacf993646bb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.846+0000 7fcda8ff9700 1 -- 172.21.15.26:0/3179794874 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0027070 con 0x557ade553210 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.846+0000 7fcda8ff9700 1 -- 172.21.15.26:0/3179794874 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb00056a0 con 0x557ade553210 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.846+0000 7fcda8ff9700 1 -- 172.21.15.26:0/3179794874 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000bba0 con 0x557ade553210 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3179794874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 msgr2=0x557ade556940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 0x557ade556940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009990 tx=0x7fcdb0005c80 comp rx=0 tx=0).stop 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3179794874 shutdown_connections 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade552860 0x557ade552c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade5573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3179794874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade553210 0x557ade556940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3179794874 >> 172.21.15.26:0/3179794874 conn(0x557ade480080 msgr2=0x557ade54ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3179794874 shutdown_connections 2024-05-20T21:09:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3179794874 wait complete. 2024-05-20T21:09:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 -- start start 2024-05-20T21:09:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.847+0000 7fcdcb587b80 1 Processor -- start 2024-05-20T21:09:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade55bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ade3a7fa0 con 0x557ade553210 2024-05-20T21:09:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557ade4437c0 con 0x557ade552860 2024-05-20T21:09:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ade55f0c0 con 0x557ade556f60 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdcb587b80 10 client.0 _ll_register_callbacks cb 0x557ade43c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41552/0 (socket says 172.21.15.26:41552) 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 -- 172.21.15.26:0/3660193845 learned_addr learned my addr 172.21.15.26:0/3660193845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade55bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 -- 172.21.15.26:0/3660193845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 msgr2=0x557ade55bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade55bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 -- 172.21.15.26:0/3660193845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 msgr2=0x557ade55b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 -- 172.21.15.26:0/3660193845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ade440690 con 0x557ade552860 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcdba59c700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55e3d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009ed0 tx=0x7fcdac0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b59ad15cbb64dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.848+0000 7fcda1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcda1ffb700 1 -- 172.21.15.26:0/3660193845 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac02b060 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb00056a0 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcda1ffb700 1 -- 172.21.15.26:0/3660193845 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcdac01cdb0 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcda1ffb700 1 -- 172.21.15.26:0/3660193845 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdac026410 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 10 client.28688 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557ade569870 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcda1ffb700 1 -- 172.21.15.26:0/3660193845 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb00056a0 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcda1ffb700 1 -- 172.21.15.26:0/3660193845 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcdac02f030 con 0x557ade552860 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 10 client.28688 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 10 client.28688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 20 client.28688 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade56e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 4 client.28688 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.849+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ade51e9e0 con 0x557ade56be50 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.850+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade56e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.850+0000 7fcda1ffb700 10 client.28688 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.850+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade56e2a0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad2172da29a5c2f server_cookie=75c8ae6fbd477d70 in_seq=0 out_seq=0 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcda1ffb700 1 -- 172.21.15.26:0/3660193845 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ade51e9e0 con 0x557ade56be50 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcda1ffb700 10 client.28688 handle_command_reply: tid=0 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 client.28688 shutdown 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 msgr2=0x557ade55e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55e3d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009ed0 tx=0x7fcdac0047c0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 shutdown_connections 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade56e2a0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade55bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcda1ffb700 0 client.28688 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55e3d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/3660193845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.851+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 >> 172.21.15.26:0/3660193845 conn(0x557ade480080 msgr2=0x557ade558350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.852+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 shutdown_connections 2024-05-20T21:09:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.852+0000 7fcdcb587b80 1 -- 172.21.15.26:0/3660193845 wait complete. 2024-05-20T21:09:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.852+0000 7fcdcb587b80 20 client.28688 trim_cache size 0 max 16384 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 Processor -- start 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 -- start start 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade573b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1bf3c0 con 0x557ade553210 2024-05-20T21:09:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1bcc50 con 0x557ade552860 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1baaa0 con 0x557ade556f60 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdb9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdb9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41364/0 (socket says 172.21.15.26:41364) 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.877+0000 7fcdb9d9b700 1 -- 172.21.15.26:0/2653954712 learned_addr learned my addr 172.21.15.26:0/2653954712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade573b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdba59c700 1 -- 172.21.15.26:0/2653954712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 msgr2=0x557ade573b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdba59c700 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade573b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdba59c700 1 -- 172.21.15.26:0/2653954712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 msgr2=0x557ade55d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdba59c700 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdba59c700 1 -- 172.21.15.26:0/2653954712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ade54d8d0 con 0x557ade552860 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade573b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcdba59c700 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55a8a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0263e0 tx=0x7fcdac00e5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26e030d7f602254 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcda3fff700 1 -- 172.21.15.26:0/2653954712 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac02b070 con 0x557ade552860 2024-05-20T21:09:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcda3fff700 1 -- 172.21.15.26:0/2653954712 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdac00ed80 con 0x557ade552860 2024-05-20T21:09:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.878+0000 7fcda3fff700 1 -- 172.21.15.26:0/2653954712 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac026b20 con 0x557ade552860 2024-05-20T21:09:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 -- 172.21.15.26:0/2653954712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 msgr2=0x557ade55a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55a8a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0263e0 tx=0x7fcdac00e5c0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 -- 172.21.15.26:0/2653954712 shutdown_connections 2024-05-20T21:09:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade556f60 0x557ade573b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade552860 0x557ade55a8a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/2653954712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade55d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 -- 172.21.15.26:0/2653954712 >> 172.21.15.26:0/2653954712 conn(0x557ade480080 msgr2=0x557ade55cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 -- 172.21.15.26:0/2653954712 shutdown_connections 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 -- 172.21.15.26:0/2653954712 wait complete. 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.879+0000 7fcdcb587b80 1 -- start start 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 Processor -- start 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade552860 0x557ade564380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade5648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade561350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1bb110 con 0x557ade553210 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1bc580 con 0x557ade556f60 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1bb7e0 con 0x557ade552860 2024-05-20T21:09:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdcb587b80 10 client.0 _ll_register_callbacks cb 0x557ade43c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade561350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade561350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41566/0 (socket says 172.21.15.26:41566) 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1494076097 learned_addr learned my addr 172.21.15.26:0/1494076097 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdba59c700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade552860 0x557ade564380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdb9d9b700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade5648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1494076097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade552860 msgr2=0x557ade564380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade552860 0x557ade564380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1494076097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 msgr2=0x557ade5648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade5648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.880+0000 7fcdbad9d700 1 -- 172.21.15.26:0/1494076097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ade561960 con 0x557ade556f60 2024-05-20T21:09:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcdbad9d700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade561350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40df4f0 tx=0x7fcdb40a8590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d28e5965b85a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcda1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcda1ffb700 1 -- 172.21.15.26:0/1494076097 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40a9430 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac00ed80 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcda1ffb700 1 -- 172.21.15.26:0/1494076097 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fcdb40a91a0 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcda1ffb700 1 -- 172.21.15.26:0/1494076097 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdb40a8d30 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcdcb587b80 10 client.28694 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557ade5d7380 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.881+0000 7fcda1ffb700 1 -- 172.21.15.26:0/1494076097 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdac00ed80 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcda1ffb700 1 -- 172.21.15.26:0/1494076097 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fcdb4012070 con 0x557ade556f60 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 10 client.28694 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 10 client.28694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 20 client.28694 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade5d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 4 client.28694 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ade4b3be0 con 0x557ade56be50 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdba59c700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade5d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdba59c700 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade5d9a40 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1791a19ad5cb61a2 server_cookie=a929d3d6cd80ba36 in_seq=0 out_seq=0 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcda1ffb700 10 client.28694 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcda1ffb700 1 -- 172.21.15.26:0/1494076097 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557ade4b3be0 con 0x557ade56be50 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcda1ffb700 10 client.28694 handle_command_reply: tid=0 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 1 client.28694 shutdown 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 msgr2=0x557ade561350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.882+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade561350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40df4f0 tx=0x7fcdb40a8590 comp rx=0 tx=0).stop 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 shutdown_connections 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557ade56be50 0x557ade5d9a40 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557ade556f60 0x557ade561350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557ade552860 0x557ade564380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcda1ffb700 0 client.28694 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 --2- 172.21.15.26:0/1494076097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ade553210 0x557ade5648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 >> 172.21.15.26:0/1494076097 conn(0x557ade480080 msgr2=0x557ade551f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 shutdown_connections 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 1 -- 172.21.15.26:0/1494076097 wait complete. 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdcb587b80 20 client.28694 trim_cache size 0 max 16384 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:09:01.889 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:09:01.890 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.883+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcda40b2880 msgr2=0x7fcda40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcda40b2880 0x7fcda40b4d40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009d20 tx=0x7fcdac009260 comp rx=0 tx=0).stop 2024-05-20T21:09:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 msgr2=0x7fcdbc10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 0x7fcdbc10f460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400aa90 tx=0x7fcdb400ada0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 shutdown_connections 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fcda40b2880 0x7fcda40b4d40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc103590 0x7fcdbc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdbc1096a0 0x7fcdbc10f460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 --2- 172.21.15.26:0/1691054487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc103f50 0x7fcdbc1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 >> 172.21.15.26:0/1691054487 conn(0x7fcdbc07a6f0 msgr2=0x7fcdbc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 shutdown_connections 2024-05-20T21:09:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:01.884+0000 7fcdc0cf1700 1 -- 172.21.15.26:0/1691054487 wait complete. 2024-05-20T21:09:02.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 294498 2.25 MB/sec execute 548 sec latency 15.858 ms 2024-05-20T21:09:03.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:03 smithi144 ceph-mon[35660]: pgmap v777: 97 pgs: 97 active+clean; 28 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 765 KiB/s wr, 67 op/s 2024-05-20T21:09:03.355 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:03 smithi026 ceph-mon[31990]: pgmap v777: 97 pgs: 97 active+clean; 28 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 765 KiB/s wr, 67 op/s 2024-05-20T21:09:03.384 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:09:03.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:09:03.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:03 smithi055 ceph-mon[35213]: pgmap v777: 97 pgs: 97 active+clean; 28 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 765 KiB/s wr, 67 op/s 2024-05-20T21:09:03.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 294921 2.25 MB/sec execute 549 sec latency 17.642 ms 2024-05-20T21:09:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 -- 172.21.15.26:0/2267607216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be8102470 msgr2=0x7f6be81028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/2267607216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be8102470 0x7f6be81028f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009990 tx=0x7f6bd8009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 -- 172.21.15.26:0/2267607216 shutdown_connections 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/2267607216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102e30 0x7f6be810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/2267607216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be81073d0 0x7f6be81077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/2267607216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be8102470 0x7f6be81028f0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 -- 172.21.15.26:0/2267607216 >> 172.21.15.26:0/2267607216 conn(0x7f6be8079480 msgr2=0x7f6be8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 -- 172.21.15.26:0/2267607216 shutdown_connections 2024-05-20T21:09:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.724+0000 7f6bef8dd700 1 -- 172.21.15.26:0/2267607216 wait complete. 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 Processor -- start 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 -- start start 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 0x7f6be81092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be8102e30 0x7f6be8109800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be81073d0 0x7f6be8109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8113650 con 0x7f6be81073d0 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81134d0 con 0x7f6be8102470 2024-05-20T21:09:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.725+0000 7f6bef8dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81137d0 con 0x7f6be8102e30 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bede7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be81073d0 0x7f6be8109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 0x7f6be81092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bede7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be81073d0 0x7f6be8109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41386/0 (socket says 172.21.15.26:41386) 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 0x7f6be81092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41598/0 (socket says 172.21.15.26:41598) 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bede7a700 1 -- 172.21.15.26:0/3272400812 learned_addr learned my addr 172.21.15.26:0/3272400812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bece78700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be8102e30 0x7f6be8109800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 -- 172.21.15.26:0/3272400812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be8102e30 msgr2=0x7f6be8109800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be8102e30 0x7f6be8109800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 -- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be81073d0 msgr2=0x7f6be8109d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be81073d0 0x7f6be8109d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 -- 172.21.15.26:0/3272400812 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be81ac340 con 0x7f6be8102470 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bece78700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be8102e30 0x7f6be8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.726+0000 7f6bed679700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 0x7f6be81092c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6be400d780 tx=0x7f6be400da90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.727+0000 7f6bde7fc700 1 -- 172.21.15.26:0/3272400812 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4016070 con 0x7f6be8102470 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.727+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be81ac630 con 0x7f6be8102470 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.727+0000 7f6bde7fc700 1 -- 172.21.15.26:0/3272400812 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6be4004830 con 0x7f6be8102470 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.727+0000 7f6bde7fc700 1 -- 172.21.15.26:0/3272400812 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4005490 con 0x7f6be8102470 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.727+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be81acb70 con 0x7f6be8102470 2024-05-20T21:09:03.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.729+0000 7f6bde7fc700 1 -- 172.21.15.26:0/3272400812 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6be400f690 con 0x7f6be8102470 2024-05-20T21:09:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.730+0000 7f6bde7fc700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bd40b2930 0x7f6bd40b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.731+0000 7f6bece78700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bd40b2930 0x7f6bd40b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.731+0000 7f6bde7fc700 1 -- 172.21.15.26:0/3272400812 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6be40e1fe0 con 0x7f6be8102470 2024-05-20T21:09:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.731+0000 7f6bece78700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bd40b2930 0x7f6bd40b4df0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6be810aa80 tx=0x7f6bd801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 Processor -- start 2024-05-20T21:09:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 -- start start 2024-05-20T21:09:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fbc9925f00 con 0x55fbc99c7c80 2024-05-20T21:09:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fbc9915aa0 con 0x55fbc99c8640 2024-05-20T21:09:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fbc9919ae0 con 0x55fbc99cc7b0 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bede7a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bece78700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bece78700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41388/0 (socket says 172.21.15.26:41388) 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.732+0000 7f6bece78700 1 -- 172.21.15.26:0/3673749705 learned_addr learned my addr 172.21.15.26:0/3673749705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bede7a700 1 -- 172.21.15.26:0/3673749705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 msgr2=0x55fbc99ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bede7a700 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bede7a700 1 -- 172.21.15.26:0/3673749705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 msgr2=0x55fbc99c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bede7a700 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bede7a700 1 -- 172.21.15.26:0/3673749705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc98bf140 con 0x55fbc99c8640 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bede7a700 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99d46b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6be00093d0 tx=0x7f6be0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485b143885ae12a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bd37fe700 1 -- 172.21.15.26:0/3673749705 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0017660 con 0x55fbc99c8640 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bd37fe700 1 -- 172.21.15.26:0/3673749705 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6be0017c90 con 0x55fbc99c8640 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.733+0000 7f6bd37fe700 1 -- 172.21.15.26:0/3673749705 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0015c10 con 0x55fbc99c8640 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.734+0000 7f6bfa173b80 1 -- 172.21.15.26:0/3673749705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 msgr2=0x55fbc99d46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99d46b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6be00093d0 tx=0x7f6be0009920 comp rx=0 tx=0).stop 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 -- 172.21.15.26:0/3673749705 shutdown_connections 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99d46b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/3673749705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 -- 172.21.15.26:0/3673749705 >> 172.21.15.26:0/3673749705 conn(0x55fbc98f33d0 msgr2=0x55fbc99c2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 -- 172.21.15.26:0/3673749705 shutdown_connections 2024-05-20T21:09:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 -- 172.21.15.26:0/3673749705 wait complete. 2024-05-20T21:09:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 -- start start 2024-05-20T21:09:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.735+0000 7f6bfa173b80 1 Processor -- start 2024-05-20T21:09:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c7c80 0x55fbc99d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c8640 0x55fbc99d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99cc7b0 0x55fbc99d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fbc981c130 con 0x55fbc99c8640 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fbc987df30 con 0x55fbc99cc7b0 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fbc99d4d10 con 0x55fbc99c7c80 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bfa173b80 10 client.0 _ll_register_callbacks cb 0x55fbc98b0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bed679700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c7c80 0x55fbc99d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bece78700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c8640 0x55fbc99d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bed679700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c7c80 0x55fbc99d43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45886/0 (socket says 172.21.15.26:45886) 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bed679700 1 -- 172.21.15.26:0/1047997752 learned_addr learned my addr 172.21.15.26:0/1047997752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99cc7b0 0x55fbc99d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bede7a700 1 -- 172.21.15.26:0/1047997752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c7c80 msgr2=0x55fbc99d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c7c80 0x55fbc99d43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bede7a700 1 -- 172.21.15.26:0/1047997752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c8640 msgr2=0x55fbc99d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c8640 0x55fbc99d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.736+0000 7f6bede7a700 1 -- 172.21.15.26:0/1047997752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc9865a60 con 0x55fbc99cc7b0 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99cc7b0 0x55fbc99d1bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009690 tx=0x7f6be0015a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5986b0b9cd92eff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1047997752 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0024030 con 0x55fbc99cc7b0 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1047997752 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6be0003a00 con 0x55fbc99cc7b0 2024-05-20T21:09:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be0017c90 con 0x55fbc99cc7b0 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1047997752 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6be0030050 con 0x55fbc99cc7b0 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bfa173b80 10 client.28706 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fbc98cf8f0 con 0x55fbc99cc7b0 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.737+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1047997752 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6be0017c90 con 0x55fbc99cc7b0 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1047997752 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f6be00051d0 con 0x55fbc99cc7b0 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bfa173b80 10 client.28706 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bfa173b80 10 client.28706 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bfa173b80 20 client.28706 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc99e1790 0x55fbc99e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bfa173b80 4 client.28706 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc9927e50 con 0x55fbc99e1790 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bed679700 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc99e1790 0x55fbc99e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bd0ff9700 10 client.28706 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.738+0000 7f6bed679700 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc99e1790 0x55fbc99e3b80 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504efb3ea166f48a server_cookie=5fceff541e7f74cd in_seq=0 out_seq=0 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1047997752 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fbc9927e50 con 0x55fbc99e1790 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bd0ff9700 10 client.28706 handle_command_reply: tid=0 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 client.28706 shutdown 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99cc7b0 msgr2=0x55fbc99d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99cc7b0 0x55fbc99d1bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009690 tx=0x7f6be0015a40 comp rx=0 tx=0).stop 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 shutdown_connections 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc99e1790 0x55fbc99e3b80 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99cc7b0 0x55fbc99d1bf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c7c80 0x55fbc99d43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1047997752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c8640 0x55fbc99d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 >> 172.21.15.26:0/1047997752 conn(0x55fbc98f33d0 msgr2=0x55fbc99c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.739+0000 7f6bd0ff9700 0 client.28706 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.740+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 shutdown_connections 2024-05-20T21:09:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.740+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1047997752 wait complete. 2024-05-20T21:09:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.740+0000 7f6bfa173b80 20 client.28706 trim_cache size 0 max 16384 2024-05-20T21:09:03.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 Processor -- start 2024-05-20T21:09:03.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 -- start start 2024-05-20T21:09:03.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81b6b70 con 0x55fbc99c7c80 2024-05-20T21:09:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81b4400 con 0x55fbc99c8640 2024-05-20T21:09:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.766+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81b2250 con 0x55fbc99cc7b0 2024-05-20T21:09:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bed679700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41632/0 (socket says 172.21.15.26:41632) 2024-05-20T21:09:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 -- 172.21.15.26:0/2800372814 learned_addr learned my addr 172.21.15.26:0/2800372814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bede7a700 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 -- 172.21.15.26:0/2800372814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 msgr2=0x55fbc99ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 -- 172.21.15.26:0/2800372814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 msgr2=0x55fbc99bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99bcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 -- 172.21.15.26:0/2800372814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc99c4080 con 0x55fbc99c8640 2024-05-20T21:09:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bed679700 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bece78700 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99cdc20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009990 tx=0x7f6bd8022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfab05378b1e4e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.767+0000 7f6bd2ffd700 1 -- 172.21.15.26:0/2800372814 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800fd20 con 0x55fbc99c8640 2024-05-20T21:09:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bd2ffd700 1 -- 172.21.15.26:0/2800372814 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bd80308b0 con 0x55fbc99c8640 2024-05-20T21:09:03.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bd2ffd700 1 -- 172.21.15.26:0/2800372814 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8023700 con 0x55fbc99c8640 2024-05-20T21:09:03.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 -- 172.21.15.26:0/2800372814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 msgr2=0x55fbc99cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99cdc20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009990 tx=0x7f6bd8022000 comp rx=0 tx=0).stop 2024-05-20T21:09:03.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 -- 172.21.15.26:0/2800372814 shutdown_connections 2024-05-20T21:09:03.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99c7c80 0x55fbc99bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99cc7b0 0x55fbc99ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/2800372814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c8640 0x55fbc99cdc20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.768+0000 7f6bfa173b80 1 -- 172.21.15.26:0/2800372814 >> 172.21.15.26:0/2800372814 conn(0x55fbc98f33d0 msgr2=0x55fbc99cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.769+0000 7f6bfa173b80 1 -- 172.21.15.26:0/2800372814 shutdown_connections 2024-05-20T21:09:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.769+0000 7f6bfa173b80 1 -- 172.21.15.26:0/2800372814 wait complete. 2024-05-20T21:09:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.769+0000 7f6bfa173b80 1 -- start start 2024-05-20T21:09:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.769+0000 7f6bfa173b80 1 Processor -- start 2024-05-20T21:09:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c7c80 0x55fbc99e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c8640 0x55fbc99e3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99cc7b0 0x55fbc99e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bed679700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c7c80 0x55fbc99e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81b28c0 con 0x55fbc99cc7b0 2024-05-20T21:09:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81b3d30 con 0x55fbc99c7c80 2024-05-20T21:09:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81b2f90 con 0x55fbc99c8640 2024-05-20T21:09:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bed679700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c7c80 0x55fbc99e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41634/0 (socket says 172.21.15.26:41634) 2024-05-20T21:09:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bed679700 1 -- 172.21.15.26:0/1788056385 learned_addr learned my addr 172.21.15.26:0/1788056385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bfa173b80 10 client.0 _ll_register_callbacks cb 0x55fbc98b0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bece78700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c8640 0x55fbc99e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.770+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99cc7b0 0x55fbc99e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bede7a700 1 -- 172.21.15.26:0/1788056385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c8640 msgr2=0x55fbc99e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c8640 0x55fbc99e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bede7a700 1 -- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c7c80 msgr2=0x55fbc99e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c7c80 0x55fbc99e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bede7a700 1 -- 172.21.15.26:0/1788056385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc99e3e10 con 0x55fbc99cc7b0 2024-05-20T21:09:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd80308b0 con 0x55fbc99cc7b0 2024-05-20T21:09:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bede7a700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99cc7b0 0x55fbc99e38a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6be0005610 tx=0x7f6be0015bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c39889499932cc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bd0ff9700 10 client.19470 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1788056385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0024040 con 0x55fbc99cc7b0 2024-05-20T21:09:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1788056385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6be0003c00 con 0x55fbc99cc7b0 2024-05-20T21:09:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1788056385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6be0009c20 con 0x55fbc99cc7b0 2024-05-20T21:09:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bfa173b80 10 client.19470 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.771+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55fbc9a4cc70 con 0x55fbc99cc7b0 2024-05-20T21:09:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1788056385 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f6be0004ed0 con 0x55fbc99cc7b0 2024-05-20T21:09:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bfa173b80 10 client.19470 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bfa173b80 10 client.19470 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:09:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bfa173b80 20 client.19470 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc9a4d130 0x55fbc9a4f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bfa173b80 4 client.19470 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-20T21:09:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55fbc9921630 con 0x55fbc9a4d130 2024-05-20T21:09:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.772+0000 7f6bed679700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc9a4d130 0x55fbc9a4f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.773+0000 7f6bd0ff9700 10 client.19470 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.773+0000 7f6bed679700 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc9a4d130 0x55fbc9a4f520 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6493bdc62cd2dca5 server_cookie=b07bc8efdb9da798 in_seq=0 out_seq=0 2024-05-20T21:09:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:03.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bd0ff9700 1 -- 172.21.15.26:0/1788056385 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fbc9921630 con 0x55fbc9a4d130 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bd0ff9700 10 client.19470 handle_command_reply: tid=0 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 client.19470 shutdown 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99cc7b0 msgr2=0x55fbc99e38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99cc7b0 0x55fbc99e38a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6be0005610 tx=0x7f6be0015bf0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 shutdown_connections 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fbc9a4d130 0x55fbc9a4f520 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fbc99cc7b0 0x55fbc99e38a0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bd0ff9700 0 client.19470 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fbc99c7c80 0x55fbc99e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 --2- 172.21.15.26:0/1788056385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fbc99c8640 0x55fbc99e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 >> 172.21.15.26:0/1788056385 conn(0x55fbc98f33d0 msgr2=0x55fbc99d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 shutdown_connections 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.909+0000 7f6bfa173b80 1 -- 172.21.15.26:0/1788056385 wait complete. 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bfa173b80 20 client.19470 trim_cache size 0 max 16384 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 50, 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "e258fdd1": { 2024-05-20T21:09:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 50, 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:09:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:03.913 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bd40b2930 msgr2=0x7f6bd40b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bd40b2930 0x7f6bd40b4df0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6be810aa80 tx=0x7f6bd801a040 comp rx=0 tx=0).stop 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 msgr2=0x7f6be81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 0x7f6be81092c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6be400d780 tx=0x7f6be400da90 comp rx=0 tx=0).stop 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 shutdown_connections 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6bd40b2930 0x7f6bd40b4df0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be81073d0 0x7f6be8109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6be8102470 0x7f6be81092c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 --2- 172.21.15.26:0/3272400812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be8102e30 0x7f6be8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 >> 172.21.15.26:0/3272400812 conn(0x7f6be8079480 msgr2=0x7f6be81002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.910+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 shutdown_connections 2024-05-20T21:09:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:03.911+0000 7f6bef8dd700 1 -- 172.21.15.26:0/3272400812 wait complete. 2024-05-20T21:09:04.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.25 MB/sec execute 550 sec latency 804.700 ms 2024-05-20T21:09:05.146 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:05 smithi026 ceph-mon[31990]: pgmap v778: 97 pgs: 97 active+clean; 28 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-05-20T21:09:05.180 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e258fdd1', quiesce took 11.2 seconds. Will release after: 18.8 2024-05-20T21:09:05.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:05 smithi144 ceph-mon[35660]: pgmap v778: 97 pgs: 97 active+clean; 28 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-05-20T21:09:05.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:05 smithi055 ceph-mon[35213]: pgmap v778: 97 pgs: 97 active+clean; 28 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-05-20T21:09:05.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.24 MB/sec execute 551 sec latency 1804.746 ms 2024-05-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 -- 172.21.15.26:0/47848258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 msgr2=0x7f1cf41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/47848258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf41028c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400b0a0 tx=0x7f1ce400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 -- 172.21.15.26:0/47848258 shutdown_connections 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/47848258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cf4102e00 0x7f1cf410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/47848258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108520 0x7f1cf4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/47848258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf41028c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 -- 172.21.15.26:0/47848258 >> 172.21.15.26:0/47848258 conn(0x7f1cf40793f0 msgr2=0x7f1cf4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 -- 172.21.15.26:0/47848258 shutdown_connections 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 -- 172.21.15.26:0/47848258 wait complete. 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 Processor -- start 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.219+0000 7f1cfac4e700 1 -- start start 2024-05-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cfac4e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cfac4e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cf4102e00 0x7f1cf419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cfac4e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108520 0x7f1cf41a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cfac4e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4112580 con 0x7f1cf4102440 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cfac4e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4112400 con 0x7f1cf4102e00 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cfac4e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4112700 con 0x7f1cf4108520 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41436/0 (socket says 172.21.15.26:41436) 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 -- 172.21.15.26:0/3648399989 learned_addr learned my addr 172.21.15.26:0/3648399989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf3fff700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cf4102e00 0x7f1cf419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf91eb700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108520 0x7f1cf41a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 -- 172.21.15.26:0/3648399989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108520 msgr2=0x7f1cf41a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108520 0x7f1cf41a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 -- 172.21.15.26:0/3648399989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cf4102e00 msgr2=0x7f1cf419b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cf4102e00 0x7f1cf419b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 -- 172.21.15.26:0/3648399989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf41a6080 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf89ea700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf419aac0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b530 tx=0x7f1cdc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc015020 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.220+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cdc011cc0 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.221+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf41a6250 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.221+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc010dc0 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.221+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a67b0 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.222+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf410a9b0 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.223+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1cdc018020 con 0x7f1cf4102440 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.224+0000 7f1cf1ffb700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1ce00b2900 0x7f1ce00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.224+0000 7f1cf3fff700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1ce00b2900 0x7f1ce00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.224+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f1cdc0e1a20 con 0x7f1cf4102440 2024-05-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.225+0000 7f1cf3fff700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1ce00b2900 0x7f1ce00b4dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4079b40 tx=0x7f1ce400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.229+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cdc01f020 con 0x7f1cf4102440 2024-05-20T21:09:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.494+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cf406a230 con 0x7f1cf4102440 2024-05-20T21:09:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.494+0000 7f1cf1ffb700 1 -- 172.21.15.26:0/3648399989 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f1cdc0a96e0 con 0x7f1cf4102440 2024-05-20T21:09:06.495 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:09:06.495 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:09:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1ce00b2900 msgr2=0x7f1ce00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1ce00b2900 0x7f1ce00b4dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4079b40 tx=0x7f1ce400b7b0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 msgr2=0x7f1cf419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf419aac0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b530 tx=0x7f1cdc00b840 comp rx=0 tx=0).stop 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 shutdown_connections 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f1ce00b2900 0x7f1ce00b4dc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf4102440 0x7f1cf419aac0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108520 0x7f1cf41a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 --2- 172.21.15.26:0/3648399989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cf4102e00 0x7f1cf419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 >> 172.21.15.26:0/3648399989 conn(0x7f1cf40793f0 msgr2=0x7f1cf4100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.497+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 shutdown_connections 2024-05-20T21:09:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:06.498+0000 7f1cfac4e700 1 -- 172.21.15.26:0/3648399989 wait complete. 2024-05-20T21:09:06.499 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:09:06.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.24 MB/sec execute 552 sec latency 2804.812 ms 2024-05-20T21:09:07.117 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:07 smithi026 ceph-mon[31990]: pgmap v779: 97 pgs: 97 active+clean; 30 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-05-20T21:09:07.118 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:07 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3648399989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:07.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:09:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:07 smithi055 ceph-mon[35213]: pgmap v779: 97 pgs: 97 active+clean; 30 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-05-20T21:09:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:07 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3648399989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:07 smithi144 ceph-mon[35660]: pgmap v779: 97 pgs: 97 active+clean; 30 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-05-20T21:09:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:07 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3648399989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:07.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.23 MB/sec execute 553 sec latency 3804.871 ms 2024-05-20T21:09:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.207+0000 7f2213e59640 1 -- 172.21.15.26:0/3580431474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 msgr2=0x7f220c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.207+0000 7f2213e59640 1 --2- 172.21.15.26:0/3580431474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c10aef0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2204009ca0 tx=0x7f220402a8c0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- 172.21.15.26:0/3580431474 shutdown_connections 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 --2- 172.21.15.26:0/3580431474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c10aef0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 --2- 172.21.15.26:0/3580431474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220c103ca0 0x7f220c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 --2- 172.21.15.26:0/3580431474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 0x7f220c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- 172.21.15.26:0/3580431474 >> 172.21.15.26:0/3580431474 conn(0x7f220c0fe040 msgr2=0x7f220c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- 172.21.15.26:0/3580431474 shutdown_connections 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- 172.21.15.26:0/3580431474 wait complete. 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 Processor -- start 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- start start 2024-05-20T21:09:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 0x7f220c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220c103ca0 0x7f220c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10e4c0 con 0x7f220c104660 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10e340 con 0x7f220c103ca0 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.208+0000 7f2213e59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10e640 con 0x7f220c102ab0 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c1a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41450/0 (socket says 172.21.15.26:41450) 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 -- 172.21.15.26:0/895909408 learned_addr learned my addr 172.21.15.26:0/895909408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22113f4640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220c103ca0 0x7f220c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2211bf5640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 0x7f220c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 -- 172.21.15.26:0/895909408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 msgr2=0x7f220c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 0x7f220c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 -- 172.21.15.26:0/895909408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220c103ca0 msgr2=0x7f220c198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220c103ca0 0x7f220c198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 -- 172.21.15.26:0/895909408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c1a06c0 con 0x7f220c104660 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2211bf5640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 0x7f220c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f22123f6640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c1a0050 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f220402aeb0 tx=0x7f2204003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2202ffd640 1 -- 172.21.15.26:0/895909408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204003e10 con 0x7f220c104660 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c1a0950 con 0x7f220c104660 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2202ffd640 1 -- 172.21.15.26:0/895909408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22040044d0 con 0x7f220c104660 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2202ffd640 1 -- 172.21.15.26:0/895909408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204030c90 con 0x7f220c104660 2024-05-20T21:09:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.209+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a0ea0 con 0x7f220c104660 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 Processor -- start 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2202ffd640 1 -- 172.21.15.26:0/895909408 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2204004680 con 0x7f220c104660 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 -- start start 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766c6c0 0x56397766cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766d0b0 0x563977671bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 0x5639776746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5639775ddc30 con 0x563977672220 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5639775cd7d0 con 0x56397766c6c0 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.212+0000 7f2216b79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5639775d1810 con 0x56397766d0b0 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 0x5639776746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 0x5639776746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41458/0 (socket says 172.21.15.26:41458) 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 -- 172.21.15.26:0/449031736 learned_addr learned my addr 172.21.15.26:0/449031736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766c6c0 0x56397766cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f2211bf5640 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766d0b0 0x563977671bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f2202ffd640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21f80b28a0 0x7f21f80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 -- 172.21.15.26:0/449031736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766d0b0 msgr2=0x563977671bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766d0b0 0x563977671bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 -- 172.21.15.26:0/449031736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766c6c0 msgr2=0x56397766cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766c6c0 0x56397766cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 -- 172.21.15.26:0/449031736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639772e2210 con 0x563977672220 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22123f6640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21f80b28a0 0x7f21f80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f2202ffd640 1 -- 172.21.15.26:0/895909408 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22041038f0 con 0x7f220c104660 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f22113f4640 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 0x5639776746b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0126e0 tx=0x7f21fc009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a87b292dba40d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.213+0000 7f21f7fff640 1 -- 172.21.15.26:0/449031736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc012d20 con 0x563977672220 2024-05-20T21:09:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f21f7fff640 1 -- 172.21.15.26:0/449031736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21fc0107d0 con 0x563977672220 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f21f7fff640 1 -- 172.21.15.26:0/449031736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc003850 con 0x563977672220 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 -- 172.21.15.26:0/449031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 msgr2=0x5639776746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f22123f6640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21f80b28a0 0x7f21f80b4d60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f22040cabb0 tx=0x7f22040bf780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 0x5639776746b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0126e0 tx=0x7f21fc009660 comp rx=0 tx=0).stop 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 -- 172.21.15.26:0/449031736 shutdown_connections 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563977672220 0x5639776746b0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766d0b0 0x563977671bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 --2- 172.21.15.26:0/449031736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766c6c0 0x56397766cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 -- 172.21.15.26:0/449031736 >> 172.21.15.26:0/449031736 conn(0x563977667d30 msgr2=0x563977668130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 -- 172.21.15.26:0/449031736 shutdown_connections 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.214+0000 7f2216b79740 1 -- 172.21.15.26:0/449031736 wait complete. 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 -- start start 2024-05-20T21:09:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 Processor -- start 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766c6c0 0x563977676810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 0x563977676d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563977672220 0x56397767dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5639772b3790 con 0x56397766d0b0 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563977589890 con 0x563977672220 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5639776751d0 con 0x56397766c6c0 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2216b79740 10 client.0 _ll_register_callbacks cb 0x5639772e1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2211bf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 0x563977676d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2211bf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 0x563977676d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41470/0 (socket says 172.21.15.26:41470) 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.215+0000 7f2211bf5640 1 -- 172.21.15.26:0/48196337 learned_addr learned my addr 172.21.15.26:0/48196337 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f22113f4640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766c6c0 0x563977676810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f22113f4640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563977672220 0x56397767dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2211bf5640 1 -- 172.21.15.26:0/48196337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766c6c0 msgr2=0x563977676810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2211bf5640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766c6c0 0x563977676810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2211bf5640 1 -- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563977672220 msgr2=0x56397767dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2211bf5640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563977672220 0x56397767dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2211bf5640 1 -- 172.21.15.26:0/48196337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639773ae620 con 0x56397766d0b0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f22113f4640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766c6c0 0x563977676810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2211bf5640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 0x563977676d50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f220c103b00 tx=0x7f22080117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47510edb9809edb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f21f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f21f57fa640 1 -- 172.21.15.26:0/48196337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208005220 con 0x56397766d0b0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f21f57fa640 1 -- 172.21.15.26:0/48196337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f22080053c0 con 0x56397766d0b0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc0107d0 con 0x56397766d0b0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f21f57fa640 1 -- 172.21.15.26:0/48196337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22080055c0 con 0x56397766d0b0 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2216b79740 10 client.19488 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563977680e40 con 0x56397766d0b0 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.216+0000 7f21f57fa640 1 -- 172.21.15.26:0/48196337 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21fc0107d0 con 0x56397766d0b0 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f21f57fa640 1 -- 172.21.15.26:0/48196337 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f2208022020 con 0x56397766d0b0 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f2216b79740 10 client.19488 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f2216b79740 10 client.19488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f2216b79740 20 client.19488 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f2216b79740 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563977682640 0x563977684a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f2216b79740 4 client.19488 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639775dfb80 con 0x563977682640 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.217+0000 7f22113f4640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563977682640 0x563977684a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f22113f4640 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563977682640 0x563977684a80 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98de361be61abda server_cookie=2fb75cd59f82ca75 in_seq=0 out_seq=0 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f21f57fa640 10 client.19488 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f21f57fa640 1 -- 172.21.15.26:0/48196337 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5639775dfb80 con 0x563977682640 2024-05-20T21:09:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f21f57fa640 10 client.19488 handle_command_reply: tid=0 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f2216b79740 1 client.19488 shutdown 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 msgr2=0x563977676d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.218+0000 7f2216b79740 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 0x563977676d50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f220c103b00 tx=0x7f22080117b0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 shutdown_connections 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x563977682640 0x563977684a80 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563977672220 0x56397767dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766d0b0 0x563977676d50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 --2- 172.21.15.26:0/48196337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56397766c6c0 0x563977676810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f21f57fa640 0 client.19488 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 >> 172.21.15.26:0/48196337 conn(0x563977667d30 msgr2=0x563977666fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 shutdown_connections 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 1 -- 172.21.15.26:0/48196337 wait complete. 2024-05-20T21:09:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.219+0000 7f2216b79740 20 client.19488 trim_cache size 0 max 16384 2024-05-20T21:09:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 Processor -- start 2024-05-20T21:09:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 -- start start 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x563977671960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 0x56397766e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1ac740 con 0x56397766c6c0 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1aa010 con 0x56397766d0b0 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2216b79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1a7e60 con 0x563977672220 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x563977671960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x563977671960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41474/0 (socket says 172.21.15.26:41474) 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 -- 172.21.15.26:0/507439135 learned_addr learned my addr 172.21.15.26:0/507439135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 -- 172.21.15.26:0/507439135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 msgr2=0x56397766e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f2211bf5640 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 0x56397766e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 -- 172.21.15.26:0/507439135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 msgr2=0x56397767dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.234+0000 7f22113f4640 1 -- 172.21.15.26:0/507439135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639776715e0 con 0x56397766c6c0 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f22113f4640 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x563977671960 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0126b0 tx=0x7f21fc0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef837d35851859c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f21f77fe640 1 -- 172.21.15.26:0/507439135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0107a0 con 0x56397766c6c0 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f21f77fe640 1 -- 172.21.15.26:0/507439135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21fc012040 con 0x56397766c6c0 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f21f77fe640 1 -- 172.21.15.26:0/507439135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0035f0 con 0x56397766c6c0 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 -- 172.21.15.26:0/507439135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 msgr2=0x563977671960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x563977671960 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0126b0 tx=0x7f21fc0095e0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 -- 172.21.15.26:0/507439135 shutdown_connections 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 0x56397766e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 --2- 172.21.15.26:0/507439135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x563977671960 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 -- 172.21.15.26:0/507439135 >> 172.21.15.26:0/507439135 conn(0x563977667d30 msgr2=0x563977668110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 -- 172.21.15.26:0/507439135 shutdown_connections 2024-05-20T21:09:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.235+0000 7f2216b79740 1 -- 172.21.15.26:0/507439135 wait complete. 2024-05-20T21:09:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 -- start start 2024-05-20T21:09:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 Processor -- start 2024-05-20T21:09:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x56397767b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 0x56397767c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1a84d0 con 0x56397766c6c0 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1a9940 con 0x56397766d0b0 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1a8ba0 con 0x563977672220 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2216b79740 10 client.0 _ll_register_callbacks cb 0x7f22080097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f22113f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x56397767b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f22113f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x56397767b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41478/0 (socket says 172.21.15.26:41478) 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f22113f4640 1 -- 172.21.15.26:0/2821505015 learned_addr learned my addr 172.21.15.26:0/2821505015 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.236+0000 7f2211bf5640 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f22113f4640 1 -- 172.21.15.26:0/2821505015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 msgr2=0x56397767c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f22113f4640 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 0x56397767c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f22113f4640 1 -- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 msgr2=0x56397767b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f22113f4640 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f22113f4640 1 -- 172.21.15.26:0/2821505015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397767cc40 con 0x56397766c6c0 2024-05-20T21:09:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f22113f4640 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x56397767b100 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f21fc010840 tx=0x7f21fc010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f31ab080ee13c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f21f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f21f57fa640 1 -- 172.21.15.26:0/2821505015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc031030 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f21f57fa640 1 -- 172.21.15.26:0/2821505015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f21fc005410 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc012040 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f21f57fa640 1 -- 172.21.15.26:0/2821505015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f21fc005070 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 10 client.19494 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56397767df50 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f21f57fa640 1 -- 172.21.15.26:0/2821505015 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21fc012040 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f21f57fa640 1 -- 172.21.15.26:0/2821505015 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f21fc02c800 con 0x56397766c6c0 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 10 client.19494 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 10 client.19494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 20 client.19494 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5639776ed6a0 0x5639776efac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 4 client.19494 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:09:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.237+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639775d9360 con 0x5639776ed6a0 2024-05-20T21:09:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.238+0000 7f2211bf5640 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5639776ed6a0 0x5639776efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.238+0000 7f2211bf5640 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5639776ed6a0 0x5639776efac0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81801bd37d479604 server_cookie=16314e09252f8204 in_seq=0 out_seq=0 2024-05-20T21:09:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.238+0000 7f21f57fa640 10 client.19494 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.265+0000 7f21f57fa640 1 -- 172.21.15.26:0/2821505015 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5639775d9360 con 0x5639776ed6a0 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.265+0000 7f21f57fa640 10 client.19494 handle_command_reply: tid=0 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.265+0000 7f2216b79740 1 client.19494 shutdown 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.265+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 msgr2=0x56397767b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.265+0000 7f2216b79740 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x56397767b100 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f21fc010840 tx=0x7f21fc010870 comp rx=0 tx=0).stop 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 shutdown_connections 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5639776ed6a0 0x5639776efac0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563977672220 0x56397767c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56397766d0b0 0x56397767b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 --2- 172.21.15.26:0/2821505015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56397766c6c0 0x56397767b100 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 >> 172.21.15.26:0/2821505015 conn(0x563977667d30 msgr2=0x56397766e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f21f57fa640 0 client.19494 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 shutdown_connections 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 1 -- 172.21.15.26:0/2821505015 wait complete. 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.266+0000 7f2216b79740 20 client.19494 trim_cache size 0 max 16384 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:09:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:09:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21f80b28a0 msgr2=0x7f21f80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21f80b28a0 0x7f21f80b4d60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f22040cabb0 tx=0x7f22040bf780 comp rx=0 tx=0).stop 2024-05-20T21:09:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 msgr2=0x7f220c1a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c1a0050 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f220402aeb0 tx=0x7f2204003c20 comp rx=0 tx=0).stop 2024-05-20T21:09:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 shutdown_connections 2024-05-20T21:09:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f21f80b28a0 0x7f21f80b4d60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220c104660 0x7f220c1a0050 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220c103ca0 0x7f220c198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 --2- 172.21.15.26:0/895909408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220c102ab0 0x7f220c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 >> 172.21.15.26:0/895909408 conn(0x7f220c0fe040 msgr2=0x7f220c106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 shutdown_connections 2024-05-20T21:09:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:08.267+0000 7f2213e59640 1 -- 172.21.15.26:0/895909408 wait complete. 2024-05-20T21:09:08.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.23 MB/sec execute 554 sec latency 4804.942 ms 2024-05-20T21:09:09.367 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:09 smithi026 ceph-mon[31990]: pgmap v780: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-20T21:09:09.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:09 smithi055 ceph-mon[35213]: pgmap v780: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-20T21:09:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 -- 172.21.15.26:0/21766070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 msgr2=0x7fafa41103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 --2- 172.21.15.26:0/21766070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41103c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b0a0 tx=0x7faf9c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 -- 172.21.15.26:0/21766070 shutdown_connections 2024-05-20T21:09:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 --2- 172.21.15.26:0/21766070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41103c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 --2- 172.21.15.26:0/21766070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa41095a0 0x7fafa4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 --2- 172.21.15.26:0/21766070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafa406bab0 0x7fafa406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 -- 172.21.15.26:0/21766070 >> 172.21.15.26:0/21766070 conn(0x7fafa40793b0 msgr2=0x7fafa40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 -- 172.21.15.26:0/21766070 shutdown_connections 2024-05-20T21:09:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.477+0000 7fafa884c700 1 -- 172.21.15.26:0/21766070 wait complete. 2024-05-20T21:09:09.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 Processor -- start 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 -- start start 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa406bab0 0x7fafa41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafa41095a0 0x7fafa419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4113580 con 0x7fafa406c470 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4113400 con 0x7fafa41095a0 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.478+0000 7fafa884c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4113700 con 0x7fafa406bab0 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41500/0 (socket says 172.21.15.26:41500) 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 -- 172.21.15.26:0/3917136890 learned_addr learned my addr 172.21.15.26:0/3917136890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7fafa259c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa406bab0 0x7fafa41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafa41095a0 0x7fafa419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 -- 172.21.15.26:0/3917136890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa406bab0 msgr2=0x7fafa41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa406bab0 0x7fafa41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 -- 172.21.15.26:0/3917136890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafa41095a0 msgr2=0x7fafa419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafa41095a0 0x7fafa419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 -- 172.21.15.26:0/3917136890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa419c280 con 0x7fafa406c470 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41a1e50 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00d950 tx=0x7faf8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf9bfff700 1 -- 172.21.15.26:0/3917136890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c016070 con 0x7fafa406c470 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa419c510 con 0x7fafa406c470 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7faf9bfff700 1 -- 172.21.15.26:0/3917136890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf8c00fd20 con 0x7fafa406c470 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.479+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa41a87e0 con 0x7fafa406c470 2024-05-20T21:09:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.480+0000 7faf9bfff700 1 -- 172.21.15.26:0/3917136890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00e3b0 con 0x7fafa406c470 2024-05-20T21:09:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 Processor -- start 2024-05-20T21:09:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 -- start start 2024-05-20T21:09:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7faf9bfff700 1 -- 172.21.15.26:0/3917136890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7faf8c019070 con 0x7fafa406c470 2024-05-20T21:09:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cc210 0x5575409cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 0x5575409d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55754092ac90 con 0x5575409cff60 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55754091a830 con 0x5575409cc210 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.483+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55754091e870 con 0x5575409cb860 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 0x5575409d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 0x5575409d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41506/0 (socket says 172.21.15.26:41506) 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7faf99d9b700 1 -- 172.21.15.26:0/311606546 learned_addr learned my addr 172.21.15.26:0/311606546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa259c700 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cc210 0x5575409cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 -- 172.21.15.26:0/311606546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cc210 msgr2=0x5575409cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cc210 0x5575409cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 -- 172.21.15.26:0/311606546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 msgr2=0x5575409d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 0x5575409d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 -- 172.21.15.26:0/311606546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55754086a8d0 con 0x5575409cb860 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7faf99d9b700 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 0x5575409d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7faf9bfff700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf900b2880 0x7faf900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409cbc40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b070 tx=0x7faf9c009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8511fb06aadd87c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.484+0000 7fafa259c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf900b2880 0x7faf900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.485+0000 7faf9bfff700 1 -- 172.21.15.26:0/3917136890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faf8c0e2040 con 0x7fafa406c470 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.485+0000 7faf98d99700 1 -- 172.21.15.26:0/311606546 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c020070 con 0x5575409cb860 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.485+0000 7faf98d99700 1 -- 172.21.15.26:0/311606546 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf9c0074e0 con 0x5575409cb860 2024-05-20T21:09:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.485+0000 7faf98d99700 1 -- 172.21.15.26:0/311606546 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c005280 con 0x5575409cb860 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafa259c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf900b2880 0x7faf900b4d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b750 tx=0x7faf9400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 -- 172.21.15.26:0/311606546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 msgr2=0x5575409cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409cbc40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b070 tx=0x7faf9c009e20 comp rx=0 tx=0).stop 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 -- 172.21.15.26:0/311606546 shutdown_connections 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cff60 0x5575409d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409cbc40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/311606546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cc210 0x5575409cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 -- 172.21.15.26:0/311606546 >> 172.21.15.26:0/311606546 conn(0x5575408f9080 msgr2=0x5575409c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 -- 172.21.15.26:0/311606546 shutdown_connections 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 -- 172.21.15.26:0/311606546 wait complete. 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 -- start start 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.486+0000 7fafb30e2b80 1 Processor -- start 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557540820fa0 con 0x5575409cc210 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5575408bc7c0 con 0x5575409cff60 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5575409d80c0 con 0x5575409cb860 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 10 client.0 _ll_register_callbacks cb 0x5575408b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41510/0 (socket says 172.21.15.26:41510) 2024-05-20T21:09:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 -- 172.21.15.26:0/1020837717 learned_addr learned my addr 172.21.15.26:0/1020837717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafa259c700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 -- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 msgr2=0x5575409d7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409d7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 -- 172.21.15.26:0/1020837717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 msgr2=0x5575409d4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409d4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 -- 172.21.15.26:0/1020837717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575408b9690 con 0x5575409cc210 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf99d9b700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4680 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0a93a0 tx=0x7faf8c0e5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40a9af2985e3b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf89ffb700 1 -- 172.21.15.26:0/1020837717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c020070 con 0x5575409cc210 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7faf89ffb700 1 -- 172.21.15.26:0/1020837717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faf8c091110 con 0x5575409cc210 2024-05-20T21:09:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.487+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9c0074e0 con 0x5575409cc210 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7faf89ffb700 1 -- 172.21.15.26:0/1020837717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf8c0a9470 con 0x5575409cc210 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 10 client.19506 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575409e27d0 con 0x5575409cc210 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7faf89ffb700 1 -- 172.21.15.26:0/1020837717 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf9c0074e0 con 0x5575409cc210 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7faf89ffb700 1 -- 172.21.15.26:0/1020837717 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7faf8c0a9900 con 0x5575409cc210 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 10 client.19506 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 10 client.19506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 20 client.19506 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5575409e4e10 0x5575409e7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 4 client.19506 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575409979e0 con 0x5575409e4e10 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.488+0000 7fafa259c700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5575409e4e10 0x5575409e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.489+0000 7fafa259c700 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5575409e4e10 0x5575409e7200 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ad19b86a32d7799 server_cookie=e98527a223c91c7f in_seq=0 out_seq=0 2024-05-20T21:09:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.489+0000 7faf89ffb700 10 client.19506 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.489+0000 7faf89ffb700 1 -- 172.21.15.26:0/1020837717 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5575409979e0 con 0x5575409e4e10 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.489+0000 7faf89ffb700 10 client.19506 handle_command_reply: tid=0 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 client.19506 shutdown 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 msgr2=0x5575409d4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4680 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0a93a0 tx=0x7faf8c0e5720 comp rx=0 tx=0).stop 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 shutdown_connections 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x5575409e4e10 0x5575409e7200 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cb860 0x5575409d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/1020837717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4680 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 >> 172.21.15.26:0/1020837717 conn(0x5575408f9080 msgr2=0x5575409c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7faf89ffb700 0 client.19506 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 shutdown_connections 2024-05-20T21:09:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 1 -- 172.21.15.26:0/1020837717 wait complete. 2024-05-20T21:09:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.490+0000 7fafb30e2b80 20 client.19506 trim_cache size 0 max 16384 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.505+0000 7fafb30e2b80 1 Processor -- start 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.505+0000 7fafb30e2b80 1 -- start start 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.505+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cb860 0x5575409cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.505+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.505+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cff60 0x5575409d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41b26a0 con 0x5575409cc210 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41aff30 con 0x5575409cb860 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41add80 con 0x5575409cff60 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41514/0 (socket says 172.21.15.26:41514) 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 -- 172.21.15.26:0/3331641580 learned_addr learned my addr 172.21.15.26:0/3331641580 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7fafa259c700 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cb860 0x5575409cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cff60 0x5575409d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 -- 172.21.15.26:0/3331641580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cff60 msgr2=0x5575409d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cff60 0x5575409d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 -- 172.21.15.26:0/3331641580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cb860 msgr2=0x5575409cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cb860 0x5575409cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 -- 172.21.15.26:0/3331641580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575409cfcb0 con 0x5575409cc210 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4cb0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7faf8c010a20 tx=0x7faf8c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef9a5cf94068146 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf8bfff700 1 -- 172.21.15.26:0/3331641580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c020050 con 0x5575409cc210 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf8bfff700 1 -- 172.21.15.26:0/3331641580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf8c014020 con 0x5575409cc210 2024-05-20T21:09:09.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.506+0000 7faf8bfff700 1 -- 172.21.15.26:0/3331641580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00fed0 con 0x5575409cc210 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3331641580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 msgr2=0x5575409d4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4cb0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7faf8c010a20 tx=0x7faf8c005910 comp rx=0 tx=0).stop 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3331641580 shutdown_connections 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cff60 0x5575409d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cb860 0x5575409cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3331641580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cc210 0x5575409d4cb0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3331641580 >> 172.21.15.26:0/3331641580 conn(0x5575408f9080 msgr2=0x5575409d5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3331641580 shutdown_connections 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.507+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3331641580 wait complete. 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 -- start start 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 Processor -- start 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 0x5575409ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cc210 0x5575409daec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41ae3f0 con 0x5575409cb860 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41af860 con 0x5575409cff60 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41aeac0 con 0x5575409cc210 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafb30e2b80 10 client.0 _ll_register_callbacks cb 0x5575408b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 0x5575409ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 0x5575409ddb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41524/0 (socket says 172.21.15.26:41524) 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 -- 172.21.15.26:0/3112620765 learned_addr learned my addr 172.21.15.26:0/3112620765 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cc210 0x5575409daec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa2d9d700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 -- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cc210 msgr2=0x5575409daec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cc210 0x5575409daec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 -- 172.21.15.26:0/3112620765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 msgr2=0x5575409db480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409db480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.508+0000 7fafa259c700 1 -- 172.21.15.26:0/3112620765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575409dbcc0 con 0x5575409cb860 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7fafa259c700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 0x5575409ddb20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7faf9400bfd0 tx=0x7faf940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41e432d0d2237f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7faf89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7faf89ffb700 1 -- 172.21.15.26:0/3112620765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94003e50 con 0x5575409cb860 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7faf89ffb700 1 -- 172.21.15.26:0/3112620765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7faf9401e880 con 0x5575409cb860 2024-05-20T21:09:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf8c014020 con 0x5575409cb860 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7faf89ffb700 1 -- 172.21.15.26:0/3112620765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf9401ea80 con 0x5575409cb860 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7fafb30e2b80 10 client.19512 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557540a4f9d0 con 0x5575409cb860 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.509+0000 7faf89ffb700 1 -- 172.21.15.26:0/3112620765 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf8c014020 con 0x5575409cb860 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7faf89ffb700 1 -- 172.21.15.26:0/3112620765 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7faf9403b050 con 0x5575409cb860 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7fafb30e2b80 10 client.19512 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7fafb30e2b80 10 client.19512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7fafb30e2b80 20 client.19512 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x557540a50090 0x557540a524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7fafb30e2b80 4 client.19512 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55754092cbe0 con 0x557540a50090 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x557540a50090 0x557540a524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7faf89ffb700 10 client.19512 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.510+0000 7faf99d9b700 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x557540a50090 0x557540a524b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d88d34bc5ad1229b server_cookie=229a3ffc6f5519a2 in_seq=0 out_seq=0 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7faf89ffb700 1 -- 172.21.15.26:0/3112620765 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55754092cbe0 con 0x557540a50090 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7faf89ffb700 10 client.19512 handle_command_reply: tid=0 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 client.19512 shutdown 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 msgr2=0x5575409ddb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 0x5575409ddb20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7faf9400bfd0 tx=0x7faf940047c0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 shutdown_connections 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x557540a50090 0x557540a524b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575409cb860 0x5575409ddb20 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5575409cff60 0x5575409db480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 --2- 172.21.15.26:0/3112620765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575409cc210 0x5575409daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 >> 172.21.15.26:0/3112620765 conn(0x5575408f9080 msgr2=0x5575409caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7faf89ffb700 0 client.19512 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:09.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 shutdown_connections 2024-05-20T21:09:09.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 1 -- 172.21.15.26:0/3112620765 wait complete. 2024-05-20T21:09:09.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.511+0000 7fafb30e2b80 20 client.19512 trim_cache size 0 max 16384 2024-05-20T21:09:09.517 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf900b2880 msgr2=0x7faf900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf900b2880 0x7faf900b4d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b750 tx=0x7faf9400b3c0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 msgr2=0x7fafa41a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41a1e50 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00d950 tx=0x7faf8c00dc60 comp rx=0 tx=0).stop 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 shutdown_connections 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafa41095a0 0x7fafa419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa406bab0 0x7fafa41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7faf900b2880 0x7faf900b4d40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 --2- 172.21.15.26:0/3917136890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafa406c470 0x7fafa41a1e50 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 >> 172.21.15.26:0/3917136890 conn(0x7fafa40793b0 msgr2=0x7fafa4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 shutdown_connections 2024-05-20T21:09:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:09.512+0000 7fafa884c700 1 -- 172.21.15.26:0/3917136890 wait complete. 2024-05-20T21:09:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:09 smithi144 ceph-mon[35660]: pgmap v780: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-20T21:09:09.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.23 MB/sec execute 555 sec latency 5804.982 ms 2024-05-20T21:09:10.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:09:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:10.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.22 MB/sec execute 556 sec latency 6805.066 ms 2024-05-20T21:09:11.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:11 smithi026 ceph-mon[31990]: pgmap v781: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-05-20T21:09:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:11 smithi055 ceph-mon[35213]: pgmap v781: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-05-20T21:09:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:11 smithi144 ceph-mon[35660]: pgmap v781: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-05-20T21:09:11.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.22 MB/sec execute 557 sec latency 7805.111 ms 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 -- 172.21.15.26:0/2542609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c90101b30 msgr2=0x7f5c90101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2542609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c90101b30 0x7f5c90101fb0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5c780099c0 tx=0x7f5c78009cd0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 -- 172.21.15.26:0/2542609 shutdown_connections 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2542609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c901024f0 0x7f5c901103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2542609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90109520 0x7f5c90109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2542609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c90101b30 0x7f5c90101fb0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 -- 172.21.15.26:0/2542609 >> 172.21.15.26:0/2542609 conn(0x7f5c90078ab0 msgr2=0x7f5c900ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 -- 172.21.15.26:0/2542609 shutdown_connections 2024-05-20T21:09:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.433+0000 7f5c963fc700 1 -- 172.21.15.26:0/2542609 wait complete. 2024-05-20T21:09:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 Processor -- start 2024-05-20T21:09:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 -- start start 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 0x7f5c9019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 0x7f5c9019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c90109520 0x7f5c90199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90113610 con 0x7f5c901024f0 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90113490 con 0x7f5c90101b30 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c963fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90113790 con 0x7f5c90109520 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 0x7f5c9019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 0x7f5c9019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c94999700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c90109520 0x7f5c90199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 0x7f5c9019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56742/0 (socket says 172.21.15.26:56742) 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c87fff700 1 -- 172.21.15.26:0/2345509755 learned_addr learned my addr 172.21.15.26:0/2345509755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.434+0000 7f5c8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 0x7f5c9019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:36530/0 (socket says 172.21.15.26:36530) 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c87fff700 1 -- 172.21.15.26:0/2345509755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c90109520 msgr2=0x7f5c90199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c90109520 0x7f5c90199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c87fff700 1 -- 172.21.15.26:0/2345509755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 msgr2=0x7f5c9019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 0x7f5c9019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c87fff700 1 -- 172.21.15.26:0/2345509755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9019a090 con 0x7f5c901024f0 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 0x7f5c9019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c94999700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c90109520 0x7f5c90199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 0x7f5c9019fcb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5c78006010 tx=0x7f5c78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c8dffb700 1 -- 172.21.15.26:0/2345509755 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78026070 con 0x7f5c901024f0 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9019a320 con 0x7f5c901024f0 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c8dffb700 1 -- 172.21.15.26:0/2345509755 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c78004630 con 0x7f5c901024f0 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c8dffb700 1 -- 172.21.15.26:0/2345509755 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800faa0 con 0x7f5c901024f0 2024-05-20T21:09:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.435+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c901a6640 con 0x7f5c901024f0 2024-05-20T21:09:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.437+0000 7f5c8dffb700 1 -- 172.21.15.26:0/2345509755 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5c78029070 con 0x7f5c901024f0 2024-05-20T21:09:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5c8dffb700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c7c0b2880 0x7f5c7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c7c0b2880 0x7f5c7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5c8dffb700 1 -- 172.21.15.26:0/2345509755 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5c780e29c0 con 0x7f5c901024f0 2024-05-20T21:09:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5ca0c92b80 1 Processor -- start 2024-05-20T21:09:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c7c0b2880 0x7f5c7c0b4d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5c9019aa70 tx=0x7f5c80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5ca0c92b80 1 -- start start 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e5095c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e5099940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.439+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5099f60 0x55f7e509a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7e4ff4c90 con 0x55f7e5096210 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f7e4fe4830 con 0x55f7e5095860 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7e4fe8870 con 0x55f7e5099f60 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e5099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e5099940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56758/0 (socket says 172.21.15.26:56758) 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 -- 172.21.15.26:0/1683376661 learned_addr learned my addr 172.21.15.26:0/1683376661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e5095c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c94999700 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5099f60 0x55f7e509a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 -- 172.21.15.26:0/1683376661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5099f60 msgr2=0x55f7e509a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5099f60 0x55f7e509a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 -- 172.21.15.26:0/1683376661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 msgr2=0x55f7e5095c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e5095c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 -- 172.21.15.26:0/1683376661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7e4f348d0 con 0x55f7e5096210 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e5095c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c87fff700 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e5099940 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5c780a6030 tx=0x7f5c78005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efe94f1fbd0ae76 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c867fc700 1 -- 172.21.15.26:0/1683376661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780a9ac0 con 0x55f7e5096210 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c867fc700 1 -- 172.21.15.26:0/1683376661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c780aa6b0 con 0x55f7e5096210 2024-05-20T21:09:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.440+0000 7f5c867fc700 1 -- 172.21.15.26:0/1683376661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780aa410 con 0x55f7e5096210 2024-05-20T21:09:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/1683376661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 msgr2=0x55f7e5099940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e5099940 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5c780a6030 tx=0x7f5c78005870 comp rx=0 tx=0).stop 2024-05-20T21:09:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/1683376661 shutdown_connections 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5099f60 0x55f7e509a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e5095c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/1683376661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e5099940 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/1683376661 >> 172.21.15.26:0/1683376661 conn(0x55f7e4fc3080 msgr2=0x55f7e508fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/1683376661 shutdown_connections 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.441+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/1683376661 wait complete. 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 -- start start 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 Processor -- start 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 0x55f7e50a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e509e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e509ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7e4eeafa0 con 0x55f7e5096210 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f7e4f867c0 con 0x55f7e5099f60 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7e50a1ee0 con 0x55f7e5095860 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.442+0000 7f5ca0c92b80 10 client.0 _ll_register_callbacks cb 0x55f7e4f7f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 0x55f7e50a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 0x55f7e50a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:53076/0 (socket says 172.21.15.26:53076) 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2422241956 learned_addr learned my addr 172.21.15.26:0/2422241956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e509e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c94999700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e509ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 msgr2=0x55f7e509ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e509ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2422241956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 msgr2=0x55f7e509e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e509e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2422241956 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7e4f83690 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e509e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 0x55f7e50a13d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c80003a50 tx=0x7f5c800047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7545fa12b533c308 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/2422241956 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8001fcd0 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.443+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c780aa6b0 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/2422241956 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5c800273e0 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/2422241956 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c80027620 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 10 client.28489 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f7e50ac5b0 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/2422241956 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c780aa6b0 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/2422241956 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5c80030020 con 0x55f7e5095860 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 10 client.28489 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 10 client.28489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 20 client.28489 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e50aeb40 0x55f7e50b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 4 client.28489 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7e50619e0 con 0x55f7e50aeb40 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.444+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e50aeb40 0x55f7e50b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.445+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e50aeb40 0x55f7e50b0f90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c9d346b7745f1a4 server_cookie=329081598ff6e789 in_seq=0 out_seq=0 2024-05-20T21:09:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.445+0000 7f5c6f7fe700 10 client.28489 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/2422241956 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f7e50619e0 con 0x55f7e50aeb40 2024-05-20T21:09:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5c6f7fe700 10 client.28489 handle_command_reply: tid=0 2024-05-20T21:09:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 client.28489 shutdown 2024-05-20T21:09:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 msgr2=0x55f7e50a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 0x55f7e50a13d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c80003a50 tx=0x7f5c800047c0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 shutdown_connections 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e50aeb40 0x55f7e50b0f90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e509ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5095860 0x55f7e50a13d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2422241956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5096210 0x55f7e509e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5c6f7fe700 0 client.28489 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 >> 172.21.15.26:0/2422241956 conn(0x55f7e4fc3080 msgr2=0x55f7e509b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 shutdown_connections 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2422241956 wait complete. 2024-05-20T21:09:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.446+0000 7f5ca0c92b80 20 client.28489 trim_cache size 0 max 16384 2024-05-20T21:09:12.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.478+0000 7f5ca0c92b80 1 Processor -- start 2024-05-20T21:09:12.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.478+0000 7f5ca0c92b80 1 -- start start 2024-05-20T21:09:12.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 0x55f7e5090470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e509d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e50a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901b0440 con 0x55f7e5095860 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901adcd0 con 0x55f7e5099f60 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901abb20 con 0x55f7e5096210 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 0x55f7e5090470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 0x55f7e5090470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56782/0 (socket says 172.21.15.26:56782) 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2236935525 learned_addr learned my addr 172.21.15.26:0/2236935525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c87fff700 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e509d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c94999700 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e50a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2236935525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 msgr2=0x55f7e509d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e509d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2236935525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 msgr2=0x55f7e50a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e50a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.479+0000 7f5c8ffff700 1 -- 172.21.15.26:0/2236935525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7e50b1610 con 0x55f7e5095860 2024-05-20T21:09:12.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.480+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 0x55f7e5090470 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f5c800094f0 tx=0x7f5c80005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf113f2eae5473d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.480+0000 7f5c85ffb700 1 -- 172.21.15.26:0/2236935525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80032940 con 0x55f7e5095860 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.480+0000 7f5c85ffb700 1 -- 172.21.15.26:0/2236935525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c80037070 con 0x55f7e5095860 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.480+0000 7f5c85ffb700 1 -- 172.21.15.26:0/2236935525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80027c20 con 0x55f7e5095860 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2236935525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 msgr2=0x55f7e5090470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 0x55f7e5090470 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f5c800094f0 tx=0x7f5c80005900 comp rx=0 tx=0).stop 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2236935525 shutdown_connections 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5095860 0x55f7e5090470 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5099f60 0x55f7e50a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/2236935525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e509d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2236935525 >> 172.21.15.26:0/2236935525 conn(0x55f7e4fc3080 msgr2=0x55f7e509ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2236935525 shutdown_connections 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/2236935525 wait complete. 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 -- start start 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.481+0000 7f5ca0c92b80 1 Processor -- start 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e50a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e5097120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 0x55f7e5097660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ac190 con 0x55f7e5099f60 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ad600 con 0x55f7e5095860 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ac860 con 0x55f7e5096210 2024-05-20T21:09:12.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 0x55f7e5097660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 0x55f7e5097660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56796/0 (socket says 172.21.15.26:56796) 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 -- 172.21.15.26:0/3405759096 learned_addr learned my addr 172.21.15.26:0/3405759096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e50a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5ca0c92b80 10 client.0 _ll_register_callbacks cb 0x55f7e4f7f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c87fff700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e5097120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 -- 172.21.15.26:0/3405759096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 msgr2=0x55f7e5097120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e5097120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 -- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 msgr2=0x55f7e50a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e50a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 -- 172.21.15.26:0/3405759096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7e5097ca0 con 0x55f7e5099f60 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e50a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c94999700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 0x55f7e5097660 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5c88003e70 tx=0x7f5c8800e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dd12bb7080171a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:12.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.482+0000 7f5c6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/3405759096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c880092d0 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/3405759096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5c88007500 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c80037070 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/3405759096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c88007740 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 10 client.19530 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f7e511d9d0 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/3405759096 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c88007d70 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/3405759096 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5c88029050 con 0x55f7e5099f60 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 10 client.19530 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 10 client.19530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 20 client.19530 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:12.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e511e0f0 0x55f7e5120510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 4 client.19530 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.483+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5c80037070 con 0x55f7e511e0f0 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e511e0f0 0x55f7e5120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5c8ffff700 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e511e0f0 0x55f7e5120510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30f92014a36e78f5 server_cookie=c14ce39538b25ed6 in_seq=0 out_seq=0 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5c6f7fe700 10 client.19530 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5c6f7fe700 1 -- 172.21.15.26:0/3405759096 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5c80037070 con 0x55f7e511e0f0 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5c6f7fe700 10 client.19530 handle_command_reply: tid=0 2024-05-20T21:09:12.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5ca0c92b80 1 client.19530 shutdown 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 msgr2=0x55f7e5097660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.484+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 0x55f7e5097660 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5c88003e70 tx=0x7f5c8800e7d0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 shutdown_connections 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55f7e511e0f0 0x55f7e5120510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7e5099f60 0x55f7e5097660 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5c6f7fe700 0 client.19530 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:12.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f7e5095860 0x55f7e50a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 --2- 172.21.15.26:0/3405759096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7e5096210 0x55f7e5097120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 >> 172.21.15.26:0/3405759096 conn(0x55f7e4fc3080 msgr2=0x55f7e5094f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:12.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 shutdown_connections 2024-05-20T21:09:12.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 1 -- 172.21.15.26:0/3405759096 wait complete. 2024-05-20T21:09:12.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5ca0c92b80 20 client.19530 trim_cache size 0 max 16384 2024-05-20T21:09:12.489 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c7c0b2880 msgr2=0x7f5c7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c7c0b2880 0x7f5c7c0b4d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5c9019aa70 tx=0x7f5c80009260 comp rx=0 tx=0).stop 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 msgr2=0x7f5c9019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.485+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 0x7f5c9019fcb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5c78006010 tx=0x7f5c78004270 comp rx=0 tx=0).stop 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 shutdown_connections 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c90109520 0x7f5c90199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c90101b30 0x7f5c9019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5c7c0b2880 0x7f5c7c0b4d40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 --2- 172.21.15.26:0/2345509755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c901024f0 0x7f5c9019fcb0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:12.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 >> 172.21.15.26:0/2345509755 conn(0x7f5c90078ab0 msgr2=0x7f5c901003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:12.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 shutdown_connections 2024-05-20T21:09:12.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:12.486+0000 7f5c963fc700 1 -- 172.21.15.26:0/2345509755 wait complete. 2024-05-20T21:09:12.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.21 MB/sec execute 558 sec latency 8805.159 ms 2024-05-20T21:09:13.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:09:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:13 smithi055 ceph-mon[35213]: pgmap v782: 97 pgs: 97 active+clean; 28 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 908 KiB/s wr, 58 op/s 2024-05-20T21:09:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:13 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:09:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:13 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:09:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:13 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:09:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:13 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:09:13.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:13 smithi026 ceph-mon[31990]: pgmap v782: 97 pgs: 97 active+clean; 28 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 908 KiB/s wr, 58 op/s 2024-05-20T21:09:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:09:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:09:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:09:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:09:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:13 smithi144 ceph-mon[35660]: pgmap v782: 97 pgs: 97 active+clean; 28 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 908 KiB/s wr, 58 op/s 2024-05-20T21:09:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:13 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:09:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:13 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:09:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:13 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:09:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:13 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:09:13.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.21 MB/sec execute 559 sec latency 9805.245 ms 2024-05-20T21:09:14.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.21 MB/sec execute 560 sec latency 10805.305 ms 2024-05-20T21:09:15.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:15 smithi026 ceph-mon[31990]: pgmap v783: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 625 KiB/s wr, 49 op/s 2024-05-20T21:09:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:15 smithi055 ceph-mon[35213]: pgmap v783: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 625 KiB/s wr, 49 op/s 2024-05-20T21:09:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:15 smithi144 ceph-mon[35660]: pgmap v783: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 625 KiB/s wr, 49 op/s 2024-05-20T21:09:15.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.20 MB/sec execute 561 sec latency 11805.427 ms 2024-05-20T21:09:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 -- 172.21.15.26:0/2076857371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 msgr2=0x7f20d8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 --2- 172.21.15.26:0/2076857371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d8103a60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b0d0 tx=0x7f20c800b3e0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 -- 172.21.15.26:0/2076857371 shutdown_connections 2024-05-20T21:09:16.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 --2- 172.21.15.26:0/2076857371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d8103fa0 0x7f20d8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 --2- 172.21.15.26:0/2076857371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d8109680 0x7f20d8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 --2- 172.21.15.26:0/2076857371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d8103a60 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 -- 172.21.15.26:0/2076857371 >> 172.21.15.26:0/2076857371 conn(0x7f20d807a6f0 msgr2=0x7f20d807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 -- 172.21.15.26:0/2076857371 shutdown_connections 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.454+0000 7f20def47700 1 -- 172.21.15.26:0/2076857371 wait complete. 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 Processor -- start 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 -- start start 2024-05-20T21:09:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d8103fa0 0x7f20d819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d8109680 0x7f20d81999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113690 con 0x7f20d81035e0 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113510 con 0x7f20d8109680 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20def47700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113810 con 0x7f20d8103fa0 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20dcce3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20dcce3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d819f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56812/0 (socket says 172.21.15.26:56812) 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.455+0000 7f20dcce3700 1 -- 172.21.15.26:0/3973203781 learned_addr learned my addr 172.21.15.26:0/3973203781 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d8103fa0 0x7f20d819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d8109680 0x7f20d81999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dcce3700 1 -- 172.21.15.26:0/3973203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d8103fa0 msgr2=0x7f20d819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dcce3700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d8103fa0 0x7f20d819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dcce3700 1 -- 172.21.15.26:0/3973203781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d8109680 msgr2=0x7f20d81999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dcce3700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d8109680 0x7f20d81999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dcce3700 1 -- 172.21.15.26:0/3973203781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d819a150 con 0x7f20d81035e0 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20dcce3700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d819f830 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d950 tx=0x7f20c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3973203781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0016070 con 0x7f20d81035e0 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d819a440 con 0x7f20d81035e0 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3973203781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c0009dd0 con 0x7f20d81035e0 2024-05-20T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3973203781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c000f3c0 con 0x7f20d81035e0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.456+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d81a6700 con 0x7f20d81035e0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.458+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3973203781 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f20c0019070 con 0x7f20d81035e0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.459+0000 7f20d5ffb700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20c40b2880 0x7f20c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3973203781 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20c00e1ee0 con 0x7f20d81035e0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20c40b2880 0x7f20c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 Processor -- start 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20c40b2880 0x7f20c40b4d40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f20d807ada0 tx=0x7f20c800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 -- start start 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a75700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5a7c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5585f59cdd70 con 0x5585f5a75320 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5585f59bd910 con 0x5585f5a6f8a0 2024-05-20T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.460+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5585f59c1950 con 0x5585f5a70260 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a75700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a75700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56828/0 (socket says 172.21.15.26:56828) 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 -- 172.21.15.26:0/542711351 learned_addr learned my addr 172.21.15.26:0/542711351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5a7c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20d7fff700 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 -- 172.21.15.26:0/542711351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 msgr2=0x5585f5a7c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5a7c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 -- 172.21.15.26:0/542711351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 msgr2=0x5585f5a6fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 -- 172.21.15.26:0/542711351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5966fb0 con 0x5585f5a75320 2024-05-20T21:09:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20dcce3700 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a75700 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f20c00a9430 tx=0x7f20c00a5490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8b6b4ad83839dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20ceffd700 1 -- 172.21.15.26:0/542711351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0020070 con 0x5585f5a75320 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20ceffd700 1 -- 172.21.15.26:0/542711351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c00910b0 con 0x5585f5a75320 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.461+0000 7f20ceffd700 1 -- 172.21.15.26:0/542711351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00b0e30 con 0x5585f5a75320 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 -- 172.21.15.26:0/542711351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 msgr2=0x5585f5a75700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a75700 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f20c00a9430 tx=0x7f20c00a5490 comp rx=0 tx=0).stop 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 -- 172.21.15.26:0/542711351 shutdown_connections 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a75700 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5a7c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/542711351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 -- 172.21.15.26:0/542711351 >> 172.21.15.26:0/542711351 conn(0x5585f599b240 msgr2=0x5585f5a6a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 -- 172.21.15.26:0/542711351 shutdown_connections 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.462+0000 7f20e97ddb80 1 -- 172.21.15.26:0/542711351 wait complete. 2024-05-20T21:09:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 -- start start 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 Processor -- start 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 0x5585f5a7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a70260 0x5585f5a79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a75320 0x5585f5a79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5585f58c3fa0 con 0x5585f5a6f8a0 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5585f5925da0 con 0x5585f5a70260 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5585f5a7c680 con 0x5585f5a75320 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20e97ddb80 10 client.0 _ll_register_callbacks cb 0x5585f5958950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20dcce3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 0x5585f5a7bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20dcce3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 0x5585f5a7bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56844/0 (socket says 172.21.15.26:56844) 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.463+0000 7f20dcce3700 1 -- 172.21.15.26:0/3406326515 learned_addr learned my addr 172.21.15.26:0/3406326515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a75320 0x5585f5a79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a70260 0x5585f5a79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dcce3700 1 -- 172.21.15.26:0/3406326515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a75320 msgr2=0x5585f5a79580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dcce3700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a75320 0x5585f5a79580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dcce3700 1 -- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a70260 msgr2=0x5585f5a79040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dcce3700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a70260 0x5585f5a79040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dcce3700 1 -- 172.21.15.26:0/3406326515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f590d8d0 con 0x5585f5a6f8a0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a75320 0x5585f5a79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a70260 0x5585f5a79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20dcce3700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 0x5585f5a7bd40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f20c00dfcf0 tx=0x7f20c00a95d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8e5f97d01630c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20b7fff700 1 -- 172.21.15.26:0/3406326515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00c2020 con 0x5585f5a6f8a0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20b7fff700 1 -- 172.21.15.26:0/3406326515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20c00b0bd0 con 0x5585f5a6f8a0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c00910b0 con 0x5585f5a6f8a0 2024-05-20T21:09:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20b7fff700 1 -- 172.21.15.26:0/3406326515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f20c0020070 con 0x5585f5a6f8a0 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20e97ddb80 10 client.19542 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.464+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585f5977760 con 0x5585f5a6f8a0 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20b7fff700 1 -- 172.21.15.26:0/3406326515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c00910b0 con 0x5585f5a6f8a0 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20b7fff700 1 -- 172.21.15.26:0/3406326515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f20c00b1f00 con 0x5585f5a6f8a0 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20e97ddb80 10 client.19542 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20e97ddb80 10 client.19542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20e97ddb80 20 client.19542 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5a89070 0x5585f5a8b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20e97ddb80 4 client.19542 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f59cfcc0 con 0x5585f5a89070 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.465+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5a89070 0x5585f5a8b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.466+0000 7f20d7fff700 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5a89070 0x5585f5a8b460 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=289d994be300e0ee server_cookie=d9c8db033c258965 in_seq=0 out_seq=0 2024-05-20T21:09:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.466+0000 7f20b7fff700 10 client.19542 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.466+0000 7f20b7fff700 1 -- 172.21.15.26:0/3406326515 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5585f59cfcc0 con 0x5585f5a89070 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.466+0000 7f20b7fff700 10 client.19542 handle_command_reply: tid=0 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.466+0000 7f20e97ddb80 1 client.19542 shutdown 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 msgr2=0x5585f5a7bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 0x5585f5a7bd40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f20c00dfcf0 tx=0x7f20c00a95d0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 shutdown_connections 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5a89070 0x5585f5a8b460 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a6f8a0 0x5585f5a7bd40 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a75320 0x5585f5a79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/3406326515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a70260 0x5585f5a79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 >> 172.21.15.26:0/3406326515 conn(0x5585f599b240 msgr2=0x5585f5a69af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20b7fff700 0 client.19542 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 shutdown_connections 2024-05-20T21:09:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 1 -- 172.21.15.26:0/3406326515 wait complete. 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.467+0000 7f20e97ddb80 20 client.19542 trim_cache size 0 max 16384 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.482+0000 7f20e97ddb80 1 Processor -- start 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.482+0000 7f20e97ddb80 1 -- start start 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.482+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.482+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 0x5585f5a72d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.482+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a75320 0x5585f5a73290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.482+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81b05c0 con 0x5585f5a70260 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ade50 con 0x5585f5a75320 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81abca0 con 0x5585f5a6f8a0 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 0x5585f5a72d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 0x5585f5a72d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56848/0 (socket says 172.21.15.26:56848) 2024-05-20T21:09:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 -- 172.21.15.26:0/1291161050 learned_addr learned my addr 172.21.15.26:0/1291161050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20dcce3700 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 -- 172.21.15.26:0/1291161050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a6f8a0 msgr2=0x5585f5a6a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 -- 172.21.15.26:0/1291161050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a75320 msgr2=0x5585f5a73290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a75320 0x5585f5a73290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 -- 172.21.15.26:0/1291161050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5a7b980 con 0x5585f5a70260 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20d7fff700 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 0x5585f5a72d50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b010 tx=0x7f20c801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5dfc788b607955 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20ce7fc700 1 -- 172.21.15.26:0/1291161050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8007970 con 0x5585f5a70260 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20ce7fc700 1 -- 172.21.15.26:0/1291161050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c8007b10 con 0x5585f5a70260 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.483+0000 7f20ce7fc700 1 -- 172.21.15.26:0/1291161050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8030af0 con 0x5585f5a70260 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 -- 172.21.15.26:0/1291161050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 msgr2=0x5585f5a72d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 0x5585f5a72d50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b010 tx=0x7f20c801e000 comp rx=0 tx=0).stop 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 -- 172.21.15.26:0/1291161050 shutdown_connections 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a75320 0x5585f5a73290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a6f8a0 0x5585f5a6a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/1291161050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a70260 0x5585f5a72d50 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 -- 172.21.15.26:0/1291161050 >> 172.21.15.26:0/1291161050 conn(0x5585f599b240 msgr2=0x5585f5a77950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 -- 172.21.15.26:0/1291161050 shutdown_connections 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 -- 172.21.15.26:0/1291161050 wait complete. 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 -- start start 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.484+0000 7f20e97ddb80 1 Processor -- start 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5aeed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5aef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a8d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ac310 con 0x5585f5a75320 2024-05-20T21:09:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ad780 con 0x5585f5a6f8a0 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ac9e0 con 0x5585f5a70260 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20e97ddb80 10 client.0 _ll_register_callbacks cb 0x5585f5958950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a8d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a8d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56860/0 (socket says 172.21.15.26:56860) 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 -- 172.21.15.26:0/2426502885 learned_addr learned my addr 172.21.15.26:0/2426502885 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dcce3700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5aeed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20d7fff700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5aef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 -- 172.21.15.26:0/2426502885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 msgr2=0x5585f5aef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5aef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 -- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 msgr2=0x5585f5aeed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5aeed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dd4e4700 1 -- 172.21.15.26:0/2426502885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5a8e0e0 con 0x5585f5a75320 2024-05-20T21:09:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20d7fff700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5aef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.485+0000 7f20dcce3700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5aeed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20dd4e4700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a8d8a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f20d0012550 tx=0x7f20d0012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677d0625644aa9bd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20b7fff700 1 -- 172.21.15.26:0/2426502885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d001b070 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20b7fff700 1 -- 172.21.15.26:0/2426502885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20d0015450 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c8007b10 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20b7fff700 1 -- 172.21.15.26:0/2426502885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f20d0026050 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 10 client.19548 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585f5af7680 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20b7fff700 1 -- 172.21.15.26:0/2426502885 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c8007b10 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20b7fff700 1 -- 172.21.15.26:0/2426502885 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f20d0014640 con 0x5585f5a75320 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 10 client.19548 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 10 client.19548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 20 client.19548 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5af7dc0 0x5585f5afa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 4 client.19548 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.486+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585f59c94a0 con 0x5585f5af7dc0 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.487+0000 7f20dcce3700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5af7dc0 0x5585f5afa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.487+0000 7f20dcce3700 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5af7dc0 0x5585f5afa1e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85756d562ae6f14c server_cookie=360e07bace03a7cd in_seq=0 out_seq=0 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.487+0000 7f20b7fff700 10 client.19548 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.487+0000 7f20b7fff700 1 -- 172.21.15.26:0/2426502885 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585f59c94a0 con 0x5585f5af7dc0 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.487+0000 7f20b7fff700 10 client.19548 handle_command_reply: tid=0 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 client.19548 shutdown 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 msgr2=0x5585f5a8d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a8d8a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f20d0012550 tx=0x7f20d0012860 comp rx=0 tx=0).stop 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 shutdown_connections 2024-05-20T21:09:16.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5585f5af7dc0 0x5585f5afa1e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585f5a75320 0x5585f5a8d8a0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5585f5a6f8a0 0x5585f5aeed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 --2- 172.21.15.26:0/2426502885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585f5a70260 0x5585f5aef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 >> 172.21.15.26:0/2426502885 conn(0x5585f599b240 msgr2=0x5585f5a72d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20b7fff700 0 client.19548 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 shutdown_connections 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 1 -- 172.21.15.26:0/2426502885 wait complete. 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.488+0000 7f20e97ddb80 20 client.19548 trim_cache size 0 max 16384 2024-05-20T21:09:16.526 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20c40b2880 msgr2=0x7f20c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20c40b2880 0x7f20c40b4d40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f20d807ada0 tx=0x7f20c800b040 comp rx=0 tx=0).stop 2024-05-20T21:09:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 msgr2=0x7f20d819f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d819f830 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d950 tx=0x7f20c000dc60 comp rx=0 tx=0).stop 2024-05-20T21:09:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 shutdown_connections 2024-05-20T21:09:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20c40b2880 0x7f20c40b4d40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d81035e0 0x7f20d819f830 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d8109680 0x7f20d81999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 --2- 172.21.15.26:0/3973203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d8103fa0 0x7f20d819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 >> 172.21.15.26:0/3973203781 conn(0x7f20d807a6f0 msgr2=0x7f20d81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 shutdown_connections 2024-05-20T21:09:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:16.489+0000 7f20def47700 1 -- 172.21.15.26:0/3973203781 wait complete. 2024-05-20T21:09:16.594 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.20 MB/sec execute 562 sec latency 12805.490 ms 2024-05-20T21:09:17.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:17 smithi026 ceph-mon[31990]: pgmap v784: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 360 KiB/s wr, 41 op/s 2024-05-20T21:09:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:17 smithi055 ceph-mon[35213]: pgmap v784: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 360 KiB/s wr, 41 op/s 2024-05-20T21:09:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:17 smithi144 ceph-mon[35660]: pgmap v784: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 360 KiB/s wr, 41 op/s 2024-05-20T21:09:17.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.19 MB/sec execute 563 sec latency 13805.536 ms 2024-05-20T21:09:18.313 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:09:18.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:09:18.579 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:18.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.19 MB/sec execute 564 sec latency 14805.576 ms 2024-05-20T21:09:19.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:19 smithi055 ceph-mon[35213]: pgmap v785: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-05-20T21:09:19.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:19 smithi026 ceph-mon[31990]: pgmap v785: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-05-20T21:09:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:19 smithi144 ceph-mon[35660]: pgmap v785: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-05-20T21:09:19.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.19 MB/sec execute 565 sec latency 15805.659 ms 2024-05-20T21:09:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 -- 172.21.15.26:0/2385430428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 msgr2=0x7f5a4c112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/2385430428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c112410 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b0a0 tx=0x7f5a4400b3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 -- 172.21.15.26:0/2385430428 shutdown_connections 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/2385430428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c112410 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/2385430428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06bc90 0x7f5a4c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/2385430428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4c06c5b0 0x7f5a4c10fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 -- 172.21.15.26:0/2385430428 >> 172.21.15.26:0/2385430428 conn(0x7f5a4c079410 msgr2=0x7f5a4c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 -- 172.21.15.26:0/2385430428 shutdown_connections 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.434+0000 7f5a50eaf700 1 -- 172.21.15.26:0/2385430428 wait complete. 2024-05-20T21:09:20.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 Processor -- start 2024-05-20T21:09:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 -- start start 2024-05-20T21:09:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4c06bc90 0x7f5a4c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06c5b0 0x7f5a4c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1157a0 con 0x7f5a4c110020 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c115620 con 0x7f5a4c06bc90 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a50eaf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c115920 con 0x7f5a4c06c5b0 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a4ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a4ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c19bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60214/0 (socket says 172.21.15.26:60214) 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/3412660922 learned_addr learned my addr 172.21.15.26:0/3412660922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4c06bc90 0x7f5a4c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.435+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06c5b0 0x7f5a4c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/3412660922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06c5b0 msgr2=0x7f5a4c1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06c5b0 0x7f5a4c1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/3412660922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4c06bc90 msgr2=0x7f5a4c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4c06bc90 0x7f5a4c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/3412660922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c19c220 con 0x7f5a4c110020 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06c5b0 0x7f5a4c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c19bac0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b830 tx=0x7f5a44004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a43fff700 1 -- 172.21.15.26:0/3412660922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44021070 con 0x7f5a4c110020 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a43fff700 1 -- 172.21.15.26:0/3412660922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a44007b80 con 0x7f5a4c110020 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c19c4b0 con 0x7f5a4c110020 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a43fff700 1 -- 172.21.15.26:0/3412660922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440052a0 con 0x7f5a4c110020 2024-05-20T21:09:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.436+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1a87d0 con 0x7f5a4c110020 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.439+0000 7f5a43fff700 1 -- 172.21.15.26:0/3412660922 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5a44024070 con 0x7f5a4c110020 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.440+0000 7f5a43fff700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a380b2880 0x7f5a380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.440+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a380b2880 0x7f5a380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.440+0000 7f5a43fff700 1 -- 172.21.15.26:0/3412660922 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5a4400f020 con 0x7f5a4c110020 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.440+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a380b2880 0x7f5a380b4d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c19cc00 tx=0x7f5a3c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.440+0000 7f5a5b745b80 1 Processor -- start 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 -- start start 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d75048e0 0x5604d7504cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d75089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7508fe0 0x5604d7509420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5604d7463d10 con 0x5604d7505290 2024-05-20T21:09:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5604d74538b0 con 0x5604d7508fe0 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604d74578f0 con 0x5604d75048e0 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d75089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d75089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60224/0 (socket says 172.21.15.26:60224) 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 -- 172.21.15.26:0/3795578918 learned_addr learned my addr 172.21.15.26:0/3795578918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7508fe0 0x5604d7509420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d75048e0 0x5604d7504cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 -- 172.21.15.26:0/3795578918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d75048e0 msgr2=0x5604d7504cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d75048e0 0x5604d7504cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 -- 172.21.15.26:0/3795578918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7508fe0 msgr2=0x5604d7509420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.441+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7508fe0 0x5604d7509420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.442+0000 7f5a41d9b700 1 -- 172.21.15.26:0/3795578918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d73a3950 con 0x5604d7505290 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.442+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7508fe0 0x5604d7509420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.442+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d75089c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5a34008c50 tx=0x7f5a34008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902b4f69d62a9211 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.442+0000 7f5a40d99700 1 -- 172.21.15.26:0/3795578918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34014070 con 0x5604d7505290 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.442+0000 7f5a40d99700 1 -- 172.21.15.26:0/3795578918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a3400edf0 con 0x5604d7505290 2024-05-20T21:09:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.442+0000 7f5a40d99700 1 -- 172.21.15.26:0/3795578918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34010430 con 0x5604d7505290 2024-05-20T21:09:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 -- 172.21.15.26:0/3795578918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 msgr2=0x5604d75089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d75089c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5a34008c50 tx=0x7f5a34008f60 comp rx=0 tx=0).stop 2024-05-20T21:09:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 -- 172.21.15.26:0/3795578918 shutdown_connections 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7508fe0 0x5604d7509420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d75048e0 0x5604d7504cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/3795578918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d75089c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 -- 172.21.15.26:0/3795578918 >> 172.21.15.26:0/3795578918 conn(0x5604d7432100 msgr2=0x5604d74fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 -- 172.21.15.26:0/3795578918 shutdown_connections 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 -- 172.21.15.26:0/3795578918 wait complete. 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 -- start start 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.443+0000 7f5a5b745b80 1 Processor -- start 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 0x5604d7510410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7505290 0x5604d750d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d750dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5604d735a020 con 0x5604d75048e0 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5604d73f5840 con 0x5604d7505290 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604d75111a0 con 0x5604d7508fe0 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a5b745b80 10 client.0 _ll_register_callbacks cb 0x5604d73ee9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 0x5604d7510410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 0x5604d7510410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60232/0 (socket says 172.21.15.26:60232) 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 -- 172.21.15.26:0/4166992427 learned_addr learned my addr 172.21.15.26:0/4166992427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d750dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7505290 0x5604d750d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 -- 172.21.15.26:0/4166992427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 msgr2=0x5604d750dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d750dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 -- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7505290 msgr2=0x5604d750d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7505290 0x5604d750d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 -- 172.21.15.26:0/4166992427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d73f2710 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d750dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 0x5604d7510410 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c003df0 tx=0x7f5a3c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d57c1d823942b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.444+0000 7f5a31ffb700 1 -- 172.21.15.26:0/4166992427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c028670 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3400edf0 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a31ffb700 1 -- 172.21.15.26:0/4166992427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5a3c028810 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a31ffb700 1 -- 172.21.15.26:0/4166992427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c028a10 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 10 client.19560 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604d751b930 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a31ffb700 1 -- 172.21.15.26:0/4166992427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a3400edf0 con 0x5604d75048e0 2024-05-20T21:09:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a31ffb700 1 -- 172.21.15.26:0/4166992427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5a3c031050 con 0x5604d75048e0 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 10 client.19560 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 20 client.19560 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d75203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 4 client.19560 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d74d0a60 con 0x5604d751dfc0 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.445+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d75203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.446+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d75203f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5249c1fc079761d9 server_cookie=6f52f533a175f5d2 in_seq=0 out_seq=0 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.446+0000 7f5a31ffb700 10 client.19560 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.446+0000 7f5a31ffb700 1 -- 172.21.15.26:0/4166992427 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5604d74d0a60 con 0x5604d751dfc0 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.446+0000 7f5a31ffb700 10 client.19560 handle_command_reply: tid=0 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 client.19560 shutdown 2024-05-20T21:09:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 msgr2=0x5604d7510410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 0x5604d7510410 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c003df0 tx=0x7f5a3c0047c0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 shutdown_connections 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d75203f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d75048e0 0x5604d7510410 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d750dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a31ffb700 0 client.19560 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/4166992427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d7505290 0x5604d750d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 >> 172.21.15.26:0/4166992427 conn(0x5604d7432100 msgr2=0x5604d74fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 shutdown_connections 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 1 -- 172.21.15.26:0/4166992427 wait complete. 2024-05-20T21:09:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.447+0000 7f5a5b745b80 20 client.19560 trim_cache size 0 max 16384 2024-05-20T21:09:20.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 Processor -- start 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 -- start start 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d750f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d74fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d7525d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1b2620 con 0x5604d7505290 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1afeb0 con 0x5604d75048e0 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1add00 con 0x5604d7508fe0 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d74fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d74fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60234/0 (socket says 172.21.15.26:60234) 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 -- 172.21.15.26:0/259378463 learned_addr learned my addr 172.21.15.26:0/259378463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 -- 172.21.15.26:0/259378463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 msgr2=0x5604d7525d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d7525d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d750f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 -- 172.21.15.26:0/259378463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 msgr2=0x5604d750f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d750f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.468+0000 7f5a41d9b700 1 -- 172.21.15.26:0/259378463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d74ffdc0 con 0x5604d7505290 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d74fed10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009010 tx=0x7f5a340053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eca64f9382da054 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a33fff700 1 -- 172.21.15.26:0/259378463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34014040 con 0x5604d7505290 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a33fff700 1 -- 172.21.15.26:0/259378463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a3400ed10 con 0x5604d7505290 2024-05-20T21:09:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a33fff700 1 -- 172.21.15.26:0/259378463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340236b0 con 0x5604d7505290 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 -- 172.21.15.26:0/259378463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 msgr2=0x5604d74fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d74fed10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009010 tx=0x7f5a340053f0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 -- 172.21.15.26:0/259378463 shutdown_connections 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7508fe0 0x5604d7525d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d750f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/259378463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7505290 0x5604d74fed10 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 -- 172.21.15.26:0/259378463 >> 172.21.15.26:0/259378463 conn(0x5604d7432100 msgr2=0x5604d750ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 -- 172.21.15.26:0/259378463 shutdown_connections 2024-05-20T21:09:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.469+0000 7f5a5b745b80 1 -- 172.21.15.26:0/259378463 wait complete. 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 -- start start 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 Processor -- start 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d7516bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7505290 0x5604d7514fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 0x5604d7515620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ae370 con 0x5604d7508fe0 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1af7e0 con 0x5604d75048e0 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1aea40 con 0x5604d7505290 2024-05-20T21:09:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a5b745b80 10 client.0 _ll_register_callbacks cb 0x5604d73ee9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a4ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 0x5604d7515620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a4ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 0x5604d7515620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60236/0 (socket says 172.21.15.26:60236) 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/238778882 learned_addr learned my addr 172.21.15.26:0/238778882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a41d9b700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7505290 0x5604d7514fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.470+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d7516bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/238778882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7505290 msgr2=0x5604d7514fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7505290 0x5604d7514fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 msgr2=0x5604d7516bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d7516bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a4ad9d700 1 -- 172.21.15.26:0/238778882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d7588bf0 con 0x5604d7508fe0 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a4ad9d700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 0x5604d7515620 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5a440aac00 tx=0x7f5a440aa760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a9bcef68c191d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a31ffb700 1 -- 172.21.15.26:0/238778882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440ab600 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a31ffb700 1 -- 172.21.15.26:0/238778882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5a44007d30 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3400ed10 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a31ffb700 1 -- 172.21.15.26:0/238778882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a440aaf00 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a5b745b80 10 client.19566 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604d7589e10 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a31ffb700 1 -- 172.21.15.26:0/238778882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a3400ed10 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.471+0000 7f5a31ffb700 1 -- 172.21.15.26:0/238778882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5a44011070 con 0x5604d7508fe0 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a5b745b80 10 client.19566 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a5b745b80 10 client.19566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a5b745b80 20 client.19566 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d758c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a5b745b80 4 client.19566 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:09:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5604d7465c60 con 0x5604d751dfc0 2024-05-20T21:09:20.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d758c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:20.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a4a59c700 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d758c5b0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce5a1f95c44e482 server_cookie=37ca199c8b873943 in_seq=0 out_seq=0 2024-05-20T21:09:20.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.472+0000 7f5a31ffb700 10 client.19566 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.473+0000 7f5a31ffb700 1 -- 172.21.15.26:0/238778882 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5604d7465c60 con 0x5604d751dfc0 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.473+0000 7f5a31ffb700 10 client.19566 handle_command_reply: tid=0 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.473+0000 7f5a5b745b80 1 client.19566 shutdown 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 msgr2=0x5604d7515620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 0x5604d7515620 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5a440aac00 tx=0x7f5a440aa760 comp rx=0 tx=0).stop 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 shutdown_connections 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5604d751dfc0 0x5604d758c5b0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604d7508fe0 0x5604d7515620 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a31ffb700 0 client.19566 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d75048e0 0x5604d7516bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 --2- 172.21.15.26:0/238778882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604d7505290 0x5604d7514fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 >> 172.21.15.26:0/238778882 conn(0x5604d7432100 msgr2=0x5604d750f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 shutdown_connections 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 1 -- 172.21.15.26:0/238778882 wait complete. 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a5b745b80 20 client.19566 trim_cache size 0 max 16384 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "58c04fa7-2395-4786-90c9-4de29a3f46d7", 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:09:20.478 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a380b2880 msgr2=0x7f5a380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a380b2880 0x7f5a380b4d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c19cc00 tx=0x7f5a3c00b3c0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 msgr2=0x7f5a4c19bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.474+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c19bac0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b830 tx=0x7f5a44004d00 comp rx=0 tx=0).stop 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 shutdown_connections 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f5a380b2880 0x7f5a380b4d40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4c110020 0x7f5a4c19bac0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a4c06bc90 0x7f5a4c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 --2- 172.21.15.26:0/3412660922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a4c06c5b0 0x7f5a4c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:20.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 >> 172.21.15.26:0/3412660922 conn(0x7f5a4c079410 msgr2=0x7f5a4c1114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:20.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 shutdown_connections 2024-05-20T21:09:20.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:20.475+0000 7f5a50eaf700 1 -- 172.21.15.26:0/3412660922 wait complete. 2024-05-20T21:09:20.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.18 MB/sec execute 566 sec latency 16805.732 ms 2024-05-20T21:09:21.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:21 smithi026 ceph-mon[31990]: pgmap v786: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-20T21:09:21.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:09:21.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:21 smithi144 ceph-mon[35660]: pgmap v786: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 -- 172.21.15.26:0/3699918658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704103560 msgr2=0x7f3704109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 --2- 172.21.15.26:0/3699918658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704103560 0x7f3704109df0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009900 tx=0x7f36fc02a410 comp rx=0 tx=0).stop 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 -- 172.21.15.26:0/3699918658 shutdown_connections 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 --2- 172.21.15.26:0/3699918658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704103560 0x7f3704109df0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 --2- 172.21.15.26:0/3699918658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3704102ba0 0x7f3704103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 --2- 172.21.15.26:0/3699918658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37041019b0 0x7f3704101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 -- 172.21.15.26:0/3699918658 >> 172.21.15.26:0/3699918658 conn(0x7f37040fcf20 msgr2=0x7f37040ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 -- 172.21.15.26:0/3699918658 shutdown_connections 2024-05-20T21:09:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.366+0000 7f370c045640 1 -- 172.21.15.26:0/3699918658 wait complete. 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 Processor -- start 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 -- start start 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37041019b0 0x7f3704198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 0x7f3704199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3704103560 0x7f37041a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f370410d390 con 0x7f3704102ba0 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f370410d210 con 0x7f3704103560 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370c045640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f370410d510 con 0x7f37041019b0 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 0x7f3704199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 0x7f3704199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60260/0 (socket says 172.21.15.26:60260) 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 -- 172.21.15.26:0/3449376468 learned_addr learned my addr 172.21.15.26:0/3449376468 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f3709de1640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37041019b0 0x7f3704198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370a5e2640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3704103560 0x7f37041a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 -- 172.21.15.26:0/3449376468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37041019b0 msgr2=0x7f3704198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37041019b0 0x7f3704198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 -- 172.21.15.26:0/3449376468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3704103560 msgr2=0x7f37041a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3704103560 0x7f37041a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f37095e0640 1 -- 172.21.15.26:0/3449376468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37041a0740 con 0x7f3704102ba0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.367+0000 7f370a5e2640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3704103560 0x7f37041a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.368+0000 7f37095e0640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 0x7f3704199060 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f36f400d770 tx=0x7f36f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.368+0000 7f36faffd640 1 -- 172.21.15.26:0/3449376468 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4017070 con 0x7f3704102ba0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.368+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37041a0a30 con 0x7f3704102ba0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.368+0000 7f36faffd640 1 -- 172.21.15.26:0/3449376468 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f40054b0 con 0x7f3704102ba0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.368+0000 7f36faffd640 1 -- 172.21.15.26:0/3449376468 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4004c60 con 0x7f3704102ba0 2024-05-20T21:09:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.368+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37041a0f30 con 0x7f3704102ba0 2024-05-20T21:09:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 Processor -- start 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 -- start start 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e796c0 0x560d52e79ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e7ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f36faffd640 1 -- 172.21.15.26:0/3449376468 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f36f4010460 con 0x7f3704102ba0 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d52deac30 con 0x560d52e7f220 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560d52dda7d0 con 0x560d52e7a0b0 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.371+0000 7f370e9fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560d52dde810 con 0x560d52e796c0 2024-05-20T21:09:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e816b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60270/0 (socket says 172.21.15.26:60270) 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 -- 172.21.15.26:0/3678218656 learned_addr learned my addr 172.21.15.26:0/3678218656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f370a5e2640 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e796c0 0x560d52e79ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f3709de1640 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e7ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 -- 172.21.15.26:0/3678218656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e796c0 msgr2=0x560d52e79ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e796c0 0x560d52e79ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 -- 172.21.15.26:0/3678218656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 msgr2=0x560d52e7ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e7ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 -- 172.21.15.26:0/3678218656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d52aef210 con 0x560d52e7f220 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f37095e0640 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e816b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f36f40e5ee0 tx=0x7f36f40ea810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d9e6c84a17c105 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f36effff640 1 -- 172.21.15.26:0/3678218656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f40f7070 con 0x560d52e7f220 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.372+0000 7f36faffd640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f36f00b28a0 0x7f36f00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f36effff640 1 -- 172.21.15.26:0/3678218656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f40a8e10 con 0x560d52e7f220 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f36effff640 1 -- 172.21.15.26:0/3678218656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f40a8a70 con 0x560d52e7f220 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370a5e2640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f36f00b28a0 0x7f36f00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f36faffd640 1 -- 172.21.15.26:0/3449376468 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f36f40e1d40 con 0x7f3704102ba0 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 -- 172.21.15.26:0/3678218656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 msgr2=0x560d52e816b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370a5e2640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f36f00b28a0 0x7f36f00b4d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f36fc005c40 tx=0x7f36fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e816b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f36f40e5ee0 tx=0x7f36f40ea810 comp rx=0 tx=0).stop 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 -- 172.21.15.26:0/3678218656 shutdown_connections 2024-05-20T21:09:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e816b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e7ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 --2- 172.21.15.26:0/3678218656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e796c0 0x560d52e79ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.373+0000 7f370e9fd740 1 -- 172.21.15.26:0/3678218656 >> 172.21.15.26:0/3678218656 conn(0x560d52e74d30 msgr2=0x560d52e75130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 -- 172.21.15.26:0/3678218656 shutdown_connections 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 -- 172.21.15.26:0/3678218656 wait complete. 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 -- start start 2024-05-20T21:09:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 Processor -- start 2024-05-20T21:09:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e83810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7a0b0 0x560d52e83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d52ac0790 con 0x560d52e7f220 2024-05-20T21:09:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560d52d96890 con 0x560d52e796c0 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560d52e821d0 con 0x560d52e7a0b0 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.374+0000 7f370e9fd740 10 client.0 _ll_register_callbacks cb 0x560d52aeea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e8ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60280/0 (socket says 172.21.15.26:60280) 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f3709de1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e83810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 -- 172.21.15.26:0/2069129343 learned_addr learned my addr 172.21.15.26:0/2069129343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f37095e0640 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7a0b0 0x560d52e83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 -- 172.21.15.26:0/2069129343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7a0b0 msgr2=0x560d52e83d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7a0b0 0x560d52e83d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 -- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 msgr2=0x560d52e83810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e83810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 -- 172.21.15.26:0/2069129343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d52bbb620 con 0x560d52e7f220 2024-05-20T21:09:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370a5e2640 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e8ad90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f36fc033000 tx=0x7f36fc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f842f59c094c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f36ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2069129343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc04d070 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f40a8e10 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.375+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2069129343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f36fc0035f0 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2069129343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36fc059050 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 10 client.19578 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d52e8dde0 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2069129343 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f40a8e10 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2069129343 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f36fc03a040 con 0x560d52e7f220 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 10 client.19578 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 10 client.19578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 20 client.19578 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52e8f4f0 0x560d52e91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 4 client.19578 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d52decb80 con 0x560d52e8f4f0 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.376+0000 7f3709de1640 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52e8f4f0 0x560d52e91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.377+0000 7f3709de1640 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52e8f4f0 0x560d52e91980 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1816f756b76d044 server_cookie=5d43a9fb958c7d1c in_seq=0 out_seq=0 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.377+0000 7f36ed7fa640 10 client.19578 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.377+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2069129343 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560d52decb80 con 0x560d52e8f4f0 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.377+0000 7f36ed7fa640 10 client.19578 handle_command_reply: tid=0 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.377+0000 7f370e9fd740 1 client.19578 shutdown 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 msgr2=0x560d52e8ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e8ad90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f36fc033000 tx=0x7f36fc02a410 comp rx=0 tx=0).stop 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 shutdown_connections 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52e8f4f0 0x560d52e91980 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7f220 0x560d52e8ad90 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7a0b0 0x560d52e83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f36ed7fa640 0 client.19578 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2069129343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e83810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 >> 172.21.15.26:0/2069129343 conn(0x560d52e74d30 msgr2=0x560d52e73fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 shutdown_connections 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 1 -- 172.21.15.26:0/2069129343 wait complete. 2024-05-20T21:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.378+0000 7f370e9fd740 20 client.19578 trim_cache size 0 max 16384 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 Processor -- start 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 -- start start 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 0x560d52e6fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e8ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37041ac7d0 con 0x560d52e796c0 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37041aa0a0 con 0x560d52e7a0b0 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f370e9fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37041a7ef0 con 0x560d52e7f220 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f3709de1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 0x560d52e6fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f3709de1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 0x560d52e6fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60284/0 (socket says 172.21.15.26:60284) 2024-05-20T21:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.393+0000 7f3709de1640 1 -- 172.21.15.26:0/649050975 learned_addr learned my addr 172.21.15.26:0/649050975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f370a5e2640 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f37095e0640 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e8ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f3709de1640 1 -- 172.21.15.26:0/649050975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 msgr2=0x560d52e77c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f3709de1640 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e77c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f3709de1640 1 -- 172.21.15.26:0/649050975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 msgr2=0x560d52e8ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f3709de1640 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e8ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f3709de1640 1 -- 172.21.15.26:0/649050975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d52e74b60 con 0x560d52e796c0 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f370a5e2640 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f3709de1640 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 0x560d52e6fe40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f37000096b0 tx=0x7f3700009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4ccc9466e58a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f36ef7fe640 1 -- 172.21.15.26:0/649050975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37000142d0 con 0x560d52e796c0 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f36ef7fe640 1 -- 172.21.15.26:0/649050975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3700003e00 con 0x560d52e796c0 2024-05-20T21:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.394+0000 7f36ef7fe640 1 -- 172.21.15.26:0/649050975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700012690 con 0x560d52e796c0 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 -- 172.21.15.26:0/649050975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 msgr2=0x560d52e6fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 0x560d52e6fe40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f37000096b0 tx=0x7f3700009840 comp rx=0 tx=0).stop 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 -- 172.21.15.26:0/649050975 shutdown_connections 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e7a0b0 0x560d52e8ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 --2- 172.21.15.26:0/649050975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e796c0 0x560d52e6fe40 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 -- 172.21.15.26:0/649050975 >> 172.21.15.26:0/649050975 conn(0x560d52e74d30 msgr2=0x560d52e75110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 -- 172.21.15.26:0/649050975 shutdown_connections 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 -- 172.21.15.26:0/649050975 wait complete. 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 -- start start 2024-05-20T21:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.395+0000 7f370e9fd740 1 Processor -- start 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 0x560d52e88930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e89940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37041a8560 con 0x560d52e7a0b0 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37041a99d0 con 0x560d52e796c0 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37041a8c30 con 0x560d52e7f220 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 10 client.0 _ll_register_callbacks cb 0x7f36fc041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 0x560d52e88930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 0x560d52e88930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60300/0 (socket says 172.21.15.26:60300) 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 -- 172.21.15.26:0/2760434668 learned_addr learned my addr 172.21.15.26:0/2760434668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 -- 172.21.15.26:0/2760434668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 msgr2=0x560d52e89940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e89940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 -- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 msgr2=0x560d52e883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 -- 172.21.15.26:0/2760434668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d52e89f30 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f37095e0640 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 0x560d52e88930 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f36f40a9290 tx=0x7f36f4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fcc1dd9c482199 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f36ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2760434668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f40b9ad0 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2760434668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f36f40e5a00 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.396+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700003e00 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2760434668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36f40b9ca0 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 10 client.19584 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d52efa1b0 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2760434668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3700003e00 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2760434668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f36f40a5590 con 0x560d52e7a0b0 2024-05-20T21:09:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 10 client.19584 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 10 client.19584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 20 client.19584 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52efa620 0x560d52efca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 4 client.19584 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d52de6360 con 0x560d52efa620 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.397+0000 7f3709de1640 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52efa620 0x560d52efca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.398+0000 7f36ed7fa640 10 client.19584 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.398+0000 7f3709de1640 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52efa620 0x560d52efca80 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19ce907329a13803 server_cookie=cf9c5474826b6c9f in_seq=0 out_seq=0 2024-05-20T21:09:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f36ed7fa640 1 -- 172.21.15.26:0/2760434668 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560d52de6360 con 0x560d52efa620 2024-05-20T21:09:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f36ed7fa640 10 client.19584 handle_command_reply: tid=0 2024-05-20T21:09:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 client.19584 shutdown 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 msgr2=0x560d52e88930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 0x560d52e88930 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f36f40a9290 tx=0x7f36f4005bb0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 shutdown_connections 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560d52efa620 0x560d52efca80 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d52e7f220 0x560d52e89940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d52e7a0b0 0x560d52e88930 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 --2- 172.21.15.26:0/2760434668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560d52e796c0 0x560d52e883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f36ed7fa640 0 client.19584 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 >> 172.21.15.26:0/2760434668 conn(0x560d52e74d30 msgr2=0x560d52e6fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 shutdown_connections 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 1 -- 172.21.15.26:0/2760434668 wait complete. 2024-05-20T21:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.416+0000 7f370e9fd740 20 client.19584 trim_cache size 0 max 16384 2024-05-20T21:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f36f00b28a0 msgr2=0x7f36f00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f36f00b28a0 0x7f36f00b4d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f36fc005c40 tx=0x7f36fc005a20 comp rx=0 tx=0).stop 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 msgr2=0x7f3704199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 0x7f3704199060 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f36f400d770 tx=0x7f36f400dc30 comp rx=0 tx=0).stop 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 shutdown_connections 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f36f00b28a0 0x7f36f00b4d60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3704103560 0x7f37041a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3704102ba0 0x7f3704199060 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 --2- 172.21.15.26:0/3449376468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37041019b0 0x7f3704198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 >> 172.21.15.26:0/3449376468 conn(0x7f37040fcf20 msgr2=0x7f37040fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 shutdown_connections 2024-05-20T21:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:21.417+0000 7f370c045640 1 -- 172.21.15.26:0/3449376468 wait complete. 2024-05-20T21:09:21.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:21 smithi055 ceph-mon[35213]: pgmap v786: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-20T21:09:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:21.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.18 MB/sec execute 567 sec latency 17805.787 ms 2024-05-20T21:09:22.414 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:22 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:09:22.414 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:22 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:22.414 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:22 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:09:22.414 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:22 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:22.415 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:22 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:09:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:22 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:09:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:22 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:22 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:09:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:22 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:22.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:22 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:09:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:22 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:09:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:22 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:22 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:09:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:22 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:22 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:09:22.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.18 MB/sec execute 568 sec latency 18805.851 ms 2024-05-20T21:09:23.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:23 smithi026 ceph-mon[31990]: pgmap v787: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 -- 172.21.15.26:0/4121260674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41041a0 msgr2=0x7f10f41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 --2- 172.21.15.26:0/4121260674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41041a0 0x7f10f41081c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f10e40099c0 tx=0x7f10e4009cd0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 -- 172.21.15.26:0/4121260674 shutdown_connections 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 --2- 172.21.15.26:0/4121260674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f41088f0 0x7f10f4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 --2- 172.21.15.26:0/4121260674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10f41037f0 0x7f10f4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 --2- 172.21.15.26:0/4121260674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41041a0 0x7f10f41081c0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 -- 172.21.15.26:0/4121260674 >> 172.21.15.26:0/4121260674 conn(0x7f10f407a6a0 msgr2=0x7f10f407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 -- 172.21.15.26:0/4121260674 shutdown_connections 2024-05-20T21:09:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.437+0000 7f10fb676700 1 -- 172.21.15.26:0/4121260674 wait complete. 2024-05-20T21:09:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 Processor -- start 2024-05-20T21:09:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 -- start start 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 0x7f10f41a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f41041a0 0x7f10f41a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10f41088f0 0x7f10f419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4113560 con 0x7f10f41037f0 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41133e0 con 0x7f10f41041a0 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10fb676700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41136e0 con 0x7f10f41088f0 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10f9412700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 0x7f10f41a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10f9412700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 0x7f10f41a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60318/0 (socket says 172.21.15.26:60318) 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.438+0000 7f10f9412700 1 -- 172.21.15.26:0/2195834632 learned_addr learned my addr 172.21.15.26:0/2195834632 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f8c11700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f41041a0 0x7f10f41a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9c13700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10f41088f0 0x7f10f419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9412700 1 -- 172.21.15.26:0/2195834632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10f41088f0 msgr2=0x7f10f419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9412700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10f41088f0 0x7f10f419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9412700 1 -- 172.21.15.26:0/2195834632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f41041a0 msgr2=0x7f10f41a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9412700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f41041a0 0x7f10f41a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9412700 1 -- 172.21.15.26:0/2195834632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f419c220 con 0x7f10f41037f0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10f9412700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 0x7f10f41a1900 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f10f00060b0 tx=0x7f10f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10ea7fc700 1 -- 172.21.15.26:0/2195834632 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f001a070 con 0x7f10f41037f0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f419c510 con 0x7f10f41037f0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10ea7fc700 1 -- 172.21.15.26:0/2195834632 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10f000ddd0 con 0x7f10f41037f0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10ea7fc700 1 -- 172.21.15.26:0/2195834632 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0014690 con 0x7f10f41037f0 2024-05-20T21:09:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.439+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f41a87d0 con 0x7f10f41037f0 2024-05-20T21:09:23.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.442+0000 7f10ea7fc700 1 -- 172.21.15.26:0/2195834632 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f10f001d070 con 0x7f10f41037f0 2024-05-20T21:09:23.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.443+0000 7f10ea7fc700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10e00b2880 0x7f10e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.443+0000 7f10f8c11700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10e00b2880 0x7f10e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.443+0000 7f10ea7fc700 1 -- 172.21.15.26:0/2195834632 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f10f00e1b40 con 0x7f10f41037f0 2024-05-20T21:09:23.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.443+0000 7f1105f0cb80 1 Processor -- start 2024-05-20T21:09:23.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10e00b2880 0x7f10e00b4d40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000c00 tx=0x7f10e4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:23.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 -- start start 2024-05-20T21:09:23.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde53700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde5a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5642cddabd70 con 0x5642cde4d8a0 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5642cdd9b910 con 0x5642cde53320 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5642cdd9f950 con 0x5642cde4e260 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde4dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60324/0 (socket says 172.21.15.26:60324) 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 -- 172.21.15.26:0/314891046 learned_addr learned my addr 172.21.15.26:0/314891046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f9412700 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde53700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f9c13700 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde5a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 -- 172.21.15.26:0/314891046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 msgr2=0x5642cde5a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde5a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 -- 172.21.15.26:0/314891046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 msgr2=0x5642cde53700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde53700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f8c11700 1 -- 172.21.15.26:0/314891046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cdd44fb0 con 0x5642cde4d8a0 2024-05-20T21:09:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.444+0000 7f10f9c13700 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde5a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.445+0000 7f10f8c11700 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde4dd20 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009fd0 tx=0x7f10e401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b3f848203dd56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.445+0000 7f10df7fe700 1 -- 172.21.15.26:0/314891046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e400bb10 con 0x5642cde4d8a0 2024-05-20T21:09:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.445+0000 7f10df7fe700 1 -- 172.21.15.26:0/314891046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10e400bcb0 con 0x5642cde4d8a0 2024-05-20T21:09:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.445+0000 7f10df7fe700 1 -- 172.21.15.26:0/314891046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4030840 con 0x5642cde4d8a0 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.445+0000 7f1105f0cb80 1 -- 172.21.15.26:0/314891046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 msgr2=0x5642cde4dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.445+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde4dd20 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009fd0 tx=0x7f10e401f000 comp rx=0 tx=0).stop 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 -- 172.21.15.26:0/314891046 shutdown_connections 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde5a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde53700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/314891046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde4dd20 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 -- 172.21.15.26:0/314891046 >> 172.21.15.26:0/314891046 conn(0x5642cdd79240 msgr2=0x5642cde483e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 -- 172.21.15.26:0/314891046 shutdown_connections 2024-05-20T21:09:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 -- 172.21.15.26:0/314891046 wait complete. 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 -- start start 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 Processor -- start 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.446+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4d8a0 0x5642cde59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 0x5642cde57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5642cdca1fa0 con 0x5642cde4e260 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5642cdd03da0 con 0x5642cde53320 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5642cde5a680 con 0x5642cde4d8a0 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 10 client.0 _ll_register_callbacks cb 0x5642cdd36950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 0x5642cde57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 0x5642cde57060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60334/0 (socket says 172.21.15.26:60334) 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 -- 172.21.15.26:0/1531387369 learned_addr learned my addr 172.21.15.26:0/1531387369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f9c13700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f9412700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4d8a0 0x5642cde59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 -- 172.21.15.26:0/1531387369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4d8a0 msgr2=0x5642cde59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4d8a0 0x5642cde59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 -- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 msgr2=0x5642cde575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 -- 172.21.15.26:0/1531387369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cdceb8d0 con 0x5642cde4e260 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f9412700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4d8a0 0x5642cde59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f9c13700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10f8c11700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 0x5642cde57060 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009d10 tx=0x7f10e4030ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149904bf41a9b4a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10dcff9700 1 -- 172.21.15.26:0/1531387369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4037070 con 0x5642cde4e260 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f10dcff9700 1 -- 172.21.15.26:0/1531387369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f10e4003680 con 0x5642cde4e260 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.447+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e400bcb0 con 0x5642cde4e260 2024-05-20T21:09:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f10dcff9700 1 -- 172.21.15.26:0/1531387369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10e4043050 con 0x5642cde4e260 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 10 client.19596 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642cdd55760 con 0x5642cde4e260 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f10dcff9700 1 -- 172.21.15.26:0/1531387369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10e400bcb0 con 0x5642cde4e260 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f10dcff9700 1 -- 172.21.15.26:0/1531387369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f10e400f870 con 0x5642cde4e260 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 10 client.19596 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 10 client.19596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 20 client.19596 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cde67000 0x5642cde693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 4 client.19596 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cddadcc0 con 0x5642cde67000 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.448+0000 7f10f9412700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cde67000 0x5642cde693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.449+0000 7f10dcff9700 10 client.19596 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.449+0000 7f10f9412700 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cde67000 0x5642cde693f0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a617405c68aaa7cc server_cookie=d3fee62db77b7d3e in_seq=0 out_seq=0 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.449+0000 7f10dcff9700 1 -- 172.21.15.26:0/1531387369 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5642cddadcc0 con 0x5642cde67000 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.449+0000 7f10dcff9700 10 client.19596 handle_command_reply: tid=0 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.449+0000 7f1105f0cb80 1 client.19596 shutdown 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.449+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 msgr2=0x5642cde57060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 0x5642cde57060 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009d10 tx=0x7f10e4030ea0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 shutdown_connections 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cde67000 0x5642cde693f0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4d8a0 0x5642cde59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/1531387369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4e260 0x5642cde57060 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 >> 172.21.15.26:0/1531387369 conn(0x5642cdd79240 msgr2=0x5642cde47af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f10dcff9700 0 client.19596 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 shutdown_connections 2024-05-20T21:09:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 1 -- 172.21.15.26:0/1531387369 wait complete. 2024-05-20T21:09:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.450+0000 7f1105f0cb80 20 client.19596 trim_cache size 0 max 16384 2024-05-20T21:09:23.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:23 smithi055 ceph-mon[35213]: pgmap v787: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 Processor -- start 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 -- start start 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde48270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde54010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde54550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41b2620 con 0x5642cde4d8a0 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41afeb0 con 0x5642cde53320 2024-05-20T21:09:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41add00 con 0x5642cde4e260 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde48270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde48270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60348/0 (socket says 172.21.15.26:60348) 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 -- 172.21.15.26:0/4291771056 learned_addr learned my addr 172.21.15.26:0/4291771056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9c13700 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde54550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f8c11700 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde54010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 -- 172.21.15.26:0/4291771056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 msgr2=0x5642cde54010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde54010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 -- 172.21.15.26:0/4291771056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 msgr2=0x5642cde54550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde54550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 -- 172.21.15.26:0/4291771056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cde56020 con 0x5642cde4d8a0 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9c13700 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde54550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.472+0000 7f10f9412700 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde48270 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f10f00a9030 tx=0x7f10f00098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d73731d977d9a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f10deffd700 1 -- 172.21.15.26:0/4291771056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f00ad450 con 0x5642cde4d8a0 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f10deffd700 1 -- 172.21.15.26:0/4291771056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10f00b1f00 con 0x5642cde4d8a0 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f10deffd700 1 -- 172.21.15.26:0/4291771056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f00a9300 con 0x5642cde4d8a0 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 -- 172.21.15.26:0/4291771056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 msgr2=0x5642cde48270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde48270 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f10f00a9030 tx=0x7f10f00098c0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 -- 172.21.15.26:0/4291771056 shutdown_connections 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde48270 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde54550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/4291771056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde54010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 -- 172.21.15.26:0/4291771056 >> 172.21.15.26:0/4291771056 conn(0x5642cdd79240 msgr2=0x5642cde4d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 -- 172.21.15.26:0/4291771056 shutdown_connections 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.473+0000 7f1105f0cb80 1 -- 172.21.15.26:0/4291771056 wait complete. 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 -- start start 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 Processor -- start 2024-05-20T21:09:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde68b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41ae370 con 0x5642cde4d8a0 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41af7e0 con 0x5642cde53320 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41aea40 con 0x5642cde4e260 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f1105f0cb80 10 client.0 _ll_register_callbacks cb 0x5642cdd36950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f10f9412700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f10f9412700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde68060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60360/0 (socket says 172.21.15.26:60360) 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.474+0000 7f10f9412700 1 -- 172.21.15.26:0/3197461693 learned_addr learned my addr 172.21.15.26:0/3197461693 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9c13700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde68b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f8c11700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9412700 1 -- 172.21.15.26:0/3197461693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 msgr2=0x5642cde685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9412700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9412700 1 -- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 msgr2=0x5642cde68b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9412700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde68b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9412700 1 -- 172.21.15.26:0/3197461693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cded0b30 con 0x5642cde4d8a0 2024-05-20T21:09:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f8c11700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9c13700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde68b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10f9412700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde68060 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f10f00a50d0 tx=0x7f10f00a98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed23004b75847f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10dcff9700 1 -- 172.21.15.26:0/3197461693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f00ad0d0 con 0x5642cde4d8a0 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f10dcff9700 1 -- 172.21.15.26:0/3197461693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f10f00a9300 con 0x5642cde4d8a0 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.475+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f00b1f00 con 0x5642cde4d8a0 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f10dcff9700 1 -- 172.21.15.26:0/3197461693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10f00a9500 con 0x5642cde4d8a0 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 10 client.19602 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642cded1e40 con 0x5642cde4d8a0 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f10dcff9700 1 -- 172.21.15.26:0/3197461693 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10f00b1f00 con 0x5642cde4d8a0 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f10dcff9700 1 -- 172.21.15.26:0/3197461693 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f10f00a4b30 con 0x5642cde4d8a0 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 10 client.19602 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 10 client.19602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 20 client.19602 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cded24a0 0x5642cded48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 4 client.19602 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642cde52d20 con 0x5642cded24a0 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.476+0000 7f10f8c11700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cded24a0 0x5642cded48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f10dcff9700 10 client.19602 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f10f8c11700 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cded24a0 0x5642cded48c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b658e3fe331c67 server_cookie=445fc693464dec39 in_seq=0 out_seq=0 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f10dcff9700 1 -- 172.21.15.26:0/3197461693 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5642cde52d20 con 0x5642cded24a0 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f10dcff9700 10 client.19602 handle_command_reply: tid=0 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 client.19602 shutdown 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 msgr2=0x5642cde68060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde68060 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f10f00a50d0 tx=0x7f10f00a98c0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 shutdown_connections 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5642cded24a0 0x5642cded48c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5642cde4d8a0 0x5642cde68060 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5642cde53320 0x5642cde68b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 --2- 172.21.15.26:0/3197461693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642cde4e260 0x5642cde685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 >> 172.21.15.26:0/3197461693 conn(0x5642cdd79240 msgr2=0x5642cde50ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f10dcff9700 0 client.19602 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 shutdown_connections 2024-05-20T21:09:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.477+0000 7f1105f0cb80 1 -- 172.21.15.26:0/3197461693 wait complete. 2024-05-20T21:09:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f1105f0cb80 20 client.19602 trim_cache size 0 max 16384 2024-05-20T21:09:23.483 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:09:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:09:23.483 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10e00b2880 msgr2=0x7f10e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10e00b2880 0x7f10e00b4d40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000c00 tx=0x7f10e4005fb0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 msgr2=0x7f10f41a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 0x7f10f41a1900 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f10f00060b0 tx=0x7f10f000d3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 shutdown_connections 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f10e00b2880 0x7f10e00b4d40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10f41037f0 0x7f10f41a1900 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10f41088f0 0x7f10f419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 --2- 172.21.15.26:0/2195834632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f41041a0 0x7f10f41a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 >> 172.21.15.26:0/2195834632 conn(0x7f10f407a6a0 msgr2=0x7f10f4101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 shutdown_connections 2024-05-20T21:09:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:23.478+0000 7f10fb676700 1 -- 172.21.15.26:0/2195834632 wait complete. 2024-05-20T21:09:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:23 smithi144 ceph-mon[35660]: pgmap v787: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-20T21:09:23.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.17 MB/sec execute 569 sec latency 19805.917 ms 2024-05-20T21:09:23.982 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:09:24.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:09:24.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:09:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:24.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.17 MB/sec execute 570 sec latency 20805.943 ms 2024-05-20T21:09:25.085 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:25.355 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:25 smithi026 ceph-mon[31990]: pgmap v788: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2024-05-20T21:09:25.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:25 smithi055 ceph-mon[35213]: pgmap v788: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2024-05-20T21:09:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:25 smithi144 ceph-mon[35660]: pgmap v788: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2024-05-20T21:09:25.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.16 MB/sec execute 571 sec latency 21806.020 ms 2024-05-20T21:09:26.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.16 MB/sec execute 572 sec latency 22806.082 ms 2024-05-20T21:09:26.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 -- 172.21.15.26:0/4017783091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e4102a80 msgr2=0x7f67e4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:26.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 --2- 172.21.15.26:0/4017783091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e4102a80 0x7f67e4102e60 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f67d400b0d0 tx=0x7f67d400b3e0 comp rx=0 tx=0).stop 2024-05-20T21:09:26.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 -- 172.21.15.26:0/4017783091 shutdown_connections 2024-05-20T21:09:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 --2- 172.21.15.26:0/4017783091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e4102a80 0x7f67e4102e60 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 --2- 172.21.15.26:0/4017783091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e41077d0 0x7f67e410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 --2- 172.21.15.26:0/4017783091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67e4103430 0x7f67e41070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 -- 172.21.15.26:0/4017783091 >> 172.21.15.26:0/4017783091 conn(0x7f67e40793f0 msgr2=0x7f67e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 -- 172.21.15.26:0/4017783091 shutdown_connections 2024-05-20T21:09:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.922+0000 7f67e99fa700 1 -- 172.21.15.26:0/4017783091 wait complete. 2024-05-20T21:09:26.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 Processor -- start 2024-05-20T21:09:26.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 -- start start 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67e4102a80 0x7f67e419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e4103430 0x7f67e419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 0x7f67e41a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41125b0 con 0x7f67e41077d0 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4112430 con 0x7f67e4102a80 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.923+0000 7f67e99fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4112730 con 0x7f67e4103430 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 0x7f67e41a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 0x7f67e41a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60380/0 (socket says 172.21.15.26:60380) 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 -- 172.21.15.26:0/3641217974 learned_addr learned my addr 172.21.15.26:0/3641217974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67da7fc700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e4103430 0x7f67e419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e2ffd700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67e4102a80 0x7f67e419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 -- 172.21.15.26:0/3641217974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e4103430 msgr2=0x7f67e419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e4103430 0x7f67e419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 -- 172.21.15.26:0/3641217974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67e4102a80 msgr2=0x7f67e419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67e4102a80 0x7f67e419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 -- 172.21.15.26:0/3641217974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e41a5fd0 con 0x7f67e41077d0 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e37fe700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 0x7f67e41a5a90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b550 tx=0x7f67dc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc015020 con 0x7f67e41077d0 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e41a6200 con 0x7f67e41077d0 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67dc011cc0 con 0x7f67e41077d0 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc010b80 con 0x7f67e41077d0 2024-05-20T21:09:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.924+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41a66c0 con 0x7f67e41077d0 2024-05-20T21:09:26.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.926+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e4103b80 con 0x7f67e41077d0 2024-05-20T21:09:26.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.927+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f67dc018020 con 0x7f67e41077d0 2024-05-20T21:09:26.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.927+0000 7f67e89f8700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67d00b2880 0x7f67d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:26.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.928+0000 7f67e2ffd700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67d00b2880 0x7f67d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:26.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.928+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f67dc0e1930 con 0x7f67e41077d0 2024-05-20T21:09:26.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.928+0000 7f67e2ffd700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67d00b2880 0x7f67d00b4d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009fd0 tx=0x7f67d400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:26.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:26.932+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67dc01f020 con 0x7f67e41077d0 2024-05-20T21:09:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.204+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67e402e720 con 0x7f67e41077d0 2024-05-20T21:09:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.205+0000 7f67e89f8700 1 -- 172.21.15.26:0/3641217974 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f67dc0a95f0 con 0x7f67e41077d0 2024-05-20T21:09:27.204 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:09:27.204 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67d00b2880 msgr2=0x7f67d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67d00b2880 0x7f67d00b4d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009fd0 tx=0x7f67d400b040 comp rx=0 tx=0).stop 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 msgr2=0x7f67e41a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 0x7f67e41a5a90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b550 tx=0x7f67dc00b860 comp rx=0 tx=0).stop 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 shutdown_connections 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67d00b2880 0x7f67d00b4d40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e41077d0 0x7f67e41a5a90 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67e4102a80 0x7f67e419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 --2- 172.21.15.26:0/3641217974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e4103430 0x7f67e419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.207+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 >> 172.21.15.26:0/3641217974 conn(0x7f67e40793f0 msgr2=0x7f67e4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.208+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 shutdown_connections 2024-05-20T21:09:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:27.208+0000 7f67e99fa700 1 -- 172.21.15.26:0/3641217974 wait complete. 2024-05-20T21:09:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:09:27.383 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:27 smithi026 ceph-mon[31990]: pgmap v789: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:27 smithi055 ceph-mon[35213]: pgmap v789: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:27 smithi144 ceph-mon[35660]: pgmap v789: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:27.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.16 MB/sec execute 573 sec latency 23806.196 ms 2024-05-20T21:09:28.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:28 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3641217974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 -- 172.21.15.26:0/3843271597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01088f0 msgr2=0x7f3de0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 --2- 172.21.15.26:0/3843271597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01088f0 0x7f3de0110470 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800b0d0 tx=0x7f3dd800b3e0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 -- 172.21.15.26:0/3843271597 shutdown_connections 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 --2- 172.21.15.26:0/3843271597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01088f0 0x7f3de0110470 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 --2- 172.21.15.26:0/3843271597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01037f0 0x7f3de0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 --2- 172.21.15.26:0/3843271597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01041a0 0x7f3de01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 -- 172.21.15.26:0/3843271597 >> 172.21.15.26:0/3843271597 conn(0x7f3de007a6a0 msgr2=0x7f3de007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 -- 172.21.15.26:0/3843271597 shutdown_connections 2024-05-20T21:09:28.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 -- 172.21.15.26:0/3843271597 wait complete. 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 Processor -- start 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.297+0000 7f3de62db700 1 -- start start 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de62db700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01037f0 0x7f3de019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de62db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 0x7f3de019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de62db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01088f0 0x7f3de0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de62db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0113590 con 0x7f3de01041a0 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de62db700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0113410 con 0x7f3de01037f0 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de62db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0113710 con 0x7f3de01088f0 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 0x7f3de019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 0x7f3de019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60396/0 (socket says 172.21.15.26:60396) 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 -- 172.21.15.26:0/2720735158 learned_addr learned my addr 172.21.15.26:0/2720735158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de4878700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01088f0 0x7f3de0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddffff700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01037f0 0x7f3de019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 -- 172.21.15.26:0/2720735158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01088f0 msgr2=0x7f3de0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01088f0 0x7f3de0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 -- 172.21.15.26:0/2720735158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01037f0 msgr2=0x7f3de019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01037f0 0x7f3de019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 -- 172.21.15.26:0/2720735158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de0199ff0 con 0x7f3de01041a0 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddffff700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01037f0 0x7f3de019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3de4878700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01088f0 0x7f3de0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.298+0000 7f3ddf7fe700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 0x7f3de019fc10 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.299+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4016070 con 0x7f3de01041a0 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.299+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de019a2e0 con 0x7f3de01041a0 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.299+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dd4010460 con 0x7f3de01041a0 2024-05-20T21:09:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.299+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400f620 con 0x7f3de01041a0 2024-05-20T21:09:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.299+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de01a65a0 con 0x7f3de01041a0 2024-05-20T21:09:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.300+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de006a230 con 0x7f3de01041a0 2024-05-20T21:09:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.301+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3dd4019070 con 0x7f3de01041a0 2024-05-20T21:09:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.302+0000 7f3ddd7fa700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3dc80b2880 0x7f3dc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.302+0000 7f3ddffff700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3dc80b2880 0x7f3dc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.303+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3dd40e1eb0 con 0x7f3de01041a0 2024-05-20T21:09:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.303+0000 7f3ddffff700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3dc80b2880 0x7f3dc80b4d40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005950 tx=0x7f3dd0005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.307+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3dd40a9b70 con 0x7f3de01041a0 2024-05-20T21:09:28.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:28 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3641217974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:28.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:28 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3641217974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:28.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.582+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3de002e560 con 0x7f3de01041a0 2024-05-20T21:09:28.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.582+0000 7f3ddd7fa700 1 -- 172.21.15.26:0/2720735158 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f3dd40a9470 con 0x7f3de01041a0 2024-05-20T21:09:28.582 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:09:28.582 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3dc80b2880 msgr2=0x7f3dc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3dc80b2880 0x7f3dc80b4d40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005950 tx=0x7f3dd0005f30 comp rx=0 tx=0).stop 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 msgr2=0x7f3de019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 0x7f3de019fc10 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400dd10 comp rx=0 tx=0).stop 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 shutdown_connections 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de01088f0 0x7f3de0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3de01037f0 0x7f3de019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3dc80b2880 0x7f3dc80b4d40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 --2- 172.21.15.26:0/2720735158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de01041a0 0x7f3de019fc10 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 >> 172.21.15.26:0/2720735158 conn(0x7f3de007a6a0 msgr2=0x7f3de0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 shutdown_connections 2024-05-20T21:09:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:28.585+0000 7f3de62db700 1 -- 172.21.15.26:0/2720735158 wait complete. 2024-05-20T21:09:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:09:28.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.15 MB/sec execute 574 sec latency 24806.226 ms 2024-05-20T21:09:28.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:09:29.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:29 smithi055 ceph-mon[35213]: pgmap v790: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:29.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:29 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2720735158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:29 smithi026 ceph-mon[31990]: pgmap v790: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:29.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:29 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2720735158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:29.543 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id e258fdd1 --release --await' 2024-05-20T21:09:29.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id e258fdd1 --release --await 2024-05-20T21:09:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:29 smithi144 ceph-mon[35660]: pgmap v790: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:29 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2720735158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:29.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.15 MB/sec execute 575 sec latency 25806.266 ms 2024-05-20T21:09:29.645 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:30.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.15 MB/sec execute 576 sec latency 26806.348 ms 2024-05-20T21:09:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:31.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:31 smithi026 ceph-mon[31990]: pgmap v791: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:31.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:31 smithi055 ceph-mon[35213]: pgmap v791: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:31 smithi144 ceph-mon[35660]: pgmap v791: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 KiB/s wr, 17 op/s 2024-05-20T21:09:31.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.14 MB/sec execute 577 sec latency 27806.427 ms 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/1752888913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0102930 msgr2=0x7fa0a0110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/1752888913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0102930 0x7fa0a0110430 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa09800b0a0 tx=0x7fa09800b3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/1752888913 shutdown_connections 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/1752888913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0102930 0x7fa0a0110430 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/1752888913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a0109720 0x7fa0a0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/1752888913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0a0101f70 0x7fa0a01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/1752888913 >> 172.21.15.26:0/1752888913 conn(0x7fa0a00776b0 msgr2=0x7fa0a0077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:32.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/1752888913 shutdown_connections 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.078+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/1752888913 wait complete. 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 Processor -- start 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 -- start start 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0a0101f70 0x7fa0a019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a0102930 0x7fa0a019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 0x7fa0a01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0113600 con 0x7fa0a0109720 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0113480 con 0x7fa0a0101f70 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a6a7c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0113780 con 0x7fa0a0102930 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a5019700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 0x7fa0a01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a5019700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 0x7fa0a01999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54652/0 (socket says 172.21.15.26:54652) 2024-05-20T21:09:32.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a5019700 1 -- 172.21.15.26:0/2842008526 learned_addr learned my addr 172.21.15.26:0/2842008526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa0a4818700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0a0101f70 0x7fa0a019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.079+0000 7fa09ffff700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a0102930 0x7fa0a019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a5019700 1 -- 172.21.15.26:0/2842008526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a0102930 msgr2=0x7fa0a019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a5019700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a0102930 0x7fa0a019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a5019700 1 -- 172.21.15.26:0/2842008526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0a0101f70 msgr2=0x7fa0a019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a5019700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0a0101f70 0x7fa0a019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a5019700 1 -- 172.21.15.26:0/2842008526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a019a100 con 0x7fa0a0109720 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a5019700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 0x7fa0a01999a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa0980062a0 tx=0x7fa098004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa09dffb700 1 -- 172.21.15.26:0/2842008526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098021070 con 0x7fa0a0109720 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a019a390 con 0x7fa0a0109720 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa09dffb700 1 -- 172.21.15.26:0/2842008526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa098007c80 con 0x7fa0a0109720 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa09dffb700 1 -- 172.21.15.26:0/2842008526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980052a0 con 0x7fa0a0109720 2024-05-20T21:09:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.080+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a0103080 con 0x7fa0a0109720 2024-05-20T21:09:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.083+0000 7fa09dffb700 1 -- 172.21.15.26:0/2842008526 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa098024070 con 0x7fa0a0109720 2024-05-20T21:09:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.083+0000 7fa09dffb700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0880b2880 0x7fa0880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0a4818700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0880b2880 0x7fa0880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa09dffb700 1 -- 172.21.15.26:0/2842008526 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa09800f020 con 0x7fa0a0109720 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0a4818700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0880b2880 0x7fa0880b4d40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0100640 tx=0x7fa090009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 Processor -- start 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 -- start start 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc16c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 0x55dc9cc1a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dc9cb75c90 con 0x55dc9cc17210 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dc9cb65830 con 0x55dc9cc16860 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc9cb69870 con 0x55dc9cc1af60 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa09ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 0x55dc9cc1a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa09ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 0x55dc9cc1a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54654/0 (socket says 172.21.15.26:54654) 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.084+0000 7fa09ffff700 1 -- 172.21.15.26:0/1467536411 learned_addr learned my addr 172.21.15.26:0/1467536411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa0a5019700 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa0a4818700 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc16c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa09ffff700 1 -- 172.21.15.26:0/1467536411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 msgr2=0x55dc9cc1b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa09ffff700 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa09ffff700 1 -- 172.21.15.26:0/1467536411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 msgr2=0x55dc9cc16c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa09ffff700 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc16c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa09ffff700 1 -- 172.21.15.26:0/1467536411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9cab58d0 con 0x55dc9cc17210 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa0a5019700 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:32.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa09ffff700 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 0x55dc9cc1a940 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa09400cc50 tx=0x7fa09400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb31605660af538 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa086ffd700 1 -- 172.21.15.26:0/1467536411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094014070 con 0x55dc9cc17210 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa086ffd700 1 -- 172.21.15.26:0/1467536411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa09400edf0 con 0x55dc9cc17210 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.085+0000 7fa086ffd700 1 -- 172.21.15.26:0/1467536411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940107d0 con 0x55dc9cc17210 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 -- 172.21.15.26:0/1467536411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 msgr2=0x55dc9cc1a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 0x55dc9cc1a940 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa09400cc50 tx=0x7fa09400cf60 comp rx=0 tx=0).stop 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 -- 172.21.15.26:0/1467536411 shutdown_connections 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc16c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/1467536411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc17210 0x55dc9cc1a940 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 -- 172.21.15.26:0/1467536411 >> 172.21.15.26:0/1467536411 conn(0x55dc9cb44080 msgr2=0x55dc9cc10e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 -- 172.21.15.26:0/1467536411 shutdown_connections 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 -- 172.21.15.26:0/1467536411 wait complete. 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.086+0000 7fa0b1312b80 1 -- start start 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 Processor -- start 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc17210 0x55dc9cc1f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dc9ca6bfa0 con 0x55dc9cc1af60 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dc9cb077c0 con 0x55dc9cc16860 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc9cc23120 con 0x55dc9cc17210 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0a4818700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0b1312b80 10 client.0 _ll_register_callbacks cb 0x55dc9cb00920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0a5019700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0a4818700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc223c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50792/0 (socket says 172.21.15.26:50792) 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa0a4818700 1 -- 172.21.15.26:0/3006008394 learned_addr learned my addr 172.21.15.26:0/3006008394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.087+0000 7fa09ffff700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc17210 0x55dc9cc1f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a4818700 1 -- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc17210 msgr2=0x55dc9cc1f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a4818700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc17210 0x55dc9cc1f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a4818700 1 -- 172.21.15.26:0/3006008394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 msgr2=0x55dc9cc1fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a4818700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a4818700 1 -- 172.21.15.26:0/3006008394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9cb04690 con 0x55dc9cc16860 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a5019700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa09ffff700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc17210 0x55dc9cc1f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0a4818700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc223c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa090015700 tx=0x7fa090004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c725a7ddd7eeb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa07ffff700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa07ffff700 1 -- 172.21.15.26:0/3006008394 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090025020 con 0x55dc9cc16860 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09400edf0 con 0x55dc9cc16860 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa07ffff700 1 -- 172.21.15.26:0/3006008394 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa090015c70 con 0x55dc9cc16860 2024-05-20T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa07ffff700 1 -- 172.21.15.26:0/3006008394 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa090030050 con 0x55dc9cc16860 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0b1312b80 10 client.28832 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.088+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc9cc2d880 con 0x55dc9cc16860 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa07ffff700 1 -- 172.21.15.26:0/3006008394 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa09400edf0 con 0x55dc9cc16860 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa07ffff700 1 -- 172.21.15.26:0/3006008394 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fa090013450 con 0x55dc9cc16860 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa0b1312b80 10 client.28832 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa0b1312b80 10 client.28832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa0b1312b80 20 client.28832 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc2feb0 0x55dc9cc322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa0b1312b80 4 client.28832 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc9cbe29e0 con 0x55dc9cc2feb0 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.089+0000 7fa09ffff700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc2feb0 0x55dc9cc322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa09ffff700 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc2feb0 0x55dc9cc322c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acade61ee70bb0d server_cookie=6fbe90b0d1222108 in_seq=0 out_seq=0 2024-05-20T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa07ffff700 10 client.28832 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa07ffff700 1 -- 172.21.15.26:0/3006008394 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc9cbe29e0 con 0x55dc9cc2feb0 2024-05-20T21:09:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa07ffff700 10 client.28832 handle_command_reply: tid=0 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 client.28832 shutdown 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 msgr2=0x55dc9cc223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc223c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa090015700 tx=0x7fa090004860 comp rx=0 tx=0).stop 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 shutdown_connections 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc2feb0 0x55dc9cc322c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc1fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc16860 0x55dc9cc223c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/3006008394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc17210 0x55dc9cc1f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa07ffff700 0 client.28832 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.090+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 >> 172.21.15.26:0/3006008394 conn(0x55dc9cb44080 msgr2=0x55dc9cc10510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.091+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 shutdown_connections 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.091+0000 7fa0b1312b80 1 -- 172.21.15.26:0/3006008394 wait complete. 2024-05-20T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.091+0000 7fa0b1312b80 20 client.28832 trim_cache size 0 max 16384 2024-05-20T21:09:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 Processor -- start 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 -- start start 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc16860 0x55dc9cc1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc10c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc37bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01b26f0 con 0x55dc9cc1af60 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01aff80 con 0x55dc9cc17210 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01addd0 con 0x55dc9cc16860 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc37bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc37bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54672/0 (socket says 172.21.15.26:54672) 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 -- 172.21.15.26:0/197826101 learned_addr learned my addr 172.21.15.26:0/197826101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa09ffff700 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc10c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a4818700 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc16860 0x55dc9cc1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 -- 172.21.15.26:0/197826101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc16860 msgr2=0x55dc9cc1fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc16860 0x55dc9cc1fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 -- 172.21.15.26:0/197826101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 msgr2=0x55dc9cc10c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc10c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a5019700 1 -- 172.21.15.26:0/197826101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9cc12d20 con 0x55dc9cc1af60 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.108+0000 7fa0a4818700 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc16860 0x55dc9cc1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.109+0000 7fa0a5019700 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc37bd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa0980a7020 tx=0x7fa09801df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393c689ff500dc42 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.109+0000 7fa0867fc700 1 -- 172.21.15.26:0/197826101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980aafa0 con 0x55dc9cc1af60 2024-05-20T21:09:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.109+0000 7fa0867fc700 1 -- 172.21.15.26:0/197826101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0980af350 con 0x55dc9cc1af60 2024-05-20T21:09:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.109+0000 7fa0867fc700 1 -- 172.21.15.26:0/197826101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980b38e0 con 0x55dc9cc1af60 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 -- 172.21.15.26:0/197826101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 msgr2=0x55dc9cc37bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc37bd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa0980a7020 tx=0x7fa09801df90 comp rx=0 tx=0).stop 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 -- 172.21.15.26:0/197826101 shutdown_connections 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc1af60 0x55dc9cc37bd0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc16860 0x55dc9cc1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/197826101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc10c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 -- 172.21.15.26:0/197826101 >> 172.21.15.26:0/197826101 conn(0x55dc9cb44080 msgr2=0x55dc9cc21780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 -- 172.21.15.26:0/197826101 shutdown_connections 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 -- 172.21.15.26:0/197826101 wait complete. 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 -- start start 2024-05-20T21:09:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 Processor -- start 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 0x55dc9cc28790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc25b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.110+0000 7fa0b1312b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc26070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ae440 con 0x55dc9cc16860 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01af8b0 con 0x55dc9cc17210 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0b1312b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01aeb10 con 0x55dc9cc1af60 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0b1312b80 10 client.0 _ll_register_callbacks cb 0x55dc9cb00920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 0x55dc9cc28790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 0x55dc9cc28790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54678/0 (socket says 172.21.15.26:54678) 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 -- 172.21.15.26:0/442849097 learned_addr learned my addr 172.21.15.26:0/442849097 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa09ffff700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc25b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a5019700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc26070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 -- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 msgr2=0x55dc9cc26070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc26070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 -- 172.21.15.26:0/442849097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 msgr2=0x55dc9cc25b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc25b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 -- 172.21.15.26:0/442849097 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9cc265b0 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a5019700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc26070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0a4818700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 0x55dc9cc28790 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa090014020 tx=0x7fa090013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8302d92053f0752d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa07ffff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa07ffff700 1 -- 172.21.15.26:0/442849097 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090025070 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa07ffff700 1 -- 172.21.15.26:0/442849097 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fa090015cc0 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.111+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0980af350 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa07ffff700 1 -- 172.21.15.26:0/442849097 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa09001b040 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 10 client.19632 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc9cc9a7e0 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa07ffff700 1 -- 172.21.15.26:0/442849097 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0980af350 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa07ffff700 1 -- 172.21.15.26:0/442849097 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fa090006550 con 0x55dc9cc16860 2024-05-20T21:09:32.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 10 client.19632 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 10 client.19632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 20 client.19632 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc9af10 0x55dc9cc9d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 4 client.19632 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc9cb77be0 con 0x55dc9cc9af10 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.112+0000 7fa09ffff700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc9af10 0x55dc9cc9d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa07ffff700 10 client.19632 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa09ffff700 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc9af10 0x55dc9cc9d330 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87c097cbcb5b5a1d server_cookie=968108028867fe25 in_seq=0 out_seq=0 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa07ffff700 1 -- 172.21.15.26:0/442849097 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc9cb77be0 con 0x55dc9cc9af10 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa07ffff700 10 client.19632 handle_command_reply: tid=0 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 client.19632 shutdown 2024-05-20T21:09:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 msgr2=0x55dc9cc28790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 0x55dc9cc28790 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa090014020 tx=0x7fa090013700 comp rx=0 tx=0).stop 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 shutdown_connections 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55dc9cc9af10 0x55dc9cc9d330 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc9cc16860 0x55dc9cc28790 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa07ffff700 0 client.19632 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9cc1af60 0x55dc9cc26070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 --2- 172.21.15.26:0/442849097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc9cc17210 0x55dc9cc25b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.113+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 >> 172.21.15.26:0/442849097 conn(0x55dc9cb44080 msgr2=0x55dc9cc20ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 shutdown_connections 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0b1312b80 1 -- 172.21.15.26:0/442849097 wait complete. 2024-05-20T21:09:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0b1312b80 20 client.19632 trim_cache size 0 max 16384 2024-05-20T21:09:32.119 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0880b2880 msgr2=0x7fa0880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0880b2880 0x7fa0880b4d40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0100640 tx=0x7fa090009190 comp rx=0 tx=0).stop 2024-05-20T21:09:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 msgr2=0x7fa0a01999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 0x7fa0a01999a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa0980062a0 tx=0x7fa098004e20 comp rx=0 tx=0).stop 2024-05-20T21:09:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 shutdown_connections 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fa0880b2880 0x7fa0880b4d40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a0109720 0x7fa0a01999a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0a0101f70 0x7fa0a019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 --2- 172.21.15.26:0/2842008526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a0102930 0x7fa0a019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.114+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 >> 172.21.15.26:0/2842008526 conn(0x7fa0a00776b0 msgr2=0x7fa0a0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.115+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 shutdown_connections 2024-05-20T21:09:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:32.115+0000 7fa0a6a7c700 1 -- 172.21.15.26:0/2842008526 wait complete. 2024-05-20T21:09:32.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295023 2.14 MB/sec execute 578 sec latency 28806.500 ms 2024-05-20T21:09:33.246 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:33 smithi026 ceph-mon[31990]: pgmap v792: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 -- 172.21.15.26:0/243740530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0104200 msgr2=0x7f37c01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 --2- 172.21.15.26:0/243740530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0104200 0x7f37c01081f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f37a80099c0 tx=0x7f37a8009cd0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 -- 172.21.15.26:0/243740530 shutdown_connections 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 --2- 172.21.15.26:0/243740530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37c0108920 0x7f37c01104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 --2- 172.21.15.26:0/243740530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0103850 0x7f37c0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 --2- 172.21.15.26:0/243740530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0104200 0x7f37c01081f0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 -- 172.21.15.26:0/243740530 >> 172.21.15.26:0/243740530 conn(0x7f37c007a5d0 msgr2=0x7f37c007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 -- 172.21.15.26:0/243740530 shutdown_connections 2024-05-20T21:09:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.361+0000 7f37c528d700 1 -- 172.21.15.26:0/243740530 wait complete. 2024-05-20T21:09:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 Processor -- start 2024-05-20T21:09:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 -- start start 2024-05-20T21:09:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 0x7f37c0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0104200 0x7f37c0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37c0108920 0x7f37c00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0113610 con 0x7f37c0103850 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0113490 con 0x7f37c0108920 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37c528d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0113790 con 0x7f37c0104200 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 0x7f37c0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 0x7f37c0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54690/0 (socket says 172.21.15.26:54690) 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37beffd700 1 -- 172.21.15.26:0/761791122 learned_addr learned my addr 172.21.15.26:0/761791122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.362+0000 7f37bf7fe700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37c0108920 0x7f37c00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37be7fc700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0104200 0x7f37c0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37beffd700 1 -- 172.21.15.26:0/761791122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0104200 msgr2=0x7f37c0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37beffd700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0104200 0x7f37c0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37beffd700 1 -- 172.21.15.26:0/761791122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37c0108920 msgr2=0x7f37c00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37beffd700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37c0108920 0x7f37c00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37beffd700 1 -- 172.21.15.26:0/761791122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c01a8030 con 0x7f37c0103850 2024-05-20T21:09:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37beffd700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 0x7f37c0075840 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f37b00060b0 tx=0x7f37b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37b7fff700 1 -- 172.21.15.26:0/761791122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b001a070 con 0x7f37c0103850 2024-05-20T21:09:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c01a8320 con 0x7f37c0103850 2024-05-20T21:09:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37b7fff700 1 -- 172.21.15.26:0/761791122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b000dd20 con 0x7f37c0103850 2024-05-20T21:09:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37b7fff700 1 -- 172.21.15.26:0/761791122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0014600 con 0x7f37c0103850 2024-05-20T21:09:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.363+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01a8880 con 0x7f37c0103850 2024-05-20T21:09:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.366+0000 7f37b7fff700 1 -- 172.21.15.26:0/761791122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f37b001d070 con 0x7f37c0103850 2024-05-20T21:09:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.366+0000 7f37b7fff700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37ac0b2880 0x7f37ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.366+0000 7f37be7fc700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37ac0b2880 0x7f37ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37b7fff700 1 -- 172.21.15.26:0/761791122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f37b00e19d0 con 0x7f37c0103850 2024-05-20T21:09:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 Processor -- start 2024-05-20T21:09:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 -- start start 2024-05-20T21:09:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37be7fc700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37ac0b2880 0x7f37ac0b4d40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f37a8000c00 tx=0x7f37a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:33.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a2cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557e7a226e70 con 0x557e7a2cd720 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557e7a216a10 con 0x557e7a2c95b0 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557e7a21aa50 con 0x557e7a2c8bf0 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a2cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a2cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54694/0 (socket says 172.21.15.26:54694) 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.367+0000 7f37beffd700 1 -- 172.21.15.26:0/896631330 learned_addr learned my addr 172.21.15.26:0/896631330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37bf7fe700 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37be7fc700 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37beffd700 1 -- 172.21.15.26:0/896631330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 msgr2=0x557e7a2c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37beffd700 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37beffd700 1 -- 172.21.15.26:0/896631330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 msgr2=0x557e7a2d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37beffd700 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37beffd700 1 -- 172.21.15.26:0/896631330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e7a1c00b0 con 0x557e7a2cd720 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37beffd700 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a2cdb00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f37b00a5010 tx=0x7f37b00a8a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45302f2fa83e8fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37b57fa700 1 -- 172.21.15.26:0/896631330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b00a9690 con 0x557e7a2cd720 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37b57fa700 1 -- 172.21.15.26:0/896631330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b00a8f90 con 0x557e7a2cd720 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.368+0000 7f37b57fa700 1 -- 172.21.15.26:0/896631330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b00b1960 con 0x557e7a2cd720 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 -- 172.21.15.26:0/896631330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 msgr2=0x557e7a2cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a2cdb00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f37b00a5010 tx=0x7f37b00a8a00 comp rx=0 tx=0).stop 2024-05-20T21:09:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 -- 172.21.15.26:0/896631330 shutdown_connections 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a2cdb00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/896631330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 -- 172.21.15.26:0/896631330 >> 172.21.15.26:0/896631330 conn(0x557e7a1f4340 msgr2=0x557e7a2c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 -- 172.21.15.26:0/896631330 shutdown_connections 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 -- 172.21.15.26:0/896631330 wait complete. 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 -- start start 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.369+0000 7f37cfb23b80 1 Processor -- start 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 0x557e7a2d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c95b0 0x557e7a2d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2cd720 0x557e7a2d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557e7a11d0a0 con 0x557e7a2c8bf0 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557e7a17eea0 con 0x557e7a2cd720 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557e7a2d5c80 con 0x557e7a2c95b0 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37cfb23b80 10 client.0 _ll_register_callbacks cb 0x557e7a1b1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 0x557e7a2d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 0x557e7a2d5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54704/0 (socket says 172.21.15.26:54704) 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 -- 172.21.15.26:0/894076152 learned_addr learned my addr 172.21.15.26:0/894076152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37bf7fe700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2cd720 0x557e7a2d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37be7fc700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c95b0 0x557e7a2d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 -- 172.21.15.26:0/894076152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c95b0 msgr2=0x557e7a2d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c95b0 0x557e7a2d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 -- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2cd720 msgr2=0x557e7a2d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2cd720 0x557e7a2d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 -- 172.21.15.26:0/894076152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7a1669d0 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37beffd700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 0x557e7a2d5330 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009510 tx=0x7f37b0015b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b259c5df7e4a0da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.370+0000 7f37a67fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37a67fc700 1 -- 172.21.15.26:0/894076152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0088060 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37a67fc700 1 -- 172.21.15.26:0/894076152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f37b00b1b70 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b00a8f90 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37a67fc700 1 -- 172.21.15.26:0/894076152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37b00c9070 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 10 client.19644 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e7a1d0860 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37a67fc700 1 -- 172.21.15.26:0/894076152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b00a8f90 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37a67fc700 1 -- 172.21.15.26:0/894076152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f37b008c030 con 0x557e7a2c8bf0 2024-05-20T21:09:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 10 client.19644 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 10 client.19644 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 20 client.19644 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a2e2700 0x557e7a2e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 4 client.19644 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.371+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e7a228dc0 con 0x557e7a2e2700 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.372+0000 7f37be7fc700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a2e2700 0x557e7a2e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.372+0000 7f37be7fc700 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a2e2700 0x557e7a2e4af0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448130b969480d57 server_cookie=d6c862792053cec7 in_seq=0 out_seq=0 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.372+0000 7f37a67fc700 10 client.19644 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37a67fc700 1 -- 172.21.15.26:0/894076152 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557e7a228dc0 con 0x557e7a2e2700 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37a67fc700 10 client.19644 handle_command_reply: tid=0 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 client.19644 shutdown 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 msgr2=0x557e7a2d5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 0x557e7a2d5330 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009510 tx=0x7f37b0015b50 comp rx=0 tx=0).stop 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 shutdown_connections 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a2e2700 0x557e7a2e4af0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c8bf0 0x557e7a2d5330 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2cd720 0x557e7a2d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/894076152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c95b0 0x557e7a2d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 >> 172.21.15.26:0/894076152 conn(0x557e7a1f4340 msgr2=0x557e7a2c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37a67fc700 0 client.19644 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 shutdown_connections 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 1 -- 172.21.15.26:0/894076152 wait complete. 2024-05-20T21:09:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.373+0000 7f37cfb23b80 20 client.19644 trim_cache size 0 max 16384 2024-05-20T21:09:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 Processor -- start 2024-05-20T21:09:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 -- start start 2024-05-20T21:09:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c8bf0 0x557e7a2bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 0x557e7a2ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2cd720 0x557e7a2ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01b2750 con 0x557e7a2c95b0 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01affe0 con 0x557e7a2c8bf0 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.394+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01ade30 con 0x557e7a2cd720 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 0x557e7a2ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 0x557e7a2ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54706/0 (socket says 172.21.15.26:54706) 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 -- 172.21.15.26:0/832461001 learned_addr learned my addr 172.21.15.26:0/832461001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37beffd700 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c8bf0 0x557e7a2bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 -- 172.21.15.26:0/832461001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2cd720 msgr2=0x557e7a2ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2cd720 0x557e7a2ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 -- 172.21.15.26:0/832461001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c8bf0 msgr2=0x557e7a2bde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c8bf0 0x557e7a2bde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 -- 172.21.15.26:0/832461001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e7a2d5b10 con 0x557e7a2c95b0 2024-05-20T21:09:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37be7fc700 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 0x557e7a2ce3c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f37a800b580 tx=0x7f37a8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c472775adc2d29a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37b4ff9700 1 -- 172.21.15.26:0/832461001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a800be30 con 0x557e7a2c95b0 2024-05-20T21:09:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37b4ff9700 1 -- 172.21.15.26:0/832461001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a8003b10 con 0x557e7a2c95b0 2024-05-20T21:09:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.395+0000 7f37b4ff9700 1 -- 172.21.15.26:0/832461001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8033a10 con 0x557e7a2c95b0 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 -- 172.21.15.26:0/832461001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 msgr2=0x557e7a2ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 0x557e7a2ce3c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f37a800b580 tx=0x7f37a8022000 comp rx=0 tx=0).stop 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 -- 172.21.15.26:0/832461001 shutdown_connections 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2cd720 0x557e7a2ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c8bf0 0x557e7a2bde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/832461001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2c95b0 0x557e7a2ce3c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 -- 172.21.15.26:0/832461001 >> 172.21.15.26:0/832461001 conn(0x557e7a1f4340 msgr2=0x557e7a2cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 -- 172.21.15.26:0/832461001 shutdown_connections 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.396+0000 7f37cfb23b80 1 -- 172.21.15.26:0/832461001 wait complete. 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 -- start start 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 Processor -- start 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a34bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01ae4a0 con 0x557e7a2cd720 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01af910 con 0x557e7a2c95b0 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01aeb70 con 0x557e7a2c8bf0 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37cfb23b80 10 client.0 _ll_register_callbacks cb 0x557e7a1b1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37be7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a34bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37be7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2e6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50848/0 (socket says 172.21.15.26:50848) 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37be7fc700 1 -- 172.21.15.26:0/685802463 learned_addr learned my addr 172.21.15.26:0/685802463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37beffd700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37bf7fe700 1 -- 172.21.15.26:0/685802463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 msgr2=0x557e7a2e5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37bf7fe700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2e5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37bf7fe700 1 -- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 msgr2=0x557e7a2e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.397+0000 7f37bf7fe700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37bf7fe700 1 -- 172.21.15.26:0/685802463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7a34c480 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37be7fc700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37bf7fe700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a34bf40 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x557e7a2cc360 tx=0x7f37b8014630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e52c5df9f43b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37a67fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37a67fc700 1 -- 172.21.15.26:0/685802463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8022060 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37a67fc700 1 -- 172.21.15.26:0/685802463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f37b8014db0 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a8003b10 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37a67fc700 1 -- 172.21.15.26:0/685802463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37b801d440 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37cfb23b80 10 client.19650 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e7a34d7f0 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.398+0000 7f37a67fc700 1 -- 172.21.15.26:0/685802463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a8003b10 con 0x557e7a2cd720 2024-05-20T21:09:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37a67fc700 1 -- 172.21.15.26:0/685802463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f37b8026030 con 0x557e7a2cd720 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37cfb23b80 10 client.19650 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37cfb23b80 10 client.19650 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37cfb23b80 20 client.19650 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a34df30 0x557e7a350350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37cfb23b80 4 client.19650 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e258fdd1", "release": true, "await": true}] 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e258fdd1", "release": true, "await": true}) -- 0x557e7a2225a0 con 0x557e7a34df30 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37beffd700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a34df30 0x557e7a350350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37beffd700 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a34df30 0x557e7a350350 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7402b644ec3c1b6 server_cookie=fc64a32c749ac564 in_seq=0 out_seq=0 2024-05-20T21:09:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.399+0000 7f37a67fc700 10 client.19650 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37a67fc700 1 -- 172.21.15.26:0/685802463 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557e7a2225a0 con 0x557e7a34df30 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37a67fc700 10 client.19650 handle_command_reply: tid=0 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 client.19650 shutdown 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 msgr2=0x557e7a34bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a34bf40 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x557e7a2cc360 tx=0x7f37b8014630 comp rx=0 tx=0).stop 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 shutdown_connections 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557e7a34df30 0x557e7a350350 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e7a2cd720 0x557e7a34bf40 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e7a2c8bf0 0x557e7a2e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37a67fc700 0 client.19650 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 --2- 172.21.15.26:0/685802463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e7a2c95b0 0x557e7a2e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 >> 172.21.15.26:0/685802463 conn(0x557e7a1f4340 msgr2=0x557e7a2ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 shutdown_connections 2024-05-20T21:09:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 1 -- 172.21.15.26:0/685802463 wait complete. 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.414+0000 7f37cfb23b80 20 client.19650 trim_cache size 0 max 16384 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 30, 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 52, 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "e258fdd1": { 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 52, 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:09:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:09:33.418 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37ac0b2880 msgr2=0x7f37ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37ac0b2880 0x7f37ac0b4d40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f37a8000c00 tx=0x7f37a8005fb0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 msgr2=0x7f37c0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 0x7f37c0075840 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f37b00060b0 tx=0x7f37b000d3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 shutdown_connections 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f37ac0b2880 0x7f37ac0b4d40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37c0103850 0x7f37c0075840 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37c0108920 0x7f37c00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 --2- 172.21.15.26:0/761791122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0104200 0x7f37c0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 >> 172.21.15.26:0/761791122 conn(0x7f37c007a5d0 msgr2=0x7f37c01019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 shutdown_connections 2024-05-20T21:09:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:33.415+0000 7f37c528d700 1 -- 172.21.15.26:0/761791122 wait complete. 2024-05-20T21:09:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:33 smithi055 ceph-mon[35213]: pgmap v792: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-20T21:09:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:33 smithi144 ceph-mon[35660]: pgmap v792: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-20T21:09:33.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295116 2.13 MB/sec execute 579 sec latency 29628.963 ms 2024-05-20T21:09:33.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:09:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3428885799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0101320 msgr2=0x7fb2f0101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3428885799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0101320 0x7fb2f0101740 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec02a410 comp rx=0 tx=0).stop 2024-05-20T21:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3428885799 shutdown_connections 2024-05-20T21:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3428885799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0102620 0x7fb2f0102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3428885799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f0101c80 0x7fb2f01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3428885799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0101320 0x7fb2f0101740 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3428885799 >> 172.21.15.26:0/3428885799 conn(0x7fb2f00fcf20 msgr2=0x7fb2f00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3428885799 shutdown_connections 2024-05-20T21:09:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3428885799 wait complete. 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.514+0000 7fb2f7c72640 1 Processor -- start 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 -- start start 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f0101320 0x7fb2f019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 0x7fb2f019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 0x7fb2f01a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01106c0 con 0x7fb2f0102620 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0110540 con 0x7fb2f0101c80 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f7c72640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0110840 con 0x7fb2f0101320 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 0x7fb2f01a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f520d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 0x7fb2f019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f520d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 0x7fb2f019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50864/0 (socket says 172.21.15.26:50864) 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 0x7fb2f01a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54744/0 (socket says 172.21.15.26:54744) 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f520d640 1 -- 172.21.15.26:0/3032586860 learned_addr learned my addr 172.21.15.26:0/3032586860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f0101320 0x7fb2f019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 -- 172.21.15.26:0/3032586860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f0101320 msgr2=0x7fb2f019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f0101320 0x7fb2f019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 -- 172.21.15.26:0/3032586860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 msgr2=0x7fb2f019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 0x7fb2f019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 -- 172.21.15.26:0/3032586860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f01a2990 con 0x7fb2f0102620 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f520d640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 0x7fb2f019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.515+0000 7fb2f620f640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 0x7fb2f01a2320 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009fd0 tx=0x7fb2e800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.516+0000 7fb2e6ffd640 1 -- 172.21.15.26:0/3032586860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800ce20 con 0x7fb2f0102620 2024-05-20T21:09:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.516+0000 7fb2e6ffd640 1 -- 172.21.15.26:0/3032586860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e8005020 con 0x7fb2f0102620 2024-05-20T21:09:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.516+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a2c80 con 0x7fb2f0102620 2024-05-20T21:09:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.516+0000 7fb2e6ffd640 1 -- 172.21.15.26:0/3032586860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8010690 con 0x7fb2f0102620 2024-05-20T21:09:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.516+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a3180 con 0x7fb2f0102620 2024-05-20T21:09:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.518+0000 7fb2fa98f740 1 Processor -- start 2024-05-20T21:09:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2e6ffd640 1 -- 172.21.15.26:0/3032586860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb2e8004a40 con 0x7fb2f0102620 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 -- start start 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 0x55cc01266ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012670b0 0x55cc0126bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc0126e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc011d7c30 con 0x55cc012666c0 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cc011c77d0 con 0x55cc0126c220 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc011cb810 con 0x55cc012670b0 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2f520d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 0x55cc01266ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.519+0000 7fb2f520d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 0x55cc01266ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54750/0 (socket says 172.21.15.26:54750) 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 -- 172.21.15.26:0/599909924 learned_addr learned my addr 172.21.15.26:0/599909924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc0126e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012670b0 0x55cc0126bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2e6ffd640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b28a0 0x7fb2dc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 -- 172.21.15.26:0/599909924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012670b0 msgr2=0x55cc0126bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012670b0 0x55cc0126bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 -- 172.21.15.26:0/599909924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 msgr2=0x55cc0126e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc0126e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 -- 172.21.15.26:0/599909924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc00edc210 con 0x55cc012666c0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2e6ffd640 1 -- 172.21.15.26:0/3032586860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2e80e29d0 con 0x7fb2f0102620 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f620f640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b28a0 0x7fb2dc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2f520d640 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 0x55cc01266ae0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0012c70 tx=0x7fb2e000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fdc83e10d5fd37 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2dbfff640 1 -- 172.21.15.26:0/599909924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0012d60 con 0x55cc012666c0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2dbfff640 1 -- 172.21.15.26:0/599909924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e00120d0 con 0x55cc012666c0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.520+0000 7fb2dbfff640 1 -- 172.21.15.26:0/599909924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00047a0 con 0x55cc012666c0 2024-05-20T21:09:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 -- 172.21.15.26:0/599909924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 msgr2=0x55cc01266ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 0x55cc01266ae0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0012c70 tx=0x7fb2e000bbe0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2f620f640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b28a0 0x7fb2dc0b4d60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80e0470 tx=0x7fb2e800c890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 -- 172.21.15.26:0/599909924 shutdown_connections 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc0126e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012670b0 0x55cc0126bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/599909924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012666c0 0x55cc01266ae0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 -- 172.21.15.26:0/599909924 >> 172.21.15.26:0/599909924 conn(0x55cc01261d30 msgr2=0x55cc01262130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 -- 172.21.15.26:0/599909924 shutdown_connections 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.521+0000 7fb2fa98f740 1 -- 172.21.15.26:0/599909924 wait complete. 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 -- start start 2024-05-20T21:09:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 Processor -- start 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01270790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01270cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01277ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc00ead790 con 0x55cc012670b0 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cc01183890 con 0x55cc012666c0 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc0126f170 con 0x55cc0126c220 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2fa98f740 10 client.0 _ll_register_callbacks cb 0x55cc00edba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2f5a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01270cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2f5a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01270cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54762/0 (socket says 172.21.15.26:54762) 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/3557118919 learned_addr learned my addr 172.21.15.26:0/3557118919 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.522+0000 7fb2f520d640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01270790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f520d640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01277ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/3557118919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 msgr2=0x55cc01277ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01277ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 msgr2=0x55cc01270790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01270790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/3557118919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc00fa8620 con 0x55cc012670b0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01270cd0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005ab0 tx=0x7fb2ec0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e742b614003d15 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2d97fa640 1 -- 172.21.15.26:0/3557118919 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec035040 con 0x55cc012670b0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e00120d0 con 0x55cc012670b0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2d97fa640 1 -- 172.21.15.26:0/3557118919 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2ec003ac0 con 0x55cc012670b0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2d97fa640 1 -- 172.21.15.26:0/3557118919 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2ec003cc0 con 0x55cc012670b0 2024-05-20T21:09:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2fa98f740 10 client.19662 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.523+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc0127acf0 con 0x55cc012670b0 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2d97fa640 1 -- 172.21.15.26:0/3557118919 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e00120d0 con 0x55cc012670b0 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2d97fa640 1 -- 172.21.15.26:0/3557118919 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fb2ec04e050 con 0x55cc012670b0 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2fa98f740 10 client.19662 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2fa98f740 10 client.19662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2fa98f740 20 client.19662 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc0127c370 0x55cc0127e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2fa98f740 4 client.19662 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc011d9b80 con 0x55cc0127c370 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2f520d640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc0127c370 0x55cc0127e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2f520d640 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc0127c370 0x55cc0127e7d0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787cfe2aca34859 server_cookie=76679479baed1af7 in_seq=0 out_seq=0 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.524+0000 7fb2d97fa640 10 client.19662 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2d97fa640 1 -- 172.21.15.26:0/3557118919 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cc011d9b80 con 0x55cc0127c370 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2d97fa640 10 client.19662 handle_command_reply: tid=0 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 client.19662 shutdown 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 msgr2=0x55cc01270cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01270cd0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005ab0 tx=0x7fb2ec0044f0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 shutdown_connections 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc0127c370 0x55cc0127e7d0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01277ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01270cd0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2d97fa640 0 client.19662 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/3557118919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01270790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 >> 172.21.15.26:0/3557118919 conn(0x55cc01261d30 msgr2=0x55cc01260fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 shutdown_connections 2024-05-20T21:09:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.525+0000 7fb2fa98f740 1 -- 172.21.15.26:0/3557118919 wait complete. 2024-05-20T21:09:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.526+0000 7fb2fa98f740 20 client.19662 trim_cache size 0 max 16384 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 Processor -- start 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 -- start start 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01260e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01277c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01283090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01ae980 con 0x55cc012670b0 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01ac250 con 0x55cc012666c0 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01aa0a0 con 0x55cc0126c220 2024-05-20T21:09:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f5a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01277c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f5a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01277c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54764/0 (socket says 172.21.15.26:54764) 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/1310453222 learned_addr learned my addr 172.21.15.26:0/1310453222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01260e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01283090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 -- 172.21.15.26:0/1310453222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 msgr2=0x55cc01283090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01283090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 -- 172.21.15.26:0/1310453222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 msgr2=0x55cc01277c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01277c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f520d640 1 -- 172.21.15.26:0/1310453222 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc01266300 con 0x55cc012666c0 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.542+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01277c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.543+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01260e30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0012c70 tx=0x7fb2e0022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af7788a9637efeac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.543+0000 7fb2db7fe640 1 -- 172.21.15.26:0/1310453222 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0034070 con 0x55cc012666c0 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.543+0000 7fb2db7fe640 1 -- 172.21.15.26:0/1310453222 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e0016040 con 0x55cc012666c0 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.543+0000 7fb2db7fe640 1 -- 172.21.15.26:0/1310453222 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004660 con 0x55cc012666c0 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.543+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1310453222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 msgr2=0x55cc01260e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.543+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01260e30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0012c70 tx=0x7fb2e0022040 comp rx=0 tx=0).stop 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1310453222 shutdown_connections 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc0126c220 0x55cc01283090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01277c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1310453222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc012666c0 0x55cc01260e30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1310453222 >> 172.21.15.26:0/1310453222 conn(0x55cc01261d30 msgr2=0x55cc01260720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1310453222 shutdown_connections 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1310453222 wait complete. 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- start start 2024-05-20T21:09:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 Processor -- start 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012666c0 0x55cc012750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01275610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc01276620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01aa710 con 0x55cc012670b0 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01abb80 con 0x55cc0126c220 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01aade0 con 0x55cc012666c0 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2fa98f740 10 client.0 _ll_register_callbacks cb 0x7fb2ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2f5a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01275610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.544+0000 7fb2f5a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01275610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54768/0 (socket says 172.21.15.26:54768) 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/1115872800 learned_addr learned my addr 172.21.15.26:0/1115872800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc01276620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012666c0 0x55cc012750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/1115872800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012666c0 msgr2=0x55cc012750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012666c0 0x55cc012750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 msgr2=0x55cc01276620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc01276620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 -- 172.21.15.26:0/1115872800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc01276c10 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012666c0 0x55cc012750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2f5a0e640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01275610 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db838db1458c3b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2d97fa640 1 -- 172.21.15.26:0/1115872800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec035070 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2d97fa640 1 -- 172.21.15.26:0/1115872800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2ec03dab0 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e0016040 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2d97fa640 1 -- 172.21.15.26:0/1115872800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2ec060050 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2fa98f740 10 client.19668 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.545+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc01277fc0 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2d97fa640 1 -- 172.21.15.26:0/1115872800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e0016040 con 0x55cc012670b0 2024-05-20T21:09:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2d97fa640 1 -- 172.21.15.26:0/1115872800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fb2ec04e040 con 0x55cc012670b0 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2fa98f740 10 client.19668 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2fa98f740 10 client.19668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2fa98f740 20 client.19668 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc012e7740 0x55cc012e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2fa98f740 4 client.19668 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc011d3360 con 0x55cc012e7740 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc012e7740 0x55cc012e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2d97fa640 10 client.19668 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.546+0000 7fb2f520d640 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc012e7740 0x55cc012e9b60 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb97f02f79f9ed6 server_cookie=e4079ad9ce73cb68 in_seq=0 out_seq=0 2024-05-20T21:09:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2d97fa640 1 -- 172.21.15.26:0/1115872800 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cc011d3360 con 0x55cc012e7740 2024-05-20T21:09:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2d97fa640 10 client.19668 handle_command_reply: tid=0 2024-05-20T21:09:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 client.19668 shutdown 2024-05-20T21:09:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 msgr2=0x55cc01275610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01275610 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec003ad0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 shutdown_connections 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55cc012e7740 0x55cc012e9b60 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cc0126c220 0x55cc01276620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc012670b0 0x55cc01275610 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 --2- 172.21.15.26:0/1115872800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc012666c0 0x55cc012750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2d97fa640 0 client.19668 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 >> 172.21.15.26:0/1115872800 conn(0x55cc01261d30 msgr2=0x55cc01260fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 shutdown_connections 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 1 -- 172.21.15.26:0/1115872800 wait complete. 2024-05-20T21:09:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.576+0000 7fb2fa98f740 20 client.19668 trim_cache size 0 max 16384 2024-05-20T21:09:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:09:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:09:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b28a0 msgr2=0x7fb2dc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b28a0 0x7fb2dc0b4d60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80e0470 tx=0x7fb2e800c890 comp rx=0 tx=0).stop 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 msgr2=0x7fb2f01a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 0x7fb2f01a2320 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009fd0 tx=0x7fb2e800ef10 comp rx=0 tx=0).stop 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 shutdown_connections 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2dc0b28a0 0x7fb2dc0b4d60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f0102620 0x7fb2f01a2320 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0101c80 0x7fb2f019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 --2- 172.21.15.26:0/3032586860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2f0101320 0x7fb2f019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 >> 172.21.15.26:0/3032586860 conn(0x7fb2f00fcf20 msgr2=0x7fb2f0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 shutdown_connections 2024-05-20T21:09:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:34.577+0000 7fb2f7c72640 1 -- 172.21.15.26:0/3032586860 wait complete. 2024-05-20T21:09:34.595 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295715 2.13 MB/sec execute 580 sec latency 26.685 ms 2024-05-20T21:09:34.750 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e258fdd1', total seconds elapsed: 40.8 2024-05-20T21:09:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:35 smithi055 ceph-mon[35213]: pgmap v793: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-05-20T21:09:35.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:35 smithi026 ceph-mon[31990]: pgmap v793: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-05-20T21:09:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:35 smithi144 ceph-mon[35660]: pgmap v793: 97 pgs: 97 active+clean; 24 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-05-20T21:09:35.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 296433 2.14 MB/sec execute 581 sec latency 17.398 ms 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 -- 172.21.15.26:0/2352761946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 msgr2=0x7f49a81028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 --2- 172.21.15.26:0/2352761946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a81028c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f499c009990 tx=0x7f499c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 -- 172.21.15.26:0/2352761946 shutdown_connections 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 --2- 172.21.15.26:0/2352761946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8102e00 0x7f49a810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 --2- 172.21.15.26:0/2352761946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8108520 0x7f49a8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 --2- 172.21.15.26:0/2352761946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a81028c0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 -- 172.21.15.26:0/2352761946 >> 172.21.15.26:0/2352761946 conn(0x7f49a80793f0 msgr2=0x7f49a8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 -- 172.21.15.26:0/2352761946 shutdown_connections 2024-05-20T21:09:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.591+0000 7f49ae652700 1 -- 172.21.15.26:0/2352761946 wait complete. 2024-05-20T21:09:36.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.592+0000 7f49ae652700 1 Processor -- start 2024-05-20T21:09:36.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.592+0000 7f49ae652700 1 -- start start 2024-05-20T21:09:36.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49ae652700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49ae652700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8102e00 0x7f49a819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49ae652700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8108520 0x7f49a8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49ae652700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81124a0 con 0x7f49a8102440 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49ae652700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112320 con 0x7f49a8108520 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49ae652700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112620 con 0x7f49a8102e00 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49acbef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8108520 0x7f49a8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8102e00 0x7f49a819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49acbef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8108520 0x7f49a8199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50940/0 (socket says 172.21.15.26:50940) 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8102e00 0x7f49a819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:55328/0 (socket says 172.21.15.26:55328) 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.593+0000 7f49acbef700 1 -- 172.21.15.26:0/452476514 learned_addr learned my addr 172.21.15.26:0/452476514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a7fff700 1 -- 172.21.15.26:0/452476514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8102e00 msgr2=0x7f49a819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a7fff700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8102e00 0x7f49a819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a7fff700 1 -- 172.21.15.26:0/452476514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8108520 msgr2=0x7f49a8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a7fff700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8108520 0x7f49a8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a7fff700 1 -- 172.21.15.26:0/452476514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a819a0c0 con 0x7f49a8102440 2024-05-20T21:09:36.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a7fff700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a819f7a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f499800c970 tx=0x7f499800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a57fa700 1 -- 172.21.15.26:0/452476514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499800e880 con 0x7f49a8102440 2024-05-20T21:09:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a819a3b0 con 0x7f49a8102440 2024-05-20T21:09:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a6670 con 0x7f49a8102440 2024-05-20T21:09:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a57fa700 1 -- 172.21.15.26:0/452476514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f499800f450 con 0x7f49a8102440 2024-05-20T21:09:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.594+0000 7f49a57fa700 1 -- 172.21.15.26:0/452476514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998018660 con 0x7f49a8102440 2024-05-20T21:09:36.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 297034 2.14 MB/sec execute 582 sec latency 21.830 ms 2024-05-20T21:09:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.597+0000 7f49a57fa700 1 -- 172.21.15.26:0/452476514 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f499800ea20 con 0x7f49a8102440 2024-05-20T21:09:36.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.598+0000 7f49a57fa700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49900b2880 0x7f49900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.598+0000 7f49a77fe700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49900b2880 0x7f49900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.598+0000 7f49a57fa700 1 -- 172.21.15.26:0/452476514 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f49980e1eb0 con 0x7f49a8102440 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.598+0000 7f49b8ee8b80 1 Processor -- start 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.598+0000 7f49b8ee8b80 1 -- start start 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a77fe700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49900b2880 0x7f49900b4d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f49a8079c60 tx=0x7f499c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b8beffed70 con 0x55b8bf0aeb50 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b8befee910 con 0x55b8bf0a08a0 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b8beff2950 con 0x55b8bf0a1260 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50950/0 (socket says 172.21.15.26:50950) 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a77fe700 1 -- 172.21.15.26:0/1276039894 learned_addr learned my addr 172.21.15.26:0/1276039894 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49acbef700 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 -- 172.21.15.26:0/1276039894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 msgr2=0x55b8bf0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 -- 172.21.15.26:0/1276039894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 msgr2=0x55b8bf0a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 -- 172.21.15.26:0/1276039894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8bef97fb0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a77fe700 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.599+0000 7f49a7fff700 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0aef30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f499800cfd0 tx=0x7f49980a9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b672d1657eca51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.600+0000 7f498e7fc700 1 -- 172.21.15.26:0/1276039894 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980ad820 con 0x55b8bf0aeb50 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.600+0000 7f498e7fc700 1 -- 172.21.15.26:0/1276039894 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49980b22d0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.600+0000 7f498e7fc700 1 -- 172.21.15.26:0/1276039894 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980b20a0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.600+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1276039894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 msgr2=0x55b8bf0aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.600+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0aef30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f499800cfd0 tx=0x7f49980a9b70 comp rx=0 tx=0).stop 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.600+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1276039894 shutdown_connections 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0aef30 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1276039894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1276039894 >> 172.21.15.26:0/1276039894 conn(0x55b8befcc240 msgr2=0x55b8bf09b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1276039894 shutdown_connections 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1276039894 wait complete. 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 -- start start 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.601+0000 7f49b8ee8b80 1 Processor -- start 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b8beef4fa0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b8bef56da0 con 0x55b8bf0a1260 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b8bf0b5ef0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49b8ee8b80 10 client.0 _ll_register_callbacks cb 0x55b8bef89950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54810/0 (socket says 172.21.15.26:54810) 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 -- 172.21.15.26:0/3186824340 learned_addr learned my addr 172.21.15.26:0/3186824340 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a77fe700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49acbef700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 -- 172.21.15.26:0/3186824340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0aeb50 msgr2=0x55b8bf0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 -- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a1260 msgr2=0x55b8bf0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a7fff700 1 -- 172.21.15.26:0/3186824340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8bef3e8d0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49acbef700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.602+0000 7f49a77fe700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49a7fff700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0b55c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f49980a54f0 tx=0x7f49980a99b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd64d0b1f61a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49877fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49877fe700 1 -- 172.21.15.26:0/3186824340 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980b2050 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49877fe700 1 -- 172.21.15.26:0/3186824340 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f49980b0ba0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49980b22d0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49877fe700 1 -- 172.21.15.26:0/3186824340 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49980b0dc0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49b8ee8b80 10 client.19680 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8befa8760 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49877fe700 1 -- 172.21.15.26:0/3186824340 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49980b22d0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.603+0000 7f49877fe700 1 -- 172.21.15.26:0/3186824340 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4998020030 con 0x55b8bf0a08a0 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49b8ee8b80 10 client.19680 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49b8ee8b80 10 client.19680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49b8ee8b80 20 client.19680 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf0c2b70 0x55b8bf0c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49b8ee8b80 4 client.19680 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8bf000cc0 con 0x55b8bf0c2b70 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49a77fe700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf0c2b70 0x55b8bf0c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49877fe700 10 client.19680 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.604+0000 7f49a77fe700 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf0c2b70 0x55b8bf0c4fb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e332a563174932cd server_cookie=883d712ed2afda3b in_seq=0 out_seq=0 2024-05-20T21:09:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49877fe700 1 -- 172.21.15.26:0/3186824340 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8bf000cc0 con 0x55b8bf0c2b70 2024-05-20T21:09:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49877fe700 10 client.19680 handle_command_reply: tid=0 2024-05-20T21:09:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 client.19680 shutdown 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 msgr2=0x55b8bf0b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0b55c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f49980a54f0 tx=0x7f49980a99b0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 shutdown_connections 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf0c2b70 0x55b8bf0c4fb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0b55c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3186824340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.605+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 >> 172.21.15.26:0/3186824340 conn(0x55b8befcc240 msgr2=0x55b8bf09b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.606+0000 7f49877fe700 0 client.19680 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.606+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 shutdown_connections 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.606+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3186824340 wait complete. 2024-05-20T21:09:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.606+0000 7f49b8ee8b80 20 client.19680 trim_cache size 0 max 16384 2024-05-20T21:09:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.625+0000 7f49b8ee8b80 1 Processor -- start 2024-05-20T21:09:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.625+0000 7f49b8ee8b80 1 -- start start 2024-05-20T21:09:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 0x55b8bf0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0aeb50 0x55b8bf09b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f499400fde0 con 0x55b8bf0a1260 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f499400d670 con 0x55b8bf0aeb50 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f499400b4c0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 0x55b8bf0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0aeb50 0x55b8bf09b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 0x55b8bf0adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54818/0 (socket says 172.21.15.26:54818) 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49a77fe700 1 -- 172.21.15.26:0/3595156088 learned_addr learned my addr 172.21.15.26:0/3595156088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49a7fff700 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 -- 172.21.15.26:0/3595156088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a08a0 msgr2=0x55b8bf0ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 -- 172.21.15.26:0/3595156088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 msgr2=0x55b8bf0adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 0x55b8bf0adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 -- 172.21.15.26:0/3595156088 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8bf09d0b0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49a77fe700 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 0x55b8bf0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:36.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.626+0000 7f49acbef700 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0aeb50 0x55b8bf09b270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009c10 tx=0x7f49a0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=495e995c1dd3ba73 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:36.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.627+0000 7f498dffb700 1 -- 172.21.15.26:0/3595156088 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0003a00 con 0x55b8bf0aeb50 2024-05-20T21:09:36.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.627+0000 7f498dffb700 1 -- 172.21.15.26:0/3595156088 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49a0003ba0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.627+0000 7f498dffb700 1 -- 172.21.15.26:0/3595156088 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00206e0 con 0x55b8bf0aeb50 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.627+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3595156088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0aeb50 msgr2=0x55b8bf09b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.627+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0aeb50 0x55b8bf09b270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009c10 tx=0x7f49a0009920 comp rx=0 tx=0).stop 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3595156088 shutdown_connections 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0aeb50 0x55b8bf09b270 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/3595156088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0a1260 0x55b8bf0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3595156088 >> 172.21.15.26:0/3595156088 conn(0x55b8befcc240 msgr2=0x55b8bf09f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3595156088 shutdown_connections 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/3595156088 wait complete. 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 -- start start 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 Processor -- start 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0bb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.628+0000 7f49b8ee8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f499400bb30 con 0x55b8bf0aeb50 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f499400cfa0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49b8ee8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f499400c200 con 0x55b8bf0a1260 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0bb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49b8ee8b80 10 client.0 _ll_register_callbacks cb 0x55b8bef89950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49acbef700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0bb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50984/0 (socket says 172.21.15.26:50984) 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 -- 172.21.15.26:0/1987362926 learned_addr learned my addr 172.21.15.26:0/1987362926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a77fe700 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 -- 172.21.15.26:0/1987362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 msgr2=0x55b8bf0b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 -- 172.21.15.26:0/1987362926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 msgr2=0x55b8bf0b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 -- 172.21.15.26:0/1987362926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8bf0b8c40 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49a7fff700 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0bb640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f499800f020 tx=0x7f49980a9030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9135465a3de074 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49877fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49877fe700 1 -- 172.21.15.26:0/1987362926 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980a84f0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a0003ba0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.629+0000 7f49877fe700 1 -- 172.21.15.26:0/1987362926 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f49980a4de0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49877fe700 1 -- 172.21.15.26:0/1987362926 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49980a5330 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 10 client.28889 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8bf0b8c40 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49877fe700 1 -- 172.21.15.26:0/1987362926 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49a0003ba0 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49877fe700 1 -- 172.21.15.26:0/1987362926 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4998020040 con 0x55b8bf0a08a0 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 10 client.28889 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 10 client.28889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 20 client.28889 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf12c820 0x55b8bf12ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 4 client.28889 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8beffa4a0 con 0x55b8bf12c820 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.630+0000 7f49a77fe700 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf12c820 0x55b8bf12ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49877fe700 10 client.28889 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49a77fe700 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf12c820 0x55b8bf12ec10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fc7dba9f52ddf64 server_cookie=5645c85c25b45daf in_seq=0 out_seq=0 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49877fe700 1 -- 172.21.15.26:0/1987362926 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8beffa4a0 con 0x55b8bf12c820 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49877fe700 10 client.28889 handle_command_reply: tid=0 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 client.28889 shutdown 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 msgr2=0x55b8bf0bb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0bb640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f499800f020 tx=0x7f49980a9030 comp rx=0 tx=0).stop 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 shutdown_connections 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55b8bf12c820 0x55b8bf12ec10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49877fe700 0 client.28889 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8bf0aeb50 0x55b8bf0b8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b8bf0a08a0 0x55b8bf0bb640 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 --2- 172.21.15.26:0/1987362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8bf0a1260 0x55b8bf0b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 >> 172.21.15.26:0/1987362926 conn(0x55b8befcc240 msgr2=0x55b8bf0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.631+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 shutdown_connections 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.632+0000 7f49b8ee8b80 1 -- 172.21.15.26:0/1987362926 wait complete. 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.632+0000 7f49b8ee8b80 20 client.28889 trim_cache size 0 max 16384 2024-05-20T21:09:36.636 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.632+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49900b2880 msgr2=0x7f49900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.632+0000 7f49ae652700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49900b2880 0x7f49900b4d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f49a8079c60 tx=0x7f499c005fb0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.632+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 msgr2=0x7f49a819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.632+0000 7f49ae652700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a819f7a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f499800c970 tx=0x7f499800cc80 comp rx=0 tx=0).stop 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 shutdown_connections 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f49900b2880 0x7f49900b4d40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a8102440 0x7f49a819f7a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49a8108520 0x7f49a8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 --2- 172.21.15.26:0/452476514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a8102e00 0x7f49a819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 >> 172.21.15.26:0/452476514 conn(0x7f49a80793f0 msgr2=0x7f49a806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 shutdown_connections 2024-05-20T21:09:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:36.633+0000 7f49ae652700 1 -- 172.21.15.26:0/452476514 wait complete. 2024-05-20T21:09:37.442 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:37 smithi026 ceph-mon[31990]: pgmap v794: 97 pgs: 97 active+clean; 24 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 116 KiB/s wr, 21 op/s 2024-05-20T21:09:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:37 smithi055 ceph-mon[35213]: pgmap v794: 97 pgs: 97 active+clean; 24 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 116 KiB/s wr, 21 op/s 2024-05-20T21:09:37.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:09:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:37 smithi144 ceph-mon[35660]: pgmap v794: 97 pgs: 97 active+clean; 24 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 116 KiB/s wr, 21 op/s 2024-05-20T21:09:37.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 297764 2.14 MB/sec execute 583 sec latency 15.291 ms 2024-05-20T21:09:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:38.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 298349 2.15 MB/sec execute 584 sec latency 18.138 ms 2024-05-20T21:09:39.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:39 smithi026 ceph-mon[31990]: pgmap v795: 97 pgs: 97 active+clean; 25 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 191 KiB/s wr, 28 op/s 2024-05-20T21:09:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:39 smithi055 ceph-mon[35213]: pgmap v795: 97 pgs: 97 active+clean; 25 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 191 KiB/s wr, 28 op/s 2024-05-20T21:09:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:39 smithi144 ceph-mon[35660]: pgmap v795: 97 pgs: 97 active+clean; 25 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 191 KiB/s wr, 28 op/s 2024-05-20T21:09:39.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 298801 2.15 MB/sec execute 585 sec latency 18.676 ms 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.636+0000 7f27e85aa700 1 -- 172.21.15.26:0/1910883686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102430 msgr2=0x7f27e01028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.636+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1910883686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102430 0x7f27e01028b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009990 tx=0x7f27d0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 -- 172.21.15.26:0/1910883686 shutdown_connections 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1910883686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102df0 0x7f27e010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1910883686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0108510 0x7f27e01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1910883686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102430 0x7f27e01028b0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 -- 172.21.15.26:0/1910883686 >> 172.21.15.26:0/1910883686 conn(0x7f27e00793b0 msgr2=0x7f27e00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 -- 172.21.15.26:0/1910883686 shutdown_connections 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.637+0000 7f27e85aa700 1 -- 172.21.15.26:0/1910883686 wait complete. 2024-05-20T21:09:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 Processor -- start 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 -- start start 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102430 0x7f27e019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0102df0 0x7f27e019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0108510 0x7f27e01a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0112470 con 0x7f27e0108510 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01122f0 con 0x7f27e0102430 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e85aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01125f0 con 0x7f27e0102df0 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e6346700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102430 0x7f27e019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e6346700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102430 0x7f27e019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50996/0 (socket says 172.21.15.26:50996) 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e6346700 1 -- 172.21.15.26:0/1511035589 learned_addr learned my addr 172.21.15.26:0/1511035589 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0108510 0x7f27e01a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.638+0000 7f27e5b45700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0102df0 0x7f27e019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e6b47700 1 -- 172.21.15.26:0/1511035589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0102df0 msgr2=0x7f27e019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0102df0 0x7f27e019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e6b47700 1 -- 172.21.15.26:0/1511035589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102430 msgr2=0x7f27e019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102430 0x7f27e019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e6b47700 1 -- 172.21.15.26:0/1511035589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e01a5f30 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0108510 0x7f27e01a59f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f27d800efc0 tx=0x7f27d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27d77fe700 1 -- 172.21.15.26:0/1511035589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80092e0 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27d77fe700 1 -- 172.21.15.26:0/1511035589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27d8004bb0 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e01a6100 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27d77fe700 1 -- 172.21.15.26:0/1511035589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8004020 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.639+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a6610 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.642+0000 7f27d77fe700 1 -- 172.21.15.26:0/1511035589 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f27d801b030 con 0x7f27e0108510 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.643+0000 7f27d77fe700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27cc0b2900 0x7f27cc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.643+0000 7f27e6346700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27cc0b2900 0x7f27cc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.643+0000 7f27d77fe700 1 -- 172.21.15.26:0/1511035589 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f27d80e1cc0 con 0x7f27e0108510 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.643+0000 7f27e6346700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27cc0b2900 0x7f27cc0b4dc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005950 tx=0x7f27dc00a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 Processor -- start 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 -- start start 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d457461050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457464d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d4574657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4573bbc90 con 0x55d457465370 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d4573ab830 con 0x55d457461620 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4573af870 con 0x55d457460c70 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27e6b47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d4574657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27e5b45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457464d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27e6b47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d4574657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54854/0 (socket says 172.21.15.26:54854) 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27e6b47700 1 -- 172.21.15.26:0/1861244714 learned_addr learned my addr 172.21.15.26:0/1861244714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27e5b45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457464d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51012/0 (socket says 172.21.15.26:51012) 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.644+0000 7f27e6346700 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d457461050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27e6b47700 1 -- 172.21.15.26:0/1861244714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 msgr2=0x55d457461050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d457461050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27e6b47700 1 -- 172.21.15.26:0/1861244714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 msgr2=0x55d457464d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457464d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27e6b47700 1 -- 172.21.15.26:0/1861244714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4572fb8d0 con 0x55d457465370 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27e6b47700 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d4574657b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f27d80a6580 tx=0x7f27d8004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4937786c0ca964eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27d4ff9700 1 -- 172.21.15.26:0/1861244714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80a9a10 con 0x55d457465370 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27d4ff9700 1 -- 172.21.15.26:0/1861244714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27d80a9310 con 0x55d457465370 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.645+0000 7f27d4ff9700 1 -- 172.21.15.26:0/1861244714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80ad3c0 con 0x55d457465370 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 -- 172.21.15.26:0/1861244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 msgr2=0x55d4574657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d4574657b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f27d80a6580 tx=0x7f27d8004e30 comp rx=0 tx=0).stop 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 -- 172.21.15.26:0/1861244714 shutdown_connections 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d4574657b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d457461050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/1861244714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457464d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 -- 172.21.15.26:0/1861244714 >> 172.21.15.26:0/1861244714 conn(0x55d45738a080 msgr2=0x55d457456e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 -- 172.21.15.26:0/1861244714 shutdown_connections 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 -- 172.21.15.26:0/1861244714 wait complete. 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.646+0000 7f27f2e40b80 1 -- start start 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 Processor -- start 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457460c70 0x55d45746c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 0x55d45746a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4572b1fa0 con 0x55d457461620 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d45734d7c0 con 0x55d457460c70 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d45745c8c0 con 0x55d457465370 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27f2e40b80 10 client.0 _ll_register_callbacks cb 0x55d457346920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 0x55d45746a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 0x55d45746a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54866/0 (socket says 172.21.15.26:54866) 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 -- 172.21.15.26:0/2780050357 learned_addr learned my addr 172.21.15.26:0/2780050357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e6346700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457460c70 0x55d45746c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e6b47700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 -- 172.21.15.26:0/2780050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 msgr2=0x55d45746a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 -- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457460c70 msgr2=0x55d45746c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457460c70 0x55d45746c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e5b45700 1 -- 172.21.15.26:0/2780050357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45734a690 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.647+0000 7f27e6346700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457460c70 0x55d45746c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27e5b45700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 0x55d45746a440 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009990 tx=0x7f27d00057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56aecfca834983 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27c9ffb700 1 -- 172.21.15.26:0/2780050357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0029060 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27c9ffb700 1 -- 172.21.15.26:0/2780050357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f27d000b6b0 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d80a9310 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27c9ffb700 1 -- 172.21.15.26:0/2780050357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27d000b8d0 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 10 client.19698 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d457473870 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27c9ffb700 1 -- 172.21.15.26:0/2780050357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27d80a9310 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27c9ffb700 1 -- 172.21.15.26:0/2780050357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f27d002d030 con 0x55d457461620 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 10 client.19698 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 20 client.19698 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d457475e00 0x55d4574781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.648+0000 7f27f2e40b80 4 client.19698 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.649+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4574289e0 con 0x55d457475e00 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.649+0000 7f27e6346700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d457475e00 0x55d4574781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.649+0000 7f27e6346700 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d457475e00 0x55d4574781f0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac414e7a360bdb25 server_cookie=e83605f4c4ad9680 in_seq=0 out_seq=0 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.649+0000 7f27c9ffb700 10 client.19698 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27c9ffb700 1 -- 172.21.15.26:0/2780050357 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d4574289e0 con 0x55d457475e00 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27c9ffb700 10 client.19698 handle_command_reply: tid=0 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 client.19698 shutdown 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 msgr2=0x55d45746a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 0x55d45746a440 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009990 tx=0x7f27d00057e0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 shutdown_connections 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d457475e00 0x55d4574781f0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457460c70 0x55d45746c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/2780050357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457461620 0x55d45746a440 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 >> 172.21.15.26:0/2780050357 conn(0x55d45738a080 msgr2=0x55d457456510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27c9ffb700 0 client.19698 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 shutdown_connections 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 1 -- 172.21.15.26:0/2780050357 wait complete. 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.650+0000 7f27f2e40b80 20 client.19698 trim_cache size 0 max 16384 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 Processor -- start 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 -- start start 2024-05-20T21:09:39.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 0x55d45745f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457456c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01ae350 con 0x55d457460c70 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01abbe0 con 0x55d457461620 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01a9a50 con 0x55d457465370 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27e6346700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 0x55d45745f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27e6346700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 0x55d45745f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59356/0 (socket says 172.21.15.26:59356) 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27e6346700 1 -- 172.21.15.26:0/3080642278 learned_addr learned my addr 172.21.15.26:0/3080642278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27e5b45700 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457456c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27e6b47700 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.668+0000 7f27e6346700 1 -- 172.21.15.26:0/3080642278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 msgr2=0x55d45746a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27e6346700 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27e6346700 1 -- 172.21.15.26:0/3080642278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 msgr2=0x55d457456c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27e6346700 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457456c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27e6346700 1 -- 172.21.15.26:0/3080642278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4574787d0 con 0x55d457460c70 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27e6b47700 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27e6346700 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 0x55d45745f6e0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00aa10 tx=0x7f27dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348821165848cd30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27cbfff700 1 -- 172.21.15.26:0/3080642278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc016070 con 0x55d457460c70 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27cbfff700 1 -- 172.21.15.26:0/3080642278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27dc01ee10 con 0x55d457460c70 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.669+0000 7f27cbfff700 1 -- 172.21.15.26:0/3080642278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc005280 con 0x55d457460c70 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3080642278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 msgr2=0x55d45745f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 0x55d45745f6e0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00aa10 tx=0x7f27dc012000 comp rx=0 tx=0).stop 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3080642278 shutdown_connections 2024-05-20T21:09:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457460c70 0x55d45745f6e0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457465370 0x55d45746a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3080642278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457456c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3080642278 >> 172.21.15.26:0/3080642278 conn(0x55d45738a080 msgr2=0x55d457464550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3080642278 shutdown_connections 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3080642278 wait complete. 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 -- start start 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.670+0000 7f27f2e40b80 1 Processor -- start 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d4574727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457472d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d45745d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01aa0a0 con 0x55d457465370 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01ab510 con 0x55d457461620 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01aa770 con 0x55d457460c70 2024-05-20T21:09:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27f2e40b80 10 client.0 _ll_register_callbacks cb 0x55d457346920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d45745d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d45745d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59366/0 (socket says 172.21.15.26:59366) 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 -- 172.21.15.26:0/3798931118 learned_addr learned my addr 172.21.15.26:0/3798931118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e5b45700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457472d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6346700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d4574727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 -- 172.21.15.26:0/3798931118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 msgr2=0x55d4574727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d4574727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 -- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 msgr2=0x55d457472d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457472d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 -- 172.21.15.26:0/3798931118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45745dad0 con 0x55d457465370 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6346700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d4574727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27e6b47700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d45745d290 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ef90 tx=0x7f27d800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7fd00610659091 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.671+0000 7f27c9ffb700 1 -- 172.21.15.26:0/3798931118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80a9310 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27c9ffb700 1 -- 172.21.15.26:0/3798931118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f27d80ad370 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27dc01ee10 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27c9ffb700 1 -- 172.21.15.26:0/3798931118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27d80a9a10 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 10 client.19704 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4574e0430 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27c9ffb700 1 -- 172.21.15.26:0/3798931118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27dc01ee10 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27c9ffb700 1 -- 172.21.15.26:0/3798931118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f27d808f2c0 con 0x55d457465370 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 10 client.19704 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 10 client.19704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 20 client.19704 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d4574e06b0 0x55d4574e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 4 client.19704 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4573bdbe0 con 0x55d4574e06b0 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.672+0000 7f27e6346700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d4574e06b0 0x55d4574e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27e6346700 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d4574e06b0 0x55d4574e2aa0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c44d36d04179bcf server_cookie=56fac432f6e42f8f in_seq=0 out_seq=0 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27c9ffb700 10 client.19704 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27c9ffb700 1 -- 172.21.15.26:0/3798931118 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4573bdbe0 con 0x55d4574e06b0 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27c9ffb700 10 client.19704 handle_command_reply: tid=0 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 client.19704 shutdown 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 msgr2=0x55d45745d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d45745d290 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ef90 tx=0x7f27d800c9b0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 shutdown_connections 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d4574e06b0 0x55d4574e2aa0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d457465370 0x55d45745d290 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d457460c70 0x55d4574727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 --2- 172.21.15.26:0/3798931118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d457461620 0x55d457472d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 >> 172.21.15.26:0/3798931118 conn(0x55d45738a080 msgr2=0x55d45745b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27c9ffb700 0 client.19704 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 shutdown_connections 2024-05-20T21:09:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 1 -- 172.21.15.26:0/3798931118 wait complete. 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.673+0000 7f27f2e40b80 20 client.19704 trim_cache size 0 max 16384 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27cc0b2900 msgr2=0x7f27cc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27cc0b2900 0x7f27cc0b4dc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005950 tx=0x7f27dc00a400 comp rx=0 tx=0).stop 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0108510 msgr2=0x7f27e01a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0108510 0x7f27e01a59f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f27d800efc0 tx=0x7f27d800c3a0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 shutdown_connections 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f27cc0b2900 0x7f27cc0b4dc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0108510 0x7f27e01a59f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27e0102430 0x7f27e019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 --2- 172.21.15.26:0/1511035589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0102df0 0x7f27e019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 >> 172.21.15.26:0/1511035589 conn(0x7f27e00793b0 msgr2=0x7f27e0100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 shutdown_connections 2024-05-20T21:09:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:39.674+0000 7f27e85aa700 1 -- 172.21.15.26:0/1511035589 wait complete. 2024-05-20T21:09:40.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 299470 2.15 MB/sec execute 586 sec latency 17.820 ms 2024-05-20T21:09:41.405 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:09:41.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:09:41.430 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:41 smithi026 ceph-mon[31990]: pgmap v796: 97 pgs: 97 active+clean; 28 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 588 KiB/s wr, 52 op/s 2024-05-20T21:09:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:41 smithi055 ceph-mon[35213]: pgmap v796: 97 pgs: 97 active+clean; 28 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 588 KiB/s wr, 52 op/s 2024-05-20T21:09:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:41 smithi144 ceph-mon[35660]: pgmap v796: 97 pgs: 97 active+clean; 28 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 588 KiB/s wr, 52 op/s 2024-05-20T21:09:41.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 300182 2.15 MB/sec execute 587 sec latency 17.091 ms 2024-05-20T21:09:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:42.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 300946 2.16 MB/sec execute 588 sec latency 18.023 ms 2024-05-20T21:09:43.431 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:43 smithi026 ceph-mon[31990]: pgmap v797: 97 pgs: 97 active+clean; 28 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 857 KiB/s wr, 60 op/s 2024-05-20T21:09:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:43 smithi055 ceph-mon[35213]: pgmap v797: 97 pgs: 97 active+clean; 28 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 857 KiB/s wr, 60 op/s 2024-05-20T21:09:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:43 smithi144 ceph-mon[35660]: pgmap v797: 97 pgs: 97 active+clean; 28 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 857 KiB/s wr, 60 op/s 2024-05-20T21:09:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 -- 172.21.15.26:0/2684505796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 msgr2=0x7f9cfc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2684505796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc1028b0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009990 tx=0x7f9ce4009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 -- 172.21.15.26:0/2684505796 shutdown_connections 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2684505796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 0x7f9cfc10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2684505796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 0x7f9cfc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2684505796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc1028b0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 -- 172.21.15.26:0/2684505796 >> 172.21.15.26:0/2684505796 conn(0x7f9cfc0793b0 msgr2=0x7f9cfc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 -- 172.21.15.26:0/2684505796 shutdown_connections 2024-05-20T21:09:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.560+0000 7f9d019d1700 1 -- 172.21.15.26:0/2684505796 wait complete. 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 Processor -- start 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 -- start start 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 0x7f9cfc1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 0x7f9cfc104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1124c0 con 0x7f9cfc102430 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc112340 con 0x7f9cfc102df0 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.561+0000 7f9d019d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc112640 con 0x7f9cfc108510 2024-05-20T21:09:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59380/0 (socket says 172.21.15.26:59380) 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2883081915 learned_addr learned my addr 172.21.15.26:0/2883081915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 0x7f9cfc1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 0x7f9cfc104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2883081915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 msgr2=0x7f9cfc104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 0x7f9cfc104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2883081915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 msgr2=0x7f9cfc1045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 0x7f9cfc1045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2883081915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc1a8030 con 0x7f9cfc102430 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 0x7f9cfc1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 0x7f9cfc104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc104070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00c9b0 tx=0x7f9cec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.562+0000 7f9d009cf700 1 -- 172.21.15.26:0/2883081915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec007b00 con 0x7f9cfc102430 2024-05-20T21:09:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.563+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc1a8320 con 0x7f9cfc102430 2024-05-20T21:09:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.563+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc1a8880 con 0x7f9cfc102430 2024-05-20T21:09:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.563+0000 7f9d009cf700 1 -- 172.21.15.26:0/2883081915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cec007ca0 con 0x7f9cfc102430 2024-05-20T21:09:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.563+0000 7f9d009cf700 1 -- 172.21.15.26:0/2883081915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec018660 con 0x7f9cfc102430 2024-05-20T21:09:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.566+0000 7f9d009cf700 1 -- 172.21.15.26:0/2883081915 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9cec00f450 con 0x7f9cfc102430 2024-05-20T21:09:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.566+0000 7f9d009cf700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9ce80b2900 0x7f9ce80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9ce80b2900 0x7f9ce80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d009cf700 1 -- 172.21.15.26:0/2883081915 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9cec0e2040 con 0x7f9cfc102430 2024-05-20T21:09:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 Processor -- start 2024-05-20T21:09:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 -- start start 2024-05-20T21:09:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 0x562826c73420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562826bcdd10 con 0x562826c72fe0 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562826bbd8b0 con 0x562826c6f290 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562826bc18f0 con 0x562826c6e8e0 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9ce80b2900 0x7f9ce80b4dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc105b00 tx=0x7f9ce4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 0x562826c73420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 0x562826c73420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59382/0 (socket says 172.21.15.26:59382) 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43878/0 (socket says 172.21.15.26:43878) 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.567+0000 7f9cfb7fe700 1 -- 172.21.15.26:0/1554948912 learned_addr learned my addr 172.21.15.26:0/1554948912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf3fff700 1 -- 172.21.15.26:0/1554948912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 msgr2=0x562826c6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf3fff700 1 -- 172.21.15.26:0/1554948912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 msgr2=0x562826c73420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 0x562826c73420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf3fff700 1 -- 172.21.15.26:0/1554948912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562826b0d950 con 0x562826c6f290 2024-05-20T21:09:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 0x562826c73420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c729c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b650 tx=0x7f9ce401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55a03d6fbffeef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.568+0000 7f9cf1ffb700 1 -- 172.21.15.26:0/1554948912 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4031070 con 0x562826c6f290 2024-05-20T21:09:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9cf1ffb700 1 -- 172.21.15.26:0/1554948912 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ce4005690 con 0x562826c6f290 2024-05-20T21:09:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9cf1ffb700 1 -- 172.21.15.26:0/1554948912 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400fdf0 con 0x562826c6f290 2024-05-20T21:09:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 -- 172.21.15.26:0/1554948912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 msgr2=0x562826c729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c729c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b650 tx=0x7f9ce401f000 comp rx=0 tx=0).stop 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 -- 172.21.15.26:0/1554948912 shutdown_connections 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c72fe0 0x562826c73420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/1554948912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c729c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.569+0000 7f9d0c267b80 1 -- 172.21.15.26:0/1554948912 >> 172.21.15.26:0/1554948912 conn(0x562826b9c100 msgr2=0x562826c68f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.570+0000 7f9d0c267b80 1 -- 172.21.15.26:0/1554948912 shutdown_connections 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.570+0000 7f9d0c267b80 1 -- 172.21.15.26:0/1554948912 wait complete. 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.570+0000 7f9d0c267b80 1 -- start start 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.570+0000 7f9d0c267b80 1 Processor -- start 2024-05-20T21:09:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6e8e0 0x562826c7a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826c777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c72fe0 0x562826c77cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562826ac4020 con 0x562826c6f290 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562826b5f840 con 0x562826c6e8e0 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562826c7b140 con 0x562826c72fe0 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9d0c267b80 10 client.0 _ll_register_callbacks cb 0x562826b589a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826c777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826c777b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59398/0 (socket says 172.21.15.26:59398) 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 -- 172.21.15.26:0/2442474295 learned_addr learned my addr 172.21.15.26:0/2442474295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c72fe0 0x562826c77cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6e8e0 0x562826c7a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 -- 172.21.15.26:0/2442474295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c72fe0 msgr2=0x562826c77cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c72fe0 0x562826c77cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 -- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6e8e0 msgr2=0x562826c7a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6e8e0 0x562826c7a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 -- 172.21.15.26:0/2442474295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562826b5c710 con 0x562826c6f290 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c72fe0 0x562826c77cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826c777b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009ce0 tx=0x7f9ce400f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809db175df6a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.571+0000 7f9cdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9cdaffd700 1 -- 172.21.15.26:0/2442474295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4031070 con 0x562826c6f290 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9cdaffd700 1 -- 172.21.15.26:0/2442474295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9ce402c410 con 0x562826c6f290 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce4005690 con 0x562826c6f290 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9cdaffd700 1 -- 172.21.15.26:0/2442474295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ce402c650 con 0x562826c6f290 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 10 client.19716 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:43.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562826c85920 con 0x562826c6f290 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9cdaffd700 1 -- 172.21.15.26:0/2442474295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ce4005690 con 0x562826c6f290 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9cdaffd700 1 -- 172.21.15.26:0/2442474295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9ce4022830 con 0x562826c6f290 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 10 client.19716 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 10 client.19716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 20 client.19716 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c87df0 0x562826c8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 4 client.19716 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.572+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562826c3aa60 con 0x562826c87df0 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.573+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c87df0 0x562826c8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.573+0000 7f9cdaffd700 10 client.19716 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.573+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c87df0 0x562826c8a1e0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73261823c79c6ec server_cookie=59ffd00b6b6968bd in_seq=0 out_seq=0 2024-05-20T21:09:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.574+0000 7f9cdaffd700 1 -- 172.21.15.26:0/2442474295 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562826c3aa60 con 0x562826c87df0 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.574+0000 7f9cdaffd700 10 client.19716 handle_command_reply: tid=0 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.574+0000 7f9d0c267b80 1 client.19716 shutdown 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 msgr2=0x562826c777b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826c777b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009ce0 tx=0x7f9ce400f7c0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 shutdown_connections 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c87df0 0x562826c8a1e0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c72fe0 0x562826c77cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6e8e0 0x562826c7a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2442474295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826c777b0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9cdaffd700 0 client.19716 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 >> 172.21.15.26:0/2442474295 conn(0x562826b9c100 msgr2=0x562826c743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 shutdown_connections 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2442474295 wait complete. 2024-05-20T21:09:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.575+0000 7f9d0c267b80 20 client.19716 trim_cache size 0 max 16384 2024-05-20T21:09:43.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 301552 2.16 MB/sec execute 589 sec latency 16.797 ms 2024-05-20T21:09:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 Processor -- start 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 -- start start 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 0x562826c76730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00fde0 con 0x562826b9c100 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00d670 con 0x562826c6f290 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00b4c0 con 0x562826c6e8e0 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 0x562826c76730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 0x562826c76730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59410/0 (socket says 172.21.15.26:59410) 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2395198021 learned_addr learned my addr 172.21.15.26:0/2395198021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2395198021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 msgr2=0x562826c796f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c796f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2395198021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 msgr2=0x562826c8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c8ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.600+0000 7f9cfaffd700 1 -- 172.21.15.26:0/2395198021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562826c690d0 con 0x562826b9c100 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.601+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 0x562826c76730 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0a9630 tx=0x7f9cec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ba4471b8e0947e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.601+0000 7f9cf17fa700 1 -- 172.21.15.26:0/2395198021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0a96a0 con 0x562826b9c100 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.601+0000 7f9cf17fa700 1 -- 172.21.15.26:0/2395198021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cec0ada50 con 0x562826b9c100 2024-05-20T21:09:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.601+0000 7f9cf17fa700 1 -- 172.21.15.26:0/2395198021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0a9900 con 0x562826b9c100 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.601+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2395198021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 msgr2=0x562826c76730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 0x562826c76730 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0a9630 tx=0x7f9cec000bc0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2395198021 shutdown_connections 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826b9c100 0x562826c76730 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826c6f290 0x562826c8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/2395198021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826c796f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2395198021 >> 172.21.15.26:0/2395198021 conn(0x562826c77370 msgr2=0x562826c78fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2395198021 shutdown_connections 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 -- 172.21.15.26:0/2395198021 wait complete. 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 -- start start 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 Processor -- start 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 0x562826cec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.602+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826ceca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9d0c267b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826ce6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00bb30 con 0x562826c6f290 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00cfa0 con 0x562826b9c100 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9d0c267b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00c200 con 0x562826c6e8e0 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9d0c267b80 10 client.0 _ll_register_callbacks cb 0x562826b589a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 0x562826cec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826ce6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 0x562826cec510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43894/0 (socket says 172.21.15.26:43894) 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfaffd700 1 -- 172.21.15.26:0/3913912958 learned_addr learned my addr 172.21.15.26:0/3913912958 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cf3fff700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826ceca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 -- 172.21.15.26:0/3913912958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 msgr2=0x562826ceca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826ceca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 -- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 msgr2=0x562826cec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 0x562826cec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 -- 172.21.15.26:0/3913912958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562826ce7080 con 0x562826c6f290 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 0x562826cec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cfb7fe700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826ce6980 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x562826c7a230 tx=0x7f9cf4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6b15f1832f4b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.603+0000 7f9cdaffd700 1 -- 172.21.15.26:0/3913912958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004db0 con 0x562826c6f290 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9cdaffd700 1 -- 172.21.15.26:0/3913912958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9cf40176e0 con 0x562826c6f290 2024-05-20T21:09:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cec0ada50 con 0x562826c6f290 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9cdaffd700 1 -- 172.21.15.26:0/3913912958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9cf40178e0 con 0x562826c6f290 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 10 client.19722 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562826cf3de0 con 0x562826c6f290 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9cdaffd700 1 -- 172.21.15.26:0/3913912958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cec0ada50 con 0x562826c6f290 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9cdaffd700 1 -- 172.21.15.26:0/3913912958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f9cf4029bb0 con 0x562826c6f290 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 10 client.19722 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 10 client.19722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 20 client.19722 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c72fe0 0x562826cf65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 4 client.19722 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562826bcfc60 con 0x562826c72fe0 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.604+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c72fe0 0x562826cf65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.605+0000 7f9cdaffd700 10 client.19722 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.605+0000 7f9cfaffd700 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c72fe0 0x562826cf65b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d540426786ff14 server_cookie=6a29a8d40170fe6c in_seq=0 out_seq=0 2024-05-20T21:09:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9cdaffd700 1 -- 172.21.15.26:0/3913912958 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562826bcfc60 con 0x562826c72fe0 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9cdaffd700 10 client.19722 handle_command_reply: tid=0 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 client.19722 shutdown 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 msgr2=0x562826ce6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826ce6980 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x562826c7a230 tx=0x7f9cf4009510 comp rx=0 tx=0).stop 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 shutdown_connections 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562826c72fe0 0x562826cf65b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562826c6f290 0x562826ce6980 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562826b9c100 0x562826cec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9cdaffd700 0 client.19722 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 --2- 172.21.15.26:0/3913912958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562826c6e8e0 0x562826ceca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 >> 172.21.15.26:0/3913912958 conn(0x562826c77370 msgr2=0x562826c83cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 shutdown_connections 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 1 -- 172.21.15.26:0/3913912958 wait complete. 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.607+0000 7f9d0c267b80 20 client.19722 trim_cache size 0 max 16384 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "4240b57d-5661-4aab-bfe0-7fc470080d54", 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:09:43.611 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9ce80b2900 msgr2=0x7f9ce80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9ce80b2900 0x7f9ce80b4dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc105b00 tx=0x7f9ce4005fb0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 msgr2=0x7f9cfc104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc104070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00c9b0 tx=0x7f9cec00ccc0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 shutdown_connections 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9ce80b2900 0x7f9ce80b4dc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc102430 0x7f9cfc104070 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cfc108510 0x7f9cfc104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 --2- 172.21.15.26:0/2883081915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cfc102df0 0x7f9cfc1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 >> 172.21.15.26:0/2883081915 conn(0x7f9cfc0793b0 msgr2=0x7f9cfc06be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 shutdown_connections 2024-05-20T21:09:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:43.608+0000 7f9d019d1700 1 -- 172.21.15.26:0/2883081915 wait complete. 2024-05-20T21:09:44.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:09:44.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 302021 2.16 MB/sec execute 590 sec latency 18.769 ms 2024-05-20T21:09:44.762 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:45.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:45 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:09:45.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:45 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:45.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:45 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:09:45.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:45 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:45.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:45 smithi026 ceph-mon[31990]: pgmap v798: 97 pgs: 97 active+clean; 27 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 973 KiB/s wr, 68 op/s 2024-05-20T21:09:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:45 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:09:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:45 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:45 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:09:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:45 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:45.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:45 smithi055 ceph-mon[35213]: pgmap v798: 97 pgs: 97 active+clean; 27 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 973 KiB/s wr, 68 op/s 2024-05-20T21:09:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:45 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:09:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:45 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:45 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:09:45.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:45 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:09:45.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:45 smithi144 ceph-mon[35660]: pgmap v798: 97 pgs: 97 active+clean; 27 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 973 KiB/s wr, 68 op/s 2024-05-20T21:09:45.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 302631 2.16 MB/sec execute 591 sec latency 28.835 ms 2024-05-20T21:09:46.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 303235 2.16 MB/sec execute 592 sec latency 61.393 ms 2024-05-20T21:09:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:47 smithi055 ceph-mon[35213]: pgmap v799: 97 pgs: 97 active+clean; 28 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-05-20T21:09:47.487 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:47 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:09:47.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:47 smithi026 ceph-mon[31990]: pgmap v799: 97 pgs: 97 active+clean; 28 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-05-20T21:09:47.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:47 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:09:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:47 smithi144 ceph-mon[35660]: pgmap v799: 97 pgs: 97 active+clean; 28 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-05-20T21:09:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:47 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:09:47.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 303829 2.16 MB/sec execute 593 sec latency 89.421 ms 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 -- 172.21.15.26:0/2960647929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b501019b0 msgr2=0x7f7b50101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 --2- 172.21.15.26:0/2960647929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b501019b0 0x7f7b50101dd0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009900 tx=0x7f7b4002a410 comp rx=0 tx=0).stop 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 -- 172.21.15.26:0/2960647929 shutdown_connections 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 --2- 172.21.15.26:0/2960647929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b50103560 0x7f7b50109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 --2- 172.21.15.26:0/2960647929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b50102ba0 0x7f7b50103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 --2- 172.21.15.26:0/2960647929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b501019b0 0x7f7b50101dd0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.672+0000 7f7b554d9640 1 -- 172.21.15.26:0/2960647929 >> 172.21.15.26:0/2960647929 conn(0x7f7b500fcf20 msgr2=0x7f7b500ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 -- 172.21.15.26:0/2960647929 shutdown_connections 2024-05-20T21:09:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 -- 172.21.15.26:0/2960647929 wait complete. 2024-05-20T21:09:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 Processor -- start 2024-05-20T21:09:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 -- start start 2024-05-20T21:09:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b501019b0 0x7f7b50198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 0x7f7b50199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b50103560 0x7f7b501a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010d390 con 0x7f7b50102ba0 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010d210 con 0x7f7b501019b0 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.673+0000 7f7b554d9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010d510 con 0x7f7b50103560 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 0x7f7b50199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b501019b0 0x7f7b50198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 0x7f7b50199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59442/0 (socket says 172.21.15.26:59442) 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b47fff640 1 -- 172.21.15.26:0/1689838175 learned_addr learned my addr 172.21.15.26:0/1689838175 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b50103560 0x7f7b501a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 -- 172.21.15.26:0/1689838175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b50103560 msgr2=0x7f7b501a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b50103560 0x7f7b501a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 -- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 msgr2=0x7f7b50199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 0x7f7b50199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 -- 172.21.15.26:0/1689838175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b501a0800 con 0x7f7b501019b0 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b47fff640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 0x7f7b50199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4effd640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b501019b0 0x7f7b50198be0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009230 tx=0x7f7b400040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4cff9640 1 -- 172.21.15.26:0/1689838175 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4003f070 con 0x7f7b501019b0 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b501a0a90 con 0x7f7b501019b0 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4cff9640 1 -- 172.21.15.26:0/1689838175 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b400044c0 con 0x7f7b501019b0 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.674+0000 7f7b4cff9640 1 -- 172.21.15.26:0/1689838175 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40035070 con 0x7f7b501019b0 2024-05-20T21:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.675+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b501a0fd0 con 0x7f7b501019b0 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.677+0000 7f7b581f1740 1 Processor -- start 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.677+0000 7f7b4cff9640 1 -- 172.21.15.26:0/1689838175 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7b400037f0 con 0x7f7b501019b0 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.677+0000 7f7b581f1740 1 -- start start 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.677+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b39ad0 0x557897b39ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.677+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3a4c0 0x557897b3efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3f630 0x557897b41ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b581f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557897aa6c30 con 0x557897b3a4c0 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b581f1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557897a967d0 con 0x557897b3f630 2024-05-20T21:09:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b581f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557897a9a810 con 0x557897b39ad0 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3a4c0 0x557897b3efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3f630 0x557897b41ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3a4c0 0x557897b3efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59450/0 (socket says 172.21.15.26:59450) 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b4effd640 1 -- 172.21.15.26:0/3467081488 learned_addr learned my addr 172.21.15.26:0/3467081488 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b39ad0 0x557897b39ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 -- 172.21.15.26:0/3467081488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b39ad0 msgr2=0x557897b39ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b39ad0 0x557897b39ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 -- 172.21.15.26:0/3467081488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3a4c0 msgr2=0x557897b3efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3a4c0 0x557897b3efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 -- 172.21.15.26:0/3467081488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578977ab210 con 0x557897b3f630 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b47fff640 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3f630 0x557897b41ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fd0 tx=0x7f7b380119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6072272c4e076a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.678+0000 7f7b4cff9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b3c0b28a0 0x7f7b3c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b457fa640 1 -- 172.21.15.26:0/3467081488 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38005020 con 0x557897b3f630 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b457fa640 1 -- 172.21.15.26:0/3467081488 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b380047a0 con 0x557897b3f630 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b457fa640 1 -- 172.21.15.26:0/3467081488 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800f8e0 con 0x557897b3f630 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b3c0b28a0 0x7f7b3c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b4cff9640 1 -- 172.21.15.26:0/1689838175 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7b401025e0 con 0x7f7b501019b0 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b3c0b28a0 0x7f7b3c0b4d60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x557897b3ad20 tx=0x7f7b480085f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b581f1740 1 -- 172.21.15.26:0/3467081488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3f630 msgr2=0x557897b41ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.679+0000 7f7b581f1740 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3f630 0x557897b41ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fd0 tx=0x7f7b380119a0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- 172.21.15.26:0/3467081488 shutdown_connections 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3f630 0x557897b41ac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3a4c0 0x557897b3efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 --2- 172.21.15.26:0/3467081488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b39ad0 0x557897b39ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- 172.21.15.26:0/3467081488 >> 172.21.15.26:0/3467081488 conn(0x557897b30d30 msgr2=0x557897b31130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- 172.21.15.26:0/3467081488 shutdown_connections 2024-05-20T21:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- 172.21.15.26:0/3467081488 wait complete. 2024-05-20T21:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- start start 2024-05-20T21:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 Processor -- start 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 0x557897b43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3a4c0 0x557897b44170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3f630 0x557897b4b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55789777c790 con 0x557897b39ad0 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557897a52890 con 0x557897b3a4c0 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557897b425e0 con 0x557897b3f630 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.680+0000 7f7b581f1740 10 client.0 _ll_register_callbacks cb 0x5578977aaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 0x557897b43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 0x557897b43c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59466/0 (socket says 172.21.15.26:59466) 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 -- 172.21.15.26:0/1084515072 learned_addr learned my addr 172.21.15.26:0/1084515072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b47fff640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3a4c0 0x557897b44170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3f630 0x557897b4b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 -- 172.21.15.26:0/1084515072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3f630 msgr2=0x557897b4b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3f630 0x557897b4b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 -- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3a4c0 msgr2=0x557897b44170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3a4c0 0x557897b44170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 -- 172.21.15.26:0/1084515072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557897877620 con 0x557897b39ad0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3f630 0x557897b4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b4effd640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 0x557897b43c30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f7b401000d0 tx=0x7f7b4003b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b683e0403de29c8d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b367fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b367fc640 1 -- 172.21.15.26:0/1084515072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40048040 con 0x557897b39ad0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b367fc640 1 -- 172.21.15.26:0/1084515072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7b40003ae0 con 0x557897b39ad0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b380047a0 con 0x557897b39ad0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b367fc640 1 -- 172.21.15.26:0/1084515072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b400c94f0 con 0x557897b39ad0 2024-05-20T21:09:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b581f1740 10 client.19734 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.681+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557897b4e190 con 0x557897b39ad0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b367fc640 1 -- 172.21.15.26:0/1084515072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b380047a0 con 0x557897b39ad0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b367fc640 1 -- 172.21.15.26:0/1084515072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7b4004b060 con 0x557897b39ad0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b581f1740 10 client.19734 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b581f1740 10 client.19734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b581f1740 20 client.19734 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b581f1740 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897b4f8c0 0x557897b51d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b581f1740 4 client.19734 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557897aa8b80 con 0x557897b4f8c0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b47fff640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897b4f8c0 0x557897b51d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.682+0000 7f7b47fff640 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897b4f8c0 0x557897b51d10 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7e958e8d233e7f server_cookie=3476f0440754f8c5 in_seq=0 out_seq=0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b367fc640 10 client.19734 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b367fc640 1 -- 172.21.15.26:0/1084515072 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557897aa8b80 con 0x557897b4f8c0 2024-05-20T21:09:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b367fc640 10 client.19734 handle_command_reply: tid=0 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 client.19734 shutdown 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 msgr2=0x557897b43c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 0x557897b43c30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f7b401000d0 tx=0x7f7b4003b690 comp rx=0 tx=0).stop 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 shutdown_connections 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897b4f8c0 0x557897b51d10 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3f630 0x557897b4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b3a4c0 0x557897b44170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 --2- 172.21.15.26:0/1084515072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b39ad0 0x557897b43c30 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 >> 172.21.15.26:0/1084515072 conn(0x557897b30d30 msgr2=0x557897b2ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b367fc640 0 client.19734 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.683+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 shutdown_connections 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.684+0000 7f7b581f1740 1 -- 172.21.15.26:0/1084515072 wait complete. 2024-05-20T21:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.684+0000 7f7b581f1740 20 client.19734 trim_cache size 0 max 16384 2024-05-20T21:09:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.702+0000 7f7b581f1740 1 Processor -- start 2024-05-20T21:09:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.702+0000 7f7b581f1740 1 -- start start 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.702+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.702+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b3bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.702+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b581f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501aa5f0 con 0x557897b3f630 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b581f1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a7ec0 con 0x557897b39ad0 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b581f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a5d10 con 0x557897b3a4c0 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b4b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59478/0 (socket says 172.21.15.26:59478) 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4f7fe640 1 -- 172.21.15.26:0/294449416 learned_addr learned my addr 172.21.15.26:0/294449416 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b47fff640 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b3bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 -- 172.21.15.26:0/294449416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 msgr2=0x557897b3bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b3bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 -- 172.21.15.26:0/294449416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 msgr2=0x557897b4b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b4b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 -- 172.21.15.26:0/294449416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557897b34c20 con 0x557897b39ad0 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b47fff640 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b4effd640 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b3b740 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b400c5c20 tx=0x7f7b40009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e89d4dddbfce74 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b44ff9640 1 -- 172.21.15.26:0/294449416 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40048070 con 0x557897b39ad0 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b44ff9640 1 -- 172.21.15.26:0/294449416 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b400eae50 con 0x557897b39ad0 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.703+0000 7f7b44ff9640 1 -- 172.21.15.26:0/294449416 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b400c5330 con 0x557897b39ad0 2024-05-20T21:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 -- 172.21.15.26:0/294449416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 msgr2=0x557897b3b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b3b740 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b400c5c20 tx=0x7f7b40009260 comp rx=0 tx=0).stop 2024-05-20T21:09:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 -- 172.21.15.26:0/294449416 shutdown_connections 2024-05-20T21:09:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 --2- 172.21.15.26:0/294449416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b3b740 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 -- 172.21.15.26:0/294449416 >> 172.21.15.26:0/294449416 conn(0x557897b30d30 msgr2=0x557897b2be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 -- 172.21.15.26:0/294449416 shutdown_connections 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.704+0000 7f7b581f1740 1 -- 172.21.15.26:0/294449416 wait complete. 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 -- start start 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 Processor -- start 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b48290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b49770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a6380 con 0x557897b3f630 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a77f0 con 0x557897b39ad0 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a6a50 con 0x557897b3a4c0 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b581f1740 10 client.0 _ll_register_callbacks cb 0x7f7b40005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b48290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b48290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43950/0 (socket says 172.21.15.26:43950) 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 -- 172.21.15.26:0/2697476527 learned_addr learned my addr 172.21.15.26:0/2697476527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b49770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b47fff640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 -- 172.21.15.26:0/2697476527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 msgr2=0x557897b487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 -- 172.21.15.26:0/2697476527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 msgr2=0x557897b49770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.705+0000 7f7b4effd640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b49770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b4effd640 1 -- 172.21.15.26:0/2697476527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557897b49d60 con 0x557897b39ad0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b4f7fe640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b49770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b4effd640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b48290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7b401003f0 tx=0x7f7b400c9bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=267bedc70a803b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b367fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b367fc640 1 -- 172.21.15.26:0/2697476527 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40048070 con 0x557897b39ad0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b367fc640 1 -- 172.21.15.26:0/2697476527 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7b400c6190 con 0x557897b39ad0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b400eae50 con 0x557897b39ad0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b367fc640 1 -- 172.21.15.26:0/2697476527 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b400c9310 con 0x557897b39ad0 2024-05-20T21:09:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b581f1740 10 client.28943 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557897b49d60 con 0x557897b39ad0 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b367fc640 1 -- 172.21.15.26:0/2697476527 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b400e3070 con 0x557897b39ad0 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b367fc640 1 -- 172.21.15.26:0/2697476527 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f7b4003bb80 con 0x557897b39ad0 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b581f1740 10 client.28943 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b581f1740 10 client.28943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.706+0000 7f7b581f1740 20 client.28943 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.707+0000 7f7b581f1740 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897bba4f0 0x557897bbc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.707+0000 7f7b581f1740 4 client.28943 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.707+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557897aa2360 con 0x557897bba4f0 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.707+0000 7f7b47fff640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897bba4f0 0x557897bbc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.707+0000 7f7b47fff640 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897bba4f0 0x557897bbc8e0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4194f0e42db682 server_cookie=2b378393e6617469 in_seq=0 out_seq=0 2024-05-20T21:09:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.707+0000 7f7b367fc640 10 client.28943 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b367fc640 1 -- 172.21.15.26:0/2697476527 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557897aa2360 con 0x557897bba4f0 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b367fc640 10 client.28943 handle_command_reply: tid=0 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 client.28943 shutdown 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 msgr2=0x557897b48290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b48290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7b401003f0 tx=0x7f7b400c9bf0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 shutdown_connections 2024-05-20T21:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x557897bba4f0 0x557897bbc8e0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b367fc640 0 client.28943 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557897b3f630 0x557897b49770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557897b3a4c0 0x557897b487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 --2- 172.21.15.26:0/2697476527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557897b39ad0 0x557897b48290 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 >> 172.21.15.26:0/2697476527 conn(0x557897b30d30 msgr2=0x557897b3b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 shutdown_connections 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.752+0000 7f7b581f1740 1 -- 172.21.15.26:0/2697476527 wait complete. 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b581f1740 20 client.28943 trim_cache size 0 max 16384 2024-05-20T21:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b3c0b28a0 msgr2=0x7f7b3c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b3c0b28a0 0x7f7b3c0b4d60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x557897b3ad20 tx=0x7f7b480085f0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b501019b0 msgr2=0x7f7b50198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b501019b0 0x7f7b50198be0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009230 tx=0x7f7b400040c0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 shutdown_connections 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7b3c0b28a0 0x7f7b3c0b4d60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b50103560 0x7f7b501a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b50102ba0 0x7f7b50199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 --2- 172.21.15.26:0/1689838175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b501019b0 0x7f7b50198be0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 >> 172.21.15.26:0/1689838175 conn(0x7f7b500fcf20 msgr2=0x7f7b5006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 shutdown_connections 2024-05-20T21:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:09:47.753+0000 7f7b554d9640 1 -- 172.21.15.26:0/1689838175 wait complete. 2024-05-20T21:09:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 -- 172.21.15.26:0/2696453413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2406bbe0 msgr2=0x7f4e241103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/2696453413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2406bbe0 0x7f4e241103d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00b0a0 tx=0x7f4e1c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 -- 172.21.15.26:0/2696453413 shutdown_connections 2024-05-20T21:09:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/2696453413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2406bbe0 0x7f4e241103d0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/2696453413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e241095b0 0x7f4e24109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/2696453413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 0x7f4e2406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 -- 172.21.15.26:0/2696453413 >> 172.21.15.26:0/2696453413 conn(0x7f4e240793f0 msgr2=0x7f4e24079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 -- 172.21.15.26:0/2696453413 shutdown_connections 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.349+0000 7f4e28ca4700 1 -- 172.21.15.26:0/2696453413 wait complete. 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 Processor -- start 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 -- start start 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 0x7f4e2419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2406bbe0 0x7f4e2419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 0x7f4e241998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241135c0 con 0x7f4e241095b0 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24113440 con 0x7f4e2406bbe0 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.350+0000 7f4e28ca4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24113740 con 0x7f4e2406b220 2024-05-20T21:09:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 0x7f4e241998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 0x7f4e241998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59502/0 (socket says 172.21.15.26:59502) 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 -- 172.21.15.26:0/4061623257 learned_addr learned my addr 172.21.15.26:0/4061623257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e2259c700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 0x7f4e2419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2406bbe0 0x7f4e2419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 -- 172.21.15.26:0/4061623257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 msgr2=0x7f4e2419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 0x7f4e2419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 -- 172.21.15.26:0/4061623257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2406bbe0 msgr2=0x7f4e2419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2406bbe0 0x7f4e2419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 -- 172.21.15.26:0/4061623257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2419a020 con 0x7f4e241095b0 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e2259c700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 0x7f4e2419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2406bbe0 0x7f4e2419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 0x7f4e241998c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00b070 tx=0x7f4e1c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e137fe700 1 -- 172.21.15.26:0/4061623257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c021070 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2419a2b0 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e137fe700 1 -- 172.21.15.26:0/4061623257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c007b80 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.351+0000 7f4e137fe700 1 -- 172.21.15.26:0/4061623257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004650 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.352+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241a65d0 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.354+0000 7f4e137fe700 1 -- 172.21.15.26:0/4061623257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4e1c01d450 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e137fe700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e0c0b2880 0x7f4e0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 Processor -- start 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e137fe700 1 -- 172.21.15.26:0/4061623257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4e1c00f020 con 0x7f4e241095b0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e2259c700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e0c0b2880 0x7f4e0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 -- start start 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 0x564db2c0e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564db2b69c90 con 0x564db2c0ef60 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564db2b59830 con 0x564db2c0b210 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.356+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564db2b5d870 con 0x564db2c0a860 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e21d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 0x564db2c0e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e2259c700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e0c0b2880 0x7f4e0c0b4d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4e2419aa00 tx=0x7f4e14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e21d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 0x564db2c0e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43974/0 (socket says 172.21.15.26:43974) 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e2259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e21d9b700 1 -- 172.21.15.26:0/3282848439 learned_addr learned my addr 172.21.15.26:0/3282848439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 -- 172.21.15.26:0/3282848439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 msgr2=0x564db2c0ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c0ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 -- 172.21.15.26:0/3282848439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 msgr2=0x564db2c0e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 0x564db2c0e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 -- 172.21.15.26:0/3282848439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db2aa98d0 con 0x564db2c0ef60 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e2259c700 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c0ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 0x564db2c0e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0f3a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0a7f00 tx=0x7f4e1c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165ef78f102214ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e10ff9700 1 -- 172.21.15.26:0/3282848439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0b33a0 con 0x564db2c0ef60 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e10ff9700 1 -- 172.21.15.26:0/3282848439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c0aa7a0 con 0x564db2c0ef60 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.357+0000 7f4e10ff9700 1 -- 172.21.15.26:0/3282848439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0b3a60 con 0x564db2c0ef60 2024-05-20T21:09:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 -- 172.21.15.26:0/3282848439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 msgr2=0x564db2c0f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0f3a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0a7f00 tx=0x7f4e1c004ee0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 -- 172.21.15.26:0/3282848439 shutdown_connections 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0f3a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c0ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/3282848439 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0b210 0x564db2c0e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 -- 172.21.15.26:0/3282848439 >> 172.21.15.26:0/3282848439 conn(0x564db2b38080 msgr2=0x564db2c04e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 -- 172.21.15.26:0/3282848439 shutdown_connections 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.358+0000 7f4e3353ab80 1 -- 172.21.15.26:0/3282848439 wait complete. 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 -- start start 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 Processor -- start 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0ef60 0x564db2c13c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564db2a5ffa0 con 0x564db2c0b210 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564db2afb7c0 con 0x564db2c0a860 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564db2c16ee0 con 0x564db2c0ef60 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e3353ab80 10 client.0 _ll_register_callbacks cb 0x564db2af4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e2259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e21d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e21d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59522/0 (socket says 172.21.15.26:59522) 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e21d9b700 1 -- 172.21.15.26:0/812547106 learned_addr learned my addr 172.21.15.26:0/812547106 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e2259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c163f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43980/0 (socket says 172.21.15.26:43980) 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.359+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0ef60 0x564db2c13c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e2259c700 1 -- 172.21.15.26:0/812547106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0ef60 msgr2=0x564db2c13c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e2259c700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0ef60 0x564db2c13c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e2259c700 1 -- 172.21.15.26:0/812547106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 msgr2=0x564db2c136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e2259c700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e2259c700 1 -- 172.21.15.26:0/812547106 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db2af8690 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0ef60 0x564db2c13c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e2259c700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c163f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4e140039b0 tx=0x7f4e140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4205ee911ecc733c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e09ffb700 1 -- 172.21.15.26:0/812547106 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1401fcd0 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c0aa7a0 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e09ffb700 1 -- 172.21.15.26:0/812547106 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4e140273e0 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e09ffb700 1 -- 172.21.15.26:0/812547106 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e14003e30 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e3353ab80 10 client.28955 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564db2c21620 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.360+0000 7f4e09ffb700 1 -- 172.21.15.26:0/812547106 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c0aa7a0 con 0x564db2c0a860 2024-05-20T21:09:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e09ffb700 1 -- 172.21.15.26:0/812547106 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4e14030020 con 0x564db2c0a860 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e3353ab80 10 client.28955 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e3353ab80 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e3353ab80 20 client.28955 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c23c50 0x564db2c260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e3353ab80 4 client.28955 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564db2bd69e0 con 0x564db2c23c50 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c23c50 0x564db2c260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c23c50 0x564db2c260b0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdc3605074c8bb3 server_cookie=7ea14bb3245e8686 in_seq=0 out_seq=0 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.361+0000 7f4e09ffb700 10 client.28955 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e09ffb700 1 -- 172.21.15.26:0/812547106 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564db2bd69e0 con 0x564db2c23c50 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e09ffb700 10 client.28955 handle_command_reply: tid=0 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 client.28955 shutdown 2024-05-20T21:09:48.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 msgr2=0x564db2c163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c163f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4e140039b0 tx=0x7f4e140047c0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 shutdown_connections 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c23c50 0x564db2c260b0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0ef60 0x564db2c13c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c163f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/812547106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 >> 172.21.15.26:0/812547106 conn(0x564db2b38080 msgr2=0x564db2c10350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e09ffb700 0 client.28955 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 shutdown_connections 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.362+0000 7f4e3353ab80 1 -- 172.21.15.26:0/812547106 wait complete. 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.363+0000 7f4e3353ab80 20 client.28955 trim_cache size 0 max 16384 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.384+0000 7f4e3353ab80 1 Processor -- start 2024-05-20T21:09:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 -- start start 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c2b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c2bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 0x564db2c09670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241ae200 con 0x564db2c0b210 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241aba90 con 0x564db2c0ef60 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241a98e0 con 0x564db2c0a860 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 0x564db2c09670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 0x564db2c09670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:43990/0 (socket says 172.21.15.26:43990) 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1156494173 learned_addr learned my addr 172.21.15.26:0/1156494173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c2bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e2259c700 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c2b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1156494173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 msgr2=0x564db2c2b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c2b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1156494173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 msgr2=0x564db2c2bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c2bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1156494173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db2c14670 con 0x564db2c0ef60 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.385+0000 7f4e2259c700 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c2b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.386+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 0x564db2c09670 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0e1130 tx=0x7f4e1c0a64f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4493f28d887ceb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.386+0000 7f4e0bfff700 1 -- 172.21.15.26:0/1156494173 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0c9020 con 0x564db2c0ef60 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.386+0000 7f4e0bfff700 1 -- 172.21.15.26:0/1156494173 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c0aa7a0 con 0x564db2c0ef60 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.386+0000 7f4e0bfff700 1 -- 172.21.15.26:0/1156494173 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0af040 con 0x564db2c0ef60 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.386+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1156494173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 msgr2=0x564db2c09670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.386+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 0x564db2c09670 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0e1130 tx=0x7f4e1c0a64f0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1156494173 shutdown_connections 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0ef60 0x564db2c09670 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0a860 0x564db2c2b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1156494173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0b210 0x564db2c2bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1156494173 >> 172.21.15.26:0/1156494173 conn(0x564db2b38080 msgr2=0x564db2c11fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1156494173 shutdown_connections 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1156494173 wait complete. 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- start start 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 Processor -- start 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c1c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0b210 0x564db2c0c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241a9f50 con 0x564db2c0ef60 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241ab3c0 con 0x564db2c0a860 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241aa620 con 0x564db2c0b210 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e3353ab80 10 client.0 _ll_register_callbacks cb 0x564db2af4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.387+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59550/0 (socket says 172.21.15.26:59550) 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e2259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c1c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1396487738 learned_addr learned my addr 172.21.15.26:0/1396487738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0b210 0x564db2c0c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1396487738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0b210 msgr2=0x564db2c0c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0b210 0x564db2c0c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 msgr2=0x564db2c1c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c1c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 -- 172.21.15.26:0/1396487738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db2c0ced0 con 0x564db2c0ef60 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e2259c700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c1c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e21d9b700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0b210 0x564db2c0c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e22d9d700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0c790 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d010 tx=0x7f4e1c0e1160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ae82cf9d65ba29 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e09ffb700 1 -- 172.21.15.26:0/1396487738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0c9050 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e09ffb700 1 -- 172.21.15.26:0/1396487738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4e1c0a7280 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c0aa7a0 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e09ffb700 1 -- 172.21.15.26:0/1396487738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c0b3a60 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e3353ab80 10 client.19758 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.388+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564db2c8e7b0 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e09ffb700 1 -- 172.21.15.26:0/1396487738 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c0aa7a0 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e09ffb700 1 -- 172.21.15.26:0/1396487738 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4e1c011070 con 0x564db2c0ef60 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e3353ab80 10 client.19758 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e3353ab80 10 client.19758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e3353ab80 20 client.19758 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c8eb20 0x564db2c90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e3353ab80 4 client.19758 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:09:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564db2b6bbe0 con 0x564db2c8eb20 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.389+0000 7f4e2259c700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c8eb20 0x564db2c90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.390+0000 7f4e2259c700 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c8eb20 0x564db2c90f10 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb26623cbd85da8 server_cookie=74546bebd3a2b5ca in_seq=0 out_seq=0 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.390+0000 7f4e09ffb700 10 client.19758 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.390+0000 7f4e09ffb700 1 -- 172.21.15.26:0/1396487738 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564db2b6bbe0 con 0x564db2c8eb20 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.390+0000 7f4e09ffb700 10 client.19758 handle_command_reply: tid=0 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.390+0000 7f4e3353ab80 1 client.19758 shutdown 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 msgr2=0x564db2c0c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0c790 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d010 tx=0x7f4e1c0e1160 comp rx=0 tx=0).stop 2024-05-20T21:09:48.485 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 shutdown_connections 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564db2c8eb20 0x564db2c90f10 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564db2c0ef60 0x564db2c0c790 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db2c0a860 0x564db2c1c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e09ffb700 0 client.19758 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 --2- 172.21.15.26:0/1396487738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db2c0b210 0x564db2c0c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 >> 172.21.15.26:0/1396487738 conn(0x564db2b38080 msgr2=0x564db2c09670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 shutdown_connections 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 1 -- 172.21.15.26:0/1396487738 wait complete. 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e3353ab80 20 client.19758 trim_cache size 0 max 16384 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e0c0b2880 msgr2=0x7f4e0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e0c0b2880 0x7f4e0c0b4d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4e2419aa00 tx=0x7f4e14009260 comp rx=0 tx=0).stop 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 msgr2=0x7f4e241998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.391+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 0x7f4e241998c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00b070 tx=0x7f4e1c004c80 comp rx=0 tx=0).stop 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 shutdown_connections 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e0c0b2880 0x7f4e0c0b4d40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e241095b0 0x7f4e241998c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e2406b220 0x7f4e2419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 --2- 172.21.15.26:0/4061623257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2406bbe0 0x7f4e2419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 >> 172.21.15.26:0/4061623257 conn(0x7f4e240793f0 msgr2=0x7f4e241010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 shutdown_connections 2024-05-20T21:09:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:48.392+0000 7f4e28ca4700 1 -- 172.21.15.26:0/4061623257 wait complete. 2024-05-20T21:09:48.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 304519 2.17 MB/sec execute 594 sec latency 28.159 ms 2024-05-20T21:09:49.258 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:49 smithi026 ceph-mon[31990]: pgmap v800: 97 pgs: 97 active+clean; 29 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-05-20T21:09:49.288 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:09:49.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:09:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:49 smithi055 ceph-mon[35213]: pgmap v800: 97 pgs: 97 active+clean; 29 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-05-20T21:09:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:49 smithi144 ceph-mon[35660]: pgmap v800: 97 pgs: 97 active+clean; 29 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-05-20T21:09:49.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 305176 2.17 MB/sec execute 595 sec latency 12.628 ms 2024-05-20T21:09:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:50.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 305612 2.17 MB/sec execute 596 sec latency 10.801 ms 2024-05-20T21:09:51.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:51 smithi026 ceph-mon[31990]: pgmap v801: 97 pgs: 97 active+clean; 26 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-05-20T21:09:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:51 smithi055 ceph-mon[35213]: pgmap v801: 97 pgs: 97 active+clean; 26 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-05-20T21:09:51.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 -- 172.21.15.26:0/2654865475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 msgr2=0x7f182c105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 --2- 172.21.15.26:0/2654865475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c105c00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1820009990 tx=0x7f1820009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 -- 172.21.15.26:0/2654865475 shutdown_connections 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 --2- 172.21.15.26:0/2654865475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c106140 0x7f182c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 --2- 172.21.15.26:0/2654865475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f182c10b780 0x7f182c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 --2- 172.21.15.26:0/2654865475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c105c00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.512+0000 7f18318f1700 1 -- 172.21.15.26:0/2654865475 >> 172.21.15.26:0/2654865475 conn(0x7f182c101270 msgr2=0x7f182c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.513+0000 7f18318f1700 1 -- 172.21.15.26:0/2654865475 shutdown_connections 2024-05-20T21:09:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.513+0000 7f18318f1700 1 -- 172.21.15.26:0/2654865475 wait complete. 2024-05-20T21:09:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.513+0000 7f18318f1700 1 Processor -- start 2024-05-20T21:09:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 -- start start 2024-05-20T21:09:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f182c106140 0x7f182c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c10b780 0x7f182c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f182c115870 con 0x7f182c105780 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1156f0 con 0x7f182c10b780 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f18318f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1159f0 con 0x7f182c106140 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f182affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f182affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60968/0 (socket says 172.21.15.26:60968) 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f182affd700 1 -- 172.21.15.26:0/1287252471 learned_addr learned my addr 172.21.15.26:0/1287252471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f182b7fe700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c10b780 0x7f182c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.514+0000 7f182a7fc700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f182c106140 0x7f182c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182b7fe700 1 -- 172.21.15.26:0/1287252471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f182c106140 msgr2=0x7f182c1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182b7fe700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f182c106140 0x7f182c1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182b7fe700 1 -- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 msgr2=0x7f182c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182b7fe700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182b7fe700 1 -- 172.21.15.26:0/1287252471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c19c410 con 0x7f182c10b780 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182affd700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f182b7fe700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c10b780 0x7f182c19bb30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f182400efb0 tx=0x7f182400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1824003a00 con 0x7f182c10b780 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f182400ce30 con 0x7f182c10b780 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182c19c700 con 0x7f182c10b780 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182c1a89e0 con 0x7f182c10b780 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.515+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18240046b0 con 0x7f182c10b780 2024-05-20T21:09:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.516+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182c106ec0 con 0x7f182c10b780 2024-05-20T21:09:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.518+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1824012050 con 0x7f182c10b780 2024-05-20T21:09:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.519+0000 7f18308ef700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18140b2880 0x7f18140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.519+0000 7f182affd700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18140b2880 0x7f18140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.519+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f18240e1bc0 con 0x7f182c10b780 2024-05-20T21:09:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.519+0000 7f182affd700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18140b2880 0x7f18140b4d40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f181c005950 tx=0x7f181c00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.523+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1824022060 con 0x7f182c10b780 2024-05-20T21:09:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:51 smithi144 ceph-mon[35660]: pgmap v801: 97 pgs: 97 active+clean; 26 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-05-20T21:09:51.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 306207 2.17 MB/sec execute 597 sec latency 30.193 ms 2024-05-20T21:09:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.833+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f182c19cff0 con 0x7f182c10b780 2024-05-20T21:09:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.834+0000 7f18308ef700 1 -- 172.21.15.26:0/1287252471 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11868 (secure 0 0 0) 0x7f1824010040 con 0x7f182c10b780 2024-05-20T21:09:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:09:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:03:14.092261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.836+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18140b2880 msgr2=0x7f18140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.836+0000 7f18318f1700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18140b2880 0x7f18140b4d40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f181c005950 tx=0x7f181c00b500 comp rx=0 tx=0).stop 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.836+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c10b780 msgr2=0x7f182c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.836+0000 7f18318f1700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c10b780 0x7f182c19bb30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f182400efb0 tx=0x7f182400c3a0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 shutdown_connections 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f18140b2880 0x7f18140b4d40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c105780 0x7f182c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f182c10b780 0x7f182c19bb30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 --2- 172.21.15.26:0/1287252471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f182c106140 0x7f182c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 >> 172.21.15.26:0/1287252471 conn(0x7f182c101270 msgr2=0x7f182c101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 shutdown_connections 2024-05-20T21:09:51.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:51.837+0000 7f18318f1700 1 -- 172.21.15.26:0/1287252471 wait complete. 2024-05-20T21:09:51.838 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-05-20T21:09:52.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:52 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1287252471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:52.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:52 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1287252471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:52.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:52 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1287252471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:09:52.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 306912 2.17 MB/sec execute 598 sec latency 19.318 ms 2024-05-20T21:09:52.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:09:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:53.332 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:53 smithi026 ceph-mon[31990]: pgmap v802: 97 pgs: 97 active+clean; 26 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:09:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:53 smithi055 ceph-mon[35213]: pgmap v802: 97 pgs: 97 active+clean; 26 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:09:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:53 smithi144 ceph-mon[35660]: pgmap v802: 97 pgs: 97 active+clean; 26 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-05-20T21:09:53.596 INFO:tasks.workunit.client.0.smithi026.stdout: 1 307551 2.17 MB/sec execute 599 sec latency 15.413 ms 2024-05-20T21:09:54.599 INFO:tasks.workunit.client.0.smithi026.stdout: 1 cleanup 600 sec 2024-05-20T21:09:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.822+0000 7fdab0918700 1 -- 172.21.15.26:0/1757057879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac1095a0 msgr2=0x7fdaac109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.822+0000 7fdab0918700 1 --2- 172.21.15.26:0/1757057879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac1095a0 0x7fdaac109980 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009990 tx=0x7fda9c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 -- 172.21.15.26:0/1757057879 shutdown_connections 2024-05-20T21:09:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 --2- 172.21.15.26:0/1757057879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac1095a0 0x7fdaac109980 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 --2- 172.21.15.26:0/1757057879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdaac06c470 0x7fdaac1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 --2- 172.21.15.26:0/1757057879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaac06bab0 0x7fdaac06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 -- 172.21.15.26:0/1757057879 >> 172.21.15.26:0/1757057879 conn(0x7fdaac0793b0 msgr2=0x7fdaac0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 -- 172.21.15.26:0/1757057879 shutdown_connections 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 -- 172.21.15.26:0/1757057879 wait complete. 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.823+0000 7fdab0918700 1 Processor -- start 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 -- start start 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 0x7fdaac19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdaac06c470 0x7fdaac19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaac1095a0 0x7fdaac1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1135e0 con 0x7fdaac06bab0 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac113460 con 0x7fdaac06c470 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdab0918700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac113760 con 0x7fdaac1095a0 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 0x7fdaac19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 0x7fdaac19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60982/0 (socket says 172.21.15.26:60982) 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 -- 172.21.15.26:0/2622213872 learned_addr learned my addr 172.21.15.26:0/2622213872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdaac06c470 0x7fdaac19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 -- 172.21.15.26:0/2622213872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaac1095a0 msgr2=0x7fdaac1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaac1095a0 0x7fdaac1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 -- 172.21.15.26:0/2622213872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdaac06c470 msgr2=0x7fdaac19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdaac06c470 0x7fdaac19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.824+0000 7fdaaa59c700 1 -- 172.21.15.26:0/2622213872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac1a0e00 con 0x7fdaac06bab0 2024-05-20T21:09:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.825+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 0x7fdaac19bdf0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.825+0000 7fda9b7fe700 1 -- 172.21.15.26:0/2622213872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c027070 con 0x7fdaac06bab0 2024-05-20T21:09:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.825+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a1090 con 0x7fdaac06bab0 2024-05-20T21:09:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.825+0000 7fda9b7fe700 1 -- 172.21.15.26:0/2622213872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda9c022470 con 0x7fdaac06bab0 2024-05-20T21:09:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.825+0000 7fda9b7fe700 1 -- 172.21.15.26:0/2622213872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c017650 con 0x7fdaac06bab0 2024-05-20T21:09:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.825+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a8970 con 0x7fdaac06bab0 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.827+0000 7fda9b7fe700 1 -- 172.21.15.26:0/2622213872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fda9c009d10 con 0x7fdaac06bab0 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.828+0000 7fda9b7fe700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda940b2880 0x7fda940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.828+0000 7fda9b7fe700 1 -- 172.21.15.26:0/2622213872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fda9c0e2d90 con 0x7fdaac06bab0 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.828+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda940b2880 0x7fda940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 Processor -- start 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda940b2880 0x7fda940b4d40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdaac104530 tx=0x7fdaa0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 -- start start 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 0x55cd51451c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd51455940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd514563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cd513b0c90 con 0x55cd51451860 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cd513a0830 con 0x55cd51452210 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cd513a4870 con 0x55cd51455f60 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 0x55cd51451c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 0x55cd51451c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60988/0 (socket says 172.21.15.26:60988) 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdaaa59c700 1 -- 172.21.15.26:0/904866842 learned_addr learned my addr 172.21.15.26:0/904866842 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.829+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd51455940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaaad9d700 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd514563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/904866842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 msgr2=0x55cd514563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd514563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/904866842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 msgr2=0x55cd51451c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 0x55cd51451c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/904866842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd512f08d0 con 0x55cd51452210 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 0x55cd51451c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd51455940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e9e0 tx=0x7fdaa0011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98db8f0589a9906d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fda98ff9700 1 -- 172.21.15.26:0/904866842 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa001b040 con 0x55cd51452210 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fda98ff9700 1 -- 172.21.15.26:0/904866842 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa0024070 con 0x55cd51452210 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.830+0000 7fda98ff9700 1 -- 172.21.15.26:0/904866842 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00136b0 con 0x55cd51452210 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/904866842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 msgr2=0x55cd51455940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd51455940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e9e0 tx=0x7fdaa0011000 comp rx=0 tx=0).stop 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/904866842 shutdown_connections 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51451860 0x55cd51451c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd514563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/904866842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd51455940 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/904866842 >> 172.21.15.26:0/904866842 conn(0x55cd5137f080 msgr2=0x55cd5144be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/904866842 shutdown_connections 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.831+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/904866842 wait complete. 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 -- start start 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 Processor -- start 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 0x55cd5145d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd5145a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd5145ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cd512a6fa0 con 0x55cd51452210 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cd513427c0 con 0x55cd51451860 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cd5145e120 con 0x55cd51455f60 2024-05-20T21:09:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdabb1aeb80 10 client.0 _ll_register_callbacks cb 0x55cd5133b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 0x55cd5145d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd5145a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 0x55cd5145d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:47222/0 (socket says 172.21.15.26:47222) 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdaaa59c700 1 -- 172.21.15.26:0/294117228 learned_addr learned my addr 172.21.15.26:0/294117228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd5145a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60996/0 (socket says 172.21.15.26:60996) 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.832+0000 7fdaaad9d700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd5145ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaaa59c700 1 -- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 msgr2=0x55cd5145ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd5145ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaaa59c700 1 -- 172.21.15.26:0/294117228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 msgr2=0x55cd5145a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd5145a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaaa59c700 1 -- 172.21.15.26:0/294117228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5133f690 con 0x55cd51451860 2024-05-20T21:09:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd5145a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 0x55cd5145d3c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005f50 tx=0x7fda9c022c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1739008761bfd724 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fda91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fda91ffb700 1 -- 172.21.15.26:0/294117228 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0b31b0 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fda91ffb700 1 -- 172.21.15.26:0/294117228 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fda9c0b2f30 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa0024070 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fda91ffb700 1 -- 172.21.15.26:0/294117228 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda9c0a9640 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdabb1aeb80 10 client.28979 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd51468830 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.833+0000 7fda91ffb700 1 -- 172.21.15.26:0/294117228 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa0024070 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fda91ffb700 1 -- 172.21.15.26:0/294117228 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fda9c0a90b0 con 0x55cd51451860 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdabb1aeb80 10 client.28979 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdabb1aeb80 10 client.28979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:09:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdabb1aeb80 20 client.28979 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd5146ad00 0x55cd5146d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdabb1aeb80 4 client.28979 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5141d9e0 con 0x55cd5146ad00 2024-05-20T21:09:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd5146ad00 0x55cd5146d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd5146ad00 0x55cd5146d160 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db039fdaa9dcb773 server_cookie=72c692cdca879786 in_seq=0 out_seq=0 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.834+0000 7fda91ffb700 10 client.28979 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fda91ffb700 1 -- 172.21.15.26:0/294117228 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cd5141d9e0 con 0x55cd5146ad00 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fda91ffb700 10 client.28979 handle_command_reply: tid=0 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 client.28979 shutdown 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 msgr2=0x55cd5145d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 0x55cd5145d3c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005f50 tx=0x7fda9c022c00 comp rx=0 tx=0).stop 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 shutdown_connections 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd5146ad00 0x55cd5146d160 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51455f60 0x55cd5145ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fda91ffb700 0 client.28979 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51451860 0x55cd5145d3c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/294117228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd5145a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 >> 172.21.15.26:0/294117228 conn(0x55cd5137f080 msgr2=0x55cd51457350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 shutdown_connections 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.835+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/294117228 wait complete. 2024-05-20T21:09:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.836+0000 7fdabb1aeb80 20 client.28979 trim_cache size 0 max 16384 2024-05-20T21:09:54.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.852+0000 7fdabb1aeb80 1 Processor -- start 2024-05-20T21:09:54.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.852+0000 7fdabb1aeb80 1 -- start start 2024-05-20T21:09:54.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd5145c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd51472a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51455f60 0x55cd51472fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b26c0 con 0x55cd51452210 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1aff50 con 0x55cd51455f60 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1adda0 con 0x55cd51451860 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd51472a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd51472a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32770/0 (socket says 172.21.15.26:32770) 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/579813771 learned_addr learned my addr 172.21.15.26:0/579813771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaaad9d700 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51455f60 0x55cd51472fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd5145c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/579813771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 msgr2=0x55cd5145c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd5145c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/579813771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51455f60 msgr2=0x55cd51472fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51455f60 0x55cd51472fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/579813771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd51458c30 con 0x55cd51452210 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd51472a80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0004790 tx=0x7fdaa0015c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02e2c786e1cf42e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.853+0000 7fda93fff700 1 -- 172.21.15.26:0/579813771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0036070 con 0x55cd51452210 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fda93fff700 1 -- 172.21.15.26:0/579813771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa0024040 con 0x55cd51452210 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fda93fff700 1 -- 172.21.15.26:0/579813771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0031c40 con 0x55cd51452210 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/579813771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 msgr2=0x55cd51472a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd51472a80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0004790 tx=0x7fdaa0015c70 comp rx=0 tx=0).stop 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/579813771 shutdown_connections 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51455f60 0x55cd51472fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd5145c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/579813771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51452210 0x55cd51472a80 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/579813771 >> 172.21.15.26:0/579813771 conn(0x55cd5137f080 msgr2=0x55cd5145bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/579813771 shutdown_connections 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.854+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/579813771 wait complete. 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 -- start start 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 Processor -- start 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd51463c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd514534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51455f60 0x55cd51453be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1ae410 con 0x55cd51455f60 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1af880 con 0x55cd51452210 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1aeae0 con 0x55cd51451860 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdabb1aeb80 10 client.0 _ll_register_callbacks cb 0x55cd5133b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:54.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd514534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaaa59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd51463c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd514534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:47242/0 (socket says 172.21.15.26:47242) 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/1585619397 learned_addr learned my addr 172.21.15.26:0/1585619397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaaad9d700 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51455f60 0x55cd51453be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 msgr2=0x55cd51463c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd51463c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/1585619397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51455f60 msgr2=0x55cd51453be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51455f60 0x55cd51453be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.855+0000 7fdaa9d9b700 1 -- 172.21.15.26:0/1585619397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd514d4950 con 0x55cd51452210 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fdaa9d9b700 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd514534a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55cd51447190 tx=0x7fdaa00097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ef0688d372c45c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fda91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:09:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fda91ffb700 1 -- 172.21.15.26:0/1585619397 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0040070 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa0024040 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fda91ffb700 1 -- 172.21.15.26:0/1585619397 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fdaa00318c0 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fda91ffb700 1 -- 172.21.15.26:0/1585619397 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdaa004c050 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fdabb1aeb80 10 client.28985 fetch_fsmap learned FSMap version 30 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd514d5b60 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fda91ffb700 1 -- 172.21.15.26:0/1585619397 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa0024040 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.856+0000 7fda91ffb700 1 -- 172.21.15.26:0/1585619397 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fdaa001e030 con 0x55cd51452210 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdabb1aeb80 10 client.28985 fetch_fsmap finished waiting for FSMap version 30 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdabb1aeb80 10 client.28985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdabb1aeb80 20 client.28985 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd514d5f30 0x55cd514d8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdabb1aeb80 4 client.28985 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd513b2be0 con 0x55cd514d5f30 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd514d5f30 0x55cd514d8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fdaaa59c700 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd514d5f30 0x55cd514d8320 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9e9f1921440e9ce server_cookie=aebba9b78a1ab5f7 in_seq=0 out_seq=0 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.857+0000 7fda91ffb700 10 client.28985 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fda91ffb700 1 -- 172.21.15.26:0/1585619397 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd513b2be0 con 0x55cd514d5f30 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fda91ffb700 10 client.28985 handle_command_reply: tid=0 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 client.28985 shutdown 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 msgr2=0x55cd514534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd514534a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55cd51447190 tx=0x7fdaa00097d0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 shutdown_connections 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x55cd514d5f30 0x55cd514d8320 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd51455f60 0x55cd51453be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd51451860 0x55cd51463c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 --2- 172.21.15.26:0/1585619397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd51452210 0x55cd514534a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 >> 172.21.15.26:0/1585619397 conn(0x55cd5137f080 msgr2=0x55cd51450f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fda91ffb700 0 client.28985 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:09:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 shutdown_connections 2024-05-20T21:09:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 1 -- 172.21.15.26:0/1585619397 wait complete. 2024-05-20T21:09:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.858+0000 7fdabb1aeb80 20 client.28985 trim_cache size 0 max 16384 2024-05-20T21:09:54.864 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda940b2880 msgr2=0x7fda940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda940b2880 0x7fda940b4d40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdaac104530 tx=0x7fdaa0009190 comp rx=0 tx=0).stop 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 msgr2=0x7fdaac19bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 0x7fdaac19bdf0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c00bfc0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 shutdown_connections 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fda940b2880 0x7fda940b4d40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaac06bab0 0x7fdaac19bdf0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaac1095a0 0x7fdaac1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 --2- 172.21.15.26:0/2622213872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdaac06c470 0x7fdaac19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:54.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 >> 172.21.15.26:0/2622213872 conn(0x7fdaac0793b0 msgr2=0x7fdaac10e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:54.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 shutdown_connections 2024-05-20T21:09:54.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:54.859+0000 7fdab0918700 1 -- 172.21.15.26:0/2622213872 wait complete. 2024-05-20T21:09:54.992 INFO:tasks.workunit.client.0.smithi026.stdout: 0 cleanup 600 sec 2024-05-20T21:09:54.992 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Operation Count AvgLat MaxLat 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: ---------------------------------------- 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: NTCreateX 41796 5.413 28487.128 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Close 30686 0.048 2.927 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Rename 1776 28.661 27804.993 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Unlink 8447 9.136 29628.955 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Qpathinfo 37920 1.581 86.470 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Qfileinfo 6626 0.075 0.645 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Qfsinfo 6965 0.319 3.607 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Sfileinfo 3406 0.643 66.502 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: Find 14668 10.853 165.685 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: WriteX 20690 0.715 11324.832 2024-05-20T21:09:54.993 INFO:tasks.workunit.client.0.smithi026.stdout: ReadX 65876 0.004 0.179 2024-05-20T21:09:54.994 INFO:tasks.workunit.client.0.smithi026.stdout: LockX 138 0.666 1.176 2024-05-20T21:09:54.994 INFO:tasks.workunit.client.0.smithi026.stdout: UnlockX 138 0.556 1.287 2024-05-20T21:09:54.994 INFO:tasks.workunit.client.0.smithi026.stdout: Flush 2932 3.065 56.603 2024-05-20T21:09:54.994 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-05-20T21:09:54.994 INFO:tasks.workunit.client.0.smithi026.stdout:Throughput 2.17178 MB/sec 1 clients 1 procs max_latency=29628.963 ms 2024-05-20T21:09:54.994 INFO:tasks.workunit.client.0.smithi026.stderr:+ dbench 10 2024-05-20T21:09:54.995 INFO:tasks.workunit.client.0.smithi026.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-20T21:09:54.995 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-05-20T21:09:54.995 INFO:tasks.workunit.client.0.smithi026.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-20T21:09:54.995 INFO:tasks.workunit.client.0.smithi026.stdout:9 of 10 processes prepared for launch 0 sec 2024-05-20T21:09:55.095 INFO:tasks.workunit.client.0.smithi026.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-20T21:09:55.095 INFO:tasks.workunit.client.0.smithi026.stdout:releasing clients 2024-05-20T21:09:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:55 smithi055 ceph-mon[35213]: pgmap v803: 97 pgs: 97 active+clean; 26 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:09:55.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:55 smithi026 ceph-mon[31990]: pgmap v803: 97 pgs: 97 active+clean; 26 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:09:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:55 smithi144 ceph-mon[35660]: pgmap v803: 97 pgs: 97 active+clean; 26 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-05-20T21:09:55.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:09:56.095 INFO:tasks.workunit.client.0.smithi026.stdout: 10 277 103.66 MB/sec warmup 1 sec latency 44.457 ms 2024-05-20T21:09:56.166 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:57.095 INFO:tasks.workunit.client.0.smithi026.stdout: 10 516 89.80 MB/sec warmup 2 sec latency 141.513 ms 2024-05-20T21:09:57.442 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:57 smithi026 ceph-mon[31990]: pgmap v804: 97 pgs: 97 active+clean; 39 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-05-20T21:09:57.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:57 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:09:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:57 smithi055 ceph-mon[35213]: pgmap v804: 97 pgs: 97 active+clean; 39 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-05-20T21:09:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:57 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:09:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:57 smithi144 ceph-mon[35660]: pgmap v804: 97 pgs: 97 active+clean; 39 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-05-20T21:09:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:57 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:09:58.095 INFO:tasks.workunit.client.0.smithi026.stdout: 10 732 70.99 MB/sec warmup 3 sec latency 102.224 ms 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 -- 172.21.15.26:0/3228161097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e41095e0 msgr2=0x7f24e41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 --2- 172.21.15.26:0/3228161097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e41095e0 0x7f24e41099c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009990 tx=0x7f24d4009ca0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 -- 172.21.15.26:0/3228161097 shutdown_connections 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 --2- 172.21.15.26:0/3228161097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e41095e0 0x7f24e41099c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 --2- 172.21.15.26:0/3228161097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e406c610 0x7f24e4110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 --2- 172.21.15.26:0/3228161097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406bc50 0x7f24e406c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 -- 172.21.15.26:0/3228161097 >> 172.21.15.26:0/3228161097 conn(0x7f24e40784b0 msgr2=0x7f24e40788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 -- 172.21.15.26:0/3228161097 shutdown_connections 2024-05-20T21:09:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.124+0000 7f24e9a20700 1 -- 172.21.15.26:0/3228161097 wait complete. 2024-05-20T21:09:58.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.125+0000 7f24e9a20700 1 Processor -- start 2024-05-20T21:09:58.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 -- start start 2024-05-20T21:09:58.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 0x7f24e419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406c610 0x7f24e419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e41095e0 0x7f24e41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4113620 con 0x7f24e406bc50 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41134a0 con 0x7f24e406c610 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e9a20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41137a0 con 0x7f24e41095e0 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 0x7f24e419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406c610 0x7f24e419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 0x7f24e419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32802/0 (socket says 172.21.15.26:32802) 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.126+0000 7f24e2ffd700 1 -- 172.21.15.26:0/2245949716 learned_addr learned my addr 172.21.15.26:0/2245949716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e37fe700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e41095e0 0x7f24e41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e27fc700 1 -- 172.21.15.26:0/2245949716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e41095e0 msgr2=0x7f24e41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e27fc700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e41095e0 0x7f24e41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e27fc700 1 -- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 msgr2=0x7f24e419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e27fc700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 0x7f24e419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e27fc700 1 -- 172.21.15.26:0/2245949716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e419a100 con 0x7f24e406c610 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 0x7f24e419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e27fc700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406c610 0x7f24e419fd20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9f0 tx=0x7f24d800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e8a1e700 1 -- 172.21.15.26:0/2245949716 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800cb70 con 0x7f24e406c610 2024-05-20T21:09:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e419a3f0 con 0x7f24e406c610 2024-05-20T21:09:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e8a1e700 1 -- 172.21.15.26:0/2245949716 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d800ee90 con 0x7f24e406c610 2024-05-20T21:09:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.127+0000 7f24e8a1e700 1 -- 172.21.15.26:0/2245949716 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8003c30 con 0x7f24e406c610 2024-05-20T21:09:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.128+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e4102470 con 0x7f24e406c610 2024-05-20T21:09:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.130+0000 7f24e8a1e700 1 -- 172.21.15.26:0/2245949716 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f24d8003ed0 con 0x7f24e406c610 2024-05-20T21:09:58.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24e8a1e700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc0b2900 0x7f24cc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24e8a1e700 1 -- 172.21.15.26:0/2245949716 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f24d80e2390 con 0x7f24e406c610 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc0b2900 0x7f24cc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 Processor -- start 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 -- start start 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fdbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdc260 0x559d84fe8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d84f39d70 con 0x559d84fe1320 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559d84f29910 con 0x559d84fdc260 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.132+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559d84f2d950 con 0x559d84fdb8a0 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32804/0 (socket says 172.21.15.26:32804) 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 -- 172.21.15.26:0/1066718739 learned_addr learned my addr 172.21.15.26:0/1066718739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc0b2900 0x7f24cc0b4dc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f24e419aae0 tx=0x7f24d401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e27fc700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fdbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e37fe700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdc260 0x559d84fe8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 -- 172.21.15.26:0/1066718739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 msgr2=0x559d84fdbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fdbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 -- 172.21.15.26:0/1066718739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdc260 msgr2=0x559d84fe8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdc260 0x559d84fe8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 -- 172.21.15.26:0/1066718739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d84ed2fb0 con 0x559d84fe1320 2024-05-20T21:09:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e27fc700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fdbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e37fe700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdc260 0x559d84fe8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.133+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe1700 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f24d40052e0 tx=0x7f24d4022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a292018fb5b653 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.134+0000 7f24c9ffb700 1 -- 172.21.15.26:0/1066718739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4003b10 con 0x559d84fe1320 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.134+0000 7f24c9ffb700 1 -- 172.21.15.26:0/1066718739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d4003cb0 con 0x559d84fe1320 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.134+0000 7f24c9ffb700 1 -- 172.21.15.26:0/1066718739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40296e0 con 0x559d84fe1320 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1066718739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 msgr2=0x559d84fe1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe1700 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f24d40052e0 tx=0x7f24d4022040 comp rx=0 tx=0).stop 2024-05-20T21:09:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1066718739 shutdown_connections 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe1700 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdc260 0x559d84fe8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1066718739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fdbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1066718739 >> 172.21.15.26:0/1066718739 conn(0x559d84f07240 msgr2=0x559d84fd63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1066718739 shutdown_connections 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1066718739 wait complete. 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.135+0000 7f24f42b6b80 1 -- start start 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 Processor -- start 2024-05-20T21:09:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 0x559d84fe5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d84e2ffa0 con 0x559d84fe1320 2024-05-20T21:09:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559d84e91da0 con 0x559d84fdb8a0 2024-05-20T21:09:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559d84fe85c0 con 0x559d84fdc260 2024-05-20T21:09:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.136+0000 7f24f42b6b80 10 client.0 _ll_register_callbacks cb 0x559d84ec4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 0x559d84fe5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 0x559d84fe5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:44428/0 (socket says 172.21.15.26:44428) 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 -- 172.21.15.26:0/2066660778 learned_addr learned my addr 172.21.15.26:0/2066660778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e37fe700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 -- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 msgr2=0x559d84fe7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 -- 172.21.15.26:0/2066660778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 msgr2=0x559d84fe5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 -- 172.21.15.26:0/2066660778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d84e798d0 con 0x559d84fdc260 2024-05-20T21:09:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e37fe700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:09:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24e27fc700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 0x559d84fe5010 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f24d80a5970 tx=0x7f24d80a92c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6edeb84a063fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:09:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.137+0000 7f24c2ffd700 1 -- 172.21.15.26:0/2066660778 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d808fe60 con 0x559d84fdc260 2024-05-20T21:09:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24c2ffd700 1 -- 172.21.15.26:0/2066660778 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f24d80cc0d0 con 0x559d84fdc260 2024-05-20T21:09:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d4003cb0 con 0x559d84fdc260 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24c2ffd700 1 -- 172.21.15.26:0/2066660778 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f24d80a9400 con 0x559d84fdc260 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24f42b6b80 10 client.28696 fetch_fsmap learned FSMap version 31 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d84ee3760 con 0x559d84fdc260 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24c2ffd700 1 -- 172.21.15.26:0/2066660778 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d4003cb0 con 0x559d84fdc260 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24c2ffd700 1 -- 172.21.15.26:0/2066660778 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f24d80b22c0 con 0x559d84fdc260 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24f42b6b80 10 client.28696 fetch_fsmap finished waiting for FSMap version 31 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24f42b6b80 10 client.28696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.138+0000 7f24f42b6b80 20 client.28696 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.139+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d84ff5110 0x559d84ff7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.139+0000 7f24f42b6b80 4 client.28696 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.139+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d84f3bcc0 con 0x559d84ff5110 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.139+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d84ff5110 0x559d84ff7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.139+0000 7f24c2ffd700 10 client.28696 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.140+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d84ff5110 0x559d84ff7520 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b67c248246941d99 server_cookie=78f452d77d05fd1b in_seq=0 out_seq=0 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.140+0000 7f24c2ffd700 1 -- 172.21.15.26:0/2066660778 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559d84f3bcc0 con 0x559d84ff5110 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.140+0000 7f24c2ffd700 10 client.28696 handle_command_reply: tid=0 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.140+0000 7f24f42b6b80 1 client.28696 shutdown 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 msgr2=0x559d84fe5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 0x559d84fe5010 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f24d80a5970 tx=0x7f24d80a92c0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 shutdown_connections 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d84ff5110 0x559d84ff7520 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fe1320 0x559d84fe5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24c2ffd700 0 client.28696 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/2066660778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdc260 0x559d84fe5010 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 >> 172.21.15.26:0/2066660778 conn(0x559d84f07240 msgr2=0x559d84fd5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 shutdown_connections 2024-05-20T21:09:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 1 -- 172.21.15.26:0/2066660778 wait complete. 2024-05-20T21:09:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.141+0000 7f24f42b6b80 20 client.28696 trim_cache size 0 max 16384 2024-05-20T21:09:58.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 Processor -- start 2024-05-20T21:09:58.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 -- start start 2024-05-20T21:09:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fd6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fe1320 0x559d84fdafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41ae3f0 con 0x559d84fdc260 2024-05-20T21:09:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41abc20 con 0x559d84fdb8a0 2024-05-20T21:09:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41a9a90 con 0x559d84fe1320 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24e27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fd6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.163+0000 7f24e27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fd6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32808/0 (socket says 172.21.15.26:32808) 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 -- 172.21.15.26:0/1491316173 learned_addr learned my addr 172.21.15.26:0/1491316173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e37fe700 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fe1320 0x559d84fdafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 -- 172.21.15.26:0/1491316173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fe1320 msgr2=0x559d84fdafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fe1320 0x559d84fdafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 -- 172.21.15.26:0/1491316173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 msgr2=0x559d84fe7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 -- 172.21.15.26:0/1491316173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d84fe0f40 con 0x559d84fdc260 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24e27fc700 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fd6270 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f24d80a95f0 tx=0x7f24d80a58e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa98c5e8fd722c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24c97fa700 1 -- 172.21.15.26:0/1491316173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d808fe60 con 0x559d84fdc260 2024-05-20T21:09:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24c97fa700 1 -- 172.21.15.26:0/1491316173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d80a9ff0 con 0x559d84fdc260 2024-05-20T21:09:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.164+0000 7f24c97fa700 1 -- 172.21.15.26:0/1491316173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80cc0d0 con 0x559d84fdc260 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1491316173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 msgr2=0x559d84fd6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fd6270 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f24d80a95f0 tx=0x7f24d80a58e0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1491316173 shutdown_connections 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fe1320 0x559d84fdafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fdb8a0 0x559d84fe7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/1491316173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fd6270 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.165+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1491316173 >> 172.21.15.26:0/1491316173 conn(0x559d84f07240 msgr2=0x559d84fda890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.166+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1491316173 shutdown_connections 2024-05-20T21:09:58.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.166+0000 7f24f42b6b80 1 -- 172.21.15.26:0/1491316173 wait complete. 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.166+0000 7f24f42b6b80 1 -- start start 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.166+0000 7f24f42b6b80 1 Processor -- start 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fde410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fde950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fe1320 0x559d84fec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41aa0e0 con 0x559d84fdc260 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fde950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fde410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fde410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:44444/0 (socket says 172.21.15.26:44444) 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e2ffd700 1 -- 172.21.15.26:0/3729876337 learned_addr learned my addr 172.21.15.26:0/3729876337 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41ab550 con 0x559d84fe1320 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41aa7b0 con 0x559d84fdb8a0 2024-05-20T21:09:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24f42b6b80 10 client.0 _ll_register_callbacks cb 0x559d84ec4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:09:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e37fe700 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fe1320 0x559d84fec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e27fc700 1 -- 172.21.15.26:0/3729876337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 msgr2=0x559d84fde410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e27fc700 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fde410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e27fc700 1 -- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fe1320 msgr2=0x559d84fec700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:09:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.167+0000 7f24e27fc700 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fe1320 0x559d84fec700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.168+0000 7f24e27fc700 1 -- 172.21.15.26:0/3729876337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d84feccf0 con 0x559d84fdc260 2024-05-20T21:09:58.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.168+0000 7f24e27fc700 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fde950 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f24d80ade60 tx=0x7f24d800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cf013371740a62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:09:58.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.168+0000 7f24c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:09:58.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24c2ffd700 1 -- 172.21.15.26:0/3729876337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80a98f0 con 0x559d84fdc260 2024-05-20T21:09:58.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24c2ffd700 1 -- 172.21.15.26:0/3729876337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f24d80cac20 con 0x559d84fdc260 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d80a9ff0 con 0x559d84fdc260 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24c2ffd700 1 -- 172.21.15.26:0/3729876337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f24d80adca0 con 0x559d84fdc260 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24f42b6b80 10 client.19800 fetch_fsmap learned FSMap version 31 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d84fedff0 con 0x559d84fdc260 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24c2ffd700 1 -- 172.21.15.26:0/3729876337 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d80041c0 con 0x559d84fdc260 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.169+0000 7f24c2ffd700 1 -- 172.21.15.26:0/3729876337 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f24d80a85c0 con 0x559d84fdc260 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24f42b6b80 10 client.19800 fetch_fsmap finished waiting for FSMap version 31 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24f42b6b80 10 client.19800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24f42b6b80 20 client.19800 populate_metadata read hostname 'smithi026' 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d8505fcc0 0x559d850620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24f42b6b80 4 client.19800 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:09:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f24d80a9ff0 con 0x559d8505fcc0 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d8505fcc0 0x559d850620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24c2ffd700 10 client.19800 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.170+0000 7f24e2ffd700 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d8505fcc0 0x559d850620b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa245596464f3b12 server_cookie=c3ff818be7292e5b in_seq=0 out_seq=0 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24c2ffd700 1 -- 172.21.15.26:0/3729876337 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f24d80a9ff0 con 0x559d8505fcc0 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24c2ffd700 10 client.19800 handle_command_reply: tid=0 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 client.19800 shutdown 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 msgr2=0x559d84fde950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fde950 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f24d80ade60 tx=0x7f24d800cf70 comp rx=0 tx=0).stop 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 shutdown_connections 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x559d8505fcc0 0x559d850620b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559d84fe1320 0x559d84fec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559d84fdb8a0 0x559d84fde410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 --2- 172.21.15.26:0/3729876337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d84fdc260 0x559d84fde950 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24c2ffd700 0 client.19800 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:09:58.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 >> 172.21.15.26:0/3729876337 conn(0x559d84f07240 msgr2=0x559d84fdb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:58.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 shutdown_connections 2024-05-20T21:09:58.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 1 -- 172.21.15.26:0/3729876337 wait complete. 2024-05-20T21:09:58.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.171+0000 7f24f42b6b80 20 client.19800 trim_cache size 0 max 16384 2024-05-20T21:09:58.175 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:09:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc0b2900 msgr2=0x7f24cc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc0b2900 0x7f24cc0b4dc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f24e419aae0 tx=0x7f24d401a040 comp rx=0 tx=0).stop 2024-05-20T21:09:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406c610 msgr2=0x7f24e419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:09:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406c610 0x7f24e419fd20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9f0 tx=0x7f24d800ed00 comp rx=0 tx=0).stop 2024-05-20T21:09:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 shutdown_connections 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f24cc0b2900 0x7f24cc0b4dc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e406bc50 0x7f24e419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e41095e0 0x7f24e41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 --2- 172.21.15.26:0/2245949716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e406c610 0x7f24e419fd20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.172+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 >> 172.21.15.26:0/2245949716 conn(0x7f24e40784b0 msgr2=0x7f24e4101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.173+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 shutdown_connections 2024-05-20T21:09:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:09:58.173+0000 7f24e9a20700 1 -- 172.21.15.26:0/2245949716 wait complete. 2024-05-20T21:09:59.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:09:59.095 INFO:tasks.workunit.client.0.smithi026.stdout: 10 900 54.40 MB/sec warmup 4 sec latency 119.764 ms 2024-05-20T21:09:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:09:59.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:09:59 smithi055 ceph-mon[35213]: pgmap v805: 97 pgs: 97 active+clean; 102 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 8.4 MiB/s wr, 136 op/s 2024-05-20T21:09:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:09:59 smithi026 ceph-mon[31990]: pgmap v805: 97 pgs: 97 active+clean; 102 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 8.4 MiB/s wr, 136 op/s 2024-05-20T21:09:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:09:59 smithi144 ceph-mon[35660]: pgmap v805: 97 pgs: 97 active+clean; 102 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 8.4 MiB/s wr, 136 op/s 2024-05-20T21:10:00.095 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1049 44.95 MB/sec warmup 5 sec latency 103.426 ms 2024-05-20T21:10:00.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:00 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:00.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:00 smithi055 ceph-mon[35213]: overall HEALTH_OK 2024-05-20T21:10:00.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:00 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:00.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:00 smithi026 ceph-mon[31990]: overall HEALTH_OK 2024-05-20T21:10:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:00 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:00.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:00 smithi144 ceph-mon[35660]: overall HEALTH_OK 2024-05-20T21:10:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 -- 172.21.15.26:0/3959396928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 msgr2=0x7fe3b410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 --2- 172.21.15.26:0/3959396928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b410aef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009860 tx=0x7fe3a402a340 comp rx=0 tx=0).stop 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 -- 172.21.15.26:0/3959396928 shutdown_connections 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 --2- 172.21.15.26:0/3959396928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b410aef0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 --2- 172.21.15.26:0/3959396928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 0x7fe3b4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 --2- 172.21.15.26:0/3959396928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 0x7fe3b4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 -- 172.21.15.26:0/3959396928 >> 172.21.15.26:0/3959396928 conn(0x7fe3b40fe020 msgr2=0x7fe3b4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 -- 172.21.15.26:0/3959396928 shutdown_connections 2024-05-20T21:10:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.881+0000 7fe3b8922640 1 -- 172.21.15.26:0/3959396928 wait complete. 2024-05-20T21:10:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.882+0000 7fe3b8922640 1 Processor -- start 2024-05-20T21:10:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.882+0000 7fe3b8922640 1 -- start start 2024-05-20T21:10:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b8922640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 0x7fe3b419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b8922640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 0x7fe3b419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b8922640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b41a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b8922640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b410e490 con 0x7fe3b4102ab0 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b8922640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b410e310 con 0x7fe3b4104660 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b8922640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b410e610 con 0x7fe3b4103ca0 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b41a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b41a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39130/0 (socket says 172.21.15.26:39130) 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 -- 172.21.15.26:0/2507844789 learned_addr learned my addr 172.21.15.26:0/2507844789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b259c640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 0x7fe3b419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 0x7fe3b419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 -- 172.21.15.26:0/2507844789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 msgr2=0x7fe3b419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 0x7fe3b419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 -- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 msgr2=0x7fe3b419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 0x7fe3b419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.883+0000 7fe3b2d9d640 1 -- 172.21.15.26:0/2507844789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b41a2960 con 0x7fe3b4104660 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 0x7fe3b419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3b259c640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 0x7fe3b419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3b2d9d640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b41a22c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009830 tx=0x7fe3a402ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3a37fe640 1 -- 172.21.15.26:0/2507844789 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4037040 con 0x7fe3b4104660 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b41a2bf0 con 0x7fe3b4104660 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3a37fe640 1 -- 172.21.15.26:0/2507844789 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3a4041070 con 0x7fe3b4104660 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3a37fe640 1 -- 172.21.15.26:0/2507844789 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4005290 con 0x7fe3b4104660 2024-05-20T21:10:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.884+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b41a30e0 con 0x7fe3b4104660 2024-05-20T21:10:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.888+0000 7fe3a37fe640 1 -- 172.21.15.26:0/2507844789 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe3a4005430 con 0x7fe3b4104660 2024-05-20T21:10:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.888+0000 7fe3bb635740 1 Processor -- start 2024-05-20T21:10:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.888+0000 7fe3bb635740 1 -- start start 2024-05-20T21:10:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5ed0b0 0x5576ed5f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3bb635740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5576ed55dc30 con 0x5576ed5ed0b0 2024-05-20T21:10:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3bb635740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5576ed54d7d0 con 0x5576ed5ec6c0 2024-05-20T21:10:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3bb635740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576ed551810 con 0x5576ed5f2220 2024-05-20T21:10:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5ed0b0 0x5576ed5f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5ecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39146/0 (socket says 172.21.15.26:39146) 2024-05-20T21:10:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b1d9b640 1 -- 172.21.15.26:0/3999094473 learned_addr learned my addr 172.21.15.26:0/3999094473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b259c640 1 -- 172.21.15.26:0/3999094473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 msgr2=0x5576ed5f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b259c640 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b259c640 1 -- 172.21.15.26:0/3999094473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 msgr2=0x5576ed5ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b259c640 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b259c640 1 -- 172.21.15.26:0/3999094473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ed262210 con 0x5576ed5ed0b0 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.889+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3a37fe640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe39c0b28a0 0x7fe39c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3b259c640 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5ed0b0 0x5576ed5f1bb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0098d0 tx=0x7fe3ac0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eea298342f0b910 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe39c0b28a0 0x7fe39c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3a37fe640 1 -- 172.21.15.26:0/2507844789 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe3a41022d0 con 0x7fe3b4104660 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3a0ff9640 1 -- 172.21.15.26:0/3999094473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00ee60 con 0x5576ed5ed0b0 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3a0ff9640 1 -- 172.21.15.26:0/3999094473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3ac01c070 con 0x5576ed5ed0b0 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.890+0000 7fe3a0ff9640 1 -- 172.21.15.26:0/3999094473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0163c0 con 0x5576ed5ed0b0 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 -- 172.21.15.26:0/3999094473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5ed0b0 msgr2=0x5576ed5f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe39c0b28a0 0x7fe39c0b4d60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5576ed5ed910 tx=0x7fe3a8009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5ed0b0 0x5576ed5f1bb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0098d0 tx=0x7fe3ac0095c0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 -- 172.21.15.26:0/3999094473 shutdown_connections 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5ed0b0 0x5576ed5f1bb0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3999094473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 -- 172.21.15.26:0/3999094473 >> 172.21.15.26:0/3999094473 conn(0x5576ed5e7d30 msgr2=0x5576ed5e8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 -- 172.21.15.26:0/3999094473 shutdown_connections 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 -- 172.21.15.26:0/3999094473 wait complete. 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.891+0000 7fe3bb635740 1 -- start start 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 Processor -- start 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5576ed233790 con 0x5576ed5fc0b0 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5576ed509890 con 0x5576ed5ec6c0 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576ed5f51d0 con 0x5576ed5f2220 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39158/0 (socket says 172.21.15.26:39158) 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 -- 172.21.15.26:0/3205603905 learned_addr learned my addr 172.21.15.26:0/3205603905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b2d9d640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3bb635740 10 client.0 _ll_register_callbacks cb 0x5576ed261a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 -- 172.21.15.26:0/3205603905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 msgr2=0x5576ed5f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 -- 172.21.15.26:0/3205603905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 msgr2=0x5576ed5fe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5fe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.892+0000 7fe3b259c640 1 -- 172.21.15.26:0/3205603905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ed32e620 con 0x5576ed5ec6c0 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe3b2d9d640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe3b259c640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5f6b40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000f80 tx=0x7fe3ac00ddd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2519ed927e114d34 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe399ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe399ffb640 1 -- 172.21.15.26:0/3205603905 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac004e50 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ac01c070 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe399ffb640 1 -- 172.21.15.26:0/3205603905 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe3ac017420 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe399ffb640 1 -- 172.21.15.26:0/3205603905 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3ac032050 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe3bb635740 10 client.29015 fetch_fsmap learned FSMap version 32 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576ed601460 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.893+0000 7fe399ffb640 1 -- 172.21.15.26:0/3205603905 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3ac01c070 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe399ffb640 1 -- 172.21.15.26:0/3205603905 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe3ac00b040 con 0x5576ed5ec6c0 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3bb635740 10 client.29015 fetch_fsmap finished waiting for FSMap version 32 2024-05-20T21:10:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3bb635740 10 client.29015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3bb635740 20 client.29015 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed605320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3bb635740 4 client.29015 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576ed55fb80 con 0x5576ed602e80 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.894+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed605320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.895+0000 7fe399ffb640 10 client.29015 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.895+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed605320 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffaa520f8f3dc103 server_cookie=437779e4a3ea755a in_seq=0 out_seq=0 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.896+0000 7fe399ffb640 1 -- 172.21.15.26:0/3205603905 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5576ed55fb80 con 0x5576ed602e80 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.896+0000 7fe399ffb640 10 client.29015 handle_command_reply: tid=0 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.896+0000 7fe3bb635740 1 client.29015 shutdown 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 msgr2=0x5576ed5f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5f6b40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000f80 tx=0x7fe3ac00ddd0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 shutdown_connections 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed605320 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe399ffb640 0 client.29015 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed5f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 --2- 172.21.15.26:0/3205603905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed5f6b40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 >> 172.21.15.26:0/3205603905 conn(0x5576ed5e7d30 msgr2=0x5576ed5e6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 shutdown_connections 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 1 -- 172.21.15.26:0/3205603905 wait complete. 2024-05-20T21:10:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.897+0000 7fe3bb635740 20 client.29015 trim_cache size 0 max 16384 2024-05-20T21:10:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 Processor -- start 2024-05-20T21:10:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 -- start start 2024-05-20T21:10:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 0x5576ed5eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5f2220 0x5576ed5eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5eb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41aa650 con 0x5576ed5fc0b0 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41a7e90 con 0x5576ed5f2220 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3bb635740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41a5ce0 con 0x5576ed5ec6c0 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 0x5576ed5eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3b1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5f2220 0x5576ed5eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 0x5576ed5eaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45864/0 (socket says 172.21.15.26:45864) 2024-05-20T21:10:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5eb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35388/0 (socket says 172.21.15.26:35388) 2024-05-20T21:10:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.920+0000 7fe3b259c640 1 -- 172.21.15.26:0/430282104 learned_addr learned my addr 172.21.15.26:0/430282104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe3b259c640 1 -- 172.21.15.26:0/430282104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5f2220 msgr2=0x5576ed5eaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe3b259c640 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5f2220 0x5576ed5eaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe3b259c640 1 -- 172.21.15.26:0/430282104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 msgr2=0x5576ed5eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe3b259c640 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5eb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe3b259c640 1 -- 172.21.15.26:0/430282104 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ed5f4a70 con 0x5576ed5ec6c0 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe3b259c640 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 0x5576ed5eaa10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e740 tx=0x7fe3ac005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c25ded0e3f9b6b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe39bfff640 1 -- 172.21.15.26:0/430282104 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0098b0 con 0x5576ed5ec6c0 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe39bfff640 1 -- 172.21.15.26:0/430282104 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3ac01c050 con 0x5576ed5ec6c0 2024-05-20T21:10:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.921+0000 7fe39bfff640 1 -- 172.21.15.26:0/430282104 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac017710 con 0x5576ed5ec6c0 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 -- 172.21.15.26:0/430282104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 msgr2=0x5576ed5eaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 0x5576ed5eaa10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e740 tx=0x7fe3ac005cf0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 -- 172.21.15.26:0/430282104 shutdown_connections 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed5eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5f2220 0x5576ed5eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 --2- 172.21.15.26:0/430282104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5ec6c0 0x5576ed5eaa10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 -- 172.21.15.26:0/430282104 >> 172.21.15.26:0/430282104 conn(0x5576ed5e7d30 msgr2=0x5576ed5e8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 -- 172.21.15.26:0/430282104 shutdown_connections 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.922+0000 7fe3bb635740 1 -- 172.21.15.26:0/430282104 wait complete. 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 -- start start 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 Processor -- start 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed6659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed667ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed66a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41a6350 con 0x5576ed5fc0b0 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41a77c0 con 0x5576ed5ec6c0 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41a6a20 con 0x5576ed5f2220 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3bb635740 10 client.0 _ll_register_callbacks cb 0x7fe3ac00e670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed66a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed6659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3b1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed667ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed6659a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39188/0 (socket says 172.21.15.26:39188) 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.923+0000 7fe3b259c640 1 -- 172.21.15.26:0/2302255968 learned_addr learned my addr 172.21.15.26:0/2302255968 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b259c640 1 -- 172.21.15.26:0/2302255968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 msgr2=0x5576ed667ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b259c640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed667ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b259c640 1 -- 172.21.15.26:0/2302255968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 msgr2=0x5576ed66a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b259c640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed66a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b259c640 1 -- 172.21.15.26:0/2302255968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ed66ac70 con 0x5576ed5ec6c0 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed667ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b2d9d640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed66a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe3b259c640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed6659a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000f80 tx=0x7fe3ac0216e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9301d50fee9128f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe399ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:10:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.924+0000 7fe399ffb640 1 -- 172.21.15.26:0/2302255968 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0098b0 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.925+0000 7fe399ffb640 1 -- 172.21.15.26:0/2302255968 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe3ac021c30 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.925+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ac01c050 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.925+0000 7fe399ffb640 1 -- 172.21.15.26:0/2302255968 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3ac017490 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.925+0000 7fe3bb635740 10 client.29021 fetch_fsmap learned FSMap version 32 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.925+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576ed66bf80 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.926+0000 7fe399ffb640 1 -- 172.21.15.26:0/2302255968 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3ac01c050 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.926+0000 7fe399ffb640 1 -- 172.21.15.26:0/2302255968 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe3ac00b040 con 0x5576ed5ec6c0 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.926+0000 7fe3bb635740 10 client.29021 fetch_fsmap finished waiting for FSMap version 32 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.926+0000 7fe3bb635740 10 client.29021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.926+0000 7fe3bb635740 20 client.29021 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.926+0000 7fe3bb635740 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed66e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.927+0000 7fe3bb635740 4 client.29021 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:10:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.927+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576ed559360 con 0x5576ed602e80 2024-05-20T21:10:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.927+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed66e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.928+0000 7fe3b1d9b640 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed66e750 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b42d63d54f52cf5 server_cookie=fced1db372c145f6 in_seq=0 out_seq=0 2024-05-20T21:10:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:00.928+0000 7fe399ffb640 10 client.29021 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:01.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1212 39.71 MB/sec warmup 6 sec latency 205.990 ms 2024-05-20T21:10:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.180+0000 7fe399ffb640 1 -- 172.21.15.26:0/2302255968 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5576ed559360 con 0x5576ed602e80 2024-05-20T21:10:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.180+0000 7fe399ffb640 10 client.29021 handle_command_reply: tid=0 2024-05-20T21:10:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.180+0000 7fe3bb635740 1 client.29021 shutdown 2024-05-20T21:10:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 msgr2=0x5576ed6659a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed6659a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000f80 tx=0x7fe3ac0216e0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 shutdown_connections 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5576ed602e80 0x5576ed66e750 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576ed5fc0b0 0x5576ed66a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576ed5f2220 0x5576ed667ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 --2- 172.21.15.26:0/2302255968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5576ed5ec6c0 0x5576ed6659a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe399ffb640 0 client.29021 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 >> 172.21.15.26:0/2302255968 conn(0x5576ed5e7d30 msgr2=0x5576ed5e6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 shutdown_connections 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 1 -- 172.21.15.26:0/2302255968 wait complete. 2024-05-20T21:10:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.181+0000 7fe3bb635740 20 client.29021 trim_cache size 0 max 16384 2024-05-20T21:10:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:10:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:10:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:10:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe39c0b28a0 msgr2=0x7fe39c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe39c0b28a0 0x7fe39c0b4d60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5576ed5ed910 tx=0x7fe3a8009780 comp rx=0 tx=0).stop 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 msgr2=0x7fe3b41a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b41a22c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009830 tx=0x7fe3a402ad70 comp rx=0 tx=0).stop 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 shutdown_connections 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe39c0b28a0 0x7fe39c0b4d60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b4104660 0x7fe3b41a22c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3b4103ca0 0x7fe3b419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 --2- 172.21.15.26:0/2507844789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b4102ab0 0x7fe3b419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 >> 172.21.15.26:0/2507844789 conn(0x7fe3b40fe020 msgr2=0x7fe3b4074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 shutdown_connections 2024-05-20T21:10:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:01.182+0000 7fe3b8922640 1 -- 172.21.15.26:0/2507844789 wait complete. 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.371+0000 7f210ce5e700 1 -- 172.21.15.26:0/722425964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806c630 msgr2=0x7f2108110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.371+0000 7f210ce5e700 1 --2- 172.21.15.26:0/722425964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806c630 0x7f2108110420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2100009fc0 tx=0x7f210000b330 comp rx=0 tx=0).stop 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 -- 172.21.15.26:0/722425964 shutdown_connections 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 --2- 172.21.15.26:0/722425964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806c630 0x7f2108110420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 --2- 172.21.15.26:0/722425964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2108109650 0x7f2108109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 --2- 172.21.15.26:0/722425964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f210806bc70 0x7f210806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 -- 172.21.15.26:0/722425964 >> 172.21.15.26:0/722425964 conn(0x7f21080782b0 msgr2=0x7f21080786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 -- 172.21.15.26:0/722425964 shutdown_connections 2024-05-20T21:10:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.372+0000 7f210ce5e700 1 -- 172.21.15.26:0/722425964 wait complete. 2024-05-20T21:10:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 Processor -- start 2024-05-20T21:10:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 -- start start 2024-05-20T21:10:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 0x7f2108102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210806c630 0x7f2108102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2108109650 0x7f21081032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2108113630 con 0x7f2108109650 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21081134b0 con 0x7f210806bc70 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210ce5e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21081137b0 con 0x7f210806c630 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 0x7f2108102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 0x7f2108102860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39214/0 (socket says 172.21.15.26:39214) 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f210659c700 1 -- 172.21.15.26:0/1673033289 learned_addr learned my addr 172.21.15.26:0/1673033289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.373+0000 7f2106d9d700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2108109650 0x7f21081032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210659c700 1 -- 172.21.15.26:0/1673033289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210806c630 msgr2=0x7f2108102da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210659c700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210806c630 0x7f2108102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210659c700 1 -- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2108109650 msgr2=0x7f21081032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210659c700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2108109650 0x7f21081032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210659c700 1 -- 172.21.15.26:0/1673033289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081ac550 con 0x7f210806bc70 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f2106d9d700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2108109650 0x7f21081032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210659c700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 0x7f2108102860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9a0 tx=0x7f20f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f20f77fe700 1 -- 172.21.15.26:0/1673033289 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800cbd0 con 0x7f210806bc70 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f20f77fe700 1 -- 172.21.15.26:0/1673033289 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f800cd70 con 0x7f210806bc70 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081ac7e0 con 0x7f210806bc70 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081acd70 con 0x7f210806bc70 2024-05-20T21:10:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.374+0000 7f20f77fe700 1 -- 172.21.15.26:0/1673033289 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8018680 con 0x7f210806bc70 2024-05-20T21:10:01.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.378+0000 7f20f77fe700 1 -- 172.21.15.26:0/1673033289 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f20f8010430 con 0x7f210806bc70 2024-05-20T21:10:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.379+0000 7f20f77fe700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20f00b2900 0x7f20f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.379+0000 7f20f77fe700 1 -- 172.21.15.26:0/1673033289 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20f80e2400 con 0x7f210806bc70 2024-05-20T21:10:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.379+0000 7f2105d9b700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20f00b2900 0x7f20f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 Processor -- start 2024-05-20T21:10:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f2105d9b700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20f00b2900 0x7f20f00b4dc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005e50 tx=0x7f20fc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 -- start start 2024-05-20T21:10:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 0x55e119de4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119deb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e119d3cd70 con 0x55e119ddf260 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e119d2c910 con 0x55e119de4320 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f21176f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e119d30950 con 0x55e119dde8a0 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 0x55e119de4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 0x55e119de4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39230/0 (socket says 172.21.15.26:39230) 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f210659c700 1 -- 172.21.15.26:0/3454673628 learned_addr learned my addr 172.21.15.26:0/3454673628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.380+0000 7f2106d9d700 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119deb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f2105d9b700 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f210659c700 1 -- 172.21.15.26:0/3454673628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 msgr2=0x55e119dded20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f210659c700 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dded20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f210659c700 1 -- 172.21.15.26:0/3454673628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 msgr2=0x55e119deb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f210659c700 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119deb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f210659c700 1 -- 172.21.15.26:0/3454673628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e119cd5fb0 con 0x55e119de4320 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f2106d9d700 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119deb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f210659c700 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 0x55e119de4700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009230 tx=0x7f20f80a91b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c3cd5006aa0499 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f20f4ff9700 1 -- 172.21.15.26:0/3454673628 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80a94d0 con 0x55e119de4320 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f20f4ff9700 1 -- 172.21.15.26:0/3454673628 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f80aa0c0 con 0x55e119de4320 2024-05-20T21:10:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.381+0000 7f20f4ff9700 1 -- 172.21.15.26:0/3454673628 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80b2390 con 0x55e119de4320 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.382+0000 7f21176f4b80 1 -- 172.21.15.26:0/3454673628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 msgr2=0x55e119de4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.382+0000 7f21176f4b80 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 0x55e119de4700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009230 tx=0x7f20f80a91b0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 -- 172.21.15.26:0/3454673628 shutdown_connections 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119deb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119de4320 0x55e119de4700 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 --2- 172.21.15.26:0/3454673628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dded20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 -- 172.21.15.26:0/3454673628 >> 172.21.15.26:0/3454673628 conn(0x55e119d0a240 msgr2=0x55e119dd93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 -- 172.21.15.26:0/3454673628 shutdown_connections 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 -- 172.21.15.26:0/3454673628 wait complete. 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 -- start start 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.383+0000 7f21176f4b80 1 Processor -- start 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119dead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119de8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e119c32fa0 con 0x55e119ddf260 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e119c94da0 con 0x55e119dde8a0 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e119deb680 con 0x55e119de4320 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f21176f4b80 10 client.0 _ll_register_callbacks cb 0x55e119cc7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119dead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119dead80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39242/0 (socket says 172.21.15.26:39242) 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 -- 172.21.15.26:0/2903256728 learned_addr learned my addr 172.21.15.26:0/2903256728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f2106d9d700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f2105d9b700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119de8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 -- 172.21.15.26:0/2903256728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 msgr2=0x55e119de85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 -- 172.21.15.26:0/2903256728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 msgr2=0x55e119de8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119de8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f210659c700 1 -- 172.21.15.26:0/2903256728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e119c7c8d0 con 0x55e119dde8a0 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.384+0000 7f2106d9d700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f210659c700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119dead80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f20f8000c00 tx=0x7f20f80b25c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2e3af359885374 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f20edffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:10:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f20edffb700 1 -- 172.21.15.26:0/2903256728 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80c2070 con 0x55e119dde8a0 2024-05-20T21:10:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f20edffb700 1 -- 172.21.15.26:0/2903256728 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20f8010720 con 0x55e119dde8a0 2024-05-20T21:10:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f80aa0c0 con 0x55e119dde8a0 2024-05-20T21:10:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f20edffb700 1 -- 172.21.15.26:0/2903256728 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20f80add70 con 0x55e119dde8a0 2024-05-20T21:10:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f21176f4b80 10 client.29033 fetch_fsmap learned FSMap version 33 2024-05-20T21:10:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.385+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e119ce6760 con 0x55e119dde8a0 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f20edffb700 1 -- 172.21.15.26:0/2903256728 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f80aa0c0 con 0x55e119dde8a0 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f20edffb700 1 -- 172.21.15.26:0/2903256728 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f20f808a070 con 0x55e119dde8a0 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f21176f4b80 10 client.29033 fetch_fsmap finished waiting for FSMap version 33 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f21176f4b80 10 client.29033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f21176f4b80 20 client.29033 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f21176f4b80 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119dfa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f21176f4b80 4 client.29033 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e119d3ecc0 con 0x55e119df80c0 2024-05-20T21:10:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.386+0000 7f2105d9b700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119dfa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.387+0000 7f20edffb700 10 client.29033 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.387+0000 7f2105d9b700 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119dfa4b0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febd9aafa8fa9e50 server_cookie=935244a4aeea67b5 in_seq=0 out_seq=0 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f20edffb700 1 -- 172.21.15.26:0/2903256728 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e119d3ecc0 con 0x55e119df80c0 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f20edffb700 10 client.29033 handle_command_reply: tid=0 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 client.29033 shutdown 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 msgr2=0x55e119dead80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119dead80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f20f8000c00 tx=0x7f20f80b25c0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 shutdown_connections 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119dfa4b0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f20edffb700 0 client.29033 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119dead80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 --2- 172.21.15.26:0/2903256728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119de8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 >> 172.21.15.26:0/2903256728 conn(0x55e119d0a240 msgr2=0x55e119dd8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 shutdown_connections 2024-05-20T21:10:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.389+0000 7f21176f4b80 1 -- 172.21.15.26:0/2903256728 wait complete. 2024-05-20T21:10:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.390+0000 7f21176f4b80 20 client.29033 trim_cache size 0 max 16384 2024-05-20T21:10:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 Processor -- start 2024-05-20T21:10:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 -- start start 2024-05-20T21:10:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119de4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119dd9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21081b6de0 con 0x55e119ddf260 2024-05-20T21:10:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21081b45e0 con 0x55e119dde8a0 2024-05-20T21:10:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f21176f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21081b2430 con 0x55e119de4320 2024-05-20T21:10:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119de4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f2105d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119dd9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f210659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119de4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39248/0 (socket says 172.21.15.26:39248) 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f210659c700 1 -- 172.21.15.26:0/661584057 learned_addr learned my addr 172.21.15.26:0/661584057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.413+0000 7f2106d9d700 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f210659c700 1 -- 172.21.15.26:0/661584057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 msgr2=0x55e119de0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f210659c700 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f210659c700 1 -- 172.21.15.26:0/661584057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 msgr2=0x55e119dd9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f210659c700 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119dd9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f210659c700 1 -- 172.21.15.26:0/661584057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e119de3f40 con 0x55e119dde8a0 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f210659c700 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119de4700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f20f800f010 tx=0x7f20f80a54e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b49a32b51c92d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f20effff700 1 -- 172.21.15.26:0/661584057 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80c2040 con 0x55e119dde8a0 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f20effff700 1 -- 172.21.15.26:0/661584057 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f8087070 con 0x55e119dde8a0 2024-05-20T21:10:01.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.414+0000 7f20effff700 1 -- 172.21.15.26:0/661584057 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8010980 con 0x55e119dde8a0 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 -- 172.21.15.26:0/661584057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 msgr2=0x55e119de4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119de4700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f20f800f010 tx=0x7f20f80a54e0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 -- 172.21.15.26:0/661584057 shutdown_connections 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119de4320 0x55e119de0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119dde8a0 0x55e119de4700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 --2- 172.21.15.26:0/661584057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119ddf260 0x55e119dd9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.415+0000 7f21176f4b80 1 -- 172.21.15.26:0/661584057 >> 172.21.15.26:0/661584057 conn(0x55e119d0a240 msgr2=0x55e119dde120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.416+0000 7f21176f4b80 1 -- 172.21.15.26:0/661584057 shutdown_connections 2024-05-20T21:10:01.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.416+0000 7f21176f4b80 1 -- 172.21.15.26:0/661584057 wait complete. 2024-05-20T21:10:01.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.416+0000 7f21176f4b80 1 -- start start 2024-05-20T21:10:01.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.416+0000 7f21176f4b80 1 Processor -- start 2024-05-20T21:10:01.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119ddf260 0x55e119df9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119de4320 0x55e119dfa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21081b2aa0 con 0x55e119de4320 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21081b3f10 con 0x55e119ddf260 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21081b3170 con 0x55e119dde8a0 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f2105d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119ddf260 0x55e119df9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f21176f4b80 10 client.0 _ll_register_callbacks cb 0x55e119cc7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dfdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45916/0 (socket says 172.21.15.26:45916) 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 -- 172.21.15.26:0/508789742 learned_addr learned my addr 172.21.15.26:0/508789742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f2106d9d700 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119de4320 0x55e119dfa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 -- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119ddf260 msgr2=0x55e119df9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119ddf260 0x55e119df9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 -- 172.21.15.26:0/508789742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119de4320 msgr2=0x55e119dfa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119de4320 0x55e119dfa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.417+0000 7f210659c700 1 -- 172.21.15.26:0/508789742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e119dfac90 con 0x55e119dde8a0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f210659c700 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dfdb80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20f80a5820 tx=0x7f20f80a89d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1f346beed91ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f20edffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f20edffb700 1 -- 172.21.15.26:0/508789742 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80c2040 con 0x55e119dde8a0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f20edffb700 1 -- 172.21.15.26:0/508789742 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f20f80a8e30 con 0x55e119dde8a0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f8087070 con 0x55e119dde8a0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f20edffb700 1 -- 172.21.15.26:0/508789742 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20f80cc300 con 0x55e119dde8a0 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f21176f4b80 10 client.28735 fetch_fsmap learned FSMap version 33 2024-05-20T21:10:01.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.418+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e119dfac90 con 0x55e119dde8a0 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f20edffb700 1 -- 172.21.15.26:0/508789742 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f8087070 con 0x55e119dde8a0 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f20edffb700 1 -- 172.21.15.26:0/508789742 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f20f808a040 con 0x55e119dde8a0 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f21176f4b80 10 client.28735 fetch_fsmap finished waiting for FSMap version 33 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f21176f4b80 10 client.28735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f21176f4b80 20 client.28735 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f21176f4b80 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119e63c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f21176f4b80 4 client.28735 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e119d384a0 con 0x55e119df80c0 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.419+0000 7f2105d9b700 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119e63c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.420+0000 7f2105d9b700 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119e63c50 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132f1d0d277c6216 server_cookie=2a462c742108e431 in_seq=0 out_seq=0 2024-05-20T21:10:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.420+0000 7f20edffb700 10 client.28735 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.421+0000 7f20edffb700 1 -- 172.21.15.26:0/508789742 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e119d384a0 con 0x55e119df80c0 2024-05-20T21:10:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.421+0000 7f20edffb700 10 client.28735 handle_command_reply: tid=0 2024-05-20T21:10:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.421+0000 7f21176f4b80 1 client.28735 shutdown 2024-05-20T21:10:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 msgr2=0x55e119dfdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dfdb80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20f80a5820 tx=0x7f20f80a89d0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 shutdown_connections 2024-05-20T21:10:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e119df80c0 0x55e119e63c50 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e119de4320 0x55e119dfa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e119dde8a0 0x55e119dfdb80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f20edffb700 0 client.28735 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 --2- 172.21.15.26:0/508789742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e119ddf260 0x55e119df9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 >> 172.21.15.26:0/508789742 conn(0x55e119d0a240 msgr2=0x55e119de98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 shutdown_connections 2024-05-20T21:10:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 1 -- 172.21.15.26:0/508789742 wait complete. 2024-05-20T21:10:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.422+0000 7f21176f4b80 20 client.28735 trim_cache size 0 max 16384 2024-05-20T21:10:01.427 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20f00b2900 msgr2=0x7f20f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20f00b2900 0x7f20f00b4dc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005e50 tx=0x7f20fc005da0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 msgr2=0x7f2108102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 0x7f2108102860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9a0 tx=0x7f20f800ecb0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 shutdown_connections 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f20f00b2900 0x7f20f00b4dc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2108109650 0x7f21081032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f210806bc70 0x7f2108102860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 --2- 172.21.15.26:0/1673033289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210806c630 0x7f2108102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 >> 172.21.15.26:0/1673033289 conn(0x7f21080782b0 msgr2=0x7f2108101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 shutdown_connections 2024-05-20T21:10:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:01.423+0000 7f210ce5e700 1 -- 172.21.15.26:0/1673033289 wait complete. 2024-05-20T21:10:01.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:01 smithi026 ceph-mon[31990]: pgmap v806: 97 pgs: 97 active+clean; 153 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 13 MiB/s wr, 148 op/s 2024-05-20T21:10:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:01 smithi055 ceph-mon[35213]: pgmap v806: 97 pgs: 97 active+clean; 153 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 13 MiB/s wr, 148 op/s 2024-05-20T21:10:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:01 smithi144 ceph-mon[35660]: pgmap v806: 97 pgs: 97 active+clean; 153 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 13 MiB/s wr, 148 op/s 2024-05-20T21:10:02.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1356 34.24 MB/sec warmup 7 sec latency 185.883 ms 2024-05-20T21:10:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:02 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:02.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:02 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:02.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:02 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:03.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1560 32.63 MB/sec warmup 8 sec latency 129.960 ms 2024-05-20T21:10:03.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-20T21:10:03.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-20T21:10:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:03 smithi055 ceph-mon[35213]: pgmap v807: 97 pgs: 97 active+clean; 212 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 19 MiB/s wr, 185 op/s 2024-05-20T21:10:03.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:03 smithi026 ceph-mon[31990]: pgmap v807: 97 pgs: 97 active+clean; 212 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 19 MiB/s wr, 185 op/s 2024-05-20T21:10:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:03 smithi144 ceph-mon[35660]: pgmap v807: 97 pgs: 97 active+clean; 212 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 19 MiB/s wr, 185 op/s 2024-05-20T21:10:03.709 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:10:04.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1722 29.26 MB/sec warmup 9 sec latency 90.052 ms 2024-05-20T21:10:05.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1880 26.66 MB/sec warmup 10 sec latency 86.024 ms 2024-05-20T21:10:05.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:05 smithi026 ceph-mon[31990]: pgmap v808: 97 pgs: 97 active+clean; 215 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 19 MiB/s wr, 294 op/s 2024-05-20T21:10:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:05 smithi055 ceph-mon[35213]: pgmap v808: 97 pgs: 97 active+clean; 215 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 19 MiB/s wr, 294 op/s 2024-05-20T21:10:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:05 smithi144 ceph-mon[35660]: pgmap v808: 97 pgs: 97 active+clean; 215 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 19 MiB/s wr, 294 op/s 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.696+0000 7f5630b49700 1 -- 172.21.15.26:0/3963371133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 msgr2=0x7f562c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.696+0000 7f5630b49700 1 --2- 172.21.15.26:0/3963371133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c102170 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f561c009990 tx=0x7f561c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 -- 172.21.15.26:0/3963371133 shutdown_connections 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 --2- 172.21.15.26:0/3963371133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 0x7f562c1165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 --2- 172.21.15.26:0/3963371133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c102170 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 --2- 172.21.15.26:0/3963371133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 0x7f562c113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 -- 172.21.15.26:0/3963371133 >> 172.21.15.26:0/3963371133 conn(0x7f562c078ab0 msgr2=0x7f562c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 -- 172.21.15.26:0/3963371133 shutdown_connections 2024-05-20T21:10:05.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.697+0000 7f5630b49700 1 -- 172.21.15.26:0/3963371133 wait complete. 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 Processor -- start 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 -- start start 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 0x7f562c1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 0x7f562c1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f562c119bb0 con 0x7f562c102740 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f562c119a30 con 0x7f562c101d90 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f562a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5630b49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f562c119d30 con 0x7f562c114140 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f562a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c19ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39280/0 (socket says 172.21.15.26:39280) 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f562a59c700 1 -- 172.21.15.26:0/2955024592 learned_addr learned my addr 172.21.15.26:0/2955024592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f562ad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 0x7f562c1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.698+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 0x7f562c1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562a59c700 1 -- 172.21.15.26:0/2955024592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 msgr2=0x7f562c1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562a59c700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 0x7f562c1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562a59c700 1 -- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 msgr2=0x7f562c1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562a59c700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 0x7f562c1a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562a59c700 1 -- 172.21.15.26:0/2955024592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c1a4f40 con 0x7f562c101d90 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562ad9d700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 0x7f562c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 0x7f562c1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:05.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f562a59c700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c19ff80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f561c004790 tx=0x7f561c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:05.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f561b7fe700 1 -- 172.21.15.26:0/2955024592 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c026070 con 0x7f562c101d90 2024-05-20T21:10:05.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f561b7fe700 1 -- 172.21.15.26:0/2955024592 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561c00fb80 con 0x7f562c101d90 2024-05-20T21:10:05.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c1a51d0 con 0x7f562c101d90 2024-05-20T21:10:05.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f561b7fe700 1 -- 172.21.15.26:0/2955024592 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c017650 con 0x7f562c101d90 2024-05-20T21:10:05.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.699+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1acab0 con 0x7f562c101d90 2024-05-20T21:10:05.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.703+0000 7f561b7fe700 1 -- 172.21.15.26:0/2955024592 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f561c029070 con 0x7f562c101d90 2024-05-20T21:10:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.704+0000 7f561b7fe700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56140b2880 0x7f56140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.704+0000 7f561b7fe700 1 -- 172.21.15.26:0/2955024592 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f561c0e32f0 con 0x7f562c101d90 2024-05-20T21:10:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.704+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56140b2880 0x7f56140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.704+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56140b2880 0x7f56140b4d40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f562c113b80 tx=0x7f5620009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.705+0000 7f563b3dfb80 1 Processor -- start 2024-05-20T21:10:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.705+0000 7f563b3dfb80 1 -- start start 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.705+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 0x55d1a172f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.705+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1729920 0x55d1a1729da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.705+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a1736100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.705+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d1a1687df0 con 0x55d1a172a2e0 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 0x55d1a172f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 0x55d1a172f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45950/0 (socket says 172.21.15.26:45950) 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 -- 172.21.15.26:0/2080772802 learned_addr learned my addr 172.21.15.26:0/2080772802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:05.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1729920 0x55d1a1729da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d1a1677990 con 0x55d1a1729920 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d1a167b9d0 con 0x55d1a172f3a0 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562ad9d700 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a1736100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 -- 172.21.15.26:0/2080772802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1729920 msgr2=0x55d1a1729da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1729920 0x55d1a1729da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 -- 172.21.15.26:0/2080772802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 msgr2=0x55d1a1736100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a1736100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 -- 172.21.15.26:0/2080772802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a1621030 con 0x55d1a172f3a0 2024-05-20T21:10:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1729920 0x55d1a1729da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f562a59c700 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 0x55d1a172f780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f561c0a7bb0 tx=0x7f561c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f4c56e91cc575a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.706+0000 7f5618ff9700 1 -- 172.21.15.26:0/2080772802 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0b3070 con 0x55d1a172f3a0 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.707+0000 7f5618ff9700 1 -- 172.21.15.26:0/2080772802 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561c0aa450 con 0x55d1a172f3a0 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.707+0000 7f5618ff9700 1 -- 172.21.15.26:0/2080772802 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c00fd30 con 0x55d1a172f3a0 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.707+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 msgr2=0x55d1a172f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.707+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 0x55d1a172f780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f561c0a7bb0 tx=0x7f561c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 shutdown_connections 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a1736100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172f3a0 0x55d1a172f780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2080772802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1729920 0x55d1a1729da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 >> 172.21.15.26:0/2080772802 conn(0x55d1a16552c0 msgr2=0x55d1a1724460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 shutdown_connections 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2080772802 wait complete. 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 -- start start 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.708+0000 7f563b3dfb80 1 Processor -- start 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 0x55d1a1735de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a17330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a1733620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d1a157e020 con 0x55d1a1729920 2024-05-20T21:10:05.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d1a15dfe20 con 0x55d1a172f3a0 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d1a167b9d0 con 0x55d1a172a2e0 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 0x55d1a1735de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 0x55d1a1735de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35462/0 (socket says 172.21.15.26:35462) 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 -- 172.21.15.26:0/3877371271 learned_addr learned my addr 172.21.15.26:0/3877371271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f563b3dfb80 10 client.0 _ll_register_callbacks cb 0x55d1a16129d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f5629d9b700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a17330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562ad9d700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a1733620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 -- 172.21.15.26:0/3877371271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 msgr2=0x55d1a17330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a17330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 -- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 msgr2=0x55d1a1733620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a1733620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.709+0000 7f562a59c700 1 -- 172.21.15.26:0/3877371271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a15c7950 con 0x55d1a1729920 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f562a59c700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 0x55d1a1735de0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f561c023000 tx=0x7f561c0cd340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3990af7f2da1b7b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f5611ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f5611ffb700 1 -- 172.21.15.26:0/3877371271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0c4060 con 0x55d1a1729920 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561c0aa450 con 0x55d1a1729920 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f5611ffb700 1 -- 172.21.15.26:0/3877371271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f561c0a6bd0 con 0x55d1a1729920 2024-05-20T21:10:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f5611ffb700 1 -- 172.21.15.26:0/3877371271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f561c0aa940 con 0x55d1a1729920 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f563b3dfb80 10 client.19848 fetch_fsmap learned FSMap version 33 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d1a16317e0 con 0x55d1a1729920 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.710+0000 7f5611ffb700 1 -- 172.21.15.26:0/3877371271 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561c0aa450 con 0x55d1a1729920 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f5611ffb700 1 -- 172.21.15.26:0/3877371271 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f561c0aabc0 con 0x55d1a1729920 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f563b3dfb80 10 client.19848 fetch_fsmap finished waiting for FSMap version 33 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f563b3dfb80 10 client.19848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f563b3dfb80 20 client.19848 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a1742e90 0x55d1a1745280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f563b3dfb80 4 client.19848 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a1689d40 con 0x55d1a1742e90 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f5629d9b700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a1742e90 0x55d1a1745280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f5629d9b700 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a1742e90 0x55d1a1745280 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78988b3eeb8697c server_cookie=c2ee95e86fb06580 in_seq=0 out_seq=0 2024-05-20T21:10:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.711+0000 7f5611ffb700 10 client.19848 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f5611ffb700 1 -- 172.21.15.26:0/3877371271 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d1a1689d40 con 0x55d1a1742e90 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f5611ffb700 10 client.19848 handle_command_reply: tid=0 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 client.19848 shutdown 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 msgr2=0x55d1a1735de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 0x55d1a1735de0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f561c023000 tx=0x7f561c0cd340 comp rx=0 tx=0).stop 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 shutdown_connections 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a1742e90 0x55d1a1745280 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a1729920 0x55d1a1735de0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f5611ffb700 0 client.19848 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a1733620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/3877371271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a17330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.712+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 >> 172.21.15.26:0/3877371271 conn(0x55d1a16552c0 msgr2=0x55d1a1723b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.713+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 shutdown_connections 2024-05-20T21:10:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.713+0000 7f563b3dfb80 1 -- 172.21.15.26:0/3877371271 wait complete. 2024-05-20T21:10:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.713+0000 7f563b3dfb80 20 client.19848 trim_cache size 0 max 16384 2024-05-20T21:10:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 Processor -- start 2024-05-20T21:10:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 -- start start 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a17242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a17307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a1742e90 0x55d1a1730ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1b6950 con 0x55d1a172a2e0 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1b4150 con 0x55d1a172f3a0 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1b1fa0 con 0x55d1a1742e90 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f5629d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a17307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.748+0000 7f562a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a17242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f5629d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a17307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39308/0 (socket says 172.21.15.26:39308) 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f5629d9b700 1 -- 172.21.15.26:0/1876212680 learned_addr learned my addr 172.21.15.26:0/1876212680 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a17242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35478/0 (socket says 172.21.15.26:35478) 2024-05-20T21:10:05.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a1742e90 0x55d1a1730ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 -- 172.21.15.26:0/1876212680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 msgr2=0x55d1a17307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a17307a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 -- 172.21.15.26:0/1876212680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 msgr2=0x55d1a17242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a17242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 -- 172.21.15.26:0/1876212680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a1725c20 con 0x55d1a1742e90 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f5629d9b700 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a17307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562a59c700 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a17242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f562ad9d700 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a1742e90 0x55d1a1730ce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f562400e980 tx=0x7f5624009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f9c5cde46ba6e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f5613fff700 1 -- 172.21.15.26:0/1876212680 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624004db0 con 0x55d1a1742e90 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f5613fff700 1 -- 172.21.15.26:0/1876212680 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56240093a0 con 0x55d1a1742e90 2024-05-20T21:10:05.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.749+0000 7f5613fff700 1 -- 172.21.15.26:0/1876212680 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624007960 con 0x55d1a1742e90 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 -- 172.21.15.26:0/1876212680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a1742e90 msgr2=0x55d1a1730ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a1742e90 0x55d1a1730ce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f562400e980 tx=0x7f5624009920 comp rx=0 tx=0).stop 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 -- 172.21.15.26:0/1876212680 shutdown_connections 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172a2e0 0x55d1a17242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a1742e90 0x55d1a1730ce0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/1876212680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a172f3a0 0x55d1a17307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 -- 172.21.15.26:0/1876212680 >> 172.21.15.26:0/1876212680 conn(0x55d1a16552c0 msgr2=0x55d1a1730090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 -- 172.21.15.26:0/1876212680 shutdown_connections 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.750+0000 7f563b3dfb80 1 -- 172.21.15.26:0/1876212680 wait complete. 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 -- start start 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 Processor -- start 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a1736cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172f3a0 0x55d1a17371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1742e90 0x55d1a17ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1b2610 con 0x55d1a172f3a0 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1b3a80 con 0x55d1a1742e90 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1b2ce0 con 0x55d1a172a2e0 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f563b3dfb80 10 client.0 _ll_register_callbacks cb 0x55d1a16129d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f562a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a1736cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f562ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1742e90 0x55d1a17ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f5629d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172f3a0 0x55d1a17371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f562a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a1736cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45988/0 (socket says 172.21.15.26:45988) 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.751+0000 7f562a59c700 1 -- 172.21.15.26:0/2242348405 learned_addr learned my addr 172.21.15.26:0/2242348405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f562ad9d700 1 -- 172.21.15.26:0/2242348405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 msgr2=0x55d1a1736cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f562ad9d700 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a1736cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f562ad9d700 1 -- 172.21.15.26:0/2242348405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172f3a0 msgr2=0x55d1a17371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f562ad9d700 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172f3a0 0x55d1a17371f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f562ad9d700 1 -- 172.21.15.26:0/2242348405 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a17addd0 con 0x55d1a1742e90 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f5629d9b700 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172f3a0 0x55d1a17371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f562ad9d700 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1742e90 0x55d1a17ad7e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f562400e980 tx=0x7f5624017840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3371d5191ef41c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f5611ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f5611ffb700 1 -- 172.21.15.26:0/2242348405 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624017cf0 con 0x55d1a1742e90 2024-05-20T21:10:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f5611ffb700 1 -- 172.21.15.26:0/2242348405 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f5624007500 con 0x55d1a1742e90 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56240093a0 con 0x55d1a1742e90 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f5611ffb700 1 -- 172.21.15.26:0/2242348405 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5624007700 con 0x55d1a1742e90 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f563b3dfb80 10 client.29057 fetch_fsmap learned FSMap version 33 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.752+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d1a17af0f0 con 0x55d1a1742e90 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f5611ffb700 1 -- 172.21.15.26:0/2242348405 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56240093a0 con 0x55d1a1742e90 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f5611ffb700 1 -- 172.21.15.26:0/2242348405 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f562401e070 con 0x55d1a1742e90 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f563b3dfb80 10 client.29057 fetch_fsmap finished waiting for FSMap version 33 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f563b3dfb80 10 client.29057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f563b3dfb80 20 client.29057 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a17af7d0 0x55d1a17b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f563b3dfb80 4 client.29057 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-20T21:10:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d1a1683520 con 0x55d1a17af7d0 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.753+0000 7f562a59c700 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a17af7d0 0x55d1a17b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.754+0000 7f5611ffb700 10 client.29057 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.754+0000 7f562a59c700 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a17af7d0 0x55d1a17b1bf0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9bb0cf2a7b58712 server_cookie=68adfaaa499126c2 in_seq=0 out_seq=0 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.754+0000 7f5611ffb700 1 -- 172.21.15.26:0/2242348405 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d1a1683520 con 0x55d1a17af7d0 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.754+0000 7f5611ffb700 10 client.29057 handle_command_reply: tid=0 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 client.29057 shutdown 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1742e90 msgr2=0x55d1a17ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1742e90 0x55d1a17ad7e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f562400e980 tx=0x7f5624017840 comp rx=0 tx=0).stop 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 shutdown_connections 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d1a17af7d0 0x55d1a17b1bf0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1a1742e90 0x55d1a17ad7e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d1a172a2e0 0x55d1a1736cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 --2- 172.21.15.26:0/2242348405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1a172f3a0 0x55d1a17371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f5611ffb700 0 client.29057 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 >> 172.21.15.26:0/2242348405 conn(0x55d1a16552c0 msgr2=0x55d1a1730920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 shutdown_connections 2024-05-20T21:10:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 1 -- 172.21.15.26:0/2242348405 wait complete. 2024-05-20T21:10:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.755+0000 7f563b3dfb80 20 client.29057 trim_cache size 0 max 16384 2024-05-20T21:10:05.760 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:10:05.760 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-05-20T21:10:05.761 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "86207ae3-de69-49a1-9f79-81898433bc47", 2024-05-20T21:10:05.761 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-05-20T21:10:05.761 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:10:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56140b2880 msgr2=0x7f56140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56140b2880 0x7f56140b4d40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f562c113b80 tx=0x7f5620009260 comp rx=0 tx=0).stop 2024-05-20T21:10:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 msgr2=0x7f562c19ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c19ff80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f561c004790 tx=0x7f561c0047c0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 shutdown_connections 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c114140 0x7f562c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f562c101d90 0x7f562c19ff80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f56140b2880 0x7f56140b4d40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 --2- 172.21.15.26:0/2955024592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f562c102740 0x7f562c1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.756+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 >> 172.21.15.26:0/2955024592 conn(0x7f562c078ab0 msgr2=0x7f562c1089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.757+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 shutdown_connections 2024-05-20T21:10:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:05.757+0000 7f5630b49700 1 -- 172.21.15.26:0/2955024592 wait complete. 2024-05-20T21:10:06.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2078 24.85 MB/sec warmup 11 sec latency 126.860 ms 2024-05-20T21:10:06.206 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.808+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f6 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP auth v8 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.290883+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=1 authpin=1 tempexporting=0 scrubqueue=0 0x5606c06b8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001f6:[<0x10000002d67/BENCHS.LWP v2>,<0x10000002c3c/WORDPRO v57>,<0x10000002c36/~dmtmp v32>,<0x10000000007/client6 v1521>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x200000001f6:[<0x10000002d67/BENCHS.LWP v8>,<0x10000002c3c/WORDPRO v329>,<0x10000002c36/~dmtmp v47>,<0x10000000007/client6 v1543>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.206 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.809+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f8 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS auth v10 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.334497+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=1 authpin=1 tempexporting=0 scrubqueue=0 0x5606c06b8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001f8:[<0x10000002d67/RESULTS.XLS v4>,<0x10000002c3c/WORDPRO v57>,<0x10000002c36/~dmtmp v32>,<0x10000000007/client6 v1521>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x200000001f8:[<0x10000002d67/RESULTS.XLS v10>,<0x10000002c3c/WORDPRO v329>,<0x10000002c36/~dmtmp v47>,<0x10000000007/client6 v1543>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.206 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.810+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fb [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP auth v12 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.371556+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=1 authpin=1 tempexporting=0 scrubqueue=0 0x5606c06b9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001fb:[<0x10000002d67/NEWS1_1.LWP v6>,<0x10000002c3c/WORDPRO v57>,<0x10000002c36/~dmtmp v32>,<0x10000000007/client6 v1521>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x200000001fb:[<0x10000002d67/NEWS1_1.LWP v12>,<0x10000002c3c/WORDPRO v329>,<0x10000002c36/~dmtmp v47>,<0x10000000007/client6 v1543>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.825+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cfa [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ auth v337 ap=1 f(v0 m2024-05-20T21:09:56.133266+0000 11=11+0) n(v0 rc2024-05-20T21:09:56.140306+0000 b2861056 12=11+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/-@100},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0745b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002cfa:[<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002cfa:[<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:09:56.133266+0000 11=11+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:09:56.140306+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:09:56.133266+0000 11=11+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:09:56.140306+0000 b2861056 12=11+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.825+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ca1 [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/ auth v315 ap=1 f(v0 m2024-05-20T21:09:55.567973+0000 2=2+0) n(v0 rc2024-05-20T21:09:55.577654+0000 b520192 3=2+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/-@100},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0746100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002ca1:[<0x10000002c40/EXCEL v14>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002ca1:[<0x10000002c40/EXCEL v315>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:09:55.567973+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:09:55.577654+0000 b520192 3=2+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:09:55.567973+0000 2=2+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:09:55.577654+0000 b520192 3=2+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.826+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002c7c [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/ auth{0=1} v417 ap=1 f(v0 m2024-05-20T21:10:02.281682+0000 5=5+0) n(v0 rc2024-05-20T21:10:02.281682+0000 b807639 6=5+1) (inest mix dirty) (ifile excl) caps={15126=pAsLsXsFsx/-@176},l=15126 | dirtyscattered=1 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5606c0746680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002c7c:[<0x10000002c40/WORD v294>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002c7c:[<0x10000002c40/WORD v417>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:10:02.281682+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:10:02.281682+0000 b807639 6=5+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:10:02.281682+0000 5=5+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:10:02.281682+0000 b807639 6=5+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.826+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d73 [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/ auth v419 ap=1 f(v0 m2024-05-20T21:09:56.312642+0000 3=3+0) n(v0 rc2024-05-20T21:09:56.317140+0000 b780288 4=3+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/-@100},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0747180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002d73:[<0x10000002c40/WORDPRO v58>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002d73:[<0x10000002c40/WORDPRO v419>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:09:56.312642+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:09:56.317140+0000 b780288 4=3+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:09:56.312642+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:09:56.317140+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.827+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e4e [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/ auth v473 ap=1 f(v0 m2024-05-20T21:09:57.253117+0000 3=3+0) n(v0 rc2024-05-20T21:10:00.409737+0000 b3244032 4=3+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/-@100},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0746c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002e4e:[<0x10000002c40/SEED v284>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002e4e:[<0x10000002c40/SEED v473>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:09:57.253117+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:10:00.409737+0000 b3244032 4=3+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:09:57.253117+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:10:00.409737+0000 b3244032 4=3+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.827+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dab [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth v645 ap=1 f(v0 m2024-05-20T21:10:05.381818+0000 24=24+0) n(v0 rc2024-05-20T21:10:05.381818+0000 b5210655 25=24+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/AsLsXsFsx@279},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5606c0747700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002dab:[<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002dab:[<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:10:05.381818+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:10:05.381818+0000 b5210655 25=24+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:10:05.381818+0000 24=24+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:10:05.381818+0000 b5210655 25=24+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.827+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d90 [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/ auth v425 ap=1 f(v0 m2024-05-20T21:09:56.414167+0000 3=3+0) n(v0 rc2024-05-20T21:09:56.422056+0000 b780288 4=3+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/-@101},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002d90:[<0x10000002c40/PM v66>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002d90:[<0x10000002c40/PM v425>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:09:56.414167+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:09:56.422056+0000 b780288 4=3+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:09:56.414167+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:09:56.422056+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.828+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cbf [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ auth v627 ap=1 f(v0 m2024-05-20T21:10:00.310364+0000 7=7+0) n(v0 rc2024-05-20T21:10:00.310364+0000 b1462582 8=7+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/-@101},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002cbf:[<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002cbf:[<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:10:00.310364+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:10:00.310364+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:10:00.310364+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:10:00.310364+0000 b1462582 8=7+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.829+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e3f [...2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/ auth v467 ap=1 f(v0 m2024-05-20T21:09:57.126383+0000 1=1+0) n(v0 rc2024-05-20T21:09:57.133366+0000 b260096 2=1+1) (inest lock) (ifile excl) caps={15126=pAsLsXsFsx/AsLsXsFsx@102},l=15126 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002e3f:[<0x10000002c40/ACCESS v112>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(2)0x10000002e3f:[<0x10000002c40/ACCESS v467>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-20T21:09:57.126383+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-05-20T21:09:57.133366+0000 b260096 2=1+1)","memory_value.dirstat":"f(v0 m2024-05-20T21:09:57.126383+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-20T21:09:57.133366+0000 b260096 2=1+1)","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.834+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d5f [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ROSE.CPT auth v42 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.100758+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d5f:[<0x10000002cfa/ROSE.CPT v20>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d5f:[<0x10000002cfa/ROSE.CPT v42>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.207 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.834+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d38 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TEAPOT2.CPT auth v34 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.970788+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d38:[<0x10000002cfa/TEAPOT2.CPT v12>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d38:[<0x10000002cfa/TEAPOT2.CPT v34>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.835+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d2e [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/MAILBOX2.CPT auth v32 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.939028+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d2e:[<0x10000002cfa/MAILBOX2.CPT v10>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d2e:[<0x10000002cfa/MAILBOX2.CPT v32>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.835+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d69 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CORELDRW.CDT auth v44 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.140306+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d69:[<0x10000002cfa/CORELDRW.CDT v22>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d69:[<0x10000002cfa/CORELDRW.CDT v44>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.835+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d55 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC2.CDR auth v40 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.070755+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d55:[<0x10000002cfa/GRAPHIC2.CDR v18>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d55:[<0x10000002cfa/GRAPHIC2.CDR v40>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.836+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d1a [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ARTISTIC.ACL auth v28 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.880770+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d1a:[<0x10000002cfa/ARTISTIC.ACL v6>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d1a:[<0x10000002cfa/ARTISTIC.ACL v28>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.836+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d42 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/SIGN2.CPT auth v36 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.004437+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d42:[<0x10000002cfa/SIGN2.CPT v14>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d42:[<0x10000002cfa/SIGN2.CPT v36>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.837+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d10 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRBARS.CFG auth v26 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.836615+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d10:[<0x10000002cfa/CDRBARS.CFG v4>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d10:[<0x10000002cfa/CDRBARS.CFG v26>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.837+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d4b [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TRAIN.CDR auth v38 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.037040+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d4b:[<0x10000002cfa/TRAIN.CDR v16>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d4b:[<0x10000002cfa/TRAIN.CDR v38>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.837+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d24 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC1.CDR auth v30 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.908298+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d24:[<0x10000002cfa/GRAPHIC1.CDR v8>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d24:[<0x10000002cfa/GRAPHIC1.CDR v30>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.837+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d04 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRROLS.CFG auth v24 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.802066+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d04:[<0x10000002cfa/CDRROLS.CFG v2>,<0x10000002c40/COREL v50>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d04:[<0x10000002cfa/CDRROLS.CFG v24>,<0x10000002c40/COREL v337>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.208 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.837+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002caa [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/PCMAGCD.XLS auth v6 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.543633+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002caa:[<0x10000002ca1/PCMAGCD.XLS v2>,<0x10000002c40/EXCEL v14>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002caa:[<0x10000002ca1/PCMAGCD.XLS v6>,<0x10000002c40/EXCEL v315>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.837+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cb5 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/SALES.XLS auth v8 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.577654+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002cb5:[<0x10000002ca1/SALES.XLS v4>,<0x10000002c40/EXCEL v14>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002cb5:[<0x10000002ca1/SALES.XLS v8>,<0x10000002c40/EXCEL v315>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.838+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002c8e [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/BASEMACH.DOC auth v36 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.473642+0000 b260096 1=1+0) caps={15126=pAsLsXsFscr/-@14},l=15126(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c083fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002c8e:[<0x10000002c7c/BASEMACH.DOC v4>,<0x10000002c40/WORD v294>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002c8e:[<0x10000002c7c/BASEMACH.DOC v36>,<0x10000002c40/WORD v417>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.838+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002c98 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/FACTS.DOC auth v60 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.502912+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@14},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0840680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002c98:[<0x10000002c7c/FACTS.DOC v6>,<0x10000002c40/WORD v294>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002c98:[<0x10000002c7c/FACTS.DOC v60>,<0x10000002c40/WORD v417>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.839+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002c85 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/CHAP10.DOC auth v8 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.448578+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@12},l=15126 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5606c0840c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002c85:[<0x10000002c7c/CHAP10.DOC v8>,<0x10000002c40/WORD v294>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002c85:[<0x10000002c7c/CHAP10.DOC v8>,<0x10000002c40/WORD v417>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.839+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e54 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/SMALL.FIL auth v14 ap=1 s=327680 n(v0 rc2024-05-20T21:10:00.387937+0000 b327680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15126 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0841700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e54:[<0x10000002e4e/SMALL.FIL v8>,<0x10000002c40/SEED v284>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e54:[<0x10000002e4e/SMALL.FIL v14>,<0x10000002c40/SEED v473>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.839+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e59 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/MEDIUM.FIL auth v16 ap=1 s=1048576 n(v0 rc2024-05-20T21:10:00.397882+0000 b1048576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15126 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0844000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e59:[<0x10000002e4e/MEDIUM.FIL v10>,<0x10000002c40/SEED v284>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e59:[<0x10000002e4e/MEDIUM.FIL v16>,<0x10000002c40/SEED v473>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.839+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e5e [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/LARGE.FIL auth v18 ap=1 s=1867776 n(v0 rc2024-05-20T21:10:00.409737+0000 b1867776 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15126 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0844580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e5e:[<0x10000002e4e/LARGE.FIL v12>,<0x10000002c40/SEED v284>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e5e:[<0x10000002e4e/LARGE.FIL v18>,<0x10000002c40/SEED v473>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.840+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d7a [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/BENCHS.LWP auth v8 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.246232+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0844b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d7a:[<0x10000002d73/BENCHS.LWP v2>,<0x10000002c40/WORDPRO v58>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d7a:[<0x10000002d73/BENCHS.LWP v8>,<0x10000002c40/WORDPRO v419>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.840+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d81 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/RESULTS.XLS auth v10 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.287824+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0845080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d81:[<0x10000002d73/RESULTS.XLS v4>,<0x10000002c40/WORDPRO v58>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d81:[<0x10000002d73/RESULTS.XLS v10>,<0x10000002c40/WORDPRO v419>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.840+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d88 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/NEWS1_1.LWP auth v12 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.317140+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0845600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d88:[<0x10000002d73/NEWS1_1.LWP v6>,<0x10000002c40/WORDPRO v58>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d88:[<0x10000002d73/NEWS1_1.LWP v12>,<0x10000002c40/WORDPRO v419>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.840+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dba [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.PX auth v206 ap=1 s=260096 n(v0 rc2024-05-20T21:10:03.261229+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0845b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dba:[<0x10000002dab/COURSES.PX v4>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dba:[<0x10000002dab/COURSES.PX v206>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.209 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.841+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dc1 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.X04 auth v208 ap=1 s=260096 n(v0 rc2024-05-20T21:10:03.261541+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0846100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dc1:[<0x10000002dab/COURSES.X04 v6>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dc1:[<0x10000002dab/COURSES.X04 v208>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.841+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002de3 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG0 auth v136 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.696754+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0846680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002de3:[<0x10000002dab/STUDENTS.YG0 v16>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002de3:[<0x10000002dab/STUDENTS.YG0 v136>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.841+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e0d [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.PX auth v148 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.887980+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0846c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e0d:[<0x10000002dab/ENROLL.PX v28>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e0d:[<0x10000002dab/ENROLL.PX v148>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.841+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dd5 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.PX auth v132 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.639664+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0847180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dd5:[<0x10000002dab/STUDENTS.PX v12>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dd5:[<0x10000002dab/STUDENTS.PX v132>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.842+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e28 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.TV auth v238 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.014658+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0847700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e28:[<0x10000002dab/STUDENTS.TV v36>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e28:[<0x10000002dab/STUDENTS.TV v238>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.842+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e2f [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.FSL auth v240 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.045158+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e2f:[<0x10000002dab/STUDENTS.FSL v38>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e2f:[<0x10000002dab/STUDENTS.FSL v240>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.842+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002df1 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG1 auth v140 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.759631+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002df1:[<0x10000002dab/STUDENTS.YG1 v20>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002df1:[<0x10000002dab/STUDENTS.YG1 v140>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.843+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e1b [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.Y03 auth v152 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.951408+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e1b:[<0x10000002dab/ENROLL.Y03 v32>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e1b:[<0x10000002dab/ENROLL.Y03 v152>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.844+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e36 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.FSL auth v242 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.077099+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e36:[<0x10000002dab/COURSES.FSL v40>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e36:[<0x10000002dab/COURSES.FSL v242>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.844+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002df8 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.VAL auth v142 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.790414+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002df8:[<0x10000002dab/STUDENTS.VAL v22>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002df8:[<0x10000002dab/STUDENTS.VAL v142>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.845+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002db2 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.DB auth v204 ap=1 s=260096 n(v0 rc2024-05-20T21:10:03.260729+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002db2:[<0x10000002dab/COURSES.DB v2>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002db2:[<0x10000002dab/COURSES.DB v204>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.210 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.845+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dc8 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.Y04 auth v88 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.572105+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dc8:[<0x10000002dab/COURSES.Y04 v8>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dc8:[<0x10000002dab/COURSES.Y04 v88>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.846+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e14 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.X03 auth v150 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.918445+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e14:[<0x10000002dab/ENROLL.X03 v30>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e14:[<0x10000002dab/ENROLL.X03 v150>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.846+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dff [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.MB auth v144 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.822099+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dff:[<0x10000002dab/STUDENTS.MB v24>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dff:[<0x10000002dab/STUDENTS.MB v144>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.847+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e22 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.VAL auth v154 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.982496+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e22:[<0x10000002dab/ENROLL.VAL v34>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e22:[<0x10000002dab/ENROLL.VAL v154>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.847+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e06 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.DB auth v146 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.856672+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c084d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e06:[<0x10000002dab/ENROLL.DB v26>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e06:[<0x10000002dab/ENROLL.DB v146>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.848+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dea [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG1 auth v138 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.727393+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0852000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dea:[<0x10000002dab/STUDENTS.XG1 v18>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dea:[<0x10000002dab/STUDENTS.XG1 v138>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.849+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ddc [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG0 auth v134 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.670245+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0852580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002ddc:[<0x10000002dab/STUDENTS.XG0 v14>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002ddc:[<0x10000002dab/STUDENTS.XG0 v134>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.850+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dcf [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.DB auth v130 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.608432+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0852b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dcf:[<0x10000002dab/STUDENTS.DB v10>,<0x10000002c40/PARADOX v108>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dcf:[<0x10000002dab/STUDENTS.DB v130>,<0x10000002c40/PARADOX v645>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.851+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d97 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/T1.XLS auth v8 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.369675+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0853080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d97:[<0x10000002d90/T1.XLS v2>,<0x10000002c40/PM v66>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d97:[<0x10000002d90/T1.XLS v8>,<0x10000002c40/PM v425>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.851+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d9d [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/EVENTS.DOC auth v10 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.393779+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0853600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002d9d:[<0x10000002d90/EVENTS.DOC v4>,<0x10000002c40/PM v66>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002d9d:[<0x10000002d90/EVENTS.DOC v10>,<0x10000002c40/PM v425>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.211 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.851+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002da4 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/MOVED.DOC auth v12 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.422056+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0853b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002da4:[<0x10000002d90/MOVED.DOC v6>,<0x10000002c40/PM v66>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002da4:[<0x10000002d90/MOVED.DOC v12>,<0x10000002c40/PM v425>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.852+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e92 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT auth v174 ap=1 s=85026 n(v0 rc2024-05-20T21:09:58.737094+0000 b85026 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0854100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e92:[<0x10000002cbf/NEWPCB.PPT v102>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e92:[<0x10000002cbf/NEWPCB.PPT v174>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.853+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cc8 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PCBENCHM.PPT auth v172 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.561051+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0854680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002cc8:[<0x10000002cbf/PCBENCHM.PPT v16>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002cc8:[<0x10000002cbf/PCBENCHM.PPT v172>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.853+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cd2 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ZD16.BMP auth v170 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.662044+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0854c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002cd2:[<0x10000002cbf/ZD16.BMP v40>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002cd2:[<0x10000002cbf/ZD16.BMP v170>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.853+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ebe [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWTIPS.PPT auth v166 ap=1 s=77076 n(v0 rc2024-05-20T21:10:00.276528+0000 b77076 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15126 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5606c0855180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002ebe:[<0x10000002cbf/NEWTIPS.PPT v166>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002ebe:[<0x10000002cbf/NEWTIPS.PPT v166>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.854+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cf0 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TRIDOTS.POT auth v178 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.748791+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0855700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002cf0:[<0x10000002cbf/TRIDOTS.POT v146>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002cf0:[<0x10000002cbf/TRIDOTS.POT v178>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.855+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ce6 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TIPS.PPT auth v176 ap=1 s=260096 n(v0 rc2024-05-20T21:09:59.052960+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c085e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002ce6:[<0x10000002cbf/TIPS.PPT v110>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002ce6:[<0x10000002cbf/TIPS.PPT v176>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.855+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002cdc [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PPTOOLS1.PPA auth v72 ap=1 s=260096 n(v0 rc2024-05-20T21:09:55.690101+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15126 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5606c085e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002cdc:[<0x10000002cbf/PPTOOLS1.PPA v72>,<0x10000002c40/PWRPNT v278>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002cdc:[<0x10000002cbf/PPTOOLS1.PPA v72>,<0x10000002c40/PWRPNT v627>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.855+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e46 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/FASTENER.MDB auth v4 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.133366+0000 b260096 1=1+0) caps={15126=pAsLsXsFsc/-@1},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c085eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e46:[<0x10000002e3f/FASTENER.MDB v2>,<0x10000002c40/ACCESS v112>,<0x10000002c35/~dmtmp v36>,<0x10000000007/client5 v1513>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e46:[<0x10000002e3f/FASTENER.MDB v4>,<0x10000002c40/ACCESS v467>,<0x10000002c35/~dmtmp v51>,<0x10000000007/client5 v1541>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.857+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dbf [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX auth v84 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.530895+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0698b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dbf:[<0x10000002db1/COURSES.PX v4>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dbf:[<0x10000002db1/COURSES.PX v84>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.858+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dc5 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04 auth v86 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.561412+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c069a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dc5:[<0x10000002db1/COURSES.X04 v6>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dc5:[<0x10000002db1/COURSES.X04 v86>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.212 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.858+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002de7 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0 auth v148 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.719578+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606bf3f2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002de7:[<0x10000002db1/STUDENTS.YG0 v16>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002de7:[<0x10000002db1/STUDENTS.YG0 v148>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.858+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e11 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX auth v160 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.907844+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c073e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e11:[<0x10000002db1/ENROLL.PX v28>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e11:[<0x10000002db1/ENROLL.PX v160>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.859+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dda [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX auth v144 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.652344+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c073e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dda:[<0x10000002db1/STUDENTS.PX v12>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dda:[<0x10000002db1/STUDENTS.PX v144>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.859+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002df5 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1 auth v152 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.781477+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c073f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002df5:[<0x10000002db1/STUDENTS.YG1 v20>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002df5:[<0x10000002db1/STUDENTS.YG1 v152>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.860+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e3a [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL auth v80 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.091366+0000 b260096 1=1+0) caps={15126=pAsLsXsFscr/-@18},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0740100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e3a:[<0x10000002db1/COURSES.FSL v40>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e3a:[<0x10000002db1/COURSES.FSL v80>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.860+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e1e [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03 auth v164 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.971559+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c073fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e1e:[<0x10000002db1/ENROLL.Y03 v32>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e1e:[<0x10000002db1/ENROLL.Y03 v164>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.860+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e33 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL auth v78 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.061776+0000 b260096 1=1+0) caps={15126=pAsLsXsFscr/-@18},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c073f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e33:[<0x10000002db1/STUDENTS.FSL v38>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e33:[<0x10000002db1/STUDENTS.FSL v78>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.861+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e2c [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV auth v76 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.030124+0000 b260096 1=1+0) caps={15126=pAsLsXsFscr/-@18},l=15126(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c073eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e2c:[<0x10000002db1/STUDENTS.TV v36>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e2c:[<0x10000002db1/STUDENTS.TV v76>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.861+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dfc [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL auth v154 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.814215+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0740680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dfc:[<0x10000002db1/STUDENTS.VAL v22>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dfc:[<0x10000002db1/STUDENTS.VAL v154>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.861+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002db8 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB auth v82 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.496855+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0740c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002db8:[<0x10000002db1/COURSES.DB v2>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002db8:[<0x10000002db1/COURSES.DB v82>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.862+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dcc [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04 auth v88 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.593551+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0741180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dcc:[<0x10000002db1/COURSES.Y04 v8>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dcc:[<0x10000002db1/COURSES.Y04 v88>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.213 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.864+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e18 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03 auth v162 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.944879+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0741700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e18:[<0x10000002db1/ENROLL.X03 v30>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e18:[<0x10000002db1/ENROLL.X03 v162>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.214 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.865+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e03 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB auth v156 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.847977+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0744000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e03:[<0x10000002db1/STUDENTS.MB v24>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e03:[<0x10000002db1/STUDENTS.MB v156>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.214 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.866+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e25 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL auth v166 ap=1 s=260096 n(v0 rc2024-05-20T21:09:57.001020+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0744580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e25:[<0x10000002db1/ENROLL.VAL v34>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e25:[<0x10000002db1/ENROLL.VAL v166>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.214 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.867+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002e0a [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB auth v158 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.876850+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0744b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002e0a:[<0x10000002db1/ENROLL.DB v26>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002e0a:[<0x10000002db1/ENROLL.DB v158>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.214 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.868+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dee [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1 auth v150 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.751257+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0745080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dee:[<0x10000002db1/STUDENTS.XG1 v18>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dee:[<0x10000002db1/STUDENTS.XG1 v150>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.214 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.870+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002de1 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0 auth v146 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.687387+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c0745600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002de1:[<0x10000002db1/STUDENTS.XG0 v14>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002de1:[<0x10000002db1/STUDENTS.XG0 v146>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.214 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:10:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:10:05.871+0000 7fb127afe700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002dd3 [2,head] /volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB auth v142 ap=1 s=260096 n(v0 rc2024-05-20T21:09:56.621783+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15126=0-4194304@1} caps={15126=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15126 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5606c06e3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002dd3:[<0x10000002db1/STUDENTS.DB v10>,<0x10000002c49/PARADOX v108>,<0x10000002c3b/~dmtmp v36>,<0x10000000007/client3 v1519>,<0x10000000006/clients v1241>,<0x10000000005/tmp v1218>,<0x10000000003/client.0 v1187>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1179>,<0x10000000001/sv_0 v1183>,<0x10000000000/qa v1170>,<0x1/volumes v1097>]//[]","memoryvalue":"(3)0x10000002dd3:[<0x10000002db1/STUDENTS.DB v142>,<0x10000002c49/PARADOX v465>,<0x10000002c3b/~dmtmp v45>,<0x10000000007/client3 v1559>,<0x10000000006/clients v1255>,<0x10000000005/tmp v1220>,<0x10000000003/client.0 v1190>,<0x10000000002/39566b0c-c8a4-4314-b820-c3de24137f95 v1181>,<0x10000000001/sv_0 v1173>,<0x10000000000/qa v1172>,<0x1/volumes v1075>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T21:10:06.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:10:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:10:07.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2243 23.52 MB/sec warmup 12 sec latency 132.903 ms 2024-05-20T21:10:07.385 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub queued for path: / 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: active paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x200000001f8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x200000001fb (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cfa (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002ca1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002c7c (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d73 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e4e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dab (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d90 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cbf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e3f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d5f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d38 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d2e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d69 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d55 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d1a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d42 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d10 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d4b (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d24 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d04 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002caa (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cb5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002c8e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002c98 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002c85 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e54 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e59 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e5e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d7a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d81 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d88 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dba (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dc1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002de3 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e0d (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dd5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e28 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e2f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002df1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e1b (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e36 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002df8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002db2 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dc8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e14 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dff (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e22 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e06 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dea (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002ddc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dcf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d97 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002d9d (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002da4 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e92 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cc8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cd2 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002ebe (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWTIPS.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cf0 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002ce6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002cdc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e46 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dbf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dc5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002de7 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e11 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dda (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002df5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e3a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e1e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e33 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e2c (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dfc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002db8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dcc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e18 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e03 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e25 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002e0a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dee (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002de1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: Scrub error on inode 0x10000002dd3 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: pgmap v809: 97 pgs: 97 active+clean; 209 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 19 MiB/s wr, 349 op/s 2024-05-20T21:10:07.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:07 smithi026 ceph-mon[31990]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub queued for path: / 2024-05-20T21:10:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:10:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: active paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x200000001f8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x200000001fb (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cfa (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002ca1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002c7c (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d73 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e4e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dab (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d90 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cbf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e3f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d5f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d38 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d2e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d69 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d55 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d1a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d42 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d10 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d4b (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d24 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d04 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002caa (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cb5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002c8e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002c98 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002c85 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e54 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e59 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e5e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d7a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d81 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d88 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dba (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dc1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002de3 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e0d (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dd5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e28 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e2f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002df1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e1b (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e36 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002df8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002db2 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dc8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e14 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dff (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e22 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e06 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dea (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002ddc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dcf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d97 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002d9d (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002da4 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e92 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cc8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cd2 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002ebe (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWTIPS.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cf0 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002ce6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002cdc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e46 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dbf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dc5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002de7 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e11 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dda (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002df5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e3a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e1e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e33 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e2c (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dfc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002db8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dcc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e18 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e03 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e25 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002e0a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dee (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002de1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: Scrub error on inode 0x10000002dd3 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: pgmap v809: 97 pgs: 97 active+clean; 209 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 19 MiB/s wr, 349 op/s 2024-05-20T21:10:07.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:07 smithi055 ceph-mon[35213]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub queued for path: / 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: active paths [/] 2024-05-20T21:10:07.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x200000001f8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x200000001fb (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cfa (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002ca1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002c7c (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d73 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e4e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dab (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d90 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cbf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e3f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.552 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d5f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d38 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d2e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d69 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d55 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d1a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d42 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d10 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d4b (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d24 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d04 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002caa (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cb5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002c8e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.553 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002c98 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002c85 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e54 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e59 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e5e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d7a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d81 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d88 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dba (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dc1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002de3 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.554 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e0d (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dd5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e28 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e2f (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002df1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e1b (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e36 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002df8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002db2 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dc8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e14 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dff (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e22 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.555 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e06 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dea (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002ddc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dcf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d97 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002d9d (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002da4 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e92 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cc8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cd2 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002ebe (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWTIPS.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cf0 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002ce6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002cdc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e46 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dbf (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dc5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002de7 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e11 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dda (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002df5 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e3a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e1e (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e33 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e2c (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.557 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dfc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002db8 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dcc (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e18 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e03 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e25 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002e0a (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dee (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002de1 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: Scrub error on inode 0x10000002dd3 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: pgmap v809: 97 pgs: 97 active+clean; 209 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 19 MiB/s wr, 349 op/s 2024-05-20T21:10:07.558 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:07 smithi144 ceph-mon[35660]: scrub summary: idle+waiting paths [/] 2024-05-20T21:10:08.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2418 22.24 MB/sec warmup 13 sec latency 87.899 ms 2024-05-20T21:10:09.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2600 21.35 MB/sec warmup 14 sec latency 81.905 ms 2024-05-20T21:10:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:09 smithi055 ceph-mon[35213]: pgmap v810: 97 pgs: 97 active+clean; 213 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 18 MiB/s wr, 407 op/s 2024-05-20T21:10:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:09 smithi026 ceph-mon[31990]: pgmap v810: 97 pgs: 97 active+clean; 213 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 18 MiB/s wr, 407 op/s 2024-05-20T21:10:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.486+0000 7ff48c00d700 1 -- 172.21.15.26:0/978214008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484102430 msgr2=0x7ff4841028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.486+0000 7ff48c00d700 1 --2- 172.21.15.26:0/978214008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484102430 0x7ff4841028b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff474009990 tx=0x7ff474009ca0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 -- 172.21.15.26:0/978214008 shutdown_connections 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 --2- 172.21.15.26:0/978214008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff484102df0 0x7ff48410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 --2- 172.21.15.26:0/978214008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484108510 0x7ff4841088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 --2- 172.21.15.26:0/978214008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484102430 0x7ff4841028b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 -- 172.21.15.26:0/978214008 >> 172.21.15.26:0/978214008 conn(0x7ff4840793b0 msgr2=0x7ff4840797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 -- 172.21.15.26:0/978214008 shutdown_connections 2024-05-20T21:10:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.487+0000 7ff48c00d700 1 -- 172.21.15.26:0/978214008 wait complete. 2024-05-20T21:10:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 Processor -- start 2024-05-20T21:10:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 -- start start 2024-05-20T21:10:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 0x7ff484074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff484102df0 0x7ff484072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484108510 0x7ff4840733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff48406d410 con 0x7ff484108510 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff489da9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 0x7ff484074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff489da9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 0x7ff484074d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39344/0 (socket says 172.21.15.26:39344) 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff489da9700 1 -- 172.21.15.26:0/909929299 learned_addr learned my addr 172.21.15.26:0/909929299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff48406d290 con 0x7ff484102430 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff48406d590 con 0x7ff484102df0 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff48a5aa700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484108510 0x7ff4840733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.488+0000 7ff4895a8700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff484102df0 0x7ff484072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff489da9700 1 -- 172.21.15.26:0/909929299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff484102df0 msgr2=0x7ff484072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff489da9700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff484102df0 0x7ff484072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff489da9700 1 -- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484108510 msgr2=0x7ff4840733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff489da9700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484108510 0x7ff4840733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff489da9700 1 -- 172.21.15.26:0/909929299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff484073a70 con 0x7ff484102430 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff48a5aa700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484108510 0x7ff4840733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff489da9700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 0x7ff484074d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff480009a20 tx=0x7ff480009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff47affd700 1 -- 172.21.15.26:0/909929299 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004020 con 0x7ff484102430 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48406e9b0 con 0x7ff484102430 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff47affd700 1 -- 172.21.15.26:0/909929299 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff48000c8f0 con 0x7ff484102430 2024-05-20T21:10:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff47affd700 1 -- 172.21.15.26:0/909929299 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800039e0 con 0x7ff484102430 2024-05-20T21:10:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.489+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48406ef10 con 0x7ff484102430 2024-05-20T21:10:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.493+0000 7ff47affd700 1 -- 172.21.15.26:0/909929299 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff480003be0 con 0x7ff484102430 2024-05-20T21:10:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.493+0000 7ff47affd700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4700b2880 0x7ff4700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.494+0000 7ff4895a8700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4700b2880 0x7ff4700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.494+0000 7ff47affd700 1 -- 172.21.15.26:0/909929299 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff4800e1720 con 0x7ff484102430 2024-05-20T21:10:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.494+0000 7ff4968a3b80 1 Processor -- start 2024-05-20T21:10:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.494+0000 7ff4968a3b80 1 -- start start 2024-05-20T21:10:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.494+0000 7ff4895a8700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4700b2880 0x7ff4700b4d40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff484074920 tx=0x7ff474005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 0x55ff3acfc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acf68a0 0x55ff3acf6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad03080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ff3ac54d70 con 0x55ff3acfc320 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ff3ac44910 con 0x55ff3acf7260 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff3ac48950 con 0x55ff3acf68a0 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 0x55ff3acfc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 0x55ff3acfc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35500/0 (socket says 172.21.15.26:35500) 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 -- 172.21.15.26:0/4182264867 learned_addr learned my addr 172.21.15.26:0/4182264867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff48a5aa700 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff4895a8700 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acf68a0 0x55ff3acf6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 -- 172.21.15.26:0/4182264867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acf68a0 msgr2=0x55ff3acf6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acf68a0 0x55ff3acf6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 -- 172.21.15.26:0/4182264867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 msgr2=0x55ff3ad03080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad03080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.495+0000 7ff489da9700 1 -- 172.21.15.26:0/4182264867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3abedfb0 con 0x55ff3acfc320 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.496+0000 7ff489da9700 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 0x55ff3acfc700 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff4800a8c70 tx=0x7ff4800a93e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603abf8660fe99af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.496+0000 7ff46ffff700 1 -- 172.21.15.26:0/4182264867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800ad090 con 0x55ff3acfc320 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.496+0000 7ff46ffff700 1 -- 172.21.15.26:0/4182264867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4800b1b40 con 0x55ff3acfc320 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.496+0000 7ff46ffff700 1 -- 172.21.15.26:0/4182264867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800a7f40 con 0x55ff3acfc320 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 -- 172.21.15.26:0/4182264867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 msgr2=0x55ff3acfc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 0x55ff3acfc700 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff4800a8c70 tx=0x7ff4800a93e0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 -- 172.21.15.26:0/4182264867 shutdown_connections 2024-05-20T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acfc320 0x55ff3acfc700 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad03080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/4182264867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acf68a0 0x55ff3acf6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 -- 172.21.15.26:0/4182264867 >> 172.21.15.26:0/4182264867 conn(0x55ff3ac22240 msgr2=0x55ff3acf13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.497+0000 7ff4968a3b80 1 -- 172.21.15.26:0/4182264867 shutdown_connections 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.498+0000 7ff4968a3b80 1 -- 172.21.15.26:0/4182264867 wait complete. 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.498+0000 7ff4968a3b80 1 -- start start 2024-05-20T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.498+0000 7ff4968a3b80 1 Processor -- start 2024-05-20T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad02d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff489da9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad02d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff489da9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad02d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35506/0 (socket says 172.21.15.26:35506) 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff489da9700 1 -- 172.21.15.26:0/908302 learned_addr learned my addr 172.21.15.26:0/908302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad00600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4895a8700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ff3ab4afa0 con 0x55ff3acf68a0 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ff3abacda0 con 0x55ff3acf7260 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff3ad03730 con 0x55ff3acfc320 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.499+0000 7ff48a5aa700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad00600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4895a8700 1 -- 172.21.15.26:0/908302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 msgr2=0x55ff3ad00600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4895a8700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad00600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4895a8700 1 -- 172.21.15.26:0/908302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 msgr2=0x55ff3ad02d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4895a8700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad02d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4895a8700 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3ab948d0 con 0x55ff3acf7260 2024-05-20T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff489da9700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad02d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4968a3b80 10 client.0 _ll_register_callbacks cb 0x55ff3abdf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800b1b40 con 0x55ff3acf7260 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff4895a8700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad000c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff47400b650 tx=0x7ff474009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1337c3cf1751dc33 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff46d7fa700 10 client.29069 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff46d7fa700 1 -- 172.21.15.26:0/908302 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474025470 con 0x55ff3acf7260 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.500+0000 7ff46d7fa700 1 -- 172.21.15.26:0/908302 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff474025610 con 0x55ff3acf7260 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff46d7fa700 1 -- 172.21.15.26:0/908302 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff474025850 con 0x55ff3acf7260 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 10 client.29069 fetch_fsmap learned FSMap version 33 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55ff3abfe760 con 0x55ff3acf7260 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff46d7fa700 1 -- 172.21.15.26:0/908302 <== mon.2 v2:172.21.15.55:3300/0 4 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7ff474021070 con 0x55ff3acf7260 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 10 client.29069 fetch_fsmap finished waiting for FSMap version 33 2024-05-20T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 10 client.29069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 20 client.29069 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad10060 0x55ff3ad124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 4 client.29069 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff3ac56cc0 con 0x55ff3ad10060 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.501+0000 7ff489da9700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad10060 0x55ff3ad124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.502+0000 7ff46d7fa700 10 client.29069 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.502+0000 7ff489da9700 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad10060 0x55ff3ad124b0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8daa9c8943d3e4 server_cookie=504d944f6952e673 in_seq=0 out_seq=0 2024-05-20T21:10:09.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff46d7fa700 1 -- 172.21.15.26:0/908302 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ff3ac56cc0 con 0x55ff3ad10060 2024-05-20T21:10:09.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff46d7fa700 10 client.29069 handle_command_reply: tid=0 2024-05-20T21:10:09.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 client.29069 shutdown 2024-05-20T21:10:09.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 msgr2=0x55ff3ad000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad000c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff47400b650 tx=0x7ff474009ca0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 shutdown_connections 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad10060 0x55ff3ad124b0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad02d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/908302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad000c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 >> 172.21.15.26:0/908302 conn(0x55ff3ac22240 msgr2=0x55ff3acf0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.504+0000 7ff46d7fa700 0 client.29069 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.505+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 shutdown_connections 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.505+0000 7ff4968a3b80 1 -- 172.21.15.26:0/908302 wait complete. 2024-05-20T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.505+0000 7ff4968a3b80 20 client.29069 trim_cache size 0 max 16384 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.535+0000 7ff4968a3b80 1 Processor -- start 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 -- start start 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf68a0 0x55ff3ad00750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 0x55ff3acf1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841b6cc0 con 0x55ff3acf7260 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841b44c0 con 0x55ff3acf68a0 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841b2310 con 0x55ff3acfc320 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4895a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 0x55ff3acf1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff48a5aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4895a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 0x55ff3acf1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35510/0 (socket says 172.21.15.26:35510) 2024-05-20T21:10:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff4895a8700 1 -- 172.21.15.26:0/230675912 learned_addr learned my addr 172.21.15.26:0/230675912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff48a5aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:46026/0 (socket says 172.21.15.26:46026) 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.536+0000 7ff489da9700 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf68a0 0x55ff3ad00750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff4895a8700 1 -- 172.21.15.26:0/230675912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 msgr2=0x55ff3ad01fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff4895a8700 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad01fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff4895a8700 1 -- 172.21.15.26:0/230675912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf68a0 msgr2=0x55ff3ad00750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff4895a8700 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf68a0 0x55ff3ad00750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff4895a8700 1 -- 172.21.15.26:0/230675912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3acf3380 con 0x55ff3acf7260 2024-05-20T21:10:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff4895a8700 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 0x55ff3acf1270 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff474025a10 tx=0x7ff47402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61e766f7b6e827 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff46f7fe700 1 -- 172.21.15.26:0/230675912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474025e30 con 0x55ff3acf7260 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff46f7fe700 1 -- 172.21.15.26:0/230675912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4800b1b40 con 0x55ff3acf7260 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.537+0000 7ff46f7fe700 1 -- 172.21.15.26:0/230675912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474031430 con 0x55ff3acf7260 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 -- 172.21.15.26:0/230675912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 msgr2=0x55ff3acf1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 0x55ff3acf1270 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff474025a10 tx=0x7ff47402d000 comp rx=0 tx=0).stop 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 -- 172.21.15.26:0/230675912 shutdown_connections 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf68a0 0x55ff3ad00750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/230675912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf7260 0x55ff3acf1270 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 -- 172.21.15.26:0/230675912 >> 172.21.15.26:0/230675912 conn(0x55ff3ac22240 msgr2=0x55ff3ad018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.538+0000 7ff4968a3b80 1 -- 172.21.15.26:0/230675912 shutdown_connections 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.539+0000 7ff4968a3b80 1 -- 172.21.15.26:0/230675912 wait complete. 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.539+0000 7ff4968a3b80 1 -- start start 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.539+0000 7ff4968a3b80 1 Processor -- start 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad17d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad18270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841b2980 con 0x55ff3acf68a0 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841b3df0 con 0x55ff3acf7260 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841b3050 con 0x55ff3acfc320 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4968a3b80 10 client.0 _ll_register_callbacks cb 0x55ff3abdf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4895a8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad17d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff489da9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4895a8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad17d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39362/0 (socket says 172.21.15.26:39362) 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4895a8700 1 -- 172.21.15.26:0/2739676319 learned_addr learned my addr 172.21.15.26:0/2739676319 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff48a5aa700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad18270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff489da9700 1 -- 172.21.15.26:0/2739676319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 msgr2=0x55ff3ad18270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff489da9700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad18270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff489da9700 1 -- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 msgr2=0x55ff3ad17d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff489da9700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad17d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff489da9700 1 -- 172.21.15.26:0/2739676319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3ad06d60 con 0x55ff3acf68a0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.540+0000 7ff4895a8700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad17d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff489da9700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad177f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff480004800 tx=0x7ff4800a5410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c38fa41d0887bae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff46d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff46d7fa700 1 -- 172.21.15.26:0/2739676319 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800a4ba0 con 0x55ff3acf68a0 2024-05-20T21:10:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff46d7fa700 1 -- 172.21.15.26:0/2739676319 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7ff4800a8f40 con 0x55ff3acf68a0 2024-05-20T21:10:09.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800b1b40 con 0x55ff3acf68a0 2024-05-20T21:10:09.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff46d7fa700 1 -- 172.21.15.26:0/2739676319 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff4800a8ab0 con 0x55ff3acf68a0 2024-05-20T21:10:09.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff4968a3b80 10 client.19872 fetch_fsmap learned FSMap version 33 2024-05-20T21:10:09.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ff3ad7aed0 con 0x55ff3acf68a0 2024-05-20T21:10:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.541+0000 7ff46d7fa700 1 -- 172.21.15.26:0/2739676319 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4800b1b40 con 0x55ff3acf68a0 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff46d7fa700 1 -- 172.21.15.26:0/2739676319 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7ff4800a4650 con 0x55ff3acf68a0 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4968a3b80 10 client.19872 fetch_fsmap finished waiting for FSMap version 33 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4968a3b80 10 client.19872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4968a3b80 20 client.19872 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad7b1b0 0x55ff3ad7d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4968a3b80 4 client.19872 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff3ac504a0 con 0x55ff3ad7b1b0 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4895a8700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad7b1b0 0x55ff3ad7d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff46d7fa700 10 client.19872 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.542+0000 7ff4895a8700 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad7b1b0 0x55ff3ad7d5a0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e60dee43a27020 server_cookie=45965ad00df311f in_seq=0 out_seq=0 2024-05-20T21:10:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff46d7fa700 1 -- 172.21.15.26:0/2739676319 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ff3ac504a0 con 0x55ff3ad7b1b0 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff46d7fa700 10 client.19872 handle_command_reply: tid=0 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 client.19872 shutdown 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 msgr2=0x55ff3ad177f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad177f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff480004800 tx=0x7ff4800a5410 comp rx=0 tx=0).stop 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 shutdown_connections 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ff3ad7b1b0 0x55ff3ad7d5a0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff3acf68a0 0x55ff3ad177f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff46d7fa700 0 client.19872 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.543+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff3acfc320 0x55ff3ad18270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.544+0000 7ff4968a3b80 1 --2- 172.21.15.26:0/2739676319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ff3acf7260 0x55ff3ad17d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.544+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 >> 172.21.15.26:0/2739676319 conn(0x55ff3ac22240 msgr2=0x55ff3acf6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.544+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 shutdown_connections 2024-05-20T21:10:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.544+0000 7ff4968a3b80 1 -- 172.21.15.26:0/2739676319 wait complete. 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.544+0000 7ff4968a3b80 20 client.19872 trim_cache size 0 max 16384 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: "86207ae3-de69-49a1-9f79-81898433bc47": { 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "86207ae3-de69-49a1-9f79-81898433bc47", 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:10:09.549 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:10:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4700b2880 msgr2=0x7ff4700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4700b2880 0x7ff4700b4d40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff484074920 tx=0x7ff474005fb0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 msgr2=0x7ff484074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 0x7ff484074d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff480009a20 tx=0x7ff480009d30 comp rx=0 tx=0).stop 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 shutdown_connections 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7ff4700b2880 0x7ff4700b4d40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff484108510 0x7ff4840733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff484102430 0x7ff484074d80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 --2- 172.21.15.26:0/909929299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff484102df0 0x7ff484072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 >> 172.21.15.26:0/909929299 conn(0x7ff4840793b0 msgr2=0x7ff484100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 shutdown_connections 2024-05-20T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:09.545+0000 7ff48c00d700 1 -- 172.21.15.26:0/909929299 wait complete. 2024-05-20T21:10:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:09 smithi144 ceph-mon[35660]: pgmap v810: 97 pgs: 97 active+clean; 213 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 18 MiB/s wr, 407 op/s 2024-05-20T21:10:10.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2762 20.74 MB/sec warmup 15 sec latency 135.705 ms 2024-05-20T21:10:10.592 INFO:tasks.cephfs.filesystem:scrub status for tag:86207ae3-de69-49a1-9f79-81898433bc47 - {'path': '/', 'tag': '86207ae3-de69-49a1-9f79-81898433bc47', 'options': 'recursive,force'} 2024-05-20T21:10:11.107 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2893 19.91 MB/sec warmup 16 sec latency 110.309 ms 2024-05-20T21:10:11.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:11 smithi055 ceph-mon[35213]: pgmap v811: 97 pgs: 97 active+clean; 215 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 13 MiB/s wr, 391 op/s 2024-05-20T21:10:11.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:11 smithi026 ceph-mon[31990]: pgmap v811: 97 pgs: 97 active+clean; 215 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 13 MiB/s wr, 391 op/s 2024-05-20T21:10:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:11 smithi144 ceph-mon[35660]: pgmap v811: 97 pgs: 97 active+clean; 215 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 13 MiB/s wr, 391 op/s 2024-05-20T21:10:12.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3031 19.23 MB/sec warmup 17 sec latency 114.512 ms 2024-05-20T21:10:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:12 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a[31986]: 2024-05-20T21:10:12.192+0000 7f0a2c4ba700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T21:10:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:12 smithi026 ceph-mon[31990]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T21:10:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:12 smithi026 ceph-mon[31990]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:active 2024-05-20T21:10:12.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:12 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:12.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:12 smithi144 ceph-mon[35660]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T21:10:12.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:12 smithi144 ceph-mon[35660]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:active 2024-05-20T21:10:12.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:12 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:12.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:12 smithi055 ceph-mon[35213]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T21:10:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:12 smithi055 ceph-mon[35213]: mds.? [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] up:active 2024-05-20T21:10:12.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:12 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:10:13.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3226 19.33 MB/sec warmup 18 sec latency 99.850 ms 2024-05-20T21:10:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:13 smithi144 ceph-mon[35660]: pgmap v812: 97 pgs: 97 active+clean; 221 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 8.6 MiB/s wr, 400 op/s 2024-05-20T21:10:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:13 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:10:13.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:13 smithi055 ceph-mon[35213]: pgmap v812: 97 pgs: 97 active+clean; 221 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 8.6 MiB/s wr, 400 op/s 2024-05-20T21:10:13.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:13 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:10:13.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:13 smithi026 ceph-mon[31990]: pgmap v812: 97 pgs: 97 active+clean; 221 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 8.6 MiB/s wr, 400 op/s 2024-05-20T21:10:13.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:13 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:10:14.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3425 19.03 MB/sec warmup 19 sec latency 91.986 ms 2024-05-20T21:10:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 -- 172.21.15.26:0/3179964695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660104690 msgr2=0x7f966010af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 --2- 172.21.15.26:0/3179964695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660104690 0x7f966010af20 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9658009ca0 tx=0x7f965802a8c0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 -- 172.21.15.26:0/3179964695 shutdown_connections 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 --2- 172.21.15.26:0/3179964695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660104690 0x7f966010af20 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 --2- 172.21.15.26:0/3179964695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9660103cd0 0x7f9660104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 --2- 172.21.15.26:0/3179964695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f9660102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 -- 172.21.15.26:0/3179964695 >> 172.21.15.26:0/3179964695 conn(0x7f96600fe070 msgr2=0x7f9660100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 -- 172.21.15.26:0/3179964695 shutdown_connections 2024-05-20T21:10:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.308+0000 7f9667324640 1 -- 172.21.15.26:0/3179964695 wait complete. 2024-05-20T21:10:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.309+0000 7f9667324640 1 Processor -- start 2024-05-20T21:10:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.309+0000 7f9667324640 1 -- start start 2024-05-20T21:10:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f9667324640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f966019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f9667324640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 0x7f966019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f9667324640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9660104690 0x7f96601a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f9667324640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9660075290 con 0x7f9660103cd0 2024-05-20T21:10:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 0x7f966019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f9667324640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9660075110 con 0x7f9660102ae0 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f9667324640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9660075410 con 0x7f9660104690 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96650c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f966019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96658c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9660104690 0x7f96601a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 0x7f966019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53772/0 (socket says 172.21.15.26:53772) 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96650c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f966019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39494/0 (socket says 172.21.15.26:39494) 2024-05-20T21:10:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 -- 172.21.15.26:0/1910107226 learned_addr learned my addr 172.21.15.26:0/1910107226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 -- 172.21.15.26:0/1910107226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9660104690 msgr2=0x7f96601a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9660104690 0x7f96601a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 -- 172.21.15.26:0/1910107226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 msgr2=0x7f966019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f966019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96648bf640 1 -- 172.21.15.26:0/1910107226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96601a2990 con 0x7f9660103cd0 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.310+0000 7f96650c0640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f966019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.311+0000 7f96648bf640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 0x7f966019b2b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f965000e9f0 tx=0x7f965000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.311+0000 7f96567fc640 1 -- 172.21.15.26:0/1910107226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000cd80 con 0x7f9660103cd0 2024-05-20T21:10:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.311+0000 7f96567fc640 1 -- 172.21.15.26:0/1910107226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96500054b0 con 0x7f9660103cd0 2024-05-20T21:10:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.311+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966010df60 con 0x7f9660103cd0 2024-05-20T21:10:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.311+0000 7f96567fc640 1 -- 172.21.15.26:0/1910107226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650010430 con 0x7f9660103cd0 2024-05-20T21:10:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.311+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966010e460 con 0x7f9660103cd0 2024-05-20T21:10:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.314+0000 7f966a044740 1 Processor -- start 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 -- start start 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96567fc640 1 -- 172.21.15.26:0/1910107226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f96500046c0 con 0x7f9660103cd0 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e74508ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e7450dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e745106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562e74479c30 con 0x562e745090b0 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562e744697d0 con 0x562e7450e220 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f966a044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562e7446d810 con 0x562e745086c0 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96658c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e74508ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96650c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e7450dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96658c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e74508ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49152/0 (socket says 172.21.15.26:49152) 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96658c1640 1 -- 172.21.15.26:0/2283092442 learned_addr learned my addr 172.21.15.26:0/2283092442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96648bf640 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e745106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96658c1640 1 -- 172.21.15.26:0/2283092442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 msgr2=0x562e745106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96658c1640 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e745106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.315+0000 7f96658c1640 1 -- 172.21.15.26:0/2283092442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 msgr2=0x562e7450dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f96658c1640 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e7450dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f96658c1640 1 -- 172.21.15.26:0/2283092442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7417e210 con 0x562e745086c0 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f96650c0640 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e7450dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f96658c1640 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e74508ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f965802ade0 tx=0x7f9658005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710dc28bb8895b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f964b7fe640 1 -- 172.21.15.26:0/2283092442 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96580316d0 con 0x562e745086c0 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f964b7fe640 1 -- 172.21.15.26:0/2283092442 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9658036030 con 0x562e745086c0 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f964b7fe640 1 -- 172.21.15.26:0/2283092442 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658004460 con 0x562e745086c0 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.316+0000 7f96567fc640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f964c0b28a0 0x7f964c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.317+0000 7f96650c0640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f964c0b28a0 0x7f964c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.317+0000 7f96567fc640 1 -- 172.21.15.26:0/1910107226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f96500e25b0 con 0x7f9660103cd0 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.317+0000 7f96650c0640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f964c0b28a0 0x7f964c0b4d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x562e745094b0 tx=0x7f965c016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.317+0000 7f966a044740 1 -- 172.21.15.26:0/2283092442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 msgr2=0x562e74508ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.317+0000 7f966a044740 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e74508ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f965802ade0 tx=0x7f9658005ef0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 -- 172.21.15.26:0/2283092442 shutdown_connections 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e745106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e7450dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 --2- 172.21.15.26:0/2283092442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e74508ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 -- 172.21.15.26:0/2283092442 >> 172.21.15.26:0/2283092442 conn(0x562e74503d30 msgr2=0x562e74504130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 -- 172.21.15.26:0/2283092442 shutdown_connections 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 -- 172.21.15.26:0/2283092442 wait complete. 2024-05-20T21:10:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 -- start start 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.318+0000 7f966a044740 1 Processor -- start 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e745126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e74512c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74519c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f96648bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e74512c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f96648bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e74512c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53792/0 (socket says 172.21.15.26:53792) 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f96650c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e745126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f96648bf640 1 -- 172.21.15.26:0/1465447127 learned_addr learned my addr 172.21.15.26:0/1465447127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562e7414f790 con 0x562e745090b0 2024-05-20T21:10:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562e74425890 con 0x562e7450e220 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562e74510ff0 con 0x562e745086c0 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f966a044740 10 client.0 _ll_register_callbacks cb 0x562e7417da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.319+0000 7f96658c1640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74519c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96648bf640 1 -- 172.21.15.26:0/1465447127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 msgr2=0x562e745126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96648bf640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e745126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96648bf640 1 -- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 msgr2=0x562e74519c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96648bf640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74519c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96648bf640 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e7424a620 con 0x562e745090b0 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96650c0640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e745126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96658c1640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74519c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f96648bf640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e74512c30 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f965000e9c0 tx=0x7f9650005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340aad6b01d49a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f9648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:10:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f9648ff9640 1 -- 172.21.15.26:0/1465447127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500a94c0 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9658036030 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f9648ff9640 1 -- 172.21.15.26:0/1465447127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f96500b2750 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f9648ff9640 1 -- 172.21.15.26:0/1465447127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f96500a5300 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f966a044740 10 client.19884 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.320+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e7451cca0 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f9648ff9640 1 -- 172.21.15.26:0/1465447127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96500c4030 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f9648ff9640 1 -- 172.21.15.26:0/1465447127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f96500a8dd0 con 0x562e745090b0 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f966a044740 10 client.19884 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f966a044740 10 client.19884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f966a044740 20 client.19884 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f966a044740 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e74520820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f966a044740 4 client.19884 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e7447bb80 con 0x562e7451e430 2024-05-20T21:10:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.321+0000 7f96650c0640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e74520820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.322+0000 7f96650c0640 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e74520820 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc91f502b2942a7c server_cookie=c2678bfc2e93c2be in_seq=0 out_seq=0 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.322+0000 7f9648ff9640 10 client.19884 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f9648ff9640 1 -- 172.21.15.26:0/1465447127 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562e7447bb80 con 0x562e7451e430 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f9648ff9640 10 client.19884 handle_command_reply: tid=0 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 client.19884 shutdown 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 msgr2=0x562e74512c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e74512c30 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f965000e9c0 tx=0x7f9650005660 comp rx=0 tx=0).stop 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 shutdown_connections 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e74520820 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74519c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f9648ff9640 0 client.19884 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745090b0 0x562e74512c30 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 --2- 172.21.15.26:0/1465447127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745086c0 0x562e745126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 >> 172.21.15.26:0/1465447127 conn(0x562e74503d30 msgr2=0x562e74502fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 shutdown_connections 2024-05-20T21:10:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 1 -- 172.21.15.26:0/1465447127 wait complete. 2024-05-20T21:10:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.323+0000 7f966a044740 20 client.19884 trim_cache size 0 max 16384 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 Processor -- start 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 -- start start 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e745119c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e745090b0 0x562e74519bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 0x562e74511f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f96601aa640 con 0x562e745086c0 2024-05-20T21:10:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96601a7e80 con 0x562e745090b0 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f966a044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96601a5cf0 con 0x562e7450e220 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f96658c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 0x562e74511f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f96650c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e745119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f96658c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 0x562e74511f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49176/0 (socket says 172.21.15.26:49176) 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f96658c1640 1 -- 172.21.15.26:0/477903047 learned_addr learned my addr 172.21.15.26:0/477903047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.345+0000 7f96648bf640 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e745090b0 0x562e74519bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96658c1640 1 -- 172.21.15.26:0/477903047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e745090b0 msgr2=0x562e74519bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96658c1640 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e745090b0 0x562e74519bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96658c1640 1 -- 172.21.15.26:0/477903047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 msgr2=0x562e745119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96658c1640 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e745119c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96658c1640 1 -- 172.21.15.26:0/477903047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e74509ec0 con 0x562e7450e220 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96650c0640 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e745119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f96658c1640 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 0x562e74511f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f965802ade0 tx=0x7f9658031800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae805b8e202b319 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f964affd640 1 -- 172.21.15.26:0/477903047 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965804a040 con 0x562e7450e220 2024-05-20T21:10:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f964affd640 1 -- 172.21.15.26:0/477903047 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9658003970 con 0x562e7450e220 2024-05-20T21:10:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.346+0000 7f964affd640 1 -- 172.21.15.26:0/477903047 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658004510 con 0x562e7450e220 2024-05-20T21:10:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 -- 172.21.15.26:0/477903047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 msgr2=0x562e74511f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 0x562e74511f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f965802ade0 tx=0x7f9658031800 comp rx=0 tx=0).stop 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 -- 172.21.15.26:0/477903047 shutdown_connections 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e7450e220 0x562e74511f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e745090b0 0x562e74519bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 --2- 172.21.15.26:0/477903047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e745119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 -- 172.21.15.26:0/477903047 >> 172.21.15.26:0/477903047 conn(0x562e74503d30 msgr2=0x562e744fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 -- 172.21.15.26:0/477903047 shutdown_connections 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 -- 172.21.15.26:0/477903047 wait complete. 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.347+0000 7f966a044740 1 -- start start 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 Processor -- start 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e74519b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745090b0 0x562e74516610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74516c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f96601a6340 con 0x562e745086c0 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96601a77b0 con 0x562e7450e220 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96601a6a10 con 0x562e745090b0 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f96650c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e74519b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f96650c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e74519b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53806/0 (socket says 172.21.15.26:53806) 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f96648bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745090b0 0x562e74516610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f96650c0640 1 -- 172.21.15.26:0/2685148892 learned_addr learned my addr 172.21.15.26:0/2685148892 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f966a044740 10 client.0 _ll_register_callbacks cb 0x7f9650004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.348+0000 7f96658c1640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74516c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96648bf640 1 -- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 msgr2=0x562e74516c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96648bf640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74516c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96648bf640 1 -- 172.21.15.26:0/2685148892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 msgr2=0x562e74519b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96648bf640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e74519b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96648bf640 1 -- 172.21.15.26:0/2685148892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e74587000 con 0x562e745090b0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96650c0640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e74519b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f96648bf640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745090b0 0x562e74516610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f96500a9870 tx=0x7f9650005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d804342297fdf3b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f9648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f9648ff9640 1 -- 172.21.15.26:0/2685148892 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500c2040 con 0x562e745090b0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f9648ff9640 1 -- 172.21.15.26:0/2685148892 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f96500e2110 con 0x562e745090b0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9658003970 con 0x562e745090b0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f9648ff9640 1 -- 172.21.15.26:0/2685148892 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f96500a92e0 con 0x562e745090b0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f966a044740 10 client.28789 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.349+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e745883b0 con 0x562e745090b0 2024-05-20T21:10:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f9648ff9640 1 -- 172.21.15.26:0/2685148892 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9658003970 con 0x562e745090b0 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f9648ff9640 1 -- 172.21.15.26:0/2685148892 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f96500a8840 con 0x562e745090b0 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f966a044740 10 client.28789 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f966a044740 10 client.28789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f966a044740 20 client.28789 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f966a044740 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e7458ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f966a044740 4 client.28789 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e74475360 con 0x562e7451e430 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f96650c0640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e7458ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f9648ff9640 10 client.28789 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.350+0000 7f96650c0640 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e7458ad90 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce10cdd22a315890 server_cookie=e371f862910fef69 in_seq=0 out_seq=0 2024-05-20T21:10:14.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:10:14.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:10:14.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:14 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:10:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f9648ff9640 1 -- 172.21.15.26:0/2685148892 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562e74475360 con 0x562e7451e430 2024-05-20T21:10:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f9648ff9640 10 client.28789 handle_command_reply: tid=0 2024-05-20T21:10:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 client.28789 shutdown 2024-05-20T21:10:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745090b0 msgr2=0x562e74516610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745090b0 0x562e74516610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f96500a9870 tx=0x7f9650005e50 comp rx=0 tx=0).stop 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 shutdown_connections 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x562e7451e430 0x562e7458ad90 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562e7450e220 0x562e74516c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e745090b0 0x562e74516610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f9648ff9640 0 client.28789 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 --2- 172.21.15.26:0/2685148892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e745086c0 0x562e74519b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.634+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 >> 172.21.15.26:0/2685148892 conn(0x562e74503d30 msgr2=0x562e745119c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 shutdown_connections 2024-05-20T21:10:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f966a044740 1 -- 172.21.15.26:0/2685148892 wait complete. 2024-05-20T21:10:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f966a044740 20 client.28789 trim_cache size 0 max 16384 2024-05-20T21:10:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:10:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:10:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f964c0b28a0 msgr2=0x7f964c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f964c0b28a0 0x7f964c0b4d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x562e745094b0 tx=0x7f965c016040 comp rx=0 tx=0).stop 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 msgr2=0x7f966019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 0x7f966019b2b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f965000e9f0 tx=0x7f965000edf0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 shutdown_connections 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f964c0b28a0 0x7f964c0b4d60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9660104690 0x7f96601a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9660103cd0 0x7f966019b2b0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 --2- 172.21.15.26:0/1910107226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9660102ae0 0x7f966019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.635+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 >> 172.21.15.26:0/1910107226 conn(0x7f96600fe070 msgr2=0x7f96600ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.636+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 shutdown_connections 2024-05-20T21:10:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:14.636+0000 7f9667324640 1 -- 172.21.15.26:0/1910107226 wait complete. 2024-05-20T21:10:14.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:10:14.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:10:14.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:14 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:10:14.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:10:14.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:10:14.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:14 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:10:15.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3637 18.67 MB/sec warmup 20 sec latency 152.688 ms 2024-05-20T21:10:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:15 smithi144 ceph-mon[35660]: pgmap v813: 97 pgs: 97 active+clean; 227 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-05-20T21:10:15.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:15 smithi055 ceph-mon[35213]: pgmap v813: 97 pgs: 97 active+clean; 227 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-05-20T21:10:15.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:15 smithi026 ceph-mon[31990]: pgmap v813: 97 pgs: 97 active+clean; 227 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-05-20T21:10:16.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3762 18.00 MB/sec warmup 21 sec latency 94.620 ms 2024-05-20T21:10:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:16 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:10:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:16 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:10:16.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:16 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:10:16.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:16 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:10:16.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:16 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:10:16.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:16 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:10:17.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3878 17.31 MB/sec warmup 22 sec latency 104.690 ms 2024-05-20T21:10:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:17 smithi144 ceph-mon[35660]: pgmap v814: 97 pgs: 97 active+clean; 231 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-05-20T21:10:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:17 smithi144 ceph-mon[35660]: scrub summary: idle 2024-05-20T21:10:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:17 smithi055 ceph-mon[35213]: pgmap v814: 97 pgs: 97 active+clean; 231 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-05-20T21:10:17.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:17 smithi055 ceph-mon[35213]: scrub summary: idle 2024-05-20T21:10:17.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:17 smithi026 ceph-mon[31990]: pgmap v814: 97 pgs: 97 active+clean; 231 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-05-20T21:10:17.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:17 smithi026 ceph-mon[31990]: scrub summary: idle 2024-05-20T21:10:18.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3996 16.83 MB/sec warmup 23 sec latency 88.842 ms 2024-05-20T21:10:19.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4178 16.86 MB/sec warmup 24 sec latency 90.828 ms 2024-05-20T21:10:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:19 smithi144 ceph-mon[35660]: pgmap v815: 97 pgs: 97 active+clean; 232 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-05-20T21:10:19.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:19 smithi055 ceph-mon[35213]: pgmap v815: 97 pgs: 97 active+clean; 232 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-05-20T21:10:19.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:19 smithi026 ceph-mon[31990]: pgmap v815: 97 pgs: 97 active+clean; 232 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-05-20T21:10:20.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4388 16.77 MB/sec warmup 25 sec latency 91.262 ms 2024-05-20T21:10:21.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4571 16.51 MB/sec warmup 26 sec latency 101.680 ms 2024-05-20T21:10:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:21 smithi144 ceph-mon[35660]: pgmap v816: 97 pgs: 97 active+clean; 231 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-20T21:10:21.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:21 smithi055 ceph-mon[35213]: pgmap v816: 97 pgs: 97 active+clean; 231 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-20T21:10:21.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:21 smithi026 ceph-mon[31990]: pgmap v816: 97 pgs: 97 active+clean; 231 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-20T21:10:22.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4689 15.95 MB/sec warmup 27 sec latency 82.239 ms 2024-05-20T21:10:23.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4841 15.59 MB/sec warmup 28 sec latency 84.915 ms 2024-05-20T21:10:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:23 smithi144 ceph-mon[35660]: pgmap v817: 97 pgs: 97 active+clean; 229 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 4.7 MiB/s wr, 263 op/s 2024-05-20T21:10:23.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:23 smithi055 ceph-mon[35213]: pgmap v817: 97 pgs: 97 active+clean; 229 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 4.7 MiB/s wr, 263 op/s 2024-05-20T21:10:23.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:23 smithi026 ceph-mon[31990]: pgmap v817: 97 pgs: 97 active+clean; 229 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 4.7 MiB/s wr, 263 op/s 2024-05-20T21:10:24.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4981 15.21 MB/sec warmup 29 sec latency 86.590 ms 2024-05-20T21:10:25.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5133 14.92 MB/sec warmup 30 sec latency 139.841 ms 2024-05-20T21:10:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:25 smithi144 ceph-mon[35660]: pgmap v818: 97 pgs: 97 active+clean; 234 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-20T21:10:25.702 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:25 smithi055 ceph-mon[35213]: pgmap v818: 97 pgs: 97 active+clean; 234 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-20T21:10:25.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:25 smithi026 ceph-mon[31990]: pgmap v818: 97 pgs: 97 active+clean; 234 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-20T21:10:26.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5316 14.71 MB/sec warmup 31 sec latency 129.362 ms 2024-05-20T21:10:27.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5493 14.56 MB/sec warmup 32 sec latency 108.001 ms 2024-05-20T21:10:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:27 smithi144 ceph-mon[35660]: pgmap v819: 97 pgs: 97 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.3 MiB/s wr, 249 op/s 2024-05-20T21:10:27.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:27 smithi055 ceph-mon[35213]: pgmap v819: 97 pgs: 97 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.3 MiB/s wr, 249 op/s 2024-05-20T21:10:27.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:27 smithi026 ceph-mon[31990]: pgmap v819: 97 pgs: 97 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.3 MiB/s wr, 249 op/s 2024-05-20T21:10:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 -- 172.21.15.26:0/2230911367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 msgr2=0x7f3d0c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 --2- 172.21.15.26:0/2230911367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c109df0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009ca0 tx=0x7f3d0402a8c0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 -- 172.21.15.26:0/2230911367 shutdown_connections 2024-05-20T21:10:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 --2- 172.21.15.26:0/2230911367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c109df0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 --2- 172.21.15.26:0/2230911367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d0c102ba0 0x7f3d0c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 --2- 172.21.15.26:0/2230911367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d0c1019b0 0x7f3d0c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 -- 172.21.15.26:0/2230911367 >> 172.21.15.26:0/2230911367 conn(0x7f3d0c0fcf20 msgr2=0x7f3d0c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 -- 172.21.15.26:0/2230911367 shutdown_connections 2024-05-20T21:10:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.758+0000 7f3d1410a640 1 -- 172.21.15.26:0/2230911367 wait complete. 2024-05-20T21:10:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 Processor -- start 2024-05-20T21:10:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 -- start start 2024-05-20T21:10:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d0c1019b0 0x7f3d0c06ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d0c102ba0 0x7f3d0c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d3e0 con 0x7f3d0c103560 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d260 con 0x7f3d0c102ba0 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.759+0000 7f3d1410a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d560 con 0x7f3d0c1019b0 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c1a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57490/0 (socket says 172.21.15.26:57490) 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 -- 172.21.15.26:0/1587820152 learned_addr learned my addr 172.21.15.26:0/1587820152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d0c1019b0 0x7f3d0c06ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d116a5640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d0c102ba0 0x7f3d0c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 -- 172.21.15.26:0/1587820152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d0c1019b0 msgr2=0x7f3d0c06ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d0c1019b0 0x7f3d0c06ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 -- 172.21.15.26:0/1587820152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d0c102ba0 msgr2=0x7f3d0c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d0c102ba0 0x7f3d0c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.760+0000 7f3d126a7640 1 -- 172.21.15.26:0/1587820152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c1a29e0 con 0x7f3d0c103560 2024-05-20T21:10:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.761+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c1a21f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3d04031040 tx=0x7f3d04003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.761+0000 7f3d02ffd640 1 -- 172.21.15.26:0/1587820152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04003d40 con 0x7f3d0c103560 2024-05-20T21:10:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.761+0000 7f3d02ffd640 1 -- 172.21.15.26:0/1587820152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d040044d0 con 0x7f3d0c103560 2024-05-20T21:10:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.761+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c1a2c10 con 0x7f3d0c103560 2024-05-20T21:10:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.761+0000 7f3d02ffd640 1 -- 172.21.15.26:0/1587820152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04030c30 con 0x7f3d0c103560 2024-05-20T21:10:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.761+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1a31b0 con 0x7f3d0c103560 2024-05-20T21:10:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.766+0000 7f3d16e24740 1 Processor -- start 2024-05-20T21:10:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.766+0000 7f3d02ffd640 1 -- 172.21.15.26:0/1587820152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3d04004040 con 0x7f3d0c103560 2024-05-20T21:10:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.766+0000 7f3d16e24740 1 -- start start 2024-05-20T21:10:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x5655419262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554192b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56554192ba30 0x56554192dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d16e24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56554188ec30 con 0x565541925ed0 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d16e24740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56554187e7d0 con 0x56554192ba30 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d16e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565541882810 con 0x5655419268c0 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x5655419262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x5655419262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57500/0 (socket says 172.21.15.26:57500) 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 -- 172.21.15.26:0/3256273104 learned_addr learned my addr 172.21.15.26:0/3256273104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56554192ba30 0x56554192dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554192b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 -- 172.21.15.26:0/3256273104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 msgr2=0x56554192b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554192b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 -- 172.21.15.26:0/3256273104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56554192ba30 msgr2=0x56554192dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56554192ba30 0x56554192dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d116a5640 1 -- 172.21.15.26:0/3256273104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565541593210 con 0x565541925ed0 2024-05-20T21:10:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.767+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554192b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:10:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.768+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x5655419262f0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009560 tx=0x7f3cfc009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4813749eca7f1cc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.768+0000 7f3cf7fff640 1 -- 172.21.15.26:0/3256273104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004050 con 0x565541925ed0 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.768+0000 7f3cf7fff640 1 -- 172.21.15.26:0/3256273104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cfc0035f0 con 0x565541925ed0 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.768+0000 7f3d02ffd640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3cf80b2940 0x7f3cf80b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.768+0000 7f3cf7fff640 1 -- 172.21.15.26:0/3256273104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00f7e0 con 0x565541925ed0 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.768+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3cf80b2940 0x7f3cf80b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d02ffd640 1 -- 172.21.15.26:0/1587820152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3d04103540 con 0x7f3d0c103560 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 -- 172.21.15.26:0/3256273104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 msgr2=0x5655419262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3cf80b2940 0x7f3cf80b4e00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x565541926cc0 tx=0x7f3d08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x5655419262f0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009560 tx=0x7f3cfc009850 comp rx=0 tx=0).stop 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 -- 172.21.15.26:0/3256273104 shutdown_connections 2024-05-20T21:10:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56554192ba30 0x56554192dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554192b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3256273104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x5655419262f0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 -- 172.21.15.26:0/3256273104 >> 172.21.15.26:0/3256273104 conn(0x565541918d30 msgr2=0x565541919130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.769+0000 7f3d16e24740 1 -- 172.21.15.26:0/3256273104 shutdown_connections 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 -- 172.21.15.26:0/3256273104 wait complete. 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 -- start start 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 Processor -- start 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541930050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x565541930590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56554192ba30 0x5655419375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565541564790 con 0x565541925ed0 2024-05-20T21:10:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56554183a890 con 0x5655419268c0 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.770+0000 7f3d16e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56554192e9e0 con 0x56554192ba30 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d16e24740 10 client.0 _ll_register_callbacks cb 0x565541592a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d11ea6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541930050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x565541930590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d11ea6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541930050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57506/0 (socket says 172.21.15.26:57506) 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d11ea6640 1 -- 172.21.15.26:0/1794602859 learned_addr learned my addr 172.21.15.26:0/1794602859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x565541930590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:39036/0 (socket says 172.21.15.26:39036) 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56554192ba30 0x5655419375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 -- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 msgr2=0x565541930590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x565541930590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 -- 172.21.15.26:0/1794602859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 msgr2=0x565541930050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541930050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 -- 172.21.15.26:0/1794602859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554165f620 con 0x56554192ba30 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56554192ba30 0x5655419375d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d04100fe0 tx=0x7f3d04004330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=903749d67ac67afa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.771+0000 7f3cf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1794602859 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040cb200 con 0x56554192ba30 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cfc0035f0 con 0x56554192ba30 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1794602859 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3d04009890 con 0x56554192ba30 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1794602859 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3d040cab00 con 0x56554192ba30 2024-05-20T21:10:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3d16e24740 10 client.28798 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56554193a520 con 0x56554192ba30 2024-05-20T21:10:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1794602859 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cfc0035f0 con 0x56554192ba30 2024-05-20T21:10:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1794602859 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3d040c6d90 con 0x56554192ba30 2024-05-20T21:10:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3d16e24740 10 client.28798 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.772+0000 7f3d16e24740 10 client.28798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3d16e24740 20 client.28798 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554193bb30 0x56554193dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3d16e24740 4 client.28798 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565541890b80 con 0x56554193bb30 2024-05-20T21:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554193bb30 0x56554193dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3cf57fa640 10 client.28798 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.773+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554193bb30 0x56554193dfb0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721d646286df2945 server_cookie=1d87607c4f81f96e in_seq=0 out_seq=0 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.777+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1794602859 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565541890b80 con 0x56554193bb30 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.777+0000 7f3cf57fa640 10 client.28798 handle_command_reply: tid=0 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.777+0000 7f3d16e24740 1 client.28798 shutdown 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.777+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56554192ba30 msgr2=0x5655419375d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.777+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56554192ba30 0x5655419375d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d04100fe0 tx=0x7f3d04004330 comp rx=0 tx=0).stop 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.777+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 shutdown_connections 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554193bb30 0x56554193dfb0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3cf57fa640 0 client.28798 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56554192ba30 0x5655419375d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x565541930590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1794602859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541930050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 >> 172.21.15.26:0/1794602859 conn(0x565541918d30 msgr2=0x565541917fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 shutdown_connections 2024-05-20T21:10:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 1 -- 172.21.15.26:0/1794602859 wait complete. 2024-05-20T21:10:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.778+0000 7f3d16e24740 20 client.28798 trim_cache size 0 max 16384 2024-05-20T21:10:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.804+0000 7f3d16e24740 1 Processor -- start 2024-05-20T21:10:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.804+0000 7f3d16e24740 1 -- start start 2024-05-20T21:10:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565541918d30 0x565541913e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541937590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554191bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d16e24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1aa5c0 con 0x565541925ed0 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d16e24740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a7e00 con 0x565541918d30 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d16e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a5c50 con 0x5655419268c0 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541937590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541937590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57510/0 (socket says 172.21.15.26:57510) 2024-05-20T21:10:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d116a5640 1 -- 172.21.15.26:0/3535843665 learned_addr learned my addr 172.21.15.26:0/3535843665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565541918d30 0x565541913e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d126a7640 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554191bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d116a5640 1 -- 172.21.15.26:0/3535843665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 msgr2=0x56554191bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.805+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554191bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3d116a5640 1 -- 172.21.15.26:0/3535843665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565541918d30 msgr2=0x565541913e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565541918d30 0x565541913e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3d116a5640 1 -- 172.21.15.26:0/3535843665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655419275f0 con 0x565541925ed0 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3d116a5640 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541937590 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009560 tx=0x7f3cfc005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af88a0dda7f6c24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3cf77fe640 1 -- 172.21.15.26:0/3535843665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc005110 con 0x565541925ed0 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3cf77fe640 1 -- 172.21.15.26:0/3535843665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cfc0052b0 con 0x565541925ed0 2024-05-20T21:10:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.806+0000 7f3cf77fe640 1 -- 172.21.15.26:0/3535843665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00f7e0 con 0x565541925ed0 2024-05-20T21:10:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.807+0000 7f3d16e24740 1 -- 172.21.15.26:0/3535843665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 msgr2=0x565541937590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.807+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541937590 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009560 tx=0x7f3cfc005610 comp rx=0 tx=0).stop 2024-05-20T21:10:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 -- 172.21.15.26:0/3535843665 shutdown_connections 2024-05-20T21:10:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5655419268c0 0x56554191bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541925ed0 0x565541937590 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 --2- 172.21.15.26:0/3535843665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565541918d30 0x565541913e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 -- 172.21.15.26:0/3535843665 >> 172.21.15.26:0/3535843665 conn(0x56554192b500 msgr2=0x565541919110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 -- 172.21.15.26:0/3535843665 shutdown_connections 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 -- 172.21.15.26:0/3535843665 wait complete. 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 -- start start 2024-05-20T21:10:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.808+0000 7f3d16e24740 1 Processor -- start 2024-05-20T21:10:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541918d30 0x565541934b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 0x5655419350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x5655419360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a62c0 con 0x565541918d30 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a7730 con 0x5655419268c0 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a6990 con 0x565541925ed0 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 0x5655419350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d11ea6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541918d30 0x565541934b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d16e24740 10 client.0 _ll_register_callbacks cb 0x7f3d04003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d116a5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 0x5655419350d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:45838/0 (socket says 172.21.15.26:45838) 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d116a5640 1 -- 172.21.15.26:0/1409454217 learned_addr learned my addr 172.21.15.26:0/1409454217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.809+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x5655419360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d116a5640 1 -- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 msgr2=0x5655419360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d116a5640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x5655419360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d116a5640 1 -- 172.21.15.26:0/1409454217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541918d30 msgr2=0x565541934b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d116a5640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541918d30 0x565541934b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d116a5640 1 -- 172.21.15.26:0/1409454217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655419366d0 con 0x565541925ed0 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d126a7640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x5655419360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541918d30 0x565541934b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d116a5640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 0x5655419350d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0056a0 tx=0x7f3cfc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2072c3209b06ad7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3cf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1409454217 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc005110 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1409454217 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3cfc00f790 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.810+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cfc0052b0 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1409454217 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc03e050 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 10 client.28804 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5655419a6750 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1409454217 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cfc0052b0 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1409454217 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3cfc01e070 con 0x565541925ed0 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 10 client.28804 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 10 client.28804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 20 client.28804 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554192ba30 0x5655419a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 4 client.28804 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:10:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56554188a360 con 0x56554192ba30 2024-05-20T21:10:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.811+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554192ba30 0x5655419a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.812+0000 7f3cf57fa640 10 client.28804 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.812+0000 7f3d11ea6640 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554192ba30 0x5655419a8e40 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a6523d202cb741 server_cookie=60bde49d231397 in_seq=0 out_seq=0 2024-05-20T21:10:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3cf57fa640 1 -- 172.21.15.26:0/1409454217 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56554188a360 con 0x56554192ba30 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3cf57fa640 10 client.28804 handle_command_reply: tid=0 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 client.28804 shutdown 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 msgr2=0x5655419350d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 0x5655419350d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0056a0 tx=0x7f3cfc003b90 comp rx=0 tx=0).stop 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 shutdown_connections 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56554192ba30 0x5655419a8e40 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655419268c0 0x5655419360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3cf57fa640 0 client.28804 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565541925ed0 0x5655419350d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 --2- 172.21.15.26:0/1409454217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565541918d30 0x565541934b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 >> 172.21.15.26:0/1409454217 conn(0x56554192b500 msgr2=0x565541913e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 shutdown_connections 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 1 -- 172.21.15.26:0/1409454217 wait complete. 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.928+0000 7f3d16e24740 20 client.28804 trim_cache size 0 max 16384 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:10:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:10:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:10:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3cf80b2940 msgr2=0x7f3cf80b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3cf80b2940 0x7f3cf80b4e00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x565541926cc0 tx=0x7f3d08005f90 comp rx=0 tx=0).stop 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 msgr2=0x7f3d0c1a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c1a21f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3d04031040 tx=0x7f3d04003ba0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 shutdown_connections 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3cf80b2940 0x7f3cf80b4e00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d0c103560 0x7f3d0c1a21f0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d0c102ba0 0x7f3d0c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 --2- 172.21.15.26:0/1587820152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d0c1019b0 0x7f3d0c06ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 >> 172.21.15.26:0/1587820152 conn(0x7f3d0c0fcf20 msgr2=0x7f3d0c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.929+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 shutdown_connections 2024-05-20T21:10:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:27.930+0000 7f3d1410a640 1 -- 172.21.15.26:0/1587820152 wait complete. 2024-05-20T21:10:28.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5614 14.29 MB/sec warmup 33 sec latency 154.662 ms 2024-05-20T21:10:29.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5756 14.22 MB/sec warmup 34 sec latency 107.326 ms 2024-05-20T21:10:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:29 smithi144 ceph-mon[35660]: pgmap v820: 97 pgs: 97 active+clean; 240 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.4 MiB/s wr, 295 op/s 2024-05-20T21:10:29.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:29 smithi055 ceph-mon[35213]: pgmap v820: 97 pgs: 97 active+clean; 240 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.4 MiB/s wr, 295 op/s 2024-05-20T21:10:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:29 smithi026 ceph-mon[31990]: pgmap v820: 97 pgs: 97 active+clean; 240 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.4 MiB/s wr, 295 op/s 2024-05-20T21:10:30.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5914 14.05 MB/sec warmup 35 sec latency 112.309 ms 2024-05-20T21:10:31.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6098 14.23 MB/sec warmup 36 sec latency 124.177 ms 2024-05-20T21:10:31.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:30 smithi055 ceph-mon[35213]: pgmap v821: 97 pgs: 97 active+clean; 235 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-05-20T21:10:31.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:30 smithi026 ceph-mon[31990]: pgmap v821: 97 pgs: 97 active+clean; 235 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-05-20T21:10:31.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:30 smithi144 ceph-mon[35660]: pgmap v821: 97 pgs: 97 active+clean; 235 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-05-20T21:10:32.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6238 13.87 MB/sec warmup 37 sec latency 138.166 ms 2024-05-20T21:10:33.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6391 13.57 MB/sec warmup 38 sec latency 93.354 ms 2024-05-20T21:10:33.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:33 smithi144 ceph-mon[35660]: pgmap v822: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-05-20T21:10:33.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:33 smithi055 ceph-mon[35213]: pgmap v822: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-05-20T21:10:33.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:33 smithi026 ceph-mon[31990]: pgmap v822: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-05-20T21:10:34.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6559 13.29 MB/sec warmup 39 sec latency 94.458 ms 2024-05-20T21:10:35.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6754 13.16 MB/sec warmup 40 sec latency 93.157 ms 2024-05-20T21:10:35.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:35 smithi144 ceph-mon[35660]: pgmap v823: 97 pgs: 97 active+clean; 224 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 4.4 MiB/s wr, 294 op/s 2024-05-20T21:10:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:35 smithi055 ceph-mon[35213]: pgmap v823: 97 pgs: 97 active+clean; 224 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 4.4 MiB/s wr, 294 op/s 2024-05-20T21:10:35.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:35 smithi026 ceph-mon[31990]: pgmap v823: 97 pgs: 97 active+clean; 224 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 4.4 MiB/s wr, 294 op/s 2024-05-20T21:10:36.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6936 12.95 MB/sec warmup 41 sec latency 88.465 ms 2024-05-20T21:10:37.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7100 12.95 MB/sec warmup 42 sec latency 91.447 ms 2024-05-20T21:10:37.300 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:37 smithi144 ceph-mon[35660]: pgmap v824: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-05-20T21:10:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:37 smithi055 ceph-mon[35213]: pgmap v824: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-05-20T21:10:37.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:37 smithi026 ceph-mon[31990]: pgmap v824: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-05-20T21:10:38.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7263 12.81 MB/sec warmup 43 sec latency 79.686 ms 2024-05-20T21:10:39.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7429 12.71 MB/sec warmup 44 sec latency 82.237 ms 2024-05-20T21:10:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:39 smithi055 ceph-mon[35213]: pgmap v825: 97 pgs: 97 active+clean; 225 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 3.8 MiB/s wr, 315 op/s 2024-05-20T21:10:39.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:39 smithi026 ceph-mon[31990]: pgmap v825: 97 pgs: 97 active+clean; 225 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 3.8 MiB/s wr, 315 op/s 2024-05-20T21:10:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:39 smithi144 ceph-mon[35660]: pgmap v825: 97 pgs: 97 active+clean; 225 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 3.8 MiB/s wr, 315 op/s 2024-05-20T21:10:40.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7627 12.82 MB/sec warmup 45 sec latency 97.008 ms 2024-05-20T21:10:40.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-20T21:10:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 -- 172.21.15.26:0/1774676649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 msgr2=0x7f7778102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 --2- 172.21.15.26:0/1774676649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 0x7f7778102aa0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7770009ca0 tx=0x7f777002a8c0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 -- 172.21.15.26:0/1774676649 shutdown_connections 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 --2- 172.21.15.26:0/1774676649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 0x7f7778102aa0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 --2- 172.21.15.26:0/1774676649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101c80 0x7f77781020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 --2- 172.21.15.26:0/1774676649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7778101320 0x7f7778101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 -- 172.21.15.26:0/1774676649 >> 172.21.15.26:0/1774676649 conn(0x7f77780fcf20 msgr2=0x7f77780ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 -- 172.21.15.26:0/1774676649 shutdown_connections 2024-05-20T21:10:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.067+0000 7f777e546640 1 -- 172.21.15.26:0/1774676649 wait complete. 2024-05-20T21:10:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 Processor -- start 2024-05-20T21:10:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 -- start start 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 0x7f777819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7778101c80 0x7f777819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 0x7f77781a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f7777fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 0x7f777819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777cae3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 0x7f77781a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f7777fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 0x7f777819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40946/0 (socket says 172.21.15.26:40946) 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f7777fff640 1 -- 172.21.15.26:0/4064823473 learned_addr learned my addr 172.21.15.26:0/4064823473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f77777fe640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7778101c80 0x7f777819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77781106b0 con 0x7f7778102620 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110530 con 0x7f7778101320 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.068+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110830 con 0x7f7778101c80 2024-05-20T21:10:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f7777fff640 1 -- 172.21.15.26:0/4064823473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7778101c80 msgr2=0x7f777819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f7777fff640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7778101c80 0x7f777819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f7777fff640 1 -- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 msgr2=0x7f77781a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f7777fff640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 0x7f77781a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f7777fff640 1 -- 172.21.15.26:0/4064823473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77781a2920 con 0x7f7778101320 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f7777fff640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 0x7f777819ad10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9f0 tx=0x7f776800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f77757fa640 1 -- 172.21.15.26:0/4064823473 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776800cd80 con 0x7f7778101320 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f77757fa640 1 -- 172.21.15.26:0/4064823473 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77680108c0 con 0x7f7778101320 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f77757fa640 1 -- 172.21.15.26:0/4064823473 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680187c0 con 0x7f7778101320 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77781a2c10 con 0x7f7778101320 2024-05-20T21:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.069+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a31b0 con 0x7f7778101320 2024-05-20T21:10:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.072+0000 7f7781267740 1 Processor -- start 2024-05-20T21:10:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.072+0000 7f77757fa640 1 -- 172.21.15.26:0/4064823473 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7768010430 con 0x7f7778101320 2024-05-20T21:10:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 -- start start 2024-05-20T21:10:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47966ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba4796bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba4796e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba478d7c30 con 0x55ba479670b0 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ba478c77d0 con 0x55ba479666c0 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7781267740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ba478cb810 con 0x55ba4796c220 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47966ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f7777fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba4796bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47966ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40952/0 (socket says 172.21.15.26:40952) 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 -- 172.21.15.26:0/2272943452 learned_addr learned my addr 172.21.15.26:0/2272943452 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f77777fe640 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba4796e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 -- 172.21.15.26:0/2272943452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 msgr2=0x55ba4796e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba4796e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 -- 172.21.15.26:0/2272943452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 msgr2=0x55ba4796bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba4796bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.073+0000 7f777cae3640 1 -- 172.21.15.26:0/2272943452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba475dc210 con 0x55ba479666c0 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f77757fa640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f77600b2920 0x7f77600b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f777cae3640 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47966ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7770009ca0 tx=0x7f7770009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56eeb70b48eaf8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f775e7fc640 1 -- 172.21.15.26:0/2272943452 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770031a80 con 0x55ba479666c0 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f77757fa640 1 -- 172.21.15.26:0/4064823473 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f77680e25d0 con 0x7f7778101320 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f7777fff640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f77600b2920 0x7f77600b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f775e7fc640 1 -- 172.21.15.26:0/2272943452 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7770036030 con 0x55ba479666c0 2024-05-20T21:10:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.074+0000 7f775e7fc640 1 -- 172.21.15.26:0/2272943452 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700044a0 con 0x55ba479666c0 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 -- 172.21.15.26:0/2272943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 msgr2=0x55ba47966ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7777fff640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f77600b2920 0x7f77600b4de0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f77680e03e0 tx=0x7f77680e8110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47966ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7770009ca0 tx=0x7f7770009a20 comp rx=0 tx=0).stop 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 -- 172.21.15.26:0/2272943452 shutdown_connections 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba4796e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba4796bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 --2- 172.21.15.26:0/2272943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47966ae0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 -- 172.21.15.26:0/2272943452 >> 172.21.15.26:0/2272943452 conn(0x55ba47961d30 msgr2=0x55ba47962130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 -- 172.21.15.26:0/2272943452 shutdown_connections 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.075+0000 7f7781267740 1 -- 172.21.15.26:0/2272943452 wait complete. 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.076+0000 7f7781267740 1 -- start start 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.076+0000 7f7781267740 1 Processor -- start 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.076+0000 7f7781267740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47970600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47970b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba47977b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47970600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47970600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40956/0 (socket says 172.21.15.26:40956) 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 -- 172.21.15.26:0/1556090055 learned_addr learned my addr 172.21.15.26:0/1556090055 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f77777fe640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47970b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba475ad790 con 0x55ba479670b0 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ba47883890 con 0x55ba479666c0 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ba4796eff0 con 0x55ba4796c220 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba47977b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 -- 172.21.15.26:0/1556090055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 msgr2=0x55ba47977b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba47977b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 -- 172.21.15.26:0/1556090055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 msgr2=0x55ba47970b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47970b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f777cae3640 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba476a8620 con 0x55ba479666c0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 10 client.0 _ll_register_callbacks cb 0x55ba475dba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f77777fe640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47970b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.077+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7770036030 con 0x55ba479666c0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f777cae3640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47970600 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f777002ade0 tx=0x7f77700095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e78f422b3e6e937 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f77577fe640 10 client.29123 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f77577fe640 1 -- 172.21.15.26:0/1556090055 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770059060 con 0x55ba479666c0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f77577fe640 1 -- 172.21.15.26:0/1556090055 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f77700043e0 con 0x55ba479666c0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f77577fe640 1 -- 172.21.15.26:0/1556090055 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77700045e0 con 0x55ba479666c0 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f7781267740 10 client.29123 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.078+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55ba4797a9c0 con 0x55ba479666c0 2024-05-20T21:10:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f77577fe640 1 -- 172.21.15.26:0/1556090055 <== mon.2 v2:172.21.15.55:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7770043070 con 0x55ba479666c0 2024-05-20T21:10:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f7781267740 10 client.29123 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f7781267740 10 client.29123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f7781267740 20 client.29123 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f7781267740 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba4797be50 0x55ba4797e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f7781267740 4 client.29123 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba478d9b80 con 0x55ba4797be50 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.079+0000 7f77777fe640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba4797be50 0x55ba4797e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.080+0000 7f77577fe640 10 client.29123 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.080+0000 7f77777fe640 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba4797be50 0x55ba4797e2b0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff33ac9cb648144b server_cookie=7092ec57b96ad382 in_seq=0 out_seq=0 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.082+0000 7f77577fe640 1 -- 172.21.15.26:0/1556090055 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ba478d9b80 con 0x55ba4797be50 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.082+0000 7f77577fe640 10 client.29123 handle_command_reply: tid=0 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.082+0000 7f7781267740 1 client.29123 shutdown 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 msgr2=0x55ba47970600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47970600 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f777002ade0 tx=0x7f77700095c0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 shutdown_connections 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba4797be50 0x55ba4797e2b0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba4796c220 0x55ba47977b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47970b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 --2- 172.21.15.26:0/1556090055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba479666c0 0x55ba47970600 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f77577fe640 0 client.29123 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 >> 172.21.15.26:0/1556090055 conn(0x55ba47961d30 msgr2=0x55ba47960fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 shutdown_connections 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 1 -- 172.21.15.26:0/1556090055 wait complete. 2024-05-20T21:10:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.083+0000 7f7781267740 20 client.29123 trim_cache size 0 max 16384 2024-05-20T21:10:41.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7846 12.83 MB/sec warmup 46 sec latency 91.861 ms 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 Processor -- start 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 -- start start 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47961730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47977b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba4796b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ae7f0 con 0x55ba479670b0 2024-05-20T21:10:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ac030 con 0x55ba4796c220 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f7781267740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77781a9e80 con 0x55ba479666c0 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47977b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47977b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40490/0 (socket says 172.21.15.26:40490) 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 -- 172.21.15.26:0/3901352351 learned_addr learned my addr 172.21.15.26:0/3901352351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f777cae3640 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba4796b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f777cae3640 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47961730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 -- 172.21.15.26:0/3901352351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 msgr2=0x55ba47961730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47961730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 -- 172.21.15.26:0/3901352351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 msgr2=0x55ba4796b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.111+0000 7f77777fe640 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba4796b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.112+0000 7f77777fe640 1 -- 172.21.15.26:0/3901352351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4796b5a0 con 0x55ba479670b0 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.112+0000 7f777cae3640 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba4796b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.112+0000 7f77777fe640 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47977b40 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x55ba4797eb20 tx=0x7f776c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b113855c2ceb8b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.112+0000 7f775dffb640 1 -- 172.21.15.26:0/3901352351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0106f0 con 0x55ba479670b0 2024-05-20T21:10:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.112+0000 7f775dffb640 1 -- 172.21.15.26:0/3901352351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f776c010d20 con 0x55ba479670b0 2024-05-20T21:10:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.112+0000 7f775dffb640 1 -- 172.21.15.26:0/3901352351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c01ed50 con 0x55ba479670b0 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 -- 172.21.15.26:0/3901352351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 msgr2=0x55ba47977b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47977b40 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x55ba4797eb20 tx=0x7f776c005f90 comp rx=0 tx=0).stop 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 -- 172.21.15.26:0/3901352351 shutdown_connections 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba4796b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47977b40 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 --2- 172.21.15.26:0/3901352351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47961730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 -- 172.21.15.26:0/3901352351 >> 172.21.15.26:0/3901352351 conn(0x55ba47961d30 msgr2=0x55ba4796f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 -- 172.21.15.26:0/3901352351 shutdown_connections 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 -- 172.21.15.26:0/3901352351 wait complete. 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.113+0000 7f7781267740 1 -- start start 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 Processor -- start 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47974fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47975510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba47976520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77781aa4f0 con 0x55ba479670b0 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ab960 con 0x55ba4796c220 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77781aabc0 con 0x55ba479666c0 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f7781267740 10 client.0 _ll_register_callbacks cb 0x7f77680e8060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f77777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47975510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f77777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47975510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40492/0 (socket says 172.21.15.26:40492) 2024-05-20T21:10:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f77777fe640 1 -- 172.21.15.26:0/228581379 learned_addr learned my addr 172.21.15.26:0/228581379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f777cae3640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47974fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.114+0000 7f777cae3640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba47976520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f777cae3640 1 -- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 msgr2=0x55ba47976520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f777cae3640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba47976520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f777cae3640 1 -- 172.21.15.26:0/228581379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 msgr2=0x55ba47975510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f777cae3640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47975510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f777cae3640 1 -- 172.21.15.26:0/228581379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba47976b10 con 0x55ba479666c0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f77777fe640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47975510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f777cae3640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47974fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7770000f80 tx=0x7f777003f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fa55fba25f3f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f77577fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f77577fe640 1 -- 172.21.15.26:0/228581379 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770030c00 con 0x55ba479666c0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c010d20 con 0x55ba479666c0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f77577fe640 1 -- 172.21.15.26:0/228581379 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7770030da0 con 0x55ba479666c0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f77577fe640 1 -- 172.21.15.26:0/228581379 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7770009b20 con 0x55ba479666c0 2024-05-20T21:10:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f7781267740 10 client.28822 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.115+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ba47977e20 con 0x55ba479666c0 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f77577fe640 1 -- 172.21.15.26:0/228581379 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f776c010d20 con 0x55ba479666c0 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f77577fe640 1 -- 172.21.15.26:0/228581379 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7770043040 con 0x55ba479666c0 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f7781267740 10 client.28822 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f7781267740 10 client.28822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f7781267740 20 client.28822 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f7781267740 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba479e7110 0x55ba479e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f7781267740 4 client.28822 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba478d3360 con 0x55ba479e7110 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.116+0000 7f77777fe640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba479e7110 0x55ba479e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.117+0000 7f77577fe640 10 client.28822 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.117+0000 7f77777fe640 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba479e7110 0x55ba479e9530 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ae12578bb4bfcb server_cookie=2cbc8b9ee9f36a26 in_seq=0 out_seq=0 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.252+0000 7f77577fe640 1 -- 172.21.15.26:0/228581379 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ba478d3360 con 0x55ba479e7110 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.252+0000 7f77577fe640 10 client.28822 handle_command_reply: tid=0 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.252+0000 7f7781267740 1 client.28822 shutdown 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.252+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 msgr2=0x55ba47974fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.252+0000 7f7781267740 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47974fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7770000f80 tx=0x7f777003f040 comp rx=0 tx=0).stop 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 shutdown_connections 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ba479e7110 0x55ba479e9530 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f77577fe640 0 client.28822 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ba4796c220 0x55ba47976520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba479670b0 0x55ba47975510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 --2- 172.21.15.26:0/228581379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba479666c0 0x55ba47974fd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 >> 172.21.15.26:0/228581379 conn(0x55ba47961d30 msgr2=0x55ba4796b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 shutdown_connections 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 1 -- 172.21.15.26:0/228581379 wait complete. 2024-05-20T21:10:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.253+0000 7f7781267740 20 client.28822 trim_cache size 0 max 16384 2024-05-20T21:10:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:10:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:10:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f77600b2920 msgr2=0x7f77600b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f77600b2920 0x7f77600b4de0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f77680e03e0 tx=0x7f77680e8110 comp rx=0 tx=0).stop 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 msgr2=0x7f777819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 0x7f777819ad10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9f0 tx=0x7f776800eeb0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 shutdown_connections 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f77600b2920 0x7f77600b4de0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7778102620 0x7f77781a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7778101c80 0x7f777819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 --2- 172.21.15.26:0/4064823473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7778101320 0x7f777819ad10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 >> 172.21.15.26:0/4064823473 conn(0x7f77780fcf20 msgr2=0x7f77780fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 shutdown_connections 2024-05-20T21:10:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:41.254+0000 7f777e546640 1 -- 172.21.15.26:0/4064823473 wait complete. 2024-05-20T21:10:41.354 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:41 smithi026 ceph-mon[31990]: pgmap v826: 97 pgs: 97 active+clean; 226 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-05-20T21:10:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:41 smithi055 ceph-mon[35213]: pgmap v826: 97 pgs: 97 active+clean; 226 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-05-20T21:10:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:41 smithi144 ceph-mon[35660]: pgmap v826: 97 pgs: 97 active+clean; 226 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-05-20T21:10:42.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8050 12.80 MB/sec warmup 47 sec latency 147.312 ms 2024-05-20T21:10:43.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8202 12.78 MB/sec warmup 48 sec latency 107.860 ms 2024-05-20T21:10:43.384 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:43 smithi026 ceph-mon[31990]: pgmap v827: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.2 MiB/s wr, 281 op/s 2024-05-20T21:10:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:43 smithi055 ceph-mon[35213]: pgmap v827: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.2 MiB/s wr, 281 op/s 2024-05-20T21:10:43.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 -- 172.21.15.26:0/3435023565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c102a80 msgr2=0x7f8d7c102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 --2- 172.21.15.26:0/3435023565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c102a80 0x7f8d7c102e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d780099c0 tx=0x7f8d78009cd0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 -- 172.21.15.26:0/3435023565 shutdown_connections 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 --2- 172.21.15.26:0/3435023565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c1077d0 0x7f8d7c10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 --2- 172.21.15.26:0/3435023565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c102a80 0x7f8d7c102e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 --2- 172.21.15.26:0/3435023565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c103430 0x7f8d7c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 -- 172.21.15.26:0/3435023565 >> 172.21.15.26:0/3435023565 conn(0x7f8d7c0793f0 msgr2=0x7f8d7c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 -- 172.21.15.26:0/3435023565 shutdown_connections 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.536+0000 7f8d840e4700 1 -- 172.21.15.26:0/3435023565 wait complete. 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 Processor -- start 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 -- start start 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 0x7f8d7c10ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c103430 0x7f8d7c110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c1077d0 0x7f8d7c110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c113630 con 0x7f8d7c102a80 2024-05-20T21:10:43.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1134b0 con 0x7f8d7c1077d0 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d840e4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1137b0 con 0x7f8d7c103430 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d81e80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 0x7f8d7c10ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d81e80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 0x7f8d7c10ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40516/0 (socket says 172.21.15.26:40516) 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d81e80700 1 -- 172.21.15.26:0/1196009607 learned_addr learned my addr 172.21.15.26:0/1196009607 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d82681700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c1077d0 0x7f8d7c110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.537+0000 7f8d8167f700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c103430 0x7f8d7c110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d82681700 1 -- 172.21.15.26:0/1196009607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c103430 msgr2=0x7f8d7c110520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d82681700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c103430 0x7f8d7c110520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d82681700 1 -- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 msgr2=0x7f8d7c10ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d82681700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 0x7f8d7c10ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d82681700 1 -- 172.21.15.26:0/1196009607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c1a7f90 con 0x7f8d7c1077d0 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d8167f700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c103430 0x7f8d7c110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d81e80700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 0x7f8d7c10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:43.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d82681700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c1077d0 0x7f8d7c110a60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400efb0 tx=0x7f8d7400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:43.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d72ffd700 1 -- 172.21.15.26:0/1196009607 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74003a00 con 0x7f8d7c1077d0 2024-05-20T21:10:43.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c1a8280 con 0x7f8d7c1077d0 2024-05-20T21:10:43.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d72ffd700 1 -- 172.21.15.26:0/1196009607 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d7400ce30 con 0x7f8d7c1077d0 2024-05-20T21:10:43.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d72ffd700 1 -- 172.21.15.26:0/1196009607 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74004640 con 0x7f8d7c1077d0 2024-05-20T21:10:43.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.538+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1a87c0 con 0x7f8d7c1077d0 2024-05-20T21:10:43.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.542+0000 7f8d72ffd700 1 -- 172.21.15.26:0/1196009607 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8d74012050 con 0x7f8d7c1077d0 2024-05-20T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.543+0000 7f8d72ffd700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d680b2880 0x7f8d680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.543+0000 7f8d81e80700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d680b2880 0x7f8d680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.544+0000 7f8d72ffd700 1 -- 172.21.15.26:0/1196009607 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8d740e1aa0 con 0x7f8d7c1077d0 2024-05-20T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.544+0000 7f8d81e80700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d680b2880 0x7f8d680b4d40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8d78003760 tx=0x7f8d78000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.544+0000 7f8d8e97ab80 1 Processor -- start 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.544+0000 7f8d8e97ab80 1 -- start start 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb788a0 0x55e54cb78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb79260 0x55e54cb85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e54cad6d70 con 0x55e54cb7e320 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e54cac6910 con 0x55e54cb788a0 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e54caca950 con 0x55e54cb79260 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d81e80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d81e80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb7e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40526/0 (socket says 172.21.15.26:40526) 2024-05-20T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d81e80700 1 -- 172.21.15.26:0/893148661 learned_addr learned my addr 172.21.15.26:0/893148661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d8167f700 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb788a0 0x55e54cb78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d82681700 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb79260 0x55e54cb85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d82681700 1 -- 172.21.15.26:0/893148661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb788a0 msgr2=0x55e54cb78d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d82681700 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb788a0 0x55e54cb78d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d82681700 1 -- 172.21.15.26:0/893148661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 msgr2=0x55e54cb7e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d82681700 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb7e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d82681700 1 -- 172.21.15.26:0/893148661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e54ca6ffb0 con 0x55e54cb79260 2024-05-20T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.545+0000 7f8d81e80700 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.546+0000 7f8d82681700 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb79260 0x55e54cb85080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d740df540 tx=0x7f8d740a8b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47e7df8e70aa1e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.546+0000 7f8d67fff700 1 -- 172.21.15.26:0/893148661 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740a97f0 con 0x55e54cb79260 2024-05-20T21:10:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.546+0000 7f8d67fff700 1 -- 172.21.15.26:0/893148661 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d740a90f0 con 0x55e54cb79260 2024-05-20T21:10:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.546+0000 7f8d67fff700 1 -- 172.21.15.26:0/893148661 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740ad120 con 0x55e54cb79260 2024-05-20T21:10:43.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/893148661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb79260 msgr2=0x55e54cb85080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb79260 0x55e54cb85080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d740df540 tx=0x7f8d740a8b10 comp rx=0 tx=0).stop 2024-05-20T21:10:43.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/893148661 shutdown_connections 2024-05-20T21:10:43.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb79260 0x55e54cb85080 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/893148661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb788a0 0x55e54cb78d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/893148661 >> 172.21.15.26:0/893148661 conn(0x55e54caa4240 msgr2=0x55e54cb733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/893148661 shutdown_connections 2024-05-20T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.547+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/893148661 wait complete. 2024-05-20T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.548+0000 7f8d8e97ab80 1 -- start start 2024-05-20T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.548+0000 7f8d8e97ab80 1 Processor -- start 2024-05-20T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb84ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb82030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb82570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e54c9ccfa0 con 0x55e54cb79260 2024-05-20T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb84ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb84ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:38888/0 (socket says 172.21.15.26:38888) 2024-05-20T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 -- 172.21.15.26:0/396195696 learned_addr learned my addr 172.21.15.26:0/396195696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8167f700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb82030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e54ca2eda0 con 0x55e54cb7e320 2024-05-20T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d82681700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb82570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e54cb856e0 con 0x55e54cb788a0 2024-05-20T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d8e97ab80 10 client.0 _ll_register_callbacks cb 0x55e54ca61950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 -- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 msgr2=0x55e54cb82570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb82570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 -- 172.21.15.26:0/396195696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 msgr2=0x55e54cb82030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.549+0000 7f8d81e80700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb82030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d81e80700 1 -- 172.21.15.26:0/396195696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e54ca168d0 con 0x55e54cb788a0 2024-05-20T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d81e80700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb84ce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009990 tx=0x7f8d78009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa015d4cf7419ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d657fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d657fa700 1 -- 172.21.15.26:0/396195696 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78024620 con 0x55e54cb788a0 2024-05-20T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d657fa700 1 -- 172.21.15.26:0/396195696 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8d780247c0 con 0x55e54cb788a0 2024-05-20T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d740a90f0 con 0x55e54cb788a0 2024-05-20T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d657fa700 1 -- 172.21.15.26:0/396195696 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8d78024a00 con 0x55e54cb788a0 2024-05-20T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d8e97ab80 10 client.28834 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.550+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e54ca80760 con 0x55e54cb788a0 2024-05-20T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d657fa700 1 -- 172.21.15.26:0/396195696 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d740a90f0 con 0x55e54cb788a0 2024-05-20T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d657fa700 1 -- 172.21.15.26:0/396195696 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8d78030940 con 0x55e54cb788a0 2024-05-20T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8e97ab80 10 client.28834 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8e97ab80 10 client.28834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8e97ab80 20 client.28834 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cb94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8e97ab80 4 client.28834 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e54cad8cc0 con 0x55e54cb92280 2024-05-20T21:10:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.551+0000 7f8d8167f700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cb94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.552+0000 7f8d657fa700 10 client.28834 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.552+0000 7f8d8167f700 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cb94740 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15713eb3260bc35 server_cookie=4acf754ec9006786 in_seq=0 out_seq=0 2024-05-20T21:10:43.563 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:43 smithi144 ceph-mon[35660]: pgmap v827: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.2 MiB/s wr, 281 op/s 2024-05-20T21:10:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d657fa700 1 -- 172.21.15.26:0/396195696 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e54cad8cc0 con 0x55e54cb92280 2024-05-20T21:10:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d657fa700 10 client.28834 handle_command_reply: tid=0 2024-05-20T21:10:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 client.28834 shutdown 2024-05-20T21:10:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 msgr2=0x55e54cb84ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb84ce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009990 tx=0x7f8d78009cd0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 shutdown_connections 2024-05-20T21:10:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cb94740 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d657fa700 0 client.28834 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:43.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb84ce0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/396195696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb82030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.554+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 >> 172.21.15.26:0/396195696 conn(0x55e54caa4240 msgr2=0x55e54cb72af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:43.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.555+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 shutdown_connections 2024-05-20T21:10:43.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.555+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/396195696 wait complete. 2024-05-20T21:10:43.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.555+0000 7f8d8e97ab80 20 client.28834 trim_cache size 0 max 16384 2024-05-20T21:10:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 Processor -- start 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 -- start start 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb7c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb79260 0x55e54cb73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb77890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1b26b0 con 0x55e54cb7e320 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1afeb0 con 0x55e54cb79260 2024-05-20T21:10:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.590+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1add00 con 0x55e54cb788a0 2024-05-20T21:10:43.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb77890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb77890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40548/0 (socket says 172.21.15.26:40548) 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 -- 172.21.15.26:0/2184323294 learned_addr learned my addr 172.21.15.26:0/2184323294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d8167f700 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb79260 0x55e54cb73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d81e80700 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb7c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 -- 172.21.15.26:0/2184323294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 msgr2=0x55e54cb7c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb7c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 -- 172.21.15.26:0/2184323294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb79260 msgr2=0x55e54cb73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb79260 0x55e54cb73270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 -- 172.21.15.26:0/2184323294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e54cb85050 con 0x55e54cb7e320 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d81e80700 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d82681700 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb77890 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8d740df8b0 tx=0x7f8d740a50e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e5b2280a92bd48 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d677fe700 1 -- 172.21.15.26:0/2184323294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740ad4a0 con 0x55e54cb7e320 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d677fe700 1 -- 172.21.15.26:0/2184323294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d740b1ec0 con 0x55e54cb7e320 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.591+0000 7f8d677fe700 1 -- 172.21.15.26:0/2184323294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740048b0 con 0x55e54cb7e320 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.592+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/2184323294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 msgr2=0x55e54cb77890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.592+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb77890 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8d740df8b0 tx=0x7f8d740a50e0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.592+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/2184323294 shutdown_connections 2024-05-20T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.592+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb7e320 0x55e54cb77890 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/2184323294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb79260 0x55e54cb73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/2184323294 >> 172.21.15.26:0/2184323294 conn(0x55e54caa4240 msgr2=0x55e54cb7d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/2184323294 shutdown_connections 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/2184323294 wait complete. 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 -- start start 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.593+0000 7f8d8e97ab80 1 Processor -- start 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb8af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb99720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb99c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1ae370 con 0x55e54cb79260 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1af7e0 con 0x55e54cb7e320 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1aea40 con 0x55e54cb788a0 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8e97ab80 10 client.0 _ll_register_callbacks cb 0x55e54ca61950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb99720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb99720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40550/0 (socket says 172.21.15.26:40550) 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 -- 172.21.15.26:0/3833114589 learned_addr learned my addr 172.21.15.26:0/3833114589 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d81e80700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb8af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d82681700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb99c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 -- 172.21.15.26:0/3833114589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 msgr2=0x55e54cb8af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb8af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 -- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 msgr2=0x55e54cb99c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb99c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 -- 172.21.15.26:0/3833114589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e54cb9a2e0 con 0x55e54cb79260 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d81e80700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb8af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d8167f700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb99720 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c111a70 tx=0x7f8d6c012a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cf7f577026eca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.594+0000 7f8d657fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d657fa700 1 -- 172.21.15.26:0/3833114589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c004a30 con 0x55e54cb79260 2024-05-20T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d657fa700 1 -- 172.21.15.26:0/3833114589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8d6c004bd0 con 0x55e54cb79260 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e54cad24a0 con 0x55e54cb79260 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d657fa700 1 -- 172.21.15.26:0/3833114589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c004dd0 con 0x55e54cb79260 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d8e97ab80 10 client.19944 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e54cbfcd90 con 0x55e54cb79260 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d657fa700 1 -- 172.21.15.26:0/3833114589 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d6c0054b0 con 0x55e54cb79260 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.595+0000 7f8d657fa700 1 -- 172.21.15.26:0/3833114589 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8d6c010b80 con 0x55e54cb79260 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d8e97ab80 10 client.19944 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d8e97ab80 10 client.19944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d8e97ab80 20 client.19944 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cbff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d8e97ab80 4 client.19944 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e54cad24a0 con 0x55e54cb92280 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d81e80700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cbff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d657fa700 10 client.19944 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.596+0000 7f8d81e80700 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cbff5c0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5564d86349e41dc server_cookie=51bef8d5da8aca4f in_seq=0 out_seq=0 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.597+0000 7f8d657fa700 1 -- 172.21.15.26:0/3833114589 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e54cad24a0 con 0x55e54cb92280 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.597+0000 7f8d657fa700 10 client.19944 handle_command_reply: tid=0 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.597+0000 7f8d8e97ab80 1 client.19944 shutdown 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.597+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 msgr2=0x55e54cb99720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.597+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb99720 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c111a70 tx=0x7f8d6c012a70 comp rx=0 tx=0).stop 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 shutdown_connections 2024-05-20T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e54cb92280 0x55e54cbff5c0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e54cb7e320 0x55e54cb99c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54cb788a0 0x55e54cb8af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d657fa700 0 client.19944 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 --2- 172.21.15.26:0/3833114589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e54cb79260 0x55e54cb99720 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 >> 172.21.15.26:0/3833114589 conn(0x55e54caa4240 msgr2=0x55e54cb77890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 shutdown_connections 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 1 -- 172.21.15.26:0/3833114589 wait complete. 2024-05-20T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.598+0000 7f8d8e97ab80 20 client.19944 trim_cache size 0 max 16384 2024-05-20T21:10:43.607 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:10:43.607 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-05-20T21:10:43.607 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-05-20T21:10:43.607 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:10:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d680b2880 msgr2=0x7f8d680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d680b2880 0x7f8d680b4d40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8d78003760 tx=0x7f8d78000bc0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c1077d0 msgr2=0x7f8d7c110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c1077d0 0x7f8d7c110a60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400efb0 tx=0x7f8d7400c3a0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 shutdown_connections 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8d680b2880 0x7f8d680b4d40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c102a80 0x7f8d7c10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7c1077d0 0x7f8d7c110a60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 --2- 172.21.15.26:0/1196009607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7c103430 0x7f8d7c110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 >> 172.21.15.26:0/1196009607 conn(0x7f8d7c0793f0 msgr2=0x7f8d7c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 shutdown_connections 2024-05-20T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:43.599+0000 7f8d840e4700 1 -- 172.21.15.26:0/1196009607 wait complete. 2024-05-20T21:10:44.108 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8351 12.71 MB/sec warmup 49 sec latency 126.475 ms 2024-05-20T21:10:44.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T21:10:44.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:10:44.735 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:10:45.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8444 12.57 MB/sec warmup 50 sec latency 117.784 ms 2024-05-20T21:10:45.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:45 smithi026 ceph-mon[31990]: pgmap v828: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.8 MiB/s wr, 280 op/s 2024-05-20T21:10:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:45 smithi055 ceph-mon[35213]: pgmap v828: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.8 MiB/s wr, 280 op/s 2024-05-20T21:10:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:45 smithi144 ceph-mon[35660]: pgmap v828: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.8 MiB/s wr, 280 op/s 2024-05-20T21:10:46.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8611 12.64 MB/sec warmup 51 sec latency 98.643 ms 2024-05-20T21:10:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 -- 172.21.15.26:0/324853403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 msgr2=0x7fb3541103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 --2- 172.21.15.26:0/324853403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 0x7fb3541103c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00b0a0 tx=0x7fb34c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 -- 172.21.15.26:0/324853403 shutdown_connections 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 --2- 172.21.15.26:0/324853403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 0x7fb3541103c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 --2- 172.21.15.26:0/324853403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3541095a0 0x7fb354109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 --2- 172.21.15.26:0/324853403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35406bab0 0x7fb35406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.675+0000 7fb358d87700 1 -- 172.21.15.26:0/324853403 >> 172.21.15.26:0/324853403 conn(0x7fb3540793b0 msgr2=0x7fb3540797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.676+0000 7fb358d87700 1 -- 172.21.15.26:0/324853403 shutdown_connections 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.676+0000 7fb358d87700 1 -- 172.21.15.26:0/324853403 wait complete. 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 Processor -- start 2024-05-20T21:10:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 -- start start 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 0x7fb35419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 0x7fb35419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3541095a0 0x7fb3541998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb354113570 con 0x7fb35406c470 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541133f0 con 0x7fb35406bab0 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb358d87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541136f0 con 0x7fb3541095a0 2024-05-20T21:10:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb35259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 0x7fb35419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb35259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 0x7fb35419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:41028/0 (socket says 172.21.15.26:41028) 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb35259c700 1 -- 172.21.15.26:0/3139495179 learned_addr learned my addr 172.21.15.26:0/3139495179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.677+0000 7fb352d9d700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3541095a0 0x7fb3541998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb35259c700 1 -- 172.21.15.26:0/3139495179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3541095a0 msgr2=0x7fb3541998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb35259c700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3541095a0 0x7fb3541998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb35259c700 1 -- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 msgr2=0x7fb35419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb35259c700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 0x7fb35419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb35259c700 1 -- 172.21.15.26:0/3139495179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35419a050 con 0x7fb35406bab0 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb352d9d700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3541095a0 0x7fb3541998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb35259c700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 0x7fb35419f730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb34400e9a0 tx=0x7fb34400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.678+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34400cbd0 con 0x7fb35406bab0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.679+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35419a2e0 con 0x7fb35406bab0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.679+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb34400cd70 con 0x7fb35406bab0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.679+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344018680 con 0x7fb35406bab0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.679+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb354102430 con 0x7fb35406bab0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.680+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb354104c60 con 0x7fb35406bab0 2024-05-20T21:10:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.682+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb344010430 con 0x7fb35406bab0 2024-05-20T21:10:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.683+0000 7fb3437fe700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb33c0b2880 0x7fb33c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.683+0000 7fb351d9b700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb33c0b2880 0x7fb33c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.683+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb3440e24c0 con 0x7fb35406bab0 2024-05-20T21:10:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.684+0000 7fb351d9b700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb33c0b2880 0x7fb33c0b4d40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb348005950 tx=0x7fb3480058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:46.688+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3440aa120 con 0x7fb35406bab0 2024-05-20T21:10:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.104+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb35419ae20 con 0x7fb35406bab0 2024-05-20T21:10:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.105+0000 7fb3437fe700 1 -- 172.21.15.26:0/3139495179 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11874 (secure 0 0 0) 0x7fb3440b21a0 con 0x7fb35406bab0 2024-05-20T21:10:47.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:10:47.106 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:10:11.369568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:10:47.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb33c0b2880 msgr2=0x7fb33c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb33c0b2880 0x7fb33c0b4d40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb348005950 tx=0x7fb3480058e0 comp rx=0 tx=0).stop 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 msgr2=0x7fb35419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 0x7fb35419f730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb34400e9a0 tx=0x7fb34400ecb0 comp rx=0 tx=0).stop 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 shutdown_connections 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3541095a0 0x7fb3541998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb35406bab0 0x7fb35419f730 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb33c0b2880 0x7fb33c0b4d40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 --2- 172.21.15.26:0/3139495179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb35406c470 0x7fb35419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.108+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 >> 172.21.15.26:0/3139495179 conn(0x7fb3540793b0 msgr2=0x7fb354101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.109+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 shutdown_connections 2024-05-20T21:10:47.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:10:47.109+0000 7fb358d87700 1 -- 172.21.15.26:0/3139495179 wait complete. 2024-05-20T21:10:47.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8779 12.65 MB/sec warmup 52 sec latency 124.779 ms 2024-05-20T21:10:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 34 2024-05-20T21:10:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:47 smithi055 ceph-mon[35213]: pgmap v829: 97 pgs: 97 active+clean; 239 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-05-20T21:10:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:47 smithi026 ceph-mon[31990]: pgmap v829: 97 pgs: 97 active+clean; 239 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-05-20T21:10:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:47 smithi144 ceph-mon[35660]: pgmap v829: 97 pgs: 97 active+clean; 239 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-05-20T21:10:47.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-20T21:10:48.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8923 12.58 MB/sec warmup 53 sec latency 119.854 ms 2024-05-20T21:10:48.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3139495179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:10:48.280 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:10:48.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3139495179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:10:48.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3139495179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:10:49.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9043 12.46 MB/sec warmup 54 sec latency 182.892 ms 2024-05-20T21:10:49.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:49 smithi055 ceph-mon[35213]: pgmap v830: 97 pgs: 97 active+clean; 242 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.5 MiB/s wr, 304 op/s 2024-05-20T21:10:49.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:49 smithi026 ceph-mon[31990]: pgmap v830: 97 pgs: 97 active+clean; 242 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.5 MiB/s wr, 304 op/s 2024-05-20T21:10:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:49 smithi144 ceph-mon[35660]: pgmap v830: 97 pgs: 97 active+clean; 242 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.5 MiB/s wr, 304 op/s 2024-05-20T21:10:50.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9176 12.37 MB/sec warmup 55 sec latency 95.445 ms 2024-05-20T21:10:51.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9294 12.19 MB/sec warmup 56 sec latency 126.060 ms 2024-05-20T21:10:51.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:51 smithi144 ceph-mon[35660]: pgmap v831: 97 pgs: 97 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 4.6 MiB/s wr, 237 op/s 2024-05-20T21:10:51.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:51 smithi026 ceph-mon[31990]: pgmap v831: 97 pgs: 97 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 4.6 MiB/s wr, 237 op/s 2024-05-20T21:10:51.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:51 smithi055 ceph-mon[35213]: pgmap v831: 97 pgs: 97 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 4.6 MiB/s wr, 237 op/s 2024-05-20T21:10:52.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9405 12.00 MB/sec warmup 57 sec latency 119.255 ms 2024-05-20T21:10:53.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9530 11.89 MB/sec warmup 58 sec latency 189.061 ms 2024-05-20T21:10:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:53 smithi055 ceph-mon[35213]: pgmap v832: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 4.7 MiB/s wr, 246 op/s 2024-05-20T21:10:53.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:53 smithi026 ceph-mon[31990]: pgmap v832: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 4.7 MiB/s wr, 246 op/s 2024-05-20T21:10:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:53 smithi144 ceph-mon[35660]: pgmap v832: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 4.7 MiB/s wr, 246 op/s 2024-05-20T21:10:54.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9674 11.76 MB/sec warmup 59 sec latency 147.027 ms 2024-05-20T21:10:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 -- 172.21.15.26:0/3063747822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 msgr2=0x7fae34103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 --2- 172.21.15.26:0/3063747822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34103f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae28009860 tx=0x7fae2802a340 comp rx=0 tx=0).stop 2024-05-20T21:10:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 -- 172.21.15.26:0/3063747822 shutdown_connections 2024-05-20T21:10:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 --2- 172.21.15.26:0/3063747822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341044a0 0x7fae3410ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 --2- 172.21.15.26:0/3063747822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34103f60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 --2- 172.21.15.26:0/3063747822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341028f0 0x7fae34102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 -- 172.21.15.26:0/3063747822 >> 172.21.15.26:0/3063747822 conn(0x7fae340fde60 msgr2=0x7fae341002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 -- 172.21.15.26:0/3063747822 shutdown_connections 2024-05-20T21:10:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.384+0000 7fae38d9e640 1 -- 172.21.15.26:0/3063747822 wait complete. 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.385+0000 7fae38d9e640 1 Processor -- start 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.385+0000 7fae38d9e640 1 -- start start 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.385+0000 7fae38d9e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341028f0 0x7fae34198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.385+0000 7fae38d9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.385+0000 7fae38d9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341044a0 0x7fae341a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.385+0000 7fae38d9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae3410e2b0 con 0x7fae341044a0 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae38d9e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae3410e130 con 0x7fae341028f0 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae38d9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae3410e430 con 0x7fae34103ae0 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341028f0 0x7fae34198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341028f0 0x7fae34198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60230/0 (socket says 172.21.15.26:60230) 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae337fe640 1 -- 172.21.15.26:0/4031694516 learned_addr learned my addr 172.21.15.26:0/4031694516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae33fff640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341044a0 0x7fae341a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae33fff640 1 -- 172.21.15.26:0/4031694516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 msgr2=0x7fae34199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae32ffd640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae33fff640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae33fff640 1 -- 172.21.15.26:0/4031694516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341028f0 msgr2=0x7fae34198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae33fff640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341028f0 0x7fae34198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae33fff640 1 -- 172.21.15.26:0/4031694516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae341a06a0 con 0x7fae341044a0 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.386+0000 7fae32ffd640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:10:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.387+0000 7fae33fff640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341044a0 0x7fae341a0030 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00ec50 tx=0x7fae2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.387+0000 7fae30ff9640 1 -- 172.21.15.26:0/4031694516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c003b70 con 0x7fae341044a0 2024-05-20T21:10:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.387+0000 7fae30ff9640 1 -- 172.21.15.26:0/4031694516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae2c003d10 con 0x7fae341044a0 2024-05-20T21:10:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.387+0000 7fae30ff9640 1 -- 172.21.15.26:0/4031694516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0107a0 con 0x7fae341044a0 2024-05-20T21:10:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.387+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae341a0990 con 0x7fae341044a0 2024-05-20T21:10:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.388+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae341a0e90 con 0x7fae341044a0 2024-05-20T21:10:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.391+0000 7fae30ff9640 1 -- 172.21.15.26:0/4031694516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fae2c003fc0 con 0x7fae341044a0 2024-05-20T21:10:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.391+0000 7fae3cdfa740 1 Processor -- start 2024-05-20T21:10:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae3cdfa740 1 -- start start 2024-05-20T21:10:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae30ff9640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae1c0b2860 0x7fae1c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae32ffd640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae1c0b2860 0x7fae1c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae30ff9640 1 -- 172.21.15.26:0/4031694516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fae2c0e27c0 con 0x7fae341044a0 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c75cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c7ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.392+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c7b400 0x561a73c7d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c75cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561a73be6c30 con 0x561a73c758a0 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561a73bd67d0 con 0x561a73c7b400 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c75cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40704/0 (socket says 172.21.15.26:40704) 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae337fe640 1 -- 172.21.15.26:0/572426921 learned_addr learned my addr 172.21.15.26:0/572426921 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae32ffd640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c7ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae3cdfa740 1 -- 172.21.15.26:0/572426921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a73bda810 con 0x561a73c76290 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c7b400 0x561a73c7d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae32ffd640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae1c0b2860 0x7fae1c0b4d20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fae28005b60 tx=0x7fae28005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 -- 172.21.15.26:0/572426921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 msgr2=0x561a73c7ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c7ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 -- 172.21.15.26:0/572426921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 msgr2=0x561a73c75cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c75cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 -- 172.21.15.26:0/572426921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a738eb210 con 0x561a73c7b400 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae32ffd640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c7ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae337fe640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c75cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:10:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.393+0000 7fae33fff640 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c7b400 0x561a73c7d890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0a9d10 tx=0x7fae2c0a9c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff39a9d948993d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.394+0000 7fae19ffb640 1 -- 172.21.15.26:0/572426921 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c009820 con 0x561a73c7b400 2024-05-20T21:10:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.394+0000 7fae19ffb640 1 -- 172.21.15.26:0/572426921 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae2c0a9890 con 0x561a73c7b400 2024-05-20T21:10:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.394+0000 7fae19ffb640 1 -- 172.21.15.26:0/572426921 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0c4420 con 0x561a73c7b400 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.394+0000 7fae3cdfa740 1 -- 172.21.15.26:0/572426921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c7b400 msgr2=0x561a73c7d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.394+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c7b400 0x561a73c7d890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0a9d10 tx=0x7fae2c0a9c10 comp rx=0 tx=0).stop 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 -- 172.21.15.26:0/572426921 shutdown_connections 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c7b400 0x561a73c7d890 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c7ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/572426921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c75cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 -- 172.21.15.26:0/572426921 >> 172.21.15.26:0/572426921 conn(0x561a73c70f10 msgr2=0x561a73c71310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 -- 172.21.15.26:0/572426921 shutdown_connections 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 -- 172.21.15.26:0/572426921 wait complete. 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 -- start start 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.395+0000 7fae3cdfa740 1 Processor -- start 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c83d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c76290 0x561a73c842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c7b400 0x561a73c8b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561a738bc790 con 0x561a73c758a0 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561a73b92890 con 0x561a73c76290 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a73c7e350 con 0x561a73c7b400 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae3cdfa740 10 client.0 _ll_register_callbacks cb 0x561a738eaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c83d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c7b400 0x561a73c8b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c83d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40712/0 (socket says 172.21.15.26:40712) 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae337fe640 1 -- 172.21.15.26:0/2996920703 learned_addr learned my addr 172.21.15.26:0/2996920703 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae32ffd640 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c76290 0x561a73c842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 -- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c76290 msgr2=0x561a73c842b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c76290 0x561a73c842b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 -- 172.21.15.26:0/2996920703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 msgr2=0x561a73c83d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c83d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 -- 172.21.15.26:0/2996920703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a739b7620 con 0x561a73c7b400 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae33fff640 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c7b400 0x561a73c8b2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0a9d10 tx=0x7fae2c0a9290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d0e44d7d380d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.396+0000 7fae12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae12ffd640 1 -- 172.21.15.26:0/2996920703 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c009820 con 0x561a73c7b400 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae12ffd640 1 -- 172.21.15.26:0/2996920703 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fae2c0a6370 con 0x561a73c7b400 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2c0a9890 con 0x561a73c7b400 2024-05-20T21:10:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae12ffd640 1 -- 172.21.15.26:0/2996920703 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae2c0a9d80 con 0x561a73c7b400 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae3cdfa740 10 client.28855 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a73c8e2d0 con 0x561a73c7b400 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae12ffd640 1 -- 172.21.15.26:0/2996920703 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae2c0a9890 con 0x561a73c7b400 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae12ffd640 1 -- 172.21.15.26:0/2996920703 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fae2c091030 con 0x561a73c7b400 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae3cdfa740 10 client.28855 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.397+0000 7fae3cdfa740 10 client.28855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae3cdfa740 20 client.28855 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c8f620 0x561a73c91a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae3cdfa740 4 client.28855 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a73be8b80 con 0x561a73c8f620 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae337fe640 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c8f620 0x561a73c91a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae337fe640 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c8f620 0x561a73c91a10 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3860bfac4e0d6a7e server_cookie=3a3e9723dd6fdb73 in_seq=0 out_seq=0 2024-05-20T21:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.398+0000 7fae12ffd640 10 client.28855 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae12ffd640 1 -- 172.21.15.26:0/2996920703 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561a73be8b80 con 0x561a73c8f620 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae12ffd640 10 client.28855 handle_command_reply: tid=0 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 client.28855 shutdown 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c7b400 msgr2=0x561a73c8b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c7b400 0x561a73c8b2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0a9d10 tx=0x7fae2c0a9290 comp rx=0 tx=0).stop 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 shutdown_connections 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c8f620 0x561a73c91a10 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c7b400 0x561a73c8b2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c76290 0x561a73c842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/2996920703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c758a0 0x561a73c83d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 >> 172.21.15.26:0/2996920703 conn(0x561a73c70f10 msgr2=0x561a73c70110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae12ffd640 0 client.28855 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 shutdown_connections 2024-05-20T21:10:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.400+0000 7fae3cdfa740 1 -- 172.21.15.26:0/2996920703 wait complete. 2024-05-20T21:10:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.401+0000 7fae3cdfa740 20 client.28855 trim_cache size 0 max 16384 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 Processor -- start 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 -- start start 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 0x561a73c73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c758a0 0x561a73c8b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae341aa560 con 0x561a73c70f10 2024-05-20T21:10:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae341a7da0 con 0x561a73c758a0 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae3cdfa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae341a5bf0 con 0x561a73c76290 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 0x561a73c73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 0x561a73c73bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40716/0 (socket says 172.21.15.26:40716) 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 -- 172.21.15.26:0/1776457016 learned_addr learned my addr 172.21.15.26:0/1776457016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae32ffd640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c758a0 0x561a73c8b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae33fff640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 -- 172.21.15.26:0/1776457016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 msgr2=0x561a73c74130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c74130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 -- 172.21.15.26:0/1776457016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c758a0 msgr2=0x561a73c8b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.424+0000 7fae337fe640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c758a0 0x561a73c8b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae337fe640 1 -- 172.21.15.26:0/1776457016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a73b986c0 con 0x561a73c70f10 2024-05-20T21:10:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae33fff640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae32ffd640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c758a0 0x561a73c8b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:10:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae337fe640 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 0x561a73c73bf0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x561a73c7e000 tx=0x7fae24009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f338ad584f1a30b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae197fa640 1 -- 172.21.15.26:0/1776457016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24004230 con 0x561a73c70f10 2024-05-20T21:10:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae197fa640 1 -- 172.21.15.26:0/1776457016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae240043d0 con 0x561a73c70f10 2024-05-20T21:10:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.425+0000 7fae197fa640 1 -- 172.21.15.26:0/1776457016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24004e50 con 0x561a73c70f10 2024-05-20T21:10:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 -- 172.21.15.26:0/1776457016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 msgr2=0x561a73c73bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 0x561a73c73bf0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x561a73c7e000 tx=0x7fae24009260 comp rx=0 tx=0).stop 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 -- 172.21.15.26:0/1776457016 shutdown_connections 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c76290 0x561a73c74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c758a0 0x561a73c8b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/1776457016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c70f10 0x561a73c73bf0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 -- 172.21.15.26:0/1776457016 >> 172.21.15.26:0/1776457016 conn(0x561a73c7aed0 msgr2=0x561a73c712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 -- 172.21.15.26:0/1776457016 shutdown_connections 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 -- 172.21.15.26:0/1776457016 wait complete. 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.426+0000 7fae3cdfa740 1 -- start start 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae3cdfa740 1 Processor -- start 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 0x561a73c88be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c758a0 0x561a73c89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae3cdfa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 0x561a73c8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 0x561a73c88be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 0x561a73c8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.427+0000 7fae337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 0x561a73c88be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:60256/0 (socket says 172.21.15.26:60256) 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 0x561a73c8a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40726/0 (socket says 172.21.15.26:40726) 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae337fe640 1 -- 172.21.15.26:0/921478046 learned_addr learned my addr 172.21.15.26:0/921478046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae341a6260 con 0x561a73c76290 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae341a76d0 con 0x561a73c70f10 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae341a6930 con 0x561a73c758a0 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae32ffd640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c758a0 0x561a73c89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 -- 172.21.15.26:0/921478046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c758a0 msgr2=0x561a73c89120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c758a0 0x561a73c89120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 -- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 msgr2=0x561a73c88be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 0x561a73c88be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a73c8a6d0 con 0x561a73c76290 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae3cdfa740 10 client.0 _ll_register_callbacks cb 0x7fae28035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae240043d0 con 0x561a73c76290 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae337fe640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 0x561a73c88be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae33fff640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 0x561a73c8a0e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0a9a80 tx=0x7fae2c0a5390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ec81d810d0c348 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.428+0000 7fae12ffd640 10 client.19965 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae12ffd640 1 -- 172.21.15.26:0/921478046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0a5c40 con 0x561a73c76290 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae12ffd640 1 -- 172.21.15.26:0/921478046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fae2c0a6370 con 0x561a73c76290 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae12ffd640 1 -- 172.21.15.26:0/921478046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae2c009820 con 0x561a73c76290 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 10 client.19965 fetch_fsmap learned FSMap version 34 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=34}) -- 0x561a73cfa160 con 0x561a73c76290 2024-05-20T21:10:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae12ffd640 1 -- 172.21.15.26:0/921478046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fae2c091040 con 0x561a73c76290 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 10 client.19965 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 10 client.19965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 20 client.19965 populate_metadata read hostname 'smithi026' 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c7b400 0x561a73cfc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 4 client.19965 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a73be2360 con 0x561a73c7b400 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.429+0000 7fae337fe640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c7b400 0x561a73cfc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.430+0000 7fae12ffd640 10 client.19965 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.430+0000 7fae337fe640 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c7b400 0x561a73cfc880 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af48d035ed9a9299 server_cookie=72f1138916b07ebf in_seq=0 out_seq=0 2024-05-20T21:10:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae12ffd640 1 -- 172.21.15.26:0/921478046 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a73be2360 con 0x561a73c7b400 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae12ffd640 10 client.19965 handle_command_reply: tid=0 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 client.19965 shutdown 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 msgr2=0x561a73c8a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 0x561a73c8a0e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0a9a80 tx=0x7fae2c0a5390 comp rx=0 tx=0).stop 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 shutdown_connections 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x561a73c7b400 0x561a73cfc880 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae12ffd640 0 client.19965 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a73c76290 0x561a73c8a0e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a73c758a0 0x561a73c89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 --2- 172.21.15.26:0/921478046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561a73c70f10 0x561a73c88be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 >> 172.21.15.26:0/921478046 conn(0x561a73c7aed0 msgr2=0x561a73c73bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 shutdown_connections 2024-05-20T21:10:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.568+0000 7fae3cdfa740 1 -- 172.21.15.26:0/921478046 wait complete. 2024-05-20T21:10:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae3cdfa740 20 client.19965 trim_cache size 0 max 16384 2024-05-20T21:10:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:10:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:10:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae1c0b2860 msgr2=0x7fae1c0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae1c0b2860 0x7fae1c0b4d20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fae28005b60 tx=0x7fae28005c90 comp rx=0 tx=0).stop 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341044a0 msgr2=0x7fae341a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341044a0 0x7fae341a0030 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00ec50 tx=0x7fae2c00c6a0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 shutdown_connections 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fae1c0b2860 0x7fae1c0b4d20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae341044a0 0x7fae341a0030 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae34103ae0 0x7fae34199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 --2- 172.21.15.26:0/4031694516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae341028f0 0x7fae34198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:10:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.569+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 >> 172.21.15.26:0/4031694516 conn(0x7fae340fde60 msgr2=0x7fae34106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:10:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.570+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 shutdown_connections 2024-05-20T21:10:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:10:54.570+0000 7fae38d9e640 1 -- 172.21.15.26:0/4031694516 wait complete. 2024-05-20T21:10:55.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9802 11.66 MB/sec warmup 60 sec latency 169.785 ms 2024-05-20T21:10:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:55 smithi055 ceph-mon[35213]: pgmap v833: 97 pgs: 97 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-05-20T21:10:55.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:55 smithi026 ceph-mon[31990]: pgmap v833: 97 pgs: 97 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-05-20T21:10:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:55 smithi144 ceph-mon[35660]: pgmap v833: 97 pgs: 97 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-05-20T21:10:56.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9937 11.60 MB/sec warmup 61 sec latency 193.782 ms 2024-05-20T21:10:57.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10067 11.47 MB/sec warmup 62 sec latency 156.282 ms 2024-05-20T21:10:57.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:57 smithi055 ceph-mon[35213]: pgmap v834: 97 pgs: 97 active+clean; 240 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-05-20T21:10:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:57 smithi026 ceph-mon[31990]: pgmap v834: 97 pgs: 97 active+clean; 240 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-05-20T21:10:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:57 smithi144 ceph-mon[35660]: pgmap v834: 97 pgs: 97 active+clean; 240 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-05-20T21:10:58.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10193 11.33 MB/sec warmup 63 sec latency 172.667 ms 2024-05-20T21:10:59.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10345 11.39 MB/sec warmup 64 sec latency 162.159 ms 2024-05-20T21:10:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:10:59 smithi055 ceph-mon[35213]: pgmap v835: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-20T21:10:59.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:10:59 smithi026 ceph-mon[31990]: pgmap v835: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-20T21:10:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:10:59 smithi144 ceph-mon[35660]: pgmap v835: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-20T21:11:00.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10499 11.35 MB/sec warmup 65 sec latency 151.699 ms 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.928+0000 7fd579e7a700 1 -- 172.21.15.26:0/2579561307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd574106140 msgr2=0x7fd574112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.928+0000 7fd579e7a700 1 --2- 172.21.15.26:0/2579561307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd574106140 0x7fd574112650 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00b0a0 tx=0x7fd56c00b3b0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 -- 172.21.15.26:0/2579561307 shutdown_connections 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 --2- 172.21.15.26:0/2579561307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd574106140 0x7fd574112650 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 --2- 172.21.15.26:0/2579561307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd57410b780 0x7fd57410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 --2- 172.21.15.26:0/2579561307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 -- 172.21.15.26:0/2579561307 >> 172.21.15.26:0/2579561307 conn(0x7fd574101270 msgr2=0x7fd574103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:00.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 -- 172.21.15.26:0/2579561307 shutdown_connections 2024-05-20T21:11:00.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.929+0000 7fd579e7a700 1 -- 172.21.15.26:0/2579561307 wait complete. 2024-05-20T21:11:00.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.930+0000 7fd579e7a700 1 Processor -- start 2024-05-20T21:11:00.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.930+0000 7fd579e7a700 1 -- start start 2024-05-20T21:11:00.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd579e7a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd579e7a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 0x7fd574075060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd579e7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd57410b780 0x7fd57406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd579e7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd574115810 con 0x7fd57410b780 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd579e7a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd574115690 con 0x7fd574106140 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd579e7a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd574115990 con 0x7fd574105780 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd573fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd57410b780 0x7fd57406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 0x7fd574075060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41516/0 (socket says 172.21.15.26:41516) 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 -- 172.21.15.26:0/351105390 learned_addr learned my addr 172.21.15.26:0/351105390 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd572ffd700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 0x7fd574075060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49710/0 (socket says 172.21.15.26:49710) 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 -- 172.21.15.26:0/351105390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 msgr2=0x7fd574075060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 0x7fd574075060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 -- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd57410b780 msgr2=0x7fd57406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd57410b780 0x7fd57406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 -- 172.21.15.26:0/351105390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57406e590 con 0x7fd574105780 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd573fff700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd57410b780 0x7fd57406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd572ffd700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 0x7fd574075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.931+0000 7fd5737fe700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574074b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd56400e9a0 tx=0x7fd56400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.932+0000 7fd570ff9700 1 -- 172.21.15.26:0/351105390 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56400ca10 con 0x7fd574105780 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.932+0000 7fd570ff9700 1 -- 172.21.15.26:0/351105390 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd56400cbb0 con 0x7fd574105780 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.932+0000 7fd570ff9700 1 -- 172.21.15.26:0/351105390 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564018680 con 0x7fd574105780 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.932+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57406e820 con 0x7fd574105780 2024-05-20T21:11:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.932+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd574072650 con 0x7fd574105780 2024-05-20T21:11:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.935+0000 7fd570ff9700 1 -- 172.21.15.26:0/351105390 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd564010430 con 0x7fd574105780 2024-05-20T21:11:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.936+0000 7fd570ff9700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd55c0b2900 0x7fd55c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd572ffd700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd55c0b2900 0x7fd55c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd570ff9700 1 -- 172.21.15.26:0/351105390 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd5640e2410 con 0x7fd574105780 2024-05-20T21:11:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 Processor -- start 2024-05-20T21:11:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 -- start start 2024-05-20T21:11:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf32700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf2cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56054be8ad70 con 0x56054bf2d260 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56054be7a910 con 0x56054bf32320 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.937+0000 7fd584710b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56054be7e950 con 0x56054bf2c8a0 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf32700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd573fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf32700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49714/0 (socket says 172.21.15.26:49714) 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 -- 172.21.15.26:0/3636094352 learned_addr learned my addr 172.21.15.26:0/3636094352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 -- 172.21.15.26:0/3636094352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 msgr2=0x56054bf2cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd572ffd700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd55c0b2900 0x7fd55c0b4dc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd57406f370 tx=0x7fd568009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf2cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 -- 172.21.15.26:0/3636094352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 msgr2=0x56054bf39080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf39080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 -- 172.21.15.26:0/3636094352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56054be23fb0 con 0x56054bf32320 2024-05-20T21:11:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd5737fe700 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf32700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd5640a59f0 tx=0x7fd5640a5180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb3e88a7a00dc2d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.938+0000 7fd559ffb700 1 -- 172.21.15.26:0/3636094352 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640b2120 con 0x56054bf32320 2024-05-20T21:11:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.939+0000 7fd559ffb700 1 -- 172.21.15.26:0/3636094352 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5640aa070 con 0x56054bf32320 2024-05-20T21:11:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.939+0000 7fd559ffb700 1 -- 172.21.15.26:0/3636094352 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640b2340 con 0x56054bf32320 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.939+0000 7fd584710b80 1 -- 172.21.15.26:0/3636094352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 msgr2=0x56054bf32700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.939+0000 7fd584710b80 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf32700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd5640a59f0 tx=0x7fd5640a5180 comp rx=0 tx=0).stop 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 -- 172.21.15.26:0/3636094352 shutdown_connections 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf39080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf32700 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 --2- 172.21.15.26:0/3636094352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf2cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 -- 172.21.15.26:0/3636094352 >> 172.21.15.26:0/3636094352 conn(0x56054be58240 msgr2=0x56054bf273e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 -- 172.21.15.26:0/3636094352 shutdown_connections 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 -- 172.21.15.26:0/3636094352 wait complete. 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 -- start start 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.940+0000 7fd584710b80 1 Processor -- start 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf38dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56054bd80fa0 con 0x56054bf2d260 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56054bde2da0 con 0x56054bf32320 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56054bf396e0 con 0x56054bf2c8a0 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf38dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd573fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53326/0 (socket says 172.21.15.26:53326) 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd572ffd700 1 -- 172.21.15.26:0/774717196 learned_addr learned my addr 172.21.15.26:0/774717196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd584710b80 10 client.0 _ll_register_callbacks cb 0x56054be15950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd5737fe700 1 -- 172.21.15.26:0/774717196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 msgr2=0x56054bf36600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd5737fe700 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf36600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd5737fe700 1 -- 172.21.15.26:0/774717196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 msgr2=0x56054bf360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd5737fe700 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.941+0000 7fd5737fe700 1 -- 172.21.15.26:0/774717196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56054bdca8d0 con 0x56054bf2c8a0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd5737fe700 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf38dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd564022000 tx=0x7fd5640a9760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbcc3c23cbd3a2d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd552ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd552ffd700 1 -- 172.21.15.26:0/774717196 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640c2070 con 0x56054bf2c8a0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd552ffd700 1 -- 172.21.15.26:0/774717196 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd56406f1f0 con 0x56054bf2c8a0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5640aa070 con 0x56054bf2c8a0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd552ffd700 1 -- 172.21.15.26:0/774717196 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd5640b2120 con 0x56054bf2c8a0 2024-05-20T21:11:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd584710b80 10 client.28870 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56054be34760 con 0x56054bf2c8a0 2024-05-20T21:11:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.942+0000 7fd552ffd700 1 -- 172.21.15.26:0/774717196 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5640cc2c0 con 0x56054bf2c8a0 2024-05-20T21:11:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd552ffd700 1 -- 172.21.15.26:0/774717196 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd5640a52e0 con 0x56054bf2c8a0 2024-05-20T21:11:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd584710b80 10 client.28870 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd584710b80 10 client.28870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd584710b80 20 client.28870 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd584710b80 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bf46130 0x56054bf48520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd584710b80 4 client.28870 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd5640aa070 con 0x56054bf46130 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd572ffd700 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bf46130 0x56054bf48520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd552ffd700 10 client.28870 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.943+0000 7fd572ffd700 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bf46130 0x56054bf48520 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5014a195b4a78e74 server_cookie=1eb4c1585f308e3b in_seq=0 out_seq=0 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.944+0000 7fd552ffd700 1 -- 172.21.15.26:0/774717196 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fd5640aa070 con 0x56054bf46130 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.944+0000 7fd552ffd700 10 client.28870 handle_command_reply: tid=0 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.944+0000 7fd584710b80 1 client.28870 shutdown 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.944+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 msgr2=0x56054bf38dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.944+0000 7fd584710b80 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf38dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd564022000 tx=0x7fd5640a9760 comp rx=0 tx=0).stop 2024-05-20T21:11:00.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 shutdown_connections 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bf46130 0x56054bf48520 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf36600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bf38dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 --2- 172.21.15.26:0/774717196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2d260 0x56054bf360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 >> 172.21.15.26:0/774717196 conn(0x56054be58240 msgr2=0x56054bf26af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd552ffd700 0 client.28870 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 shutdown_connections 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 1 -- 172.21.15.26:0/774717196 wait complete. 2024-05-20T21:11:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.945+0000 7fd584710b80 20 client.28870 trim_cache size 0 max 16384 2024-05-20T21:11:00.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 Processor -- start 2024-05-20T21:11:00.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 -- start start 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2c8a0 0x56054bf27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 0x56054bf38000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741b6a10 con 0x56054bf2c8a0 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741b4210 con 0x56054bf32320 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd584710b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741b2060 con 0x56054bf2d260 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 0x56054bf38000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 0x56054bf38000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41540/0 (socket says 172.21.15.26:41540) 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd572ffd700 1 -- 172.21.15.26:0/4112727965 learned_addr learned my addr 172.21.15.26:0/4112727965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.973+0000 7fd5737fe700 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2c8a0 0x56054bf27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd573fff700 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd572ffd700 1 -- 172.21.15.26:0/4112727965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 msgr2=0x56054bf2fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd572ffd700 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf2fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd572ffd700 1 -- 172.21.15.26:0/4112727965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2c8a0 msgr2=0x56054bf27270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd572ffd700 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2c8a0 0x56054bf27270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd572ffd700 1 -- 172.21.15.26:0/4112727965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56054bf31f40 con 0x56054bf2d260 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd5737fe700 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2c8a0 0x56054bf27270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd572ffd700 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 0x56054bf38000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5680095c0 tx=0x7fd568014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf0e8cbc4c6af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd5597fa700 1 -- 172.21.15.26:0/4112727965 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568003ce0 con 0x56054bf2d260 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd5597fa700 1 -- 172.21.15.26:0/4112727965 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd56802d070 con 0x56054bf2d260 2024-05-20T21:11:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.974+0000 7fd5597fa700 1 -- 172.21.15.26:0/4112727965 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56801e650 con 0x56054bf2d260 2024-05-20T21:11:00.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 -- 172.21.15.26:0/4112727965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 msgr2=0x56054bf38000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 0x56054bf38000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5680095c0 tx=0x7fd568014000 comp rx=0 tx=0).stop 2024-05-20T21:11:00.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 -- 172.21.15.26:0/4112727965 shutdown_connections 2024-05-20T21:11:00.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf2c8a0 0x56054bf27270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf32320 0x56054bf2fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 --2- 172.21.15.26:0/4112727965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2d260 0x56054bf38000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.975+0000 7fd584710b80 1 -- 172.21.15.26:0/4112727965 >> 172.21.15.26:0/4112727965 conn(0x56054be58240 msgr2=0x56054bf378f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.976+0000 7fd584710b80 1 -- 172.21.15.26:0/4112727965 shutdown_connections 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.976+0000 7fd584710b80 1 -- 172.21.15.26:0/4112727965 wait complete. 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.976+0000 7fd584710b80 1 -- start start 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.976+0000 7fd584710b80 1 Processor -- start 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.976+0000 7fd584710b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bfa8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd584710b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf2d260 0x56054bfa92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd584710b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf32320 0x56054bf48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd584710b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741b26d0 con 0x56054bf32320 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd584710b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741b3b40 con 0x56054bf2d260 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd584710b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741b2da0 con 0x56054bf2c8a0 2024-05-20T21:11:00.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd584710b80 10 client.0 _ll_register_callbacks cb 0x56054be15950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf2d260 0x56054bfa92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf2d260 0x56054bfa92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49734/0 (socket says 172.21.15.26:49734) 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd572ffd700 1 -- 172.21.15.26:0/883586278 learned_addr learned my addr 172.21.15.26:0/883586278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd5737fe700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bfa8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf32320 0x56054bf48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 -- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 msgr2=0x56054bfa8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bfa8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 -- 172.21.15.26:0/883586278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf2d260 msgr2=0x56054bfa92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf2d260 0x56054bfa92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 -- 172.21.15.26:0/883586278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56054bf48f00 con 0x56054bf32320 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd573fff700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf32320 0x56054bf48800 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd574075740 tx=0x7fd56c007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0cfdce21a31c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.977+0000 7fd552ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd552ffd700 1 -- 172.21.15.26:0/883586278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c007c90 con 0x56054bf32320 2024-05-20T21:11:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd552ffd700 1 -- 172.21.15.26:0/883586278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd56c025450 con 0x56054bf32320 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd56802d070 con 0x56054bf32320 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd552ffd700 1 -- 172.21.15.26:0/883586278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd56c025690 con 0x56054bf32320 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd584710b80 10 client.19983 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56054bfb0550 con 0x56054bf32320 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.978+0000 7fd552ffd700 1 -- 172.21.15.26:0/883586278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd56c025cc0 con 0x56054bf32320 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd552ffd700 1 -- 172.21.15.26:0/883586278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd56c02d030 con 0x56054bf32320 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd584710b80 10 client.19983 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd584710b80 10 client.19983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd584710b80 20 client.19983 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd584710b80 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bfb0ce0 0x56054bfb3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd584710b80 4 client.19983 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:11:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 --> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd56802d070 con 0x56054bfb0ce0 2024-05-20T21:11:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.979+0000 7fd5737fe700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bfb0ce0 0x56054bfb3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd552ffd700 10 client.19983 ms_handle_connect on v2:172.21.15.144:6838/2122637820 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd5737fe700 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bfb0ce0 0x56054bfb3100 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcef7875e7e0a739 server_cookie=66f2515441ab301f in_seq=0 out_seq=0 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd552ffd700 1 -- 172.21.15.26:0/883586278 <== mds.2 v2:172.21.15.144:6838/2122637820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd56802d070 con 0x56054bfb0ce0 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd552ffd700 10 client.19983 handle_command_reply: tid=0 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 client.19983 shutdown 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf32320 msgr2=0x56054bf48800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf32320 0x56054bf48800 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd574075740 tx=0x7fd56c007710 comp rx=0 tx=0).stop 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 shutdown_connections 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:6838/2122637820,v1:172.21.15.144:6839/2122637820] conn(0x56054bfb0ce0 0x56054bfb3100 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd552ffd700 0 client.19983 ms_handle_reset on v2:172.21.15.144:6838/2122637820 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56054bf32320 0x56054bf48800 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56054bf2c8a0 0x56054bfa8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 --2- 172.21.15.26:0/883586278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56054bf2d260 0x56054bfa92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.980+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 >> 172.21.15.26:0/883586278 conn(0x56054be58240 msgr2=0x56054bf2c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 shutdown_connections 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd584710b80 1 -- 172.21.15.26:0/883586278 wait complete. 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd584710b80 20 client.19983 trim_cache size 0 max 16384 2024-05-20T21:11:00.985 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd55c0b2900 msgr2=0x7fd55c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd579e7a700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd55c0b2900 0x7fd55c0b4dc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd57406f370 tx=0x7fd568009260 comp rx=0 tx=0).stop 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 msgr2=0x7fd574074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.981+0000 7fd579e7a700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574074b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd56400e9a0 tx=0x7fd56400ecb0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 shutdown_connections 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd55c0b2900 0x7fd55c0b4dc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd57410b780 0x7fd57406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574105780 0x7fd574074b20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 --2- 172.21.15.26:0/351105390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd574106140 0x7fd574075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 >> 172.21.15.26:0/351105390 conn(0x7fd574101270 msgr2=0x7fd574101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 shutdown_connections 2024-05-20T21:11:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:00.982+0000 7fd579e7a700 1 -- 172.21.15.26:0/351105390 wait complete. 2024-05-20T21:11:01.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10656 11.26 MB/sec warmup 66 sec latency 97.971 ms 2024-05-20T21:11:01.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:01 smithi055 ceph-mon[35213]: pgmap v836: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-05-20T21:11:01.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:01 smithi026 ceph-mon[31990]: pgmap v836: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-05-20T21:11:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:01 smithi144 ceph-mon[35660]: pgmap v836: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-05-20T21:11:01.918 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-20T21:11:02.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10824 11.14 MB/sec warmup 67 sec latency 80.561 ms 2024-05-20T21:11:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:03.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11018 11.11 MB/sec warmup 68 sec latency 81.409 ms 2024-05-20T21:11:03.418 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:03 smithi026 ceph-mon[31990]: pgmap v837: 97 pgs: 97 active+clean; 230 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.4 MiB/s wr, 277 op/s 2024-05-20T21:11:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:03 smithi055 ceph-mon[35213]: pgmap v837: 97 pgs: 97 active+clean; 230 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.4 MiB/s wr, 277 op/s 2024-05-20T21:11:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:03 smithi144 ceph-mon[35660]: pgmap v837: 97 pgs: 97 active+clean; 230 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.4 MiB/s wr, 277 op/s 2024-05-20T21:11:04.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11204 11.04 MB/sec warmup 69 sec latency 122.664 ms 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/1438073691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f8109dd0 msgr2=0x7fd6f810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/1438073691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f8109dd0 0x7fd6f810a1b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00099c0 tx=0x7fd6f0009cd0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/1438073691 shutdown_connections 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/1438073691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f8109dd0 0x7fd6f810a1b0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/1438073691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f810eeb0 0x7fd6f8116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/1438073691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/1438073691 >> 172.21.15.26:0/1438073691 conn(0x7fd6f8076d10 msgr2=0x7fd6f8108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/1438073691 shutdown_connections 2024-05-20T21:11:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.255+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/1438073691 wait complete. 2024-05-20T21:11:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.256+0000 7fd6ff5a0700 1 Processor -- start 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.256+0000 7fd6ff5a0700 1 -- start start 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6ff5a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f8109dd0 0x7fd6f81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6ff5a0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f81a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6ff5a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f810eeb0 0x7fd6f81a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6ff5a0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8119c10 con 0x7fd6f810eeb0 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6ff5a0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8119a90 con 0x7fd6f810a780 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6ff5a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8119d90 con 0x7fd6f8109dd0 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f8109dd0 0x7fd6f81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fdb3d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f810eeb0 0x7fd6f81a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f81a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f81a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49748/0 (socket says 172.21.15.26:49748) 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fdb3d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f810eeb0 0x7fd6f81a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53354/0 (socket says 172.21.15.26:53354) 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f8109dd0 0x7fd6f81a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41572/0 (socket says 172.21.15.26:41572) 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/631737782 learned_addr learned my addr 172.21.15.26:0/631737782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/631737782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f8109dd0 msgr2=0x7fd6f81a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f8109dd0 0x7fd6f81a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f810eeb0 msgr2=0x7fd6f81a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f810eeb0 0x7fd6f81a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.257+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/631737782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f81a51f0 con 0x7fd6f810a780 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.258+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f81a0720 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00efc0 tx=0x7fd6ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.258+0000 7fd6ea7fc700 1 -- 172.21.15.26:0/631737782 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00c9c0 con 0x7fd6f810a780 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.258+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f81a5480 con 0x7fd6f810a780 2024-05-20T21:11:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.258+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f81acd60 con 0x7fd6f810a780 2024-05-20T21:11:04.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.258+0000 7fd6ea7fc700 1 -- 172.21.15.26:0/631737782 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6ec019070 con 0x7fd6f810a780 2024-05-20T21:11:04.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.258+0000 7fd6ea7fc700 1 -- 172.21.15.26:0/631737782 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec014480 con 0x7fd6f810a780 2024-05-20T21:11:04.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.261+0000 7fd6ea7fc700 1 -- 172.21.15.26:0/631737782 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd6ec012050 con 0x7fd6f810a780 2024-05-20T21:11:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.262+0000 7fd6ea7fc700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6e40b2900 0x7fd6e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.262+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6e40b2900 0x7fd6e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.262+0000 7fd6ea7fc700 1 -- 172.21.15.26:0/631737782 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd6ec0e2050 con 0x7fd6f810a780 2024-05-20T21:11:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.263+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6e40b2900 0x7fd6e40b4dc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0000c00 tx=0x7fd6f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.263+0000 7fd709e36b80 1 Processor -- start 2024-05-20T21:11:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 -- start start 2024-05-20T21:11:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b763a320 0x55d3b763a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b76348a0 0x55d3b7634d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7641080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d3b7592d70 con 0x55d3b7635260 2024-05-20T21:11:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d3b7582910 con 0x55d3b763a320 2024-05-20T21:11:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd709e36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3b7586950 con 0x55d3b76348a0 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd6fdb3d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7641080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd6fdb3d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7641080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53362/0 (socket says 172.21.15.26:53362) 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd6fcb3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b76348a0 0x55d3b7634d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd6fdb3d700 1 -- 172.21.15.26:0/2401098971 learned_addr learned my addr 172.21.15.26:0/2401098971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b763a320 0x55d3b763a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.264+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/2401098971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b763a320 msgr2=0x55d3b763a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b763a320 0x55d3b763a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/2401098971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 msgr2=0x55d3b7641080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7641080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/2401098971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b752bfb0 con 0x55d3b76348a0 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6fdb3d700 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7641080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b76348a0 0x55d3b7634d20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0dfb40 tx=0x7fd6ec0a9510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=829e9235e6a9f57d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:04.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6e37fe700 1 -- 172.21.15.26:0/2401098971 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0a9d10 con 0x55d3b76348a0 2024-05-20T21:11:04.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6e37fe700 1 -- 172.21.15.26:0/2401098971 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6ec0a9610 con 0x55d3b76348a0 2024-05-20T21:11:04.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.265+0000 7fd6e37fe700 1 -- 172.21.15.26:0/2401098971 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00cb60 con 0x55d3b76348a0 2024-05-20T21:11:04.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 -- 172.21.15.26:0/2401098971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b76348a0 msgr2=0x55d3b7634d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b76348a0 0x55d3b7634d20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0dfb40 tx=0x7fd6ec0a9510 comp rx=0 tx=0).stop 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 -- 172.21.15.26:0/2401098971 shutdown_connections 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7641080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b763a320 0x55d3b763a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 --2- 172.21.15.26:0/2401098971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b76348a0 0x55d3b7634d20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 -- 172.21.15.26:0/2401098971 >> 172.21.15.26:0/2401098971 conn(0x55d3b7560240 msgr2=0x55d3b762f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 -- 172.21.15.26:0/2401098971 shutdown_connections 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.266+0000 7fd709e36b80 1 -- 172.21.15.26:0/2401098971 wait complete. 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.267+0000 7fd709e36b80 1 -- start start 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.267+0000 7fd709e36b80 1 Processor -- start 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b7640dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b763e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b7640dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b763e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b7640dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53364/0 (socket says 172.21.15.26:53364) 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 -- 172.21.15.26:0/580975751 learned_addr learned my addr 172.21.15.26:0/580975751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d3b7488fa0 con 0x55d3b76348a0 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d3b74eada0 con 0x55d3b7635260 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3b76416e0 con 0x55d3b763a320 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b763e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fdb3d700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b763e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd709e36b80 10 client.0 _ll_register_callbacks cb 0x55d3b751d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 -- 172.21.15.26:0/580975751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 msgr2=0x55d3b763e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b763e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 -- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 msgr2=0x55d3b763e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b763e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.268+0000 7fd6fd33c700 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b74d28d0 con 0x55d3b76348a0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd6fdb3d700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b763e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b7640dc0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000b580 tx=0x7fd6f0005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6828ebde8c88bce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd6e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/580975751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0035060 con 0x55d3b76348a0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6ec0a9610 con 0x55d3b76348a0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/580975751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd6f000b9a0 con 0x55d3b76348a0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/580975751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd6f000bba0 con 0x55d3b76348a0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd709e36b80 10 client.19995 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.269+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3b753c760 con 0x55d3b76348a0 2024-05-20T21:11:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.270+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/580975751 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6ec0a9610 con 0x55d3b76348a0 2024-05-20T21:11:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.270+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/580975751 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd6f0023460 con 0x55d3b76348a0 2024-05-20T21:11:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.270+0000 7fd709e36b80 10 client.19995 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.270+0000 7fd709e36b80 10 client.19995 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:11:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.270+0000 7fd709e36b80 20 client.19995 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.271+0000 7fd709e36b80 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b764e040 0x55d3b7650480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.271+0000 7fd709e36b80 4 client.19995 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.271+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3b7594cc0 con 0x55d3b764e040 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.271+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b764e040 0x55d3b7650480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.271+0000 7fd6e0ff9700 10 client.19995 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.271+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b764e040 0x55d3b7650480 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a567532daf85d01 server_cookie=e03c77f8c2e28f7a in_seq=0 out_seq=0 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.272+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/580975751 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d3b7594cc0 con 0x55d3b764e040 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.272+0000 7fd6e0ff9700 10 client.19995 handle_command_reply: tid=0 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.272+0000 7fd709e36b80 1 client.19995 shutdown 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 msgr2=0x55d3b7640dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b7640dc0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000b580 tx=0x7fd6f0005050 comp rx=0 tx=0).stop 2024-05-20T21:11:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 shutdown_connections 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b764e040 0x55d3b7650480 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b7640dc0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b763e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 --2- 172.21.15.26:0/580975751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b763e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 >> 172.21.15.26:0/580975751 conn(0x55d3b7560240 msgr2=0x55d3b762eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd6e0ff9700 0 client.19995 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 shutdown_connections 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 1 -- 172.21.15.26:0/580975751 wait complete. 2024-05-20T21:11:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.273+0000 7fd709e36b80 20 client.19995 trim_cache size 0 max 16384 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.298+0000 7fd709e36b80 1 Processor -- start 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.298+0000 7fd709e36b80 1 -- start start 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b764a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b762f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7633fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd709e36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc00fe70 con 0x55d3b76348a0 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b764a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd709e36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc00d670 con 0x55d3b7635260 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd709e36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc00b4c0 con 0x55d3b763a320 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b764a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53376/0 (socket says 172.21.15.26:53376) 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd6fd33c700 1 -- 172.21.15.26:0/3648547254 learned_addr learned my addr 172.21.15.26:0/3648547254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.299+0000 7fd6fdb3d700 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7633fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6fd33c700 1 -- 172.21.15.26:0/3648547254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 msgr2=0x55d3b7633fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7633fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6fd33c700 1 -- 172.21.15.26:0/3648547254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 msgr2=0x55d3b762f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b762f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6fd33c700 1 -- 172.21.15.26:0/3648547254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b7639f40 con 0x55d3b76348a0 2024-05-20T21:11:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b764a770 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0023850 tx=0x7fd6f0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ea6953aba8e429 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6e2ffd700 1 -- 172.21.15.26:0/3648547254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0035040 con 0x55d3b76348a0 2024-05-20T21:11:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6e2ffd700 1 -- 172.21.15.26:0/3648547254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6f000b9a0 con 0x55d3b76348a0 2024-05-20T21:11:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.300+0000 7fd6e2ffd700 1 -- 172.21.15.26:0/3648547254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0023c90 con 0x55d3b76348a0 2024-05-20T21:11:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 -- 172.21.15.26:0/3648547254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 msgr2=0x55d3b764a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b764a770 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0023850 tx=0x7fd6f0009310 comp rx=0 tx=0).stop 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 -- 172.21.15.26:0/3648547254 shutdown_connections 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b76348a0 0x55d3b764a770 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7633fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 --2- 172.21.15.26:0/3648547254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b7635260 0x55d3b762f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 -- 172.21.15.26:0/3648547254 >> 172.21.15.26:0/3648547254 conn(0x55d3b7560240 msgr2=0x55d3b7633890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.301+0000 7fd709e36b80 1 -- 172.21.15.26:0/3648547254 shutdown_connections 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.302+0000 7fd709e36b80 1 -- 172.21.15.26:0/3648547254 wait complete. 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.302+0000 7fd709e36b80 1 -- start start 2024-05-20T21:11:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.302+0000 7fd709e36b80 1 Processor -- start 2024-05-20T21:11:04.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b76348a0 0x55d3b7647090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7644430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7644990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc00bb30 con 0x55d3b7635260 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b76348a0 0x55d3b7647090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc00cfa0 con 0x55d3b76348a0 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc00c200 con 0x55d3b763a320 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fd33c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b76348a0 0x55d3b7647090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49782/0 (socket says 172.21.15.26:49782) 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fd33c700 1 -- 172.21.15.26:0/1865823572 learned_addr learned my addr 172.21.15.26:0/1865823572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd709e36b80 10 client.0 _ll_register_callbacks cb 0x55d3b751d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7644430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fdb3d700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7644990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/1865823572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 msgr2=0x55d3b7644990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7644990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b76348a0 msgr2=0x55d3b7647090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b76348a0 0x55d3b7647090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.303+0000 7fd6fcb3b700 1 -- 172.21.15.26:0/1865823572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b76451d0 con 0x55d3b7635260 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6fcb3b700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7644430 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0090a0 tx=0x7fd6ec0a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533a37b5595756ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/1865823572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0a9d10 con 0x55d3b7635260 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/1865823572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd6ec0b21f0 con 0x55d3b7635260 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6f000b9a0 con 0x55d3b7635260 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/1865823572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd6ec0a9610 con 0x55d3b7635260 2024-05-20T21:11:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 10 client.20001 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3b76b8cc0 con 0x55d3b7635260 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/1865823572 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6ec0ad9c0 con 0x55d3b7635260 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/1865823572 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd6ec090b20 con 0x55d3b7635260 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 10 client.20001 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 10 client.20001 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 20 client.20001 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b76b8fb0 0x55d3b76bb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 4 client.20001 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.304+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 --> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd6f000b9a0 con 0x55d3b76b8fb0 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.305+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b76b8fb0 0x55d3b76bb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.305+0000 7fd6e0ff9700 10 client.20001 ms_handle_connect on v2:172.21.15.55:6836/353197532 2024-05-20T21:11:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.305+0000 7fd6fd33c700 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b76b8fb0 0x55d3b76bb3a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5f5400cf8ade22e server_cookie=63e17d8bc26e5b85 in_seq=0 out_seq=0 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd6e0ff9700 1 -- 172.21.15.26:0/1865823572 <== mds.1 v2:172.21.15.55:6836/353197532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+19562 (crc 0 0 0) 0x7fd6f000b9a0 con 0x55d3b76b8fb0 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd6e0ff9700 10 client.20001 handle_command_reply: tid=0 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 client.20001 shutdown 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 msgr2=0x55d3b7644430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7644430 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0090a0 tx=0x7fd6ec0a5600 comp rx=0 tx=0).stop 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 shutdown_connections 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:6836/353197532,v1:172.21.15.55:6837/353197532] conn(0x55d3b76b8fb0 0x55d3b76bb3a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd6e0ff9700 0 client.20001 ms_handle_reset on v2:172.21.15.55:6836/353197532 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b763a320 0x55d3b7644990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d3b76348a0 0x55d3b7647090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 --2- 172.21.15.26:0/1865823572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3b7635260 0x55d3b7644430 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 >> 172.21.15.26:0/1865823572 conn(0x55d3b7560240 msgr2=0x55d3b7634120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 shutdown_connections 2024-05-20T21:11:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.307+0000 7fd709e36b80 1 -- 172.21.15.26:0/1865823572 wait complete. 2024-05-20T21:11:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.308+0000 7fd709e36b80 20 client.20001 trim_cache size 0 max 16384 2024-05-20T21:11:04.311 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-05-20T21:11:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 44218144, 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639332, 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC1.CDR" 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 47735812, 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639569, 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX" 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 54991942, 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639432, 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 103788678, 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639453, 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/EVENTS.DOC" 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 197521273, 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639641, 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/MEDIUM.FIL" 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 231725301, 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639742, 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWTIPS.PPT" 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 254790455, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639527, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0" 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 323163291, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639562, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB" 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 328769020, 2024-05-20T21:11:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639579, 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.Y03" 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 379511940, 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639582, 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03" 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 569762323, 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639164, 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD" 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 579737019, 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639460, 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/MOVED.DOC" 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 627655524, 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639516, 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG0" 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 629341141, 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639603, 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL" 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 647160875, 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639537, 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG1" 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 677553704, 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639698, 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT" 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 704466652, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639474, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.DB" 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 717518892, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639572, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.X03" 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 738158466, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639447, 2024-05-20T21:11:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/T1.XLS" 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 979430620, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639523, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG0" 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1003791986, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639507, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB" 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1018632631, 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639503, 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.DB" 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1065588826, 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639401, 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CORELDRW.CDT" 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1266876651, 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639231, 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT" 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1268775360, 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639362, 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/SIGN2.CPT" 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1280457131, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639182, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/BASEMACH.DOC" 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1353521433, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639290, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL" 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1358769960, 2024-05-20T21:11:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639201, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL" 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1371692659, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639534, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1" 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1405098003, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639615, 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS" 2024-05-20T21:11:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1660412617, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639610, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL" 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1787424025, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639606, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.FSL" 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1801687754, 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639509, 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.PX" 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1833936579, 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639544, 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.VAL" 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1860300750, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639411, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO" 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1932682183, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639391, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ROSE.CPT" 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2001491895, 2024-05-20T21:11:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639586, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.VAL" 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2051753661, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639192, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/FACTS.DOC" 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2124062772, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639300, 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRROLS.CFG" 2024-05-20T21:11:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2126925419, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639530, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG1" 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2231445796, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 2199023256059, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2292677282, 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639630, 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED" 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2348845759, 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639589, 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL" 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2349090964, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639489, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.X04" 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2439870346, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639551, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.MB" 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2527792425, 2024-05-20T21:11:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639487, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX" 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2528478409, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639500, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04" 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2529502806, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639440, 2024-05-20T21:11:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM" 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2541486423, 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639240, 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2699893321, 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639514, 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX" 2024-05-20T21:11:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2727409312, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639260, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2796191925, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639599, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.FSL" 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2861094881, 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639270, 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TIPS.PPT" 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2876337672, 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639322, 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ARTISTIC.ACL" 2024-05-20T21:11:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2912501947, 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639496, 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.Y04" 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2953028741, 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639480, 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB" 2024-05-20T21:11:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2957513845, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639173, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/CHAP10.DOC" 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3087067414, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639646, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/LARGE.FIL" 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3125608254, 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639221, 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/SALES.XLS" 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3159934142, 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639371, 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TRAIN.CDR" 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3190875669, 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639210, 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3239728539, 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639521, 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0" 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3254754513, 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639592, 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.TV" 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3264656675, 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639381, 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC2.CDR" 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3277119635, 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639565, 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.PX" 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3324122306, 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639482, 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.PX" 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3397470655, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 2199023256056, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS" 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3417015843, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639555, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB" 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3446671954, 2024-05-20T21:11:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639425, 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/RESULTS.XLS" 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3516728476, 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639342, 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/MAILBOX2.CPT" 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3559883831, 2024-05-20T21:11:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639352, 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TEAPOT2.CPT" 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.308+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6e40b2900 msgr2=0x7fd6e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.308+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6e40b2900 0x7fd6e40b4dc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0000c00 tx=0x7fd6f0005fb0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.308+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 msgr2=0x7fd6f81a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.308+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f81a0720 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00efc0 tx=0x7fd6ec00c3a0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 shutdown_connections 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd6e40b2900 0x7fd6e40b4dc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6f810eeb0 0x7fd6f81a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6f8109dd0 0x7fd6f81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 --2- 172.21.15.26:0/631737782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6f810a780 0x7fd6f81a0720 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 >> 172.21.15.26:0/631737782 conn(0x7fd6f8076d10 msgr2=0x7fd6f8078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 shutdown_connections 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:04.309+0000 7fd6ff5a0700 1 -- 172.21.15.26:0/631737782 wait complete. 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3597774530, 2024-05-20T21:11:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639622, 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/FASTENER.MDB" 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3611825292, 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639280, 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3659037241, 2024-05-20T21:11:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639312, 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRBARS.CFG" 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3720786040, 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639493, 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04" 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3756978936, 2024-05-20T21:11:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639418, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/BENCHS.LWP" 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3802642551, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639576, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03" 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3872244222, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639548, 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL" 2024-05-20T21:11:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3878729776, 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639596, 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV" 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3945285185, 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639636, 2024-05-20T21:11:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/SMALL.FIL" 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4001710755, 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639467, 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX" 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4005605087, 2024-05-20T21:11:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 2199023256054, 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP" 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4144478307, 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639541, 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1" 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4166940676, 2024-05-20T21:11:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639250, 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ZD16.BMP" 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4265572797, 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511639558, 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.DB" 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:04.353 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-05-20T21:11:05.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11358 10.97 MB/sec warmup 70 sec latency 90.523 ms 2024-05-20T21:11:05.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-20T21:11:05.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:05 smithi055 ceph-mon[35213]: pgmap v838: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-20T21:11:05.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:05 smithi026 ceph-mon[31990]: pgmap v838: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-20T21:11:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:05 smithi144 ceph-mon[35660]: pgmap v838: 97 pgs: 97 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-20T21:11:05.566 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:06.109 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11508 10.97 MB/sec warmup 71 sec latency 86.473 ms 2024-05-20T21:11:07.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11681 11.01 MB/sec warmup 72 sec latency 123.109 ms 2024-05-20T21:11:07.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:07 smithi026 ceph-mon[31990]: pgmap v839: 97 pgs: 97 active+clean; 233 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-05-20T21:11:07.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:07 smithi055 ceph-mon[35213]: pgmap v839: 97 pgs: 97 active+clean; 233 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-05-20T21:11:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:07 smithi144 ceph-mon[35660]: pgmap v839: 97 pgs: 97 active+clean; 233 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 -- 172.21.15.26:0/4046409161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 msgr2=0x7f99ac106140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 --2- 172.21.15.26:0/4046409161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac106140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009900 tx=0x7f99a002a400 comp rx=0 tx=0).stop 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 -- 172.21.15.26:0/4046409161 shutdown_connections 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 --2- 172.21.15.26:0/4046409161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 0x7f99ac106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 --2- 172.21.15.26:0/4046409161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac106140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 --2- 172.21.15.26:0/4046409161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ac101360 0x7f99ac103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 -- 172.21.15.26:0/4046409161 >> 172.21.15.26:0/4046409161 conn(0x7f99ac0fcf60 msgr2=0x7f99ac0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.711+0000 7f99b2472640 1 -- 172.21.15.26:0/4046409161 shutdown_connections 2024-05-20T21:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.712+0000 7f99b2472640 1 -- 172.21.15.26:0/4046409161 wait complete. 2024-05-20T21:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.712+0000 7f99b2472640 1 Processor -- start 2024-05-20T21:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.712+0000 7f99b2472640 1 -- start start 2024-05-20T21:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99b2472640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ac101360 0x7f99ac19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99b2472640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99b2472640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 0x7f99ac1a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99b2472640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac114ac0 con 0x7f99ac101360 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99b2472640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac114940 con 0x7f99ac106680 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99b2472640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac114c40 con 0x7f99ac103cd0 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41628/0 (socket says 172.21.15.26:41628) 2024-05-20T21:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.713+0000 7f99ab7fe640 1 -- 172.21.15.26:0/330186148 learned_addr learned my addr 172.21.15.26:0/330186148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 0x7f99ac1a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99ab7fe640 1 -- 172.21.15.26:0/330186148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 msgr2=0x7f99ac1a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 0x7f99ac1a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99ab7fe640 1 -- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ac101360 msgr2=0x7f99ac19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ac101360 0x7f99ac19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99ab7fe640 1 -- 172.21.15.26:0/330186148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac1a6ce0 con 0x7f99ac103cd0 2024-05-20T21:11:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 0x7f99ac1a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac19f650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f99a0000c00 tx=0x7f99a0032c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.714+0000 7f99a97fa640 1 -- 172.21.15.26:0/330186148 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a003f030 con 0x7f99ac103cd0 2024-05-20T21:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.715+0000 7f99a97fa640 1 -- 172.21.15.26:0/330186148 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99a0032e20 con 0x7f99ac103cd0 2024-05-20T21:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.715+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac1a6f70 con 0x7f99ac103cd0 2024-05-20T21:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.715+0000 7f99a97fa640 1 -- 172.21.15.26:0/330186148 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0035070 con 0x7f99ac103cd0 2024-05-20T21:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.715+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a73e0 con 0x7f99ac103cd0 2024-05-20T21:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 Processor -- start 2024-05-20T21:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 -- start start 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe52cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52d0b0 0x559efe531bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe5346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559efe49dc30 con 0x559efe52d0b0 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559efe48d7d0 con 0x559efe52c6c0 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99b518c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559efe491810 con 0x559efe532220 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.718+0000 7f99a97fa640 1 -- 172.21.15.26:0/330186148 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f99a0042030 con 0x7f99ac103cd0 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe52cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe52cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49812/0 (socket says 172.21.15.26:49812) 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 -- 172.21.15.26:0/27228475 learned_addr learned my addr 172.21.15.26:0/27228475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99abfff640 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52d0b0 0x559efe531bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 -- 172.21.15.26:0/27228475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 msgr2=0x559efe5346b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:11:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe5346b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 -- 172.21.15.26:0/27228475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52d0b0 msgr2=0x559efe531bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52d0b0 0x559efe531bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 -- 172.21.15.26:0/27228475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559efe1a2210 con 0x559efe52c6c0 2024-05-20T21:11:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99abfff640 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52d0b0 0x559efe531bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.719+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe52cae0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009fb0 tx=0x7f99a4010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27c0d00728575db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.720+0000 7f99927fc640 1 -- 172.21.15.26:0/27228475 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4016070 con 0x559efe52c6c0 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.720+0000 7f99927fc640 1 -- 172.21.15.26:0/27228475 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99a4012ba0 con 0x559efe52c6c0 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.720+0000 7f99927fc640 1 -- 172.21.15.26:0/27228475 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a401bba0 con 0x559efe52c6c0 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.720+0000 7f99a97fa640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99940b2920 0x7f99940b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.720+0000 7f99a97fa640 1 -- 172.21.15.26:0/330186148 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f99a0102900 con 0x7f99ac103cd0 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99abfff640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99940b2920 0x7f99940b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 -- 172.21.15.26:0/27228475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 msgr2=0x559efe52cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe52cae0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009fb0 tx=0x7f99a4010800 comp rx=0 tx=0).stop 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 -- 172.21.15.26:0/27228475 shutdown_connections 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe5346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52d0b0 0x559efe531bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 --2- 172.21.15.26:0/27228475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe52cae0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 -- 172.21.15.26:0/27228475 >> 172.21.15.26:0/27228475 conn(0x559efe527d30 msgr2=0x559efe528130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 -- 172.21.15.26:0/27228475 shutdown_connections 2024-05-20T21:11:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99b518c740 1 -- 172.21.15.26:0/27228475 wait complete. 2024-05-20T21:11:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.721+0000 7f99abfff640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99940b2920 0x7f99940b4de0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559efe52d4b0 tx=0x7f999c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.722+0000 7f99b518c740 1 -- start start 2024-05-20T21:11:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.722+0000 7f99b518c740 1 Processor -- start 2024-05-20T21:11:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.722+0000 7f99b518c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52c6c0 0x559efe536990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.722+0000 7f99b518c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52d0b0 0x559efe536ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.722+0000 7f99b518c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe53dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b518c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559efe173790 con 0x559efe52c6c0 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b518c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559efe449890 con 0x559efe52d0b0 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe53dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe53dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41650/0 (socket says 172.21.15.26:41650) 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 -- 172.21.15.26:0/1309516592 learned_addr learned my addr 172.21.15.26:0/1309516592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99abfff640 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52c6c0 0x559efe536990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b518c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559efe5351d0 con 0x559efe532220 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b518c740 10 client.0 _ll_register_callbacks cb 0x559efe1a1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 -- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52d0b0 msgr2=0x559efe536ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52d0b0 0x559efe536ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 -- 172.21.15.26:0/1309516592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52c6c0 msgr2=0x559efe536990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52c6c0 0x559efe536990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b0a0f640 1 -- 172.21.15.26:0/1309516592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559efe26e620 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.723+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a4012ba0 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe53dea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99a401b530 tx=0x7f99a4009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ab697f3a8a7e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f998b7fe640 10 client.28903 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f998b7fe640 1 -- 172.21.15.26:0/1309516592 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4016040 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f998b7fe640 1 -- 172.21.15.26:0/1309516592 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f99a4012510 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f998b7fe640 1 -- 172.21.15.26:0/1309516592 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99a4012710 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f99b518c740 10 client.28903 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.724+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=34}) -- 0x559efe540dd0 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f998b7fe640 1 -- 172.21.15.26:0/1309516592 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f99a400f040 con 0x559efe532220 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99b518c740 10 client.28903 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99b518c740 10 client.28903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99b518c740 20 client.28903 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99b518c740 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe542930 0x559efe544d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99b518c740 4 client.28903 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559efe49fb80 con 0x559efe542930 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.725+0000 7f99abfff640 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe542930 0x559efe544d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.726+0000 7f99abfff640 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe542930 0x559efe544d70 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5603adfabf191a62 server_cookie=c70d89c2f76bf809 in_seq=0 out_seq=0 2024-05-20T21:11:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.726+0000 7f998b7fe640 10 client.28903 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f998b7fe640 1 -- 172.21.15.26:0/1309516592 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559efe49fb80 con 0x559efe542930 2024-05-20T21:11:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f998b7fe640 10 client.28903 handle_command_reply: tid=0 2024-05-20T21:11:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 client.28903 shutdown 2024-05-20T21:11:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 msgr2=0x559efe53dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe53dea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99a401b530 tx=0x7f99a4009ac0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 shutdown_connections 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe542930 0x559efe544d70 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe532220 0x559efe53dea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52d0b0 0x559efe536ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 --2- 172.21.15.26:0/1309516592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe52c6c0 0x559efe536990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 >> 172.21.15.26:0/1309516592 conn(0x559efe527d30 msgr2=0x559efe526fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.728+0000 7f998b7fe640 0 client.28903 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.729+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 shutdown_connections 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.729+0000 7f99b518c740 1 -- 172.21.15.26:0/1309516592 wait complete. 2024-05-20T21:11:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.729+0000 7f99b518c740 20 client.28903 trim_cache size 0 max 16384 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 -- 172.21.15.26:0/1723403961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28102810 msgr2=0x7f4e28106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1723403961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28102810 0x7f4e28106810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009990 tx=0x7f4e1c009ca0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 -- 172.21.15.26:0/1723403961 shutdown_connections 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1723403961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28106ef0 0x7f4e2810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1723403961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e28101e60 0x7f4e28102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1723403961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28102810 0x7f4e28106810 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 -- 172.21.15.26:0/1723403961 >> 172.21.15.26:0/1723403961 conn(0x7f4e28078ab0 msgr2=0x7f4e280ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 -- 172.21.15.26:0/1723403961 shutdown_connections 2024-05-20T21:11:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.747+0000 7f4e2d366700 1 -- 172.21.15.26:0/1723403961 wait complete. 2024-05-20T21:11:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 Processor -- start 2024-05-20T21:11:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 -- start start 2024-05-20T21:11:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101e60 0x7f4e2819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e28102810 0x7f4e2819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 0x7f4e28199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28111410 con 0x7f4e28101e60 2024-05-20T21:11:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28111290 con 0x7f4e28106ef0 2024-05-20T21:11:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.749+0000 7f4e2d366700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28111590 con 0x7f4e28102810 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 0x7f4e28199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 0x7f4e28199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49826/0 (socket says 172.21.15.26:49826) 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 -- 172.21.15.26:0/1481698322 learned_addr learned my addr 172.21.15.26:0/1481698322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101e60 0x7f4e2819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 -- 172.21.15.26:0/1481698322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e28102810 msgr2=0x7f4e2819fcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e28102810 0x7f4e2819fcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 -- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101e60 msgr2=0x7f4e2819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101e60 0x7f4e2819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 -- 172.21.15.26:0/1481698322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2819a0f0 con 0x7f4e28106ef0 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.750+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 0x7f4e28199990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000ea50 tx=0x7f4e2000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.751+0000 7f4e0ffff700 1 -- 172.21.15.26:0/1481698322 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000cad0 con 0x7f4e28106ef0 2024-05-20T21:11:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.751+0000 7f4e0ffff700 1 -- 172.21.15.26:0/1481698322 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e20004cc0 con 0x7f4e28106ef0 2024-05-20T21:11:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.751+0000 7f4e0ffff700 1 -- 172.21.15.26:0/1481698322 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20005680 con 0x7f4e28106ef0 2024-05-20T21:11:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.751+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2819a3e0 con 0x7f4e28106ef0 2024-05-20T21:11:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.751+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e28108a90 con 0x7f4e28106ef0 2024-05-20T21:11:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.755+0000 7f99b518c740 1 Processor -- start 2024-05-20T21:11:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.755+0000 7f4e0ffff700 1 -- 172.21.15.26:0/1481698322 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4e20004750 con 0x7f4e28106ef0 2024-05-20T21:11:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.755+0000 7f99b518c740 1 -- start start 2024-05-20T21:11:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99b518c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe5277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99b518c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99abfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe5277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99abfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe5277d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49828/0 (socket says 172.21.15.26:49828) 2024-05-20T21:11:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99b518c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe535fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e0ffff700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e100b2900 0x7f4e100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99abfff640 1 -- 172.21.15.26:0/3998289534 learned_addr learned my addr 172.21.15.26:0/3998289534 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 Processor -- start 2024-05-20T21:11:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1b2d20 con 0x559efe532220 2024-05-20T21:11:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1b0560 con 0x559efe52c6c0 2024-05-20T21:11:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 -- start start 2024-05-20T21:11:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e100b2900 0x7f4e100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1ae3b0 con 0x559efe52d0b0 2024-05-20T21:11:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe535fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b8860 0x55d90c6b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 0x55d90c6bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6bcf60 0x55d90c6bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d90c617c90 con 0x55d90c6b9210 2024-05-20T21:11:07.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d90c607830 con 0x55d90c6bcf60 2024-05-20T21:11:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99abfff640 1 -- 172.21.15.26:0/3998289534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 msgr2=0x559efe53de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99abfff640 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99abfff640 1 -- 172.21.15.26:0/3998289534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 msgr2=0x559efe535fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99abfff640 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe535fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99abfff640 1 -- 172.21.15.26:0/3998289534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559efe44f6c0 con 0x559efe52c6c0 2024-05-20T21:11:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe535fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.756+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d90c60b870 con 0x55d90c6b8860 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e0ffff700 1 -- 172.21.15.26:0/1481698322 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4e200e20a0 con 0x7f4e28106ef0 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b8860 0x55d90c6b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e267fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 0x55d90c6bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6bcf60 0x55d90c6bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e267fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 0x55d90c6bc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53424/0 (socket says 172.21.15.26:53424) 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e267fc700 1 -- 172.21.15.26:0/2904459396 learned_addr learned my addr 172.21.15.26:0/2904459396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 -- 172.21.15.26:0/2904459396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b8860 msgr2=0x55d90c6b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b8860 0x55d90c6b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 -- 172.21.15.26:0/2904459396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 msgr2=0x55d90c6bc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 0x55d90c6bc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 -- 172.21.15.26:0/2904459396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90c5578d0 con 0x55d90c6bcf60 2024-05-20T21:11:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e267fc700 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 0x55d90c6bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e100b2900 0x7f4e100b4dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4e28106d20 tx=0x7f4e1800f9a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f99abfff640 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe5277d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x559efe528c00 tx=0x7f999c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4cec72c93c8a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f4e277fe700 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6bcf60 0x55d90c6bd3a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4e200dfbd0 tx=0x7f4e200a95a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c00f0d393bb240 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.757+0000 7f9991ffb640 1 -- 172.21.15.26:0/3998289534 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c016070 con 0x559efe52c6c0 2024-05-20T21:11:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.758+0000 7f9991ffb640 1 -- 172.21.15.26:0/3998289534 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f999c014070 con 0x559efe52c6c0 2024-05-20T21:11:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.758+0000 7f9991ffb640 1 -- 172.21.15.26:0/3998289534 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c004b60 con 0x559efe52c6c0 2024-05-20T21:11:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.758+0000 7f4e0d7fa700 1 -- 172.21.15.26:0/2904459396 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e200c3070 con 0x55d90c6bcf60 2024-05-20T21:11:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.758+0000 7f4e0d7fa700 1 -- 172.21.15.26:0/2904459396 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e200a96a0 con 0x55d90c6bcf60 2024-05-20T21:11:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.758+0000 7f4e0d7fa700 1 -- 172.21.15.26:0/2904459396 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e200910b0 con 0x55d90c6bcf60 2024-05-20T21:11:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 msgr2=0x559efe5277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe5277d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x559efe528c00 tx=0x7f999c012000 comp rx=0 tx=0).stop 2024-05-20T21:11:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/2904459396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6bcf60 msgr2=0x55d90c6bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 shutdown_connections 2024-05-20T21:11:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe535fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 --2- 172.21.15.26:0/3998289534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe5277d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 >> 172.21.15.26:0/3998289534 conn(0x559efe527d30 msgr2=0x559efe536990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6bcf60 0x55d90c6bd3a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4e200dfbd0 tx=0x7f4e200a95a0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 shutdown_connections 2024-05-20T21:11:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/2904459396 shutdown_connections 2024-05-20T21:11:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6bcf60 0x55d90c6bd3a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 -- 172.21.15.26:0/3998289534 wait complete. 2024-05-20T21:11:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b8860 0x55d90c6b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/2904459396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b9210 0x55d90c6bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/2904459396 >> 172.21.15.26:0/2904459396 conn(0x55d90c5e6080 msgr2=0x55d90c6b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/2904459396 shutdown_connections 2024-05-20T21:11:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/2904459396 wait complete. 2024-05-20T21:11:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f99b518c740 1 -- start start 2024-05-20T21:11:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 -- start start 2024-05-20T21:11:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.759+0000 7f4e37bfcb80 1 Processor -- start 2024-05-20T21:11:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 Processor -- start 2024-05-20T21:11:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe53ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe5ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b8860 0x55d90c6c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b9210 0x55d90c6c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1aea20 con 0x559efe532220 2024-05-20T21:11:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1afe90 con 0x559efe52c6c0 2024-05-20T21:11:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1af0f0 con 0x559efe52d0b0 2024-05-20T21:11:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b518c740 10 client.0 _ll_register_callbacks cb 0x7f999c005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99abfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe53ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6bcf60 0x55d90c6c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b8860 0x55d90c6c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b8860 0x55d90c6c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49840/0 (socket says 172.21.15.26:49840) 2024-05-20T21:11:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b0a0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe5ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41696/0 (socket says 172.21.15.26:41696) 2024-05-20T21:11:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99ab7fe640 1 -- 172.21.15.26:0/596704122 learned_addr learned my addr 172.21.15.26:0/596704122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99b0a0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe5ab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53428/0 (socket says 172.21.15.26:53428) 2024-05-20T21:11:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f99abfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe53ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49834/0 (socket says 172.21.15.26:49834) 2024-05-20T21:11:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d90c50dfa0 con 0x55d90c6bcf60 2024-05-20T21:11:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e26ffd700 1 -- 172.21.15.26:0/1491535453 learned_addr learned my addr 172.21.15.26:0/1491535453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6bcf60 0x55d90c6c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d90c5a97c0 con 0x55d90c6b8860 2024-05-20T21:11:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d90c6c5110 con 0x55d90c6b9210 2024-05-20T21:11:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e267fc700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b9210 0x55d90c6c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.760+0000 7f4e37bfcb80 10 client.0 _ll_register_callbacks cb 0x55d90c5a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99ab7fe640 1 -- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 msgr2=0x559efe53ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe53ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99ab7fe640 1 -- 172.21.15.26:0/596704122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 msgr2=0x559efe5ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe5ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99ab7fe640 1 -- 172.21.15.26:0/596704122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559efe5aba80 con 0x559efe52d0b0 2024-05-20T21:11:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99b0a0f640 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe5ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e267fc700 1 -- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b8860 msgr2=0x55d90c6c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e267fc700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b8860 0x55d90c6c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e267fc700 1 -- 172.21.15.26:0/1491535453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6bcf60 msgr2=0x55d90c6c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e267fc700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6bcf60 0x55d90c6c1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e267fc700 1 -- 172.21.15.26:0/1491535453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90c5a6690 con 0x55d90c6b9210 2024-05-20T21:11:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f99ab7fe640 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53b2b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f99a00c5ee0 tx=0x7f99a00c9650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb3fedbcb725f16e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f998b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e267fc700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b9210 0x55d90c6c17d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098e0 tx=0x7f4e1c00ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177ccf3aa81f71cd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f998b7fe640 1 -- 172.21.15.26:0/596704122 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0049040 con 0x559efe52d0b0 2024-05-20T21:11:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f998b7fe640 1 -- 172.21.15.26:0/596704122 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f99a00c60f0 con 0x559efe52d0b0 2024-05-20T21:11:07.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1491535453 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004230 con 0x55d90c6b9210 2024-05-20T21:11:07.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.761+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1491535453 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4e1c003890 con 0x55d90c6b9210 2024-05-20T21:11:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999c014070 con 0x559efe52d0b0 2024-05-20T21:11:07.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e200a96a0 con 0x55d90c6b9210 2024-05-20T21:11:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f998b7fe640 1 -- 172.21.15.26:0/596704122 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99a00c9e60 con 0x559efe52d0b0 2024-05-20T21:11:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f99b518c740 10 client.28915 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559efe5acd80 con 0x559efe52d0b0 2024-05-20T21:11:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1491535453 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c00be90 con 0x55d90c6b9210 2024-05-20T21:11:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 10 client.28918 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d90c6d8150 con 0x55d90c6b9210 2024-05-20T21:11:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f998b7fe640 1 -- 172.21.15.26:0/596704122 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f999c014070 con 0x559efe52d0b0 2024-05-20T21:11:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f998b7fe640 1 -- 172.21.15.26:0/596704122 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f99a00c90c0 con 0x559efe52d0b0 2024-05-20T21:11:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1491535453 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e200a96a0 con 0x55d90c6b9210 2024-05-20T21:11:07.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1491535453 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4e1c00f8e0 con 0x55d90c6b9210 2024-05-20T21:11:07.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 10 client.28918 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f99b518c740 10 client.28915 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 10 client.28918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 20 client.28918 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c6da730 0x55d90c6dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f4e37bfcb80 4 client.28918 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d90c6849e0 con 0x55d90c6da730 2024-05-20T21:11:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.762+0000 7f99b518c740 10 client.28915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f99b518c740 20 client.28915 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f99b518c740 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe5ad730 0x559efe5afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c6da730 0x55d90c6dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f99b518c740 4 client.28915 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:11:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559efe536780 con 0x559efe5ad730 2024-05-20T21:11:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f99abfff640 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe5ad730 0x559efe5afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f998b7fe640 10 client.28915 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c6da730 0x55d90c6dcb60 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881e9a8701ee1731 server_cookie=abb2714bf1916147 in_seq=0 out_seq=0 2024-05-20T21:11:07.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.763+0000 7f4e0aff5700 10 client.28918 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.764+0000 7f99abfff640 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe5ad730 0x559efe5afb50 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8f7046f1eb297f server_cookie=39866b8169df163d in_seq=0 out_seq=0 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.764+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1491535453 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d90c6849e0 con 0x55d90c6da730 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.764+0000 7f4e0aff5700 10 client.28918 handle_command_reply: tid=0 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.764+0000 7f4e37bfcb80 1 client.28918 shutdown 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b9210 msgr2=0x55d90c6c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b9210 0x55d90c6c17d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098e0 tx=0x7f4e1c00ba40 comp rx=0 tx=0).stop 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 shutdown_connections 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c6da730 0x55d90c6dcb60 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6bcf60 0x55d90c6c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b8860 0x55d90c6c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1491535453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6b9210 0x55d90c6c17d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 >> 172.21.15.26:0/1491535453 conn(0x55d90c5e6080 msgr2=0x55d90c6be350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e0aff5700 0 client.28918 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 shutdown_connections 2024-05-20T21:11:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1491535453 wait complete. 2024-05-20T21:11:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.765+0000 7f4e37bfcb80 20 client.28918 trim_cache size 0 max 16384 2024-05-20T21:11:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 Processor -- start 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 -- start start 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ae400 con 0x55d90c6b8860 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281abc00 con 0x55d90c6b9210 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281a9a50 con 0x55d90c6bcf60 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e267fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:41710/0 (socket says 172.21.15.26:41710) 2024-05-20T21:11:07.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.790+0000 7f4e277fe700 1 -- 172.21.15.26:0/419403574 learned_addr learned my addr 172.21.15.26:0/419403574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e267fc700 1 -- 172.21.15.26:0/419403574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 msgr2=0x55d90c6c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e267fc700 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e267fc700 1 -- 172.21.15.26:0/419403574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 msgr2=0x55d90c6d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e267fc700 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e267fc700 1 -- 172.21.15.26:0/419403574 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90c6b3d40 con 0x55d90c6b9210 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e277fe700 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.791+0000 7f4e267fc700 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6c00c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009990 tx=0x7f4e1c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48420c9eafbf0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.792+0000 7f4e0cff9700 1 -- 172.21.15.26:0/419403574 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00bd90 con 0x55d90c6b9210 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.792+0000 7f4e0cff9700 1 -- 172.21.15.26:0/419403574 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c027070 con 0x55d90c6b9210 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.792+0000 7f4e0cff9700 1 -- 172.21.15.26:0/419403574 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00f460 con 0x55d90c6b9210 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/419403574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 msgr2=0x55d90c6c00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6c00c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009990 tx=0x7f4e1c001e50 comp rx=0 tx=0).stop 2024-05-20T21:11:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/419403574 shutdown_connections 2024-05-20T21:11:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/419403574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6c00c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.793+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/419403574 >> 172.21.15.26:0/419403574 conn(0x55d90c5e6080 msgr2=0x55d90c6c1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.794+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/419403574 shutdown_connections 2024-05-20T21:11:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.794+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/419403574 wait complete. 2024-05-20T21:11:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.794+0000 7f4e37bfcb80 1 -- start start 2024-05-20T21:11:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.794+0000 7f4e37bfcb80 1 Processor -- start 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281aa0c0 con 0x55d90c6b8860 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ab530 con 0x55d90c6b9210 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281aa790 con 0x55d90c6bcf60 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e37bfcb80 10 client.0 _ll_register_callbacks cb 0x55d90c5a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e267fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53456/0 (socket says 172.21.15.26:53456) 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e26ffd700 1 -- 172.21.15.26:0/1898358052 learned_addr learned my addr 172.21.15.26:0/1898358052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.795+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e277fe700 1 -- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 msgr2=0x55d90c6d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e277fe700 1 -- 172.21.15.26:0/1898358052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 msgr2=0x55d90c6d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e277fe700 1 -- 172.21.15.26:0/1898358052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90c6d0960 con 0x55d90c6bcf60 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e277fe700 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6d0260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e200a6910 tx=0x7f4e200a5690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c752620ac9c492 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:07.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1898358052 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e200c3040 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1898358052 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4e200b2070 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c027070 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1898358052 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4e200a96a0 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e37bfcb80 10 client.28924 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.796+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d90c744ea0 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1898358052 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e200ada50 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1898358052 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4e2008b020 con 0x55d90c6bcf60 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e37bfcb80 10 client.28924 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e37bfcb80 10 client.28924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e37bfcb80 20 client.28924 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c745180 0x55d90c747570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e37bfcb80 4 client.28924 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-20T21:11:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.797+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4e1c027070 con 0x55d90c745180 2024-05-20T21:11:07.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.798+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c745180 0x55d90c747570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:07.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.798+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c745180 0x55d90c747570 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c88bcea78346345 server_cookie=b05002c0688c0f30 in_seq=0 out_seq=0 2024-05-20T21:11:07.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.798+0000 7f4e0aff5700 10 client.28924 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.810+0000 7f4e0aff5700 1 -- 172.21.15.26:0/1898358052 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4e1c027070 con 0x55d90c745180 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.810+0000 7f4e0aff5700 10 client.28924 handle_command_reply: tid=0 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.810+0000 7f4e37bfcb80 1 client.28924 shutdown 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 msgr2=0x55d90c6d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6d0260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e200a6910 tx=0x7f4e200a5690 comp rx=0 tx=0).stop 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 shutdown_connections 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d90c745180 0x55d90c747570 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d90c6b8860 0x55d90c6d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d90c6bcf60 0x55d90c6d0260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 --2- 172.21.15.26:0/1898358052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d90c6b9210 0x55d90c6d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.811+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 >> 172.21.15.26:0/1898358052 conn(0x55d90c5e6080 msgr2=0x55d90c6bbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.814+0000 7f4e0aff5700 0 client.28924 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.817+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 shutdown_connections 2024-05-20T21:11:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.819+0000 7f4e37bfcb80 1 -- 172.21.15.26:0/1898358052 wait complete. 2024-05-20T21:11:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.820+0000 7f4e37bfcb80 20 client.28924 trim_cache size 0 max 16384 2024-05-20T21:11:07.820 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-05-20T21:11:07.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.821+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e100b2900 msgr2=0x7f4e100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.821+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e100b2900 0x7f4e100b4dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4e28106d20 tx=0x7f4e1800f9a0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.821+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 msgr2=0x7f4e28199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.821+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 0x7f4e28199990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000ea50 tx=0x7f4e2000ed60 comp rx=0 tx=0).stop 2024-05-20T21:11:07.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.822+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 shutdown_connections 2024-05-20T21:11:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.822+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4e100b2900 0x7f4e100b4dc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.822+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101e60 0x7f4e2819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.822+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e28106ef0 0x7f4e28199990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.822+0000 7f4e2d366700 1 --2- 172.21.15.26:0/1481698322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e28102810 0x7f4e2819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.822+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 >> 172.21.15.26:0/1481698322 conn(0x7f4e28078ab0 msgr2=0x7f4e280ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.823+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 shutdown_connections 2024-05-20T21:11:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:07.823+0000 7f4e2d366700 1 -- 172.21.15.26:0/1481698322 wait complete. 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f998b7fe640 1 -- 172.21.15.26:0/596704122 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559efe536780 con 0x559efe5ad730 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f998b7fe640 10 client.28915 handle_command_reply: tid=0 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 client.28915 shutdown 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 msgr2=0x559efe53b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53b2b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f99a00c5ee0 tx=0x7f99a00c9650 comp rx=0 tx=0).stop 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 shutdown_connections 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x559efe5ad730 0x559efe5afb50 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559efe532220 0x559efe5ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f998b7fe640 0 client.28915 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559efe52d0b0 0x559efe53b2b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 --2- 172.21.15.26:0/596704122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559efe52c6c0 0x559efe53ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 >> 172.21.15.26:0/596704122 conn(0x559efe527d30 msgr2=0x559efe536950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.886+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 shutdown_connections 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b518c740 1 -- 172.21.15.26:0/596704122 wait complete. 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b518c740 20 client.28915 trim_cache size 0 max 16384 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99940b2920 msgr2=0x7f99940b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99940b2920 0x7f99940b4de0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559efe52d4b0 tx=0x7f999c00a2d0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 msgr2=0x7f99ac19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac19f650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f99a0000c00 tx=0x7f99a0032c30 comp rx=0 tx=0).stop 2024-05-20T21:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 shutdown_connections 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99940b2920 0x7f99940b4de0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99ac106680 0x7f99ac1a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99ac103cd0 0x7f99ac19f650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 --2- 172.21.15.26:0/330186148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ac101360 0x7f99ac19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.887+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 >> 172.21.15.26:0/330186148 conn(0x7f99ac0fcf60 msgr2=0x7f99ac1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.888+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 shutdown_connections 2024-05-20T21:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:07.888+0000 7f99b2472640 1 -- 172.21.15.26:0/330186148 wait complete. 2024-05-20T21:11:08.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11802 10.94 MB/sec warmup 73 sec latency 161.091 ms 2024-05-20T21:11:08.771 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 44218144, 'ino': 1099511639332, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 47735812, 'ino': 1099511639569, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 54991942, 'ino': 1099511639432, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 103788678, 'ino': 1099511639453, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 197521273, 'ino': 1099511639641, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 231725301, 'ino': 1099511639742, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWTIPS.PPT'}, {'damage_type': 'backtrace', 'id': 254790455, 'ino': 1099511639527, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 323163291, 'ino': 1099511639562, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 328769020, 'ino': 1099511639579, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 379511940, 'ino': 1099511639582, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 569762323, 'ino': 1099511639164, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD'}, {'damage_type': 'backtrace', 'id': 579737019, 'ino': 1099511639460, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 627655524, 'ino': 1099511639516, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 629341141, 'ino': 1099511639603, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 647160875, 'ino': 1099511639537, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 677553704, 'ino': 1099511639698, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT'}, {'damage_type': 'backtrace', 'id': 704466652, 'ino': 1099511639474, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 717518892, 'ino': 1099511639572, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 738158466, 'ino': 1099511639447, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 979430620, 'ino': 1099511639523, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 1003791986, 'ino': 1099511639507, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 1018632631, 'ino': 1099511639503, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 1065588826, 'ino': 1099511639401, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 1266876651, 'ino': 1099511639231, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT'}, {'damage_type': 'backtrace', 'id': 1268775360, 'ino': 1099511639362, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 1280457131, 'ino': 1099511639182, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 1353521433, 'ino': 1099511639290, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL'}, {'damage_type': 'backtrace', 'id': 1358769960, 'ino': 1099511639201, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL'}, {'damage_type': 'backtrace', 'id': 1371692659, 'ino': 1099511639534, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 1405098003, 'ino': 1099511639615, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS'}, {'damage_type': 'backtrace', 'id': 1660412617, 'ino': 1099511639610, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 1787424025, 'ino': 1099511639606, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 1801687754, 'ino': 1099511639509, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 1833936579, 'ino': 1099511639544, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 1860300750, 'ino': 1099511639411, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO'}, {'damage_type': 'backtrace', 'id': 1932682183, 'ino': 1099511639391, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2001491895, 'ino': 1099511639586, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 2051753661, 'ino': 1099511639192, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 2124062772, 'ino': 1099511639300, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 2126925419, 'ino': 1099511639530, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 2231445796, 'ino': 2199023256059, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 2292677282, 'ino': 1099511639630, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED'}, {'damage_type': 'backtrace', 'id': 2348845759, 'ino': 1099511639589, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 2349090964, 'ino': 1099511639489, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 2439870346, 'ino': 1099511639551, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 2527792425, 'ino': 1099511639487, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX'}, {'damage_type': 'backtrace', 'id': 2528478409, 'ino': 1099511639500, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 2529502806, 'ino': 1099511639440, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PM'}, {'damage_type': 'backtrace', 'id': 2541486423, 'ino': 1099511639240, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 2699893321, 'ino': 1099511639514, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 2727409312, 'ino': 1099511639260, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 2796191925, 'ino': 1099511639599, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 2861094881, 'ino': 1099511639270, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TIPS.PPT'}, {'damage_type': 'backtrace', 'id': 2876337672, 'ino': 1099511639322, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 2912501947, 'ino': 1099511639496, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 2953028741, 'ino': 1099511639480, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 2957513845, 'ino': 1099511639173, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 3087067414, 'ino': 1099511639646, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 3125608254, 'ino': 1099511639221, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 3159934142, 'ino': 1099511639371, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 3190875669, 'ino': 1099511639210, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 3239728539, 'ino': 1099511639521, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 3254754513, 'ino': 1099511639592, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 3264656675, 'ino': 1099511639381, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 3277119635, 'ino': 1099511639565, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 3324122306, 'ino': 1099511639482, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/COURSES.PX'}, {'damage_type': 'backtrace', 'id': 3397470655, 'ino': 2199023256056, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 3417015843, 'ino': 1099511639555, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 3446671954, 'ino': 1099511639425, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 3516728476, 'ino': 1099511639342, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 3559883831, 'ino': 1099511639352, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 3597774530, 'ino': 1099511639622, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 3611825292, 'ino': 1099511639280, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 3659037241, 'ino': 1099511639312, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 3720786040, 'ino': 1099511639493, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 3756978936, 'ino': 1099511639418, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 3802642551, 'ino': 1099511639576, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 3872244222, 'ino': 1099511639548, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 3878729776, 'ino': 1099511639596, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 3945285185, 'ino': 1099511639636, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 4001710755, 'ino': 1099511639467, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX'}, {'damage_type': 'backtrace', 'id': 4005605087, 'ino': 2199023256054, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 4144478307, 'ino': 1099511639541, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 4166940676, 'ino': 1099511639250, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 4265572797, 'ino': 1099511639558, 'path': '/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client5/~dmtmp/PARADOX/ENROLL.DB'}] 2024-05-20T21:11:08.772 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-05-20T21:11:09.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11998 11.08 MB/sec warmup 74 sec latency 101.687 ms 2024-05-20T21:11:09.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:09 smithi055 ceph-mon[35213]: pgmap v840: 97 pgs: 97 active+clean; 237 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-05-20T21:11:09.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:09 smithi026 ceph-mon[31990]: pgmap v840: 97 pgs: 97 active+clean; 237 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-05-20T21:11:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:09 smithi144 ceph-mon[35660]: pgmap v840: 97 pgs: 97 active+clean; 237 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-05-20T21:11:10.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12207 11.11 MB/sec warmup 75 sec latency 119.368 ms 2024-05-20T21:11:11.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12416 11.16 MB/sec warmup 76 sec latency 105.217 ms 2024-05-20T21:11:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:11 smithi055 ceph-mon[35213]: pgmap v841: 97 pgs: 97 active+clean; 237 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-05-20T21:11:11.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:11 smithi026 ceph-mon[31990]: pgmap v841: 97 pgs: 97 active+clean; 237 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-05-20T21:11:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:11 smithi144 ceph-mon[35660]: pgmap v841: 97 pgs: 97 active+clean; 237 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-05-20T21:11:12.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12540 11.06 MB/sec warmup 77 sec latency 83.687 ms 2024-05-20T21:11:13.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12634 10.96 MB/sec warmup 78 sec latency 111.320 ms 2024-05-20T21:11:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:13 smithi055 ceph-mon[35213]: pgmap v842: 97 pgs: 97 active+clean; 247 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-20T21:11:13.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:13 smithi026 ceph-mon[31990]: pgmap v842: 97 pgs: 97 active+clean; 247 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-20T21:11:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:13 smithi144 ceph-mon[35660]: pgmap v842: 97 pgs: 97 active+clean; 247 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-20T21:11:14.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12756 10.91 MB/sec warmup 79 sec latency 119.884 ms 2024-05-20T21:11:15.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12929 10.98 MB/sec warmup 80 sec latency 123.736 ms 2024-05-20T21:11:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:15 smithi055 ceph-mon[35213]: pgmap v843: 97 pgs: 97 active+clean; 253 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 4.4 MiB/s wr, 246 op/s 2024-05-20T21:11:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:15 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:11:15.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:15 smithi026 ceph-mon[31990]: pgmap v843: 97 pgs: 97 active+clean; 253 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 4.4 MiB/s wr, 246 op/s 2024-05-20T21:11:15.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:15 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:11:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:15 smithi144 ceph-mon[35660]: pgmap v843: 97 pgs: 97 active+clean; 253 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 4.4 MiB/s wr, 246 op/s 2024-05-20T21:11:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:15 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:11:16.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13103 10.95 MB/sec warmup 81 sec latency 93.954 ms 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: Unable to set osd_memory_target on smithi026 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:11:16.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:16 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: Unable to set osd_memory_target on smithi026 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:11:16.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:16 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:11:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:11:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-20T21:11:16.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: Unable to set osd_memory_target on smithi026 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-20T21:11:16.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:11:16.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:11:16.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:16 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:11:17.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13259 10.90 MB/sec warmup 82 sec latency 119.377 ms 2024-05-20T21:11:17.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:17 smithi055 ceph-mon[35213]: pgmap v844: 97 pgs: 97 active+clean; 249 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-05-20T21:11:17.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:17 smithi026 ceph-mon[31990]: pgmap v844: 97 pgs: 97 active+clean; 249 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-05-20T21:11:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:17 smithi144 ceph-mon[35660]: pgmap v844: 97 pgs: 97 active+clean; 249 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-05-20T21:11:18.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13406 10.86 MB/sec warmup 83 sec latency 91.792 ms 2024-05-20T21:11:19.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13543 10.76 MB/sec warmup 84 sec latency 91.671 ms 2024-05-20T21:11:19.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:19 smithi026 ceph-mon[31990]: pgmap v845: 97 pgs: 97 active+clean; 248 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 4.7 MiB/s wr, 263 op/s 2024-05-20T21:11:19.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:19 smithi055 ceph-mon[35213]: pgmap v845: 97 pgs: 97 active+clean; 248 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 4.7 MiB/s wr, 263 op/s 2024-05-20T21:11:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:19 smithi144 ceph-mon[35660]: pgmap v845: 97 pgs: 97 active+clean; 248 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 4.7 MiB/s wr, 263 op/s 2024-05-20T21:11:20.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13706 10.71 MB/sec warmup 85 sec latency 98.450 ms 2024-05-20T21:11:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 -- 172.21.15.26:0/958068070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 msgr2=0x7f8ea410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 --2- 172.21.15.26:0/958068070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea410d110 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009ca0 tx=0x7f8e9c02a8c0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 -- 172.21.15.26:0/958068070 shutdown_connections 2024-05-20T21:11:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 --2- 172.21.15.26:0/958068070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea410d110 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 --2- 172.21.15.26:0/958068070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4105ec0 0x7f8ea4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 --2- 172.21.15.26:0/958068070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4104cd0 0x7f8ea41050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 -- 172.21.15.26:0/958068070 >> 172.21.15.26:0/958068070 conn(0x7f8ea4100240 msgr2=0x7f8ea4102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 -- 172.21.15.26:0/958068070 shutdown_connections 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.020+0000 7f8eaa314640 1 -- 172.21.15.26:0/958068070 wait complete. 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.021+0000 7f8eaa314640 1 Processor -- start 2024-05-20T21:11:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.021+0000 7f8eaa314640 1 -- start start 2024-05-20T21:11:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.021+0000 7f8eaa314640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 0x7f8ea419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.021+0000 7f8eaa314640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4105ec0 0x7f8ea419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.021+0000 7f8eaa314640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea41a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8eaa314640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4110700 con 0x7f8ea4106880 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8eaa314640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4110580 con 0x7f8ea4104cd0 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8eaa314640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4110880 con 0x7f8ea4105ec0 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 0x7f8ea419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 0x7f8ea419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51482/0 (socket says 172.21.15.26:51482) 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 -- 172.21.15.26:0/4212328539 learned_addr learned my addr 172.21.15.26:0/4212328539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea41a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4105ec0 0x7f8ea419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 -- 172.21.15.26:0/4212328539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4105ec0 msgr2=0x7f8ea419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4105ec0 0x7f8ea419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 -- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 msgr2=0x7f8ea41a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea41a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea3fff640 1 -- 172.21.15.26:0/4212328539 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea41a2960 con 0x7f8ea4104cd0 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4105ec0 0x7f8ea419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.022+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea41a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.023+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 0x7f8ea419ad40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e9c0 tx=0x7f8e9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.023+0000 7f8ea17fa640 1 -- 172.21.15.26:0/4212328539 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400cd80 con 0x7f8ea4104cd0 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.023+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea41a2c50 con 0x7f8ea4104cd0 2024-05-20T21:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.023+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea41a31a0 con 0x7f8ea4104cd0 2024-05-20T21:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.023+0000 7f8ea17fa640 1 -- 172.21.15.26:0/4212328539 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e940035f0 con 0x7f8ea4104cd0 2024-05-20T21:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.023+0000 7f8ea17fa640 1 -- 172.21.15.26:0/4212328539 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940044c0 con 0x7f8ea4104cd0 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.026+0000 7f8ea17fa640 1 -- 172.21.15.26:0/4212328539 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8e94003fb0 con 0x7f8ea4104cd0 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.026+0000 7f8ead02d740 1 Processor -- start 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 -- start start 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55646744ec30 con 0x5564674de0b0 2024-05-20T21:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55646743e7d0 con 0x5564674dd6c0 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ead02d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556467442810 con 0x5564674e3220 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea88b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea88b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ddae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:51488/0 (socket says 172.21.15.26:51488) 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea88b1640 1 -- 172.21.15.26:0/1357709778 learned_addr learned my addr 172.21.15.26:0/1357709778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea17fa640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e8c0b2920 0x7f8e8c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.027+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e8c0b2920 0x7f8e8c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea17fa640 1 -- 172.21.15.26:0/4212328539 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8e940e2650 con 0x7f8ea4104cd0 2024-05-20T21:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1357709778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 msgr2=0x5564674e56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674e56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1357709778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 msgr2=0x5564674ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1357709778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556467153210 con 0x5564674de0b0 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e8c0b2920 0x7f8e8c0b4de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0319e0 tx=0x7f8e9c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674e2bb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f8e940e0140 tx=0x7f8e94003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3310771c32b955 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8e8a7fc640 1 -- 172.21.15.26:0/1357709778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a9380 con 0x5564674de0b0 2024-05-20T21:11:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8e8a7fc640 1 -- 172.21.15.26:0/1357709778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e94091070 con 0x5564674de0b0 2024-05-20T21:11:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.028+0000 7f8e8a7fc640 1 -- 172.21.15.26:0/1357709778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a53a0 con 0x5564674de0b0 2024-05-20T21:11:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.029+0000 7f8ead02d740 1 -- 172.21.15.26:0/1357709778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 msgr2=0x5564674e2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.029+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674e2bb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f8e940e0140 tx=0x7f8e94003a10 comp rx=0 tx=0).stop 2024-05-20T21:11:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 -- 172.21.15.26:0/1357709778 shutdown_connections 2024-05-20T21:11:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674e56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674e2bb0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1357709778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 -- 172.21.15.26:0/1357709778 >> 172.21.15.26:0/1357709778 conn(0x5564674d8d30 msgr2=0x5564674d9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 -- 172.21.15.26:0/1357709778 shutdown_connections 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 -- 172.21.15.26:0/1357709778 wait complete. 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 -- start start 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.030+0000 7f8ead02d740 1 Processor -- start 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674de0b0 0x5564674e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674e3220 0x5564674eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556467124790 con 0x5564674e3220 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5564673fa890 con 0x5564674de0b0 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49238/0 (socket says 172.21.15.26:49238) 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1973445580 learned_addr learned my addr 172.21.15.26:0/1973445580 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564674e6170 con 0x5564674dd6c0 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674de0b0 0x5564674e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674e3220 0x5564674eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674de0b0 msgr2=0x5564674e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674de0b0 0x5564674e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1973445580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674e3220 msgr2=0x5564674eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ead02d740 10 client.0 _ll_register_callbacks cb 0x556467152a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.031+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674e3220 0x5564674eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8ea3fff640 1 -- 172.21.15.26:0/1973445580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646721f620 con 0x5564674dd6c0 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e94091070 con 0x5564674dd6c0 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674de0b0 0x5564674e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674e77e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e940a5c90 tx=0x7f8e940a9790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed6d74d9a48f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8e837fe640 10 client.28936 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8e837fe640 1 -- 172.21.15.26:0/1973445580 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a9380 con 0x5564674dd6c0 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8e837fe640 1 -- 172.21.15.26:0/1973445580 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8e94003aa0 con 0x5564674dd6c0 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8e837fe640 1 -- 172.21.15.26:0/1973445580 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e940aa310 con 0x5564674dd6c0 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8ead02d740 10 client.28936 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.032+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5564674f1be0 con 0x5564674dd6c0 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8e837fe640 1 -- 172.21.15.26:0/1973445580 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8e940c40a0 con 0x5564674dd6c0 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ead02d740 10 client.28936 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ead02d740 10 client.28936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ead02d740 20 client.28936 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564674f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ead02d740 4 client.28936 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556467450b80 con 0x5564674f2f30 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.033+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564674f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.034+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564674f5320 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dba3d372529db29 server_cookie=255ee11e5aa0164d in_seq=0 out_seq=0 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.034+0000 7f8e837fe640 10 client.28936 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8e837fe640 1 -- 172.21.15.26:0/1973445580 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556467450b80 con 0x5564674f2f30 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8e837fe640 10 client.28936 handle_command_reply: tid=0 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 client.28936 shutdown 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 msgr2=0x5564674e77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674e77e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e940a5c90 tx=0x7f8e940a9790 comp rx=0 tx=0).stop 2024-05-20T21:11:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 shutdown_connections 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564674f5320 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674e3220 0x5564674eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8e837fe640 0 client.28936 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674de0b0 0x5564674e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 --2- 172.21.15.26:0/1973445580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674e77e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 >> 172.21.15.26:0/1973445580 conn(0x5564674d8d30 msgr2=0x5564674d7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 shutdown_connections 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.035+0000 7f8ead02d740 1 -- 172.21.15.26:0/1973445580 wait complete. 2024-05-20T21:11:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.036+0000 7f8ead02d740 20 client.28936 trim_cache size 0 max 16384 2024-05-20T21:11:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 Processor -- start 2024-05-20T21:11:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 -- start start 2024-05-20T21:11:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674e3220 0x5564674fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41ae9f0 con 0x5564674de0b0 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41ac230 con 0x5564674e3220 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.058+0000 7f8ead02d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41aa080 con 0x5564674dd6c0 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea88b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674e3220 0x5564674fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33602/0 (socket says 172.21.15.26:33602) 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674d3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:49250/0 (socket says 172.21.15.26:49250) 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea37fe640 1 -- 172.21.15.26:0/2096340011 learned_addr learned my addr 172.21.15.26:0/2096340011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 -- 172.21.15.26:0/2096340011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674e3220 msgr2=0x5564674fab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674e3220 0x5564674fab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 -- 172.21.15.26:0/2096340011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 msgr2=0x5564674eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 -- 172.21.15.26:0/2096340011 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564674d8b60 con 0x5564674dd6c0 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674e3220 0x5564674fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674d3e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e940a61b0 tx=0x7f8e940ac000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40db8dd9990e967f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8e89ffb640 1 -- 172.21.15.26:0/2096340011 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a9380 con 0x5564674dd6c0 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8e89ffb640 1 -- 172.21.15.26:0/2096340011 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e94091070 con 0x5564674dd6c0 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.059+0000 7f8e89ffb640 1 -- 172.21.15.26:0/2096340011 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940e2b00 con 0x5564674dd6c0 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 -- 172.21.15.26:0/2096340011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 msgr2=0x5564674d3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674d3e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e940a61b0 tx=0x7f8e940ac000 comp rx=0 tx=0).stop 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 -- 172.21.15.26:0/2096340011 shutdown_connections 2024-05-20T21:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674e3220 0x5564674fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 --2- 172.21.15.26:0/2096340011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674dd6c0 0x5564674d3e40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 -- 172.21.15.26:0/2096340011 >> 172.21.15.26:0/2096340011 conn(0x5564674d8d30 msgr2=0x5564674d9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 -- 172.21.15.26:0/2096340011 shutdown_connections 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 -- 172.21.15.26:0/2096340011 wait complete. 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.060+0000 7f8ead02d740 1 -- start start 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 Processor -- start 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41aa6f0 con 0x5564674de0b0 2024-05-20T21:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41abb60 con 0x5564674dd6c0 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41aadc0 con 0x5564674e3220 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ead02d740 10 client.0 _ll_register_callbacks cb 0x7f8e94005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33614/0 (socket says 172.21.15.26:33614) 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ea37fe640 1 -- 172.21.15.26:0/376193048 learned_addr learned my addr 172.21.15.26:0/376193048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ea88b1640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.061+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea37fe640 1 -- 172.21.15.26:0/376193048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 msgr2=0x5564674eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea37fe640 1 -- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 msgr2=0x5564674ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea37fe640 1 -- 172.21.15.26:0/376193048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564674ee070 con 0x5564674de0b0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eca70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea419c250 tx=0x7f8e980035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f5168de366ab6a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8e837fe640 1 -- 172.21.15.26:0/376193048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801e580 con 0x5564674de0b0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8e837fe640 1 -- 172.21.15.26:0/376193048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8e9801e720 con 0x5564674de0b0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e94091070 con 0x5564674de0b0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8e837fe640 1 -- 172.21.15.26:0/376193048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e9801e920 con 0x5564674de0b0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ead02d740 10 client.20049 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55646755dc00 con 0x5564674de0b0 2024-05-20T21:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.062+0000 7f8e837fe640 1 -- 172.21.15.26:0/376193048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e94091070 con 0x5564674de0b0 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8e837fe640 1 -- 172.21.15.26:0/376193048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8e98030410 con 0x5564674de0b0 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ead02d740 10 client.20049 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ead02d740 10 client.20049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ead02d740 20 client.20049 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ead02d740 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564675606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ead02d740 4 client.20049 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55646744a360 con 0x5564674f2f30 2024-05-20T21:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564675606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8e837fe640 10 client.20049 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.063+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564675606c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a654f4e90de4be28 server_cookie=56137038540cb8b9 in_seq=0 out_seq=0 2024-05-20T21:11:21.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13882 10.68 MB/sec warmup 86 sec latency 98.008 ms 2024-05-20T21:11:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.237+0000 7f8e837fe640 1 -- 172.21.15.26:0/376193048 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55646744a360 con 0x5564674f2f30 2024-05-20T21:11:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.237+0000 7f8e837fe640 10 client.20049 handle_command_reply: tid=0 2024-05-20T21:11:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.237+0000 7f8ead02d740 1 client.20049 shutdown 2024-05-20T21:11:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 msgr2=0x5564674eca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eca70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea419c250 tx=0x7f8e980035f0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 shutdown_connections 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5564674f2f30 0x5564675606c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564674e3220 0x5564674eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8e837fe640 0 client.20049 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564674de0b0 0x5564674eca70 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 --2- 172.21.15.26:0/376193048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5564674dd6c0 0x5564674ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 >> 172.21.15.26:0/376193048 conn(0x5564674d8d30 msgr2=0x5564674d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 shutdown_connections 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 1 -- 172.21.15.26:0/376193048 wait complete. 2024-05-20T21:11:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.238+0000 7f8ead02d740 20 client.20049 trim_cache size 0 max 16384 2024-05-20T21:11:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:11:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:11:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:11:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e8c0b2920 msgr2=0x7f8e8c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e8c0b2920 0x7f8e8c0b4de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0319e0 tx=0x7f8e9c02a8c0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 msgr2=0x7f8ea419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 0x7f8ea419ad40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e9c0 tx=0x7f8e9400ee80 comp rx=0 tx=0).stop 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 shutdown_connections 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8e8c0b2920 0x7f8e8c0b4de0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4106880 0x7f8ea41a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea4105ec0 0x7f8ea419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 --2- 172.21.15.26:0/4212328539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea4104cd0 0x7f8ea419ad40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 >> 172.21.15.26:0/4212328539 conn(0x7f8ea4100240 msgr2=0x7f8ea41091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.239+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 shutdown_connections 2024-05-20T21:11:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:21.240+0000 7f8eaa314640 1 -- 172.21.15.26:0/4212328539 wait complete. 2024-05-20T21:11:21.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:21 smithi055 ceph-mon[35213]: pgmap v846: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 4.7 MiB/s wr, 245 op/s 2024-05-20T21:11:21.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:21 smithi026 ceph-mon[31990]: pgmap v846: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 4.7 MiB/s wr, 245 op/s 2024-05-20T21:11:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:21 smithi144 ceph-mon[35660]: pgmap v846: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 4.7 MiB/s wr, 245 op/s 2024-05-20T21:11:22.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14067 10.68 MB/sec warmup 87 sec latency 187.788 ms 2024-05-20T21:11:23.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14195 10.60 MB/sec warmup 88 sec latency 110.533 ms 2024-05-20T21:11:23.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:23 smithi055 ceph-mon[35213]: pgmap v847: 97 pgs: 97 active+clean; 252 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 4.4 MiB/s wr, 247 op/s 2024-05-20T21:11:23.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:23 smithi026 ceph-mon[31990]: pgmap v847: 97 pgs: 97 active+clean; 252 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 4.4 MiB/s wr, 247 op/s 2024-05-20T21:11:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:23 smithi144 ceph-mon[35660]: pgmap v847: 97 pgs: 97 active+clean; 252 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 4.4 MiB/s wr, 247 op/s 2024-05-20T21:11:24.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14356 10.63 MB/sec warmup 89 sec latency 114.712 ms 2024-05-20T21:11:25.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14541 10.64 MB/sec warmup 90 sec latency 88.154 ms 2024-05-20T21:11:25.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:25 smithi055 ceph-mon[35213]: pgmap v848: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-20T21:11:25.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:25 smithi026 ceph-mon[31990]: pgmap v848: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-20T21:11:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:25 smithi144 ceph-mon[35660]: pgmap v848: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-20T21:11:26.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14715 10.68 MB/sec warmup 91 sec latency 106.687 ms 2024-05-20T21:11:26.180 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-20T21:11:26.180 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-05-20T21:11:26.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:11:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:27.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14835 10.63 MB/sec warmup 92 sec latency 113.848 ms 2024-05-20T21:11:27.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:27 smithi055 ceph-mon[35213]: pgmap v849: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-05-20T21:11:27.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:27 smithi026 ceph-mon[31990]: pgmap v849: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-05-20T21:11:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:27 smithi144 ceph-mon[35660]: pgmap v849: 97 pgs: 97 active+clean; 250 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-05-20T21:11:28.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14961 10.56 MB/sec warmup 93 sec latency 177.291 ms 2024-05-20T21:11:29.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15086 10.49 MB/sec warmup 94 sec latency 192.979 ms 2024-05-20T21:11:29.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:29 smithi055 ceph-mon[35213]: pgmap v850: 97 pgs: 97 active+clean; 247 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 4.1 MiB/s wr, 313 op/s 2024-05-20T21:11:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:29 smithi026 ceph-mon[31990]: pgmap v850: 97 pgs: 97 active+clean; 247 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 4.1 MiB/s wr, 313 op/s 2024-05-20T21:11:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:29 smithi144 ceph-mon[35660]: pgmap v850: 97 pgs: 97 active+clean; 247 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 4.1 MiB/s wr, 313 op/s 2024-05-20T21:11:30.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15242 10.43 MB/sec warmup 95 sec latency 207.600 ms 2024-05-20T21:11:31.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15402 10.38 MB/sec warmup 96 sec latency 101.181 ms 2024-05-20T21:11:31.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:31 smithi055 ceph-mon[35213]: pgmap v851: 97 pgs: 97 active+clean; 248 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.1 MiB/s wr, 301 op/s 2024-05-20T21:11:31.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:31 smithi026 ceph-mon[31990]: pgmap v851: 97 pgs: 97 active+clean; 248 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.1 MiB/s wr, 301 op/s 2024-05-20T21:11:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:31 smithi144 ceph-mon[35660]: pgmap v851: 97 pgs: 97 active+clean; 248 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.1 MiB/s wr, 301 op/s 2024-05-20T21:11:32.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15527 10.32 MB/sec warmup 97 sec latency 194.541 ms 2024-05-20T21:11:33.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15664 10.24 MB/sec warmup 98 sec latency 136.127 ms 2024-05-20T21:11:33.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:33 smithi055 ceph-mon[35213]: pgmap v852: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.8 MiB/s wr, 302 op/s 2024-05-20T21:11:33.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:33 smithi026 ceph-mon[31990]: pgmap v852: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.8 MiB/s wr, 302 op/s 2024-05-20T21:11:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:33 smithi144 ceph-mon[35660]: pgmap v852: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.8 MiB/s wr, 302 op/s 2024-05-20T21:11:34.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15812 10.22 MB/sec warmup 99 sec latency 155.455 ms 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 -- 172.21.15.26:0/1886025124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e281035a0 msgr2=0x7f7e28109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 --2- 172.21.15.26:0/1886025124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e281035a0 0x7f7e28109e30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009ca0 tx=0x7f7e2002a8c0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 -- 172.21.15.26:0/1886025124 shutdown_connections 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 --2- 172.21.15.26:0/1886025124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e281035a0 0x7f7e28109e30 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 --2- 172.21.15.26:0/1886025124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e28102be0 0x7f7e28103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 --2- 172.21.15.26:0/1886025124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e281019f0 0x7f7e28101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 -- 172.21.15.26:0/1886025124 >> 172.21.15.26:0/1886025124 conn(0x7f7e280fcf60 msgr2=0x7f7e280ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 -- 172.21.15.26:0/1886025124 shutdown_connections 2024-05-20T21:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.353+0000 7f7e3014e640 1 -- 172.21.15.26:0/1886025124 wait complete. 2024-05-20T21:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 Processor -- start 2024-05-20T21:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 -- start start 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 0x7f7e2806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28102be0 0x7f7e2819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e281035a0 0x7f7e2819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2810d3b0 con 0x7f7e28102be0 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2810d230 con 0x7f7e281035a0 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.354+0000 7f7e3014e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2810d530 con 0x7f7e281019f0 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 0x7f7e2806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2d6e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28102be0 0x7f7e2819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 0x7f7e2806b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:60580/0 (socket says 172.21.15.26:60580) 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 -- 172.21.15.26:0/2097526427 learned_addr learned my addr 172.21.15.26:0/2097526427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e281035a0 0x7f7e2819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 -- 172.21.15.26:0/2097526427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e281035a0 msgr2=0x7f7e2819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e281035a0 0x7f7e2819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 -- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28102be0 msgr2=0x7f7e2819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28102be0 0x7f7e2819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 -- 172.21.15.26:0/2097526427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e281a06f0 con 0x7f7e281019f0 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e2deea640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 0x7f7e2806b820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400e9c0 tx=0x7f7e2400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.355+0000 7f7e1effd640 1 -- 172.21.15.26:0/2097526427 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400cd00 con 0x7f7e281019f0 2024-05-20T21:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.356+0000 7f7e1effd640 1 -- 172.21.15.26:0/2097526427 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e240044b0 con 0x7f7e281019f0 2024-05-20T21:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.356+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e281a09e0 con 0x7f7e281019f0 2024-05-20T21:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.356+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281a0f80 con 0x7f7e281019f0 2024-05-20T21:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.356+0000 7f7e1effd640 1 -- 172.21.15.26:0/2097526427 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24010430 con 0x7f7e281019f0 2024-05-20T21:11:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.359+0000 7f7e1effd640 1 -- 172.21.15.26:0/2097526427 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7e240105d0 con 0x7f7e281019f0 2024-05-20T21:11:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.359+0000 7f7e32e6a740 1 Processor -- start 2024-05-20T21:11:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.359+0000 7f7e32e6a740 1 -- start start 2024-05-20T21:11:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526936ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e52693bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e52693e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5268a7c30 con 0x55e5269370b0 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e5268977d0 con 0x55e5269366c0 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e52689b810 con 0x55e52693c220 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2deea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e52693bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2e6eb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526936ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2deea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e52693bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53934/0 (socket says 172.21.15.26:53934) 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2deea640 1 -- 172.21.15.26:0/3378909623 learned_addr learned my addr 172.21.15.26:0/3378909623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e52693e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/3378909623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 msgr2=0x55e52693e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e52693e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/3378909623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 msgr2=0x55e52693bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e52693bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/3378909623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5265ac210 con 0x55e5269366c0 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.360+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e52693e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e1effd640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e140b2970 0x7f7e140b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526936ae0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7e20031040 tx=0x7f7e20009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7fd934651ad2455 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e2deea640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e140b2970 0x7f7e140b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e13fff640 1 -- 172.21.15.26:0/3378909623 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20031650 con 0x55e5269366c0 2024-05-20T21:11:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e1effd640 1 -- 172.21.15.26:0/2097526427 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7e240e2540 con 0x7f7e281019f0 2024-05-20T21:11:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e13fff640 1 -- 172.21.15.26:0/3378909623 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e20036030 con 0x55e5269366c0 2024-05-20T21:11:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.361+0000 7f7e13fff640 1 -- 172.21.15.26:0/3378909623 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200043b0 con 0x55e5269366c0 2024-05-20T21:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3378909623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 msgr2=0x55e526936ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e2deea640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e140b2970 0x7f7e140b4e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7e240a9a90 tx=0x7f7e240e80e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526936ae0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7e20031040 tx=0x7f7e20009a20 comp rx=0 tx=0).stop 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3378909623 shutdown_connections 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e52693e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e52693bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3378909623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526936ae0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3378909623 >> 172.21.15.26:0/3378909623 conn(0x55e526931d30 msgr2=0x55e526932130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3378909623 shutdown_connections 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.362+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3378909623 wait complete. 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 -- start start 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 Processor -- start 2024-05-20T21:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 0x55e526940600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269370b0 0x55e526940b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526947b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e52657d790 con 0x55e5269366c0 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e526853890 con 0x55e52693c220 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.363+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e52693eff0 con 0x55e5269370b0 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 0x55e526940600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 0x55e526940600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53938/0 (socket says 172.21.15.26:53938) 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1715324113 learned_addr learned my addr 172.21.15.26:0/1715324113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e32e6a740 10 client.0 _ll_register_callbacks cb 0x55e5265aba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269370b0 0x55e526940b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526947b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1715324113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269370b0 msgr2=0x55e526940b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269370b0 0x55e526940b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 msgr2=0x55e526947b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526947b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1715324113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e526678620 con 0x55e5269366c0 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269370b0 0x55e526940b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 0x55e526940600 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7e2002a900 tx=0x7f7e200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561a429b53ae325e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.364+0000 7f7e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e117fa640 1 -- 172.21.15.26:0/1715324113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20058060 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e117fa640 1 -- 172.21.15.26:0/1715324113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7e20031d80 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e20036030 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e117fa640 1 -- 172.21.15.26:0/1715324113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7e2004a040 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 10 client.20061 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e52694ab60 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e117fa640 1 -- 172.21.15.26:0/1715324113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e20036030 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e117fa640 1 -- 172.21.15.26:0/1715324113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7e20050070 con 0x55e5269366c0 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 10 client.20061 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 10 client.20061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 20 client.20061 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e52694e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 4 client.20061 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.365+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5268a9b80 con 0x55e52694bfe0 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.366+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e52694e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.366+0000 7f7e117fa640 10 client.20061 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.366+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e52694e3d0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd05ef29ad1d75d1 server_cookie=e2711ac15566fe3 in_seq=0 out_seq=0 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e117fa640 1 -- 172.21.15.26:0/1715324113 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e5268a9b80 con 0x55e52694bfe0 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e117fa640 10 client.20061 handle_command_reply: tid=0 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 client.20061 shutdown 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 msgr2=0x55e526940600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 0x55e526940600 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7e2002a900 tx=0x7f7e200095c0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 shutdown_connections 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e52694e3d0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526947b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269370b0 0x55e526940b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e117fa640 0 client.20061 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1715324113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269366c0 0x55e526940600 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 >> 172.21.15.26:0/1715324113 conn(0x55e526931d30 msgr2=0x55e526930fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 shutdown_connections 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1715324113 wait complete. 2024-05-20T21:11:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.368+0000 7f7e32e6a740 20 client.20061 trim_cache size 0 max 16384 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.393+0000 7f7e32e6a740 1 Processor -- start 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 -- start start 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526940aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526947b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e526934cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c010f70 con 0x55e5269370b0 2024-05-20T21:11:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00e7b0 con 0x55e5269366c0 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00c600 con 0x55e52693c220 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526947b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526947b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53948/0 (socket says 172.21.15.26:53948) 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 -- 172.21.15.26:0/3304209945 learned_addr learned my addr 172.21.15.26:0/3304209945 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526940aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e526934cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 -- 172.21.15.26:0/3304209945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 msgr2=0x55e526934cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e526934cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 -- 172.21.15.26:0/3304209945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 msgr2=0x55e526940aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526940aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2d6e9640 1 -- 172.21.15.26:0/3304209945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e526931b60 con 0x55e5269370b0 2024-05-20T21:11:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.394+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526940aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.395+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526947b40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000c00 tx=0x7f7e18005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed52927bdb13518 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.395+0000 7f7e137fe640 1 -- 172.21.15.26:0/3304209945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18010630 con 0x55e5269370b0 2024-05-20T21:11:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.395+0000 7f7e137fe640 1 -- 172.21.15.26:0/3304209945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e18010c60 con 0x55e5269370b0 2024-05-20T21:11:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.395+0000 7f7e137fe640 1 -- 172.21.15.26:0/3304209945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18027b20 con 0x55e5269370b0 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3304209945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 msgr2=0x55e526947b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526947b40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000c00 tx=0x7f7e18005f90 comp rx=0 tx=0).stop 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3304209945 shutdown_connections 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e52693c220 0x55e526934cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526947b40 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/3304209945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5269366c0 0x55e526940aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3304209945 >> 172.21.15.26:0/3304209945 conn(0x55e526931d30 msgr2=0x55e52692ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3304209945 shutdown_connections 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.396+0000 7f7e32e6a740 1 -- 172.21.15.26:0/3304209945 wait complete. 2024-05-20T21:11:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 -- start start 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 Processor -- start 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269366c0 0x55e526944d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526945240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526946250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00cc70 con 0x55e5269370b0 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00e0e0 con 0x55e52693c220 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00d340 con 0x55e5269366c0 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e32e6a740 10 client.0 _ll_register_callbacks cb 0x7f7e240e5ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e2d6e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526945240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2d6e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526945240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53952/0 (socket says 172.21.15.26:53952) 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.397+0000 7f7e2e6eb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269366c0 0x55e526944d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2d6e9640 1 -- 172.21.15.26:0/1627101517 learned_addr learned my addr 172.21.15.26:0/1627101517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526946250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 msgr2=0x55e526946250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526946250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1627101517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 msgr2=0x55e526945240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526945240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 -- 172.21.15.26:0/1627101517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e526946840 con 0x55e5269366c0 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e2e6eb640 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269366c0 0x55e526944d00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e20056040 tx=0x7f7e20031760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3936b3e207dd6350 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e117fa640 1 -- 172.21.15.26:0/1627101517 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20058040 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e18010c60 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.398+0000 7f7e117fa640 1 -- 172.21.15.26:0/1627101517 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7e20031a80 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e117fa640 1 -- 172.21.15.26:0/1627101517 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7e20031c80 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 10 client.28960 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e526947b30 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e117fa640 1 -- 172.21.15.26:0/1627101517 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e18010c60 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e117fa640 1 -- 172.21.15.26:0/1627101517 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7e20050070 con 0x55e5269366c0 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 10 client.28960 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 10 client.28960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 20 client.28960 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e5269b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 4 client.28960 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:11:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5268a3360 con 0x55e52694bfe0 2024-05-20T21:11:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.399+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e5269b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.400+0000 7f7e2d6e9640 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e5269b8fc0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476311d454c4f9c server_cookie=190809bad94ddc62 in_seq=0 out_seq=0 2024-05-20T21:11:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.400+0000 7f7e117fa640 10 client.28960 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.531+0000 7f7e117fa640 1 -- 172.21.15.26:0/1627101517 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e5268a3360 con 0x55e52694bfe0 2024-05-20T21:11:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.531+0000 7f7e117fa640 10 client.28960 handle_command_reply: tid=0 2024-05-20T21:11:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.531+0000 7f7e32e6a740 1 client.28960 shutdown 2024-05-20T21:11:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269366c0 msgr2=0x55e526944d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269366c0 0x55e526944d00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e20056040 tx=0x7f7e20031760 comp rx=0 tx=0).stop 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 shutdown_connections 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55e52694bfe0 0x55e5269b8fc0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e52693c220 0x55e526946250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e117fa640 0 client.28960 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5269370b0 0x55e526945240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 --2- 172.21.15.26:0/1627101517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5269366c0 0x55e526944d00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 >> 172.21.15.26:0/1627101517 conn(0x55e526931d30 msgr2=0x55e52693b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 shutdown_connections 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 1 -- 172.21.15.26:0/1627101517 wait complete. 2024-05-20T21:11:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.532+0000 7f7e32e6a740 20 client.28960 trim_cache size 0 max 16384 2024-05-20T21:11:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:11:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:11:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e140b2970 msgr2=0x7f7e140b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e140b2970 0x7f7e140b4e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7e240a9a90 tx=0x7f7e240e80e0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 msgr2=0x7f7e2806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 0x7f7e2806b820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400e9c0 tx=0x7f7e2400ee80 comp rx=0 tx=0).stop 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 shutdown_connections 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7e140b2970 0x7f7e140b4e30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7e281035a0 0x7f7e2819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28102be0 0x7f7e2819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 --2- 172.21.15.26:0/2097526427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e281019f0 0x7f7e2806b820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 >> 172.21.15.26:0/2097526427 conn(0x7f7e280fcf60 msgr2=0x7f7e28105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 shutdown_connections 2024-05-20T21:11:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:34.533+0000 7f7e3014e640 1 -- 172.21.15.26:0/2097526427 wait complete. 2024-05-20T21:11:35.110 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15921 10.14 MB/sec warmup 100 sec latency 204.306 ms 2024-05-20T21:11:35.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:35 smithi055 ceph-mon[35213]: pgmap v853: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-05-20T21:11:35.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:35 smithi026 ceph-mon[31990]: pgmap v853: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-05-20T21:11:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:35 smithi144 ceph-mon[35660]: pgmap v853: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-05-20T21:11:36.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16073 10.10 MB/sec warmup 101 sec latency 181.390 ms 2024-05-20T21:11:37.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16235 10.13 MB/sec warmup 102 sec latency 194.236 ms 2024-05-20T21:11:37.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:37 smithi055 ceph-mon[35213]: pgmap v854: 97 pgs: 97 active+clean; 243 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-05-20T21:11:37.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:37 smithi026 ceph-mon[31990]: pgmap v854: 97 pgs: 97 active+clean; 243 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-05-20T21:11:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:37 smithi144 ceph-mon[35660]: pgmap v854: 97 pgs: 97 active+clean; 243 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-05-20T21:11:38.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16419 10.18 MB/sec warmup 103 sec latency 228.222 ms 2024-05-20T21:11:39.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16637 10.22 MB/sec warmup 104 sec latency 121.661 ms 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 -- 172.21.15.26:0/2833962441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01077d0 msgr2=0x7f4af010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/2833962441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01077d0 0x7f4af010f370 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800d360 tx=0x7f4ae800d670 comp rx=0 tx=0).stop 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 -- 172.21.15.26:0/2833962441 shutdown_connections 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/2833962441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01077d0 0x7f4af010f370 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/2833962441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0102a80 0x7f4af0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/2833962441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af0103430 0x7f4af01070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.146+0000 7f4af5ce7700 1 -- 172.21.15.26:0/2833962441 >> 172.21.15.26:0/2833962441 conn(0x7f4af00793f0 msgr2=0x7f4af0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.147+0000 7f4af5ce7700 1 -- 172.21.15.26:0/2833962441 shutdown_connections 2024-05-20T21:11:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.147+0000 7f4af5ce7700 1 -- 172.21.15.26:0/2833962441 wait complete. 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.147+0000 7f4af5ce7700 1 Processor -- start 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 -- start start 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af0102a80 0x7f4af0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 0x7f4af0197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af01077d0 0x7f4af01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01114a0 con 0x7f4af0102a80 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4af0111320 con 0x7f4af01077d0 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4af5ce7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4af0111620 con 0x7f4af0103430 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4aeeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 0x7f4af0197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4aeeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 0x7f4af0197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:60634/0 (socket says 172.21.15.26:60634) 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af01077d0 0x7f4af01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4aeeffd700 1 -- 172.21.15.26:0/1737780706 learned_addr learned my addr 172.21.15.26:0/1737780706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.148+0000 7f4aef7fe700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af0102a80 0x7f4af0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aeeffd700 1 -- 172.21.15.26:0/1737780706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af01077d0 msgr2=0x7f4af01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aeeffd700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af01077d0 0x7f4af01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aeeffd700 1 -- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af0102a80 msgr2=0x7f4af0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aeeffd700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af0102a80 0x7f4af0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aeeffd700 1 -- 172.21.15.26:0/1737780706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af01ac3f0 con 0x7f4af0103430 2024-05-20T21:11:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aeeffd700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 0x7f4af0197c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400ea30 tx=0x7f4ae400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400cbd0 con 0x7f4af0103430 2024-05-20T21:11:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae40044b0 con 0x7f4af0103430 2024-05-20T21:11:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.149+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4003dd0 con 0x7f4af0103430 2024-05-20T21:11:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.150+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af01ac6e0 con 0x7f4af0103430 2024-05-20T21:11:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.150+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af01acc20 con 0x7f4af0103430 2024-05-20T21:11:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.151+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af006b4c0 con 0x7f4af0103430 2024-05-20T21:11:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.152+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4ae4003f70 con 0x7f4af0103430 2024-05-20T21:11:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.153+0000 7f4aecff9700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ad80b29d0 0x7f4ad80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.153+0000 7f4aef7fe700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ad80b29d0 0x7f4ad80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.154+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f4ae40e2270 con 0x7f4af0103430 2024-05-20T21:11:39.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.154+0000 7f4aef7fe700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ad80b29d0 0x7f4ad80b4e90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009bf0 tx=0x7f4ae0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.159+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ae40a9f30 con 0x7f4af0103430 2024-05-20T21:11:39.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:39 smithi055 ceph-mon[35213]: pgmap v855: 97 pgs: 97 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-05-20T21:11:39.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:39 smithi026 ceph-mon[31990]: pgmap v855: 97 pgs: 97 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-05-20T21:11:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:39 smithi144 ceph-mon[35660]: pgmap v855: 97 pgs: 97 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-05-20T21:11:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.594+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4af0198c70 con 0x7f4af0103430 2024-05-20T21:11:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.595+0000 7f4aecff9700 1 -- 172.21.15.26:0/1737780706 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11874 (secure 0 0 0) 0x7f4ae40a9830 con 0x7f4af0103430 2024-05-20T21:11:39.595 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:11:39.595 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:10:11.369568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ad80b29d0 msgr2=0x7f4ad80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ad80b29d0 0x7f4ad80b4e90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009bf0 tx=0x7f4ae0009260 comp rx=0 tx=0).stop 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 msgr2=0x7f4af0197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 0x7f4af0197c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400ea30 tx=0x7f4ae400ed40 comp rx=0 tx=0).stop 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 shutdown_connections 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f4ad80b29d0 0x7f4ad80b4e90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af0102a80 0x7f4af0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af01077d0 0x7f4af01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 --2- 172.21.15.26:0/1737780706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4af0103430 0x7f4af0197c80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.598+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 >> 172.21.15.26:0/1737780706 conn(0x7f4af00793f0 msgr2=0x7f4af0100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.599+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 shutdown_connections 2024-05-20T21:11:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:39.599+0000 7f4af5ce7700 1 -- 172.21.15.26:0/1737780706 wait complete. 2024-05-20T21:11:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 34 2024-05-20T21:11:40.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16825 10.27 MB/sec warmup 105 sec latency 83.989 ms 2024-05-20T21:11:40.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:40 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1737780706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:40.428 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-05-20T21:11:40.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-05-20T21:11:40.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:40 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1737780706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:40.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:40 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1737780706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:40.774 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:41.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16976 10.28 MB/sec warmup 106 sec latency 81.684 ms 2024-05-20T21:11:41.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:41 smithi026 ceph-mon[31990]: pgmap v856: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-05-20T21:11:41.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:41 smithi055 ceph-mon[35213]: pgmap v856: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-05-20T21:11:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:41 smithi144 ceph-mon[35660]: pgmap v856: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-05-20T21:11:42.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17106 10.24 MB/sec warmup 107 sec latency 104.613 ms 2024-05-20T21:11:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 -- 172.21.15.26:0/4154865896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea806be30 msgr2=0x7f6ea81081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/4154865896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea806be30 0x7f6ea81081d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6e980099c0 tx=0x7f6e98009cd0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 -- 172.21.15.26:0/4154865896 shutdown_connections 2024-05-20T21:11:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/4154865896 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea8108900 0x7f6ea8110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/4154865896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea806b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/4154865896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea806be30 0x7f6ea81081d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 -- 172.21.15.26:0/4154865896 >> 172.21.15.26:0/4154865896 conn(0x7f6ea80793e0 msgr2=0x7f6ea80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 -- 172.21.15.26:0/4154865896 shutdown_connections 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.877+0000 7f6eaf3df700 1 -- 172.21.15.26:0/4154865896 wait complete. 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.878+0000 7f6eaf3df700 1 Processor -- start 2024-05-20T21:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.878+0000 7f6eaf3df700 1 -- start start 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eaf3df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eaf3df700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea806be30 0x7f6ea819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6ead17b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6ead17b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:39862/0 (socket says 172.21.15.26:39862) 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6ead17b700 1 -- 172.21.15.26:0/2055080769 learned_addr learned my addr 172.21.15.26:0/2055080769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8108900 0x7f6ea81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eac97a700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea806be30 0x7f6ea819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81135e0 con 0x7f6ea8108900 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8113460 con 0x7f6ea806be30 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8113760 con 0x7f6ea806b480 2024-05-20T21:11:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.879+0000 7f6ead97c700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8108900 0x7f6ea81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6ead17b700 1 -- 172.21.15.26:0/2055080769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea806be30 msgr2=0x7f6ea819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6ead17b700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea806be30 0x7f6ea819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6ead17b700 1 -- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8108900 msgr2=0x7f6ea81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6ead17b700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8108900 0x7f6ea81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6ead17b700 1 -- 172.21.15.26:0/2055080769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea8199fa0 con 0x7f6ea806b480 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6ead17b700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea819f5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e9c0 tx=0x7f6ea400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6e9e7fc700 1 -- 172.21.15.26:0/2055080769 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400cbd0 con 0x7f6ea806b480 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6e9e7fc700 1 -- 172.21.15.26:0/2055080769 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ea400ee50 con 0x7f6ea806b480 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6e9e7fc700 1 -- 172.21.15.26:0/2055080769 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40054e0 con 0x7f6ea806b480 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea819a290 con 0x7f6ea806b480 2024-05-20T21:11:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.880+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea8102440 con 0x7f6ea806b480 2024-05-20T21:11:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.883+0000 7f6e9e7fc700 1 -- 172.21.15.26:0/2055080769 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6ea4004750 con 0x7f6ea806b480 2024-05-20T21:11:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.884+0000 7f6e9e7fc700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e940b2930 0x7f6e940b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.885+0000 7f6eac97a700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e940b2930 0x7f6e940b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.885+0000 7f6e9e7fc700 1 -- 172.21.15.26:0/2055080769 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6ea40e2580 con 0x7f6ea806b480 2024-05-20T21:11:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.885+0000 7f6eac97a700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e940b2930 0x7f6e940b4df0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6ea819ae30 tx=0x7f6e98000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:42.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.886+0000 7f6eb9c75b80 1 Processor -- start 2024-05-20T21:11:42.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.886+0000 7f6eb9c75b80 1 -- start start 2024-05-20T21:11:42.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945eccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ed9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560945e2be20 con 0x560945ecc970 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560945e1b9c0 con 0x560945ecd330 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560945e1fa00 con 0x560945ed2340 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ed9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ed9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49906/0 (socket says 172.21.15.26:49906) 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead17b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 -- 172.21.15.26:0/637949571 learned_addr learned my addr 172.21.15.26:0/637949571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6eac97a700 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945eccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 -- 172.21.15.26:0/637949571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 msgr2=0x560945ed2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 -- 172.21.15.26:0/637949571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 msgr2=0x560945eccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945eccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 -- 172.21.15.26:0/637949571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560945d6ba60 con 0x560945ecd330 2024-05-20T21:11:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.887+0000 7f6ead97c700 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ed9170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0011630 tx=0x7f6ea0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c82f1383f4a02bb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.888+0000 7f6e937fe700 1 -- 172.21.15.26:0/637949571 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0003a00 con 0x560945ecd330 2024-05-20T21:11:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.888+0000 7f6e937fe700 1 -- 172.21.15.26:0/637949571 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ea001a070 con 0x560945ecd330 2024-05-20T21:11:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.888+0000 7f6e937fe700 1 -- 172.21.15.26:0/637949571 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0007760 con 0x560945ecd330 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/637949571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 msgr2=0x560945ed9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ed9170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0011630 tx=0x7f6ea0009920 comp rx=0 tx=0).stop 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/637949571 shutdown_connections 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ed9170 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/637949571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945eccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/637949571 >> 172.21.15.26:0/637949571 conn(0x560945dfa210 msgr2=0x560945ec7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/637949571 shutdown_connections 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/637949571 wait complete. 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 -- start start 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.889+0000 7f6eb9c75b80 1 Processor -- start 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ecc970 0x560945ed8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecd330 0x560945ed61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ed2340 0x560945ed66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560945d22130 con 0x560945ecd330 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560945dbd950 con 0x560945ed2340 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560945ed96b0 con 0x560945ecc970 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eac97a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecd330 0x560945ed61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eb9c75b80 10 client.0 _ll_register_callbacks cb 0x560945db6ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eac97a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecd330 0x560945ed61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59056/0 (socket says 172.21.15.26:59056) 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6eac97a700 1 -- 172.21.15.26:0/1729923481 learned_addr learned my addr 172.21.15.26:0/1729923481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ecc970 0x560945ed8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead97c700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ed2340 0x560945ed66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead17b700 1 -- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ed2340 msgr2=0x560945ed66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ed2340 0x560945ed66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead17b700 1 -- 172.21.15.26:0/1729923481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecd330 msgr2=0x560945ed61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecd330 0x560945ed61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.890+0000 7f6ead17b700 1 -- 172.21.15.26:0/1729923481 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560945dba820 con 0x560945ecc970 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.891+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ecc970 0x560945ed8e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40a6d50 tx=0x7f6ea40b2280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ff3fb00164628 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.891+0000 7f6e90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.891+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1729923481 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40a9ae0 con 0x560945ecc970 2024-05-20T21:11:42.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.891+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1729923481 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6ea40a60d0 con 0x560945ecc970 2024-05-20T21:11:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.891+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea001a070 con 0x560945ecc970 2024-05-20T21:11:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.891+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1729923481 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ea40ade90 con 0x560945ecc970 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 10 client.28978 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560945ee3e60 con 0x560945ecc970 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1729923481 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ea40b2940 con 0x560945ecc970 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1729923481 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6ea40a5450 con 0x560945ecc970 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 10 client.28978 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 10 client.28978 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 20 client.28978 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945ee8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 4 client.28978 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ea001a070 con 0x560945ee6440 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.892+0000 7f6eac97a700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945ee8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.893+0000 7f6e90ff9700 10 client.28978 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.893+0000 7f6eac97a700 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945ee8860 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40f6c07f0c01833 server_cookie=364cac85ee732bbb in_seq=0 out_seq=0 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.894+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1729923481 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f6ea001a070 con 0x560945ee6440 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.894+0000 7f6e90ff9700 10 client.28978 handle_command_reply: tid=0 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.894+0000 7f6eb9c75b80 1 client.28978 shutdown 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.894+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ecc970 msgr2=0x560945ed8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ecc970 0x560945ed8e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40a6d50 tx=0x7f6ea40b2280 comp rx=0 tx=0).stop 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 shutdown_connections 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945ee8860 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6e90ff9700 0 client.28978 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ed2340 0x560945ed66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ecc970 0x560945ed8e50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1729923481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecd330 0x560945ed61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 >> 172.21.15.26:0/1729923481 conn(0x560945dfa210 msgr2=0x560945ec1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 shutdown_connections 2024-05-20T21:11:42.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1729923481 wait complete. 2024-05-20T21:11:42.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.895+0000 7f6eb9c75b80 20 client.28978 trim_cache size 0 max 16384 2024-05-20T21:11:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.932+0000 7f6eb9c75b80 1 Processor -- start 2024-05-20T21:11:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.932+0000 7f6eb9c75b80 1 -- start start 2024-05-20T21:11:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945ed6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ee2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81b25a0 con 0x560945ecc970 2024-05-20T21:11:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81afda0 con 0x560945ecd330 2024-05-20T21:11:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81adbf0 con 0x560945ed2340 2024-05-20T21:11:42.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eac97a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ee2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6ead17b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945ed6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eac97a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ee2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49924/0 (socket says 172.21.15.26:49924) 2024-05-20T21:11:42.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6eac97a700 1 -- 172.21.15.26:0/3650993881 learned_addr learned my addr 172.21.15.26:0/3650993881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:42.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6ead97c700 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6ead97c700 1 -- 172.21.15.26:0/3650993881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 msgr2=0x560945ee2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.933+0000 7f6ead97c700 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ee2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6ead97c700 1 -- 172.21.15.26:0/3650993881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 msgr2=0x560945ed6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6ead97c700 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945ed6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6ead97c700 1 -- 172.21.15.26:0/3650993881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560945ec9450 con 0x560945ed2340 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6ead17b700 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945ed6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6ead97c700 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed8990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00047d0 tx=0x7f6ea0009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c151b71c325598e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6e92ffd700 1 -- 172.21.15.26:0/3650993881 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0003c00 con 0x560945ed2340 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6e92ffd700 1 -- 172.21.15.26:0/3650993881 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ea001a070 con 0x560945ed2340 2024-05-20T21:11:42.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.934+0000 7f6e92ffd700 1 -- 172.21.15.26:0/3650993881 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0004340 con 0x560945ed2340 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/3650993881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 msgr2=0x560945ed8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed8990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00047d0 tx=0x7f6ea0009f20 comp rx=0 tx=0).stop 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/3650993881 shutdown_connections 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945ed6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945ed8990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/3650993881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945ee2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/3650993881 >> 172.21.15.26:0/3650993881 conn(0x560945dfa210 msgr2=0x560945ed06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/3650993881 shutdown_connections 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.935+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/3650993881 wait complete. 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 -- start start 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 Processor -- start 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945edf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945edb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945edbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae260 con 0x560945ecc970 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81af6d0 con 0x560945ecd330 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae930 con 0x560945ed2340 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.936+0000 7f6eb9c75b80 10 client.0 _ll_register_callbacks cb 0x560945db6ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945edb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6ead17b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945edf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945edb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:49940/0 (socket says 172.21.15.26:49940) 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6ead17b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945edf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59072/0 (socket says 172.21.15.26:59072) 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 -- 172.21.15.26:0/1064856709 learned_addr learned my addr 172.21.15.26:0/1064856709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6ead97c700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945edbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 -- 172.21.15.26:0/1064856709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 msgr2=0x560945edbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945edbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 -- 172.21.15.26:0/1064856709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 msgr2=0x560945edf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945edf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 -- 172.21.15.26:0/1064856709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560945edc740 con 0x560945ecd330 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945edf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6ead97c700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945edbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6eac97a700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945edb910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6e980099c0 tx=0x7f6e98009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db122e1fa0df2674 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.937+0000 7f6e90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:11:42.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1064856709 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98003a90 con 0x560945ecd330 2024-05-20T21:11:42.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1064856709 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6e98003c30 con 0x560945ecd330 2024-05-20T21:11:42.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea001a070 con 0x560945ecd330 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1064856709 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6e98003e30 con 0x560945ecd330 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6eb9c75b80 10 client.29294 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560945f521b0 con 0x560945ecd330 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1064856709 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e98037070 con 0x560945ecd330 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.938+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1064856709 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6e9803c6a0 con 0x560945ecd330 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6eb9c75b80 10 client.29294 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6eb9c75b80 10 client.29294 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6eb9c75b80 20 client.29294 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945f54990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6eb9c75b80 4 client.29294 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-05-20T21:11:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x7f6ea001a070 con 0x560945ee6440 2024-05-20T21:11:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945f54990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6e90ff9700 10 client.29294 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:42.939+0000 7f6ead17b700 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945f54990 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f5fb8b76d67773 server_cookie=e8715ebf3444e0c6 in_seq=0 out_seq=0 2024-05-20T21:11:43.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17207 10.21 MB/sec warmup 108 sec latency 229.193 ms 2024-05-20T21:11:43.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:43 smithi055 ceph-mon[35213]: pgmap v857: 97 pgs: 97 active+clean; 255 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-05-20T21:11:43.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:43 smithi026 ceph-mon[31990]: pgmap v857: 97 pgs: 97 active+clean; 255 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6e90ff9700 1 -- 172.21.15.26:0/1064856709 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6ea001a070 con 0x560945ee6440 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6e90ff9700 10 client.29294 handle_command_reply: tid=0 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 client.29294 shutdown 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 msgr2=0x560945edb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945edb910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6e980099c0 tx=0x7f6e98009cd0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 shutdown_connections 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x560945ee6440 0x560945f54990 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560945ecc970 0x560945edf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560945ed2340 0x560945edbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6e90ff9700 0 client.29294 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 --2- 172.21.15.26:0/1064856709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560945ecd330 0x560945edb910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.485+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 >> 172.21.15.26:0/1064856709 conn(0x560945dfa210 msgr2=0x560945ecfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 shutdown_connections 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eb9c75b80 1 -- 172.21.15.26:0/1064856709 wait complete. 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eb9c75b80 20 client.29294 trim_cache size 0 max 16384 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 34, 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 54, 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:11:43.487 INFO:teuthology.orchestra.run.smithi026.stdout: "1fa160df": { 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 54, 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.4, 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:11:43.488 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:11:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e940b2930 msgr2=0x7f6e940b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e940b2930 0x7f6e940b4df0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6ea819ae30 tx=0x7f6e98000bc0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 msgr2=0x7f6ea819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.486+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea819f5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e9c0 tx=0x7f6ea400ecd0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 shutdown_connections 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6e940b2930 0x7f6e940b4df0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8108900 0x7f6ea81996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea806b480 0x7f6ea819f5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 --2- 172.21.15.26:0/2055080769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ea806be30 0x7f6ea819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 >> 172.21.15.26:0/2055080769 conn(0x7f6ea80793e0 msgr2=0x7f6ea8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 shutdown_connections 2024-05-20T21:11:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:43.487+0000 7f6eaf3df700 1 -- 172.21.15.26:0/2055080769 wait complete. 2024-05-20T21:11:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:43 smithi144 ceph-mon[35660]: pgmap v857: 97 pgs: 97 active+clean; 255 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-05-20T21:11:44.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 10.17 MB/sec warmup 109 sec latency 1155.782 ms 2024-05-20T21:11:44.348 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1fa160df', quiesce took 18.2 seconds. Will release after: 2.1999999999999993 2024-05-20T21:11:45.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 10.08 MB/sec warmup 110 sec latency 2155.902 ms 2024-05-20T21:11:45.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:45 smithi055 ceph-mon[35213]: pgmap v858: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-05-20T21:11:45.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:45 smithi026 ceph-mon[31990]: pgmap v858: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-05-20T21:11:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:45 smithi144 ceph-mon[35660]: pgmap v858: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-05-20T21:11:46.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 9.99 MB/sec warmup 111 sec latency 3156.003 ms 2024-05-20T21:11:46.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:11:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:47.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 9.90 MB/sec warmup 112 sec latency 4156.062 ms 2024-05-20T21:11:47.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:47 smithi026 ceph-mon[31990]: pgmap v859: 97 pgs: 97 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-05-20T21:11:47.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:47 smithi055 ceph-mon[35213]: pgmap v859: 97 pgs: 97 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-05-20T21:11:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:47 smithi144 ceph-mon[35660]: pgmap v859: 97 pgs: 97 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-05-20T21:11:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 -- 172.21.15.26:0/1550863132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 msgr2=0x7f7a641030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 --2- 172.21.15.26:0/1550863132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a641030d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009900 tx=0x7f7a4c02a410 comp rx=0 tx=0).stop 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 -- 172.21.15.26:0/1550863132 shutdown_connections 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 --2- 172.21.15.26:0/1550863132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a64103610 0x7f7a64109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 --2- 172.21.15.26:0/1550863132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a641030d0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 --2- 172.21.15.26:0/1550863132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a64101a60 0x7f7a64101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 -- 172.21.15.26:0/1550863132 >> 172.21.15.26:0/1550863132 conn(0x7f7a640fcfd0 msgr2=0x7f7a640ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 -- 172.21.15.26:0/1550863132 shutdown_connections 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 -- 172.21.15.26:0/1550863132 wait complete. 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.631+0000 7f7a694f8640 1 Processor -- start 2024-05-20T21:11:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 -- start start 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a64101a60 0x7f7a6406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a6419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a64103610 0x7f7a6419ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6410d3d0 con 0x7f7a64102c50 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6410d250 con 0x7f7a64101a60 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a694f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6410d550 con 0x7f7a64103610 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a6419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a6419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59086/0 (socket says 172.21.15.26:59086) 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/3110663938 learned_addr learned my addr 172.21.15.26:0/3110663938 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a637fe640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a64103610 0x7f7a6419ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a64101a60 0x7f7a6406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/3110663938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a64103610 msgr2=0x7f7a6419ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a64103610 0x7f7a6419ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/3110663938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a64101a60 msgr2=0x7f7a6406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a64101a60 0x7f7a6406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.632+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/3110663938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a641a0820 con 0x7f7a64102c50 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.633+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a6419d8f0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009230 tx=0x7f7a4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.633+0000 7f7a60ff9640 1 -- 172.21.15.26:0/3110663938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c03f030 con 0x7f7a64102c50 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.633+0000 7f7a60ff9640 1 -- 172.21.15.26:0/3110663938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a4c003c90 con 0x7f7a64102c50 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.633+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a641a0a50 con 0x7f7a64102c50 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.633+0000 7f7a60ff9640 1 -- 172.21.15.26:0/3110663938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c035070 con 0x7f7a64102c50 2024-05-20T21:11:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.633+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a641a0f40 con 0x7f7a64102c50 2024-05-20T21:11:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 Processor -- start 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a60ff9640 1 -- 172.21.15.26:0/3110663938 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7a4c042030 con 0x7f7a64102c50 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 -- start start 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba7fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba84bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee2b9f0c30 con 0x55ee2ba800b0 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee2b9e07d0 con 0x55ee2ba85220 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a6c210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee2b9e4810 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba84bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba84bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59102/0 (socket says 172.21.15.26:59102) 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a62ffd640 1 -- 172.21.15.26:0/101558857 learned_addr learned my addr 172.21.15.26:0/101558857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.636+0000 7f7a637fe640 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba7fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a62ffd640 1 -- 172.21.15.26:0/101558857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 msgr2=0x55ee2ba7fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba7fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a62ffd640 1 -- 172.21.15.26:0/101558857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 msgr2=0x55ee2ba876b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba876b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a62ffd640 1 -- 172.21.15.26:0/101558857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2b6f5210 con 0x55ee2ba800b0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba84bb0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400bcb0 tx=0x7f7a540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98420da5d04addb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a60ff9640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a500b28a0 0x7f7a500b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a597fa640 1 -- 172.21.15.26:0/101558857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54005020 con 0x55ee2ba800b0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a597fa640 1 -- 172.21.15.26:0/101558857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a540047a0 con 0x55ee2ba800b0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a597fa640 1 -- 172.21.15.26:0/101558857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011640 con 0x55ee2ba800b0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a637fe640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a500b28a0 0x7f7a500b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.637+0000 7f7a60ff9640 1 -- 172.21.15.26:0/3110663938 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7a4c102930 con 0x7f7a64102c50 2024-05-20T21:11:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 -- 172.21.15.26:0/101558857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 msgr2=0x55ee2ba84bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a637fe640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a500b28a0 0x7f7a500b4d60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55ee2ba80910 tx=0x7f7a5c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba84bb0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400bcb0 tx=0x7f7a540107a0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 -- 172.21.15.26:0/101558857 shutdown_connections 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba876b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba84bb0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 --2- 172.21.15.26:0/101558857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba7fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 -- 172.21.15.26:0/101558857 >> 172.21.15.26:0/101558857 conn(0x55ee2ba7ad30 msgr2=0x55ee2ba7b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 -- 172.21.15.26:0/101558857 shutdown_connections 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 -- 172.21.15.26:0/101558857 wait complete. 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 -- start start 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 Processor -- start 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.638+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba800b0 0x55ee2ba89de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba85220 0x55ee2ba90db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee2b6c6790 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee2b99c890 con 0x55ee2ba800b0 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee2ba881d0 con 0x55ee2ba85220 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 10 client.0 _ll_register_callbacks cb 0x55ee2b6f4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba898a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59114/0 (socket says 172.21.15.26:59114) 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 -- 172.21.15.26:0/3763794642 learned_addr learned my addr 172.21.15.26:0/3763794642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba800b0 0x55ee2ba89de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a637fe640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba85220 0x55ee2ba90db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 -- 172.21.15.26:0/3763794642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba85220 msgr2=0x55ee2ba90db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba85220 0x55ee2ba90db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 -- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba800b0 msgr2=0x55ee2ba89de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba800b0 0x55ee2ba89de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 -- 172.21.15.26:0/3763794642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2b7c1620 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba800b0 0x55ee2ba89de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba898a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7a54011010 tx=0x7f7a5400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9694b36a1ecd80a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/3763794642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5402b060 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/3763794642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7a54004b60 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a540047a0 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/3763794642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a54005000 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.639+0000 7f7a6c210740 10 client.20103 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee2ba93d90 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/3763794642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a540047a0 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/3763794642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5402f030 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 10 client.20103 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 10 client.20103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 20 client.20103 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2ba95190 0x55ee2ba975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 4 client.20103 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee2b9f2b80 con 0x55ee2ba95190 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.640+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2ba95190 0x55ee2ba975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.641+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2ba95190 0x55ee2ba975a0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1071ac1aec81db server_cookie=91df6d987fad18c1 in_seq=0 out_seq=0 2024-05-20T21:11:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.641+0000 7f7a4a7fc640 10 client.20103 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.641+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/3763794642 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ee2b9f2b80 con 0x55ee2ba95190 2024-05-20T21:11:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.641+0000 7f7a4a7fc640 10 client.20103 handle_command_reply: tid=0 2024-05-20T21:11:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.641+0000 7f7a6c210740 1 client.20103 shutdown 2024-05-20T21:11:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 msgr2=0x55ee2ba898a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba898a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7a54011010 tx=0x7f7a5400bfd0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 shutdown_connections 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2ba95190 0x55ee2ba975a0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba85220 0x55ee2ba90db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a4a7fc640 0 client.20103 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba800b0 0x55ee2ba89de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 --2- 172.21.15.26:0/3763794642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba898a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 >> 172.21.15.26:0/3763794642 conn(0x55ee2ba7ad30 msgr2=0x55ee2ba79fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 shutdown_connections 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 1 -- 172.21.15.26:0/3763794642 wait complete. 2024-05-20T21:11:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.642+0000 7f7a6c210740 20 client.20103 trim_cache size 0 max 16384 2024-05-20T21:11:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 Processor -- start 2024-05-20T21:11:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 -- start start 2024-05-20T21:11:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba88ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba90d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba89400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641ac830 con 0x55ee2ba800b0 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641aa070 con 0x55ee2ba85220 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a6c210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641a7ec0 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.657+0000 7f7a5a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba90d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba90d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59128/0 (socket says 172.21.15.26:59128) 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/2162006539 learned_addr learned my addr 172.21.15.26:0/2162006539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba88ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a637fe640 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba89400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/2162006539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 msgr2=0x55ee2ba88ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba88ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/2162006539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 msgr2=0x55ee2ba89400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba89400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 -- 172.21.15.26:0/2162006539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2ba880b0 con 0x55ee2ba800b0 2024-05-20T21:11:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba90d70 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0c9d60 tx=0x7f7a4c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a49562d204c8e45 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a58ff9640 1 -- 172.21.15.26:0/2162006539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c049040 con 0x55ee2ba800b0 2024-05-20T21:11:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a58ff9640 1 -- 172.21.15.26:0/2162006539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a4c003e40 con 0x55ee2ba800b0 2024-05-20T21:11:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.658+0000 7f7a58ff9640 1 -- 172.21.15.26:0/2162006539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0d2e10 con 0x55ee2ba800b0 2024-05-20T21:11:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 -- 172.21.15.26:0/2162006539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 msgr2=0x55ee2ba90d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba90d70 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0c9d60 tx=0x7f7a4c005e60 comp rx=0 tx=0).stop 2024-05-20T21:11:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 -- 172.21.15.26:0/2162006539 shutdown_connections 2024-05-20T21:11:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba89400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba800b0 0x55ee2ba90d70 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 --2- 172.21.15.26:0/2162006539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba88ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 -- 172.21.15.26:0/2162006539 >> 172.21.15.26:0/2162006539 conn(0x55ee2ba7ad30 msgr2=0x55ee2ba79fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 -- 172.21.15.26:0/2162006539 shutdown_connections 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.659+0000 7f7a6c210740 1 -- 172.21.15.26:0/2162006539 wait complete. 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 -- start start 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 Processor -- start 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba8e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba800b0 0x55ee2ba8e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641a8530 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641a99a0 con 0x55ee2ba85220 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641a8c00 con 0x55ee2ba800b0 2024-05-20T21:11:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a6c210740 10 client.0 _ll_register_callbacks cb 0x7f7a4c108420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba8e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba8e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59140/0 (socket says 172.21.15.26:59140) 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a62ffd640 1 -- 172.21.15.26:0/4198828193 learned_addr learned my addr 172.21.15.26:0/4198828193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba800b0 0x55ee2ba8e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.660+0000 7f7a637fe640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a62ffd640 1 -- 172.21.15.26:0/4198828193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba800b0 msgr2=0x55ee2ba8e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba800b0 0x55ee2ba8e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a62ffd640 1 -- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 msgr2=0x55ee2ba8f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba8f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a62ffd640 1 -- 172.21.15.26:0/4198828193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2ba8fea0 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a62ffd640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba8e3b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7a54013010 tx=0x7f7a54011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14810424726a36bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/4198828193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5402b070 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/4198828193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f7a540051a0 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee2b9ec360 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/4198828193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a54004780 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 10 client.20109 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee2ba911b0 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/4198828193 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ee2b9ec360 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/4198828193 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7a54041020 con 0x55ee2ba7f6c0 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 10 client.20109 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 10 client.20109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 20 client.20109 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.661+0000 7f7a6c210740 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2bb00510 0x55ee2bb02930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.662+0000 7f7a6c210740 4 client.20109 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.662+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee2b9665f0 con 0x55ee2bb00510 2024-05-20T21:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.662+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2bb00510 0x55ee2bb02930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.662+0000 7f7a4a7fc640 10 client.20109 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.662+0000 7f7a5a7fc640 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2bb00510 0x55ee2bb02930 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83f3278191d17c server_cookie=8cc59dd750e742bc in_seq=0 out_seq=0 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a4a7fc640 1 -- 172.21.15.26:0/4198828193 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ee2b9665f0 con 0x55ee2bb00510 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a4a7fc640 10 client.20109 handle_command_reply: tid=0 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 client.20109 shutdown 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 msgr2=0x55ee2ba8e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba8e3b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7a54013010 tx=0x7f7a54011670 comp rx=0 tx=0).stop 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 shutdown_connections 2024-05-20T21:11:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55ee2bb00510 0x55ee2bb02930 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee2ba85220 0x55ee2ba8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a4a7fc640 0 client.20109 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2ba800b0 0x55ee2ba8e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 --2- 172.21.15.26:0/4198828193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee2ba7f6c0 0x55ee2ba8e3b0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 >> 172.21.15.26:0/4198828193 conn(0x55ee2ba7ad30 msgr2=0x55ee2ba79fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 shutdown_connections 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.737+0000 7f7a6c210740 1 -- 172.21.15.26:0/4198828193 wait complete. 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a6c210740 20 client.20109 trim_cache size 0 max 16384 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:11:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:11:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:11:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a500b28a0 msgr2=0x7f7a500b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a500b28a0 0x7f7a500b4d60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55ee2ba80910 tx=0x7f7a5c009600 comp rx=0 tx=0).stop 2024-05-20T21:11:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 msgr2=0x7f7a6419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a6419d8f0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009230 tx=0x7f7a4c0042f0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 shutdown_connections 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f7a500b28a0 0x7f7a500b4d60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a64103610 0x7f7a6419ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a64102c50 0x7f7a6419d8f0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 --2- 172.21.15.26:0/3110663938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a64101a60 0x7f7a6406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.738+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 >> 172.21.15.26:0/3110663938 conn(0x7f7a640fcfd0 msgr2=0x7f7a64105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.739+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 shutdown_connections 2024-05-20T21:11:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:11:47.739+0000 7f7a694f8640 1 -- 172.21.15.26:0/3110663938 wait complete. 2024-05-20T21:11:48.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 9.81 MB/sec warmup 113 sec latency 5156.137 ms 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2165890977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a01095b0 msgr2=0x7fd0a0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2165890977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a01095b0 0x7fd0a0109990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd090009990 tx=0x7fd090009ca0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2165890977 shutdown_connections 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2165890977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a01095b0 0x7fd0a0109990 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2165890977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a006bbe0 0x7fd0a01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2165890977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a006b220 0x7fd0a006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2165890977 >> 172.21.15.26:0/2165890977 conn(0x7fd0a00793f0 msgr2=0x7fd0a0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2165890977 shutdown_connections 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.648+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2165890977 wait complete. 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 Processor -- start 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 -- start start 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 0x7fd0a0102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a006bbe0 0x7fd0a0102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a01095b0 0x7fd0a0102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0113570 con 0x7fd0a006b220 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01133f0 con 0x7fd0a01095b0 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.649+0000 7fd0a5c16700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01136f0 con 0x7fd0a006bbe0 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 0x7fd0a0102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 0x7fd0a0102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59168/0 (socket says 172.21.15.26:59168) 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 -- 172.21.15.26:0/2681314442 learned_addr learned my addr 172.21.15.26:0/2681314442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd096ffd700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a006bbe0 0x7fd0a0102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:48.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09ffff700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a01095b0 0x7fd0a0102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 -- 172.21.15.26:0/2681314442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a006bbe0 msgr2=0x7fd0a0102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a006bbe0 0x7fd0a0102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 -- 172.21.15.26:0/2681314442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a01095b0 msgr2=0x7fd0a0102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a01095b0 0x7fd0a0102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 -- 172.21.15.26:0/2681314442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a01a8050 con 0x7fd0a006b220 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09ffff700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a01095b0 0x7fd0a0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09f7fe700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 0x7fd0a0102510 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd090000c00 tx=0x7fd0900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090027070 con 0x7fd0a006b220 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a01a82e0 con 0x7fd0a006b220 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd09000fb80 con 0x7fd0a006b220 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.650+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090022d00 con 0x7fd0a006b220 2024-05-20T21:11:48.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.651+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01a8740 con 0x7fd0a006b220 2024-05-20T21:11:48.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.651+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a010ba90 con 0x7fd0a006b220 2024-05-20T21:11:48.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.653+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd09002a070 con 0x7fd0a006b220 2024-05-20T21:11:48.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.654+0000 7fd09d7fa700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd08c0b2880 0x7fd08c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:48.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.654+0000 7fd096ffd700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd08c0b2880 0x7fd08c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:48.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.654+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd0900e3550 con 0x7fd0a006b220 2024-05-20T21:11:48.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.654+0000 7fd096ffd700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd08c0b2880 0x7fd08c0b4d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0079b30 tx=0x7fd08800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.657+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd090031070 con 0x7fd0a006b220 2024-05-20T21:11:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.928+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a0103a40 con 0x7fd0a006b220 2024-05-20T21:11:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.929+0000 7fd09d7fa700 1 -- 172.21.15.26:0/2681314442 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11874 (secure 0 0 0) 0x7fd090013070 con 0x7fd0a006b220 2024-05-20T21:11:48.929 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:11:48.929 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:10:11.369568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd08c0b2880 msgr2=0x7fd08c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd08c0b2880 0x7fd08c0b4d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0079b30 tx=0x7fd08800a300 comp rx=0 tx=0).stop 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 msgr2=0x7fd0a0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 0x7fd0a0102510 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd090000c00 tx=0x7fd0900047b0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 shutdown_connections 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd08c0b2880 0x7fd08c0b4d40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a006b220 0x7fd0a0102510 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a01095b0 0x7fd0a0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 --2- 172.21.15.26:0/2681314442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a006bbe0 0x7fd0a0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 >> 172.21.15.26:0/2681314442 conn(0x7fd0a00793f0 msgr2=0x7fd0a0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 shutdown_connections 2024-05-20T21:11:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:48.931+0000 7fd0a5c16700 1 -- 172.21.15.26:0/2681314442 wait complete. 2024-05-20T21:11:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 34 2024-05-20T21:11:49.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 9.73 MB/sec warmup 114 sec latency 6156.183 ms 2024-05-20T21:11:49.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:49 smithi026 ceph-mon[31990]: pgmap v860: 97 pgs: 97 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-05-20T21:11:49.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:49 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2681314442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:49.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:49 smithi055 ceph-mon[35213]: pgmap v860: 97 pgs: 97 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-05-20T21:11:49.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:49 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2681314442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:49 smithi144 ceph-mon[35660]: pgmap v860: 97 pgs: 97 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-05-20T21:11:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:49 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2681314442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:49.739 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 1fa160df --release --await' 2024-05-20T21:11:49.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 1fa160df --release --await 2024-05-20T21:11:50.038 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:50.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 9.64 MB/sec warmup 115 sec latency 7156.212 ms 2024-05-20T21:11:51.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17238 9.56 MB/sec warmup 116 sec latency 8156.243 ms 2024-05-20T21:11:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:51 smithi026 ceph-mon[31990]: pgmap v861: 97 pgs: 97 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 3.4 MiB/s wr, 182 op/s 2024-05-20T21:11:51.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:51 smithi055 ceph-mon[35213]: pgmap v861: 97 pgs: 97 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 3.4 MiB/s wr, 182 op/s 2024-05-20T21:11:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:51 smithi144 ceph-mon[35660]: pgmap v861: 97 pgs: 97 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 3.4 MiB/s wr, 182 op/s 2024-05-20T21:11:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 -- 172.21.15.26:0/2328249219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406b220 msgr2=0x7fbda406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2328249219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406b220 0x7fbda406b6a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009990 tx=0x7fbd98009ca0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 -- 172.21.15.26:0/2328249219 shutdown_connections 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2328249219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda406bbe0 0x7fbda41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2328249219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 0x7fbda4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2328249219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406b220 0x7fbda406b6a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 -- 172.21.15.26:0/2328249219 >> 172.21.15.26:0/2328249219 conn(0x7fbda40793f0 msgr2=0x7fbda4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 -- 172.21.15.26:0/2328249219 shutdown_connections 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.855+0000 7fbdaa091700 1 -- 172.21.15.26:0/2328249219 wait complete. 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 Processor -- start 2024-05-20T21:11:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 -- start start 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda406b220 0x7fbda419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 0x7fbda419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 0x7fbda4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41136c0 con 0x7fbda406bbe0 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbda4113540 con 0x7fbda41095b0 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbdaa091700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbda4113840 con 0x7fbda406b220 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.856+0000 7fbda2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 0x7fbda419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 0x7fbda419ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32874/0 (socket says 172.21.15.26:32874) 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 -- 172.21.15.26:0/2064098874 learned_addr learned my addr 172.21.15.26:0/2064098874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda3fff700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 0x7fbda4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda37fe700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda406b220 0x7fbda419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 -- 172.21.15.26:0/2064098874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda406b220 msgr2=0x7fbda419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda406b220 0x7fbda419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 -- 172.21.15.26:0/2064098874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 msgr2=0x7fbda4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 0x7fbda4199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 -- 172.21.15.26:0/2064098874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda419a440 con 0x7fbda406bbe0 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda3fff700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 0x7fbda4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 0x7fbda419ff80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbd98005910 tx=0x7fbd98009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda0ff9700 1 -- 172.21.15.26:0/2064098874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98027070 con 0x7fbda406bbe0 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda0ff9700 1 -- 172.21.15.26:0/2064098874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd9800fce0 con 0x7fbda406bbe0 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda419a6d0 con 0x7fbda406bbe0 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbda0ff9700 1 -- 172.21.15.26:0/2064098874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd980173f0 con 0x7fbda406bbe0 2024-05-20T21:11:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.857+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda41a67f0 con 0x7fbda406bbe0 2024-05-20T21:11:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.860+0000 7fbda0ff9700 1 -- 172.21.15.26:0/2064098874 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbd9802a070 con 0x7fbda406bbe0 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbda0ff9700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbd8c0b2880 0x7fbd8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbda37fe700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbd8c0b2880 0x7fbd8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbda0ff9700 1 -- 172.21.15.26:0/2064098874 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fbd980e3310 con 0x7fbda406bbe0 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 Processor -- start 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbda37fe700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbd8c0b2880 0x7fbd8c0b4d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006fd0 tx=0x7fbd94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 -- start start 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 0x55d28b81eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b818cf0 0x55d28b819170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b825450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d28b773d90 con 0x55d28b81e740 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d28b763930 con 0x55d28b8196b0 2024-05-20T21:11:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.861+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d28b767970 con 0x55d28b818cf0 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 0x55d28b81eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 0x55d28b81eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32880/0 (socket says 172.21.15.26:32880) 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 -- 172.21.15.26:0/972116588 learned_addr learned my addr 172.21.15.26:0/972116588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b818cf0 0x55d28b819170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 -- 172.21.15.26:0/972116588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b818cf0 msgr2=0x55d28b819170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b818cf0 0x55d28b819170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 -- 172.21.15.26:0/972116588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 msgr2=0x55d28b825450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda3fff700 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b825450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b825450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 -- 172.21.15.26:0/972116588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d28b6b39d0 con 0x55d28b81e740 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b818cf0 0x55d28b819170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbda37fe700 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 0x55d28b81eb20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d750 tx=0x7fbd94010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd455b3876a4a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbd89ffb700 1 -- 172.21.15.26:0/972116588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94024070 con 0x55d28b81e740 2024-05-20T21:11:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbd89ffb700 1 -- 172.21.15.26:0/972116588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd94014ad0 con 0x55d28b81e740 2024-05-20T21:11:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.862+0000 7fbd89ffb700 1 -- 172.21.15.26:0/972116588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9401fa70 con 0x55d28b81e740 2024-05-20T21:11:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 -- 172.21.15.26:0/972116588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 msgr2=0x55d28b81eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 0x55d28b81eb20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d750 tx=0x7fbd94010000 comp rx=0 tx=0).stop 2024-05-20T21:11:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 -- 172.21.15.26:0/972116588 shutdown_connections 2024-05-20T21:11:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b81e740 0x55d28b81eb20 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b825450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/972116588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b818cf0 0x55d28b819170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 -- 172.21.15.26:0/972116588 >> 172.21.15.26:0/972116588 conn(0x55d28b742180 msgr2=0x55d28b80f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 -- 172.21.15.26:0/972116588 shutdown_connections 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.863+0000 7fbdb4927b80 1 -- 172.21.15.26:0/972116588 wait complete. 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 -- start start 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 Processor -- start 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b825170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b8224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b822a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d28b66a0a0 con 0x55d28b818cf0 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d28b7058c0 con 0x55d28b8196b0 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d28b825bd0 con 0x55d28b81e740 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbdb4927b80 10 client.0 _ll_register_callbacks cb 0x55d28b6fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b825170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b825170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32896/0 (socket says 172.21.15.26:32896) 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbda37fe700 1 -- 172.21.15.26:0/232791236 learned_addr learned my addr 172.21.15.26:0/232791236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b8224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.864+0000 7fbda3fff700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b822a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbda37fe700 1 -- 172.21.15.26:0/232791236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 msgr2=0x55d28b822a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbda37fe700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b822a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbda37fe700 1 -- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 msgr2=0x55d28b8224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbda37fe700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b8224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbda37fe700 1 -- 172.21.15.26:0/232791236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28b702790 con 0x55d28b818cf0 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbda37fe700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b825170 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d440 tx=0x7fbd94014990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b908e9510c0260e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbd82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbd82ffd700 1 -- 172.21.15.26:0/232791236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9402e030 con 0x55d28b818cf0 2024-05-20T21:11:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbd82ffd700 1 -- 172.21.15.26:0/232791236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fbd9400f040 con 0x55d28b818cf0 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd94014ad0 con 0x55d28b818cf0 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbd82ffd700 1 -- 172.21.15.26:0/232791236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbd94038050 con 0x55d28b818cf0 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbdb4927b80 10 client.20127 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.865+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d28b830380 con 0x55d28b818cf0 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbd82ffd700 1 -- 172.21.15.26:0/232791236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd94014ad0 con 0x55d28b818cf0 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbd82ffd700 1 -- 172.21.15.26:0/232791236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fbd94027030 con 0x55d28b818cf0 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbdb4927b80 10 client.20127 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbdb4927b80 10 client.20127 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbdb4927b80 20 client.20127 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b832960 0x55d28b834d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbdb4927b80 4 client.20127 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d28b7e0ae0 con 0x55d28b832960 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.866+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b832960 0x55d28b834d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.867+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b832960 0x55d28b834d50 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd79e39124b20c1b server_cookie=3b7d29a63e7efdf3 in_seq=0 out_seq=0 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.867+0000 7fbd82ffd700 10 client.20127 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.867+0000 7fbd82ffd700 1 -- 172.21.15.26:0/232791236 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d28b7e0ae0 con 0x55d28b832960 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.867+0000 7fbd82ffd700 10 client.20127 handle_command_reply: tid=0 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.867+0000 7fbdb4927b80 1 client.20127 shutdown 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 msgr2=0x55d28b825170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b825170 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d440 tx=0x7fbd94014990 comp rx=0 tx=0).stop 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 shutdown_connections 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b832960 0x55d28b834d50 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b825170 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b822a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/232791236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b8224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbd82ffd700 0 client.20127 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 >> 172.21.15.26:0/232791236 conn(0x55d28b742180 msgr2=0x55d28b809ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 shutdown_connections 2024-05-20T21:11:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 1 -- 172.21.15.26:0/232791236 wait complete. 2024-05-20T21:11:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.868+0000 7fbdb4927b80 20 client.20127 trim_cache size 0 max 16384 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 Processor -- start 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 -- start start 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b82eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b83a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b83aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41ae3b0 con 0x55d28b818cf0 2024-05-20T21:11:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41abbb0 con 0x55d28b8196b0 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41a9a00 con 0x55d28b81e740 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.888+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b82eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b82eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32904/0 (socket says 172.21.15.26:32904) 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 -- 172.21.15.26:0/126896600 learned_addr learned my addr 172.21.15.26:0/126896600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b83a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 -- 172.21.15.26:0/126896600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 msgr2=0x55d28b83aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b83aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 -- 172.21.15.26:0/126896600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 msgr2=0x55d28b83a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b83a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 -- 172.21.15.26:0/126896600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d28b80f730 con 0x55d28b818cf0 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbda37fe700 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b82eca0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009210 tx=0x7fbd94009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6915965270049d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbd897fa700 1 -- 172.21.15.26:0/126896600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9402e070 con 0x55d28b818cf0 2024-05-20T21:11:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbd897fa700 1 -- 172.21.15.26:0/126896600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd94009600 con 0x55d28b818cf0 2024-05-20T21:11:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.889+0000 7fbd897fa700 1 -- 172.21.15.26:0/126896600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94024040 con 0x55d28b818cf0 2024-05-20T21:11:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 -- 172.21.15.26:0/126896600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 msgr2=0x55d28b82eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b82eca0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009210 tx=0x7fbd94009e60 comp rx=0 tx=0).stop 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 -- 172.21.15.26:0/126896600 shutdown_connections 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b82eca0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b83aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/126896600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b83a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 -- 172.21.15.26:0/126896600 >> 172.21.15.26:0/126896600 conn(0x55d28b742180 msgr2=0x55d28b814160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 -- 172.21.15.26:0/126896600 shutdown_connections 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 -- 172.21.15.26:0/126896600 wait complete. 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 -- start start 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.890+0000 7fbdb4927b80 1 Processor -- start 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b828f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b829450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b89c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41aa070 con 0x55d28b818cf0 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41ab4e0 con 0x55d28b8196b0 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41aa740 con 0x55d28b81e740 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbdb4927b80 10 client.0 _ll_register_callbacks cb 0x55d28b6fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b828f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b828f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32920/0 (socket says 172.21.15.26:32920) 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 -- 172.21.15.26:0/1658718595 learned_addr learned my addr 172.21.15.26:0/1658718595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b829450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 -- 172.21.15.26:0/1658718595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 msgr2=0x55d28b89c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b89c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 -- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 msgr2=0x55d28b829450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b829450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 -- 172.21.15.26:0/1658718595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28b829d10 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbda37fe700 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b828f10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbd94008010 tx=0x7fbd9402a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2061c3ae63653ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbd82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.891+0000 7fbd82ffd700 1 -- 172.21.15.26:0/1658718595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94024070 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd94009600 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbd82ffd700 1 -- 172.21.15.26:0/1658718595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fbd94014b80 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbd82ffd700 1 -- 172.21.15.26:0/1658718595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbd9404e050 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 10 client.20133 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d28b89da40 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbd82ffd700 1 -- 172.21.15.26:0/1658718595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd94009600 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbd82ffd700 1 -- 172.21.15.26:0/1658718595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fbd94027040 con 0x55d28b818cf0 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 10 client.20133 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 10 client.20133 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:11:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 20 client.20133 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b89e040 0x55d28b8a0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 4 client.20133 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1fa160df", "release": true, "await": true}] 2024-05-20T21:11:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1fa160df", "release": true, "await": true}) -- 0x55d28b775ce0 con 0x55d28b89e040 2024-05-20T21:11:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.892+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b89e040 0x55d28b8a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.893+0000 7fbda2ffd700 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b89e040 0x55d28b8a0460 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0442a8495797deb server_cookie=c8c49579a153c00c in_seq=0 out_seq=0 2024-05-20T21:11:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.893+0000 7fbd82ffd700 10 client.20133 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbd82ffd700 1 -- 172.21.15.26:0/1658718595 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d28b775ce0 con 0x55d28b89e040 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbd82ffd700 10 client.20133 handle_command_reply: tid=0 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 client.20133 shutdown 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 msgr2=0x55d28b828f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b828f10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbd94008010 tx=0x7fbd9402a4d0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 shutdown_connections 2024-05-20T21:11:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d28b89e040 0x55d28b8a0460 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d28b818cf0 0x55d28b828f10 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d28b81e740 0x55d28b89c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 --2- 172.21.15.26:0/1658718595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d28b8196b0 0x55d28b829450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 >> 172.21.15.26:0/1658718595 conn(0x55d28b742180 msgr2=0x55d28b809410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbd82ffd700 0 client.20133 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 shutdown_connections 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 1 -- 172.21.15.26:0/1658718595 wait complete. 2024-05-20T21:11:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.955+0000 7fbdb4927b80 20 client.20133 trim_cache size 0 max 16384 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 34, 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 56, 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "1fa160df": { 2024-05-20T21:11:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 56, 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.4, 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:11:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:51.960 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:11:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbd8c0b2880 msgr2=0x7fbd8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbd8c0b2880 0x7fbd8c0b4d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006fd0 tx=0x7fbd94008040 comp rx=0 tx=0).stop 2024-05-20T21:11:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 msgr2=0x7fbda419ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 0x7fbda419ff80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbd98005910 tx=0x7fbd98009f90 comp rx=0 tx=0).stop 2024-05-20T21:11:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 shutdown_connections 2024-05-20T21:11:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbda41095b0 0x7fbda4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda406b220 0x7fbda419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fbd8c0b2880 0x7fbd8c0b4d40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 --2- 172.21.15.26:0/2064098874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda406bbe0 0x7fbda419ff80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:51.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.956+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 >> 172.21.15.26:0/2064098874 conn(0x7fbda40793f0 msgr2=0x7fbda41013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:51.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.957+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 shutdown_connections 2024-05-20T21:11:51.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:51.957+0000 7fbdaa091700 1 -- 172.21.15.26:0/2064098874 wait complete. 2024-05-20T21:11:52.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17262 9.48 MB/sec warmup 117 sec latency 9023.687 ms 2024-05-20T21:11:52.821 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1fa160df', total seconds elapsed: 26.6 2024-05-20T21:11:52.821 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-05-20T21:11:52.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:11:53.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17417 9.50 MB/sec warmup 118 sec latency 117.408 ms 2024-05-20T21:11:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:53.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:53 smithi055 ceph-mon[35213]: pgmap v862: 97 pgs: 97 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-05-20T21:11:53.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:53 smithi026 ceph-mon[31990]: pgmap v862: 97 pgs: 97 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-05-20T21:11:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:53 smithi144 ceph-mon[35660]: pgmap v862: 97 pgs: 97 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-05-20T21:11:54.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17556 9.49 MB/sec warmup 119 sec latency 127.540 ms 2024-05-20T21:11:55.411 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:55 smithi026 ceph-mon[31990]: pgmap v863: 97 pgs: 97 active+clean; 261 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-05-20T21:11:55.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:55 smithi055 ceph-mon[35213]: pgmap v863: 97 pgs: 97 active+clean; 261 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-05-20T21:11:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 -- 172.21.15.26:0/3840141217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4801095b0 msgr2=0x7fd480109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 --2- 172.21.15.26:0/3840141217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4801095b0 0x7fd480109990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474009c00 comp rx=0 tx=0).stop 2024-05-20T21:11:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 -- 172.21.15.26:0/3840141217 shutdown_connections 2024-05-20T21:11:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 --2- 172.21.15.26:0/3840141217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48006bbe0 0x7fd4801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 --2- 172.21.15.26:0/3840141217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4801095b0 0x7fd480109990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 --2- 172.21.15.26:0/3840141217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006b220 0x7fd48006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 -- 172.21.15.26:0/3840141217 >> 172.21.15.26:0/3840141217 conn(0x7fd4800793f0 msgr2=0x7fd480079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 -- 172.21.15.26:0/3840141217 shutdown_connections 2024-05-20T21:11:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.458+0000 7fd48704e700 1 -- 172.21.15.26:0/3840141217 wait complete. 2024-05-20T21:11:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 Processor -- start 2024-05-20T21:11:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 -- start start 2024-05-20T21:11:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 0x7fd480102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006bbe0 0x7fd480102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4801095b0 0x7fd480102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801135d0 con 0x7fd4801095b0 2024-05-20T21:11:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113450 con 0x7fd48006b220 2024-05-20T21:11:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd48704e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113750 con 0x7fd48006bbe0 2024-05-20T21:11:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd484dea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 0x7fd480102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd484dea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 0x7fd480102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50912/0 (socket says 172.21.15.26:50912) 2024-05-20T21:11:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd484dea700 1 -- 172.21.15.26:0/2698258995 learned_addr learned my addr 172.21.15.26:0/2698258995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.459+0000 7fd4855eb700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4801095b0 0x7fd480102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd47ffff700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006bbe0 0x7fd480102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd484dea700 1 -- 172.21.15.26:0/2698258995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006bbe0 msgr2=0x7fd480102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd484dea700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006bbe0 0x7fd480102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd484dea700 1 -- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4801095b0 msgr2=0x7fd480102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd484dea700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4801095b0 0x7fd480102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd484dea700 1 -- 172.21.15.26:0/2698258995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801a81b0 con 0x7fd48006b220 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd484dea700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 0x7fd480102510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd474005950 tx=0x7fd47400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47400b6a0 con 0x7fd48006b220 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.460+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801a8440 con 0x7fd48006b220 2024-05-20T21:11:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.461+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a88f0 con 0x7fd48006b220 2024-05-20T21:11:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.461+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd474005040 con 0x7fd48006b220 2024-05-20T21:11:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.461+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740038e0 con 0x7fd48006b220 2024-05-20T21:11:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.462+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd464005330 con 0x7fd48006b220 2024-05-20T21:11:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.464+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd47400b840 con 0x7fd48006b220 2024-05-20T21:11:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.465+0000 7fd47dffb700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4700b2900 0x7fd4700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.465+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fd4740e3480 con 0x7fd48006b220 2024-05-20T21:11:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.465+0000 7fd47ffff700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4700b2900 0x7fd4700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.466+0000 7fd47ffff700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4700b2900 0x7fd4700b4dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd480079bc0 tx=0x7fd46c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:55.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.469+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd474013030 con 0x7fd48006b220 2024-05-20T21:11:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:55 smithi144 ceph-mon[35660]: pgmap v863: 97 pgs: 97 active+clean; 261 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-05-20T21:11:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.883+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd464005620 con 0x7fd48006b220 2024-05-20T21:11:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.884+0000 7fd47dffb700 1 -- 172.21.15.26:0/2698258995 <== mon.2 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11874 (secure 0 0 0) 0x7fd4740ab1d0 con 0x7fd48006b220 2024-05-20T21:11:55.884 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:11:55.884 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:10:11.369568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24455,24476]},"id":1}]} 2024-05-20T21:11:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4700b2900 msgr2=0x7fd4700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4700b2900 0x7fd4700b4dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd480079bc0 tx=0x7fd46c009260 comp rx=0 tx=0).stop 2024-05-20T21:11:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 msgr2=0x7fd480102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 0x7fd480102510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd474005950 tx=0x7fd47400fa00 comp rx=0 tx=0).stop 2024-05-20T21:11:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 shutdown_connections 2024-05-20T21:11:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fd4700b2900 0x7fd4700b4dc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4801095b0 0x7fd480102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd48006b220 0x7fd480102510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 --2- 172.21.15.26:0/2698258995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006bbe0 0x7fd480102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 >> 172.21.15.26:0/2698258995 conn(0x7fd4800793f0 msgr2=0x7fd480101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 shutdown_connections 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:55.887+0000 7fd48704e700 1 -- 172.21.15.26:0/2698258995 wait complete. 2024-05-20T21:11:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 34 2024-05-20T21:11:56.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17846 5.79 MB/sec execute 1 sec latency 109.671 ms 2024-05-20T21:11:56.159 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:56 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2698258995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:56.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:56 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2698258995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:56.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:56 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2698258995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:11:56.720 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-05-20T21:11:56.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-05-20T21:11:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:11:57.111 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17955 3.82 MB/sec execute 2 sec latency 87.090 ms 2024-05-20T21:11:57.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:57 smithi055 ceph-mon[35213]: pgmap v864: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-05-20T21:11:57.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:57 smithi026 ceph-mon[31990]: pgmap v864: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-05-20T21:11:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:57 smithi144 ceph-mon[35660]: pgmap v864: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-05-20T21:11:58.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18091 4.12 MB/sec execute 3 sec latency 153.879 ms 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 -- 172.21.15.26:0/3401891451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06be30 msgr2=0x7f116c1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 --2- 172.21.15.26:0/3401891451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06be30 0x7f116c1081d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f11540099c0 tx=0x7f1154009cd0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 -- 172.21.15.26:0/3401891451 shutdown_connections 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 --2- 172.21.15.26:0/3401891451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f116c108900 0x7f116c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 --2- 172.21.15.26:0/3401891451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f116c06b480 0x7f116c06b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 --2- 172.21.15.26:0/3401891451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06be30 0x7f116c1081d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 -- 172.21.15.26:0/3401891451 >> 172.21.15.26:0/3401891451 conn(0x7f116c0793e0 msgr2=0x7f116c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:59.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 -- 172.21.15.26:0/3401891451 shutdown_connections 2024-05-20T21:11:59.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.055+0000 7f11729a2700 1 -- 172.21.15.26:0/3401891451 wait complete. 2024-05-20T21:11:59.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.056+0000 7f11729a2700 1 Processor -- start 2024-05-20T21:11:59.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.056+0000 7f11729a2700 1 -- start start 2024-05-20T21:11:59.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f11729a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 0x7f116c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f11729a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f116c06be30 0x7f116c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f11729a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f116c108900 0x7f116c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f11729a2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f116c113570 con 0x7f116c06b480 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f11729a2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1133f0 con 0x7f116c06be30 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f11729a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1136f0 con 0x7f116c108900 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 0x7f116c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 0x7f116c102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32952/0 (socket says 172.21.15.26:32952) 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 -- 172.21.15.26:0/1751700130 learned_addr learned my addr 172.21.15.26:0/1751700130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f1170f3f700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f116c108900 0x7f116c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 -- 172.21.15.26:0/1751700130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f116c108900 msgr2=0x7f116c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f116c108900 0x7f116c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 -- 172.21.15.26:0/1751700130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f116c06be30 msgr2=0x7f116c102a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f116c06be30 0x7f116c102a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 -- 172.21.15.26:0/1751700130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c1a7f20 con 0x7f116c06b480 2024-05-20T21:11:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.057+0000 7f116bfff700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 0x7f116c102510 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f115c00e9a0 tx=0x7f115c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.058+0000 7f1169ffb700 1 -- 172.21.15.26:0/1751700130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c00cbd0 con 0x7f116c06b480 2024-05-20T21:11:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.058+0000 7f1169ffb700 1 -- 172.21.15.26:0/1751700130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c00cd70 con 0x7f116c06b480 2024-05-20T21:11:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.058+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c1a8210 con 0x7f116c06b480 2024-05-20T21:11:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.058+0000 7f1169ffb700 1 -- 172.21.15.26:0/1751700130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c018680 con 0x7f116c06b480 2024-05-20T21:11:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.058+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c1a87a0 con 0x7f116c06b480 2024-05-20T21:11:59.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.062+0000 7f1169ffb700 1 -- 172.21.15.26:0/1751700130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f115c010950 con 0x7f116c06b480 2024-05-20T21:11:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f1169ffb700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11580b2a30 0x7f11580b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f1163fff700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11580b2a30 0x7f11580b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f1169ffb700 1 -- 172.21.15.26:0/1751700130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f115c0e2580 con 0x7f116c06b480 2024-05-20T21:11:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f117d238b80 1 Processor -- start 2024-05-20T21:11:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f117d238b80 1 -- start start 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f1163fff700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11580b2a30 0x7f11580b4ef0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f11540098e0 tx=0x7f1154000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.063+0000 7f117d238b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de7f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f117d238b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de79df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f117d238b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f117d238b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d42ddd8e20 con 0x55d42de7f340 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f117d238b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d42ddc89c0 con 0x55d42de7a330 2024-05-20T21:11:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f117d238b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d42ddcca00 con 0x55d42de79970 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de79df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1170f3f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de79df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48930/0 (socket says 172.21.15.26:48930) 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1170f3f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de86170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50936/0 (socket says 172.21.15.26:50936) 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 -- 172.21.15.26:0/356057859 learned_addr learned my addr 172.21.15.26:0/356057859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f116bfff700 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de7f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 -- 172.21.15.26:0/356057859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 msgr2=0x55d42de86170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de86170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 -- 172.21.15.26:0/356057859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 msgr2=0x55d42de7f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de7f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 -- 172.21.15.26:0/356057859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42dd18a60 con 0x55d42de79970 2024-05-20T21:11:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.064+0000 7f1163fff700 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de79df0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11540099c0 tx=0x7f115401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e806f6bca320f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.065+0000 7f11627fc700 1 -- 172.21.15.26:0/356057859 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11540037c0 con 0x55d42de79970 2024-05-20T21:11:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.065+0000 7f11627fc700 1 -- 172.21.15.26:0/356057859 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1154003df0 con 0x55d42de79970 2024-05-20T21:11:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.065+0000 7f11627fc700 1 -- 172.21.15.26:0/356057859 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115402e460 con 0x55d42de79970 2024-05-20T21:11:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 -- 172.21.15.26:0/356057859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 msgr2=0x55d42de79df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de79df0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11540099c0 tx=0x7f115401d000 comp rx=0 tx=0).stop 2024-05-20T21:11:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 -- 172.21.15.26:0/356057859 shutdown_connections 2024-05-20T21:11:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de7f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 --2- 172.21.15.26:0/356057859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de79df0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 -- 172.21.15.26:0/356057859 >> 172.21.15.26:0/356057859 conn(0x55d42dda7210 msgr2=0x55d42de74280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 -- 172.21.15.26:0/356057859 shutdown_connections 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 -- 172.21.15.26:0/356057859 wait complete. 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.066+0000 7f117d238b80 1 -- start start 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.067+0000 7f117d238b80 1 Processor -- start 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.067+0000 7f117d238b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de85e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.067+0000 7f117d238b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.067+0000 7f117d238b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.067+0000 7f116bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de85e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de85e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48942/0 (socket says 172.21.15.26:48942) 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 -- 172.21.15.26:0/3107193070 learned_addr learned my addr 172.21.15.26:0/3107193070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d42dccf130 con 0x55d42de7f340 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d42dd6a950 con 0x55d42de7a330 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d42de86710 con 0x55d42de79970 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f117d238b80 10 client.0 _ll_register_callbacks cb 0x55d42dd63ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f1170f3f700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f1163fff700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 -- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 msgr2=0x55d42de83190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de83190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 -- 172.21.15.26:0/3107193070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 msgr2=0x55d42de836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42dd67820 con 0x55d42de79970 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f116bfff700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de85e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f115c0a5b60 tx=0x7f115c0b2790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7a3c22fc1e0b47 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f114b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f114b7fe700 1 -- 172.21.15.26:0/3107193070 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0c3060 con 0x55d42de79970 2024-05-20T21:11:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.068+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1154003df0 con 0x55d42de79970 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f114b7fe700 1 -- 172.21.15.26:0/3107193070 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f115c0a8f50 con 0x55d42de79970 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f114b7fe700 1 -- 172.21.15.26:0/3107193070 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f115c0a5270 con 0x55d42de79970 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 10 client.29032 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d42de90f30 con 0x55d42de79970 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f114b7fe700 1 -- 172.21.15.26:0/3107193070 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1154003df0 con 0x55d42de79970 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f114b7fe700 1 -- 172.21.15.26:0/3107193070 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f115c0cc0d0 con 0x55d42de79970 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 10 client.29032 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 10 client.29032 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 20 client.29032 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42de93450 0x55d42de95860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 4 client.29032 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.069+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42de45b70 con 0x55d42de93450 2024-05-20T21:11:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.070+0000 7f1163fff700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42de93450 0x55d42de95860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.070+0000 7f1163fff700 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42de93450 0x55d42de95860 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd094397a280b4 server_cookie=2834be839cbd090d in_seq=0 out_seq=0 2024-05-20T21:11:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.070+0000 7f114b7fe700 10 client.29032 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.073+0000 7f114b7fe700 1 -- 172.21.15.26:0/3107193070 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d42de45b70 con 0x55d42de93450 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.073+0000 7f114b7fe700 10 client.29032 handle_command_reply: tid=0 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.073+0000 7f117d238b80 1 client.29032 shutdown 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 msgr2=0x55d42de85e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de85e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f115c0a5b60 tx=0x7f115c0b2790 comp rx=0 tx=0).stop 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 shutdown_connections 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42de93450 0x55d42de95860 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f114b7fe700 0 client.29032 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7f340 0x55d42de836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de79970 0x55d42de85e90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 --2- 172.21.15.26:0/3107193070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7a330 0x55d42de83190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 >> 172.21.15.26:0/3107193070 conn(0x55d42dda7210 msgr2=0x55d42de6ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 shutdown_connections 2024-05-20T21:11:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 1 -- 172.21.15.26:0/3107193070 wait complete. 2024-05-20T21:11:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.074+0000 7f117d238b80 20 client.29032 trim_cache size 0 max 16384 2024-05-20T21:11:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 Processor -- start 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 -- start start 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de82150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7f340 0x55d42de859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 0x55d42de7be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b2710 con 0x55d42de7a330 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1aff10 con 0x55d42de7f340 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.110+0000 7f117d238b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1add60 con 0x55d42de85f10 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 0x55d42de7be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1163fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7f340 0x55d42de859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 0x55d42de7be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:48954/0 (socket says 172.21.15.26:48954) 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 -- 172.21.15.26:0/3894533563 learned_addr learned my addr 172.21.15.26:0/3894533563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f116bfff700 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de82150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 -- 172.21.15.26:0/3894533563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7f340 msgr2=0x55d42de859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7f340 0x55d42de859d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 -- 172.21.15.26:0/3894533563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 msgr2=0x55d42de82150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de82150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 -- 172.21.15.26:0/3894533563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42de90d40 con 0x55d42de85f10 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1170f3f700 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 0x55d42de7be20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1164007600 tx=0x7f116400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40472a02ff3b7ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.111+0000 7f1161ffb700 1 -- 172.21.15.26:0/3894533563 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164016810 con 0x55d42de85f10 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.112+0000 7f1161ffb700 1 -- 172.21.15.26:0/3894533563 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164007e40 con 0x55d42de85f10 2024-05-20T21:11:59.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.112+0000 7f1161ffb700 1 -- 172.21.15.26:0/3894533563 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116401ecf0 con 0x55d42de85f10 2024-05-20T21:11:59.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18246 4.47 MB/sec execute 4 sec latency 113.948 ms 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.112+0000 7f117d238b80 1 -- 172.21.15.26:0/3894533563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 msgr2=0x55d42de7be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.112+0000 7f117d238b80 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 0x55d42de7be20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1164007600 tx=0x7f116400b040 comp rx=0 tx=0).stop 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 -- 172.21.15.26:0/3894533563 shutdown_connections 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de82150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de85f10 0x55d42de7be20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 --2- 172.21.15.26:0/3894533563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de7f340 0x55d42de859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 -- 172.21.15.26:0/3894533563 >> 172.21.15.26:0/3894533563 conn(0x55d42dda7210 msgr2=0x55d42de6ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 -- 172.21.15.26:0/3894533563 shutdown_connections 2024-05-20T21:11:59.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 -- 172.21.15.26:0/3894533563 wait complete. 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 -- start start 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.113+0000 7f117d238b80 1 Processor -- start 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de7f340 0x55d42de86960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 0x55d42de86ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1ae3d0 con 0x55d42de7a330 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1af840 con 0x55d42de85f10 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1aeaa0 con 0x55d42de7f340 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f117d238b80 10 client.0 _ll_register_callbacks cb 0x55d42dd63ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 0x55d42de86ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 0x55d42de86ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:50966/0 (socket says 172.21.15.26:50966) 2024-05-20T21:11:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 -- 172.21.15.26:0/1390634073 learned_addr learned my addr 172.21.15.26:0/1390634073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1163fff700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de7f340 0x55d42de86960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f116bfff700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 -- 172.21.15.26:0/1390634073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de7f340 msgr2=0x55d42de86960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de7f340 0x55d42de86960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 -- 172.21.15.26:0/1390634073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 msgr2=0x55d42de8c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de8c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.114+0000 7f1170f3f700 1 -- 172.21.15.26:0/1390634073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42de87730 con 0x55d42de85f10 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f1170f3f700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 0x55d42de86ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1164009010 tx=0x7f1164004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70daf9be2d1a26ed server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f114b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f114b7fe700 1 -- 172.21.15.26:0/1390634073 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116401ed80 con 0x55d42de85f10 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f114b7fe700 1 -- 172.21.15.26:0/1390634073 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f1164016610 con 0x55d42de85f10 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d42dddad70 con 0x55d42de85f10 2024-05-20T21:11:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f114b7fe700 1 -- 172.21.15.26:0/1390634073 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1164016810 con 0x55d42de85f10 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f117d238b80 10 client.29339 fetch_fsmap learned FSMap version 34 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.115+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d42df078e0 con 0x55d42de85f10 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f114b7fe700 1 -- 172.21.15.26:0/1390634073 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d42dddad70 con 0x55d42de85f10 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f114b7fe700 1 -- 172.21.15.26:0/1390634073 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1164031020 con 0x55d42de85f10 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f117d238b80 10 client.29339 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f117d238b80 10 client.29339 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f117d238b80 20 client.29339 populate_metadata read hostname 'smithi026' 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f117d238b80 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42df07f80 0x55d42df0a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f117d238b80 4 client.29339 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x55d42ddd4550 con 0x55d42df07f80 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.116+0000 7f116bfff700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42df07f80 0x55d42df0a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:11:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.117+0000 7f114b7fe700 10 client.29339 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.117+0000 7f116bfff700 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42df07f80 0x55d42df0a3a0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18e772b7a2d1c84 server_cookie=83cf44e32a2c655d in_seq=0 out_seq=0 2024-05-20T21:11:59.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:11:59 smithi055 ceph-mon[35213]: pgmap v865: 97 pgs: 97 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-05-20T21:11:59.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:11:59 smithi026 ceph-mon[31990]: pgmap v865: 97 pgs: 97 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-05-20T21:11:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:11:59 smithi144 ceph-mon[35660]: pgmap v865: 97 pgs: 97 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-05-20T21:11:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.616+0000 7f114b7fe700 1 -- 172.21.15.26:0/1390634073 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d42ddd4550 con 0x55d42df07f80 2024-05-20T21:11:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.616+0000 7f114b7fe700 10 client.29339 handle_command_reply: tid=0 2024-05-20T21:11:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.616+0000 7f117d238b80 1 client.29339 shutdown 2024-05-20T21:11:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.616+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 msgr2=0x55d42de86ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.616+0000 7f117d238b80 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 0x55d42de86ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1164009010 tx=0x7f1164004d00 comp rx=0 tx=0).stop 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 shutdown_connections 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55d42df07f80 0x55d42df0a3a0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d42de7a330 0x55d42de8c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d42de85f10 0x55d42de86ec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f114b7fe700 0 client.29339 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 --2- 172.21.15.26:0/1390634073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42de7f340 0x55d42de86960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 >> 172.21.15.26:0/1390634073 conn(0x55d42dda7210 msgr2=0x55d42de7cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 shutdown_connections 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 1 -- 172.21.15.26:0/1390634073 wait complete. 2024-05-20T21:11:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f117d238b80 20 client.29339 trim_cache size 0 max 16384 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 34, 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 58, 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "5a97a73c": { 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 58, 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:11:59.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.4, 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:59.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.617+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11580b2a30 msgr2=0x7f11580b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11580b2a30 0x7f11580b4ef0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f11540098e0 tx=0x7f1154000bc0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 msgr2=0x7f116c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:11:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 0x7f116c102510 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f115c00e9a0 tx=0x7f115c00ed60 comp rx=0 tx=0).stop 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 shutdown_connections 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f11580b2a30 0x7f11580b4ef0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f116c06b480 0x7f116c102510 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f116c108900 0x7f116c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 --2- 172.21.15.26:0/1751700130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f116c06be30 0x7f116c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 >> 172.21.15.26:0/1751700130 conn(0x7f116c0793e0 msgr2=0x7f116c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 shutdown_connections 2024-05-20T21:11:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:11:59.618+0000 7f11729a2700 1 -- 172.21.15.26:0/1751700130 wait complete. 2024-05-20T21:12:00.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 3.65 MB/sec execute 5 sec latency 715.031 ms 2024-05-20T21:12:00.598 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5a97a73c', quiesce took 7.8 seconds. Will release after: 12.599999999999998 2024-05-20T21:12:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 -- 172.21.15.26:0/1984710496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 msgr2=0x7f9c54105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/1984710496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54105980 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009900 tx=0x7f9c5002a410 comp rx=0 tx=0).stop 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 -- 172.21.15.26:0/1984710496 shutdown_connections 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/1984710496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c541087f0 0x7f9c5410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/1984710496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c54105ec0 0x7f9c541082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/1984710496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54105980 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 -- 172.21.15.26:0/1984710496 >> 172.21.15.26:0/1984710496 conn(0x7f9c540fcf60 msgr2=0x7f9c540ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 -- 172.21.15.26:0/1984710496 shutdown_connections 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.834+0000 7f9c5ba54640 1 -- 172.21.15.26:0/1984710496 wait complete. 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 Processor -- start 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 -- start start 2024-05-20T21:12:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c54105ec0 0x7f9c54101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c541087f0 0x7f9c54101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5410e520 con 0x7f9c54103590 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5410e3a0 con 0x7f9c54105ec0 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c5ba54640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5410e6a0 con 0x7f9c541087f0 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c597f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c597f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39096/0 (socket says 172.21.15.26:39096) 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c597f0640 1 -- 172.21.15.26:0/462717659 learned_addr learned my addr 172.21.15.26:0/462717659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c58fef640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c54105ec0 0x7f9c54101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.835+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c541087f0 0x7f9c54101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c597f0640 1 -- 172.21.15.26:0/462717659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c541087f0 msgr2=0x7f9c54101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c597f0640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c541087f0 0x7f9c54101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c597f0640 1 -- 172.21.15.26:0/462717659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c54105ec0 msgr2=0x7f9c54101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c597f0640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c54105ec0 0x7f9c54101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c597f0640 1 -- 172.21.15.26:0/462717659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c54102310 con 0x7f9c54103590 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c597f0640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54102f00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9c50031460 tx=0x7f9c50003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c4a7fc640 1 -- 172.21.15.26:0/462717659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5003f070 con 0x7f9c54103590 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c4a7fc640 1 -- 172.21.15.26:0/462717659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c50031da0 con 0x7f9c54103590 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c4a7fc640 1 -- 172.21.15.26:0/462717659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50035030 con 0x7f9c54103590 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c541a2c80 con 0x7f9c54103590 2024-05-20T21:12:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.836+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a31d0 con 0x7f9c54103590 2024-05-20T21:12:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c4a7fc640 1 -- 172.21.15.26:0/462717659 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9c50042070 con 0x7f9c54103590 2024-05-20T21:12:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 Processor -- start 2024-05-20T21:12:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 -- start start 2024-05-20T21:12:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636a9556c30 con 0x5636a95eb220 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5636a95467d0 con 0x5636a95e60b0 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636a954a810 con 0x5636a95e56c0 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c58fef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c59ff1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c58fef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95ed6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39100/0 (socket says 172.21.15.26:39100) 2024-05-20T21:12:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c58fef640 1 -- 172.21.15.26:0/1875583696 learned_addr learned my addr 172.21.15.26:0/1875583696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.839+0000 7f9c597f0640 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c58fef640 1 -- 172.21.15.26:0/1875583696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 msgr2=0x5636a95e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c58fef640 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c58fef640 1 -- 172.21.15.26:0/1875583696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 msgr2=0x5636a95eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c58fef640 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95eabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c58fef640 1 -- 172.21.15.26:0/1875583696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a925b210 con 0x5636a95eb220 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c597f0640 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c4a7fc640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c400b2920 0x7f9c400b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c58fef640 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95ed6b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400abb0 tx=0x7f9c4400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d30078623bc7062 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c400b2920 0x7f9c400b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c4a7fc640 1 -- 172.21.15.26:0/462717659 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9c50102a80 con 0x7f9c54103590 2024-05-20T21:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c3f7fe640 1 -- 172.21.15.26:0/1875583696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4401d070 con 0x5636a95eb220 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c3f7fe640 1 -- 172.21.15.26:0/1875583696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c44018ad0 con 0x5636a95eb220 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.840+0000 7f9c3f7fe640 1 -- 172.21.15.26:0/1875583696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44010980 con 0x5636a95eb220 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1875583696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 msgr2=0x5636a95ed6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95ed6b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400abb0 tx=0x7f9c4400f790 comp rx=0 tx=0).stop 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c400b2920 0x7f9c400b4de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5636a95e6910 tx=0x7f9c4c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1875583696 shutdown_connections 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95ed6b0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1875583696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1875583696 >> 172.21.15.26:0/1875583696 conn(0x5636a95e0d30 msgr2=0x5636a95e1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1875583696 shutdown_connections 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1875583696 wait complete. 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 -- start start 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.841+0000 7f9c5e76b740 1 Processor -- start 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 0x5636a95ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95eb220 0x5636a95f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636a922c790 con 0x5636a95e56c0 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5636a9502890 con 0x5636a95e60b0 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636a95ee1d0 con 0x5636a95eb220 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c5e76b740 10 client.0 _ll_register_callbacks cb 0x5636a925aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 0x5636a95ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 0x5636a95ef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39114/0 (socket says 172.21.15.26:39114) 2024-05-20T21:12:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 -- 172.21.15.26:0/2880541672 learned_addr learned my addr 172.21.15.26:0/2880541672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95eb220 0x5636a95f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c58fef640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 -- 172.21.15.26:0/2880541672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95eb220 msgr2=0x5636a95f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95eb220 0x5636a95f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 -- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 msgr2=0x5636a95efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 -- 172.21.15.26:0/2880541672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a9327620 con 0x5636a95e56c0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 0x5636a95ef7f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b4fddcce3cf605 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2880541672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50049040 con 0x5636a95e56c0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.842+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2880541672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9c500ca350 con 0x5636a95e56c0 2024-05-20T21:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44018ad0 con 0x5636a95e56c0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2880541672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9c500c9c00 con 0x5636a95e56c0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 10 client.20169 fetch_fsmap learned FSMap version 34 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5636a95f9d50 con 0x5636a95e56c0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2880541672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c44018ad0 con 0x5636a95e56c0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2880541672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9c500ec040 con 0x5636a95e56c0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 10 client.20169 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 10 client.20169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 20 client.20169 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a95fb500 0x5636a95fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 4 client.20169 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636a9558b80 con 0x5636a95fb500 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.843+0000 7f9c58fef640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a95fb500 0x5636a95fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.844+0000 7f9c58fef640 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a95fb500 0x5636a95fd9a0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66af2bcf524b088 server_cookie=915a5d5dc1e877d0 in_seq=0 out_seq=0 2024-05-20T21:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.844+0000 7f9c3cff9640 10 client.20169 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.844+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2880541672 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636a9558b80 con 0x5636a95fb500 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.844+0000 7f9c3cff9640 10 client.20169 handle_command_reply: tid=0 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 client.20169 shutdown 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 msgr2=0x5636a95ef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 0x5636a95ef7f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50009650 comp rx=0 tx=0).stop 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 shutdown_connections 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a95fb500 0x5636a95fd9a0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95eb220 0x5636a95f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e60b0 0x5636a95efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c3cff9640 0 client.20169 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2880541672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e56c0 0x5636a95ef7f0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 >> 172.21.15.26:0/2880541672 conn(0x5636a95e0d30 msgr2=0x5636a95dffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 shutdown_connections 2024-05-20T21:12:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2880541672 wait complete. 2024-05-20T21:12:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.845+0000 7f9c5e76b740 20 client.20169 trim_cache size 0 max 16384 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 Processor -- start 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 -- start start 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 0x5636a95f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95eb220 0x5636a95ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541ae9e0 con 0x5636a95e60b0 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541ac220 con 0x5636a95eb220 2024-05-20T21:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541aa070 con 0x5636a95e56c0 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.860+0000 7f9c58fef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 0x5636a95f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 0x5636a95f6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39128/0 (socket says 172.21.15.26:39128) 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 -- 172.21.15.26:0/1934804939 learned_addr learned my addr 172.21.15.26:0/1934804939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 -- 172.21.15.26:0/1934804939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 msgr2=0x5636a95e1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 -- 172.21.15.26:0/1934804939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95eb220 msgr2=0x5636a95ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95eb220 0x5636a95ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 -- 172.21.15.26:0/1934804939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a95ea620 con 0x5636a95e60b0 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c58fef640 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 0x5636a95f6d30 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400b010 tx=0x7f9c4400a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eafd267f799271 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c3effd640 1 -- 172.21.15.26:0/1934804939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44029070 con 0x5636a95e60b0 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c3effd640 1 -- 172.21.15.26:0/1934804939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c4400a7b0 con 0x5636a95e60b0 2024-05-20T21:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.861+0000 7f9c3effd640 1 -- 172.21.15.26:0/1934804939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4401d040 con 0x5636a95e60b0 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1934804939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 msgr2=0x5636a95f6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 0x5636a95f6d30 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400b010 tx=0x7f9c4400a740 comp rx=0 tx=0).stop 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1934804939 shutdown_connections 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95eb220 0x5636a95ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95e60b0 0x5636a95f6d30 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/1934804939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e56c0 0x5636a95e1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1934804939 >> 172.21.15.26:0/1934804939 conn(0x5636a95e0d30 msgr2=0x5636a95ea9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1934804939 shutdown_connections 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 -- 172.21.15.26:0/1934804939 wait complete. 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 -- start start 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 Processor -- start 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e56c0 0x5636a95f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.862+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e60b0 0x5636a95f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541aa6e0 con 0x5636a95eb220 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541abb50 con 0x5636a95e56c0 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541aadb0 con 0x5636a95e60b0 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 10 client.0 _ll_register_callbacks cb 0x7f9c50005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39142/0 (socket says 172.21.15.26:39142) 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 -- 172.21.15.26:0/2208470805 learned_addr learned my addr 172.21.15.26:0/2208470805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c58fef640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e60b0 0x5636a95f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e56c0 0x5636a95f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 -- 172.21.15.26:0/2208470805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e60b0 msgr2=0x5636a95f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e60b0 0x5636a95f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 -- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e56c0 msgr2=0x5636a95f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e56c0 0x5636a95f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 -- 172.21.15.26:0/2208470805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a95f6170 con 0x5636a95eb220 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c58fef640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e60b0 0x5636a95f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e56c0 0x5636a95f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c59ff1640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95f5b80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c011f20 tx=0x7f9c4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547ad9ccc8f83983 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2208470805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c004d20 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2208470805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9c4c020b00 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636a9552360 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2208470805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c020d40 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 10 client.20175 fetch_fsmap learned FSMap version 34 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.863+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5636a96663c0 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2208470805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5636a9552360 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2208470805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9c4c027030 con 0x5636a95eb220 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c5e76b740 10 client.20175 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c5e76b740 10 client.20175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c5e76b740 20 client.20175 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a9666ed0 0x5636a96692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c5e76b740 4 client.20175 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:12:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636a94cc5f0 con 0x5636a9666ed0 2024-05-20T21:12:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.864+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a9666ed0 0x5636a96692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.865+0000 7f9c597f0640 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a9666ed0 0x5636a96692f0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4dfd94394e5c2db server_cookie=2add8327be25cf2f in_seq=0 out_seq=0 2024-05-20T21:12:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.865+0000 7f9c3cff9640 10 client.20175 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.944+0000 7f9c3cff9640 1 -- 172.21.15.26:0/2208470805 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5636a94cc5f0 con 0x5636a9666ed0 2024-05-20T21:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c3cff9640 10 client.20175 handle_command_reply: tid=0 2024-05-20T21:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 client.20175 shutdown 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 msgr2=0x5636a95f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95f5b80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c011f20 tx=0x7f9c4c004ab0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 shutdown_connections 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5636a9666ed0 0x5636a96692f0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636a95eb220 0x5636a95f5b80 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636a95e60b0 0x5636a95f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 --2- 172.21.15.26:0/2208470805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a95e56c0 0x5636a95f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 >> 172.21.15.26:0/2208470805 conn(0x5636a95e0d30 msgr2=0x5636a95ea960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c3cff9640 0 client.20175 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 shutdown_connections 2024-05-20T21:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 1 -- 172.21.15.26:0/2208470805 wait complete. 2024-05-20T21:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.945+0000 7f9c5e76b740 20 client.20175 trim_cache size 0 max 16384 2024-05-20T21:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c400b2920 msgr2=0x7f9c400b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c400b2920 0x7f9c400b4de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5636a95e6910 tx=0x7f9c4c009600 comp rx=0 tx=0).stop 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 msgr2=0x7f9c54102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54102f00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9c50031460 tx=0x7f9c50003c80 comp rx=0 tx=0).stop 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 shutdown_connections 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c400b2920 0x7f9c400b4de0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c541087f0 0x7f9c54101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c54105ec0 0x7f9c54101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 --2- 172.21.15.26:0/462717659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c54103590 0x7f9c54102f00 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 >> 172.21.15.26:0/462717659 conn(0x7f9c540fcf60 msgr2=0x7f9c54104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 shutdown_connections 2024-05-20T21:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:00.946+0000 7f9c5ba54640 1 -- 172.21.15.26:0/462717659 wait complete. 2024-05-20T21:12:01.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 3.04 MB/sec execute 6 sec latency 1715.119 ms 2024-05-20T21:12:01.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:01 smithi055 ceph-mon[35213]: pgmap v866: 97 pgs: 97 active+clean; 265 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-05-20T21:12:01.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:01 smithi026 ceph-mon[31990]: pgmap v866: 97 pgs: 97 active+clean; 265 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-05-20T21:12:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:01 smithi144 ceph-mon[35660]: pgmap v866: 97 pgs: 97 active+clean; 265 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-05-20T21:12:02.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 2.61 MB/sec execute 7 sec latency 2715.214 ms 2024-05-20T21:12:03.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 2.28 MB/sec execute 8 sec latency 3715.303 ms 2024-05-20T21:12:03.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:03 smithi055 ceph-mon[35213]: pgmap v867: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-05-20T21:12:03.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:03 smithi026 ceph-mon[31990]: pgmap v867: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-05-20T21:12:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:03 smithi144 ceph-mon[35660]: pgmap v867: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-05-20T21:12:04.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 2.03 MB/sec execute 9 sec latency 4715.394 ms 2024-05-20T21:12:05.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.83 MB/sec execute 10 sec latency 5715.522 ms 2024-05-20T21:12:05.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:05 smithi055 ceph-mon[35213]: pgmap v868: 97 pgs: 97 active+clean; 247 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-20T21:12:05.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:05 smithi026 ceph-mon[31990]: pgmap v868: 97 pgs: 97 active+clean; 247 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-20T21:12:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:05 smithi144 ceph-mon[35660]: pgmap v868: 97 pgs: 97 active+clean; 247 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-20T21:12:06.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.66 MB/sec execute 11 sec latency 6715.633 ms 2024-05-20T21:12:07.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.52 MB/sec execute 12 sec latency 7715.738 ms 2024-05-20T21:12:07.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:07 smithi055 ceph-mon[35213]: pgmap v869: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-05-20T21:12:07.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:07 smithi026 ceph-mon[31990]: pgmap v869: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-05-20T21:12:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:07 smithi144 ceph-mon[35660]: pgmap v869: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-05-20T21:12:08.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.40 MB/sec execute 13 sec latency 8715.793 ms 2024-05-20T21:12:09.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.30 MB/sec execute 14 sec latency 9715.897 ms 2024-05-20T21:12:09.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:09 smithi055 ceph-mon[35213]: pgmap v870: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-05-20T21:12:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:09 smithi026 ceph-mon[31990]: pgmap v870: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-05-20T21:12:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:09 smithi144 ceph-mon[35660]: pgmap v870: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-05-20T21:12:10.112 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.22 MB/sec execute 15 sec latency 10715.970 ms 2024-05-20T21:12:11.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.14 MB/sec execute 16 sec latency 11716.010 ms 2024-05-20T21:12:11.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:11 smithi055 ceph-mon[35213]: pgmap v871: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 955 KiB/s wr, 123 op/s 2024-05-20T21:12:11.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:11 smithi026 ceph-mon[31990]: pgmap v871: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 955 KiB/s wr, 123 op/s 2024-05-20T21:12:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:11 smithi144 ceph-mon[35660]: pgmap v871: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 955 KiB/s wr, 123 op/s 2024-05-20T21:12:12.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.07 MB/sec execute 17 sec latency 12716.054 ms 2024-05-20T21:12:13.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 1.01 MB/sec execute 18 sec latency 13716.151 ms 2024-05-20T21:12:13.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:12:13.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:13 smithi055 ceph-mon[35213]: pgmap v872: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 461 KiB/s wr, 103 op/s 2024-05-20T21:12:13.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:13 smithi026 ceph-mon[31990]: pgmap v872: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 461 KiB/s wr, 103 op/s 2024-05-20T21:12:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:12:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:13 smithi144 ceph-mon[35660]: pgmap v872: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 461 KiB/s wr, 103 op/s 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 -- 172.21.15.26:0/1504763079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854100b50 msgr2=0x7f9854100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1504763079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854100b50 0x7f9854100fd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f983c009900 tx=0x7f983c02a410 comp rx=0 tx=0).stop 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 -- 172.21.15.26:0/1504763079 shutdown_connections 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1504763079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9854101510 0x7f985410b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1504763079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854100b50 0x7f9854100fd0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1504763079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98541040e0 0x7f9854104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 -- 172.21.15.26:0/1504763079 >> 172.21.15.26:0/1504763079 conn(0x7f98540fc6d0 msgr2=0x7f98540feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 -- 172.21.15.26:0/1504763079 shutdown_connections 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 -- 172.21.15.26:0/1504763079 wait complete. 2024-05-20T21:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.041+0000 7f9858ba4640 1 Processor -- start 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 -- start start 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854100b50 0x7f9854198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 0x7f9854199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98541040e0 0x7f98541a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f985410e5c0 con 0x7f9854101510 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f985410e440 con 0x7f98541040e0 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9858ba4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f985410e740 con 0x7f9854100b50 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 0x7f9854199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 0x7f9854199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38930/0 (socket says 172.21.15.26:38930) 2024-05-20T21:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 -- 172.21.15.26:0/1195097687 learned_addr learned my addr 172.21.15.26:0/1195097687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f9852d9d640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98541040e0 0x7f98541a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f985259c640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854100b50 0x7f9854198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 -- 172.21.15.26:0/1195097687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854100b50 msgr2=0x7f9854198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854100b50 0x7f9854198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 -- 172.21.15.26:0/1195097687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98541040e0 msgr2=0x7f98541a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98541040e0 0x7f98541a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 -- 172.21.15.26:0/1195097687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98541a0940 con 0x7f9854101510 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984bfff640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 0x7f9854199270 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f983c009230 tx=0x7f983c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.042+0000 7f984b7fe640 1 -- 172.21.15.26:0/1195097687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c03f030 con 0x7f9854101510 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.043+0000 7f984b7fe640 1 -- 172.21.15.26:0/1195097687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f983c003c90 con 0x7f9854101510 2024-05-20T21:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.043+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98541a0bd0 con 0x7f9854101510 2024-05-20T21:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.043+0000 7f984b7fe640 1 -- 172.21.15.26:0/1195097687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c035070 con 0x7f9854101510 2024-05-20T21:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.043+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541a10c0 con 0x7f9854101510 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.045+0000 7f984b7fe640 1 -- 172.21.15.26:0/1195097687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f983c042030 con 0x7f9854101510 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.045+0000 7f985b997740 1 Processor -- start 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.045+0000 7f985b997740 1 -- start start 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985b997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985b997740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985b997740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985b997740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5614eb722c30 con 0x5614eb7bb630 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985b997740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5614eb7127d0 con 0x5614eb7b64c0 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985b997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614eb716810 con 0x5614eb7b5ad0 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38944/0 (socket says 172.21.15.26:38944) 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 -- 172.21.15.26:0/3260695790 learned_addr learned my addr 172.21.15.26:0/3260695790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f985259c640 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f9852d9d640 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 -- 172.21.15.26:0/3260695790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 msgr2=0x5614eb7b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 -- 172.21.15.26:0/3260695790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 msgr2=0x5614eb7bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 -- 172.21.15.26:0/3260695790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614eb427210 con 0x5614eb7bb630 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984b7fe640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98400b28a0 0x7f98400b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.046+0000 7f984bfff640 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7bdac0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f983c106ce0 tx=0x7f983c106dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d91e7689598b3df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f9852d9d640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98400b28a0 0x7f98400b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f9848ff9640 1 -- 172.21.15.26:0/3260695790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0c5f30 con 0x5614eb7bb630 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f984b7fe640 1 -- 172.21.15.26:0/1195097687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f983c102b40 con 0x7f9854101510 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f9848ff9640 1 -- 172.21.15.26:0/3260695790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f983c003e40 con 0x5614eb7bb630 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f9848ff9640 1 -- 172.21.15.26:0/3260695790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0c9870 con 0x5614eb7bb630 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f985b997740 1 -- 172.21.15.26:0/3260695790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 msgr2=0x5614eb7bdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f9852d9d640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98400b28a0 0x7f98400b4d60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5614eb7b6d20 tx=0x7f984c00e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f985b997740 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7bdac0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f983c106ce0 tx=0x7f983c106dd0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f985b997740 1 -- 172.21.15.26:0/3260695790 shutdown_connections 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f985b997740 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7bdac0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f985b997740 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.047+0000 7f985b997740 1 --2- 172.21.15.26:0/3260695790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- 172.21.15.26:0/3260695790 >> 172.21.15.26:0/3260695790 conn(0x5614eb7acd30 msgr2=0x5614eb7ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- 172.21.15.26:0/3260695790 shutdown_connections 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- 172.21.15.26:0/3260695790 wait complete. 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- start start 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 Processor -- start 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 0x5614eb7bf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7bb630 0x5614eb7c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5614eb3f8790 con 0x5614eb7b5ad0 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5614eb6ce890 con 0x5614eb7b64c0 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614eb7be3a0 con 0x5614eb7bb630 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985b997740 10 client.0 _ll_register_callbacks cb 0x5614eb426a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 0x5614eb7bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 0x5614eb7bf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38950/0 (socket says 172.21.15.26:38950) 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f985259c640 1 -- 172.21.15.26:0/1537861235 learned_addr learned my addr 172.21.15.26:0/1537861235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f9852d9d640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7bb630 0x5614eb7c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.048+0000 7f984bfff640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985259c640 1 -- 172.21.15.26:0/1537861235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7bb630 msgr2=0x5614eb7c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985259c640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7bb630 0x5614eb7c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985259c640 1 -- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 msgr2=0x5614eb7bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985259c640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985259c640 1 -- 172.21.15.26:0/1537861235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614eb4f3620 con 0x5614eb7b5ad0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985259c640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 0x5614eb7bf980 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f985406af80 tx=0x7f984400f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faf8d1afdf5038d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f9839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f9839ffb640 1 -- 172.21.15.26:0/1537861235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844013850 con 0x5614eb7b5ad0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f9839ffb640 1 -- 172.21.15.26:0/1537861235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f98440139f0 con 0x5614eb7b5ad0 2024-05-20T21:12:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5614eb724b80 con 0x5614eb7b5ad0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f9839ffb640 1 -- 172.21.15.26:0/1537861235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9844013bf0 con 0x5614eb7b5ad0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985b997740 10 client.20187 fetch_fsmap learned FSMap version 34 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5614eb7c9fb0 con 0x5614eb7b5ad0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.049+0000 7f9839ffb640 1 -- 172.21.15.26:0/1537861235 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5614eb724b80 con 0x5614eb7b5ad0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f9839ffb640 1 -- 172.21.15.26:0/1537861235 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9844024050 con 0x5614eb7b5ad0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f985b997740 10 client.20187 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f985b997740 10 client.20187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f985b997740 20 client.20187 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f985b997740 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb7cb6a0 0x5614eb7cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f985b997740 4 client.20187 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614eb71e360 con 0x5614eb7cb6a0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f984bfff640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb7cb6a0 0x5614eb7cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f984bfff640 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb7cb6a0 0x5614eb7cdab0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688f5194902e45ec server_cookie=d0d561f88ed3e208 in_seq=0 out_seq=0 2024-05-20T21:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.050+0000 7f9839ffb640 10 client.20187 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.051+0000 7f9839ffb640 1 -- 172.21.15.26:0/1537861235 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5614eb71e360 con 0x5614eb7cb6a0 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.051+0000 7f9839ffb640 10 client.20187 handle_command_reply: tid=0 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.051+0000 7f985b997740 1 client.20187 shutdown 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 msgr2=0x5614eb7bf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 0x5614eb7bf980 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f985406af80 tx=0x7f984400f950 comp rx=0 tx=0).stop 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 shutdown_connections 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb7cb6a0 0x5614eb7cdab0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7bb630 0x5614eb7c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f9839ffb640 0 client.20187 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 --2- 172.21.15.26:0/1537861235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7b5ad0 0x5614eb7bf980 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 >> 172.21.15.26:0/1537861235 conn(0x5614eb7acd30 msgr2=0x5614eb7abfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 shutdown_connections 2024-05-20T21:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 1 -- 172.21.15.26:0/1537861235 wait complete. 2024-05-20T21:12:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.052+0000 7f985b997740 20 client.20187 trim_cache size 0 max 16384 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 Processor -- start 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 -- start start 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b64c0 0x5614eb7b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98541aa770 con 0x5614eb7bb630 2024-05-20T21:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98541a7fb0 con 0x5614eb7b5ad0 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985b997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98541a5e00 con 0x5614eb7b64c0 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f9852d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f9852d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38952/0 (socket says 172.21.15.26:38952) 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f9852d9d640 1 -- 172.21.15.26:0/4151270657 learned_addr learned my addr 172.21.15.26:0/4151270657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f985259c640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f984bfff640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b64c0 0x5614eb7b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.067+0000 7f9852d9d640 1 -- 172.21.15.26:0/4151270657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b64c0 msgr2=0x5614eb7b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f9852d9d640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b64c0 0x5614eb7b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f9852d9d640 1 -- 172.21.15.26:0/4151270657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b5ad0 msgr2=0x5614eb7b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f9852d9d640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f9852d9d640 1 -- 172.21.15.26:0/4151270657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614eb7b68a0 con 0x5614eb7bb630 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985259c640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f984bfff640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b64c0 0x5614eb7b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f9852d9d640 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6ec0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f984c01aa20 tx=0x7f984c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9a60b6381f746d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f983bfff640 1 -- 172.21.15.26:0/4151270657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c02a070 con 0x5614eb7bb630 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f983bfff640 1 -- 172.21.15.26:0/4151270657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f984c018da0 con 0x5614eb7bb630 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f983bfff640 1 -- 172.21.15.26:0/4151270657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c016680 con 0x5614eb7bb630 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 -- 172.21.15.26:0/4151270657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 msgr2=0x5614eb7c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6ec0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f984c01aa20 tx=0x7f984c012000 comp rx=0 tx=0).stop 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 -- 172.21.15.26:0/4151270657 shutdown_connections 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6ec0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b64c0 0x5614eb7b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 --2- 172.21.15.26:0/4151270657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b5ad0 0x5614eb7b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.068+0000 7f985b997740 1 -- 172.21.15.26:0/4151270657 >> 172.21.15.26:0/4151270657 conn(0x5614eb7acd30 msgr2=0x5614eb7afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 -- 172.21.15.26:0/4151270657 shutdown_connections 2024-05-20T21:12:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 -- 172.21.15.26:0/4151270657 wait complete. 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 -- start start 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 Processor -- start 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98541a6470 con 0x5614eb7bb630 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98541a78e0 con 0x5614eb7b64c0 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98541a6b40 con 0x5614eb7b5ad0 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985b997740 10 client.0 _ll_register_callbacks cb 0x7f984400a4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f9852d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f9852d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38958/0 (socket says 172.21.15.26:38958) 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f9852d9d640 1 -- 172.21.15.26:0/334609954 learned_addr learned my addr 172.21.15.26:0/334609954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f985259c640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.069+0000 7f984bfff640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9852d9d640 1 -- 172.21.15.26:0/334609954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 msgr2=0x5614eb7c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9852d9d640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9852d9d640 1 -- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 msgr2=0x5614eb7c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9852d9d640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9852d9d640 1 -- 172.21.15.26:0/334609954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614eb7c6640 con 0x5614eb7bb630 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9852d9d640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6050 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f984c01a9f0 tx=0x7f984c025b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4923e492828b639 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9839ffb640 1 -- 172.21.15.26:0/334609954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c02a040 con 0x5614eb7bb630 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9839ffb640 1 -- 172.21.15.26:0/334609954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f984c030450 con 0x5614eb7bb630 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f984c018da0 con 0x5614eb7bb630 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9839ffb640 1 -- 172.21.15.26:0/334609954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f984c01a670 con 0x5614eb7bb630 2024-05-20T21:12:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 10 client.20193 fetch_fsmap learned FSMap version 34 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5614eb836670 con 0x5614eb7bb630 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9839ffb640 1 -- 172.21.15.26:0/334609954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f984c018da0 con 0x5614eb7bb630 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f9839ffb640 1 -- 172.21.15.26:0/334609954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f984c024070 con 0x5614eb7bb630 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 10 client.20193 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 10 client.20193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 20 client.20193 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb836fa0 0x5614eb8393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 4 client.20193 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.070+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614eb6985f0 con 0x5614eb836fa0 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.071+0000 7f985259c640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb836fa0 0x5614eb8393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.071+0000 7f985259c640 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb836fa0 0x5614eb8393c0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560a321e16993586 server_cookie=5cb89670c258b55f in_seq=0 out_seq=0 2024-05-20T21:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.071+0000 7f9839ffb640 10 client.20193 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f9839ffb640 1 -- 172.21.15.26:0/334609954 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5614eb6985f0 con 0x5614eb836fa0 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f9839ffb640 10 client.20193 handle_command_reply: tid=0 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 client.20193 shutdown 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 msgr2=0x5614eb7c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6050 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f984c01a9f0 tx=0x7f984c025b70 comp rx=0 tx=0).stop 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 shutdown_connections 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5614eb836fa0 0x5614eb8393c0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eb7bb630 0x5614eb7c6050 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614eb7b64c0 0x5614eb7c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f9839ffb640 0 client.20193 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 --2- 172.21.15.26:0/334609954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614eb7b5ad0 0x5614eb7c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 >> 172.21.15.26:0/334609954 conn(0x5614eb7acd30 msgr2=0x5614eb7b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 shutdown_connections 2024-05-20T21:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 1 -- 172.21.15.26:0/334609954 wait complete. 2024-05-20T21:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.098+0000 7f985b997740 20 client.20193 trim_cache size 0 max 16384 2024-05-20T21:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98400b28a0 msgr2=0x7f98400b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98400b28a0 0x7f98400b4d60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5614eb7b6d20 tx=0x7f984c00e040 comp rx=0 tx=0).stop 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 msgr2=0x7f9854199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 0x7f9854199270 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f983c009230 tx=0x7f983c0042f0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 shutdown_connections 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f98400b28a0 0x7f98400b4d60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98541040e0 0x7f98541a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9854101510 0x7f9854199270 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 --2- 172.21.15.26:0/1195097687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854100b50 0x7f9854198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 >> 172.21.15.26:0/1195097687 conn(0x7f98540fc6d0 msgr2=0x7f9854107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 shutdown_connections 2024-05-20T21:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:14.099+0000 7f9858ba4640 1 -- 172.21.15.26:0/1195097687 wait complete. 2024-05-20T21:12:14.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 0.96 MB/sec execute 19 sec latency 14716.212 ms 2024-05-20T21:12:15.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 0.91 MB/sec execute 20 sec latency 15716.261 ms 2024-05-20T21:12:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 -- 172.21.15.26:0/2617722686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6406bd10 msgr2=0x7f6a6406c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/2617722686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6406bd10 0x7f6a6406c190 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6a540099c0 tx=0x7f6a54009cd0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 -- 172.21.15.26:0/2617722686 shutdown_connections 2024-05-20T21:12:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/2617722686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a6406c6d0 0x7f6a64110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/2617722686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64109630 0x7f6a64109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/2617722686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6406bd10 0x7f6a6406c190 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 -- 172.21.15.26:0/2617722686 >> 172.21.15.26:0/2617722686 conn(0x7f6a640794e0 msgr2=0x7f6a640798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 -- 172.21.15.26:0/2617722686 shutdown_connections 2024-05-20T21:12:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.401+0000 7f6a6b77a700 1 -- 172.21.15.26:0/2617722686 wait complete. 2024-05-20T21:12:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 Processor -- start 2024-05-20T21:12:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 -- start start 2024-05-20T21:12:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6406bd10 0x7f6a641024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a6406c6d0 0x7f6a641029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 0x7f6a64102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64113690 con 0x7f6a64109630 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64113510 con 0x7f6a6406c6d0 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a6b77a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64113810 con 0x7f6a6406bd10 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 0x7f6a64102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 0x7f6a64102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38984/0 (socket says 172.21.15.26:38984) 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 -- 172.21.15.26:0/1870337333 learned_addr learned my addr 172.21.15.26:0/1870337333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69516700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6406bd10 0x7f6a641024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a68d15700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a6406c6d0 0x7f6a641029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 -- 172.21.15.26:0/1870337333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6406bd10 msgr2=0x7f6a641024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6406bd10 0x7f6a641024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 -- 172.21.15.26:0/1870337333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a6406c6d0 msgr2=0x7f6a641029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a6406c6d0 0x7f6a641029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.402+0000 7f6a69d17700 1 -- 172.21.15.26:0/1870337333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a641a80f0 con 0x7f6a64109630 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.403+0000 7f6a69d17700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 0x7f6a64102f30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007ee0 tx=0x7f6a5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.403+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00dd20 con 0x7f6a64109630 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.403+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a5c004020 con 0x7f6a64109630 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.403+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a641a83e0 con 0x7f6a64109630 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.403+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c015660 con 0x7f6a64109630 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.403+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a641a88f0 con 0x7f6a64109630 2024-05-20T21:12:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.404+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6410bac0 con 0x7f6a64109630 2024-05-20T21:12:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.405+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6a5c0041d0 con 0x7f6a64109630 2024-05-20T21:12:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.407+0000 7f6a5a7fc700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a500b2900 0x7f6a500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.407+0000 7f6a69516700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a500b2900 0x7f6a500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.407+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6a5c0e23c0 con 0x7f6a64109630 2024-05-20T21:12:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.407+0000 7f6a69516700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a500b2900 0x7f6a500b4dc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6a600045b0 tx=0x7f6a6000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.410+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a5c010070 con 0x7f6a64109630 2024-05-20T21:12:15.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:15 smithi055 ceph-mon[35213]: pgmap v873: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 252 KiB/s wr, 79 op/s 2024-05-20T21:12:15.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:15 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:15.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:15 smithi026 ceph-mon[31990]: pgmap v873: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 252 KiB/s wr, 79 op/s 2024-05-20T21:12:15.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:15 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:15 smithi144 ceph-mon[35660]: pgmap v873: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 252 KiB/s wr, 79 op/s 2024-05-20T21:12:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:15 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.671+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a64104ca0 con 0x7f6a64109630 2024-05-20T21:12:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.672+0000 7f6a5a7fc700 1 -- 172.21.15.26:0/1870337333 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11873 (secure 0 0 0) 0x7f6a64104ca0 con 0x7f6a64109630 2024-05-20T21:12:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:12:15.672 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:12:13.224420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T21:12:15.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a500b2900 msgr2=0x7f6a500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:15.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a500b2900 0x7f6a500b4dc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6a600045b0 tx=0x7f6a6000b3c0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 msgr2=0x7f6a64102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:15.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 0x7f6a64102f30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007ee0 tx=0x7f6a5c00d3b0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 shutdown_connections 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6a500b2900 0x7f6a500b4dc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a64109630 0x7f6a64102f30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6406bd10 0x7f6a641024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 --2- 172.21.15.26:0/1870337333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a6406c6d0 0x7f6a641029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 >> 172.21.15.26:0/1870337333 conn(0x7f6a640794e0 msgr2=0x7f6a64101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 shutdown_connections 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:15.674+0000 7f6a6b77a700 1 -- 172.21.15.26:0/1870337333 wait complete. 2024-05-20T21:12:15.675 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 35 2024-05-20T21:12:16.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 0.87 MB/sec execute 21 sec latency 16716.290 ms 2024-05-20T21:12:16.406 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:16 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/1870337333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:12:16.437 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 5a97a73c --release --await' 2024-05-20T21:12:16.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 5a97a73c --release --await 2024-05-20T21:12:16.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:16 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/1870337333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:12:16.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:16 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/1870337333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:12:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:12:17.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 0.83 MB/sec execute 22 sec latency 17716.374 ms 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:17 smithi055 ceph-mon[35213]: pgmap v874: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:17 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:17 smithi026 ceph-mon[31990]: pgmap v874: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-05-20T21:12:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:12:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:17 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:17 smithi144 ceph-mon[35660]: pgmap v874: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-05-20T21:12:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:12:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:17.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:17 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:18.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18264 0.79 MB/sec execute 23 sec latency 18716.481 ms 2024-05-20T21:12:18.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:18 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi144 to 176.1M 2024-05-20T21:12:18.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:18 smithi026 ceph-mon[31990]: Unable to set osd_memory_target on smithi144 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.468+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3361874819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2281076b0 msgr2=0x7fb22810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.468+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3361874819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2281076b0 0x7fb22810f280 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb22000b0d0 tx=0x7fb22000b3e0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3361874819 shutdown_connections 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3361874819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2281076b0 0x7fb22810f280 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3361874819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281025d0 0x7fb2281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3361874819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb228102f80 0x7fb228106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3361874819 >> 172.21.15.26:0/3361874819 conn(0x7fb2280fde60 msgr2=0x7fb228100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3361874819 shutdown_connections 2024-05-20T21:12:18.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.469+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3361874819 wait complete. 2024-05-20T21:12:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 Processor -- start 2024-05-20T21:12:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 -- start start 2024-05-20T21:12:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281025d0 0x7fb22819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 0x7fb22819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2281076b0 0x7fb228198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb228112400 con 0x7fb228102f80 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb228112280 con 0x7fb2281076b0 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22e7a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb228112580 con 0x7fb2281025d0 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 0x7fb22819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22d7a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281025d0 0x7fb22819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 0x7fb22819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39006/0 (socket says 172.21.15.26:39006) 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22cf9f700 1 -- 172.21.15.26:0/3651106302 learned_addr learned my addr 172.21.15.26:0/3651106302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22dfa1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2281076b0 0x7fb228198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.470+0000 7fb22cf9f700 1 -- 172.21.15.26:0/3651106302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281025d0 msgr2=0x7fb22819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281025d0 0x7fb22819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22cf9f700 1 -- 172.21.15.26:0/3651106302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2281076b0 msgr2=0x7fb228198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2281076b0 0x7fb228198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22cf9f700 1 -- 172.21.15.26:0/3651106302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb228199040 con 0x7fb228102f80 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22dfa1700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2281076b0 0x7fb228198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 0x7fb22819eb80 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb22400d980 tx=0x7fb22400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:18.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb21a7fc700 1 -- 172.21.15.26:0/3651106302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224017070 con 0x7fb228102f80 2024-05-20T21:12:18.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb21a7fc700 1 -- 172.21.15.26:0/3651106302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb22400fd20 con 0x7fb228102f80 2024-05-20T21:12:18.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb21a7fc700 1 -- 172.21.15.26:0/3651106302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22400e3b0 con 0x7fb228102f80 2024-05-20T21:12:18.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb228199330 con 0x7fb228102f80 2024-05-20T21:12:18.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.471+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2281a55c0 con 0x7fb228102f80 2024-05-20T21:12:18.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.474+0000 7fb21a7fc700 1 -- 172.21.15.26:0/3651106302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb22400f890 con 0x7fb228102f80 2024-05-20T21:12:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.474+0000 7fb21a7fc700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2140b2880 0x7fb2140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.474+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2140b2880 0x7fb2140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.474+0000 7fb21a7fc700 1 -- 172.21.15.26:0/3651106302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2240e1c70 con 0x7fb228102f80 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 Processor -- start 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 -- start start 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2140b2880 0x7fb2140b4d40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007be0 tx=0x7fb21c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1918b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f19252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645f1877d90 con 0x5645f191e420 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5645f1867930 con 0x5645f19191a0 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5645f186b970 con 0x5645f191edd0 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb22d7a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb22d7a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39020/0 (socket says 172.21.15.26:39020) 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb22d7a0700 1 -- 172.21.15.26:0/2530845342 learned_addr learned my addr 172.21.15.26:0/2530845342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.475+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1918b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22dfa1700 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f19252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2530845342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 msgr2=0x5645f19252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f19252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2530845342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 msgr2=0x5645f191e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2530845342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f17b79d0 con 0x5645f191edd0 2024-05-20T21:12:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1918b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb2240df710 tx=0x7fb22400fb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e19cfc9c8ec99 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb2137fe700 1 -- 172.21.15.26:0/2530845342 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224020040 con 0x5645f191edd0 2024-05-20T21:12:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb2137fe700 1 -- 172.21.15.26:0/2530845342 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2240a8d40 con 0x5645f191edd0 2024-05-20T21:12:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.476+0000 7fb2137fe700 1 -- 172.21.15.26:0/2530845342 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2240dfbd0 con 0x5645f191edd0 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2530845342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 msgr2=0x5645f1918b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1918b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb2240df710 tx=0x7fb22400fb80 comp rx=0 tx=0).stop 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2530845342 shutdown_connections 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f19252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2530845342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1918b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2530845342 >> 172.21.15.26:0/2530845342 conn(0x5645f1846180 msgr2=0x5645f19133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2530845342 shutdown_connections 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2530845342 wait complete. 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.477+0000 7fb23a29ab80 1 -- start start 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 Processor -- start 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1922260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f19227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645f176e0a0 con 0x5645f191e420 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5645f18098c0 con 0x5645f19191a0 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5645f1925800 con 0x5645f191edd0 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb23a29ab80 10 client.0 _ll_register_callbacks cb 0x5645f1802a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1922260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1922260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39034/0 (socket says 172.21.15.26:39034) 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb22cf9f700 1 -- 172.21.15.26:0/1719904832 learned_addr learned my addr 172.21.15.26:0/1719904832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.478+0000 7fb22dfa1700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f19227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22cf9f700 1 -- 172.21.15.26:0/1719904832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 msgr2=0x5645f19227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f19227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22cf9f700 1 -- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 msgr2=0x5645f1924f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22cf9f700 1 -- 172.21.15.26:0/1719904832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f1806790 con 0x5645f191e420 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1922260 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb2240a52b0 tx=0x7fb224010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbdadee54d61305 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb210ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb210ff9700 1 -- 172.21.15.26:0/1719904832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22406d060 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb210ff9700 1 -- 172.21.15.26:0/1719904832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2240dfde0 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2240a8d40 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb210ff9700 1 -- 172.21.15.26:0/1719904832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb224020070 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb23a29ab80 10 client.20211 fetch_fsmap learned FSMap version 35 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5645f192fed0 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.479+0000 7fb210ff9700 1 -- 172.21.15.26:0/1719904832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2240a8d40 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb210ff9700 1 -- 172.21.15.26:0/1719904832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fb2240b1c00 con 0x5645f191e420 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb23a29ab80 10 client.20211 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb23a29ab80 10 client.20211 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb23a29ab80 20 client.20211 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f19323a0 0x5645f1934790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb23a29ab80 4 client.20211 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645f18e4ae0 con 0x5645f19323a0 2024-05-20T21:12:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f19323a0 0x5645f1934790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f19323a0 0x5645f1934790 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa495a59e27f891 server_cookie=8fd14969ab12940f in_seq=0 out_seq=0 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.480+0000 7fb210ff9700 10 client.20211 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb210ff9700 1 -- 172.21.15.26:0/1719904832 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5645f18e4ae0 con 0x5645f19323a0 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb210ff9700 10 client.20211 handle_command_reply: tid=0 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 client.20211 shutdown 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 msgr2=0x5645f1922260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1922260 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb2240a52b0 tx=0x7fb224010840 comp rx=0 tx=0).stop 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 shutdown_connections 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f19323a0 0x5645f1934790 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f19227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb210ff9700 0 client.20211 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/1719904832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1922260 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.481+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 >> 172.21.15.26:0/1719904832 conn(0x5645f1846180 msgr2=0x5645f190de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:18.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.482+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 shutdown_connections 2024-05-20T21:12:18.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.482+0000 7fb23a29ab80 1 -- 172.21.15.26:0/1719904832 wait complete. 2024-05-20T21:12:18.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.482+0000 7fb23a29ab80 20 client.20211 trim_cache size 0 max 16384 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 Processor -- start 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 -- start start 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f191f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281ae1c0 con 0x5645f191e420 2024-05-20T21:12:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281ab9f0 con 0x5645f19191a0 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281a9840 con 0x5645f191edd0 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39046/0 (socket says 172.21.15.26:39046) 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.502+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2468918969 learned_addr learned my addr 172.21.15.26:0/2468918969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22dfa1700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f191f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2468918969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 msgr2=0x5645f191f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f191f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2468918969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 msgr2=0x5645f1924100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2468918969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f1922030 con 0x5645f191e420 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22dfa1700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f191f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191db60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb2240dfa80 tx=0x7fb22400fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61bd5fc428e866b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb212ffd700 1 -- 172.21.15.26:0/2468918969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22406d070 con 0x5645f191e420 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb212ffd700 1 -- 172.21.15.26:0/2468918969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2240a8d40 con 0x5645f191e420 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.503+0000 7fb212ffd700 1 -- 172.21.15.26:0/2468918969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2240b2090 con 0x5645f191e420 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2468918969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 msgr2=0x5645f191db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191db60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb2240dfa80 tx=0x7fb22400fb80 comp rx=0 tx=0).stop 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2468918969 shutdown_connections 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f191f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1924100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2468918969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f191db60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2468918969 >> 172.21.15.26:0/2468918969 conn(0x5645f1846180 msgr2=0x5645f191c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2468918969 shutdown_connections 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2468918969 wait complete. 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 -- start start 2024-05-20T21:12:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.504+0000 7fb23a29ab80 1 Processor -- start 2024-05-20T21:12:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1936430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1936970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1936eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281a9eb0 con 0x5645f191e420 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281ab320 con 0x5645f19191a0 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281aa580 con 0x5645f191edd0 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1936970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb23a29ab80 10 client.0 _ll_register_callbacks cb 0x5645f1802a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1936970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39056/0 (socket says 172.21.15.26:39056) 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2317139311 learned_addr learned my addr 172.21.15.26:0/2317139311 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1936430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22dfa1700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1936eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2317139311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 msgr2=0x5645f1936eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1936eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 msgr2=0x5645f1936430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1936430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 -- 172.21.15.26:0/2317139311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f199cbf0 con 0x5645f191e420 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1936430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb22cf9f700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1936970 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb22400d950 tx=0x7fb224005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625a2c06129978dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.505+0000 7fb210ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb210ff9700 1 -- 172.21.15.26:0/2317139311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22406d070 con 0x5645f191e420 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb210ff9700 1 -- 172.21.15.26:0/2317139311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb2240a4c90 con 0x5645f191e420 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2240a8d40 con 0x5645f191e420 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb210ff9700 1 -- 172.21.15.26:0/2317139311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb2240ad1c0 con 0x5645f191e420 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 10 client.20217 fetch_fsmap learned FSMap version 35 2024-05-20T21:12:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5645f199cbf0 con 0x5645f191e420 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb210ff9700 1 -- 172.21.15.26:0/2317139311 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2240a8d40 con 0x5645f191e420 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb210ff9700 1 -- 172.21.15.26:0/2317139311 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fb2240c3c40 con 0x5645f191e420 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 10 client.20217 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 10 client.20217 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 20 client.20217 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f199e0c0 0x5645f19a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 4 client.20217 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5a97a73c", "release": true, "await": true}] 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5a97a73c", "release": true, "await": true}) -- 0x5645f1879ce0 con 0x5645f199e0c0 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.506+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f199e0c0 0x5645f19a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.507+0000 7fb22d7a0700 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f199e0c0 0x5645f19a04e0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724d52321b6b15f1 server_cookie=4f52cfe0ce519201 in_seq=0 out_seq=0 2024-05-20T21:12:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.507+0000 7fb210ff9700 10 client.20217 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:18 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi144 to 176.1M 2024-05-20T21:12:18.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:18 smithi144 ceph-mon[35660]: Unable to set osd_memory_target on smithi144 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-20T21:12:18.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb210ff9700 1 -- 172.21.15.26:0/2317139311 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5645f1879ce0 con 0x5645f199e0c0 2024-05-20T21:12:18.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb210ff9700 10 client.20217 handle_command_reply: tid=0 2024-05-20T21:12:18.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 client.20217 shutdown 2024-05-20T21:12:18.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 msgr2=0x5645f1936970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1936970 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb22400d950 tx=0x7fb224005c40 comp rx=0 tx=0).stop 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 shutdown_connections 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5645f199e0c0 0x5645f19a04e0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645f191edd0 0x5645f1936eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5645f19191a0 0x5645f1936430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb210ff9700 0 client.20217 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 --2- 172.21.15.26:0/2317139311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645f191e420 0x5645f1936970 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 >> 172.21.15.26:0/2317139311 conn(0x5645f1846180 msgr2=0x5645f190d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 shutdown_connections 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.553+0000 7fb23a29ab80 1 -- 172.21.15.26:0/2317139311 wait complete. 2024-05-20T21:12:18.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb23a29ab80 20 client.20217 trim_cache size 0 max 16384 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 35, 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 60, 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "5a97a73c": { 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 60, 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:12:18.555 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.4, 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:12:18.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:12:18.557 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:12:18.557 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:12:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2140b2880 msgr2=0x7fb2140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2140b2880 0x7fb2140b4d40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007be0 tx=0x7fb21c0073d0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 msgr2=0x7fb22819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 0x7fb22819eb80 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb22400d980 tx=0x7fb22400dc90 comp rx=0 tx=0).stop 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 shutdown_connections 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2281076b0 0x7fb228198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281025d0 0x7fb22819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb2140b2880 0x7fb2140b4d40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 --2- 172.21.15.26:0/3651106302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb228102f80 0x7fb22819eb80 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.554+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 >> 172.21.15.26:0/3651106302 conn(0x7fb2280fde60 msgr2=0x7fb228100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.555+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 shutdown_connections 2024-05-20T21:12:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:12:18.555+0000 7fb22e7a2700 1 -- 172.21.15.26:0/3651106302 wait complete. 2024-05-20T21:12:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:18 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi144 to 176.1M 2024-05-20T21:12:18.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:18 smithi055 ceph-mon[35213]: Unable to set osd_memory_target on smithi144 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-20T21:12:19.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18360 0.90 MB/sec execute 24 sec latency 19162.032 ms 2024-05-20T21:12:19.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: pgmap v875: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-20T21:12:19.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: Unable to set osd_memory_target on smithi055 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:12:19.439 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:12:19.475 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5a97a73c', total seconds elapsed: 26.7 2024-05-20T21:12:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: pgmap v875: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-20T21:12:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-05-20T21:12:19.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: Unable to set osd_memory_target on smithi055 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-20T21:12:19.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:12:19.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:12:19.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: pgmap v875: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: Unable to set osd_memory_target on smithi055 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-20T21:12:19.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:12:19.603 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:12:19.603 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:12:20.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18511 1.03 MB/sec execute 25 sec latency 99.032 ms 2024-05-20T21:12:21.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18700 1.49 MB/sec execute 26 sec latency 102.427 ms 2024-05-20T21:12:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:21 smithi144 ceph-mon[35660]: pgmap v876: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-05-20T21:12:21.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:21 smithi055 ceph-mon[35213]: pgmap v876: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-05-20T21:12:21.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:21 smithi026 ceph-mon[31990]: pgmap v876: 97 pgs: 97 active+clean; 245 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-05-20T21:12:22.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18884 1.61 MB/sec execute 27 sec latency 91.070 ms 2024-05-20T21:12:22.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:22 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:22.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:22 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:22.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:22 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:23.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19046 1.96 MB/sec execute 28 sec latency 108.655 ms 2024-05-20T21:12:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:23 smithi144 ceph-mon[35660]: pgmap v877: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 467 KiB/s wr, 26 op/s 2024-05-20T21:12:23.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:23 smithi055 ceph-mon[35213]: pgmap v877: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 467 KiB/s wr, 26 op/s 2024-05-20T21:12:23.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:23 smithi026 ceph-mon[31990]: pgmap v877: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 467 KiB/s wr, 26 op/s 2024-05-20T21:12:24.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19239 2.37 MB/sec execute 29 sec latency 77.768 ms 2024-05-20T21:12:25.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19410 2.53 MB/sec execute 30 sec latency 100.245 ms 2024-05-20T21:12:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:25 smithi144 ceph-mon[35660]: pgmap v878: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.3 MiB/s wr, 89 op/s 2024-05-20T21:12:25.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:25 smithi055 ceph-mon[35213]: pgmap v878: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.3 MiB/s wr, 89 op/s 2024-05-20T21:12:25.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:25 smithi026 ceph-mon[31990]: pgmap v878: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.3 MiB/s wr, 89 op/s 2024-05-20T21:12:26.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19610 2.75 MB/sec execute 31 sec latency 99.654 ms 2024-05-20T21:12:27.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19768 2.85 MB/sec execute 32 sec latency 97.651 ms 2024-05-20T21:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 -- 172.21.15.26:0/1457229215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454108790 msgr2=0x7f845410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1457229215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454108790 0x7f845410abe0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f844c009ca0 tx=0x7f844c02a8c0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 -- 172.21.15.26:0/1457229215 shutdown_connections 2024-05-20T21:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1457229215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454108790 0x7f845410abe0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1457229215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454105e60 0x7f8454108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1457229215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 0x7f8454105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 -- 172.21.15.26:0/1457229215 >> 172.21.15.26:0/1457229215 conn(0x7f84540fcf20 msgr2=0x7f84540ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 -- 172.21.15.26:0/1457229215 shutdown_connections 2024-05-20T21:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.228+0000 7f845c3ac640 1 -- 172.21.15.26:0/1457229215 wait complete. 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 Processor -- start 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 -- start start 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 0x7f8454102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454105e60 0x7f84541014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454108790 0x7f8454101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f845410e500 con 0x7f8454105e60 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f845410e380 con 0x7f8454103530 2024-05-20T21:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845c3ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f845410e680 con 0x7f8454108790 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f8459947640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454105e60 0x7f84541014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845a148640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 0x7f8454102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845a949640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454108790 0x7f8454101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845a148640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 0x7f8454102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40674/0 (socket says 172.21.15.26:40674) 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.229+0000 7f845a148640 1 -- 172.21.15.26:0/1775753329 learned_addr learned my addr 172.21.15.26:0/1775753329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845a949640 1 -- 172.21.15.26:0/1775753329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 msgr2=0x7f8454102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845a949640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 0x7f8454102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845a949640 1 -- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454105e60 msgr2=0x7f84541014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845a949640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454105e60 0x7f84541014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845a949640 1 -- 172.21.15.26:0/1775753329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84541022b0 con 0x7f8454108790 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845a949640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454108790 0x7f8454101a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f844c031040 tx=0x7f844c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f844b7fe640 1 -- 172.21.15.26:0/1775753329 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c003d10 con 0x7f8454108790 2024-05-20T21:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f844b7fe640 1 -- 172.21.15.26:0/1775753329 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f844c0044d0 con 0x7f8454108790 2024-05-20T21:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84541a2c70 con 0x7f8454108790 2024-05-20T21:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.230+0000 7f844b7fe640 1 -- 172.21.15.26:0/1775753329 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c030b90 con 0x7f8454108790 2024-05-20T21:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.231+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84541a31c0 con 0x7f8454108790 2024-05-20T21:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.234+0000 7f844b7fe640 1 -- 172.21.15.26:0/1775753329 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f844c004040 con 0x7f8454108790 2024-05-20T21:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.234+0000 7f845f19b740 1 Processor -- start 2024-05-20T21:12:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.234+0000 7f845f19b740 1 -- start start 2024-05-20T21:12:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845f19b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228ef6c0 0x55a7228efae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845f19b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845f19b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845f19b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a722860c30 con 0x55a7228f00b0 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845f19b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a7228507d0 con 0x55a7228ef6c0 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845f19b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a722854810 con 0x55a7228f5220 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f8459947640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228ef6c0 0x55a7228efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43250/0 (socket says 172.21.15.26:43250) 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 -- 172.21.15.26:0/1866771268 learned_addr learned my addr 172.21.15.26:0/1866771268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f8459947640 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 -- 172.21.15.26:0/1866771268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 msgr2=0x55a7228f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 -- 172.21.15.26:0/1866771268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228ef6c0 msgr2=0x55a7228efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228ef6c0 0x55a7228efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 -- 172.21.15.26:0/1866771268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a722565210 con 0x55a7228f00b0 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f8459947640 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228ef6c0 0x55a7228efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f844b7fe640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84400b2920 0x7f84400b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.235+0000 7f845a148640 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f4bb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8450009dc0 tx=0x7f8450011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa664080b4947b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.236+0000 7f8459947640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84400b2920 0x7f84400b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.236+0000 7f844b7fe640 1 -- 172.21.15.26:0/1775753329 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f844c1034a0 con 0x7f8454108790 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.236+0000 7f8448bfe640 1 -- 172.21.15.26:0/1866771268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450005220 con 0x55a7228f00b0 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.236+0000 7f8448bfe640 1 -- 172.21.15.26:0/1866771268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8450004c30 con 0x55a7228f00b0 2024-05-20T21:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.236+0000 7f8448bfe640 1 -- 172.21.15.26:0/1866771268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450019760 con 0x55a7228f00b0 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 -- 172.21.15.26:0/1866771268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 msgr2=0x55a7228f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f4bb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8450009dc0 tx=0x7f8450011810 comp rx=0 tx=0).stop 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f8459947640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84400b2920 0x7f84400b4de0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55a7228f0910 tx=0x7f8444004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 -- 172.21.15.26:0/1866771268 shutdown_connections 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f4bb0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 --2- 172.21.15.26:0/1866771268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228ef6c0 0x55a7228efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 -- 172.21.15.26:0/1866771268 >> 172.21.15.26:0/1866771268 conn(0x55a7228ead30 msgr2=0x55a7228eb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 -- 172.21.15.26:0/1866771268 shutdown_connections 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 -- 172.21.15.26:0/1866771268 wait complete. 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 -- start start 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.237+0000 7f845f19b740 1 Processor -- start 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a722900d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a722536790 con 0x55a7228f00b0 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a72280c890 con 0x55a7228f5220 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7228f81d0 con 0x55a7228ef6c0 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f8459947640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228f9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:36402/0 (socket says 172.21.15.26:36402) 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 -- 172.21.15.26:0/332333313 learned_addr learned my addr 172.21.15.26:0/332333313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f8459947640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43256/0 (socket says 172.21.15.26:43256) 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a949640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a722900d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845f19b740 10 client.0 _ll_register_callbacks cb 0x55a722564a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 -- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 msgr2=0x55a722900d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a722900d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 -- 172.21.15.26:0/332333313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 msgr2=0x55a7228f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f845a148640 1 -- 172.21.15.26:0/332333313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a722631620 con 0x55a7228ef6c0 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.238+0000 7f8459947640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f845a148640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228f9810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8450009350 tx=0x7f8450005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62cfb8f75bb389ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f843dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f843dffb640 1 -- 172.21.15.26:0/332333313 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450027070 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f843dffb640 1 -- 172.21.15.26:0/332333313 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8450004450 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8450004c30 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f843dffb640 1 -- 172.21.15.26:0/332333313 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8450033050 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f845f19b740 10 client.29095 fetch_fsmap learned FSMap version 36 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a722903d00 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.239+0000 7f843dffb640 1 -- 172.21.15.26:0/332333313 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8450004c30 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f843dffb640 1 -- 172.21.15.26:0/332333313 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f84500194a0 con 0x55a7228ef6c0 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f845f19b740 10 client.29095 fetch_fsmap finished waiting for FSMap version 36 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f845f19b740 10 client.29095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f845f19b740 20 client.29095 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f845f19b740 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722905290 0x55a7229076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f845f19b740 4 client.29095 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a722862b80 con 0x55a722905290 2024-05-20T21:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.240+0000 7f8459947640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722905290 0x55a7229076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.241+0000 7f843dffb640 10 client.29095 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.241+0000 7f8459947640 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722905290 0x55a7229076e0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77977ecb17ec17e server_cookie=d813aad2061c44d5 in_seq=0 out_seq=0 2024-05-20T21:12:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.242+0000 7f843dffb640 1 -- 172.21.15.26:0/332333313 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a722862b80 con 0x55a722905290 2024-05-20T21:12:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.242+0000 7f843dffb640 10 client.29095 handle_command_reply: tid=0 2024-05-20T21:12:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.242+0000 7f845f19b740 1 client.29095 shutdown 2024-05-20T21:12:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.242+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 msgr2=0x55a7228f9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.242+0000 7f845f19b740 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228f9810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8450009350 tx=0x7f8450005230 comp rx=0 tx=0).stop 2024-05-20T21:12:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 shutdown_connections 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722905290 0x55a7229076e0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a722900d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f843dffb640 0 client.29095 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 --2- 172.21.15.26:0/332333313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228f9810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 >> 172.21.15.26:0/332333313 conn(0x55a7228ead30 msgr2=0x55a7228e9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 shutdown_connections 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 1 -- 172.21.15.26:0/332333313 wait complete. 2024-05-20T21:12:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.243+0000 7f845f19b740 20 client.29095 trim_cache size 0 max 16384 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 Processor -- start 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 -- start start 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 0x55a7228e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f00b0 0x55a722900ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84541aea30 con 0x55a7228ef6c0 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84541ac2d0 con 0x55a7228f00b0 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.265+0000 7f845f19b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84541aa100 con 0x55a7228f5220 2024-05-20T21:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 0x55a7228e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a949640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 0x55a7228e5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43264/0 (socket says 172.21.15.26:43264) 2024-05-20T21:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 -- 172.21.15.26:0/2731896857 learned_addr learned my addr 172.21.15.26:0/2731896857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f8459947640 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f00b0 0x55a722900ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 -- 172.21.15.26:0/2731896857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 msgr2=0x55a7228f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 -- 172.21.15.26:0/2731896857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f00b0 msgr2=0x55a722900ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f00b0 0x55a722900ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 -- 172.21.15.26:0/2731896857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a722907d60 con 0x55a7228ef6c0 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f845a148640 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 0x55a7228e5e40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f845000a010 tx=0x7f84500098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b48b500f7a5291c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.266+0000 7f843ffff640 1 -- 172.21.15.26:0/2731896857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450027070 con 0x55a7228ef6c0 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.267+0000 7f843ffff640 1 -- 172.21.15.26:0/2731896857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8450019d00 con 0x55a7228ef6c0 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.267+0000 7f843ffff640 1 -- 172.21.15.26:0/2731896857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450004a40 con 0x55a7228ef6c0 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.267+0000 7f845f19b740 1 -- 172.21.15.26:0/2731896857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 msgr2=0x55a7228e5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.267+0000 7f845f19b740 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 0x55a7228e5e40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f845000a010 tx=0x7f84500098a0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- 172.21.15.26:0/2731896857 shutdown_connections 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228f5220 0x55a7228f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f00b0 0x55a722900ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 --2- 172.21.15.26:0/2731896857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228ef6c0 0x55a7228e5e40 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- 172.21.15.26:0/2731896857 >> 172.21.15.26:0/2731896857 conn(0x55a7228ead30 msgr2=0x55a7228f49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- 172.21.15.26:0/2731896857 shutdown_connections 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- 172.21.15.26:0/2731896857 wait complete. 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- start start 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 Processor -- start 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a7229766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84541aa790 con 0x55a7228f00b0 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84541abc00 con 0x55a7228f5220 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84541aae60 con 0x55a7228ef6c0 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.268+0000 7f845f19b740 10 client.0 _ll_register_callbacks cb 0x7f8450009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f8459947640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:36420/0 (socket says 172.21.15.26:36420) 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a949640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a7229766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 -- 172.21.15.26:0/2863333220 learned_addr learned my addr 172.21.15.26:0/2863333220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 -- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 msgr2=0x55a7229766c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a7229766c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 -- 172.21.15.26:0/2863333220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 msgr2=0x55a7228fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228fe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 -- 172.21.15.26:0/2863333220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a722976c00 con 0x55a7228ef6c0 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f8459947640 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845a148640 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228fdbf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f8450009a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d7fe803fab483a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f843dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f843dffb640 1 -- 172.21.15.26:0/2863333220 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450046060 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f843dffb640 1 -- 172.21.15.26:0/2863333220 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f84500049d0 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.269+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8450019d00 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f843dffb640 1 -- 172.21.15.26:0/2863333220 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8450027040 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 10 client.29101 fetch_fsmap learned FSMap version 36 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a722977e30 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f843dffb640 1 -- 172.21.15.26:0/2863333220 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8450019d00 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f843dffb640 1 -- 172.21.15.26:0/2863333220 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f84500037f0 con 0x55a7228ef6c0 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 10 client.29101 fetch_fsmap finished waiting for FSMap version 36 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 10 client.29101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 20 client.29101 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722978760 0x55a72297ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 4 client.29101 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:12:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a72285c360 con 0x55a722978760 2024-05-20T21:12:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.270+0000 7f8459947640 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722978760 0x55a72297ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.271+0000 7f843dffb640 10 client.29101 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.271+0000 7f8459947640 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722978760 0x55a72297ab80 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679663cae68b6c2e server_cookie=b61d23f58dd922f in_seq=0 out_seq=0 2024-05-20T21:12:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f843dffb640 1 -- 172.21.15.26:0/2863333220 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a72285c360 con 0x55a722978760 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f843dffb640 10 client.29101 handle_command_reply: tid=0 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 client.29101 shutdown 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 msgr2=0x55a7228fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228fdbf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f8450009a40 comp rx=0 tx=0).stop 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 shutdown_connections 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55a722978760 0x55a72297ab80 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a7228f5220 0x55a7229766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f843dffb640 0 client.29101 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7228f00b0 0x55a7228fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 --2- 172.21.15.26:0/2863333220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7228ef6c0 0x55a7228fdbf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 >> 172.21.15.26:0/2863333220 conn(0x55a7228ead30 msgr2=0x55a7228edc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 shutdown_connections 2024-05-20T21:12:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 1 -- 172.21.15.26:0/2863333220 wait complete. 2024-05-20T21:12:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.393+0000 7f845f19b740 20 client.29101 trim_cache size 0 max 16384 2024-05-20T21:12:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:12:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:12:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84400b2920 msgr2=0x7f84400b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84400b2920 0x7f84400b4de0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55a7228f0910 tx=0x7f8444004730 comp rx=0 tx=0).stop 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454108790 msgr2=0x7f8454101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454108790 0x7f8454101a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f844c031040 tx=0x7f844c003ba0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 shutdown_connections 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f84400b2920 0x7f84400b4de0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8454108790 0x7f8454101a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8454105e60 0x7f84541014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 --2- 172.21.15.26:0/1775753329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8454103530 0x7f8454102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 >> 172.21.15.26:0/1775753329 conn(0x7f84540fcf20 msgr2=0x7f8454103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 shutdown_connections 2024-05-20T21:12:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:27.394+0000 7f845c3ac640 1 -- 172.21.15.26:0/1775753329 wait complete. 2024-05-20T21:12:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:27 smithi144 ceph-mon[35660]: pgmap v879: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.7 MiB/s wr, 108 op/s 2024-05-20T21:12:27.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:27 smithi055 ceph-mon[35213]: pgmap v879: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.7 MiB/s wr, 108 op/s 2024-05-20T21:12:27.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:27 smithi026 ceph-mon[31990]: pgmap v879: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.7 MiB/s wr, 108 op/s 2024-05-20T21:12:28.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19929 2.95 MB/sec execute 33 sec latency 155.405 ms 2024-05-20T21:12:29.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20100 3.06 MB/sec execute 34 sec latency 99.737 ms 2024-05-20T21:12:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:29 smithi144 ceph-mon[35660]: pgmap v880: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-05-20T21:12:29.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:29 smithi055 ceph-mon[35213]: pgmap v880: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-05-20T21:12:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:29 smithi026 ceph-mon[31990]: pgmap v880: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-05-20T21:12:30.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20263 3.28 MB/sec execute 35 sec latency 84.626 ms 2024-05-20T21:12:31.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20392 3.31 MB/sec execute 36 sec latency 102.307 ms 2024-05-20T21:12:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:31 smithi144 ceph-mon[35660]: pgmap v881: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-05-20T21:12:31.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:31 smithi055 ceph-mon[35213]: pgmap v881: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-05-20T21:12:31.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:31 smithi026 ceph-mon[31990]: pgmap v881: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-05-20T21:12:32.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20567 3.49 MB/sec execute 37 sec latency 92.136 ms 2024-05-20T21:12:33.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20824 3.96 MB/sec execute 38 sec latency 100.278 ms 2024-05-20T21:12:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:33 smithi144 ceph-mon[35660]: pgmap v882: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-05-20T21:12:33.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:33 smithi055 ceph-mon[35213]: pgmap v882: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-05-20T21:12:33.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:33 smithi026 ceph-mon[31990]: pgmap v882: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-05-20T21:12:34.113 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21016 4.13 MB/sec execute 39 sec latency 114.038 ms 2024-05-20T21:12:35.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21190 4.30 MB/sec execute 40 sec latency 116.376 ms 2024-05-20T21:12:35.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:35 smithi144 ceph-mon[35660]: pgmap v883: 97 pgs: 97 active+clean; 255 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-05-20T21:12:35.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:35 smithi055 ceph-mon[35213]: pgmap v883: 97 pgs: 97 active+clean; 255 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-05-20T21:12:35.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:35 smithi026 ceph-mon[31990]: pgmap v883: 97 pgs: 97 active+clean; 255 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-05-20T21:12:36.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21362 4.55 MB/sec execute 41 sec latency 110.681 ms 2024-05-20T21:12:37.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21478 4.53 MB/sec execute 42 sec latency 89.533 ms 2024-05-20T21:12:37.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:37 smithi026 ceph-mon[31990]: pgmap v884: 97 pgs: 97 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-05-20T21:12:37.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:37 smithi026 ceph-mon[31990]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:37 smithi144 ceph-mon[35660]: pgmap v884: 97 pgs: 97 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-05-20T21:12:37.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:37 smithi144 ceph-mon[35660]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:37 smithi055 ceph-mon[35213]: pgmap v884: 97 pgs: 97 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-05-20T21:12:37.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:37 smithi055 ceph-mon[35213]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T21:12:38.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21658 4.83 MB/sec execute 43 sec latency 93.155 ms 2024-05-20T21:12:39.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21827 4.96 MB/sec execute 44 sec latency 106.689 ms 2024-05-20T21:12:39.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:39 smithi144 ceph-mon[35660]: pgmap v885: 97 pgs: 97 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-05-20T21:12:39.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:39 smithi055 ceph-mon[35213]: pgmap v885: 97 pgs: 97 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-05-20T21:12:39.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:39 smithi026 ceph-mon[31990]: pgmap v885: 97 pgs: 97 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-05-20T21:12:40.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21965 4.97 MB/sec execute 45 sec latency 101.653 ms 2024-05-20T21:12:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 -- 172.21.15.26:0/852354339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc1019b0 msgr2=0x7f83cc101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 --2- 172.21.15.26:0/852354339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc1019b0 0x7f83cc101dd0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009900 tx=0x7f83c802a410 comp rx=0 tx=0).stop 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 -- 172.21.15.26:0/852354339 shutdown_connections 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 --2- 172.21.15.26:0/852354339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83cc103560 0x7f83cc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 --2- 172.21.15.26:0/852354339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83cc102ba0 0x7f83cc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 --2- 172.21.15.26:0/852354339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc1019b0 0x7f83cc101dd0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.514+0000 7f83d47c1640 1 -- 172.21.15.26:0/852354339 >> 172.21.15.26:0/852354339 conn(0x7f83cc0fcf20 msgr2=0x7f83cc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.515+0000 7f83d47c1640 1 -- 172.21.15.26:0/852354339 shutdown_connections 2024-05-20T21:12:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.515+0000 7f83d47c1640 1 -- 172.21.15.26:0/852354339 wait complete. 2024-05-20T21:12:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.515+0000 7f83d47c1640 1 Processor -- start 2024-05-20T21:12:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.515+0000 7f83d47c1640 1 -- start start 2024-05-20T21:12:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d47c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83cc1019b0 0x7f83cc06acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d47c1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83cc102ba0 0x7f83cc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d47c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 0x7f83cc19ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d47c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc10d360 con 0x7f83cc103560 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d47c1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc10d1e0 con 0x7f83cc102ba0 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d47c1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc10d4e0 con 0x7f83cc1019b0 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 0x7f83cc19ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 0x7f83cc19ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41626/0 (socket says 172.21.15.26:41626) 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 -- 172.21.15.26:0/2485998353 learned_addr learned my addr 172.21.15.26:0/2485998353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d255d640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83cc1019b0 0x7f83cc06acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83cc102ba0 0x7f83cc19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 -- 172.21.15.26:0/2485998353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83cc1019b0 msgr2=0x7f83cc06acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83cc1019b0 0x7f83cc06acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 -- 172.21.15.26:0/2485998353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83cc102ba0 msgr2=0x7f83cc19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83cc102ba0 0x7f83cc19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 -- 172.21.15.26:0/2485998353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1a07f0 con 0x7f83cc103560 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.516+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 0x7f83cc19ff30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f83c400efd0 tx=0x7f83c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.517+0000 7f83c37fe640 1 -- 172.21.15.26:0/2485998353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4003b70 con 0x7f83cc103560 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.517+0000 7f83c37fe640 1 -- 172.21.15.26:0/2485998353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83c4003d10 con 0x7f83cc103560 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.517+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1a0ae0 con 0x7f83cc103560 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.517+0000 7f83c37fe640 1 -- 172.21.15.26:0/2485998353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4003b70 con 0x7f83cc103560 2024-05-20T21:12:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.517+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a0fe0 con 0x7f83cc103560 2024-05-20T21:12:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.520+0000 7f83d75b5740 1 Processor -- start 2024-05-20T21:12:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.520+0000 7f83c37fe640 1 -- 172.21.15.26:0/2485998353 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f83c4004020 con 0x7f83cc103560 2024-05-20T21:12:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 -- start start 2024-05-20T21:12:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8ceafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ceb630 0x5556e8cedac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556e8c52c30 con 0x5556e8ce5ad0 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5556e8c427d0 con 0x5556e8ce64c0 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d75b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556e8c46810 con 0x5556e8ceb630 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41636/0 (socket says 172.21.15.26:41636) 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 -- 172.21.15.26:0/4056031078 learned_addr learned my addr 172.21.15.26:0/4056031078 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d255d640 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8ceafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 -- 172.21.15.26:0/4056031078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ceb630 msgr2=0x5556e8cedac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ceb630 0x5556e8cedac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 -- 172.21.15.26:0/4056031078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 msgr2=0x5556e8ceafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8ceafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d1d5c640 1 -- 172.21.15.26:0/4056031078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e8957210 con 0x5556e8ce5ad0 2024-05-20T21:12:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.521+0000 7f83d255d640 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8ceafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83c37fe640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83b80b2a90 0x7f83b80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce5ef0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009c70 tx=0x7f83bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a03f9db8e02eee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83c37fe640 1 -- 172.21.15.26:0/2485998353 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f83c40e22a0 con 0x7f83cc103560 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83c0ff9640 1 -- 172.21.15.26:0/4056031078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc01e070 con 0x5556e8ce5ad0 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83d255d640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83b80b2a90 0x7f83b80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83c0ff9640 1 -- 172.21.15.26:0/4056031078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83bc009e20 con 0x5556e8ce5ad0 2024-05-20T21:12:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.522+0000 7f83c0ff9640 1 -- 172.21.15.26:0/4056031078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc019450 con 0x5556e8ce5ad0 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 -- 172.21.15.26:0/4056031078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 msgr2=0x5556e8ce5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d255d640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83b80b2a90 0x7f83b80b4f50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f83c80057c0 tx=0x7f83c8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce5ef0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009c70 tx=0x7f83bc011810 comp rx=0 tx=0).stop 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 -- 172.21.15.26:0/4056031078 shutdown_connections 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ceb630 0x5556e8cedac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8ceafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 --2- 172.21.15.26:0/4056031078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce5ef0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 -- 172.21.15.26:0/4056031078 >> 172.21.15.26:0/4056031078 conn(0x5556e8cdcd30 msgr2=0x5556e8cdd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 -- 172.21.15.26:0/4056031078 shutdown_connections 2024-05-20T21:12:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.523+0000 7f83d75b5740 1 -- 172.21.15.26:0/4056031078 wait complete. 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 -- start start 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 Processor -- start 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8cefc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556e8928790 con 0x5556e8ceb630 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5556e8bfe890 con 0x5556e8ce64c0 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556e8cee5e0 con 0x5556e8ce5ad0 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d75b5740 10 client.0 _ll_register_callbacks cb 0x5556e8956a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d2d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d1d5c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d2d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41650/0 (socket says 172.21.15.26:41650) 2024-05-20T21:12:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d1d5c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57642/0 (socket says 172.21.15.26:57642) 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d2d5e640 1 -- 172.21.15.26:0/3227317149 learned_addr learned my addr 172.21.15.26:0/3227317149 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.524+0000 7f83d255d640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8cefc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d1d5c640 1 -- 172.21.15.26:0/3227317149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 msgr2=0x5556e8cefc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8cefc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d1d5c640 1 -- 172.21.15.26:0/3227317149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 msgr2=0x5556e8cf71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d1d5c640 1 -- 172.21.15.26:0/3227317149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e8a23620 con 0x5556e8ce64c0 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf01a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f83bc014040 tx=0x7f83bc003ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec884b901e47d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3227317149 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc029060 con 0x5556e8ce64c0 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3227317149 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f83bc0040d0 con 0x5556e8ce64c0 2024-05-20T21:12:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83bc009e20 con 0x5556e8ce64c0 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3227317149 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f83bc01e040 con 0x5556e8ce64c0 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d75b5740 10 client.29414 fetch_fsmap learned FSMap version 37 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.525+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5556e8cfa1c0 con 0x5556e8ce64c0 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3227317149 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83bc009e20 con 0x5556e8ce64c0 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3227317149 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f83bc02d030 con 0x5556e8ce64c0 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d75b5740 10 client.29414 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d75b5740 10 client.29414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d75b5740 20 client.29414 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8cfb730 0x5556e8cfdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d75b5740 4 client.29414 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556e8c54b80 con 0x5556e8cfb730 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.526+0000 7f83d255d640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8cfb730 0x5556e8cfdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.527+0000 7f83b5ffb640 10 client.29414 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.527+0000 7f83d255d640 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8cfb730 0x5556e8cfdb20 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2215bc12fc29b0 server_cookie=b387eb23f2a029c0 in_seq=0 out_seq=0 2024-05-20T21:12:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.528+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3227317149 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5556e8c54b80 con 0x5556e8cfb730 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.528+0000 7f83b5ffb640 10 client.29414 handle_command_reply: tid=0 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.528+0000 7f83d75b5740 1 client.29414 shutdown 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 msgr2=0x5556e8cf01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf01a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f83bc014040 tx=0x7f83bc003ee0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 shutdown_connections 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8cfb730 0x5556e8cfdb20 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf01a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83b5ffb640 0 client.29414 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3227317149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8cefc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 >> 172.21.15.26:0/3227317149 conn(0x5556e8cdcd30 msgr2=0x5556e8cdbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 shutdown_connections 2024-05-20T21:12:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 1 -- 172.21.15.26:0/3227317149 wait complete. 2024-05-20T21:12:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.529+0000 7f83d75b5740 20 client.29414 trim_cache size 0 max 16384 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 Processor -- start 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 -- start start 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cdbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1aa5f0 con 0x5556e8ceb630 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1a7ec0 con 0x5556e8ce64c0 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d75b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1a5d10 con 0x5556e8ce5ad0 2024-05-20T21:12:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d2d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cdbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d2d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cdbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41652/0 (socket says 172.21.15.26:41652) 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d2d5e640 1 -- 172.21.15.26:0/1469931969 learned_addr learned my addr 172.21.15.26:0/1469931969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d255d640 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.558+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83d2d5e640 1 -- 172.21.15.26:0/1469931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 msgr2=0x5556e8ce6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83d2d5e640 1 -- 172.21.15.26:0/1469931969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 msgr2=0x5556e8cf71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83d2d5e640 1 -- 172.21.15.26:0/1469931969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e8cf0de0 con 0x5556e8ceb630 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cdbe30 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f83c40a6ad0 tx=0x7f83c40b0f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b977483aa673dd7d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83b7fff640 1 -- 172.21.15.26:0/1469931969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40a9f60 con 0x5556e8ceb630 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83b7fff640 1 -- 172.21.15.26:0/1469931969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83c40c3070 con 0x5556e8ceb630 2024-05-20T21:12:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.559+0000 7f83b7fff640 1 -- 172.21.15.26:0/1469931969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40a5af0 con 0x5556e8ceb630 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 -- 172.21.15.26:0/1469931969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 msgr2=0x5556e8cdbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cdbe30 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f83c40a6ad0 tx=0x7f83c40b0f90 comp rx=0 tx=0).stop 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 -- 172.21.15.26:0/1469931969 shutdown_connections 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cdbe30 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce64c0 0x5556e8cf71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 --2- 172.21.15.26:0/1469931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce5ad0 0x5556e8ce6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 -- 172.21.15.26:0/1469931969 >> 172.21.15.26:0/1469931969 conn(0x5556e8cdcd30 msgr2=0x5556e8cdb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 -- 172.21.15.26:0/1469931969 shutdown_connections 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 -- 172.21.15.26:0/1469931969 wait complete. 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.560+0000 7f83d75b5740 1 -- start start 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 Processor -- start 2024-05-20T21:12:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 0x5556e8cf4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce64c0 0x5556e8cf5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1a6380 con 0x5556e8ceb630 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1a77f0 con 0x5556e8ce5ad0 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1a6a50 con 0x5556e8ce64c0 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d75b5740 10 client.0 _ll_register_callbacks cb 0x7f83bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d255d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 0x5556e8cf4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d255d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 0x5556e8cf4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:57650/0 (socket says 172.21.15.26:57650) 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d255d640 1 -- 172.21.15.26:0/3285250446 learned_addr learned my addr 172.21.15.26:0/3285250446 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce64c0 0x5556e8cf5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d255d640 1 -- 172.21.15.26:0/3285250446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce64c0 msgr2=0x5556e8cf5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d255d640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce64c0 0x5556e8cf5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.561+0000 7f83d255d640 1 -- 172.21.15.26:0/3285250446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 msgr2=0x5556e8cf62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d255d640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d255d640 1 -- 172.21.15.26:0/3285250446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e8cf68d0 con 0x5556e8ce5ad0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d2d5e640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d255d640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 0x5556e8cf4e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f83c8031690 tx=0x7f83c8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a926601f8c00be server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3285250446 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8035040 con 0x5556e8ce5ad0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c40c3070 con 0x5556e8ce5ad0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3285250446 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f83c80035f0 con 0x5556e8ce5ad0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3285250446 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f83c8060050 con 0x5556e8ce5ad0 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d75b5740 10 client.29420 fetch_fsmap learned FSMap version 37 2024-05-20T21:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.562+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5556e8d667f0 con 0x5556e8ce5ad0 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3285250446 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83c40c3070 con 0x5556e8ce5ad0 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3285250446 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f83c8044040 con 0x5556e8ce5ad0 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d75b5740 10 client.29420 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d75b5740 10 client.29420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d75b5740 20 client.29420 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8d671c0 0x5556e8d695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d75b5740 4 client.29420 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556e8ce7290 con 0x5556e8d671c0 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8d671c0 0x5556e8d695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83d1d5c640 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8d671c0 0x5556e8d695e0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a899bd87687228b server_cookie=2f58c46d9c2d8404 in_seq=0 out_seq=0 2024-05-20T21:12:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.563+0000 7f83b5ffb640 10 client.29420 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83b5ffb640 1 -- 172.21.15.26:0/3285250446 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5556e8ce7290 con 0x5556e8d671c0 2024-05-20T21:12:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83b5ffb640 10 client.29420 handle_command_reply: tid=0 2024-05-20T21:12:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 client.29420 shutdown 2024-05-20T21:12:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 msgr2=0x5556e8cf4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 0x5556e8cf4e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f83c8031690 tx=0x7f83c8009260 comp rx=0 tx=0).stop 2024-05-20T21:12:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 shutdown_connections 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5556e8d671c0 0x5556e8d695e0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83b5ffb640 0 client.29420 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556e8ceb630 0x5556e8cf62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556e8ce64c0 0x5556e8cf5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 --2- 172.21.15.26:0/3285250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5556e8ce5ad0 0x5556e8cf4e00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 >> 172.21.15.26:0/3285250446 conn(0x5556e8cdcd30 msgr2=0x5556e8cdbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 shutdown_connections 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 1 -- 172.21.15.26:0/3285250446 wait complete. 2024-05-20T21:12:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.775+0000 7f83d75b5740 20 client.29420 trim_cache size 0 max 16384 2024-05-20T21:12:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:12:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:12:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:12:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83b80b2a90 msgr2=0x7f83b80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83b80b2a90 0x7f83b80b4f50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f83c80057c0 tx=0x7f83c8031400 comp rx=0 tx=0).stop 2024-05-20T21:12:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 msgr2=0x7f83cc19ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 0x7f83cc19ff30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f83c400efd0 tx=0x7f83c400c6a0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 shutdown_connections 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f83b80b2a90 0x7f83b80b4f50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83cc103560 0x7f83cc19ff30 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83cc102ba0 0x7f83cc19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 --2- 172.21.15.26:0/2485998353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83cc1019b0 0x7f83cc06acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 >> 172.21.15.26:0/2485998353 conn(0x7f83cc0fcf20 msgr2=0x7f83cc106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 shutdown_connections 2024-05-20T21:12:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:40.776+0000 7f83d47c1640 1 -- 172.21.15.26:0/2485998353 wait complete. 2024-05-20T21:12:41.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22086 4.92 MB/sec execute 46 sec latency 131.079 ms 2024-05-20T21:12:41.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:41 smithi144 ceph-mon[35660]: pgmap v886: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 4.3 MiB/s wr, 229 op/s 2024-05-20T21:12:41.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:41 smithi055 ceph-mon[35213]: pgmap v886: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 4.3 MiB/s wr, 229 op/s 2024-05-20T21:12:41.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:41 smithi026 ceph-mon[31990]: pgmap v886: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 4.3 MiB/s wr, 229 op/s 2024-05-20T21:12:42.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22228 5.05 MB/sec execute 47 sec latency 220.501 ms 2024-05-20T21:12:43.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22361 5.02 MB/sec execute 48 sec latency 144.184 ms 2024-05-20T21:12:43.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:43 smithi144 ceph-mon[35660]: pgmap v887: 97 pgs: 97 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 4.8 MiB/s wr, 239 op/s 2024-05-20T21:12:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:43 smithi055 ceph-mon[35213]: pgmap v887: 97 pgs: 97 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 4.8 MiB/s wr, 239 op/s 2024-05-20T21:12:43.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:43 smithi026 ceph-mon[31990]: pgmap v887: 97 pgs: 97 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 4.8 MiB/s wr, 239 op/s 2024-05-20T21:12:44.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22518 5.11 MB/sec execute 49 sec latency 103.209 ms 2024-05-20T21:12:45.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22670 5.18 MB/sec execute 50 sec latency 170.515 ms 2024-05-20T21:12:45.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:45 smithi144 ceph-mon[35660]: pgmap v888: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 5.0 MiB/s wr, 261 op/s 2024-05-20T21:12:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:45 smithi055 ceph-mon[35213]: pgmap v888: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 5.0 MiB/s wr, 261 op/s 2024-05-20T21:12:45.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:45 smithi026 ceph-mon[31990]: pgmap v888: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 5.0 MiB/s wr, 261 op/s 2024-05-20T21:12:46.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22829 5.17 MB/sec execute 51 sec latency 96.669 ms 2024-05-20T21:12:47.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22958 5.13 MB/sec execute 52 sec latency 200.781 ms 2024-05-20T21:12:47.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:47 smithi144 ceph-mon[35660]: pgmap v889: 97 pgs: 97 active+clean; 271 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 4.8 MiB/s wr, 273 op/s 2024-05-20T21:12:47.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:47 smithi055 ceph-mon[35213]: pgmap v889: 97 pgs: 97 active+clean; 271 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 4.8 MiB/s wr, 273 op/s 2024-05-20T21:12:47.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:47 smithi026 ceph-mon[31990]: pgmap v889: 97 pgs: 97 active+clean; 271 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 4.8 MiB/s wr, 273 op/s 2024-05-20T21:12:48.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23112 5.19 MB/sec execute 53 sec latency 117.350 ms 2024-05-20T21:12:49.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23264 5.20 MB/sec execute 54 sec latency 114.659 ms 2024-05-20T21:12:49.503 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:49 smithi055 ceph-mon[35213]: pgmap v890: 97 pgs: 97 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-05-20T21:12:49.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:49 smithi144 ceph-mon[35660]: pgmap v890: 97 pgs: 97 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-05-20T21:12:49.561 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:49 smithi026 ceph-mon[31990]: pgmap v890: 97 pgs: 97 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-05-20T21:12:50.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23452 5.37 MB/sec execute 55 sec latency 113.485 ms 2024-05-20T21:12:51.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23657 5.57 MB/sec execute 56 sec latency 78.502 ms 2024-05-20T21:12:51.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:51 smithi144 ceph-mon[35660]: pgmap v891: 97 pgs: 97 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-05-20T21:12:51.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:51 smithi055 ceph-mon[35213]: pgmap v891: 97 pgs: 97 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-05-20T21:12:51.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:51 smithi026 ceph-mon[31990]: pgmap v891: 97 pgs: 97 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-05-20T21:12:52.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23804 5.52 MB/sec execute 57 sec latency 103.054 ms 2024-05-20T21:12:53.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23988 5.60 MB/sec execute 58 sec latency 115.377 ms 2024-05-20T21:12:53.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:53 smithi144 ceph-mon[35660]: pgmap v892: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.3 MiB/s wr, 299 op/s 2024-05-20T21:12:53.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:53 smithi055 ceph-mon[35213]: pgmap v892: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.3 MiB/s wr, 299 op/s 2024-05-20T21:12:53.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:53 smithi026 ceph-mon[31990]: pgmap v892: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.3 MiB/s wr, 299 op/s 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 -- 172.21.15.26:0/1353728061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf40754b0 msgr2=0x7efcf4075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1353728061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf40754b0 0x7efcf4075930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efce4009890 tx=0x7efce402a360 comp rx=0 tx=0).stop 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 -- 172.21.15.26:0/1353728061 shutdown_connections 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1353728061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf410f080 0x7efcf4111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1353728061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf40754b0 0x7efcf4075930 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1353728061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 0x7efcf4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 -- 172.21.15.26:0/1353728061 >> 172.21.15.26:0/1353728061 conn(0x7efcf40fe040 msgr2=0x7efcf4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 -- 172.21.15.26:0/1353728061 shutdown_connections 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.908+0000 7efcfbb89640 1 -- 172.21.15.26:0/1353728061 wait complete. 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.909+0000 7efcfbb89640 1 Processor -- start 2024-05-20T21:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.909+0000 7efcfbb89640 1 -- start start 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.909+0000 7efcfbb89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 0x7efcf419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.909+0000 7efcfbb89640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 0x7efcf419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.909+0000 7efcfbb89640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf410f080 0x7efcf41a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfbb89640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4114aa0 con 0x7efcf40754b0 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfbb89640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4114920 con 0x7efcf410f080 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfbb89640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4114c20 con 0x7efcf4076a90 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcf9925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 0x7efcf419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcf9925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 0x7efcf419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40696/0 (socket says 172.21.15.26:40696) 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcf9925640 1 -- 172.21.15.26:0/1875812815 learned_addr learned my addr 172.21.15.26:0/1875812815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf410f080 0x7efcf41a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcf9124640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 0x7efcf419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 -- 172.21.15.26:0/1875812815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 msgr2=0x7efcf419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 0x7efcf419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 -- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 msgr2=0x7efcf419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 0x7efcf419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 -- 172.21.15.26:0/1875812815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf41a6d70 con 0x7efcf410f080 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcf9124640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 0x7efcf419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcf9925640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 0x7efcf419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.910+0000 7efcfa126640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf410f080 0x7efcf41a6700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efcec00ec50 tx=0x7efcec00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.911+0000 7efceaffd640 1 -- 172.21.15.26:0/1875812815 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec003b70 con 0x7efcf410f080 2024-05-20T21:12:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.911+0000 7efceaffd640 1 -- 172.21.15.26:0/1875812815 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcec003d10 con 0x7efcf410f080 2024-05-20T21:12:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.911+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf41a7060 con 0x7efcf410f080 2024-05-20T21:12:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.911+0000 7efceaffd640 1 -- 172.21.15.26:0/1875812815 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0107a0 con 0x7efcf410f080 2024-05-20T21:12:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.911+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf41a7530 con 0x7efcf410f080 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.914+0000 7efcfe980740 1 Processor -- start 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.914+0000 7efceaffd640 1 -- 172.21.15.26:0/1875812815 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efcec003fc0 con 0x7efcf410f080 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.914+0000 7efcfe980740 1 -- start start 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcfe980740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f772ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcfe980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f777bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcfe980740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f77a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcfe980740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56143f6e3c30 con 0x56143f7726c0 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcfe980740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56143f6d37d0 con 0x56143f778220 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcfe980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56143f6d7810 con 0x56143f7730b0 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f777bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9124640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f772ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f777bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59806/0 (socket says 172.21.15.26:59806) 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 -- 172.21.15.26:0/4284101523 learned_addr learned my addr 172.21.15.26:0/4284101523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9124640 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f77a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 -- 172.21.15.26:0/4284101523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 msgr2=0x56143f77a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f77a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 -- 172.21.15.26:0/4284101523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 msgr2=0x56143f772ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f772ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 -- 172.21.15.26:0/4284101523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143f3e8210 con 0x56143f7730b0 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9124640 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f77a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efceaffd640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efce00b2920 0x7efce00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9925640 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f777bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcf00115c0 tx=0x7efcf0011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da158f6812c3793 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.915+0000 7efcf9124640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efce00b2920 0x7efce00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efceaffd640 1 -- 172.21.15.26:0/1875812815 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efcec0e27c0 con 0x7efcf410f080 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcdffff640 1 -- 172.21.15.26:0/4284101523 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf000f6c0 con 0x56143f7730b0 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcdffff640 1 -- 172.21.15.26:0/4284101523 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcf000fcf0 con 0x56143f7730b0 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcdffff640 1 -- 172.21.15.26:0/4284101523 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0019ca0 con 0x56143f7730b0 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcf9124640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efce00b2920 0x7efce00b4de0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efce402a950 tx=0x7efce4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 -- 172.21.15.26:0/4284101523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 msgr2=0x56143f777bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f777bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcf00115c0 tx=0x7efcf0011a80 comp rx=0 tx=0).stop 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 -- 172.21.15.26:0/4284101523 shutdown_connections 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f77a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f777bb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 --2- 172.21.15.26:0/4284101523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f772ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.916+0000 7efcfe980740 1 -- 172.21.15.26:0/4284101523 >> 172.21.15.26:0/4284101523 conn(0x56143f76dd30 msgr2=0x56143f76e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 -- 172.21.15.26:0/4284101523 shutdown_connections 2024-05-20T21:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 -- 172.21.15.26:0/4284101523 wait complete. 2024-05-20T21:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 -- start start 2024-05-20T21:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 Processor -- start 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f77c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f77cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f778220 0x56143f783c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56143f3b9790 con 0x56143f7726c0 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56143f68f890 con 0x56143f7730b0 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56143f77b1d0 con 0x56143f778220 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcfe980740 10 client.0 _ll_register_callbacks cb 0x56143f3e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.917+0000 7efcf9925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f77c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f77cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f77cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:46850/0 (socket says 172.21.15.26:46850) 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f77c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40724/0 (socket says 172.21.15.26:40724) 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 -- 172.21.15.26:0/1149727722 learned_addr learned my addr 172.21.15.26:0/1149727722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcfa126640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f778220 0x56143f783c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 -- 172.21.15.26:0/1149727722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f778220 msgr2=0x56143f783c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f778220 0x56143f783c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 -- 172.21.15.26:0/1149727722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 msgr2=0x56143f77c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f77c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 -- 172.21.15.26:0/1149727722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143f4b4620 con 0x56143f7730b0 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcfa126640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f778220 0x56143f783c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcf9124640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f77cc40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efce402afd0 tx=0x7efce402a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a4022be6883a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.918+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1149727722 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce404e020 con 0x56143f7730b0 2024-05-20T21:12:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1149727722 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efce4004240 con 0x56143f7730b0 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf000fcf0 con 0x56143f7730b0 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1149727722 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efce4059050 con 0x56143f7730b0 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 10 client.29432 fetch_fsmap learned FSMap version 37 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56143f786cd0 con 0x56143f7730b0 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1149727722 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcf000fcf0 con 0x56143f7730b0 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1149727722 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7efce403a070 con 0x56143f7730b0 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 10 client.29432 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 10 client.29432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 20 client.29432 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f788470 0x56143f78a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 4 client.29432 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143f6e5b80 con 0x56143f788470 2024-05-20T21:12:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.919+0000 7efcf9925640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f788470 0x56143f78a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.920+0000 7efcdd7fa640 10 client.29432 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.920+0000 7efcf9925640 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f788470 0x56143f78a880 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fee786ec2bb97c server_cookie=44fcc3d3c94722ff in_seq=0 out_seq=0 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1149727722 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56143f6e5b80 con 0x56143f788470 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcdd7fa640 10 client.29432 handle_command_reply: tid=0 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 client.29432 shutdown 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 msgr2=0x56143f77cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f77cc40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efce402afd0 tx=0x7efce402a360 comp rx=0 tx=0).stop 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 shutdown_connections 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f788470 0x56143f78a880 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f778220 0x56143f783c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f77cc40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcdd7fa640 0 client.29432 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 --2- 172.21.15.26:0/1149727722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f77c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 >> 172.21.15.26:0/1149727722 conn(0x56143f76dd30 msgr2=0x56143f76cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 shutdown_connections 2024-05-20T21:12:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.921+0000 7efcfe980740 1 -- 172.21.15.26:0/1149727722 wait complete. 2024-05-20T21:12:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.922+0000 7efcfe980740 20 client.29432 trim_cache size 0 max 16384 2024-05-20T21:12:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.945+0000 7efcfe980740 1 Processor -- start 2024-05-20T21:12:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.945+0000 7efcfe980740 1 -- start start 2024-05-20T21:12:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.945+0000 7efcfe980740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f774290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcfe980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f7747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcfe980740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f783c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcf9925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f774290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcf9925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f774290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40734/0 (socket says 172.21.15.26:40734) 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcfe980740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41b2e10 con 0x56143f7726c0 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcfe980740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41b06e0 con 0x56143f778220 2024-05-20T21:12:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcfe980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41ae530 con 0x56143f7730b0 2024-05-20T21:12:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcf9925640 1 -- 172.21.15.26:0/644572399 learned_addr learned my addr 172.21.15.26:0/644572399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.946+0000 7efcf9124640 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f7747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.947+0000 7efcf9925640 1 -- 172.21.15.26:0/644572399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 msgr2=0x56143f7747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.947+0000 7efcf9925640 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f7747d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.947+0000 7efcf9925640 1 -- 172.21.15.26:0/644572399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 msgr2=0x56143f783c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.947+0000 7efcf9925640 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f783c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.947+0000 7efcf9925640 1 -- 172.21.15.26:0/644572399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143f773d70 con 0x56143f7726c0 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.947+0000 7efcf9925640 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f774290 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7efcf00096c0 tx=0x7efcf0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ebe8e86e619b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.948+0000 7efcdf7fe640 1 -- 172.21.15.26:0/644572399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0009d90 con 0x56143f7726c0 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.948+0000 7efcdf7fe640 1 -- 172.21.15.26:0/644572399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcf0019810 con 0x56143f7726c0 2024-05-20T21:12:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.948+0000 7efcdf7fe640 1 -- 172.21.15.26:0/644572399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf000f9b0 con 0x56143f7726c0 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 -- 172.21.15.26:0/644572399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 msgr2=0x56143f774290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f774290 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7efcf00096c0 tx=0x7efcf0019720 comp rx=0 tx=0).stop 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 -- 172.21.15.26:0/644572399 shutdown_connections 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f778220 0x56143f783c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7730b0 0x56143f7747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 --2- 172.21.15.26:0/644572399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f7726c0 0x56143f774290 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 -- 172.21.15.26:0/644572399 >> 172.21.15.26:0/644572399 conn(0x56143f76dd30 msgr2=0x56143f77c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 -- 172.21.15.26:0/644572399 shutdown_connections 2024-05-20T21:12:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 -- 172.21.15.26:0/644572399 wait complete. 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.949+0000 7efcfe980740 1 -- start start 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcfe980740 1 Processor -- start 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcfe980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7726c0 0x56143f7813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcfe980740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f781930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcfe980740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f778220 0x56143f782940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcfe980740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41aeba0 con 0x56143f778220 2024-05-20T21:12:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcfe980740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41b0010 con 0x56143f7730b0 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.950+0000 7efcf9925640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7726c0 0x56143f7813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfe980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41af270 con 0x56143f7726c0 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcf9925640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7726c0 0x56143f7813f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:59830/0 (socket says 172.21.15.26:59830) 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcf9925640 1 -- 172.21.15.26:0/1387158491 learned_addr learned my addr 172.21.15.26:0/1387158491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f778220 0x56143f782940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfe980740 10 client.0 _ll_register_callbacks cb 0x7efce40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcf9124640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f781930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 -- 172.21.15.26:0/1387158491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7726c0 msgr2=0x56143f7813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7726c0 0x56143f7813f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 -- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 msgr2=0x56143f781930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f781930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 -- 172.21.15.26:0/1387158491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143f782f30 con 0x56143f778220 2024-05-20T21:12:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcf9124640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f781930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.951+0000 7efcfa126640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f778220 0x56143f782940 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7efcec0e02b0 tx=0x7efcec0a9be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b45e9bf28d2ed7c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1387158491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0a94f0 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1387158491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efcec0a6370 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0019810 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1387158491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efcec0042b0 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcfe980740 10 client.20271 fetch_fsmap learned FSMap version 37 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.952+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56143f7f3130 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1387158491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcf0019810 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1387158491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7efcec091040 con 0x56143f778220 2024-05-20T21:12:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcfe980740 10 client.20271 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcfe980740 10 client.20271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcfe980740 20 client.20271 populate_metadata read hostname 'smithi026' 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcfe980740 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f7f3b00 0x56143f7f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcfe980740 4 client.20271 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56143f6df360 con 0x56143f7f3b00 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.953+0000 7efcf9925640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f7f3b00 0x56143f7f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.954+0000 7efcdd7fa640 10 client.20271 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:53.954+0000 7efcf9925640 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f7f3b00 0x56143f7f5f20 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7401b2146a5b35 server_cookie=f948362002971f6 in_seq=0 out_seq=0 2024-05-20T21:12:54.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24125 5.55 MB/sec execute 59 sec latency 137.753 ms 2024-05-20T21:12:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcdd7fa640 1 -- 172.21.15.26:0/1387158491 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56143f6df360 con 0x56143f7f3b00 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcdd7fa640 10 client.20271 handle_command_reply: tid=0 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 client.20271 shutdown 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f778220 msgr2=0x56143f782940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f778220 0x56143f782940 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7efcec0e02b0 tx=0x7efcec0a9be0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 shutdown_connections 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56143f7f3b00 0x56143f7f5f20 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56143f778220 0x56143f782940 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56143f7730b0 0x56143f781930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcdd7fa640 0 client.20271 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 --2- 172.21.15.26:0/1387158491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56143f7726c0 0x56143f7813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 >> 172.21.15.26:0/1387158491 conn(0x56143f76dd30 msgr2=0x56143f774290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 shutdown_connections 2024-05-20T21:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.126+0000 7efcfe980740 1 -- 172.21.15.26:0/1387158491 wait complete. 2024-05-20T21:12:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfe980740 20 client.20271 trim_cache size 0 max 16384 2024-05-20T21:12:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:12:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:12:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efce00b2920 msgr2=0x7efce00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efce00b2920 0x7efce00b4de0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efce402a950 tx=0x7efce4005d30 comp rx=0 tx=0).stop 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf410f080 msgr2=0x7efcf41a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf410f080 0x7efcf41a6700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efcec00ec50 tx=0x7efcec00c6a0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 shutdown_connections 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efce00b2920 0x7efce00b4de0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcf410f080 0x7efcf41a6700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf4076a90 0x7efcf419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 --2- 172.21.15.26:0/1875812815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf40754b0 0x7efcf419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 >> 172.21.15.26:0/1875812815 conn(0x7efcf40fe040 msgr2=0x7efcf4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:12:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.127+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 shutdown_connections 2024-05-20T21:12:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:12:54.128+0000 7efcfbb89640 1 -- 172.21.15.26:0/1875812815 wait complete. 2024-05-20T21:12:55.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24307 5.56 MB/sec execute 60 sec latency 145.697 ms 2024-05-20T21:12:55.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:55 smithi144 ceph-mon[35660]: pgmap v893: 97 pgs: 97 active+clean; 263 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 3.9 MiB/s wr, 283 op/s 2024-05-20T21:12:55.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:55 smithi055 ceph-mon[35213]: pgmap v893: 97 pgs: 97 active+clean; 263 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 3.9 MiB/s wr, 283 op/s 2024-05-20T21:12:55.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:55 smithi026 ceph-mon[31990]: pgmap v893: 97 pgs: 97 active+clean; 263 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 3.9 MiB/s wr, 283 op/s 2024-05-20T21:12:56.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24489 5.60 MB/sec execute 61 sec latency 100.504 ms 2024-05-20T21:12:57.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24635 5.59 MB/sec execute 62 sec latency 101.518 ms 2024-05-20T21:12:57.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:57 smithi144 ceph-mon[35660]: pgmap v894: 97 pgs: 97 active+clean; 259 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.6 MiB/s wr, 272 op/s 2024-05-20T21:12:57.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:57 smithi055 ceph-mon[35213]: pgmap v894: 97 pgs: 97 active+clean; 259 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.6 MiB/s wr, 272 op/s 2024-05-20T21:12:57.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:57 smithi026 ceph-mon[31990]: pgmap v894: 97 pgs: 97 active+clean; 259 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.6 MiB/s wr, 272 op/s 2024-05-20T21:12:58.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24774 5.57 MB/sec execute 63 sec latency 111.813 ms 2024-05-20T21:12:59.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24963 5.65 MB/sec execute 64 sec latency 88.908 ms 2024-05-20T21:12:59.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:12:59 smithi144 ceph-mon[35660]: pgmap v895: 97 pgs: 97 active+clean; 261 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 276 op/s 2024-05-20T21:12:59.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:12:59 smithi055 ceph-mon[35213]: pgmap v895: 97 pgs: 97 active+clean; 261 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 276 op/s 2024-05-20T21:12:59.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:12:59 smithi026 ceph-mon[31990]: pgmap v895: 97 pgs: 97 active+clean; 261 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 276 op/s 2024-05-20T21:13:00.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25223 5.95 MB/sec execute 65 sec latency 89.391 ms 2024-05-20T21:13:01.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25407 6.00 MB/sec execute 66 sec latency 106.711 ms 2024-05-20T21:13:01.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:01 smithi144 ceph-mon[35660]: pgmap v896: 97 pgs: 97 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-05-20T21:13:01.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:01 smithi055 ceph-mon[35213]: pgmap v896: 97 pgs: 97 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-05-20T21:13:01.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:01 smithi026 ceph-mon[31990]: pgmap v896: 97 pgs: 97 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-05-20T21:13:02.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25576 6.07 MB/sec execute 67 sec latency 122.429 ms 2024-05-20T21:13:03.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25739 6.14 MB/sec execute 68 sec latency 115.154 ms 2024-05-20T21:13:03.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:03 smithi144 ceph-mon[35660]: pgmap v897: 97 pgs: 97 active+clean; 271 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-20T21:13:03.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:03 smithi055 ceph-mon[35213]: pgmap v897: 97 pgs: 97 active+clean; 271 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-20T21:13:03.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:03 smithi026 ceph-mon[31990]: pgmap v897: 97 pgs: 97 active+clean; 271 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-20T21:13:04.114 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25860 6.16 MB/sec execute 69 sec latency 116.608 ms 2024-05-20T21:13:05.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26028 6.25 MB/sec execute 70 sec latency 84.201 ms 2024-05-20T21:13:05.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:05 smithi144 ceph-mon[35660]: pgmap v898: 97 pgs: 97 active+clean; 273 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-05-20T21:13:05.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:05 smithi055 ceph-mon[35213]: pgmap v898: 97 pgs: 97 active+clean; 273 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-05-20T21:13:05.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:05 smithi026 ceph-mon[31990]: pgmap v898: 97 pgs: 97 active+clean; 273 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-05-20T21:13:06.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26234 6.42 MB/sec execute 71 sec latency 87.492 ms 2024-05-20T21:13:07.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26363 6.41 MB/sec execute 72 sec latency 127.071 ms 2024-05-20T21:13:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.260+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/3610106788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb81019b0 msgr2=0x7f8bb8101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.260+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/3610106788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb81019b0 0x7f8bb8101dd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009900 tx=0x7f8ba802a3f0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/3610106788 shutdown_connections 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/3610106788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8103560 0x7f8bb8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/3610106788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb8102ba0 0x7f8bb8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/3610106788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb81019b0 0x7f8bb8101dd0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/3610106788 >> 172.21.15.26:0/3610106788 conn(0x7f8bb80fcf20 msgr2=0x7f8bb80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/3610106788 shutdown_connections 2024-05-20T21:13:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/3610106788 wait complete. 2024-05-20T21:13:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 Processor -- start 2024-05-20T21:13:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.261+0000 7f8bbe2a0640 1 -- start start 2024-05-20T21:13:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbe2a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb81019b0 0x7f8bb806ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbe2a0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8102ba0 0x7f8bb819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbe2a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 0x7f8bb819ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbe2a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810d360 con 0x7f8bb8103560 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbe2a0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810d1e0 con 0x7f8bb8102ba0 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbe2a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810d4e0 con 0x7f8bb81019b0 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbc83d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 0x7f8bb819ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8102ba0 0x7f8bb819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbc83d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 0x7f8bb819ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44012/0 (socket says 172.21.15.26:44012) 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8102ba0 0x7f8bb819d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59588/0 (socket says 172.21.15.26:59588) 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bbc83d640 1 -- 172.21.15.26:0/272176391 learned_addr learned my addr 172.21.15.26:0/272176391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.262+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb81019b0 0x7f8bb806ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb7fff640 1 -- 172.21.15.26:0/272176391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8102ba0 msgr2=0x7f8bb819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8102ba0 0x7f8bb819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb7fff640 1 -- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 msgr2=0x7f8bb819ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 0x7f8bb819ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb7fff640 1 -- 172.21.15.26:0/272176391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb8197780 con 0x7f8bb81019b0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 0x7f8bb819ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb81019b0 0x7f8bb806ac40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009230 tx=0x7f8ba80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb57fa640 1 -- 172.21.15.26:0/272176391 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba803f070 con 0x7f8bb81019b0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb57fa640 1 -- 172.21.15.26:0/272176391 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba80044c0 con 0x7f8bb81019b0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bb57fa640 1 -- 172.21.15.26:0/272176391 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8035070 con 0x7f8bb81019b0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb8197a10 con 0x7f8bb81019b0 2024-05-20T21:13:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.263+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb8197eb0 con 0x7f8bb81019b0 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.266+0000 7f8bb57fa640 1 -- 172.21.15.26:0/272176391 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8ba80037f0 con 0x7f8bb81019b0 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 Processor -- start 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 -- start start 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facedbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facedc0b0 0x556facee0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facee1220 0x556facee36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556face4cc30 con 0x556facee1220 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556face3c7d0 con 0x556facedc0b0 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556face40810 con 0x556facedb6c0 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bbc83d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facedbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facedc0b0 0x556facee0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bbc83d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facedbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.26:50656/0 (socket says 172.21.15.26:50656) 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bbc83d640 1 -- 172.21.15.26:0/2015422662 learned_addr learned my addr 172.21.15.26:0/2015422662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.267+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facee1220 0x556facee36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bb57fa640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ba00b2920 0x7f8ba00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 -- 172.21.15.26:0/2015422662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facedc0b0 msgr2=0x556facee0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facedc0b0 0x556facee0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 -- 172.21.15.26:0/2015422662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facee1220 msgr2=0x556facee36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facee1220 0x556facee36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 -- 172.21.15.26:0/2015422662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556facb51210 con 0x556facedb6c0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bb57fa640 1 -- 172.21.15.26:0/272176391 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8ba8102440 con 0x7f8bb81019b0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ba00b2920 0x7f8ba00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facedbae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00118b0 tx=0x7f8bb0011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c230d3511e42e018 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8b9e7fc640 1 -- 172.21.15.26:0/2015422662 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000fca0 con 0x556facedb6c0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8b9e7fc640 1 -- 172.21.15.26:0/2015422662 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bb001b8c0 con 0x556facedb6c0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.268+0000 7f8b9e7fc640 1 -- 172.21.15.26:0/2015422662 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00109b0 con 0x556facedb6c0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ba00b2920 0x7f8ba00b4de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0012fd0 tx=0x7f8bb0019780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/2015422662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 msgr2=0x556facedbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facedbae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00118b0 tx=0x7f8bb0011d70 comp rx=0 tx=0).stop 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/2015422662 shutdown_connections 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facee1220 0x556facee36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facedc0b0 0x556facee0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/2015422662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facedbae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.269+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/2015422662 >> 172.21.15.26:0/2015422662 conn(0x556faced6d30 msgr2=0x556faced7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/2015422662 shutdown_connections 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/2015422662 wait complete. 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 -- start start 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 Processor -- start 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facee5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedc0b0 0x556facee5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.270+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556facb22790 con 0x556facedc0b0 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556facdf8890 con 0x556facee1220 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556facee41d0 con 0x556facedb6c0 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedc0b0 0x556facee5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bbc83d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facee5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedc0b0 0x556facee5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44028/0 (socket says 172.21.15.26:44028) 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bbc83d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceecc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:59602/0 (socket says 172.21.15.26:59602) 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb77fe640 1 -- 172.21.15.26:0/3367303777 learned_addr learned my addr 172.21.15.26:0/3367303777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bc0fbf740 10 client.0 _ll_register_callbacks cb 0x556facb50a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 -- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 msgr2=0x556faceecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 -- 172.21.15.26:0/3367303777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedc0b0 msgr2=0x556facee5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedc0b0 0x556facee5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 -- 172.21.15.26:0/3367303777 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556facc1d620 con 0x556facedb6c0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facee5780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80fff30 tx=0x7f8ba80c5970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b15a72448cb1005 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8b977fe640 1 -- 172.21.15.26:0/3367303777 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8048070 con 0x556facedb6c0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8b977fe640 1 -- 172.21.15.26:0/3367303777 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8ba80c60a0 con 0x556facedb6c0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.271+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb001b8c0 con 0x556facedb6c0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8b977fe640 1 -- 172.21.15.26:0/3367303777 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8ba80c9220 con 0x556facedb6c0 2024-05-20T21:13:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8bc0fbf740 10 client.29143 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556faceefc70 con 0x556facedb6c0 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8b977fe640 1 -- 172.21.15.26:0/3367303777 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba80dee50 con 0x556facedb6c0 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8b977fe640 1 -- 172.21.15.26:0/3367303777 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f8ba804b060 con 0x556facedb6c0 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8bc0fbf740 10 client.29143 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8bc0fbf740 10 client.29143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8bc0fbf740 20 client.29143 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.272+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facef1300 0x556facef3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.273+0000 7f8bc0fbf740 4 client.29143 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.273+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8bb001b8c0 con 0x556facef1300 2024-05-20T21:13:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.273+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facef1300 0x556facef3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.273+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facef1300 0x556facef3770 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6483a1af69caf8ee server_cookie=6458ca62f13cd8cb in_seq=0 out_seq=0 2024-05-20T21:13:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.273+0000 7f8b977fe640 10 client.29143 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.275+0000 7f8b977fe640 1 -- 172.21.15.26:0/3367303777 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f8bb001b8c0 con 0x556facef1300 2024-05-20T21:13:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.275+0000 7f8b977fe640 10 client.29143 handle_command_reply: tid=0 2024-05-20T21:13:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.275+0000 7f8bc0fbf740 1 client.29143 shutdown 2024-05-20T21:13:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 msgr2=0x556facee5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facee5780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80fff30 tx=0x7f8ba80c5970 comp rx=0 tx=0).stop 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 shutdown_connections 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facef1300 0x556facef3770 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedc0b0 0x556facee5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/3367303777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedb6c0 0x556facee5780 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 >> 172.21.15.26:0/3367303777 conn(0x556faced6d30 msgr2=0x556faced5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8b977fe640 0 client.29143 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 shutdown_connections 2024-05-20T21:13:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/3367303777 wait complete. 2024-05-20T21:13:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.276+0000 7f8bc0fbf740 20 client.29143 trim_cache size 0 max 16384 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.297+0000 7f8bc0fbf740 1 Processor -- start 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.297+0000 7f8bc0fbf740 1 -- start start 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556facee5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceda120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81aa640 con 0x556facedb6c0 2024-05-20T21:13:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81a7f10 con 0x556facee1220 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81a5d60 con 0x556facedc0b0 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556facee5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556facee5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44040/0 (socket says 172.21.15.26:44040) 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 -- 172.21.15.26:0/1653912037 learned_addr learned my addr 172.21.15.26:0/1653912037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceda120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 -- 172.21.15.26:0/1653912037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 msgr2=0x556faceecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 -- 172.21.15.26:0/1653912037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 msgr2=0x556faceda120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceda120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb7fff640 1 -- 172.21.15.26:0/1653912037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556facef3dc0 con 0x556facedb6c0 2024-05-20T21:13:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceda120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.298+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.299+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556facee5c20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80c9da0 tx=0x7f8ba80c5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e531499878f85bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.299+0000 7f8b9dffb640 1 -- 172.21.15.26:0/1653912037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8048070 con 0x556facedb6c0 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.299+0000 7f8b9dffb640 1 -- 172.21.15.26:0/1653912037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba80dee50 con 0x556facedb6c0 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.299+0000 7f8b9dffb640 1 -- 172.21.15.26:0/1653912037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80c5240 con 0x556facedb6c0 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/1653912037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 msgr2=0x556facee5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556facee5c20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80c9da0 tx=0x7f8ba80c5f10 comp rx=0 tx=0).stop 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/1653912037 shutdown_connections 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceda120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/1653912037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556facee5c20 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/1653912037 >> 172.21.15.26:0/1653912037 conn(0x556faced6d30 msgr2=0x556faced9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/1653912037 shutdown_connections 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/1653912037 wait complete. 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.300+0000 7f8bc0fbf740 1 -- start start 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 Processor -- start 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556faceec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceeab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81a63d0 con 0x556facedb6c0 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81a7840 con 0x556facee1220 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81a6aa0 con 0x556facedc0b0 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bc0fbf740 10 client.0 _ll_register_callbacks cb 0x7f8ba8105a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556faceec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556faceec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44048/0 (socket says 172.21.15.26:44048) 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 -- 172.21.15.26:0/141727013 learned_addr learned my addr 172.21.15.26:0/141727013 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bbc83d640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceeab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 -- 172.21.15.26:0/141727013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 msgr2=0x556faceecab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 -- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 msgr2=0x556faceeab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceeab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.301+0000 7f8bb7fff640 1 -- 172.21.15.26:0/141727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556faceeb310 con 0x556facedb6c0 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8bb7fff640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556faceec570 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8032a50 tx=0x7f8ba80c5270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb9a04b80b8502 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8b977fe640 1 -- 172.21.15.26:0/141727013 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8048070 con 0x556facedb6c0 2024-05-20T21:13:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8b977fe640 1 -- 172.21.15.26:0/141727013 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f8ba80d25e0 con 0x556facedb6c0 2024-05-20T21:13:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba80dee50 con 0x556facedb6c0 2024-05-20T21:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8b977fe640 1 -- 172.21.15.26:0/141727013 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8ba80c9880 con 0x556facedb6c0 2024-05-20T21:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8bc0fbf740 10 client.20289 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556faceeb310 con 0x556facedb6c0 2024-05-20T21:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.302+0000 7f8b977fe640 1 -- 172.21.15.26:0/141727013 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba8105bc0 con 0x556facedb6c0 2024-05-20T21:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8b977fe640 1 -- 172.21.15.26:0/141727013 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f8ba80e5020 con 0x556facedb6c0 2024-05-20T21:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bc0fbf740 10 client.20289 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bc0fbf740 10 client.20289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bc0fbf740 20 client.20289 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facf5cfa0 0x556facf5f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bc0fbf740 4 client.20289 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:13:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8ba80dee50 con 0x556facf5cfa0 2024-05-20T21:13:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facf5cfa0 0x556facf5f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8b977fe640 10 client.20289 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.303+0000 7f8bb77fe640 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facf5cfa0 0x556facf5f390 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e6cf9518f3eb20 server_cookie=c1a0d842c72be1d3 in_seq=0 out_seq=0 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.434+0000 7f8b977fe640 1 -- 172.21.15.26:0/141727013 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f8ba80dee50 con 0x556facf5cfa0 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.434+0000 7f8b977fe640 10 client.20289 handle_command_reply: tid=0 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.434+0000 7f8bc0fbf740 1 client.20289 shutdown 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 msgr2=0x556faceec570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556faceec570 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8032a50 tx=0x7f8ba80c5270 comp rx=0 tx=0).stop 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 shutdown_connections 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x556facf5cfa0 0x556facf5f390 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556facee1220 0x556faceeab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556facedc0b0 0x556faceecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 --2- 172.21.15.26:0/141727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556facedb6c0 0x556faceec570 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 >> 172.21.15.26:0/141727013 conn(0x556faced6d30 msgr2=0x556facee5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8b977fe640 0 client.20289 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 shutdown_connections 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 1 -- 172.21.15.26:0/141727013 wait complete. 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.435+0000 7f8bc0fbf740 20 client.20289 trim_cache size 0 max 16384 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:13:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:13:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.436+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ba00b2920 msgr2=0x7f8ba00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.436+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ba00b2920 0x7f8ba00b4de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0012fd0 tx=0x7f8bb0019780 comp rx=0 tx=0).stop 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.436+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb81019b0 msgr2=0x7f8bb806ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.436+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb81019b0 0x7f8bb806ac40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009230 tx=0x7f8ba80040c0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 shutdown_connections 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f8ba00b2920 0x7f8ba00b4de0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8103560 0x7f8bb819ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb8102ba0 0x7f8bb819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 --2- 172.21.15.26:0/272176391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb81019b0 0x7f8bb806ac40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 >> 172.21.15.26:0/272176391 conn(0x7f8bb80fcf20 msgr2=0x7f8bb8106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 shutdown_connections 2024-05-20T21:13:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:07.437+0000 7f8bbe2a0640 1 -- 172.21.15.26:0/272176391 wait complete. 2024-05-20T21:13:07.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:07 smithi144 ceph-mon[35660]: pgmap v899: 97 pgs: 97 active+clean; 276 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.9 MiB/s wr, 259 op/s 2024-05-20T21:13:07.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:07 smithi055 ceph-mon[35213]: pgmap v899: 97 pgs: 97 active+clean; 276 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.9 MiB/s wr, 259 op/s 2024-05-20T21:13:07.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:07 smithi026 ceph-mon[31990]: pgmap v899: 97 pgs: 97 active+clean; 276 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.9 MiB/s wr, 259 op/s 2024-05-20T21:13:08.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26496 6.43 MB/sec execute 73 sec latency 162.339 ms 2024-05-20T21:13:09.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26629 6.43 MB/sec execute 74 sec latency 91.068 ms 2024-05-20T21:13:09.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:09 smithi144 ceph-mon[35660]: pgmap v900: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 4.4 MiB/s wr, 286 op/s 2024-05-20T21:13:09.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:09 smithi055 ceph-mon[35213]: pgmap v900: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 4.4 MiB/s wr, 286 op/s 2024-05-20T21:13:09.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:09 smithi026 ceph-mon[31990]: pgmap v900: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 4.4 MiB/s wr, 286 op/s 2024-05-20T21:13:10.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26782 6.45 MB/sec execute 75 sec latency 94.452 ms 2024-05-20T21:13:11.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26909 6.40 MB/sec execute 76 sec latency 139.412 ms 2024-05-20T21:13:11.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:11 smithi144 ceph-mon[35660]: pgmap v901: 97 pgs: 97 active+clean; 284 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 4.9 MiB/s wr, 242 op/s 2024-05-20T21:13:11.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:11 smithi055 ceph-mon[35213]: pgmap v901: 97 pgs: 97 active+clean; 284 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 4.9 MiB/s wr, 242 op/s 2024-05-20T21:13:11.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:11 smithi026 ceph-mon[31990]: pgmap v901: 97 pgs: 97 active+clean; 284 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 4.9 MiB/s wr, 242 op/s 2024-05-20T21:13:12.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27043 6.35 MB/sec execute 77 sec latency 112.252 ms 2024-05-20T21:13:13.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27200 6.36 MB/sec execute 78 sec latency 120.741 ms 2024-05-20T21:13:13.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:13 smithi144 ceph-mon[35660]: pgmap v902: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 5.1 MiB/s wr, 281 op/s 2024-05-20T21:13:13.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:13 smithi055 ceph-mon[35213]: pgmap v902: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 5.1 MiB/s wr, 281 op/s 2024-05-20T21:13:13.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:13 smithi026 ceph-mon[31990]: pgmap v902: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 5.1 MiB/s wr, 281 op/s 2024-05-20T21:13:14.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27369 6.42 MB/sec execute 79 sec latency 121.925 ms 2024-05-20T21:13:15.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27519 6.40 MB/sec execute 80 sec latency 101.245 ms 2024-05-20T21:13:15.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:15 smithi144 ceph-mon[35660]: pgmap v903: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-05-20T21:13:15.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:15 smithi055 ceph-mon[35213]: pgmap v903: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-05-20T21:13:15.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:15 smithi026 ceph-mon[31990]: pgmap v903: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-05-20T21:13:16.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27683 6.41 MB/sec execute 81 sec latency 129.591 ms 2024-05-20T21:13:17.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27884 6.51 MB/sec execute 82 sec latency 109.004 ms 2024-05-20T21:13:17.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:17 smithi144 ceph-mon[35660]: pgmap v904: 97 pgs: 97 active+clean; 278 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-05-20T21:13:17.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:17 smithi055 ceph-mon[35213]: pgmap v904: 97 pgs: 97 active+clean; 278 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-05-20T21:13:17.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:17 smithi026 ceph-mon[31990]: pgmap v904: 97 pgs: 97 active+clean; 278 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-05-20T21:13:18.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28036 6.51 MB/sec execute 83 sec latency 95.046 ms 2024-05-20T21:13:19.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28192 6.52 MB/sec execute 84 sec latency 93.135 ms 2024-05-20T21:13:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:19 smithi144 ceph-mon[35660]: pgmap v905: 97 pgs: 97 active+clean; 273 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.3 MiB/s wr, 310 op/s 2024-05-20T21:13:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:19 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:13:19.559 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:19 smithi026 ceph-mon[31990]: pgmap v905: 97 pgs: 97 active+clean; 273 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.3 MiB/s wr, 310 op/s 2024-05-20T21:13:19.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:19 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:13:19.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:19 smithi055 ceph-mon[35213]: pgmap v905: 97 pgs: 97 active+clean; 273 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.3 MiB/s wr, 310 op/s 2024-05-20T21:13:19.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:19 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:13:20.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28394 6.57 MB/sec execute 85 sec latency 89.613 ms 2024-05-20T21:13:20.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:20 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:13:20.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:20 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:13:20.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:20 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.569+0000 7f959d3c9640 1 -- 172.21.15.26:0/163364220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598103cd0 msgr2=0x7f9598104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.569+0000 7f959d3c9640 1 --2- 172.21.15.26:0/163364220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598103cd0 0x7f9598104150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f958c009860 tx=0x7f958c02a340 comp rx=0 tx=0).stop 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 -- 172.21.15.26:0/163364220 shutdown_connections 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 --2- 172.21.15.26:0/163364220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9598104690 0x7f959810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 --2- 172.21.15.26:0/163364220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598103cd0 0x7f9598104150 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 --2- 172.21.15.26:0/163364220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9598102ae0 0x7f9598102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 -- 172.21.15.26:0/163364220 >> 172.21.15.26:0/163364220 conn(0x7f95980fe070 msgr2=0x7f9598100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 -- 172.21.15.26:0/163364220 shutdown_connections 2024-05-20T21:13:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 -- 172.21.15.26:0/163364220 wait complete. 2024-05-20T21:13:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.570+0000 7f959d3c9640 1 Processor -- start 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 -- start start 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 0x7f95980767b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9598103cd0 0x7f9598074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9598104690 0x7f9598075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f959810e4a0 con 0x7f9598103cd0 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f959810e320 con 0x7f9598102ae0 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f959d3c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f959810e620 con 0x7f9598104690 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 0x7f95980767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f95967fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9598103cd0 0x7f9598074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 0x7f95980767b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54540/0 (socket says 172.21.15.26:54540) 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f9596ffd640 1 -- 172.21.15.26:0/3258930254 learned_addr learned my addr 172.21.15.26:0/3258930254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.571+0000 7f95977fe640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9598104690 0x7f9598075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f9596ffd640 1 -- 172.21.15.26:0/3258930254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9598104690 msgr2=0x7f9598075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f9596ffd640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9598104690 0x7f9598075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f9596ffd640 1 -- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9598103cd0 msgr2=0x7f9598074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f9596ffd640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9598103cd0 0x7f9598074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f9596ffd640 1 -- 172.21.15.26:0/3258930254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9598075c40 con 0x7f9598102ae0 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f9596ffd640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 0x7f95980767b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f958800c5d0 tx=0x7f958800ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f957ffff640 1 -- 172.21.15.26:0/3258930254 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588004020 con 0x7f9598102ae0 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f957ffff640 1 -- 172.21.15.26:0/3258930254 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95880035f0 con 0x7f9598102ae0 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981987d0 con 0x7f9598102ae0 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f957ffff640 1 -- 172.21.15.26:0/3258930254 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958801e720 con 0x7f9598102ae0 2024-05-20T21:13:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.572+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9598198d40 con 0x7f9598102ae0 2024-05-20T21:13:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.575+0000 7f957ffff640 1 -- 172.21.15.26:0/3258930254 <== mon.2 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9588025020 con 0x7f9598102ae0 2024-05-20T21:13:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 Processor -- start 2024-05-20T21:13:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 -- start start 2024-05-20T21:13:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf7012ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7017bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fdf6f83c30 con 0x55fdf70126c0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fdf6f737d0 con 0x55fdf70130b0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95a01bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fdf6f77810 con 0x55fdf7018220 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f957ffff640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f95800b2920 0x7f95800b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf7012ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f95977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf7012ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49852/0 (socket says 172.21.15.26:49852) 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.576+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7017bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f95977fe640 1 -- 172.21.15.26:0/184788265 learned_addr learned my addr 172.21.15.26:0/184788265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f95967fc640 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f957ffff640 1 -- 172.21.15.26:0/3258930254 <== mon.2 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f95880ea280 con 0x7f9598102ae0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f95977fe640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f95800b2920 0x7f95800b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f9596ffd640 1 -- 172.21.15.26:0/184788265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 msgr2=0x55fdf701a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f9596ffd640 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f9596ffd640 1 -- 172.21.15.26:0/184788265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 msgr2=0x55fdf7012ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f9596ffd640 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf7012ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f9596ffd640 1 -- 172.21.15.26:0/184788265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdf6c88210 con 0x55fdf70130b0 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f9596ffd640 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7017bb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f95880e7d70 tx=0x7f9588003fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11aaa39ded0d7972 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.577+0000 7f95977fe640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f95800b2920 0x7f95800b4de0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55fdf7013910 tx=0x7f95900095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.578+0000 7f957d7fa640 1 -- 172.21.15.26:0/184788265 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880b1840 con 0x55fdf70130b0 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.578+0000 7f957d7fa640 1 -- 172.21.15.26:0/184788265 <== mon.2 v2:172.21.15.55:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95880b5bf0 con 0x55fdf70130b0 2024-05-20T21:13:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.578+0000 7f957d7fa640 1 -- 172.21.15.26:0/184788265 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880adad0 con 0x55fdf70130b0 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.578+0000 7f95a01bd740 1 -- 172.21.15.26:0/184788265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 msgr2=0x55fdf7017bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7017bb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f95880e7d70 tx=0x7f9588003fb0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 -- 172.21.15.26:0/184788265 shutdown_connections 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7017bb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- 172.21.15.26:0/184788265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf7012ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 -- 172.21.15.26:0/184788265 >> 172.21.15.26:0/184788265 conn(0x55fdf700dd30 msgr2=0x55fdf700e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 -- 172.21.15.26:0/184788265 shutdown_connections 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 -- 172.21.15.26:0/184788265 wait complete. 2024-05-20T21:13:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 -- start start 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 Processor -- start 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf701cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.579+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf7018220 0x55fdf7023d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95a01bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fdf6c59790 con 0x55fdf70126c0 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95a01bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fdf6f2f890 con 0x55fdf7018220 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95a01bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fdf701b1d0 con 0x55fdf70130b0 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95a01bd740 10 client.0 _ll_register_callbacks cb 0x55fdf6c87a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49860/0 (socket says 172.21.15.26:49860) 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 -- 172.21.15.26:0/2103576736 learned_addr learned my addr 172.21.15.26:0/2103576736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95977fe640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf7018220 0x55fdf7023d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95967fc640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf701cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 -- 172.21.15.26:0/2103576736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 msgr2=0x55fdf701cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf701cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 -- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf7018220 msgr2=0x55fdf7023d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf7018220 0x55fdf7023d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 -- 172.21.15.26:0/2103576736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdf6d54620 con 0x55fdf70126c0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95967fc640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf701cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f9596ffd640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c810 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f95880ad8c0 tx=0x7f95880b1610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6ef8014cfd863c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f957aff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f957aff5640 1 -- 172.21.15.26:0/2103576736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588003aa0 con 0x55fdf70126c0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f957aff5640 1 -- 172.21.15.26:0/2103576736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f95880ba210 con 0x55fdf70126c0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.580+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95880b5bf0 con 0x55fdf70126c0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f957aff5640 1 -- 172.21.15.26:0/2103576736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95880ba450 con 0x55fdf70126c0 2024-05-20T21:13:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 10 client.20301 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fdf7026d70 con 0x55fdf70126c0 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f957aff5640 1 -- 172.21.15.26:0/2103576736 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95880b5bf0 con 0x55fdf70126c0 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f957aff5640 1 -- 172.21.15.26:0/2103576736 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f95880b0aa0 con 0x55fdf70126c0 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 10 client.20301 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 10 client.20301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 20 client.20301 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf702a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 4 client.20301 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.581+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdf6f85b80 con 0x55fdf70280c0 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.582+0000 7f95967fc640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf702a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.582+0000 7f957aff5640 10 client.20301 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.582+0000 7f95967fc640 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf702a4b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51163e10d231fdb0 server_cookie=70125a3c497cc631 in_seq=0 out_seq=0 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f957aff5640 1 -- 172.21.15.26:0/2103576736 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fdf6f85b80 con 0x55fdf70280c0 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f957aff5640 10 client.20301 handle_command_reply: tid=0 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 client.20301 shutdown 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 msgr2=0x55fdf701c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c810 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f95880ad8c0 tx=0x7f95880b1610 comp rx=0 tx=0).stop 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 shutdown_connections 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf702a4b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf7018220 0x55fdf7023d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f957aff5640 0 client.20301 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf701cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2103576736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c810 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 >> 172.21.15.26:0/2103576736 conn(0x55fdf700dd30 msgr2=0x55fdf700cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 shutdown_connections 2024-05-20T21:13:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 1 -- 172.21.15.26:0/2103576736 wait complete. 2024-05-20T21:13:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.584+0000 7f95a01bd740 20 client.20301 trim_cache size 0 max 16384 2024-05-20T21:13:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 Processor -- start 2024-05-20T21:13:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 -- start start 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7023d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95981aa580 con 0x55fdf70126c0 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a7e50 con 0x55fdf70130b0 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.608+0000 7f95a01bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a5ca0 con 0x55fdf7018220 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49862/0 (socket says 172.21.15.26:49862) 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 -- 172.21.15.26:0/1790194811 learned_addr learned my addr 172.21.15.26:0/1790194811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f95967fc640 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7023d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f95977fe640 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 -- 172.21.15.26:0/1790194811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 msgr2=0x55fdf701c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 -- 172.21.15.26:0/1790194811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 msgr2=0x55fdf7023d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7023d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 -- 172.21.15.26:0/1790194811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdf700db60 con 0x55fdf70126c0 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f9596ffd640 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c420 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9588000c00 tx=0x7f9588003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a429304a87a6b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f957cff9640 1 -- 172.21.15.26:0/1790194811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880b5bf0 con 0x55fdf70126c0 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f957cff9640 1 -- 172.21.15.26:0/1790194811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95880ade30 con 0x55fdf70126c0 2024-05-20T21:13:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.609+0000 7f957cff9640 1 -- 172.21.15.26:0/1790194811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588003aa0 con 0x55fdf70126c0 2024-05-20T21:13:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 -- 172.21.15.26:0/1790194811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 msgr2=0x55fdf701c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c420 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9588000c00 tx=0x7f9588003fb0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 -- 172.21.15.26:0/1790194811 shutdown_connections 2024-05-20T21:13:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf7018220 0x55fdf701c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70130b0 0x55fdf7023d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 --2- 172.21.15.26:0/1790194811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf70126c0 0x55fdf701c420 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.610+0000 7f95a01bd740 1 -- 172.21.15.26:0/1790194811 >> 172.21.15.26:0/1790194811 conn(0x55fdf700dd30 msgr2=0x55fdf7010d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 -- 172.21.15.26:0/1790194811 shutdown_connections 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 -- 172.21.15.26:0/1790194811 wait complete. 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 -- start start 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 Processor -- start 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 0x55fdf7021150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf7021690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.611+0000 7f95a01bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf7018220 0x55fdf70226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95a01bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a6310 con 0x55fdf7018220 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95a01bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a7780 con 0x55fdf70126c0 2024-05-20T21:13:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95a01bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a69e0 con 0x55fdf70130b0 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95a01bd740 10 client.0 _ll_register_callbacks cb 0x7f9590009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 0x55fdf7021150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 0x55fdf7021150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:54582/0 (socket says 172.21.15.26:54582) 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 -- 172.21.15.26:0/2532967714 learned_addr learned my addr 172.21.15.26:0/2532967714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95977fe640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf7018220 0x55fdf70226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95967fc640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf7021690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 -- 172.21.15.26:0/2532967714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 msgr2=0x55fdf7021690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf7021690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 -- 172.21.15.26:0/2532967714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf7018220 msgr2=0x55fdf70226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf7018220 0x55fdf70226a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 -- 172.21.15.26:0/2532967714 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdf7022c90 con 0x55fdf70126c0 2024-05-20T21:13:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f95967fc640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf7021690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f9596ffd640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 0x55fdf7021150 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f95880b0e40 tx=0x7f9588004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4716837411c7ab0c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.612+0000 7f957aff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f957aff5640 1 -- 172.21.15.26:0/2532967714 <== mon.2 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880b1610 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f957aff5640 1 -- 172.21.15.26:0/2532967714 <== mon.2 v2:172.21.15.55:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f95880041c0 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95880ade30 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f957aff5640 1 -- 172.21.15.26:0/2532967714 <== mon.2 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95880d2ba0 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 10 client.29471 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fdf7022c90 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f957aff5640 1 -- 172.21.15.26:0/2532967714 <== mon.2 v2:172.21.15.55:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95880ade30 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f957aff5640 1 -- 172.21.15.26:0/2532967714 <== mon.2 v2:172.21.15.55:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f95880ad1b0 con 0x55fdf70126c0 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 10 client.29471 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 10 client.29471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 20 client.29471 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf7094bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 4 client.29471 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:13:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdf6f7f360 con 0x55fdf70280c0 2024-05-20T21:13:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.613+0000 7f95967fc640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf7094bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.614+0000 7f95967fc640 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf7094bd0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4b4ba7b57e4905 server_cookie=4be868a7debd8d06 in_seq=0 out_seq=0 2024-05-20T21:13:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.614+0000 7f957aff5640 10 client.29471 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:20.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:20 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:13:20.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:20 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:13:20.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:20 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:13:20.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:20 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:13:20.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:20 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:13:20.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:20 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:13:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.761+0000 7f957aff5640 1 -- 172.21.15.26:0/2532967714 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fdf6f7f360 con 0x55fdf70280c0 2024-05-20T21:13:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.761+0000 7f957aff5640 10 client.29471 handle_command_reply: tid=0 2024-05-20T21:13:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.761+0000 7f95a01bd740 1 client.29471 shutdown 2024-05-20T21:13:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.761+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 msgr2=0x55fdf7021150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.761+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 0x55fdf7021150 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f95880b0e40 tx=0x7f9588004750 comp rx=0 tx=0).stop 2024-05-20T21:13:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.762+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 shutdown_connections 2024-05-20T21:13:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.762+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55fdf70280c0 0x55fdf7094bd0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.762+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdf7018220 0x55fdf70226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.762+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdf70130b0 0x55fdf7021690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.762+0000 7f95a01bd740 1 --2- 172.21.15.26:0/2532967714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fdf70126c0 0x55fdf7021150 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.763+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 >> 172.21.15.26:0/2532967714 conn(0x55fdf700dd30 msgr2=0x55fdf7010d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.763+0000 7f957aff5640 0 client.29471 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.764+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 shutdown_connections 2024-05-20T21:13:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.764+0000 7f95a01bd740 1 -- 172.21.15.26:0/2532967714 wait complete. 2024-05-20T21:13:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.768+0000 7f95a01bd740 20 client.29471 trim_cache size 0 max 16384 2024-05-20T21:13:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:13:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:13:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.769+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f95800b2920 msgr2=0x7f95800b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.769+0000 7f959d3c9640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f95800b2920 0x7f95800b4de0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55fdf7013910 tx=0x7f95900095c0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.769+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 msgr2=0x7f95980767b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.769+0000 7f959d3c9640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 0x7f95980767b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f958800c5d0 tx=0x7f958800ca90 comp rx=0 tx=0).stop 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 shutdown_connections 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f95800b2920 0x7f95800b4de0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9598104690 0x7f9598075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9598103cd0 0x7f9598074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 --2- 172.21.15.26:0/3258930254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9598102ae0 0x7f95980767b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 >> 172.21.15.26:0/3258930254 conn(0x7f95980fe070 msgr2=0x7f9598107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 shutdown_connections 2024-05-20T21:13:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:20.770+0000 7f959d3c9640 1 -- 172.21.15.26:0/3258930254 wait complete. 2024-05-20T21:13:21.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28571 6.58 MB/sec execute 86 sec latency 177.595 ms 2024-05-20T21:13:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:21 smithi144 ceph-mon[35660]: pgmap v906: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-20T21:13:21.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:21 smithi055 ceph-mon[35213]: pgmap v906: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-20T21:13:21.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:21 smithi026 ceph-mon[31990]: pgmap v906: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-20T21:13:22.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28709 6.57 MB/sec execute 87 sec latency 97.603 ms 2024-05-20T21:13:23.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28887 6.58 MB/sec execute 88 sec latency 87.741 ms 2024-05-20T21:13:23.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:23 smithi144 ceph-mon[35660]: pgmap v907: 97 pgs: 97 active+clean; 266 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-05-20T21:13:23.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:23 smithi055 ceph-mon[35213]: pgmap v907: 97 pgs: 97 active+clean; 266 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-05-20T21:13:23.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:23 smithi026 ceph-mon[31990]: pgmap v907: 97 pgs: 97 active+clean; 266 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-05-20T21:13:24.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29053 6.60 MB/sec execute 89 sec latency 90.653 ms 2024-05-20T21:13:25.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29191 6.60 MB/sec execute 90 sec latency 133.626 ms 2024-05-20T21:13:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:25 smithi144 ceph-mon[35660]: pgmap v908: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-05-20T21:13:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:25 smithi055 ceph-mon[35213]: pgmap v908: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-05-20T21:13:25.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:25 smithi026 ceph-mon[31990]: pgmap v908: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-05-20T21:13:26.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29338 6.60 MB/sec execute 91 sec latency 98.128 ms 2024-05-20T21:13:27.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29552 6.73 MB/sec execute 92 sec latency 111.499 ms 2024-05-20T21:13:27.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:27 smithi144 ceph-mon[35660]: pgmap v909: 97 pgs: 97 active+clean; 270 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.8 MiB/s wr, 301 op/s 2024-05-20T21:13:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:27 smithi055 ceph-mon[35213]: pgmap v909: 97 pgs: 97 active+clean; 270 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.8 MiB/s wr, 301 op/s 2024-05-20T21:13:27.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:27 smithi026 ceph-mon[31990]: pgmap v909: 97 pgs: 97 active+clean; 270 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.8 MiB/s wr, 301 op/s 2024-05-20T21:13:28.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29787 6.83 MB/sec execute 93 sec latency 108.729 ms 2024-05-20T21:13:29.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29984 6.87 MB/sec execute 94 sec latency 113.099 ms 2024-05-20T21:13:29.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:29 smithi144 ceph-mon[35660]: pgmap v910: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-20T21:13:29.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:29 smithi055 ceph-mon[35213]: pgmap v910: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-20T21:13:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:29 smithi026 ceph-mon[31990]: pgmap v910: 97 pgs: 97 active+clean; 272 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-20T21:13:30.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30092 6.84 MB/sec execute 95 sec latency 104.271 ms 2024-05-20T21:13:31.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30216 6.88 MB/sec execute 96 sec latency 124.656 ms 2024-05-20T21:13:31.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:31 smithi144 ceph-mon[35660]: pgmap v911: 97 pgs: 97 active+clean; 276 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-05-20T21:13:31.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:31 smithi055 ceph-mon[35213]: pgmap v911: 97 pgs: 97 active+clean; 276 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-05-20T21:13:31.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:31 smithi026 ceph-mon[31990]: pgmap v911: 97 pgs: 97 active+clean; 276 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-05-20T21:13:32.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30369 6.90 MB/sec execute 97 sec latency 121.451 ms 2024-05-20T21:13:33.119 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30578 7.00 MB/sec execute 98 sec latency 103.310 ms 2024-05-20T21:13:33.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:33 smithi144 ceph-mon[35660]: pgmap v912: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.3 MiB/s wr, 266 op/s 2024-05-20T21:13:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:33 smithi055 ceph-mon[35213]: pgmap v912: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.3 MiB/s wr, 266 op/s 2024-05-20T21:13:33.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:33 smithi026 ceph-mon[31990]: pgmap v912: 97 pgs: 97 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.3 MiB/s wr, 266 op/s 2024-05-20T21:13:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 -- 172.21.15.26:0/3290264226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 msgr2=0x7fe64c074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 --2- 172.21.15.26:0/3290264226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c074ee0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009900 tx=0x7fe63c02a410 comp rx=0 tx=0).stop 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 -- 172.21.15.26:0/3290264226 shutdown_connections 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 --2- 172.21.15.26:0/3290264226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 0x7fe64c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 --2- 172.21.15.26:0/3290264226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 --2- 172.21.15.26:0/3290264226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c074ee0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 -- 172.21.15.26:0/3290264226 >> 172.21.15.26:0/3290264226 conn(0x7fe64c0fe070 msgr2=0x7fe64c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 -- 172.21.15.26:0/3290264226 shutdown_connections 2024-05-20T21:13:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.893+0000 7fe651be4640 1 -- 172.21.15.26:0/3290264226 wait complete. 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.894+0000 7fe651be4640 1 Processor -- start 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.894+0000 7fe651be4640 1 -- start start 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe651be4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe651be4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe651be4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 0x7fe64c1a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe651be4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c110710 con 0x7fe64c076a90 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe651be4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c110590 con 0x7fe64c0754b0 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe651be4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c110890 con 0x7fe64c10a9c0 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.26:40808/0 (socket says 172.21.15.26:40808) 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1178279282 learned_addr learned my addr 172.21.15.26:0/1178279282 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64bfff640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 0x7fe64c1a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64bfff640 1 -- 172.21.15.26:0/1178279282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 msgr2=0x7fe64c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64bfff640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64bfff640 1 -- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 msgr2=0x7fe64c19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64bfff640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64bfff640 1 -- 172.21.15.26:0/1178279282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1a27e0 con 0x7fe64c10a9c0 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64affd640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.895+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.896+0000 7fe64bfff640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 0x7fe64c1a2170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe644009fd0 tx=0x7fe64400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.896+0000 7fe648ff9640 1 -- 172.21.15.26:0/1178279282 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64400ce20 con 0x7fe64c10a9c0 2024-05-20T21:13:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.896+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c1a2ad0 con 0x7fe64c10a9c0 2024-05-20T21:13:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.896+0000 7fe648ff9640 1 -- 172.21.15.26:0/1178279282 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe644005020 con 0x7fe64c10a9c0 2024-05-20T21:13:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.896+0000 7fe648ff9640 1 -- 172.21.15.26:0/1178279282 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644010690 con 0x7fe64c10a9c0 2024-05-20T21:13:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.896+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1a3070 con 0x7fe64c10a9c0 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.899+0000 7fe6549dc740 1 Processor -- start 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.899+0000 7fe648ff9640 1 -- 172.21.15.26:0/1178279282 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe644004a40 con 0x7fe64c10a9c0 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.899+0000 7fe6549dc740 1 -- start start 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe6549dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5641eb644c30 con 0x5641eb6d36c0 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe6549dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641eb6347d0 con 0x5641eb6d9220 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe6549dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641eb638810 con 0x5641eb6d40b0 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60406/0 (socket says 172.21.15.26:60406) 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64affd640 1 -- 172.21.15.26:0/1501534726 learned_addr learned my addr 172.21.15.26:0/1501534726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64affd640 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1501534726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 msgr2=0x5641eb6db6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6db6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1501534726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 msgr2=0x5641eb6d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1501534726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641eb349210 con 0x5641eb6d40b0 2024-05-20T21:13:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.900+0000 7fe648ff9640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6340b2920 0x7fe6340b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.901+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6d8bb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe63c02a930 tx=0x7fe63c004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b64902820d82c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.901+0000 7fe64affd640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6340b2920 0x7fe6340b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.901+0000 7fe648ff9640 1 -- 172.21.15.26:0/1178279282 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe6440e2930 con 0x7fe64c10a9c0 2024-05-20T21:13:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.901+0000 7fe631ffb640 1 -- 172.21.15.26:0/1501534726 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c035030 con 0x5641eb6d40b0 2024-05-20T21:13:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.901+0000 7fe631ffb640 1 -- 172.21.15.26:0/1501534726 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe63c040070 con 0x5641eb6d40b0 2024-05-20T21:13:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.901+0000 7fe631ffb640 1 -- 172.21.15.26:0/1501534726 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c045e30 con 0x5641eb6d40b0 2024-05-20T21:13:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe64affd640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6340b2920 0x7fe6340b4de0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5641eb6d4910 tx=0x7fe640009b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 -- 172.21.15.26:0/1501534726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 msgr2=0x5641eb6d8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6d8bb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe63c02a930 tx=0x7fe63c004470 comp rx=0 tx=0).stop 2024-05-20T21:13:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 -- 172.21.15.26:0/1501534726 shutdown_connections 2024-05-20T21:13:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6d8bb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1501534726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 -- 172.21.15.26:0/1501534726 >> 172.21.15.26:0/1501534726 conn(0x5641eb6ced30 msgr2=0x5641eb6cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 -- 172.21.15.26:0/1501534726 shutdown_connections 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.902+0000 7fe6549dc740 1 -- 172.21.15.26:0/1501534726 wait complete. 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 -- start start 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 Processor -- start 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d40b0 0x5641eb6ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d9220 0x5641eb6e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5641eb31a790 con 0x5641eb6d36c0 2024-05-20T21:13:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641eb5f0890 con 0x5641eb6d40b0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641eb6dc170 con 0x5641eb6d9220 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6dd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60408/0 (socket says 172.21.15.26:60408) 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1820607494 learned_addr learned my addr 172.21.15.26:0/1820607494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe6549dc740 10 client.0 _ll_register_callbacks cb 0x5641eb348a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.903+0000 7fe64affd640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d40b0 0x5641eb6ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64bfff640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d9220 0x5641eb6e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1820607494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d9220 msgr2=0x5641eb6e4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d9220 0x5641eb6e4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d40b0 msgr2=0x5641eb6ddcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d40b0 0x5641eb6ddcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64b7fe640 1 -- 172.21.15.26:0/1820607494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641eb415620 con 0x5641eb6d36c0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6dd7b0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe63c044000 tx=0x7fe63c045da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb61a3f02120426b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe62affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe62affd640 1 -- 172.21.15.26:0/1820607494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c035040 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe62affd640 1 -- 172.21.15.26:0/1820607494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe63c003800 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.904+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe63c040070 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe62affd640 1 -- 172.21.15.26:0/1820607494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe63c003a00 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 10 client.20319 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641eb6e7d10 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe62affd640 1 -- 172.21.15.26:0/1820607494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe63c040070 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe62affd640 1 -- 172.21.15.26:0/1820607494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7fe63c031b70 con 0x5641eb6d36c0 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 10 client.20319 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 10 client.20319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 20 client.20319 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb6e9060 0x5641eb6eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 4 client.20319 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:13:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641eb646b80 con 0x5641eb6e9060 2024-05-20T21:13:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.905+0000 7fe64affd640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb6e9060 0x5641eb6eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.906+0000 7fe62affd640 10 client.20319 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.906+0000 7fe64affd640 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb6e9060 0x5641eb6eb450 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f114e55b618c382 server_cookie=9342b5fdb46f4d8 in_seq=0 out_seq=0 2024-05-20T21:13:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.907+0000 7fe62affd640 1 -- 172.21.15.26:0/1820607494 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5641eb646b80 con 0x5641eb6e9060 2024-05-20T21:13:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.907+0000 7fe62affd640 10 client.20319 handle_command_reply: tid=0 2024-05-20T21:13:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.907+0000 7fe6549dc740 1 client.20319 shutdown 2024-05-20T21:13:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 msgr2=0x5641eb6dd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6dd7b0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe63c044000 tx=0x7fe63c045da0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 shutdown_connections 2024-05-20T21:13:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb6e9060 0x5641eb6eb450 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d9220 0x5641eb6e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d40b0 0x5641eb6ddcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 --2- 172.21.15.26:0/1820607494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6dd7b0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 >> 172.21.15.26:0/1820607494 conn(0x5641eb6ced30 msgr2=0x5641eb6cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe62affd640 0 client.20319 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 shutdown_connections 2024-05-20T21:13:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 1 -- 172.21.15.26:0/1820607494 wait complete. 2024-05-20T21:13:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.908+0000 7fe6549dc740 20 client.20319 trim_cache size 0 max 16384 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.939+0000 7fe6549dc740 1 Processor -- start 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.939+0000 7fe6549dc740 1 -- start start 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe6549dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1ae6e0 con 0x5641eb6d36c0 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe6549dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1abfb0 con 0x5641eb6d9220 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe6549dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1a9e00 con 0x5641eb6d40b0 2024-05-20T21:13:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60418/0 (socket says 172.21.15.26:60418) 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 -- 172.21.15.26:0/3828756069 learned_addr learned my addr 172.21.15.26:0/3828756069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64affd640 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64bfff640 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 -- 172.21.15.26:0/3828756069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 msgr2=0x5641eb6dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6dcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 -- 172.21.15.26:0/3828756069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 msgr2=0x5641eb6dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6dd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64b7fe640 1 -- 172.21.15.26:0/3828756069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641eb6eba30 con 0x5641eb6d36c0 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.940+0000 7fe64affd640 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.941+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d23a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9af59b447d16b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.941+0000 7fe6317fa640 1 -- 172.21.15.26:0/3828756069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c035040 con 0x5641eb6d36c0 2024-05-20T21:13:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.941+0000 7fe6317fa640 1 -- 172.21.15.26:0/3828756069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe63c040040 con 0x5641eb6d36c0 2024-05-20T21:13:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.941+0000 7fe6317fa640 1 -- 172.21.15.26:0/3828756069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c03cde0 con 0x5641eb6d36c0 2024-05-20T21:13:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 -- 172.21.15.26:0/3828756069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 msgr2=0x5641eb6d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d23a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c005900 comp rx=0 tx=0).stop 2024-05-20T21:13:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 -- 172.21.15.26:0/3828756069 shutdown_connections 2024-05-20T21:13:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d9220 0x5641eb6dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 --2- 172.21.15.26:0/3828756069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d36c0 0x5641eb6d23a0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 -- 172.21.15.26:0/3828756069 >> 172.21.15.26:0/3828756069 conn(0x5641eb6ced30 msgr2=0x5641eb6d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 -- 172.21.15.26:0/3828756069 shutdown_connections 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 -- 172.21.15.26:0/3828756069 wait complete. 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 -- start start 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.942+0000 7fe6549dc740 1 Processor -- start 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d36c0 0x5641eb6e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 0x5641eb6e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1aa470 con 0x5641eb6d9220 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1ab8e0 con 0x5641eb6d36c0 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1aab40 con 0x5641eb6d40b0 2024-05-20T21:13:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe6549dc740 10 client.0 _ll_register_callbacks cb 0x7fe640009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 0x5641eb6e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d36c0 0x5641eb6e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 0x5641eb6e2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60428/0 (socket says 172.21.15.26:60428) 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 -- 172.21.15.26:0/2363332782 learned_addr learned my addr 172.21.15.26:0/2363332782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64affd640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 -- 172.21.15.26:0/2363332782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 msgr2=0x5641eb6e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 -- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d36c0 msgr2=0x5641eb6e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d36c0 0x5641eb6e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64bfff640 1 -- 172.21.15.26:0/2363332782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641eb6e35a0 con 0x5641eb6d9220 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64affd640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.943+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d36c0 0x5641eb6e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T21:13:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe64bfff640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 0x5641eb6e2fb0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe644005fd0 tx=0x7fe6440aa650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4afcb6c2fec1965 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe62affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe62affd640 1 -- 172.21.15.26:0/2363332782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440c9060 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe62affd640 1 -- 172.21.15.26:0/2363332782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fe6440051d0 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe63c040040 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe62affd640 1 -- 172.21.15.26:0/2363332782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe6440b2cf0 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe6549dc740 10 client.20325 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641eb753ba0 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe62affd640 1 -- 172.21.15.26:0/2363332782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6440b2860 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.944+0000 7fe62affd640 1 -- 172.21.15.26:0/2363332782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7fe6440a8b60 con 0x5641eb6d9220 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe6549dc740 10 client.20325 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe6549dc740 10 client.20325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe6549dc740 20 client.20325 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe6549dc740 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb7543c0 0x5641eb7567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe6549dc740 4 client.20325 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:13:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe63c040040 con 0x5641eb7543c0 2024-05-20T21:13:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb7543c0 0x5641eb7567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe62affd640 10 client.20325 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:33.945+0000 7fe64b7fe640 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb7543c0 0x5641eb7567e0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5efea9e5c6bb521 server_cookie=134d7057b4dfd6e4 in_seq=0 out_seq=0 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe62affd640 1 -- 172.21.15.26:0/2363332782 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe63c040040 con 0x5641eb7543c0 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe62affd640 10 client.20325 handle_command_reply: tid=0 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 client.20325 shutdown 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 msgr2=0x5641eb6e2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 0x5641eb6e2fb0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe644005fd0 tx=0x7fe6440aa650 comp rx=0 tx=0).stop 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 shutdown_connections 2024-05-20T21:13:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x5641eb7543c0 0x5641eb7567e0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641eb6d9220 0x5641eb6e2fb0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe62affd640 0 client.20325 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641eb6d40b0 0x5641eb6e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 --2- 172.21.15.26:0/2363332782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641eb6d36c0 0x5641eb6e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 >> 172.21.15.26:0/2363332782 conn(0x5641eb6ced30 msgr2=0x5641eb6cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 shutdown_connections 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 1 -- 172.21.15.26:0/2363332782 wait complete. 2024-05-20T21:13:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.086+0000 7fe6549dc740 20 client.20325 trim_cache size 0 max 16384 2024-05-20T21:13:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:13:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:13:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6340b2920 msgr2=0x7fe6340b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6340b2920 0x7fe6340b4de0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5641eb6d4910 tx=0x7fe640009b70 comp rx=0 tx=0).stop 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 msgr2=0x7fe64c1a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 0x7fe64c1a2170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe644009fd0 tx=0x7fe64400ef10 comp rx=0 tx=0).stop 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 shutdown_connections 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fe6340b2920 0x7fe6340b4de0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe64c10a9c0 0x7fe64c1a2170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe64c076a90 0x7fe64c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 --2- 172.21.15.26:0/1178279282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe64c0754b0 0x7fe64c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 >> 172.21.15.26:0/1178279282 conn(0x7fe64c0fe070 msgr2=0x7fe64c1068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 shutdown_connections 2024-05-20T21:13:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:34.087+0000 7fe651be4640 1 -- 172.21.15.26:0/1178279282 wait complete. 2024-05-20T21:13:34.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30762 7.07 MB/sec execute 99 sec latency 182.151 ms 2024-05-20T21:13:35.115 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30900 7.05 MB/sec execute 100 sec latency 185.515 ms 2024-05-20T21:13:35.247 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-20T21:13:35.247 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.2 2024-05-20T21:13:35.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:13:35.588 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:35.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:35 smithi055 ceph-mon[35213]: pgmap v913: 97 pgs: 97 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-20T21:13:35.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:35 smithi026 ceph-mon[31990]: pgmap v913: 97 pgs: 97 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-20T21:13:35.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:35 smithi144 ceph-mon[35660]: pgmap v913: 97 pgs: 97 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-20T21:13:36.116 INFO:tasks.workunit.client.0.smithi026.stdout: 10 31013 7.04 MB/sec execute 101 sec latency 83.567 ms 2024-05-20T21:13:37.116 INFO:tasks.workunit.client.0.smithi026.stdout: 10 31141 7.03 MB/sec execute 102 sec latency 186.383 ms 2024-05-20T21:13:37.363 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:37 smithi026 ceph-mon[31990]: pgmap v914: 97 pgs: 97 active+clean; 287 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 4.2 MiB/s wr, 236 op/s 2024-05-20T21:13:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.519+0000 7f6025ffb700 1 -- 172.21.15.26:0/576641180 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c003a30 con 0x7f60281041a0 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 -- 172.21.15.26:0/576641180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 msgr2=0x7f60281081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 --2- 172.21.15.26:0/576641180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f60281081c0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099c0 tx=0x7f601c009cd0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 -- 172.21.15.26:0/576641180 shutdown_connections 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 --2- 172.21.15.26:0/576641180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60281088f0 0x7f6028110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 --2- 172.21.15.26:0/576641180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60281037f0 0x7f6028103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 --2- 172.21.15.26:0/576641180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f60281081c0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 -- 172.21.15.26:0/576641180 >> 172.21.15.26:0/576641180 conn(0x7f602807a6a0 msgr2=0x7f602807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 -- 172.21.15.26:0/576641180 shutdown_connections 2024-05-20T21:13:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.520+0000 7f602d6dc700 1 -- 172.21.15.26:0/576641180 wait complete. 2024-05-20T21:13:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 Processor -- start 2024-05-20T21:13:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 -- start start 2024-05-20T21:13:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60281037f0 0x7f602819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f602819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60281088f0 0x7f60281998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6028113590 con 0x7f60281041a0 2024-05-20T21:13:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6028113410 con 0x7f60281088f0 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.521+0000 7f602d6dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6028113710 con 0x7f60281037f0 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f602819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f602819fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60458/0 (socket says 172.21.15.26:60458) 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 -- 172.21.15.26:0/3298797419 learned_addr learned my addr 172.21.15.26:0/3298797419 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 -- 172.21.15.26:0/3298797419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60281037f0 msgr2=0x7f602819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60281037f0 0x7f602819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 -- 172.21.15.26:0/3298797419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60281088f0 msgr2=0x7f60281998c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60281088f0 0x7f60281998c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 -- 172.21.15.26:0/3298797419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602819a020 con 0x7f60281041a0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.522+0000 7f60267fc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f602819fc40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f601c0098e0 tx=0x7f601c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.523+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c009e60 con 0x7f60281041a0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.523+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602819a2b0 con 0x7f60281041a0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.523+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f601c003d50 con 0x7f60281041a0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.523+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a65d0 con 0x7f60281041a0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.523+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c005020 con 0x7f60281041a0 2024-05-20T21:13:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.524+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602806a230 con 0x7f60281041a0 2024-05-20T21:13:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.526+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f601c004830 con 0x7f60281041a0 2024-05-20T21:13:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.527+0000 7f600ffff700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60100b2880 0x7f60100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.528+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f601c0e3070 con 0x7f60281041a0 2024-05-20T21:13:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.528+0000 7f6026ffd700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60100b2880 0x7f60100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.528+0000 7f6026ffd700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60100b2880 0x7f60100b4d40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6018005d90 tx=0x7f6018005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.533+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f601c013070 con 0x7f60281041a0 2024-05-20T21:13:37.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:37 smithi055 ceph-mon[35213]: pgmap v914: 97 pgs: 97 active+clean; 287 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 4.2 MiB/s wr, 236 op/s 2024-05-20T21:13:37.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:37 smithi144 ceph-mon[35660]: pgmap v914: 97 pgs: 97 active+clean; 287 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 4.2 MiB/s wr, 236 op/s 2024-05-20T21:13:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.981+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f602802e560 con 0x7f60281041a0 2024-05-20T21:13:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.982+0000 7f600ffff700 1 -- 172.21.15.26:0/3298797419 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11869 (secure 0 0 0) 0x7f601c0aadc0 con 0x7f60281041a0 2024-05-20T21:13:37.981 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:13:37.981 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:12:36.164744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T21:13:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.985+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60100b2880 msgr2=0x7f60100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.985+0000 7f602d6dc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60100b2880 0x7f60100b4d40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6018005d90 tx=0x7f6018005ce0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.985+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 msgr2=0x7f602819fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.985+0000 7f602d6dc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f602819fc40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f601c0098e0 tx=0x7f601c004690 comp rx=0 tx=0).stop 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 shutdown_connections 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60281088f0 0x7f60281998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60281037f0 0x7f602819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f60100b2880 0x7f60100b4d40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 --2- 172.21.15.26:0/3298797419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60281041a0 0x7f602819fc40 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 >> 172.21.15.26:0/3298797419 conn(0x7f602807a6a0 msgr2=0x7f6028101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 shutdown_connections 2024-05-20T21:13:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:37.986+0000 7f602d6dc700 1 -- 172.21.15.26:0/3298797419 wait complete. 2024-05-20T21:13:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 37 2024-05-20T21:13:38.116 INFO:tasks.workunit.client.0.smithi026.stdout: 10 31286 7.06 MB/sec execute 103 sec latency 112.436 ms 2024-05-20T21:13:38.309 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:38 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3298797419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:13:38.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:38 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3298797419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:13:38.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:38 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3298797419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:13:38.820 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.2 --await-for 45.0' 2024-05-20T21:13:38.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.2 --await-for 45.0 2024-05-20T21:13:39.116 INFO:tasks.workunit.client.0.smithi026.stdout: 10 31436 7.03 MB/sec execute 104 sec latency 109.214 ms 2024-05-20T21:13:39.187 INFO:tasks.workunit.client.0.smithi026.stdout:[25524] write failed on handle 11100 (Bad file descriptor) 2024-05-20T21:13:39.187 INFO:tasks.workunit.client.0.smithi026.stdout:Child failed with status 1 2024-05-20T21:13:39.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:13:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:39.202 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-20T21:13:39.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-20T21:13:39.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:39 smithi026 ceph-mon[31990]: pgmap v915: 97 pgs: 97 active+clean; 292 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 4.5 MiB/s wr, 254 op/s 2024-05-20T21:13:39.511 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ebd1d5361d457738957dc0455bba90102296634 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-20T21:13:39.769 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ab11c32662754336bc56741b3b161c7b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ebd1d5361d457738957dc0455bba90102296634 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-20T21:13:39.771 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-20T21:13:39.781 INFO:tasks.quiescer:joining Quiescers 2024-05-20T21:13:39.781 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-20T21:13:39.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-20T21:13:39.786 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:39 smithi055 ceph-mon[35213]: pgmap v915: 97 pgs: 97 active+clean; 292 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 4.5 MiB/s wr, 254 op/s 2024-05-20T21:13:39.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:39 smithi144 ceph-mon[35660]: pgmap v915: 97 pgs: 97 active+clean; 292 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 4.5 MiB/s wr, 254 op/s 2024-05-20T21:13:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:41.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:41 smithi026 ceph-mon[31990]: pgmap v916: 97 pgs: 97 active+clean; 291 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-20T21:13:41.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:41 smithi055 ceph-mon[35213]: pgmap v916: 97 pgs: 97 active+clean; 291 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-20T21:13:41.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:41 smithi144 ceph-mon[35660]: pgmap v916: 97 pgs: 97 active+clean; 291 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-20T21:13:42.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.878+0000 7f34cbb35700 1 -- 172.21.15.26:0/3019801730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c4104030 msgr2=0x7f34c4110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.878+0000 7f34cbb35700 1 --2- 172.21.15.26:0/3019801730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c4104030 0x7f34c4110450 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009990 tx=0x7f34bc009ca0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.878+0000 7f34cbb35700 1 -- 172.21.15.26:0/3019801730 shutdown_connections 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.878+0000 7f34cbb35700 1 --2- 172.21.15.26:0/3019801730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c4104030 0x7f34c4110450 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 --2- 172.21.15.26:0/3019801730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34c41084e0 0x7f34c41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 --2- 172.21.15.26:0/3019801730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34c4103690 0x7f34c4103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 -- 172.21.15.26:0/3019801730 >> 172.21.15.26:0/3019801730 conn(0x7f34c407a730 msgr2=0x7f34c407ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 -- 172.21.15.26:0/3019801730 shutdown_connections 2024-05-20T21:13:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 -- 172.21.15.26:0/3019801730 wait complete. 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 Processor -- start 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 -- start start 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.879+0000 7f34cbb35700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34c4103690 0x7f34c41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34cbb35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34c4104030 0x7f34c41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34cbb35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 0x7f34c419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34cbb35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34c4113670 con 0x7f34c41084e0 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34cbb35700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41134f0 con 0x7f34c4103690 2024-05-20T21:13:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34cbb35700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41137f0 con 0x7f34c4104030 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 0x7f34c419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 0x7f34c419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38552/0 (socket says 172.21.15.26:38552) 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34c4103690 0x7f34c41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 -- 172.21.15.26:0/1777448396 learned_addr learned my addr 172.21.15.26:0/1777448396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34c90d0700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34c4104030 0x7f34c41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 -- 172.21.15.26:0/1777448396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34c4104030 msgr2=0x7f34c41a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34c4104030 0x7f34c41a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 -- 172.21.15.26:0/1777448396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34c4103690 msgr2=0x7f34c41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34c4103690 0x7f34c41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 -- 172.21.15.26:0/1777448396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c419c230 con 0x7f34c41084e0 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 0x7f34c419bad0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0047b0 tx=0x7f34bc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.880+0000 7f34baffd700 1 -- 172.21.15.26:0/1777448396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc026070 con 0x7f34c41084e0 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.881+0000 7f34baffd700 1 -- 172.21.15.26:0/1777448396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34bc004c70 con 0x7f34c41084e0 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.881+0000 7f34baffd700 1 -- 172.21.15.26:0/1777448396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00f670 con 0x7f34c41084e0 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.881+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c419c4c0 con 0x7f34c41084e0 2024-05-20T21:13:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.881+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c41a87e0 con 0x7f34c41084e0 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.883+0000 7f34baffd700 1 -- 172.21.15.26:0/1777448396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f34bc029070 con 0x7f34c41084e0 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.884+0000 7f34baffd700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34b00b2930 0x7f34b00b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.884+0000 7f34c98d1700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34b00b2930 0x7f34b00b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.884+0000 7f34baffd700 1 -- 172.21.15.26:0/1777448396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f34bc0e2b50 con 0x7f34c41084e0 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.884+0000 7f34d63cbb80 1 Processor -- start 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 -- start start 2024-05-20T21:13:42.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34b00b2930 0x7f34b00b4df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f34c41046c0 tx=0x7f34c000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9dc80 0x56420ef9e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efaa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56420eefbf00 con 0x56420efa27b0 2024-05-20T21:13:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56420eeebaa0 con 0x56420ef9e640 2024-05-20T21:13:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56420eeefae0 con 0x56420ef9dc80 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38564/0 (socket says 172.21.15.26:38564) 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 -- 172.21.15.26:0/2580551929 learned_addr learned my addr 172.21.15.26:0/2580551929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efaa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c90d0700 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9dc80 0x56420ef9e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 -- 172.21.15.26:0/2580551929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9dc80 msgr2=0x56420ef9e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9dc80 0x56420ef9e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 -- 172.21.15.26:0/2580551929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 msgr2=0x56420efaa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efaa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 -- 172.21.15.26:0/2580551929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420ee95140 con 0x56420efa27b0 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efaa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.885+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa2b90 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f34c0016730 tx=0x7f34c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe647428f0844be server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34affff700 1 -- 172.21.15.26:0/2580551929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0015780 con 0x56420efa27b0 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34affff700 1 -- 172.21.15.26:0/2580551929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34c0025070 con 0x56420efa27b0 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34affff700 1 -- 172.21.15.26:0/2580551929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00209a0 con 0x56420efa27b0 2024-05-20T21:13:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2580551929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 msgr2=0x56420efa2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa2b90 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f34c0016730 tx=0x7f34c0011000 comp rx=0 tx=0).stop 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2580551929 shutdown_connections 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa2b90 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efaa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.886+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2580551929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9dc80 0x56420ef9e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2580551929 >> 172.21.15.26:0/2580551929 conn(0x56420eec93d0 msgr2=0x56420ef98970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2580551929 shutdown_connections 2024-05-20T21:13:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2580551929 wait complete. 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- start start 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 Processor -- start 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9dc80 0x56420efaa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efa7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56420edf2130 con 0x56420efa27b0 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56420ee53f30 con 0x56420ef9dc80 2024-05-20T21:13:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.887+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56420efaad10 con 0x56420ef9e640 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34d63cbb80 10 client.0 _ll_register_callbacks cb 0x56420ee86ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38566/0 (socket says 172.21.15.26:38566) 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 -- 172.21.15.26:0/2666850130 learned_addr learned my addr 172.21.15.26:0/2666850130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9dc80 0x56420efaa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34c90d0700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efa7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 -- 172.21.15.26:0/2666850130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 msgr2=0x56420efa7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efa7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 -- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9dc80 msgr2=0x56420efaa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9dc80 0x56420efaa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 -- 172.21.15.26:0/2666850130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56420ee3ba60 con 0x56420efa27b0 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34c90d0700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efa7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa7b90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f34bc005af0 tx=0x7f34bc0b2dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34193f29fad44c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ad7fa700 1 -- 172.21.15.26:0/2666850130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0aa8a0 con 0x56420efa27b0 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ad7fa700 1 -- 172.21.15.26:0/2666850130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f34bc0a9610 con 0x56420efa27b0 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c0025070 con 0x56420efa27b0 2024-05-20T21:13:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34ad7fa700 1 -- 172.21.15.26:0/2666850130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f34bc0aa1a0 con 0x56420efa27b0 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34d63cbb80 10 client.20343 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.888+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56420eea58f0 con 0x56420efa27b0 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34ad7fa700 1 -- 172.21.15.26:0/2666850130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34c0025070 con 0x56420efa27b0 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34ad7fa700 1 -- 172.21.15.26:0/2666850130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f34bc0e0ab0 con 0x56420efa27b0 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34d63cbb80 10 client.20343 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34d63cbb80 10 client.20343 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34d63cbb80 20 client.20343 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420efb7730 0x56420efb9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34d63cbb80 4 client.20343 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56420eefde50 con 0x56420efb7730 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.889+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420efb7730 0x56420efb9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420efb7730 0x56420efb9b20 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9283c9d5ca98d44 server_cookie=f1f8f806d8e7115d in_seq=0 out_seq=0 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34ad7fa700 10 client.20343 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34ad7fa700 1 -- 172.21.15.26:0/2666850130 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56420eefde50 con 0x56420efb7730 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34ad7fa700 10 client.20343 handle_command_reply: tid=0 2024-05-20T21:13:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34d63cbb80 1 client.20343 shutdown 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 msgr2=0x56420efa7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.890+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa7b90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f34bc005af0 tx=0x7f34bc0b2dc0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 shutdown_connections 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420efb7730 0x56420efb9b20 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420efa27b0 0x56420efa7b90 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9dc80 0x56420efaa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34ad7fa700 0 client.20343 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2666850130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efa7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 >> 172.21.15.26:0/2666850130 conn(0x56420eec93d0 msgr2=0x56420ef9cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 shutdown_connections 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2666850130 wait complete. 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.891+0000 7f34d63cbb80 20 client.20343 trim_cache size 0 max 16384 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 Processor -- start 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 -- start start 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420ef92f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efbf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420efa27b0 0x56420efbf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41b2570 con 0x56420ef9dc80 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41afe00 con 0x56420ef9e640 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41adc50 con 0x56420efa27b0 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420ef92f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420ef92f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38574/0 (socket says 172.21.15.26:38574) 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 -- 172.21.15.26:0/2605553948 learned_addr learned my addr 172.21.15.26:0/2605553948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c90d0700 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efbf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420efa27b0 0x56420efbf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 -- 172.21.15.26:0/2605553948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420efa27b0 msgr2=0x56420efbf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420efa27b0 0x56420efbf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 -- 172.21.15.26:0/2605553948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 msgr2=0x56420efbf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efbf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34c98d1700 1 -- 172.21.15.26:0/2605553948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420efba180 con 0x56420ef9dc80 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.912+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420efa27b0 0x56420efbf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.913+0000 7f34c98d1700 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420ef92f00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f34c0016700 tx=0x7f34c0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d2e45f63d80a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.913+0000 7f34af7fe700 1 -- 172.21.15.26:0/2605553948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0039070 con 0x56420ef9dc80 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.913+0000 7f34af7fe700 1 -- 172.21.15.26:0/2605553948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34c0025050 con 0x56420ef9dc80 2024-05-20T21:13:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.913+0000 7f34af7fe700 1 -- 172.21.15.26:0/2605553948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00157e0 con 0x56420ef9dc80 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2605553948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 msgr2=0x56420ef92f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420ef92f00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f34c0016700 tx=0x7f34c0015000 comp rx=0 tx=0).stop 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2605553948 shutdown_connections 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420ef92f00 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420efa27b0 0x56420efbf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/2605553948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420ef9e640 0x56420efbf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2605553948 >> 172.21.15.26:0/2605553948 conn(0x56420eec93d0 msgr2=0x56420ef9d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2605553948 shutdown_connections 2024-05-20T21:13:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 -- 172.21.15.26:0/2605553948 wait complete. 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 -- start start 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.914+0000 7f34d63cbb80 1 Processor -- start 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420efb0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420efa27b0 0x56420efad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41ae2c0 con 0x56420ef9dc80 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41af730 con 0x56420efa27b0 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41ae990 con 0x56420ef9e640 2024-05-20T21:13:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 10 client.0 _ll_register_callbacks cb 0x56420ee86ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420efb0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420efb0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38576/0 (socket says 172.21.15.26:38576) 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 -- 172.21.15.26:0/160975710 learned_addr learned my addr 172.21.15.26:0/160975710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420efa27b0 0x56420efad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c90d0700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 -- 172.21.15.26:0/160975710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 msgr2=0x56420efacf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efacf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 -- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420efa27b0 msgr2=0x56420efad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420efa27b0 0x56420efad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 -- 172.21.15.26:0/160975710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56420f022290 con 0x56420ef9dc80 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c90d0700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efacf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34ca0d2700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420efa27b0 0x56420efad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34c98d1700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420efb0740 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f34c0004c20 tx=0x7f34c000aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87747bd3430b1a7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34ad7fa700 1 -- 172.21.15.26:0/160975710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0042060 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34ad7fa700 1 -- 172.21.15.26:0/160975710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f34c0020850 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.915+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c0025050 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34ad7fa700 1 -- 172.21.15.26:0/160975710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f34c0039050 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 10 client.20349 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56420f0233b0 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34ad7fa700 1 -- 172.21.15.26:0/160975710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34c0025050 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34ad7fa700 1 -- 172.21.15.26:0/160975710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f34c00342e0 con 0x56420ef9dc80 2024-05-20T21:13:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 10 client.20349 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 10 client.20349 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 20 client.20349 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420f0238b0 0x56420f025cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 4 client.20349 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.2, "await_for": 45.0}] 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.2, "await_for": 45.0}) -- 0x56420eef7630 con 0x56420f0238b0 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.916+0000 7f34c90d0700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420f0238b0 0x56420f025cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.917+0000 7f34c90d0700 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420f0238b0 0x56420f025cd0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f13660fc8fbf5c server_cookie=270da5d645cf1d37 in_seq=0 out_seq=0 2024-05-20T21:13:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:41.917+0000 7f34ad7fa700 10 client.20349 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34ad7fa700 1 -- 172.21.15.26:0/160975710 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56420eef7630 con 0x56420f0238b0 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34ad7fa700 10 client.20349 handle_command_reply: tid=0 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 client.20349 shutdown 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 37, 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:13:42.425 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 62, 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "48f5eda": { 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 62, 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:42.426 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.2, 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:42.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:42.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:42.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:42.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:42.428 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:13:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 msgr2=0x56420efb0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420efb0740 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f34c0004c20 tx=0x7f34c000aeb0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 shutdown_connections 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x56420f0238b0 0x56420f025cd0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56420ef9dc80 0x56420efb0740 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56420efa27b0 0x56420efad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34ad7fa700 0 client.20349 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 --2- 172.21.15.26:0/160975710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56420ef9e640 0x56420efacf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 >> 172.21.15.26:0/160975710 conn(0x56420eec93d0 msgr2=0x56420efa3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 shutdown_connections 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 1 -- 172.21.15.26:0/160975710 wait complete. 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.308+0000 7f34d63cbb80 20 client.20349 trim_cache size 0 max 16384 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34b00b2930 msgr2=0x7f34b00b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34b00b2930 0x7f34b00b4df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f34c41046c0 tx=0x7f34c000d040 comp rx=0 tx=0).stop 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 msgr2=0x7f34c419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 0x7f34c419bad0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0047b0 tx=0x7f34bc005e80 comp rx=0 tx=0).stop 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 shutdown_connections 2024-05-20T21:13:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f34b00b2930 0x7f34b00b4df0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c41084e0 0x7f34c419bad0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34c4103690 0x7f34c41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 --2- 172.21.15.26:0/1777448396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34c4104030 0x7f34c41a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 >> 172.21.15.26:0/1777448396 conn(0x7f34c407a730 msgr2=0x7f34c4101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 shutdown_connections 2024-05-20T21:13:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:42.309+0000 7f34cbb35700 1 -- 172.21.15.26:0/1777448396 wait complete. 2024-05-20T21:13:43.429 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '48f5eda', quiesce took 8.2 seconds. Will release after: 13.0 2024-05-20T21:13:43.659 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:43 smithi026 ceph-mon[31990]: pgmap v917: 97 pgs: 97 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 4.5 MiB/s wr, 247 op/s 2024-05-20T21:13:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3178460264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88102f80 msgr2=0x7efc88103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3178460264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88102f80 0x7efc88103360 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7efc78009990 tx=0x7efc78009ca0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3178460264 shutdown_connections 2024-05-20T21:13:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3178460264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88102f80 0x7efc88103360 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3178460264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc88110020 0x7efc88112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3178460264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc881038a0 0x7efc8810fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3178460264 >> 172.21.15.26:0/3178460264 conn(0x7efc880784b0 msgr2=0x7efc880788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3178460264 shutdown_connections 2024-05-20T21:13:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3178460264 wait complete. 2024-05-20T21:13:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.669+0000 7efc8d2d0700 1 Processor -- start 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 -- start start 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc88102f80 0x7efc8819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc881038a0 0x7efc8819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 0x7efc881a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc881157f0 con 0x7efc88110020 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efc88115670 con 0x7efc881038a0 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc8d2d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc88115970 con 0x7efc88102f80 2024-05-20T21:13:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 0x7efc881a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 0x7efc881a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38594/0 (socket says 172.21.15.26:38594) 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 -- 172.21.15.26:0/3331516507 learned_addr learned my addr 172.21.15.26:0/3331516507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc867fc700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc881038a0 0x7efc8819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc86ffd700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc88102f80 0x7efc8819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 -- 172.21.15.26:0/3331516507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc88102f80 msgr2=0x7efc8819bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc88102f80 0x7efc8819bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 -- 172.21.15.26:0/3331516507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc881038a0 msgr2=0x7efc8819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc881038a0 0x7efc8819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 -- 172.21.15.26:0/3331516507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc881a0db0 con 0x7efc88110020 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.670+0000 7efc877fe700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 0x7efc881a0670 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7efc8000ef90 tx=0x7efc8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.671+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800092e0 con 0x7efc88110020 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.671+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc881a1040 con 0x7efc88110020 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.671+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc80004bb0 con 0x7efc88110020 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.671+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80004020 con 0x7efc88110020 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.671+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc881a8920 con 0x7efc88110020 2024-05-20T21:13:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.672+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc88102a90 con 0x7efc88110020 2024-05-20T21:13:43.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.673+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efc8001b030 con 0x7efc88110020 2024-05-20T21:13:43.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.674+0000 7efc6ffff700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc700b2880 0x7efc700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.674+0000 7efc86ffd700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc700b2880 0x7efc700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.674+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7efc800e2770 con 0x7efc88110020 2024-05-20T21:13:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.674+0000 7efc86ffd700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc700b2880 0x7efc700b4d40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b2e0 tx=0x7efc78005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.677+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc80022020 con 0x7efc88110020 2024-05-20T21:13:43.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:43 smithi055 ceph-mon[35213]: pgmap v917: 97 pgs: 97 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 4.5 MiB/s wr, 247 op/s 2024-05-20T21:13:43.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:43 smithi144 ceph-mon[35660]: pgmap v917: 97 pgs: 97 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 4.5 MiB/s wr, 247 op/s 2024-05-20T21:13:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.940+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc8819d100 con 0x7efc88110020 2024-05-20T21:13:43.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.941+0000 7efc6ffff700 1 -- 172.21.15.26:0/3331516507 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11869 (secure 0 0 0) 0x7efc80010040 con 0x7efc88110020 2024-05-20T21:13:43.940 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:13:43.940 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1873553840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1873553840},{"type":"v1","addr":"172.21.15.26:6835","nonce":1873553840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1148458694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1148458694},{"type":"v1","addr":"172.21.15.26:6837","nonce":1148458694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/841729682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":841729682},{"type":"v1","addr":"172.21.15.26:6839","nonce":841729682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3119421551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3119421551},{"type":"v1","addr":"172.21.15.26:6841","nonce":3119421551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3730419260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3730419260},{"type":"v1","addr":"172.21.15.144:6833","nonce":3730419260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2396855261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2396855261},{"type":"v1","addr":"172.21.15.55:6833","nonce":2396855261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T20:52:47.191542+0000","modified":"2024-05-20T21:12:36.164744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24455,"mds_2":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1712709614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1712709614},{"type":"v1","addr":"172.21.15.144:6835","nonce":1712709614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.55:6835/2325650145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2325650145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2325650145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.144:6839/2122637820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":2122637820},{"type":"v1","addr":"172.21.15.144:6839","nonce":2122637820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":270,"addr":"172.21.15.55:6837/353197532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":353197532},{"type":"v1","addr":"172.21.15.55:6837","nonce":353197532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.55:6839/1704683185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":1704683185},{"type":"v1","addr":"172.21.15.55:6839","nonce":1704683185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26755":{"gid":26755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/3257965779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3257965779},{"type":"v1","addr":"172.21.15.144:6837","nonce":3257965779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24445,24476,24455]},"id":1}]} 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc700b2880 msgr2=0x7efc700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc700b2880 0x7efc700b4d40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b2e0 tx=0x7efc78005fb0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 msgr2=0x7efc881a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 0x7efc881a0670 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7efc8000ef90 tx=0x7efc8000c3a0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 shutdown_connections 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7efc700b2880 0x7efc700b4d40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc88110020 0x7efc881a0670 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc88102f80 0x7efc8819bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 --2- 172.21.15.26:0/3331516507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc881038a0 0x7efc8819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.943+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 >> 172.21.15.26:0/3331516507 conn(0x7efc880784b0 msgr2=0x7efc88111700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.944+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 shutdown_connections 2024-05-20T21:13:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:43.944+0000 7efc8d2d0700 1 -- 172.21.15.26:0/3331516507 wait complete. 2024-05-20T21:13:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 37 2024-05-20T21:13:44.565 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:44 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3331516507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:13:44.594 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --cancel --all' 2024-05-20T21:13:44.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --cancel --all 2024-05-20T21:13:44.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:44 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3331516507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:13:44.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:44 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3331516507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T21:13:44.847 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:45.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:45 smithi055 ceph-mon[35213]: pgmap v918: 97 pgs: 97 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-05-20T21:13:45.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:45 smithi026 ceph-mon[31990]: pgmap v918: 97 pgs: 97 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-05-20T21:13:45.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:45 smithi144 ceph-mon[35660]: pgmap v918: 97 pgs: 97 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-05-20T21:13:46.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 -- 172.21.15.26:0/3941345822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 msgr2=0x7f82cc10fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3941345822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc10fa70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009990 tx=0x7f82bc009ca0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 -- 172.21.15.26:0/3941345822 shutdown_connections 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3941345822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82cc10ffb0 0x7f82cc1123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3941345822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 0x7f82cc075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3941345822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc10fa70 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 -- 172.21.15.26:0/3941345822 >> 172.21.15.26:0/3941345822 conn(0x7f82cc07a700 msgr2=0x7f82cc07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:46.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 -- 172.21.15.26:0/3941345822 shutdown_connections 2024-05-20T21:13:46.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.701+0000 7f82d43cc700 1 -- 172.21.15.26:0/3941345822 wait complete. 2024-05-20T21:13:46.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 Processor -- start 2024-05-20T21:13:46.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 -- start start 2024-05-20T21:13:46.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 0x7f82cc19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82cc10ffb0 0x7f82cc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1157f0 con 0x7f82cc076090 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc115670 con 0x7f82cc10ffb0 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d43cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc115970 con 0x7f82cc075770 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d1967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d1967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38620/0 (socket says 172.21.15.26:38620) 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d1967700 1 -- 172.21.15.26:0/3633406592 learned_addr learned my addr 172.21.15.26:0/3633406592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d2168700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 0x7f82cc19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.702+0000 7f82d2969700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82cc10ffb0 0x7f82cc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d1967700 1 -- 172.21.15.26:0/3633406592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 msgr2=0x7f82cc19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d1967700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 0x7f82cc19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d1967700 1 -- 172.21.15.26:0/3633406592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82cc10ffb0 msgr2=0x7f82cc1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d1967700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82cc10ffb0 0x7f82cc1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d1967700 1 -- 172.21.15.26:0/3633406592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc1a0dd0 con 0x7f82cc076090 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d2168700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 0x7f82cc19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d1967700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc19c300 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f82bc000c00 tx=0x7f82bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:46.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82c37fe700 1 -- 172.21.15.26:0/3633406592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc027070 con 0x7f82cc076090 2024-05-20T21:13:46.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82c37fe700 1 -- 172.21.15.26:0/3633406592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82bc00fd00 con 0x7f82cc076090 2024-05-20T21:13:46.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc1a1060 con 0x7f82cc076090 2024-05-20T21:13:46.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82c37fe700 1 -- 172.21.15.26:0/3633406592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc022e20 con 0x7f82cc076090 2024-05-20T21:13:46.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.703+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc1a8940 con 0x7f82cc076090 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.706+0000 7f82c37fe700 1 -- 172.21.15.26:0/3633406592 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f82bc02a070 con 0x7f82cc076090 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.706+0000 7f82c37fe700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f82b80b2900 0x7f82b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82d2168700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f82b80b2900 0x7f82b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82c37fe700 1 -- 172.21.15.26:0/3633406592 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f82bc0e2fb0 con 0x7f82cc076090 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 Processor -- start 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82d2168700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f82b80b2900 0x7f82b80b4dc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005950 tx=0x7f82c8005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 -- start start 2024-05-20T21:13:46.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d23096a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23091fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d2309e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d22fefde0 con 0x564d23092520 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564d22fdf980 con 0x564d23091b60 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.707+0000 7f82dec62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d22fe39c0 con 0x564d23096690 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d2309e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d2309e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38628/0 (socket says 172.21.15.26:38628) 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 -- 172.21.15.26:0/2647356327 learned_addr learned my addr 172.21.15.26:0/2647356327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2168700 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d23096a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d1967700 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23091fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 -- 172.21.15.26:0/2647356327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 msgr2=0x564d23096a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d23096a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 -- 172.21.15.26:0/2647356327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 msgr2=0x564d23091fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23091fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 -- 172.21.15.26:0/2647356327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d22f89020 con 0x564d23092520 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d1967700 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23091fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82d2969700 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d2309e590 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f82c400a430 tx=0x7f82c4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df5fc2c09271158 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82c0d70700 1 -- 172.21.15.26:0/2647356327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4014620 con 0x564d23092520 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82c0d70700 1 -- 172.21.15.26:0/2647356327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82c4019070 con 0x564d23092520 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.708+0000 7f82c0d70700 1 -- 172.21.15.26:0/2647356327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4013940 con 0x564d23092520 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 -- 172.21.15.26:0/2647356327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 msgr2=0x564d2309e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d2309e590 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f82c400a430 tx=0x7f82c4009650 comp rx=0 tx=0).stop 2024-05-20T21:13:46.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 -- 172.21.15.26:0/2647356327 shutdown_connections 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d2309e590 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d23096a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2647356327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23091fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 -- 172.21.15.26:0/2647356327 >> 172.21.15.26:0/2647356327 conn(0x564d22fbd2b0 msgr2=0x564d2308c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 -- 172.21.15.26:0/2647356327 shutdown_connections 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.709+0000 7f82dec62b80 1 -- 172.21.15.26:0/2647356327 wait complete. 2024-05-20T21:13:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 -- start start 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 Processor -- start 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 0x564d2309e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23092520 0x564d2309b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d2309ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d22ee6010 con 0x564d23091b60 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564d22f47e10 con 0x564d23092520 2024-05-20T21:13:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d2309ebf0 con 0x564d23096690 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82dec62b80 10 client.0 _ll_register_callbacks cb 0x564d22f7a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 0x564d2309e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 0x564d2309e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38634/0 (socket says 172.21.15.26:38634) 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 -- 172.21.15.26:0/822598269 learned_addr learned my addr 172.21.15.26:0/822598269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d1967700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23092520 0x564d2309b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2969700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d2309ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 -- 172.21.15.26:0/822598269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 msgr2=0x564d2309ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d2309ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 -- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23092520 msgr2=0x564d2309b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23092520 0x564d2309b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.710+0000 7f82d2168700 1 -- 172.21.15.26:0/822598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d22f2f940 con 0x564d23091b60 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82d2168700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 0x564d2309e210 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f82c80109e0 tx=0x7f82c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82125d4eff45cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82b5ffb700 1 -- 172.21.15.26:0/822598269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004a30 con 0x564d23091b60 2024-05-20T21:13:46.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82b5ffb700 1 -- 172.21.15.26:0/822598269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f82c8004bd0 con 0x564d23091b60 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c4019070 con 0x564d23091b60 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82b5ffb700 1 -- 172.21.15.26:0/822598269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f82c8004dd0 con 0x564d23091b60 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82dec62b80 10 client.20367 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d22f997d0 con 0x564d23091b60 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82b5ffb700 1 -- 172.21.15.26:0/822598269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82c4019070 con 0x564d23091b60 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82b5ffb700 1 -- 172.21.15.26:0/822598269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f82c802e020 con 0x564d23091b60 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82dec62b80 10 client.20367 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:46.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.711+0000 7f82dec62b80 10 client.20367 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82dec62b80 20 client.20367 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82dec62b80 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d230ab6c0 0x564d230adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82dec62b80 4 client.20367 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d22ff1d30 con 0x564d230ab6c0 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82d1967700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d230ab6c0 0x564d230adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82d1967700 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d230ab6c0 0x564d230adb30 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08b08dccdbcbb3a server_cookie=d0314dd9691a336a in_seq=0 out_seq=0 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.712+0000 7f82b5ffb700 10 client.20367 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:46.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82b5ffb700 1 -- 172.21.15.26:0/822598269 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564d22ff1d30 con 0x564d230ab6c0 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82b5ffb700 10 client.20367 handle_command_reply: tid=0 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 client.20367 shutdown 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 msgr2=0x564d2309e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 0x564d2309e210 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f82c80109e0 tx=0x7f82c8004750 comp rx=0 tx=0).stop 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 shutdown_connections 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d230ab6c0 0x564d230adb30 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23091b60 0x564d2309e210 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d2309ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 --2- 172.21.15.26:0/822598269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23092520 0x564d2309b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82b5ffb700 0 client.20367 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 >> 172.21.15.26:0/822598269 conn(0x564d22fbd2b0 msgr2=0x564d23090b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 shutdown_connections 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 1 -- 172.21.15.26:0/822598269 wait complete. 2024-05-20T21:13:46.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.713+0000 7f82dec62b80 20 client.20367 trim_cache size 0 max 16384 2024-05-20T21:13:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 Processor -- start 2024-05-20T21:13:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 -- start start 2024-05-20T21:13:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23086de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d23097a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d230b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b2760 con 0x564d23092520 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1afff0 con 0x564d23091b60 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82dec62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1ade40 con 0x564d23096690 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82d1967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d23097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82d1967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d23097a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38650/0 (socket says 172.21.15.26:38650) 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82d1967700 1 -- 172.21.15.26:0/2718803382 learned_addr learned my addr 172.21.15.26:0/2718803382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.734+0000 7f82d2168700 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23086de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82d1967700 1 -- 172.21.15.26:0/2718803382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 msgr2=0x564d230b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82d1967700 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d230b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82d1967700 1 -- 172.21.15.26:0/2718803382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 msgr2=0x564d23086de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82d1967700 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23086de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82d1967700 1 -- 172.21.15.26:0/2718803382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2309e9f0 con 0x564d23092520 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82d1967700 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d23097a40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0aa300 tx=0x7f82bc092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba0b0b4b6e32518 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82b7fff700 1 -- 172.21.15.26:0/2718803382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0aa600 con 0x564d23092520 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82b7fff700 1 -- 172.21.15.26:0/2718803382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82bc0ae9b0 con 0x564d23092520 2024-05-20T21:13:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.735+0000 7f82b7fff700 1 -- 172.21.15.26:0/2718803382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0aa860 con 0x564d23092520 2024-05-20T21:13:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- 172.21.15.26:0/2718803382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 msgr2=0x564d23097a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d23097a40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0aa300 tx=0x7f82bc092040 comp rx=0 tx=0).stop 2024-05-20T21:13:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- 172.21.15.26:0/2718803382 shutdown_connections 2024-05-20T21:13:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23096690 0x564d230b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23091b60 0x564d23086de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- 172.21.15.26:0/2718803382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d23097a40 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- 172.21.15.26:0/2718803382 >> 172.21.15.26:0/2718803382 conn(0x564d22fbd2b0 msgr2=0x564d23097330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- 172.21.15.26:0/2718803382 shutdown_connections 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- 172.21.15.26:0/2718803382 wait complete. 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- start start 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 Processor -- start 2024-05-20T21:13:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23091b60 0x564d230a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d230a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23096690 0x564d230a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1ae4b0 con 0x564d23092520 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1af920 con 0x564d23096690 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1aeb80 con 0x564d23091b60 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.736+0000 7f82dec62b80 10 client.0 _ll_register_callbacks cb 0x564d22f7a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d230a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d230a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38658/0 (socket says 172.21.15.26:38658) 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 -- 172.21.15.26:0/3381744902 learned_addr learned my addr 172.21.15.26:0/3381744902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d2168700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23091b60 0x564d230a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d2969700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23096690 0x564d230a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 -- 172.21.15.26:0/3381744902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23091b60 msgr2=0x564d230a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23091b60 0x564d230a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 -- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23096690 msgr2=0x564d230a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23096690 0x564d230a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 -- 172.21.15.26:0/3381744902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d230a0b10 con 0x564d23092520 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d2969700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23096690 0x564d230a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82d1967700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d230a43a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098e0 tx=0x7f82bc009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587b5ee33539399f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82b5ffb700 1 -- 172.21.15.26:0/3381744902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0c4e60 con 0x564d23092520 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82b5ffb700 1 -- 172.21.15.26:0/3381744902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f82bc022570 con 0x564d23092520 2024-05-20T21:13:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82bc0ae9b0 con 0x564d23092520 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.737+0000 7f82b5ffb700 1 -- 172.21.15.26:0/3381744902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f82bc0aa860 con 0x564d23092520 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 10 client.20373 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d23116d60 con 0x564d23092520 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82b5ffb700 1 -- 172.21.15.26:0/3381744902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82bc0ae9b0 con 0x564d23092520 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82b5ffb700 1 -- 172.21.15.26:0/3381744902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f82bc00f870 con 0x564d23092520 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 10 client.20373 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 10 client.20373 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-20T21:13:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 20 client.20373 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d231174a0 0x564d231198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 4 client.20373 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x564d22feb510 con 0x564d231174a0 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.738+0000 7f82d2168700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d231174a0 0x564d231198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82b5ffb700 10 client.20373 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82d2168700 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d231174a0 0x564d231198c0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235ed237befbd21f server_cookie=1179b9a2572e240d in_seq=0 out_seq=0 2024-05-20T21:13:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82b5ffb700 1 -- 172.21.15.26:0/3381744902 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x564d22feb510 con 0x564d231174a0 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82b5ffb700 10 client.20373 handle_command_reply: tid=0 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 client.20373 shutdown 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 msgr2=0x564d230a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d230a43a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098e0 tx=0x7f82bc009540 comp rx=0 tx=0).stop 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 shutdown_connections 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x564d231174a0 0x564d231198c0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d23096690 0x564d230a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d23091b60 0x564d230a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82b5ffb700 0 client.20373 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 --2- 172.21.15.26:0/3381744902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d23092520 0x564d230a43a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 >> 172.21.15.26:0/3381744902 conn(0x564d22fbd2b0 msgr2=0x564d23085dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 shutdown_connections 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.739+0000 7f82dec62b80 1 -- 172.21.15.26:0/3381744902 wait complete. 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82dec62b80 20 client.20373 trim_cache size 0 max 16384 2024-05-20T21:13:46.748 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:13:46.749 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 37, 2024-05-20T21:13:46.749 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24476, 2024-05-20T21:13:46.749 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 63, 2024-05-20T21:13:46.749 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": {} 2024-05-20T21:13:46.749 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:13:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f82b80b2900 msgr2=0x7f82b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f82b80b2900 0x7f82b80b4dc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005950 tx=0x7f82c8005f50 comp rx=0 tx=0).stop 2024-05-20T21:13:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 msgr2=0x7f82cc19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc19c300 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f82bc000c00 tx=0x7f82bc0047b0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 shutdown_connections 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82cc10ffb0 0x7f82cc1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc075770 0x7f82cc19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f82b80b2900 0x7f82b80b4dc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 --2- 172.21.15.26:0/3633406592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc076090 0x7f82cc19c300 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 >> 172.21.15.26:0/3633406592 conn(0x7f82cc07a700 msgr2=0x7f82cc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 shutdown_connections 2024-05-20T21:13:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:46.740+0000 7f82d43cc700 1 -- 172.21.15.26:0/3633406592 wait complete. 2024-05-20T21:13:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 -- 172.21.15.26:0/3883345184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0102ac0 msgr2=0x7f6df0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 --2- 172.21.15.26:0/3883345184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0102ac0 0x7f6df0102ee0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de002a410 comp rx=0 tx=0).stop 2024-05-20T21:13:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 -- 172.21.15.26:0/3883345184 shutdown_connections 2024-05-20T21:13:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 --2- 172.21.15.26:0/3883345184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0104670 0x7f6df010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 --2- 172.21.15.26:0/3883345184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0103cb0 0x7f6df0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 --2- 172.21.15.26:0/3883345184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0102ac0 0x7f6df0102ee0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 -- 172.21.15.26:0/3883345184 >> 172.21.15.26:0/3883345184 conn(0x7f6df00fe030 msgr2=0x7f6df0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 -- 172.21.15.26:0/3883345184 shutdown_connections 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 -- 172.21.15.26:0/3883345184 wait complete. 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 Processor -- start 2024-05-20T21:13:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.184+0000 7f6df6c83640 1 -- start start 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0102ac0 0x7f6df019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 0x7f6df019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0104670 0x7f6df01a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010e4f0 con 0x7f6df0103cb0 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010e370 con 0x7f6df0104670 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010e670 con 0x7f6df0102ac0 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 0x7f6df019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 0x7f6df019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38688/0 (socket says 172.21.15.26:38688) 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 -- 172.21.15.26:0/2867736930 learned_addr learned my addr 172.21.15.26:0/2867736930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df5220640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0104670 0x7f6df01a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0102ac0 0x7f6df019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 -- 172.21.15.26:0/2867736930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0102ac0 msgr2=0x7f6df019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0102ac0 0x7f6df019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 -- 172.21.15.26:0/2867736930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0104670 msgr2=0x7f6df01a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0104670 0x7f6df01a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 -- 172.21.15.26:0/2867736930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df01a29d0 con 0x7f6df0103cb0 2024-05-20T21:13:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0102ac0 0x7f6df019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df5220640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0104670 0x7f6df01a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6deffff640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 0x7f6df019b2f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c890 tx=0x7f6dd800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6dedffb640 1 -- 172.21.15.26:0/2867736930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004220 con 0x7f6df0103cb0 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df01a2cc0 con 0x7f6df0103cb0 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.185+0000 7f6dedffb640 1 -- 172.21.15.26:0/2867736930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd8007e40 con 0x7f6df0103cb0 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.186+0000 7f6dedffb640 1 -- 172.21.15.26:0/2867736930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004a00 con 0x7f6df0103cb0 2024-05-20T21:13:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.186+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df01a3200 con 0x7f6df0103cb0 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.188+0000 7f6dedffb640 1 -- 172.21.15.26:0/2867736930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6dd80079b0 con 0x7f6df0103cb0 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.188+0000 7f6df99a2740 1 Processor -- start 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.188+0000 7f6df99a2740 1 -- start start 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74910ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c7491116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df99a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c74907ac30 con 0x55c74910f220 2024-05-20T21:13:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df99a2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c74906a7d0 con 0x55c7491096c0 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df99a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c74906e810 con 0x55c74910a0b0 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c7491116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c7491116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38690/0 (socket says 172.21.15.26:38690) 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 -- 172.21.15.26:0/1017146240 learned_addr learned my addr 172.21.15.26:0/1017146240 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df5220640 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74910ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 -- 172.21.15.26:0/1017146240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 msgr2=0x55c74910ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74910ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 -- 172.21.15.26:0/1017146240 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 msgr2=0x55c749109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6dedffb640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ddc0b28a0 0x7f6ddc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6deffff640 1 -- 172.21.15.26:0/1017146240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c748d7f210 con 0x55c74910f220 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6df5220640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ddc0b28a0 0x7f6ddc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.189+0000 7f6dedffb640 1 -- 172.21.15.26:0/2867736930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6dd80e20c0 con 0x7f6df0103cb0 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6deffff640 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c7491116b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80a82f0 tx=0x7f6dd80a5d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a49171efff7f5c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6de6ffd640 1 -- 172.21.15.26:0/1017146240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801d030 con 0x55c74910f220 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6de6ffd640 1 -- 172.21.15.26:0/1017146240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd80056a0 con 0x55c74910f220 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6de6ffd640 1 -- 172.21.15.26:0/1017146240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80a8f70 con 0x55c74910f220 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 -- 172.21.15.26:0/1017146240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 msgr2=0x55c7491116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df5220640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ddc0b28a0 0x7f6ddc0b4d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55c74910a910 tx=0x7f6de800a730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c7491116b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80a82f0 tx=0x7f6dd80a5d30 comp rx=0 tx=0).stop 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 -- 172.21.15.26:0/1017146240 shutdown_connections 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c7491116b0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74910ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1017146240 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 -- 172.21.15.26:0/1017146240 >> 172.21.15.26:0/1017146240 conn(0x55c749104d30 msgr2=0x55c749105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.190+0000 7f6df99a2740 1 -- 172.21.15.26:0/1017146240 shutdown_connections 2024-05-20T21:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 -- 172.21.15.26:0/1017146240 wait complete. 2024-05-20T21:13:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 -- start start 2024-05-20T21:13:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 Processor -- start 2024-05-20T21:13:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7491096c0 0x55c7491137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 0x55c749113d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c74911ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c748d50790 con 0x55c74910a0b0 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c749026890 con 0x55c74910f220 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c7491121d0 con 0x55c7491096c0 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df99a2740 10 client.0 _ll_register_callbacks cb 0x55c748d7ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6deffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 0x55c749113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6deffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 0x55c749113d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38696/0 (socket says 172.21.15.26:38696) 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6deffff640 1 -- 172.21.15.26:0/178258438 learned_addr learned my addr 172.21.15.26:0/178258438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.191+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7491096c0 0x55c7491137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6df5220640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c74911ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6deffff640 1 -- 172.21.15.26:0/178258438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7491096c0 msgr2=0x55c7491137e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6deffff640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7491096c0 0x55c7491137e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6deffff640 1 -- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 msgr2=0x55c74911ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6deffff640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c74911ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6deffff640 1 -- 172.21.15.26:0/178258438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c748e4b620 con 0x55c74910a0b0 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6df5220640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c74911ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6deffff640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 0x55c749113d20 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80dfb00 tx=0x7f6dd80a4ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3964b4249f4bbfde server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6dcffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6dcffff640 1 -- 172.21.15.26:0/178258438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801d040 con 0x55c74910a0b0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6dcffff640 1 -- 172.21.15.26:0/178258438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6dd80a4f90 con 0x55c74910a0b0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd80056a0 con 0x55c74910a0b0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6dcffff640 1 -- 172.21.15.26:0/178258438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6dd8007ca0 con 0x55c74910a0b0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6df99a2740 10 client.20385 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c74911dd40 con 0x55c74910a0b0 2024-05-20T21:13:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.192+0000 7f6dcffff640 1 -- 172.21.15.26:0/178258438 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd80056a0 con 0x55c74910a0b0 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6dcffff640 1 -- 172.21.15.26:0/178258438 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f6dd80a9880 con 0x55c74910a0b0 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df99a2740 10 client.20385 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df99a2740 10 client.20385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df99a2740 20 client.20385 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df99a2740 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74911f090 0x55c749121480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df99a2740 4 client.20385 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T21:13:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c74907cb80 con 0x55c74911f090 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74911f090 0x55c749121480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74911f090 0x55c749121480 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fb95174404d7ea server_cookie=f9a13dfe2b58f943 in_seq=0 out_seq=0 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.193+0000 7f6dcffff640 10 client.20385 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6dcffff640 1 -- 172.21.15.26:0/178258438 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c74907cb80 con 0x55c74911f090 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6dcffff640 10 client.20385 handle_command_reply: tid=0 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 client.20385 shutdown 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 msgr2=0x55c749113d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 0x55c749113d20 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80dfb00 tx=0x7f6dd80a4ee0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 shutdown_connections 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74911f090 0x55c749121480 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c74911ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910a0b0 0x55c749113d20 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6dcffff640 0 client.20385 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 --2- 172.21.15.26:0/178258438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7491096c0 0x55c7491137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 >> 172.21.15.26:0/178258438 conn(0x55c749104d30 msgr2=0x55c749103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 shutdown_connections 2024-05-20T21:13:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 1 -- 172.21.15.26:0/178258438 wait complete. 2024-05-20T21:13:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.194+0000 7f6df99a2740 20 client.20385 trim_cache size 0 max 16384 2024-05-20T21:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 Processor -- start 2024-05-20T21:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 -- start start 2024-05-20T21:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74911ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c749125be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01aeb20 con 0x55c74910f220 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01ac3f0 con 0x55c7491096c0 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df99a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01aa240 con 0x55c74910a0b0 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df5220640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c749125be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df5220640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c749125be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38704/0 (socket says 172.21.15.26:38704) 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.210+0000 7f6df5220640 1 -- 172.21.15.26:0/1640169222 learned_addr learned my addr 172.21.15.26:0/1640169222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6deffff640 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74911ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df5220640 1 -- 172.21.15.26:0/1640169222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 msgr2=0x55c74911ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df5220640 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74911ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df5220640 1 -- 172.21.15.26:0/1640169222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 msgr2=0x55c749108120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df5220640 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749108120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df5220640 1 -- 172.21.15.26:0/1640169222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c749104b60 con 0x55c74910f220 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6df5220640 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c749125be0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6de801f730 tx=0x7f6de8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f0813fffab7068 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6de67fc640 1 -- 172.21.15.26:0/1640169222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de802c070 con 0x55c74910f220 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6de67fc640 1 -- 172.21.15.26:0/1640169222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de801f320 con 0x55c74910f220 2024-05-20T21:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.211+0000 7f6de67fc640 1 -- 172.21.15.26:0/1640169222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de801bbf0 con 0x55c74910f220 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 -- 172.21.15.26:0/1640169222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 msgr2=0x55c749125be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c749125be0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6de801f730 tx=0x7f6de8014000 comp rx=0 tx=0).stop 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 -- 172.21.15.26:0/1640169222 shutdown_connections 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c74910f220 0x55c749125be0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c74911ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 --2- 172.21.15.26:0/1640169222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c7491096c0 0x55c749108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 -- 172.21.15.26:0/1640169222 >> 172.21.15.26:0/1640169222 conn(0x55c749104d30 msgr2=0x55c749107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 -- 172.21.15.26:0/1640169222 shutdown_connections 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 -- 172.21.15.26:0/1640169222 wait complete. 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 -- start start 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.212+0000 7f6df99a2740 1 Processor -- start 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 0x55c749118700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c749118c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c749119c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01aa8b0 con 0x55c7491096c0 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01abd20 con 0x55c74910f220 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01aaf80 con 0x55c74910a0b0 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 10 client.0 _ll_register_callbacks cb 0x7f6de800a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 0x55c749118700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 0x55c749118700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38718/0 (socket says 172.21.15.26:38718) 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 -- 172.21.15.26:0/4149961346 learned_addr learned my addr 172.21.15.26:0/4149961346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df5220640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c749119c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6deffff640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c749118c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 -- 172.21.15.26:0/4149961346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 msgr2=0x55c749118c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c749118c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 -- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 msgr2=0x55c749119c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c749119c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 -- 172.21.15.26:0/4149961346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c749112310 con 0x55c7491096c0 2024-05-20T21:13:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6deffff640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c749118c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df4a1f640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 0x55c749118700 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9178296189eb3b71 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6dcffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6dcffff640 1 -- 172.21.15.26:0/4149961346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0035040 con 0x55c7491096c0 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6dcffff640 1 -- 172.21.15.26:0/4149961346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f6de00039b0 con 0x55c7491096c0 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.213+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de801f320 con 0x55c7491096c0 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6dcffff640 1 -- 172.21.15.26:0/4149961346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6de0053050 con 0x55c7491096c0 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 10 client.20391 fetch_fsmap learned FSMap version 37 2024-05-20T21:13:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c749189bf0 con 0x55c7491096c0 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6dcffff640 1 -- 172.21.15.26:0/4149961346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de801f320 con 0x55c7491096c0 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6dcffff640 1 -- 172.21.15.26:0/4149961346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f6de0031ca0 con 0x55c7491096c0 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 10 client.20391 fetch_fsmap finished waiting for FSMap version 37 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 10 client.20391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 20 client.20391 populate_metadata read hostname 'smithi026' 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74918a620 0x55c74918ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 4 client.20391 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 --> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c749076360 con 0x55c74918a620 2024-05-20T21:13:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.214+0000 7f6deffff640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74918a620 0x55c74918ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.215+0000 7f6deffff640 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74918a620 0x55c74918ca40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ee1ecae897d042 server_cookie=d36586ea0f6ee0 in_seq=0 out_seq=0 2024-05-20T21:13:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.215+0000 7f6dcffff640 10 client.20391 ms_handle_connect on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6dcffff640 1 -- 172.21.15.26:0/4149961346 <== mds.0 v2:172.21.15.55:6838/1704683185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c749076360 con 0x55c74918a620 2024-05-20T21:13:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6dcffff640 10 client.20391 handle_command_reply: tid=0 2024-05-20T21:13:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 client.20391 shutdown 2024-05-20T21:13:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 msgr2=0x55c749118700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 0x55c749118700 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de00035f0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 shutdown_connections 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:6838/1704683185,v1:172.21.15.55:6839/1704683185] conn(0x55c74918a620 0x55c74918ca40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c74910f220 0x55c749119c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c74910a0b0 0x55c749118c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6dcffff640 0 client.20391 ms_handle_reset on v2:172.21.15.55:6838/1704683185 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 --2- 172.21.15.26:0/4149961346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7491096c0 0x55c749118700 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.281+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 >> 172.21.15.26:0/4149961346 conn(0x55c749104d30 msgr2=0x55c7491082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 shutdown_connections 2024-05-20T21:13:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df99a2740 1 -- 172.21.15.26:0/4149961346 wait complete. 2024-05-20T21:13:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df99a2740 20 client.20391 trim_cache size 0 max 16384 2024-05-20T21:13:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-05-20T21:13:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-05-20T21:13:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-05-20T21:13:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ddc0b28a0 msgr2=0x7f6ddc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ddc0b28a0 0x7f6ddc0b4d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55c74910a910 tx=0x7f6de800a730 comp rx=0 tx=0).stop 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 msgr2=0x7f6df019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 0x7f6df019b2f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c890 tx=0x7f6dd800cd50 comp rx=0 tx=0).stop 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 shutdown_connections 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f6ddc0b28a0 0x7f6ddc0b4d60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df0104670 0x7f6df01a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0103cb0 0x7f6df019b2f0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 --2- 172.21.15.26:0/2867736930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df0102ac0 0x7f6df019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 >> 172.21.15.26:0/2867736930 conn(0x7f6df00fe030 msgr2=0x7f6df0074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.282+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 shutdown_connections 2024-05-20T21:13:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-05-20T21:13:47.283+0000 7f6df6c83640 1 -- 172.21.15.26:0/2867736930 wait complete. 2024-05-20T21:13:47.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:47 smithi026 ceph-mon[31990]: pgmap v919: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.9 MiB/s wr, 204 op/s 2024-05-20T21:13:47.531 INFO:tasks.quiescer:done joining Quiescers 2024-05-20T21:13:47.531 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-20T21:13:47.550 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T21:13:47.558 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-20T21:13:47.567 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-20T21:13:47.568 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-20T21:13:47.572 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-20T21:13:47.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T21:13:47.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:47 smithi055 ceph-mon[35213]: pgmap v919: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.9 MiB/s wr, 204 op/s 2024-05-20T21:13:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 -- 172.21.15.26:0/799287916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff60076a90 msgr2=0x7eff60074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 --2- 172.21.15.26:0/799287916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff60076a90 0x7eff60074ee0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff5002a410 comp rx=0 tx=0).stop 2024-05-20T21:13:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 -- 172.21.15.26:0/799287916 shutdown_connections 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 --2- 172.21.15.26:0/799287916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6010a980 0x7eff6010ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 --2- 172.21.15.26:0/799287916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff600754b0 0x7eff60075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 --2- 172.21.15.26:0/799287916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff60076a90 0x7eff60074ee0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 -- 172.21.15.26:0/799287916 >> 172.21.15.26:0/799287916 conn(0x7eff600fe050 msgr2=0x7eff60100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 -- 172.21.15.26:0/799287916 shutdown_connections 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 -- 172.21.15.26:0/799287916 wait complete. 2024-05-20T21:13:47.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 Processor -- start 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.707+0000 7eff65254640 1 -- start start 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff65254640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 0x7eff6019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff65254640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff60076a90 0x7eff6019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff65254640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6010a980 0x7eff601a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff65254640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7eff601107b0 con 0x7eff600754b0 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff65254640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7eff60110630 con 0x7eff60076a90 2024-05-20T21:13:47.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff65254640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff60110930 con 0x7eff6010a980 2024-05-20T21:13:47.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 0x7eff6019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 0x7eff6019afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38732/0 (socket says 172.21.15.26:38732) 2024-05-20T21:13:47.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 -- 172.21.15.26:0/62192616 learned_addr learned my addr 172.21.15.26:0/62192616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:47.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5f59e640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6010a980 0x7eff601a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff57fff640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff60076a90 0x7eff6019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 -- 172.21.15.26:0/62192616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6010a980 msgr2=0x7eff601a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6010a980 0x7eff601a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 -- 172.21.15.26:0/62192616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff60076a90 msgr2=0x7eff6019b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff60076a90 0x7eff6019b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 -- 172.21.15.26:0/62192616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601a2bc0 con 0x7eff600754b0 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff57fff640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff60076a90 0x7eff6019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:13:47.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5ed9d640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 0x7eff6019afa0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7eff50031460 tx=0x7eff50003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.708+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5003f070 con 0x7eff600754b0 2024-05-20T21:13:47.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.709+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff50031da0 con 0x7eff600754b0 2024-05-20T21:13:47.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.709+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50035030 con 0x7eff600754b0 2024-05-20T21:13:47.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.709+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a2e50 con 0x7eff600754b0 2024-05-20T21:13:47.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.709+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a32e0 con 0x7eff600754b0 2024-05-20T21:13:47.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.710+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff601096b0 con 0x7eff600754b0 2024-05-20T21:13:47.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.712+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7eff50042070 con 0x7eff600754b0 2024-05-20T21:13:47.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.713+0000 7eff5cd99640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7eff4c0b27f0 0x7eff4c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:47.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.713+0000 7eff57fff640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7eff4c0b27f0 0x7eff4c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:47.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.713+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7eff501032f0 con 0x7eff600754b0 2024-05-20T21:13:47.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.713+0000 7eff57fff640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7eff4c0b27f0 0x7eff4c0b4cb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff48006fd0 tx=0x7eff48008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:47.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.716+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff500cafb0 con 0x7eff600754b0 2024-05-20T21:13:47.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:47 smithi144 ceph-mon[35660]: pgmap v919: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.9 MiB/s wr, 204 op/s 2024-05-20T21:13:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.910+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff6002ddf0 con 0x7eff600754b0 2024-05-20T21:13:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.911+0000 7eff5cd99640 1 -- 172.21.15.26:0/62192616 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22791 (secure 0 0 0) 0x7eff500ca8b0 con 0x7eff600754b0 2024-05-20T21:13:47.911 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:13:47.911 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":84,"fsid":"9006c62a-16e9-11ef-bc9a-c7b262605968","created":"2024-05-20T20:44:33.648165+0000","modified":"2024-05-20T21:03:13.415076+0000","last_up_change":"2024-05-20T20:51:44.088305+0000","last_in_change":"2024-05-20T20:51:24.313094+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T20:47:57.693552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T20:52:41.091910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T20:52:43.961192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2c397ed-d333-4e7b-ba99-70756dd5fe29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6803","nonce":3292543364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6805","nonce":3292543364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6809","nonce":3292543364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3292543364},{"type":"v1","addr":"172.21.15.26:6807","nonce":3292543364}]},"public_addr":"172.21.15.26:6803/3292543364","cluster_addr":"172.21.15.26:6805/3292543364","heartbeat_back_addr":"172.21.15.26:6809/3292543364","heartbeat_front_addr":"172.21.15.26:6807/3292543364","state":["exists","up"]},{"osd":1,"uuid":"d138e804-8e04-4eaf-a954-9ff798e9c4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6801","nonce":1106200301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6803","nonce":1106200301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6807","nonce":1106200301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1106200301},{"type":"v1","addr":"172.21.15.55:6805","nonce":1106200301}]},"public_addr":"172.21.15.55:6801/1106200301","cluster_addr":"172.21.15.55:6803/1106200301","heartbeat_back_addr":"172.21.15.55:6807/1106200301","heartbeat_front_addr":"172.21.15.55:6805/1106200301","state":["exists","up"]},{"osd":2,"uuid":"f7569bb5-2603-43fa-8f88-5bbba4b6e930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6801","nonce":632346919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6803","nonce":632346919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6807","nonce":632346919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":632346919},{"type":"v1","addr":"172.21.15.144:6805","nonce":632346919}]},"public_addr":"172.21.15.144:6801/632346919","cluster_addr":"172.21.15.144:6803/632346919","heartbeat_back_addr":"172.21.15.144:6807/632346919","heartbeat_front_addr":"172.21.15.144:6805/632346919","state":["exists","up"]},{"osd":3,"uuid":"eda9b415-a29a-489c-8d1c-2d893c0170e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6811","nonce":2074985040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6813","nonce":2074985040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6817","nonce":2074985040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2074985040},{"type":"v1","addr":"172.21.15.26:6815","nonce":2074985040}]},"public_addr":"172.21.15.26:6811/2074985040","cluster_addr":"172.21.15.26:6813/2074985040","heartbeat_back_addr":"172.21.15.26:6817/2074985040","heartbeat_front_addr":"172.21.15.26:6815/2074985040","state":["exists","up"]},{"osd":4,"uuid":"7ca8decb-66db-4743-bcb4-ca3efeae40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6809","nonce":1113585138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6811","nonce":1113585138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6815","nonce":1113585138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1113585138},{"type":"v1","addr":"172.21.15.55:6813","nonce":1113585138}]},"public_addr":"172.21.15.55:6809/1113585138","cluster_addr":"172.21.15.55:6811/1113585138","heartbeat_back_addr":"172.21.15.55:6815/1113585138","heartbeat_front_addr":"172.21.15.55:6813/1113585138","state":["exists","up"]},{"osd":5,"uuid":"090b604a-f758-4fa3-b06f-dc15da11739f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6809","nonce":1448183794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6811","nonce":1448183794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6815","nonce":1448183794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1448183794},{"type":"v1","addr":"172.21.15.144:6813","nonce":1448183794}]},"public_addr":"172.21.15.144:6809/1448183794","cluster_addr":"172.21.15.144:6811/1448183794","heartbeat_back_addr":"172.21.15.144:6815/1448183794","heartbeat_front_addr":"172.21.15.144:6813/1448183794","state":["exists","up"]},{"osd":6,"uuid":"54095128-6ffb-408b-8ac6-e8728acec871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6819","nonce":1035586360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6821","nonce":1035586360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6825","nonce":1035586360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1035586360},{"type":"v1","addr":"172.21.15.26:6823","nonce":1035586360}]},"public_addr":"172.21.15.26:6819/1035586360","cluster_addr":"172.21.15.26:6821/1035586360","heartbeat_back_addr":"172.21.15.26:6825/1035586360","heartbeat_front_addr":"172.21.15.26:6823/1035586360","state":["exists","up"]},{"osd":7,"uuid":"e9320773-2918-438a-8cf7-f9171c836fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6817","nonce":1141160889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6819","nonce":1141160889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6823","nonce":1141160889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":1141160889},{"type":"v1","addr":"172.21.15.55:6821","nonce":1141160889}]},"public_addr":"172.21.15.55:6817/1141160889","cluster_addr":"172.21.15.55:6819/1141160889","heartbeat_back_addr":"172.21.15.55:6823/1141160889","heartbeat_front_addr":"172.21.15.55:6821/1141160889","state":["exists","up"]},{"osd":8,"uuid":"c564749e-8c86-4759-9818-af2408d1b735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6817","nonce":2473562796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6819","nonce":2473562796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6823","nonce":2473562796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2473562796},{"type":"v1","addr":"172.21.15.144:6821","nonce":2473562796}]},"public_addr":"172.21.15.144:6817/2473562796","cluster_addr":"172.21.15.144:6819/2473562796","heartbeat_back_addr":"172.21.15.144:6823/2473562796","heartbeat_front_addr":"172.21.15.144:6821/2473562796","state":["exists","up"]},{"osd":9,"uuid":"258768d8-6e31-4618-9ec1-12e184f159a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6827","nonce":2188316601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6829","nonce":2188316601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6833","nonce":2188316601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2188316601},{"type":"v1","addr":"172.21.15.26:6831","nonce":2188316601}]},"public_addr":"172.21.15.26:6827/2188316601","cluster_addr":"172.21.15.26:6829/2188316601","heartbeat_back_addr":"172.21.15.26:6833/2188316601","heartbeat_front_addr":"172.21.15.26:6831/2188316601","state":["exists","up"]},{"osd":10,"uuid":"ea416660-9136-4ac1-a30e-f449d892b3cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6825","nonce":46093047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6827","nonce":46093047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6831","nonce":46093047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":46093047},{"type":"v1","addr":"172.21.15.55:6829","nonce":46093047}]},"public_addr":"172.21.15.55:6825/46093047","cluster_addr":"172.21.15.55:6827/46093047","heartbeat_back_addr":"172.21.15.55:6831/46093047","heartbeat_front_addr":"172.21.15.55:6829/46093047","state":["exists","up"]},{"osd":11,"uuid":"d926e90a-ceaf-4f1d-b43b-94f2b3d1ba19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6825","nonce":2410307143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6827","nonce":2410307143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6831","nonce":2410307143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2410307143},{"type":"v1","addr":"172.21.15.144:6829","nonce":2410307143}]},"public_addr":"172.21.15.144:6825/2410307143","cluster_addr":"172.21.15.144:6827/2410307143","heartbeat_back_addr":"172.21.15.144:6831/2410307143","heartbeat_front_addr":"172.21.15.144:6829/2410307143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:08.140330+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:32.289101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:47:55.323597+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:19.656615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:48:45.061123+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:09.639593+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:49:34.660928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:00.894524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:25.080381+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:50:51.064055+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:16.656087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T20:51:42.369620+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3367150269":"2024-05-21T20:45:35.597719+0000","172.21.15.26:6800/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/1750454874":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/3909957816":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6800/1825381346":"2024-05-21T20:45:08.150176+0000","172.21.15.26:0/1784334339":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/767936803":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/2532144649":"2024-05-21T20:45:35.597719+0000","172.21.15.26:0/2879790538":"2024-05-21T20:45:08.150176+0000","172.21.15.26:6801/1825381346":"2024-05-21T20:45:08.150176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7eff4c0b27f0 msgr2=0x7eff4c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7eff4c0b27f0 0x7eff4c0b4cb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff48006fd0 tx=0x7eff48008040 comp rx=0 tx=0).stop 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 msgr2=0x7eff6019afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 0x7eff6019afa0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7eff50031460 tx=0x7eff50003d50 comp rx=0 tx=0).stop 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 shutdown_connections 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7eff4c0b27f0 0x7eff4c0b4cb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6010a980 0x7eff601a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff60076a90 0x7eff6019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 --2- 172.21.15.26:0/62192616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff600754b0 0x7eff6019afa0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.913+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 >> 172.21.15.26:0/62192616 conn(0x7eff600fe050 msgr2=0x7eff60106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.914+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 shutdown_connections 2024-05-20T21:13:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:47.914+0000 7eff65254640 1 -- 172.21.15.26:0/62192616 wait complete. 2024-05-20T21:13:47.925 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T20:47:57.693552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T20:52:41.091910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T20:52:43.961192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T21:13:47.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T21:13:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 -- 172.21.15.26:0/621724971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c108790 msgr2=0x7f678c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 --2- 172.21.15.26:0/621724971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c108790 0x7f678c10abe0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6784009ca0 tx=0x7f678402a8c0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 -- 172.21.15.26:0/621724971 shutdown_connections 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 --2- 172.21.15.26:0/621724971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c108790 0x7f678c10abe0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 --2- 172.21.15.26:0/621724971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c105e60 0x7f678c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 --2- 172.21.15.26:0/621724971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c103530 0x7f678c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 -- 172.21.15.26:0/621724971 >> 172.21.15.26:0/621724971 conn(0x7f678c0fcf40 msgr2=0x7f678c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 -- 172.21.15.26:0/621724971 shutdown_connections 2024-05-20T21:13:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.034+0000 7f679467e640 1 -- 172.21.15.26:0/621724971 wait complete. 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 Processor -- start 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 -- start start 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c103530 0x7f678c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 0x7f678c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c108790 0x7f678c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e4e0 con 0x7f678c105e60 2024-05-20T21:13:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e360 con 0x7f678c108790 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679467e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e660 con 0x7f678c103530 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f6791c19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 0x7f678c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f6791c19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 0x7f678c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38754/0 (socket says 172.21.15.26:38754) 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f6791c19640 1 -- 172.21.15.26:0/2734922161 learned_addr learned my addr 172.21.15.26:0/2734922161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f679241a640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c103530 0x7f678c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.035+0000 7f6792c1b640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c108790 0x7f678c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6791c19640 1 -- 172.21.15.26:0/2734922161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c103530 msgr2=0x7f678c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6791c19640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c103530 0x7f678c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6791c19640 1 -- 172.21.15.26:0/2734922161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c108790 msgr2=0x7f678c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6791c19640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c108790 0x7f678c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6791c19640 1 -- 172.21.15.26:0/2734922161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1a0800 con 0x7f678c105e60 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6792c1b640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c108790 0x7f678c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f6791c19640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 0x7f678c199120 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c970 tx=0x7f677c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c007bb0 con 0x7f678c105e60 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f677c007d50 con 0x7f678c105e60 2024-05-20T21:13:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c004e10 con 0x7f678c105e60 2024-05-20T21:13:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c1a0af0 con 0x7f678c105e60 2024-05-20T21:13:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.036+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c1a0ff0 con 0x7f678c105e60 2024-05-20T21:13:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.037+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c0fea30 con 0x7f678c105e60 2024-05-20T21:13:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.038+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f677c005000 con 0x7f678c105e60 2024-05-20T21:13:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.039+0000 7f67837fe640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67780b27f0 0x7f67780b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.039+0000 7f679241a640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67780b27f0 0x7f67780b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.039+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f677c0e2360 con 0x7f678c105e60 2024-05-20T21:13:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.040+0000 7f679241a640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67780b27f0 0x7f67780b4cb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67880058c0 tx=0x7f678800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.043+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f677c014030 con 0x7f678c105e60 2024-05-20T21:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.235+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f678c02dd30 con 0x7f678c105e60 2024-05-20T21:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.235+0000 7f67837fe640 1 -- 172.21.15.26:0/2734922161 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f677c0a9fc0 con 0x7f678c105e60 2024-05-20T21:13:48.235 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-05-20T21:13:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67780b27f0 msgr2=0x7f67780b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67780b27f0 0x7f67780b4cb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67880058c0 tx=0x7f678800a2d0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 msgr2=0x7f678c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 0x7f678c199120 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c970 tx=0x7f677c00ce30 comp rx=0 tx=0).stop 2024-05-20T21:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 shutdown_connections 2024-05-20T21:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f67780b27f0 0x7f67780b4cb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c108790 0x7f678c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c105e60 0x7f678c199120 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 --2- 172.21.15.26:0/2734922161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c103530 0x7f678c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 >> 172.21.15.26:0/2734922161 conn(0x7f678c0fcf40 msgr2=0x7f678c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 shutdown_connections 2024-05-20T21:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.237+0000 7f679467e640 1 -- 172.21.15.26:0/2734922161 wait complete. 2024-05-20T21:13:48.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T21:13:48.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 -- 172.21.15.26:0/1573052802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4103ca0 msgr2=0x7f9cb4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/1573052802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4103ca0 0x7f9cb4104120 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009900 tx=0x7f9ca802a410 comp rx=0 tx=0).stop 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 -- 172.21.15.26:0/1573052802 shutdown_connections 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/1573052802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb4104660 0x7f9cb410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/1573052802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4103ca0 0x7f9cb4104120 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/1573052802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 -- 172.21.15.26:0/1573052802 >> 172.21.15.26:0/1573052802 conn(0x7f9cb40fe020 msgr2=0x7f9cb4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 -- 172.21.15.26:0/1573052802 shutdown_connections 2024-05-20T21:13:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 -- 172.21.15.26:0/1573052802 wait complete. 2024-05-20T21:13:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.357+0000 7f9cb9eda640 1 Processor -- start 2024-05-20T21:13:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 -- start start 2024-05-20T21:13:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb4103ca0 0x7f9cb41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cb4104660 0x7f9cb41a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb410e490 con 0x7f9cb4102ab0 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb410e310 con 0x7f9cb4104660 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb9eda640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb410e610 con 0x7f9cb4103ca0 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38780/0 (socket says 172.21.15.26:38780) 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 -- 172.21.15.26:0/2786015681 learned_addr learned my addr 172.21.15.26:0/2786015681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb3fff640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cb4104660 0x7f9cb41a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb2ffd640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb4103ca0 0x7f9cb41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 -- 172.21.15.26:0/2786015681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb4103ca0 msgr2=0x7f9cb41991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb4103ca0 0x7f9cb41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 -- 172.21.15.26:0/2786015681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cb4104660 msgr2=0x7f9cb41a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cb4104660 0x7f9cb41a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 -- 172.21.15.26:0/2786015681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb41a0870 con 0x7f9cb4102ab0 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.358+0000 7f9cb37fe640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4198c60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400d770 tx=0x7f9ca400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.359+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4017070 con 0x7f9cb4102ab0 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.359+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca4009c30 con 0x7f9cb4102ab0 2024-05-20T21:13:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.359+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb41a0b60 con 0x7f9cb4102ab0 2024-05-20T21:13:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.359+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4003c50 con 0x7f9cb4102ab0 2024-05-20T21:13:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.359+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb41a10b0 con 0x7f9cb4102ab0 2024-05-20T21:13:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.360+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb4109ef0 con 0x7f9cb4102ab0 2024-05-20T21:13:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.361+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9ca4003df0 con 0x7f9cb4102ab0 2024-05-20T21:13:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.362+0000 7f9cb0ff9640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c9c0b27f0 0x7f9c9c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.362+0000 7f9cb2ffd640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c9c0b27f0 0x7f9c9c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.362+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f9ca40e1d00 con 0x7f9cb4102ab0 2024-05-20T21:13:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.362+0000 7f9cb2ffd640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c9c0b27f0 0x7f9c9c0b4cb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8031460 tx=0x7f9ca8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.366+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ca40a99c0 con 0x7f9cb4102ab0 2024-05-20T21:13:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.557+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9cb402de20 con 0x7f9cb4102ab0 2024-05-20T21:13:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.557+0000 7f9cb0ff9640 1 -- 172.21.15.26:0/2786015681 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f9ca4009de0 con 0x7f9cb4102ab0 2024-05-20T21:13:48.557 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-05-20T21:13:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c9c0b27f0 msgr2=0x7f9c9c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c9c0b27f0 0x7f9c9c0b4cb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8031460 tx=0x7f9ca8000bc0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 msgr2=0x7f9cb4198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4198c60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400d770 tx=0x7f9ca400dc30 comp rx=0 tx=0).stop 2024-05-20T21:13:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 shutdown_connections 2024-05-20T21:13:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f9c9c0b27f0 0x7f9c9c0b4cb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cb4104660 0x7f9cb41a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb4103ca0 0x7f9cb41991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 --2- 172.21.15.26:0/2786015681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cb4102ab0 0x7f9cb4198c60 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 >> 172.21.15.26:0/2786015681 conn(0x7f9cb40fe020 msgr2=0x7f9cb41070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 shutdown_connections 2024-05-20T21:13:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.559+0000 7f9cb9eda640 1 -- 172.21.15.26:0/2786015681 wait complete. 2024-05-20T21:13:48.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/62192616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T21:13:48.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:48 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2734922161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T21:13:48.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T21:13:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/62192616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T21:13:48.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:48 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2734922161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T21:13:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 -- 172.21.15.26:0/2395097320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 msgr2=0x7f929c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 --2- 172.21.15.26:0/2395097320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c101db0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f928c009900 tx=0x7f928c02a410 comp rx=0 tx=0).stop 2024-05-20T21:13:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 -- 172.21.15.26:0/2395097320 shutdown_connections 2024-05-20T21:13:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 --2- 172.21.15.26:0/2395097320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c103540 0x7f929c109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 --2- 172.21.15.26:0/2395097320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f929c102b80 0x7f929c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 --2- 172.21.15.26:0/2395097320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c101db0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 -- 172.21.15.26:0/2395097320 >> 172.21.15.26:0/2395097320 conn(0x7f929c0fcf00 msgr2=0x7f929c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 -- 172.21.15.26:0/2395097320 shutdown_connections 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.719+0000 7f92a1415640 1 -- 172.21.15.26:0/2395097320 wait complete. 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 Processor -- start 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 -- start start 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c102b80 0x7f929c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f929c103540 0x7f929c19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f929c10d370 con 0x7f929c101990 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f929c10d1f0 con 0x7f929c103540 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f92a1415640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f929c10d4f0 con 0x7f929c102b80 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f929affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f929affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c06a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38794/0 (socket says 172.21.15.26:38794) 2024-05-20T21:13:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.720+0000 7f929affd640 1 -- 172.21.15.26:0/3936296225 learned_addr learned my addr 172.21.15.26:0/3936296225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929b7fe640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f929c103540 0x7f929c19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f92927fc640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c102b80 0x7f929c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929affd640 1 -- 172.21.15.26:0/3936296225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c102b80 msgr2=0x7f929c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929affd640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c102b80 0x7f929c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929affd640 1 -- 172.21.15.26:0/3936296225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f929c103540 msgr2=0x7f929c19ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929affd640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f929c103540 0x7f929c19ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929affd640 1 -- 172.21.15.26:0/3936296225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929c1a0890 con 0x7f929c101990 2024-05-20T21:13:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f929affd640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c06a700 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f928c005ab0 tx=0x7f928c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c03f030 con 0x7f929c101990 2024-05-20T21:13:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929c1a0b20 con 0x7f929c101990 2024-05-20T21:13:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f928c003c90 con 0x7f929c101990 2024-05-20T21:13:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c035070 con 0x7f929c101990 2024-05-20T21:13:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.721+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f929c1a1010 con 0x7f929c101990 2024-05-20T21:13:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.722+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929c108ec0 con 0x7f929c101990 2024-05-20T21:13:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.723+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f928c042030 con 0x7f929c101990 2024-05-20T21:13:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.724+0000 7f9298ff9640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92880b27f0 0x7f92880b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.724+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f928c102d10 con 0x7f929c101990 2024-05-20T21:13:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.724+0000 7f92927fc640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92880b27f0 0x7f92880b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.725+0000 7f92927fc640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92880b27f0 0x7f92880b4cb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9284005fd0 tx=0x7f9284005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.728+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f928c0ca9d0 con 0x7f929c101990 2024-05-20T21:13:48.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/62192616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T21:13:48.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:48 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2734922161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T21:13:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.919+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f929c0636f0 con 0x7f929c101990 2024-05-20T21:13:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.920+0000 7f9298ff9640 1 -- 172.21.15.26:0/3936296225 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f928c0ca2d0 con 0x7f929c101990 2024-05-20T21:13:48.920 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-05-20T21:13:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92880b27f0 msgr2=0x7f92880b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92880b27f0 0x7f92880b4cb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9284005fd0 tx=0x7f9284005850 comp rx=0 tx=0).stop 2024-05-20T21:13:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 msgr2=0x7f929c06a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c06a700 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f928c005ab0 tx=0x7f928c0042f0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 shutdown_connections 2024-05-20T21:13:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f92880b27f0 0x7f92880b4cb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f929c103540 0x7f929c19ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c102b80 0x7f929c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 --2- 172.21.15.26:0/3936296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c101990 0x7f929c06a700 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 >> 172.21.15.26:0/3936296225 conn(0x7f929c0fcf00 msgr2=0x7f929c106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 shutdown_connections 2024-05-20T21:13:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:48.922+0000 7f92a1415640 1 -- 172.21.15.26:0/3936296225 wait complete. 2024-05-20T21:13:48.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-20T21:13:49.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 -- 172.21.15.26:0/3076918942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8104650 msgr2=0x7f3cf810aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 --2- 172.21.15.26:0/3076918942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8104650 0x7f3cf810aee0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009ca0 tx=0x7f3cf002a8c0 comp rx=0 tx=0).stop 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 -- 172.21.15.26:0/3076918942 shutdown_connections 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 --2- 172.21.15.26:0/3076918942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8104650 0x7f3cf810aee0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 --2- 172.21.15.26:0/3076918942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8103c90 0x7f3cf8104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 --2- 172.21.15.26:0/3076918942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cf8102aa0 0x7f3cf8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 -- 172.21.15.26:0/3076918942 >> 172.21.15.26:0/3076918942 conn(0x7f3cf80fe030 msgr2=0x7f3cf8100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 -- 172.21.15.26:0/3076918942 shutdown_connections 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.039+0000 7f3cffa59640 1 -- 172.21.15.26:0/3076918942 wait complete. 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 Processor -- start 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 -- start start 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cf8102aa0 0x7f3cf8076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 0x7f3cf8074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8104650 0x7f3cf8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf810e4d0 con 0x7f3cf8103c90 2024-05-20T21:13:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf810e350 con 0x7f3cf8104650 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cffa59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf810e650 con 0x7f3cf8102aa0 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cfcff4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 0x7f3cf8074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cfcff4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 0x7f3cf8074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38826/0 (socket says 172.21.15.26:38826) 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cfcff4640 1 -- 172.21.15.26:0/397149844 learned_addr learned my addr 172.21.15.26:0/397149844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cfd7f5640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cf8102aa0 0x7f3cf8076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.040+0000 7f3cfdff6640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8104650 0x7f3cf8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:49.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfcff4640 1 -- 172.21.15.26:0/397149844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cf8102aa0 msgr2=0x7f3cf8076800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfcff4640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cf8102aa0 0x7f3cf8076800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfcff4640 1 -- 172.21.15.26:0/397149844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8104650 msgr2=0x7f3cf8075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfcff4640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8104650 0x7f3cf8075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfcff4640 1 -- 172.21.15.26:0/397149844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf8075c10 con 0x7f3cf8103c90 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfdff6640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8104650 0x7f3cf8075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cfcff4640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 0x7f3cf8074e40 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800e9f0 tx=0x7f3ce800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800cd80 con 0x7f3cf8103c90 2024-05-20T21:13:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ce80108c0 con 0x7f3cf8103c90 2024-05-20T21:13:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf81a2ca0 con 0x7f3cf8103c90 2024-05-20T21:13:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8018810 con 0x7f3cf8103c90 2024-05-20T21:13:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.041+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf81a31a0 con 0x7f3cf8103c90 2024-05-20T21:13:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.042+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf8109e80 con 0x7f3cf8103c90 2024-05-20T21:13:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.044+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3ce8010430 con 0x7f3cf8103c90 2024-05-20T21:13:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.045+0000 7f3cee7fc640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ce40b27f0 0x7f3ce40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:13:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.045+0000 7f3cfd7f5640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ce40b27f0 0x7f3ce40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:13:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.045+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f3ce80e2b50 con 0x7f3cf8103c90 2024-05-20T21:13:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.046+0000 7f3cfd7f5640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ce40b27f0 0x7f3ce40b4cb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8103af0 tx=0x7f3cf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:13:49.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.049+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ce80aa810 con 0x7f3cf8103c90 2024-05-20T21:13:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.295+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3cf802dd70 con 0x7f3cf8103c90 2024-05-20T21:13:49.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.305+0000 7f3cee7fc640 1 -- 172.21.15.26:0/397149844 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+815642 (secure 0 0 0) 0x7f3ce80aa110 con 0x7f3cf8103c90 2024-05-20T21:13:49.315 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-20T21:13:49.316 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:13:49.316 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 20, 2024-05-20T21:13:49.316 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.316 INFO:teuthology.orchestra.run.smithi026.stdout: "active_gid": 14150, 2024-05-20T21:13:49.316 INFO:teuthology.orchestra.run.smithi026.stdout: "active_name": "x", 2024-05-20T21:13:49.316 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addrs": { 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6800", 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2583575856 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6801", 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2583575856 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addr": "172.21.15.26:6801/2583575856", 2024-05-20T21:13:49.318 INFO:teuthology.orchestra.run.smithi026.stdout: "active_change": "2024-05-20T20:45:35.597832+0000", 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "active_mgr_features": 4540701547738038271, 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "standbys": [ 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24109, 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "z", 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-05-20T21:13:49.319 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:49.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-05-20T21:13:49.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.323 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-05-20T21:13:49.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.334 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-05-20T21:13:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-05-20T21:13:49.337 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.338 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.340 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-05-20T21:13:49.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.341 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.345 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.346 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-05-20T21:13:49.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.349 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-05-20T21:13:49.350 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-05-20T21:13:49.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T21:13:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-05-20T21:13:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-05-20T21:13:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-20T21:13:49.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.355 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-05-20T21:13:49.356 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.357 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-05-20T21:13:49.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-05-20T21:13:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:49.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-20T21:13:49.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.364 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.369 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-05-20T21:13:49.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.373 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-05-20T21:13:49.375 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-05-20T21:13:49.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:49.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-20T21:13:49.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.381 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:49.382 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.383 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.384 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-05-20T21:13:49.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-20T21:13:49.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-05-20T21:13:49.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-05-20T21:13:49.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-05-20T21:13:49.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-20T21:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.397 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-05-20T21:13:49.398 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-05-20T21:13:49.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-20T21:13:49.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-20T21:13:49.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-05-20T21:13:49.408 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.409 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.409 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-20T21:13:49.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-05-20T21:13:49.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:49.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-20T21:13:49.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.417 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-05-20T21:13:49.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-05-20T21:13:49.420 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.422 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.426 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.427 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-05-20T21:13:49.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-05-20T21:13:49.429 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.431 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-20T21:13:49.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-05-20T21:13:49.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.432 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.435 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-05-20T21:13:49.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-20T21:13:49.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-05-20T21:13:49.441 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-05-20T21:13:49.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-20T21:13:49.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.448 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.450 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.452 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.454 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.455 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.459 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.461 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.463 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.464 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.465 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-05-20T21:13:49.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-05-20T21:13:49.474 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.478 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.479 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.480 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-05-20T21:13:49.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.484 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.485 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.486 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.492 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.495 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-05-20T21:13:49.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-05-20T21:13:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-20T21:13:49.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-05-20T21:13:49.502 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.504 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.505 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.506 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.507 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.509 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.510 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.512 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.513 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.514 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.516 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-20T21:13:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.519 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.522 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-20T21:13:49.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-20T21:13:49.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.522 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.525 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-05-20T21:13:49.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-20T21:13:49.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.535 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.540 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.541 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.542 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-20T21:13:49.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.545 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.548 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-20T21:13:49.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.551 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.552 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.555 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-20T21:13:49.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "[]", 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: "MULTICLUSTER_CONFIG": { 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-20T21:13:49.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "{}", 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.560 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.566 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-20T21:13:49.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.571 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.574 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-20T21:13:49.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.582 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-20T21:13:49.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.589 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.590 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-05-20T21:13:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-20T21:13:49.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.596 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.598 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-05-20T21:13:49.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-05-20T21:13:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-05-20T21:13:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-05-20T21:13:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.640 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.644 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-05-20T21:13:49.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.650 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:49.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.657 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.659 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-05-20T21:13:49.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.668 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.669 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-05-20T21:13:49.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-05-20T21:13:49.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-20T21:13:49.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.684 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.687 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.692 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-05-20T21:13:49.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-05-20T21:13:49.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.714 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.721 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.731 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-05-20T21:13:49.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.733 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.736 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.741 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-05-20T21:13:49.746 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.750 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.752 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.753 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.754 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-05-20T21:13:49.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-05-20T21:13:49.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-20T21:13:49.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.768 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.770 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.774 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.775 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.776 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.781 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.785 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.790 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.792 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.793 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.795 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-05-20T21:13:49.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-20T21:13:49.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.805 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.809 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-05-20T21:13:49.814 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.817 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.818 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.819 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.823 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.830 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.833 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.834 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.846 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.847 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:49.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.850 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.851 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-05-20T21:13:49.853 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-05-20T21:13:49.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.856 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-05-20T21:13:49.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.860 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.865 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.867 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:49.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.871 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-05-20T21:13:49.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.876 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.879 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.883 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-05-20T21:13:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.891 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.896 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-05-20T21:13:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-05-20T21:13:49.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.904 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.908 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-05-20T21:13:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.924 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.925 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.926 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.929 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:49.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.938 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.939 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.940 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.944 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.945 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.946 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.947 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.951 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.953 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.956 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.963 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.965 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.968 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.969 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-05-20T21:13:49.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-05-20T21:13:49.977 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:49.979 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-05-20T21:13:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smb", 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "update_orchestration": { 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "update_orchestration", 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-05-20T21:13:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.014 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.038 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.040 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.041 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.044 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-05-20T21:13:50.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.048 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.061 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.062 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-05-20T21:13:50.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-20T21:13:50.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-05-20T21:13:50.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-05-20T21:13:50.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-05-20T21:13:50.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.085 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.089 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.090 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-05-20T21:13:50.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.095 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.096 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.097 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.101 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.102 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.103 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.104 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.110 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.112 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.115 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.117 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.118 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-05-20T21:13:50.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.125 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.126 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.129 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.130 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.131 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.133 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.135 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.137 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.138 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.140 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.141 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.145 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24116, 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "y", 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-05-20T21:13:50.151 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-05-20T21:13:50.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.155 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.157 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.159 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.163 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.166 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.167 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-05-20T21:13:50.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-05-20T21:13:50.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-05-20T21:13:50.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.180 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-05-20T21:13:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-20T21:13:50.184 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-05-20T21:13:50.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-20T21:13:50.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.190 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-05-20T21:13:50.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.194 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-05-20T21:13:50.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-05-20T21:13:50.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-20T21:13:50.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.222 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-05-20T21:13:50.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.224 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.225 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-05-20T21:13:50.226 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.227 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-05-20T21:13:50.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-20T21:13:50.230 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-05-20T21:13:50.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-05-20T21:13:50.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-20T21:13:50.238 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.239 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.240 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.241 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.242 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-20T21:13:50.243 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.244 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-05-20T21:13:50.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-05-20T21:13:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.250 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.251 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-05-20T21:13:50.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.255 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-05-20T21:13:50.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-05-20T21:13:50.267 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-05-20T21:13:50.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-20T21:13:50.269 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.270 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.271 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-05-20T21:13:50.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.274 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.277 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.279 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.280 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.281 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.282 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-20T21:13:50.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-20T21:13:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-05-20T21:13:50.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-05-20T21:13:50.296 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.299 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.301 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-05-20T21:13:50.302 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-20T21:13:50.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.306 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-05-20T21:13:50.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.310 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.311 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.312 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.313 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-05-20T21:13:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-20T21:13:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.320 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-05-20T21:13:50.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.323 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.325 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.327 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.328 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.331 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-05-20T21:13:50.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.337 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-05-20T21:13:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.340 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-05-20T21:13:50.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.343 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-20T21:13:50.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.348 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.350 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.352 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.353 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-20T21:13:50.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.358 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-20T21:13:50.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.364 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.365 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.368 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.369 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.370 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.373 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-20T21:13:50.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "[]", 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "MULTICLUSTER_CONFIG": { 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "{}", 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-05-20T21:13:50.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.383 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.384 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-20T21:13:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-20T21:13:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-05-20T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-20T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.415 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-20T21:13:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-05-20T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-05-20T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.432 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-05-20T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-05-20T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.446 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.447 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.448 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-05-20T21:13:50.449 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.450 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-05-20T21:13:50.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-05-20T21:13:50.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.456 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.458 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.462 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.467 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.468 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-05-20T21:13:50.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-05-20T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-05-20T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.496 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-05-20T21:13:50.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.500 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.503 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.508 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.510 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-05-20T21:13:50.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-05-20T21:13:50.512 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-05-20T21:13:50.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.518 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.519 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.520 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-05-20T21:13:50.521 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.522 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.525 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.526 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.529 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-05-20T21:13:50.535 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.536 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.537 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.541 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.542 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.544 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.545 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.546 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.547 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-05-20T21:13:50.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-20T21:13:50.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.552 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.557 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.565 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.569 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.574 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.581 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.585 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.588 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.590 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.592 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.595 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.597 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.601 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.608 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.610 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.611 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-05-20T21:13:50.618 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.622 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.625 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.626 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-05-20T21:13:50.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.634 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.636 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.640 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.641 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.644 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.645 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.647 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.651 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.658 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.662 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-05-20T21:13:50.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-05-20T21:13:50.671 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-05-20T21:13:50.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.676 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.680 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:50.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-05-20T21:13:50.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.689 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-05-20T21:13:50.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-20T21:13:50.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.698 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.702 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-05-20T21:13:50.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.707 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-05-20T21:13:50.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.718 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.719 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.720 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.721 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.722 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.723 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-05-20T21:13:50.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.731 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-05-20T21:13:50.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.734 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.735 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.737 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.760 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.762 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.764 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.765 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.768 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-05-20T21:13:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.774 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.777 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.780 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.782 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-05-20T21:13:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.795 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-05-20T21:13:50.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.799 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.800 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-05-20T21:13:50.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-05-20T21:13:50.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smb", 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.811 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.814 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.819 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "update_orchestration": { 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "update_orchestration", 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.822 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-05-20T21:13:50.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-05-20T21:13:50.825 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.827 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.832 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.833 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.839 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.841 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.842 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.843 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.844 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.846 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.847 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.849 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.851 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.852 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.853 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.855 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.858 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.861 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.866 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.867 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.870 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.876 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-05-20T21:13:50.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.880 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-05-20T21:13:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.887 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-05-20T21:13:50.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-05-20T21:13:50.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.894 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-05-20T21:13:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard", 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-05-20T21:13:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-05-20T21:13:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-05-20T21:13:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-05-20T21:13:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-05-20T21:13:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-20T21:13:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-05-20T21:13:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-05-20T21:13:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-20T21:13:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-05-20T21:13:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-05-20T21:13:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-05-20T21:13:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-20T21:13:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-05-20T21:13:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-05-20T21:13:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-05-20T21:13:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-20T21:13:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-20T21:13:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-05-20T21:13:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-20T21:13:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-05-20T21:13:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-05-20T21:13:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-05-20T21:13:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-05-20T21:13:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-20T21:13:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-05-20T21:13:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-05-20T21:13:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-05-20T21:13:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-05-20T21:13:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-20T21:13:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-05-20T21:13:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-05-20T21:13:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-05-20T21:13:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-20T21:13:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-05-20T21:13:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-05-20T21:13:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-05-20T21:13:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-20T21:13:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-05-20T21:13:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-05-20T21:13:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-20T21:13:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-05-20T21:13:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-20T21:13:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-20T21:13:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-20T21:13:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "[]", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "MULTICLUSTER_CONFIG": { 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "{}", 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-20T21:13:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-20T21:13:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-20T21:13:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-20T21:13:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-05-20T21:13:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-20T21:13:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-05-20T21:13:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-05-20T21:13:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-05-20T21:13:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-05-20T21:13:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-05-20T21:13:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-20T21:13:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-05-20T21:13:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-05-20T21:13:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-05-20T21:13:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-05-20T21:13:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-20T21:13:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-05-20T21:13:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-05-20T21:13:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-05-20T21:13:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-05-20T21:13:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-05-20T21:13:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-05-20T21:13:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-05-20T21:13:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-05-20T21:13:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-05-20T21:13:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-05-20T21:13:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-05-20T21:13:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-05-20T21:13:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-05-20T21:13:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smb", 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "update_orchestration": { 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "update_orchestration", 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-05-20T21:13:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-20T21:13:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-05-20T21:13:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-05-20T21:13:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-05-20T21:13:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-05-20T21:13:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-20T21:13:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-05-20T21:13:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-05-20T21:13:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-05-20T21:13:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-05-20T21:13:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-05-20T21:13:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "sqlite3_killpoint": { 2024-05-20T21:13:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sqlite3_killpoint", 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "dev", 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-20T21:13:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard": "https://172.21.15.26:8443/" 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "always_on_modules": { 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus": [ 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-05-20T21:13:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-05-20T21:13:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific": [ 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-05-20T21:13:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy": [ 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-05-20T21:13:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "reef": [ 2024-05-20T21:13:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-05-20T21:13:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "squid": [ 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-05-20T21:13:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "last_failure_osd_epoch": 3, 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "active_clients": [ 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-05-20T21:13:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 432920170 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "libcephsqlite", 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1062974415 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-05-20T21:13:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1998339259 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3316882063 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-20T21:13:51.806 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:13:51.807 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:49 smithi144 ceph-mon[35660]: pgmap v920: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-05-20T21:13:51.807 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:49 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2786015681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T21:13:51.807 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:49 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/3936296225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:49 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/397149844' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:51 smithi144 ceph-mon[35660]: pgmap v921: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:49 smithi055 ceph-mon[35213]: pgmap v920: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:49 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2786015681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:49 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/3936296225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:49 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/397149844' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-20T21:13:51.808 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:51 smithi055 ceph-mon[35213]: pgmap v921: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-20T21:13:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ce40b27f0 msgr2=0x7f3ce40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ce40b27f0 0x7f3ce40b4cb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8103af0 tx=0x7f3cf4005f90 comp rx=0 tx=0).stop 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 msgr2=0x7f3cf8074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 0x7f3cf8074e40 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800e9f0 tx=0x7f3ce800eeb0 comp rx=0 tx=0).stop 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 shutdown_connections 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f3ce40b27f0 0x7f3ce40b4cb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf8104650 0x7f3cf8075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf8103c90 0x7f3cf8074e40 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 --2- 172.21.15.26:0/397149844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cf8102aa0 0x7f3cf8076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:13:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 >> 172.21.15.26:0/397149844 conn(0x7f3cf80fe030 msgr2=0x7f3cf8107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:13:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 shutdown_connections 2024-05-20T21:13:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:13:49.311+0000 7f3cffa59640 1 -- 172.21.15.26:0/397149844 wait complete. 2024-05-20T21:13:51.817 DEBUG:tasks.check_counter:Getting stats from a 2024-05-20T21:13:51.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-20T21:13:51.818 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:49 smithi026 ceph-mon[31990]: pgmap v920: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-05-20T21:13:51.818 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:49 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2786015681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T21:13:51.819 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:49 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/3936296225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T21:13:51.819 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:49 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/397149844' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-20T21:13:51.819 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:51 smithi026 ceph-mon[31990]: pgmap v921: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-20T21:13:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:53.608 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:53 smithi026 ceph-mon[31990]: pgmap v922: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 719 KiB/s wr, 97 op/s 2024-05-20T21:13:53.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:53 smithi055 ceph-mon[35213]: pgmap v922: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 719 KiB/s wr, 97 op/s 2024-05-20T21:13:53.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:53 smithi144 ceph-mon[35660]: pgmap v922: 97 pgs: 97 active+clean; 281 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 719 KiB/s wr, 97 op/s 2024-05-20T21:13:54.010 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:13:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:13:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001588747, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-05-20T21:13:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000271512, 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 352, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 345, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 852912, 2024-05-20T21:13:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 212064, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.046189770, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.019237157, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.020656289, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.006636756, 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:13:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 345, 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.012233721, 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000035460 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 852912, 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 212064 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 262, 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2024-05-20T21:13:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 203584, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.017053847, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.013413857, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001351397, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 260, 2024-05-20T21:13:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010391661, 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000039967 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 203584 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-20T21:13:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:13:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:13:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:13:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-20T21:13:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148704, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 29, 2024-05-20T21:13:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:13:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-05-20T21:13:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-20T21:13:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 350, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 800749, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 350, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 350, 2024-05-20T21:13:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 800749, 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:54.029 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:13:54.924 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-20T21:13:54.925 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-20T21:13:54.925 DEBUG:tasks.check_counter:Getting stats from d 2024-05-20T21:13:54.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-20T21:13:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:55.634 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:55 smithi026 ceph-mon[31990]: pgmap v923: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 261 KiB/s wr, 71 op/s 2024-05-20T21:13:55.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:55 smithi055 ceph-mon[35213]: pgmap v923: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 261 KiB/s wr, 71 op/s 2024-05-20T21:13:55.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:55 smithi144 ceph-mon[35660]: pgmap v923: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 261 KiB/s wr, 71 op/s 2024-05-20T21:13:57.175 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:13:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:13:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 2960, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 448, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001740342, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000039889, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000647974, 2024-05-20T21:13:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000814772, 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000271590 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 354, 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 349, 2024-05-20T21:13:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 851584, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 215456, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.048178792, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.019453327, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.022946724, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.006701278, 2024-05-20T21:13:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 348, 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.012533573, 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000036016 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 851584, 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 215456 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 266, 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2024-05-20T21:13:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 204288, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.017460385, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.013643964, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001480804, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 264, 2024-05-20T21:13:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010653320, 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000040353 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:13:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 204288 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:13:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:13:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-20T21:13:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-20T21:13:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:13:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148744, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-20T21:13:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-20T21:13:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:13:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 356, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 801529, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 356, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-20T21:13:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 356, 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 801529, 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:13:57.194 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:13:57.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:57 smithi026 ceph-mon[31990]: pgmap v924: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 39 op/s 2024-05-20T21:13:57.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:57 smithi055 ceph-mon[35213]: pgmap v924: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 39 op/s 2024-05-20T21:13:57.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:57 smithi144 ceph-mon[35660]: pgmap v924: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 39 op/s 2024-05-20T21:13:57.993 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-20T21:13:57.994 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-20T21:13:57.994 DEBUG:tasks.check_counter:Getting stats from g 2024-05-20T21:13:57.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-20T21:13:58.249 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:13:59.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:13:59 smithi055 ceph-mon[35213]: pgmap v925: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-05-20T21:13:59.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:13:59 smithi026 ceph-mon[31990]: pgmap v925: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-05-20T21:13:59.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:13:59 smithi144 ceph-mon[35660]: pgmap v925: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-05-20T21:14:00.160 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:14:00.160 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:00.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 350, 2024-05-20T21:14:00.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 345, 2024-05-20T21:14:00.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 850496, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 212896, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.048407870, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.018979831, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.023889784, 2024-05-20T21:14:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.006521036, 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 344, 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.011433881, 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000033238 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:00.162 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 850496, 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 212896 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:00.163 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 6, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 265, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 3280, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 204208, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.017914208, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.013369796, 2024-05-20T21:14:00.164 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.002466078, 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 264, 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010225923, 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000038734 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:00.165 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3280, 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 204208 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:00.166 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.000365523, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000287595, 2024-05-20T21:14:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-05-20T21:14:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148704, 2024-05-20T21:14:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 29, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-05-20T21:14:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-20T21:14:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 352, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 801009, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 352, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 352, 2024-05-20T21:14:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 801009, 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:14:00.179 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:14:01.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:01 smithi055 ceph-mon[35213]: pgmap v926: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 25 op/s 2024-05-20T21:14:01.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:01 smithi026 ceph-mon[31990]: pgmap v926: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 25 op/s 2024-05-20T21:14:01.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:01 smithi144 ceph-mon[35660]: pgmap v926: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 25 op/s 2024-05-20T21:14:02.041 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-20T21:14:02.041 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-20T21:14:02.041 DEBUG:tasks.check_counter:Getting stats from j 2024-05-20T21:14:02.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-20T21:14:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:14:03.665 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:03 smithi026 ceph-mon[31990]: pgmap v927: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 16 op/s 2024-05-20T21:14:03.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:03 smithi055 ceph-mon[35213]: pgmap v927: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 16 op/s 2024-05-20T21:14:03.900 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:03 smithi144 ceph-mon[35660]: pgmap v927: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 16 op/s 2024-05-20T21:14:05.270 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-20T21:14:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4, 2024-05-20T21:14:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4, 2024-05-20T21:14:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 2960, 2024-05-20T21:14:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 448, 2024-05-20T21:14:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001791765, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000043650, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000783475, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2024-05-20T21:14:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000839895, 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000279965 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 357, 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 352, 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 852400, 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 217376, 2024-05-20T21:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.049292301, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.019837294, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.023833243, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.006741030, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 351, 2024-05-20T21:14:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.012780966, 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000036413 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 852400, 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 217376 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 269, 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2024-05-20T21:14:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 204816, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.018248399, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.014309191, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001562475, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 267, 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.011500763, 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000043074 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.279 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 204816 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-20T21:14:05.280 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:05.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:05.283 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:05.284 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148722, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:05.285 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-20T21:14:05.286 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-20T21:14:05.287 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 359, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 801919, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 359, 2024-05-20T21:14:05.288 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 359, 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 801919, 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-20T21:14:05.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:14:05.290 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-20T21:14:05.290 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-20T21:14:05.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:05 smithi026 ceph-mon[31990]: pgmap v928: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-05-20T21:14:05.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:05 smithi055 ceph-mon[35213]: pgmap v928: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-05-20T21:14:05.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:05 smithi144 ceph-mon[35660]: pgmap v928: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-05-20T21:14:05.935 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-20T21:14:05.935 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-20T21:14:05.935 DEBUG:tasks.check_counter:Getting stats from b 2024-05-20T21:14:05.936 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-20T21:14:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T21:14:07.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:07 smithi055 ceph-mon[35213]: pgmap v929: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-20T21:14:07.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:07 smithi026 ceph-mon[31990]: pgmap v929: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-20T21:14:07.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:07 smithi144 ceph-mon[35660]: pgmap v929: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-20T21:14:08.880 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 38978, 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 46709, 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 10035652, 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 118324181, 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 398, 2024-05-20T21:14:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 3.260580450, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 1.297672266, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 1.080406996, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.766660502, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 46707, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 4.330552116, 2024-05-20T21:14:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000092717 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 35855, 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.006927211, 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000193 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 40032, 2024-05-20T21:14:08.883 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 5120 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 950637, 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 1044390, 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 285145235, 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 706078102, 2024-05-20T21:14:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 39, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 41.583648191, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 16.724718631, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 29.338442682, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.764537766, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 1044389, 2024-05-20T21:14:08.885 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 20.893819851, 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000020005 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 1917116, 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.256371038, 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000133 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.886 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 1009648, 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 283568 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 35436, 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 39129, 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 9258056, 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 60795532, 2024-05-20T21:14:08.887 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 8, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 2.900947222, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 1.295779335, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 1.153411609, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.500260959, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 39128, 2024-05-20T21:14:08.888 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 3.169574691, 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000081005 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 43307, 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.008016984, 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000185 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.889 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 553248 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 1, 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: "finisher-MDSRank": { 2024-05-20T21:14:08.890 INFO:teuthology.orchestra.run.smithi055.stdout: "queue_len": 0, 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: "complete_latency": { 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 58886, 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 70.176889384, 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001191741 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: "finisher-PurgeQueue": { 2024-05-20T21:14:08.891 INFO:teuthology.orchestra.run.smithi055.stdout: "queue_len": 0, 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: "complete_latency": { 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 26772, 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 4.457100925, 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000166483 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": { 2024-05-20T21:14:08.892 INFO:teuthology.orchestra.run.smithi055.stdout: "request": 856333, 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "reply": 855523, 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "reply_latency": { 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 855523, 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 1363.028532852, 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001593210 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "slow_reply": 0, 2024-05-20T21:14:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: "forward": 810, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_fetch_complete": 5, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_fetch_keys": 2, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_commit": 2564, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_split": 0, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_merge": 0, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes": 802, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_top": 83, 2024-05-20T21:14:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_bottom": 663, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_pin_tail": 56, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_pinned": 526, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_expired": 170, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_with_caps": 513, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": 513, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "subtrees": 14, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse": 1653449, 2024-05-20T21:14:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_hit": 1639116, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_forward": 608, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_discover": 5, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_dir_fetch": 0, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_remote_ino": 0, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_lock": 3, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "load_cent": 0, 2024-05-20T21:14:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: "q": 0, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "exported": 12, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "exported_inodes": 161, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "imported": 3, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "imported_inodes": 42, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_dir_fetch": 0, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_backtrace_fetch": 2804, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_peer_discover": 0, 2024-05-20T21:14:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rfiles": 703, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rbytes": 224342856, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rsnaps": 0, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_backtrace_fetch": 2804, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_set_tag": 0, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_inotable_repaired": 0, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dir_inodes": 618, 2024-05-20T21:14:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dir_base_inodes": 29, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dirfrag_rstats": 589, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_file_inodes": 2186, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_inode_file_caps": 0, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_revoke": 43551, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_grant": 62921, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T21:14:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_caps": 67643, 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_caps_dirty": 21666, 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_cap_release": 697, 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "process_request_cap_release": 60067 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_cache": { 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_update": 15, 2024-05-20T21:14:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_update_receipt": 0, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_try_discover": 0, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_send_discover": 5, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_handle_discover": 130, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays": 0, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays_delayed": 0, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays_enqueuing": 0, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_created": 16352, 2024-05-20T21:14:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_enqueued": 16352, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_reintegrated": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_migrated": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_processing": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_enqueued": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_prioritized": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "recovery_started": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "recovery_completed": 0, 2024-05-20T21:14:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_quiesce_path": 16, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_quiesce_inode": 2719, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_enqueue_scrub": 29, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_exportdir": 13, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_flush": 0, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_fragmentdir": 0, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_fragstats": 0, 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_inodestats": 0 2024-05-20T21:14:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_log": { 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "evlrg": 0, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "evadd": 100750, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "evex": 100705, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "evtrm": 100705, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "ev": 45, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "evexg": 0, 2024-05-20T21:14:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: "evexd": 1485, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "segadd": 177, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "segex": 176, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "segtrm": 176, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "seg": 1, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "segmjr": 1, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "segexg": 0, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "segexd": 2, 2024-05-20T21:14:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: "expos": 208718052, 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "wrpos": 208832652, 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "rdpos": 0, 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "jlat": { 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 29667, 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 85.799749266, 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.002892093 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.906 INFO:teuthology.orchestra.run.smithi055.stdout: "replayed": 0 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_mem": { 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: "ino": 541, 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: "ino+": 17074, 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: "ino-": 16533, 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: "dir": 108, 2024-05-20T21:14:08.907 INFO:teuthology.orchestra.run.smithi055.stdout: "dir+": 126, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "dir-": 18, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "dn": 802, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "dn+": 26645, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "dn-": 25843, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "cap": 513, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "cap+": 17035, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "cap-": 16522, 2024-05-20T21:14:08.908 INFO:teuthology.orchestra.run.smithi055.stdout: "rss": 79988, 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "heap": 231800 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_server": { 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "dispatch_client_request": 888207, 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "dispatch_server_request": 674, 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_request": 856333, 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_session": 1852, 2024-05-20T21:14:08.909 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_peer_request": 3680, 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "req_create_latency": { 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 16885, 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 219.458584217, 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.012997251 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getattr_latency": { 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 779800, 2024-05-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 788.220505702, 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001010798 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getfilelock_latency": { 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.911 INFO:teuthology.orchestra.run.smithi055.stdout: "req_link_latency": { 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookup_latency": { 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 7776, 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 30.879300992, 2024-05-20T21:14:08.912 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.003971103 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookuphash_latency": { 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupino_latency": { 2024-05-20T21:14:08.913 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupname_latency": { 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupparent_latency": { 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupsnap_latency": { 2024-05-20T21:14:08.915 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.916 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.916 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lssnap_latency": { 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mkdir_latency": { 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 107, 2024-05-20T21:14:08.917 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.345529049, 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.003229243 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mknod_latency": { 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.918 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mksnap_latency": { 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "req_open_latency": { 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 8196, 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 14.269906414, 2024-05-20T21:14:08.919 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001741081 2024-05-20T21:14:08.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.920 INFO:teuthology.orchestra.run.smithi055.stdout: "req_readdir_latency": { 2024-05-20T21:14:08.920 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 22422, 2024-05-20T21:14:08.922 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 93.207237829, 2024-05-20T21:14:08.922 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.004156954 2024-05-20T21:14:08.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.922 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rename_latency": { 2024-05-20T21:14:08.922 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3455, 2024-05-20T21:14:08.922 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 46.057318229, 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.013330627 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: "req_renamesnap_latency": { 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.923 INFO:teuthology.orchestra.run.smithi055.stdout: "req_snapdiff_latency": { 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmdir_latency": { 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 1, 2024-05-20T21:14:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.001259818, 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001259818 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmsnap_latency": { 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmxattr_latency": { 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setattr_latency": { 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 5, 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 75.502055291, 2024-05-20T21:14:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 15.100411058 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setdirlayout_latency": { 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setfilelock_latency": { 2024-05-20T21:14:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 516, 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.530588894, 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001028273 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setlayout_latency": { 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setxattr_latency": { 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 8, 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.020247449, 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.002530931 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "req_symlink_latency": { 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "req_unlink_latency": { 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 16352, 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 94.535998968, 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.005781311 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_revoke_eviction": 0, 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_acquisition_throttle": 0, 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getvxattr_latency": { 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_sessions": { 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "session_count": 1, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "session_add": 3, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "session_remove": 2, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions_open": 1, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions_stale": 0, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "total_load": 119502, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "average_load": 119502, 2024-05-20T21:14:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "avg_session_uptime": 1101, 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "mdthresh_evicted": 0 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 296247, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 109, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 7912, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 166, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:08.939 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 2092050, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 14342, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.940 INFO:teuthology.orchestra.run.smithi055.stdout: "objecter": { 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_active": 0, 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_laggy": 0, 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_send": 61891, 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_send_bytes": 217172440, 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_resend": 37, 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_reply": 61891, 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "op_latency": { 2024-05-20T21:14:08.941 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 61891, 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 295.365339230, 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.004772347 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "op_inflight": 0, 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "oplen_avg": { 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 61891, 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 79385 2024-05-20T21:14:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "op": 61891, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "op_r": 4266, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "op_w": 57625, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "op_rmw": 0, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "op_pg": 0, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_stat": 2459, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_create": 5096, 2024-05-20T21:14:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_read": 1426, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_write": 31064, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_writefull": 1051, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_writesame": 0, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_append": 0, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_zero": 0, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_truncate": 0, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_delete": 16960, 2024-05-20T21:14:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_mapext": 0, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_sparse_read": 0, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_clonerange": 0, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_getxattr": 2840, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_setxattr": 10192, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_cmpxattr": 0, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_rmxattr": 0, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_resetxattrs": 0, 2024-05-20T21:14:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_call": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_watch": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_notify": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_pgls": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_pgls_filter": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_other": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_active": 0, 2024-05-20T21:14:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_send": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_resend": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_ping": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_active": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_send": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_resend": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_active": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_send": 0, 2024-05-20T21:14:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_resend": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_active": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_send": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_resend": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "command_active": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "command_send": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "command_resend": 0, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "map_epoch": 84, 2024-05-20T21:14:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "map_full": 0, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "map_inc": 6, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_sessions": 12, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_session_open": 12, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_session_close": 0, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_laggy": 0, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_wr": 6552, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_rd": 14, 2024-05-20T21:14:08.949 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_del": 1731 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: "oft": { 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_objs": 1, 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_kv_pairs": 0, 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_updates": 0, 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_removes": 0 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.950 INFO:teuthology.orchestra.run.smithi055.stdout: "osd-slow-ops": { 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "slow_ops_count": 1924 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "purge_queue": { 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_ops": 0, 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_ops_high_water": 10, 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing": 0, 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_high_water": 10, 2024-05-20T21:14:08.951 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executed_ops": 16352, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executed": 16352, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_item_in_journal": 0 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 1025047, 2024-05-20T21:14:08.952 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 212147971, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 1025047, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 1025047, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 212147971, 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:08.953 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-objecter_bytes": { 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:08.954 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 61891, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 218823904, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 61891, 2024-05-20T21:14:08.955 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 218823904, 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-objecter_ops": { 2024-05-20T21:14:08.956 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 1024, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 61891, 2024-05-20T21:14:08.957 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 61891, 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 61891, 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 61891, 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-write_buf_throttle": { 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 3758096384, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 16352, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 1651552, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:08.959 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 16352, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 1397, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 1651552, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.960 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-write_buf_throttle-0x5627c4440500": { 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 3758096384, 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 100750, 2024-05-20T21:14:08.961 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 204638348, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 100750, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 29667, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 204638348, 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:08.962 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:08.963 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:08.963 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:08.963 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.963 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:08.963 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T21:14:09.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:09 smithi055 ceph-mon[35213]: pgmap v930: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-05-20T21:14:09.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:09 smithi026 ceph-mon[31990]: pgmap v930: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-05-20T21:14:09.714 INFO:tasks.check_counter:Daemon mds.b mds.exported=12 2024-05-20T21:14:09.714 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-05-20T21:14:09.714 DEBUG:tasks.check_counter:Getting stats from e 2024-05-20T21:14:09.715 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-20T21:14:09.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:09 smithi144 ceph-mon[35660]: pgmap v930: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-05-20T21:14:09.984 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T21:14:11.561 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:11 smithi055 ceph-mon[35213]: pgmap v931: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-20T21:14:11.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:11 smithi026 ceph-mon[31990]: pgmap v931: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-20T21:14:11.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:11 smithi144 ceph-mon[35660]: pgmap v931: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-20T21:14:12.409 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T21:14:12.409 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 3485, 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 3485, 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 26158409, 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 1107413, 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 7, 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 3, 2024-05-20T21:14:12.410 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.254259510, 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.074253190, 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.076485425, 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.083643945, 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3484, 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.093190942, 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000026748 2024-05-20T21:14:12.411 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T21:14:12.412 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 642, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 624, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 17767054, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 302324, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 5, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 4, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.080681996, 2024-05-20T21:14:12.413 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.025040854, 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.041048326, 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.014364763, 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 623, 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.022743743, 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000036506 2024-05-20T21:14:12.414 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 923696, 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 219264 2024-05-20T21:14:12.415 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 119, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 386, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 4429379, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 671046, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 7, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 4, 2024-05-20T21:14:12.416 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.025077028, 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.012856080, 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.005966626, 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.002896549, 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 384, 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.016348083, 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000042573 2024-05-20T21:14:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:14:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 633840 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 1, 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "finisher-MDSRank": { 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "queue_len": 0, 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "complete_latency": { 2024-05-20T21:14:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 6569, 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.145707023, 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000022181 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "finisher-PurgeQueue": { 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "queue_len": 0, 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "complete_latency": { 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": { 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "request": 0, 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "reply": 0, 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "reply_latency": { 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "slow_reply": 0, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "forward": 0, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_fetch_complete": 0, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_fetch_keys": 0, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_commit": 0, 2024-05-20T21:14:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_split": 0, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_merge": 0, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes": 4014, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_top": 0, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_bottom": 4014, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_pin_tail": 0, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_pinned": 4014, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_expired": 0, 2024-05-20T21:14:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_with_caps": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "subtrees": 10, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_hit": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_forward": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_discover": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_dir_fetch": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_remote_ino": 0, 2024-05-20T21:14:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_lock": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "load_cent": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "q": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "exported": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "exported_inodes": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "imported": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "imported_inodes": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_dir_fetch": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_backtrace_fetch": 0, 2024-05-20T21:14:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_peer_discover": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rfiles": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rbytes": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rsnaps": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_set_tag": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_inotable_repaired": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dir_inodes": 0, 2024-05-20T21:14:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_file_inodes": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_inode_file_caps": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_grant": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T21:14:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_caps": 0, 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_caps_dirty": 0, 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_cap_release": 0, 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "process_request_cap_release": 0 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_cache": { 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_update": 0, 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_update_receipt": 0, 2024-05-20T21:14:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_try_discover": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_send_discover": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_handle_discover": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays_delayed": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays_enqueuing": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_created": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_enqueued": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_reintegrated": 0, 2024-05-20T21:14:12.429 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_migrated": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_processing": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_enqueued": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_prioritized": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "recovery_started": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "recovery_completed": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_quiesce_path": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_quiesce_inode": 0, 2024-05-20T21:14:12.430 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_exportdir": 0, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_flush": 0, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_fragmentdir": 0, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_fragstats": 0, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_inodestats": 0 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_log": { 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "evlrg": 0, 2024-05-20T21:14:12.431 INFO:teuthology.orchestra.run.smithi055.stdout: "evadd": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "evex": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "evtrm": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "ev": 24053, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "evexg": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "evexd": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "segadd": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "segex": 0, 2024-05-20T21:14:12.432 INFO:teuthology.orchestra.run.smithi055.stdout: "segtrm": 0, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "seg": 35, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "segmjr": 3, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "segexg": 0, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "segexd": 0, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "expos": 4194304, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "wrpos": 50536607, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "rdpos": 50536033, 2024-05-20T21:14:12.433 INFO:teuthology.orchestra.run.smithi055.stdout: "jlat": { 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "replayed": 24053 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_mem": { 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "ino": 262, 2024-05-20T21:14:12.434 INFO:teuthology.orchestra.run.smithi055.stdout: "ino+": 3842, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "ino-": 3580, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "dir": 55, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "dir+": 55, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "dir-": 0, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "dn": 4014, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "dn+": 4018, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "dn-": 4, 2024-05-20T21:14:12.435 INFO:teuthology.orchestra.run.smithi055.stdout: "cap": 0, 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "cap+": 0, 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "cap-": 0, 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "rss": 56784, 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "heap": 215416 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_server": { 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "dispatch_client_request": 0, 2024-05-20T21:14:12.436 INFO:teuthology.orchestra.run.smithi055.stdout: "dispatch_server_request": 0, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_request": 0, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_session": 0, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_peer_request": 0, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "req_create_latency": { 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.437 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getattr_latency": { 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getfilelock_latency": { 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.438 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: "req_link_latency": { 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookup_latency": { 2024-05-20T21:14:12.439 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookuphash_latency": { 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.440 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupino_latency": { 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupname_latency": { 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.441 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupparent_latency": { 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupsnap_latency": { 2024-05-20T21:14:12.442 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lssnap_latency": { 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.444 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mkdir_latency": { 2024-05-20T21:14:12.444 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.444 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mknod_latency": { 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.445 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mksnap_latency": { 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "req_open_latency": { 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.446 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: "req_readdir_latency": { 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rename_latency": { 2024-05-20T21:14:12.447 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: "req_renamesnap_latency": { 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "req_snapdiff_latency": { 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmdir_latency": { 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.449 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmsnap_latency": { 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmxattr_latency": { 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setattr_latency": { 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.451 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setdirlayout_latency": { 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.452 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setfilelock_latency": { 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setlayout_latency": { 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.453 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setxattr_latency": { 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: "req_symlink_latency": { 2024-05-20T21:14:12.454 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "req_unlink_latency": { 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.455 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_revoke_eviction": 0, 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_acquisition_throttle": 0, 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getvxattr_latency": { 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.456 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_sessions": { 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "session_count": 1, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "session_add": 2, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "session_remove": 1, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions_open": 2, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions_stale": 0, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "total_load": 0, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "average_load": 0, 2024-05-20T21:14:12.457 INFO:teuthology.orchestra.run.smithi055.stdout: "avg_session_uptime": 3532, 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "mdthresh_evicted": 0 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:12.458 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:12.459 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:12.460 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:12.461 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 215191, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 66, 2024-05-20T21:14:12.462 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:12.463 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 7912, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 166, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 3446925, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 17338, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "objecter": { 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "op_active": 0, 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "op_laggy": 0, 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "op_send": 3859, 2024-05-20T21:14:12.465 INFO:teuthology.orchestra.run.smithi055.stdout: "op_send_bytes": 21, 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "op_resend": 3, 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "op_reply": 3859, 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "op_latency": { 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3859, 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 2.146958755, 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000556351 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: "op_inflight": 0, 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "oplen_avg": { 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3859, 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 3860 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "op": 3859, 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "op_r": 3859, 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "op_w": 0, 2024-05-20T21:14:12.467 INFO:teuthology.orchestra.run.smithi055.stdout: "op_rmw": 0, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "op_pg": 0, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_stat": 2369, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_create": 0, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_read": 1488, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_write": 0, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_writefull": 0, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_writesame": 0, 2024-05-20T21:14:12.468 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_append": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_zero": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_truncate": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_delete": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_mapext": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_sparse_read": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_clonerange": 0, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_getxattr": 1, 2024-05-20T21:14:12.469 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_setxattr": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_cmpxattr": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_rmxattr": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_resetxattrs": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_call": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_watch": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_notify": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T21:14:12.470 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_pgls": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_pgls_filter": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_other": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_active": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_send": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_resend": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_ping": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_active": 0, 2024-05-20T21:14:12.471 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_send": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_resend": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_active": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_send": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_resend": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_active": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_send": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_resend": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "command_active": 0, 2024-05-20T21:14:12.472 INFO:teuthology.orchestra.run.smithi055.stdout: "command_send": 0, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "command_resend": 0, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "map_epoch": 84, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "map_full": 0, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "map_inc": 6, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_sessions": 8, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_session_open": 9, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_session_close": 1, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_laggy": 0, 2024-05-20T21:14:12.473 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_wr": 0, 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_rd": 2, 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_del": 0 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "oft": { 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_objs": 0, 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_kv_pairs": 0, 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_updates": 0, 2024-05-20T21:14:12.474 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_removes": 0 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: "osd-slow-ops": { 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: "slow_ops_count": 0 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: "purge_queue": { 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_ops": 0, 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T21:14:12.475 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing": 0, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_high_water": 0, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executed_ops": 0, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executed": 0, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_item_in_journal": 0 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:12.476 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 4242, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 47950490, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 4242, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:12.477 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 4242, 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 47950490, 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-objecter_bytes": { 2024-05-20T21:14:12.478 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 3859, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 46342308, 2024-05-20T21:14:12.479 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 3859, 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 46342308, 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-objecter_ops": { 2024-05-20T21:14:12.480 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 1024, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 3859, 2024-05-20T21:14:12.481 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 3859, 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 3859, 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 3859, 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.482 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-write_buf_throttle": { 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 3758096384, 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:12.483 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.484 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-write_buf_throttle-0x55b5df8226e0": { 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 3758096384, 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:12.485 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:12.486 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:12.487 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:12.487 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.487 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:12.487 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T21:14:13.222 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-20T21:14:13.222 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-20T21:14:13.222 DEBUG:tasks.check_counter:Getting stats from h 2024-05-20T21:14:13.222 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-20T21:14:13.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:13 smithi055 ceph-mon[35213]: pgmap v932: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-20T21:14:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T21:14:13.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:13 smithi026 ceph-mon[31990]: pgmap v932: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-20T21:14:13.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:13 smithi144 ceph-mon[35660]: pgmap v932: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-20T21:14:15.417 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T21:14:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 179505, 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 199203, 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 53275893, 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 148108967, 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 70, 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 7, 2024-05-20T21:14:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 9.308331665, 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 3.802448709, 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 6.467854212, 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.147320731, 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 199199, 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 3.541695729, 2024-05-20T21:14:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000017779 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 362556, 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.062760478, 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000173 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-20T21:14:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 736 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 26744, 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 23575, 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 6640999, 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 19120086, 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 16, 2024-05-20T21:14:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 1.731485925, 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.786485465, 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 1.058588442, 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.091372284, 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 23576, 2024-05-20T21:14:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.532088058, 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000022569 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 43336, 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.007731079, 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000178 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 547424 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 4475, 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 4454, 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 2345678, 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 10779553, 2024-05-20T21:14:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 8, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.367744699, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.131116311, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.101492980, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.114572002, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 4453, 2024-05-20T21:14:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.211429007, 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000047480 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 939952, 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 255136 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 1, 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "finisher-MDSRank": { 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "queue_len": 0, 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "complete_latency": { 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 13489, 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 16.515901882, 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001224397 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "finisher-PurgeQueue": { 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "queue_len": 0, 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "complete_latency": { 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 5914, 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.780077946, 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000131903 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": { 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "request": 154622, 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "reply": 153692, 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "reply_latency": { 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 153692, 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 166.847567637, 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001085596 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "slow_reply": 0, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "forward": 930, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_fetch_complete": 0, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_fetch_keys": 0, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_commit": 11, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_split": 0, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_merge": 0, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes": 450, 2024-05-20T21:14:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_top": 0, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_bottom": 440, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_pin_tail": 10, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_pinned": 445, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_expired": 4, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "inodes_with_caps": 243, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": 243, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "subtrees": 11, 2024-05-20T21:14:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse": 295839, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_hit": 292193, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_forward": 329, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_discover": 19, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_dir_fetch": 0, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_remote_ino": 0, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "traverse_lock": 31, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "load_cent": 0, 2024-05-20T21:14:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "q": 0, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "exported": 2, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "exported_inodes": 4, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "imported": 8, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "imported_inodes": 126, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_dir_fetch": 0, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_backtrace_fetch": 211, 2024-05-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "openino_peer_discover": 0, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rfiles": 703, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rbytes": 224342856, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "root_rsnaps": 0, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_backtrace_fetch": 211, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_set_tag": 0, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_inotable_repaired": 0, 2024-05-20T21:14:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dir_inodes": 27, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_dirfrag_rstats": 27, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_file_inodes": 184, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_inode_file_caps": 0, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_revoke": 9852, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_grant": 13806, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T21:14:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_caps": 15882, 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_caps_dirty": 4722, 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_cap_release": 196, 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "process_request_cap_release": 12919 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_cache": { 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_update": 75, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_update_receipt": 26, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_try_discover": 18, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_send_discover": 32, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "dir_handle_discover": 79, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays": 0, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays_delayed": 0, 2024-05-20T21:14:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "num_strays_enqueuing": 0, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_created": 3576, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_enqueued": 3576, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_reintegrated": 0, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "strays_migrated": 0, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_processing": 0, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_enqueued": 0, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "num_recovering_prioritized": 0, 2024-05-20T21:14:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "recovery_started": 0, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "recovery_completed": 0, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_quiesce_path": 16, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_quiesce_inode": 894, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_exportdir": 2, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_flush": 0, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_fragmentdir": 0, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_fragstats": 0, 2024-05-20T21:14:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "ireq_inodestats": 0 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_log": { 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "evlrg": 0, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "evadd": 24053, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "evex": 0, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "evtrm": 0, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "ev": 24053, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "evexg": 0, 2024-05-20T21:14:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "evexd": 0, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "segadd": 35, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "segex": 0, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "segtrm": 0, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "seg": 35, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "segmjr": 3, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "segexg": 0, 2024-05-20T21:14:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "segexd": 0, 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "expos": 4194304, 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "wrpos": 50536607, 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "rdpos": 0, 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "jlat": { 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 6795, 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 16.820300409, 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.002475393 2024-05-20T21:14:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "replayed": 0 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_mem": { 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "ino": 274, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "ino+": 3854, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "ino-": 3580, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "dir": 67, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "dir+": 71, 2024-05-20T21:14:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "dir-": 4, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "dn": 450, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "dn+": 4534, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "dn-": 4084, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "cap": 243, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "cap+": 3823, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "cap-": 3580, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "rss": 64276, 2024-05-20T21:14:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "heap": 223608 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_server": { 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "dispatch_client_request": 162533, 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "dispatch_server_request": 1535, 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_request": 154622, 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_client_session": 364, 2024-05-20T21:14:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "handle_peer_request": 3541, 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "req_create_latency": { 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3684, 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 34.499866955, 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.009364784 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getattr_latency": { 2024-05-20T21:14:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 136525, 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 67.833712616, 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000496859 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getfilelock_latency": { 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "req_link_latency": { 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookup_latency": { 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 1909, 2024-05-20T21:14:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 5.568971377, 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.002917219 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookuphash_latency": { 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupino_latency": { 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupname_latency": { 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupparent_latency": { 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lookupsnap_latency": { 2024-05-20T21:14:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "req_lssnap_latency": { 2024-05-20T21:14:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mkdir_latency": { 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 8, 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.059267110, 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.007408388 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mknod_latency": { 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "req_mksnap_latency": { 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "req_open_latency": { 2024-05-20T21:14:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 2293, 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 2.420501105, 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.001055604 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "req_readdir_latency": { 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 4878, 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 18.249738653, 2024-05-20T21:14:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.003741233 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rename_latency": { 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 690, 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 10.154130539, 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.014716131 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "req_renamesnap_latency": { 2024-05-20T21:14:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "req_snapdiff_latency": { 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmdir_latency": { 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmsnap_latency": { 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "req_rmxattr_latency": { 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setattr_latency": { 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 1, 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 18.853189958, 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 18.853189958 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setdirlayout_latency": { 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setfilelock_latency": { 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 128, 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.083496716, 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000652318 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setlayout_latency": { 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "req_setxattr_latency": { 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "req_symlink_latency": { 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "req_unlink_latency": { 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3576, 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 9.124692608, 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.002551647 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_revoke_eviction": 0, 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_acquisition_throttle": 0, 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "req_getvxattr_latency": { 2024-05-20T21:14:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_sessions": { 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "session_count": 1, 2024-05-20T21:14:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "session_add": 2, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "session_remove": 1, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions_open": 1, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions_stale": 0, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "total_load": 56828, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "average_load": 56828, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "avg_session_uptime": 1108, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "mdthresh_evicted": 0 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-05-20T21:14:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 288727, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 105, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 7912, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 166, 2024-05-20T21:14:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 1101109, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 7511, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-05-20T21:14:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "objecter": { 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "op_active": 0, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "op_laggy": 0, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "op_send": 11814, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "op_send_bytes": 46975298, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "op_resend": 29, 2024-05-20T21:14:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "op_reply": 11814, 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "op_latency": { 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 11814, 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 30.012428074, 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.002540412 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "op_inflight": 0, 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "oplen_avg": { 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 11814, 2024-05-20T21:14:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 12236 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "op": 11814, 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "op_r": 593, 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "op_w": 11221, 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "op_rmw": 0, 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "op_pg": 0, 2024-05-20T21:14:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_stat": 0, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_create": 10, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_read": 382, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_write": 7177, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_writefull": 215, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_writesame": 0, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_append": 0, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_zero": 0, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_truncate": 0, 2024-05-20T21:14:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_delete": 3599, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_mapext": 0, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_sparse_read": 0, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_clonerange": 0, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_getxattr": 211, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_setxattr": 20, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_cmpxattr": 0, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_rmxattr": 0, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_resetxattrs": 0, 2024-05-20T21:14:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_call": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_watch": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_notify": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_pgls": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_pgls_filter": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "osdop_other": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_active": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_send": 0, 2024-05-20T21:14:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_resend": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "linger_ping": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_active": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_send": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "poolop_resend": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_active": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_send": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "poolstat_resend": 0, 2024-05-20T21:14:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_active": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_send": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "statfs_resend": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "command_active": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "command_send": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "command_resend": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "map_epoch": 84, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "map_full": 0, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "map_inc": 6, 2024-05-20T21:14:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_sessions": 12, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_session_open": 16, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_session_close": 4, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_laggy": 0, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_wr": 423, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_rd": 0, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_del": 199 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "oft": { 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_objs": 1, 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_kv_pairs": 0, 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_updates": 0, 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "omap_total_removes": 0 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "osd-slow-ops": { 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "slow_ops_count": 397 2024-05-20T21:14:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "purge_queue": { 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_ops": 0, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_ops_high_water": 10, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing": 0, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executing_high_water": 10, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executed_ops": 3576, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_executed": 3576, 2024-05-20T21:14:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "pq_item_in_journal": 0 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 210720, 2024-05-20T21:14:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 43273551, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 210720, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 210720, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 43273551, 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-objecter_bytes": { 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 11814, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 47336474, 2024-05-20T21:14:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 11814, 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 47336474, 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-objecter_ops": { 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 1024, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 0, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 0, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 11814, 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 11814, 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 11814, 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 11814, 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-write_buf_throttle": { 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 3758096384, 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 3576, 2024-05-20T21:14:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 361176, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 3576, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 382, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 361176, 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-write_buf_throttle-0x5606c0248500": { 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 3758096384, 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 24053, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 46342303, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 24053, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 6795, 2024-05-20T21:14:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 46342303, 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:15.498 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T21:14:15.499 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:15 smithi055 ceph-mon[35213]: pgmap v933: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-20T21:14:15.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:15 smithi026 ceph-mon[31990]: pgmap v933: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-20T21:14:15.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:15 smithi144 ceph-mon[35660]: pgmap v933: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-20T21:14:16.257 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-05-20T21:14:16.258 INFO:tasks.check_counter:Daemon mds.h mds.imported=8 2024-05-20T21:14:16.258 DEBUG:tasks.check_counter:Getting stats from k 2024-05-20T21:14:16.258 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-20T21:14:16.509 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T21:14:17.629 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:17 smithi055 ceph-mon[35213]: pgmap v934: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-20T21:14:17.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:17 smithi026 ceph-mon[31990]: pgmap v934: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-20T21:14:17.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:17 smithi144 ceph-mon[35660]: pgmap v934: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-20T21:14:18.218 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T21:14:18.218 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:18.218 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 0, 2024-05-20T21:14:18.218 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 2, 2024-05-20T21:14:18.218 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 0, 2024-05-20T21:14:18.218 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 0, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 0, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.001297088, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.000000000, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.000282825, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:18.219 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.220 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 365, 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 358, 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 856448, 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 220384, 2024-05-20T21:14:18.221 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 2, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 1, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.052355912, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.018612985, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.026575817, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.008302129, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 358, 2024-05-20T21:14:18.222 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.015403762, 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000043027 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 856448, 2024-05-20T21:14:18.223 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 220384 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 2, 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 272, 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 320, 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 205344, 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:18.224 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 2, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.013867731, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.009951967, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.001137312, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 270, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.013243716, 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000049050 2024-05-20T21:14:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:14:18.226 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 205344 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 0, 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 148704, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 29, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 928, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 2, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 363, 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 802439, 2024-05-20T21:14:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 363, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 363, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 802439, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-05-20T21:14:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-05-20T21:14:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-05-20T21:14:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T21:14:18.237 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T21:14:18.910 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-20T21:14:18.910 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-20T21:14:18.910 DEBUG:tasks.check_counter:Getting stats from c 2024-05-20T21:14:18.911 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-20T21:14:19.162 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T21:14:19.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:19 smithi144 ceph-mon[35660]: pgmap v935: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-20T21:14:19.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:19 smithi055 ceph-mon[35213]: pgmap v935: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-20T21:14:19.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:19 smithi026 ceph-mon[31990]: pgmap v935: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-20T21:14:20.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:20 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:14:20.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:20 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:14:20.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:20 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T21:14:20.994 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-20T21:14:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 852, 2024-05-20T21:14:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 844, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 49874311, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 320533, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.098681825, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.027214621, 2024-05-20T21:14:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.046327303, 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.018582618, 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 843, 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.029768546, 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000035312 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 247520, 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 111280 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1306, 2024-05-20T21:14:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1440, 2024-05-20T21:14:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 46524933, 2024-05-20T21:14:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 742723, 2024-05-20T21:14:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.157331593, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.040996803, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.063818178, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.040561287, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1439, 2024-05-20T21:14:20.999 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.056992148, 2024-05-20T21:14:21.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000039605 2024-05-20T21:14:21.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.000 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:21.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.000 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.001 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-20T21:14:21.001 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 328944 2024-05-20T21:14:21.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.002 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:21.002 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 561, 2024-05-20T21:14:21.003 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 563, 2024-05-20T21:14:21.003 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 46269274, 2024-05-20T21:14:21.003 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 178405, 2024-05-20T21:14:21.004 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-05-20T21:14:21.004 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-05-20T21:14:21.004 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.065703301, 2024-05-20T21:14:21.004 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.013032846, 2024-05-20T21:14:21.005 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.036478602, 2024-05-20T21:14:21.005 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.012657432, 2024-05-20T21:14:21.005 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:21.006 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 561, 2024-05-20T21:14:21.006 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.018624630, 2024-05-20T21:14:21.006 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033198 2024-05-20T21:14:21.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.006 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:21.006 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.007 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4287, 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.212883785, 2024-05-20T21:14:21.008 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000049657 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.009 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.010 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-05-20T21:14:21.011 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 12094, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 7890, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 4204, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 95, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-05-20T21:14:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 11, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-05-20T21:14:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-05-20T21:14:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 703, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 224342856, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T21:14:21.015 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-05-20T21:14:21.016 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-05-20T21:14:21.017 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-05-20T21:14:21.018 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-05-20T21:14:21.019 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-05-20T21:14:21.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-05-20T21:14:21.021 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 70331, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 3, 2024-05-20T21:14:21.022 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 49, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 208718052, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 208842171, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 208840224, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.023 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 70331 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 524, 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 11782, 2024-05-20T21:14:21.024 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 11258, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 97, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 120, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 23, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 12094, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 12253, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 159, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-05-20T21:14:21.025 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 61384, 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215392 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-05-20T21:14:21.026 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-05-20T21:14:21.027 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-05-20T21:14:21.027 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-05-20T21:14:21.027 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-05-20T21:14:21.027 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.028 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.028 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.028 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-05-20T21:14:21.029 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.029 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.029 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.029 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-05-20T21:14:21.030 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.030 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.030 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.031 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.032 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-05-20T21:14:21.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.032 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.036 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.037 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-05-20T21:14:21.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.037 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-05-20T21:14:21.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.040 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-05-20T21:14:21.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.042 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.042 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-05-20T21:14:21.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.042 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.043 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.045 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.047 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.049 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 0, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-05-20T21:14:21.050 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 2320, 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:21.051 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:21.052 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:21.053 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:21.054 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 231598, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 73, 2024-05-20T21:14:21.055 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:21.056 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7912, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 166, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 9738797, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 51068, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:21.057 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 2520, 2024-05-20T21:14:21.058 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 26, 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 4, 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 2520, 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2520, 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.802967836, 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000715463 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.059 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2520, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2521 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 2520, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 2520, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-05-20T21:14:21.060 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 1320, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1197, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-05-20T21:14:21.061 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 2, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-05-20T21:14:21.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-05-20T21:14:21.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-05-20T21:14:21.064 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-05-20T21:14:21.065 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 0, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 0, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 10, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 10, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-05-20T21:14:21.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.067 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-05-20T21:14:21.068 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:21.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 2715, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 142411220, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 2715, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2715, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 142411220, 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:21.070 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:21.071 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2520, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 141723167, 2024-05-20T21:14:21.072 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2520, 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 141723167, 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.073 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:21.074 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2520, 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 2520, 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2520, 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2520, 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:21.076 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5650078846e0": { 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:21.078 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:21.079 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:21.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:21.080 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-20T21:14:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:21 smithi144 ceph-mon[35660]: pgmap v936: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-20T21:14:21.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:14:21.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:14:21.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:21 smithi144 ceph-mon[35660]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:14:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:21 smithi055 ceph-mon[35213]: pgmap v936: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-20T21:14:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:14:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:14:21.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:21 smithi055 ceph-mon[35213]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:14:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:21 smithi026 ceph-mon[31990]: pgmap v936: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-20T21:14:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T21:14:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T21:14:21.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:21 smithi026 ceph-mon[31990]: from='mgr.14150 172.21.15.26:0/1099226187' entity='mgr.x' 2024-05-20T21:14:21.754 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-20T21:14:21.754 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-20T21:14:21.754 DEBUG:tasks.check_counter:Getting stats from f 2024-05-20T21:14:21.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-20T21:14:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T21:14:23.503 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:23 smithi144 ceph-mon[35660]: pgmap v937: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-20T21:14:23.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:23 smithi055 ceph-mon[35213]: pgmap v937: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-20T21:14:23.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:23 smithi026 ceph-mon[31990]: pgmap v937: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-20T21:14:23.775 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-20T21:14:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 501, 2024-05-20T21:14:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 483, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 2655990, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 258074, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.067311761, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.024452119, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.031362794, 2024-05-20T21:14:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.010673396, 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 482, 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.020843992, 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000043244 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 924240, 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 220544 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1281, 2024-05-20T21:14:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1550, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 8705286, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 957717, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.137030439, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.043323586, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.034522233, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.046054032, 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1548, 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.059376801, 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000038357 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 550672 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2569, 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2569, 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 9156007, 2024-05-20T21:14:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 816176, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.252598864, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.076251134, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.073797949, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.079935296, 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2568, 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.100815775, 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000039258 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-20T21:14:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 6551, 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.178137463, 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000027192 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-20T21:14:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-05-20T21:14:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-05-20T21:14:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 1594, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 1594, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 1594, 2024-05-20T21:14:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 7, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-05-20T21:14:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-05-20T21:14:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-05-20T21:14:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T21:14:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-05-20T21:14:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-05-20T21:14:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-05-20T21:14:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 10367, 2024-05-20T21:14:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 14, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-05-20T21:14:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 22672694, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 22672120, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 10367 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 102, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 1576, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 1474, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 34, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 34, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-05-20T21:14:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 1594, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 1632, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 38, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 55584, 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215416 2024-05-20T21:14:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-05-20T21:14:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-05-20T21:14:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-05-20T21:14:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-05-20T21:14:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-05-20T21:14:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 2324, 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 207128, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 63, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7912, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 166, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 1382661, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 6997, 2024-05-20T21:14:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-05-20T21:14:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 3963, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 3963, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3963, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2.122962319, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000535695 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3963, 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 3964 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 3963, 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 3963, 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-05-20T21:14:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 2464, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1497, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-05-20T21:14:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-05-20T21:14:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-05-20T21:14:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-05-20T21:14:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-05-20T21:14:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 84, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 6, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 7, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 8, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 1, 2024-05-20T21:14:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-05-20T21:14:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T21:14:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 4347, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 20103420, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 4347, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 4347, 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 20103420, 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3963, 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 18478395, 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3963, 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 18478395, 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3963, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 3963, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3963, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 3963, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5591799566e0": { 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-20T21:14:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-20T21:14:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:23.852 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-20T21:14:24.546 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-20T21:14:24.546 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-20T21:14:24.546 DEBUG:tasks.check_counter:Getting stats from i 2024-05-20T21:14:24.546 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-20T21:14:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T21:14:25.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:25 smithi144 ceph-mon[35660]: pgmap v938: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-20T21:14:25.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:25 smithi055 ceph-mon[35213]: pgmap v938: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-20T21:14:25.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:25 smithi026 ceph-mon[31990]: pgmap v938: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-20T21:14:26.533 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-20T21:14:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 83985, 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 92239, 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 24930357, 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 60265058, 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 17, 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 4.232865555, 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 1.680305607, 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 3.266578150, 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.043505371, 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 92238, 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.332647702, 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000014447 2024-05-20T21:14:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 171525, 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.022328860, 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000130 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 4544, 2024-05-20T21:14:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 1888 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 6598, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 7212, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1890803, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 11890509, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 68, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.408667220, 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.153358608, 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.198724732, 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.068521151, 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 7210, 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.167226502, 2024-05-20T21:14:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000023193 2024-05-20T21:14:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 7072, 2024-05-20T21:14:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000938444, 2024-05-20T21:14:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000132 2024-05-20T21:14:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 638752 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 23977, 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 16138, 2024-05-20T21:14:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 6843644, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 8266339, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 9, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 1.333156848, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.510607565, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 1.055567369, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.053500706, 2024-05-20T21:14:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 16137, 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.310304821, 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000019229 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 35877, 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.005520990, 2024-05-20T21:14:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000153 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 936096, 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 225728 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-05-20T21:14:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 7755, 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 4.004277518, 2024-05-20T21:14:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000516347 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3143, 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.142427716, 2024-05-20T21:14:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000045315 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 76031, 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 75224, 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-05-20T21:14:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 75224, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 39.374514557, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000523430 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 807, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-05-20T21:14:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 160, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 5, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 135, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 20, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 152, 2024-05-20T21:14:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 45, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 80, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 80, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 8, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 147911, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 145220, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 273, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 18, 2024-05-20T21:14:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 1, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 38, 2024-05-20T21:14:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 4, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 35, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 84, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 703, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 224342856, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-05-20T21:14:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 84, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 10, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 10, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 74, 2024-05-20T21:14:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 4104, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 5408, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 6009, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 1611, 2024-05-20T21:14:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 154, 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 5571 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 1, 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 178, 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 95, 2024-05-20T21:14:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 179, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 7, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 1450, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 1450, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-05-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 16, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 419, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T21:14:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 2, 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-05-20T21:14:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 10367, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 10367, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 14, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-05-20T21:14:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 14, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 22672694, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-05-20T21:14:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3780, 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 9.369794611, 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002478781 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-05-20T21:14:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 118, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 1599, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 1481, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 48, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 130, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 82, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 160, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 1896, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 1736, 2024-05-20T21:14:26.561 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 80, 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 1555, 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 1475, 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 60192, 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223608 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-05-20T21:14:26.562 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 80339, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 1559, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 76031, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 133, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 4079, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1512, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2.764836027, 2024-05-20T21:14:26.563 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001828595 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 68088, 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 16.164902398, 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000237411 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-05-20T21:14:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 886, 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2.085273366, 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002353581 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-05-20T21:14:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-05-20T21:14:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-05-20T21:14:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4, 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.007931300, 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001982825 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 841, 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.524401972, 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000623545 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2007, 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 5.409669949, 2024-05-20T21:14:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002695401 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 384, 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.120614269, 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002918266 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-05-20T21:14:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-05-20T21:14:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-05-20T21:14:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-05-20T21:14:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 52, 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.013880244, 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000266927 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-05-20T21:14:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1450, 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 11.283005032, 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.007781382 2024-05-20T21:14:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 27887, 2024-05-20T21:14:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 27887, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 1121, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 287431, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 101, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7912, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 166, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-20T21:14:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 466085, 2024-05-20T21:14:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 2799, 2024-05-20T21:14:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-20T21:14:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-20T21:14:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 6307, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 18740694, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 28, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 6307, 2024-05-20T21:14:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 6307, 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 15.563722795, 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002467690 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 6307, 2024-05-20T21:14:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 6659 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 6307, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 376, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 5931, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-05-20T21:14:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 292, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 4072, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 175, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 1466, 2024-05-20T21:14:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 84, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-05-20T21:14:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-05-20T21:14:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-05-20T21:14:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-05-20T21:14:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-05-20T21:14:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-05-20T21:14:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-05-20T21:14:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-05-20T21:14:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-05-20T21:14:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-05-20T21:14:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-05-20T21:14:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-05-20T21:14:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-05-20T21:14:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-05-20T21:14:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-05-20T21:14:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-05-20T21:14:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-05-20T21:14:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 84, 2024-05-20T21:14:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-05-20T21:14:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 6, 2024-05-20T21:14:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 12, 2024-05-20T21:14:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 16, 2024-05-20T21:14:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 4, 2024-05-20T21:14:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-05-20T21:14:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 373, 2024-05-20T21:14:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-05-20T21:14:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 167 2024-05-20T21:14:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-05-20T21:14:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-05-20T21:14:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-05-20T21:14:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-05-20T21:14:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-05-20T21:14:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-05-20T21:14:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 228 2024-05-20T21:14:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-05-20T21:14:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-05-20T21:14:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 10, 2024-05-20T21:14:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 10, 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 1450, 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 1450, 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 114556, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 23330497, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 114556, 2024-05-20T21:14:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 114556, 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 23330497, 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 6307, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 18887144, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 6307, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 18887144, 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-05-20T21:14:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 6307, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 6307, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 6307, 2024-05-20T21:14:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 6307, 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 1450, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 146450, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 1450, 2024-05-20T21:14:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 292, 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 146450, 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x561ba960c500": { 2024-05-20T21:14:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-20T21:14:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 10367, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 18478390, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 10367, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3780, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 18478390, 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:26.620 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-20T21:14:27.384 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-05-20T21:14:27.385 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-05-20T21:14:27.385 DEBUG:tasks.check_counter:Getting stats from l 2024-05-20T21:14:27.385 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-20T21:14:27.625 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T21:14:27.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:27 smithi055 ceph-mon[35213]: pgmap v939: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-20T21:14:27.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:27 smithi026 ceph-mon[31990]: pgmap v939: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-20T21:14:27.747 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:27 smithi144 ceph-mon[35660]: pgmap v939: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-20T21:14:29.320 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-20T21:14:29.320 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T21:14:29.320 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 369, 2024-05-20T21:14:29.320 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 362, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 857536, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 222944, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 0, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.055562074, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.021553498, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.024830629, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.008060708, 2024-05-20T21:14:29.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 362, 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.015872370, 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000043846 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:29.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 857536, 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 222944 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2, 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 275, 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 320, 2024-05-20T21:14:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 205872, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.016751444, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.012548054, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.001083415, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:29.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 273, 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.017547192, 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000064275 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:29.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 205872 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 0, 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2, 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 0, 2024-05-20T21:14:29.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 0, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 2, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.000274950, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.000000000, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.000194225, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:29.327 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.328 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 0, 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-20T21:14:29.329 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T21:14:29.330 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-20T21:14:29.331 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T21:14:29.332 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T21:14:29.333 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 148704, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 29, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-20T21:14:29.334 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 928, 2024-05-20T21:14:29.335 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 2, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-20T21:14:29.336 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 367, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 802959, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 367, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 367, 2024-05-20T21:14:29.337 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 802959, 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-20T21:14:29.338 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-20T21:14:29.679 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:29 smithi144 ceph-mon[35660]: pgmap v940: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-20T21:14:29.700 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:29 smithi055 ceph-mon[35213]: pgmap v940: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-20T21:14:29.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:29 smithi026 ceph-mon[31990]: pgmap v940: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-20T21:14:30.044 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-20T21:14:30.045 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-20T21:14:30.045 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-20T21:14:30.057 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-20T21:14:30.057 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T21:14:30.057 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T21:14:30.079 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T21:14:30.079 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T21:14:30.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-20T21:14:30.441 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:14:31.679 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:31 smithi026 ceph-mon[31990]: pgmap v941: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-20T21:14:31.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:31 smithi055 ceph-mon[35213]: pgmap v941: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-20T21:14:31.734 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:31 smithi144 ceph-mon[35660]: pgmap v941: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-20T21:14:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 -- 172.21.15.26:0/2178125994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 msgr2=0x7f995406c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 --2- 172.21.15.26:0/2178125994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995406c2f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f99500099c0 tx=0x7f9950009cd0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 -- 172.21.15.26:0/2178125994 shutdown_connections 2024-05-20T21:14:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 --2- 172.21.15.26:0/2178125994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995406c2f0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 --2- 172.21.15.26:0/2178125994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9954108860 0x7f9954110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 --2- 172.21.15.26:0/2178125994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f995406c8c0 0x7f9954108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.236+0000 7f995bbea700 1 -- 172.21.15.26:0/2178125994 >> 172.21.15.26:0/2178125994 conn(0x7f9954078490 msgr2=0x7f99540788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 -- 172.21.15.26:0/2178125994 shutdown_connections 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 -- 172.21.15.26:0/2178125994 wait complete. 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 Processor -- start 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 -- start start 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995406c8c0 0x7f995419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954108860 0x7f9954199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99541136e0 con 0x7f995406bf10 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9954113560 con 0x7f9954108860 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f995bbea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9954113860 con 0x7f995406c8c0 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.237+0000 7f9959986700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56168/0 (socket says 172.21.15.26:56168) 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 -- 172.21.15.26:0/2027471699 learned_addr learned my addr 172.21.15.26:0/2027471699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959185700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995406c8c0 0x7f995419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:32.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 -- 172.21.15.26:0/2027471699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995406c8c0 msgr2=0x7f995419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995406c8c0 0x7f995419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 -- 172.21.15.26:0/2027471699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954108860 msgr2=0x7f9954199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954108860 0x7f9954199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 -- 172.21.15.26:0/2027471699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995419a1c0 con 0x7f995406bf10 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959185700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995406c8c0 0x7f995419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f9959986700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995419f8a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9950000c00 tx=0x7f995000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950026070 con 0x7f995406bf10 2024-05-20T21:14:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99500054c0 con 0x7f995406bf10 2024-05-20T21:14:32.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995419a450 con 0x7f995406bf10 2024-05-20T21:14:32.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950004c20 con 0x7f995406bf10 2024-05-20T21:14:32.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.238+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99541a6770 con 0x7f995406bf10 2024-05-20T21:14:32.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.239+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995406a230 con 0x7f995406bf10 2024-05-20T21:14:32.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.241+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9950029070 con 0x7f995406bf10 2024-05-20T21:14:32.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.242+0000 7f994affd700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99400b2880 0x7f99400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:32.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.242+0000 7f9959185700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99400b2880 0x7f99400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:32.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.242+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7f99500e32c0 con 0x7f995406bf10 2024-05-20T21:14:32.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.243+0000 7f9959185700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99400b2880 0x7f99400b4d40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f99541020a0 tx=0x7f9944008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:14:32.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.246+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9950030020 con 0x7f995406bf10 2024-05-20T21:14:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.480+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f995402e560 con 0x7f995406bf10 2024-05-20T21:14:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.481+0000 7f994affd700 1 -- 172.21.15.26:0/2027471699 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+591 (secure 0 0 0) 0x7f9950013070 con 0x7f995406bf10 2024-05-20T21:14:32.481 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3367150269 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:32.481 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2532144649 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:32.481 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1750454874 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3909957816 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/1825381346 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1784334339 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/767936803 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2532144649 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2879790538 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:32.482 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/1825381346 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99400b2880 msgr2=0x7f99400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99400b2880 0x7f99400b4d40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f99541020a0 tx=0x7f9944008040 comp rx=0 tx=0).stop 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 msgr2=0x7f995419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995419f8a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9950000c00 tx=0x7f995000f740 comp rx=0 tx=0).stop 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 shutdown_connections 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7f99400b2880 0x7f99400b4d40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f995406bf10 0x7f995419f8a0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954108860 0x7f9954199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 --2- 172.21.15.26:0/2027471699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995406c8c0 0x7f995419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.483+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 >> 172.21.15.26:0/2027471699 conn(0x7f9954078490 msgr2=0x7f9954109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:14:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.484+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 shutdown_connections 2024-05-20T21:14:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:32.484+0000 7f995bbea700 1 -- 172.21.15.26:0/2027471699 wait complete. 2024-05-20T21:14:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2024-05-20T21:14:33.227 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-20T21:14:33.227 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T21:14:33.245 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634 shell --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-20T21:14:33.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:33 smithi026 ceph-mon[31990]: pgmap v942: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-20T21:14:33.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:33 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/2027471699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T21:14:33.539 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T21:14:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:33 smithi055 ceph-mon[35213]: pgmap v942: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-20T21:14:33.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:33 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/2027471699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T21:14:33.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:33 smithi144 ceph-mon[35660]: pgmap v942: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-20T21:14:33.801 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:33 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/2027471699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T21:14:35.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 -- 172.21.15.26:0/1866136295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 msgr2=0x7fb0ac06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:35.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/1866136295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac06c0f0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009990 tx=0x7fb0a0009ca0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 -- 172.21.15.26:0/1866136295 shutdown_connections 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/1866136295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac06c630 0x7fb0ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/1866136295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0ac1095b0 0x7fb0ac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/1866136295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac06c0f0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 -- 172.21.15.26:0/1866136295 >> 172.21.15.26:0/1866136295 conn(0x7fb0ac0793f0 msgr2=0x7fb0ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 -- 172.21.15.26:0/1866136295 shutdown_connections 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 -- 172.21.15.26:0/1866136295 wait complete. 2024-05-20T21:14:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.347+0000 7fb0b18b0700 1 Processor -- start 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 -- start start 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac06c630 0x7fb0ac19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0ac1095b0 0x7fb0ac199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1135e0 con 0x7fb0ac06bc70 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac113460 con 0x7fb0ac06c630 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0b18b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac113760 con 0x7fb0ac1095b0 2024-05-20T21:14:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac19f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56176/0 (socket says 172.21.15.26:56176) 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 -- 172.21.15.26:0/308321619 learned_addr learned my addr 172.21.15.26:0/308321619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aa7fc700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac06c630 0x7fb0ac19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0ab7fe700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0ac1095b0 0x7fb0ac199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 -- 172.21.15.26:0/308321619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0ac1095b0 msgr2=0x7fb0ac199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0ac1095b0 0x7fb0ac199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 -- 172.21.15.26:0/308321619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac06c630 msgr2=0x7fb0ac19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.348+0000 7fb0aaffd700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac06c630 0x7fb0ac19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0aaffd700 1 -- 172.21.15.26:0/308321619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac19a1b0 con 0x7fb0ac06bc70 2024-05-20T21:14:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0aaffd700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac19f890 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d980 tx=0x7fb09c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:14:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c017070 con 0x7fb0ac06bc70 2024-05-20T21:14:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb09c005020 con 0x7fb0ac06bc70 2024-05-20T21:14:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac19a4a0 con 0x7fb0ac06bc70 2024-05-20T21:14:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c009ab0 con 0x7fb0ac06bc70 2024-05-20T21:14:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.349+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac1a6760 con 0x7fb0ac06bc70 2024-05-20T21:14:35.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.350+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac06a230 con 0x7fb0ac06bc70 2024-05-20T21:14:35.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.352+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb09c0051d0 con 0x7fb0ac06bc70 2024-05-20T21:14:35.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.353+0000 7fb0b08ae700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0940b2880 0x7fb0940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T21:14:35.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.353+0000 7fb0aa7fc700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0940b2880 0x7fb0940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T21:14:35.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.353+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8327+0+0 (secure 0 0 0) 0x7fb09c0e2540 con 0x7fb0ac06bc70 2024-05-20T21:14:35.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.353+0000 7fb0aa7fc700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0940b2880 0x7fb0940b4d40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0003760 tx=0x7fb0a0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T21:14:35.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.356+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb09c020020 con 0x7fb0ac06bc70 2024-05-20T21:14:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.590+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb0ac02e560 con 0x7fb0ac06bc70 2024-05-20T21:14:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.590+0000 7fb0b08ae700 1 -- 172.21.15.26:0/308321619 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+591 (secure 0 0 0) 0x7fb09c0aa200 con 0x7fb0ac06bc70 2024-05-20T21:14:35.590 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3367150269 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:35.590 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2532144649 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:35.590 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1750454874 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:35.590 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3909957816 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:35.591 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/1825381346 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:35.591 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1784334339 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:35.591 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/767936803 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:35.591 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2532144649 2024-05-21T20:45:35.597719+0000 2024-05-20T21:14:35.591 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2879790538 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:35.591 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/1825381346 2024-05-21T20:45:08.150176+0000 2024-05-20T21:14:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0940b2880 msgr2=0x7fb0940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0940b2880 0x7fb0940b4d40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0003760 tx=0x7fb0a0005fd0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 msgr2=0x7fb0ac19f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T21:14:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac19f890 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d980 tx=0x7fb09c00dc90 comp rx=0 tx=0).stop 2024-05-20T21:14:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 shutdown_connections 2024-05-20T21:14:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:6800/2583575856,v1:172.21.15.26:6801/2583575856] conn(0x7fb0940b2880 0x7fb0940b4d40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0ac06bc70 0x7fb0ac19f890 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0ac1095b0 0x7fb0ac199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 --2- 172.21.15.26:0/308321619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac06c630 0x7fb0ac19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T21:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 >> 172.21.15.26:0/308321619 conn(0x7fb0ac0793f0 msgr2=0x7fb0ac101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T21:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 shutdown_connections 2024-05-20T21:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-20T21:14:35.593+0000 7fb0b18b0700 1 -- 172.21.15.26:0/308321619 wait complete. 2024-05-20T21:14:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2024-05-20T21:14:35.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:35 smithi055 ceph-mon[35213]: pgmap v943: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-20T21:14:35.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:35 smithi026 ceph-mon[31990]: pgmap v943: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-20T21:14:35.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:35 smithi144 ceph-mon[35660]: pgmap v943: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-20T21:14:36.374 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-05-20T21:14:36.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T21:14:36.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-20T21:14:36.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[88380]: fuse finished with error 0 and tester_r 0 2024-05-20T21:14:36.420 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T21:14:36.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:36 smithi055 ceph-mon[35213]: from='client.? 172.21.15.26:0/308321619' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T21:14:36.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:36 smithi026 ceph-mon[31990]: from='client.? 172.21.15.26:0/308321619' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T21:14:36.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:36 smithi144 ceph-mon[35660]: from='client.? 172.21.15.26:0/308321619' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T21:14:37.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:37 smithi055 ceph-mon[35213]: pgmap v944: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-20T21:14:37.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:37 smithi026 ceph-mon[31990]: pgmap v944: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-20T21:14:37.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:37 smithi144 ceph-mon[35660]: pgmap v944: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-20T21:14:39.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:39 smithi055 ceph-mon[35213]: pgmap v945: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-20T21:14:39.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:39 smithi026 ceph-mon[31990]: pgmap v945: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-20T21:14:39.802 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:39 smithi144 ceph-mon[35660]: pgmap v945: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-20T21:14:41.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:41 smithi055 ceph-mon[35213]: pgmap v946: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-20T21:14:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:41 smithi026 ceph-mon[31990]: pgmap v946: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-20T21:14:41.800 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:41 smithi144 ceph-mon[35660]: pgmap v946: 97 pgs: 97 active+clean; 282 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-20T21:14:42.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T21:14:42.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T21:14:42.523 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T21:14:42.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T21:14:42.550 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-05-20T21:14:42.609 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T21:14:42.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T21:14:42.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T21:14:42.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T21:14:42.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-05-20T21:14:42.783 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-20T21:14:42.799 INFO:tasks.cephadm:Teardown begin 2024-05-20T21:14:42.799 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-20T21:14:42.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T21:14:42.831 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T21:14:42.873 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T21:14:42.907 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-20T21:14:42.907 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-20T21:14:42.924 DEBUG:teuthology.orchestra.run.smithi055:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-20T21:14:42.943 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-20T21:14:42.965 INFO:tasks.cephadm:Stopping all daemons... 2024-05-20T21:14:42.966 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-20T21:14:42.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a 2024-05-20T21:14:43.282 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:43 smithi026 systemd[1]: Stopping Ceph mon.a for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:43.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:43 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a[31986]: 2024-05-20T21:14:43.282+0000 7f0a2f4c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:14:43.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:43 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a[31986]: 2024-05-20T21:14:43.282+0000 7f0a2f4c0700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-20T21:14:43.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:43 smithi026 podman[142653]: 2024-05-20 21:14:43.333693832 +0000 UTC m=+0.171242163 container died 1dab537c29f14448c9792f303cfb52938d4dedd19b6662773eeb19f02f193ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-20T21:14:44.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:43 smithi026 podman[142653]: 2024-05-20 21:14:43.925342034 +0000 UTC m=+0.762890348 container cleanup 1dab537c29f14448c9792f303cfb52938d4dedd19b6662773eeb19f02f193ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-20T21:14:44.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:May 20 21:14:43 smithi026 bash[142653]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-a 2024-05-20T21:14:44.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-20T21:14:44.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:44.407 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-20T21:14:44.407 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-20T21:14:44.407 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.b 2024-05-20T21:14:44.701 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:44 smithi055 systemd[1]: Stopping Ceph mon.b for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:45.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-b[35209]: 2024-05-20T21:14:44.793+0000 7f8a8fc2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:14:45.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-b[35209]: 2024-05-20T21:14:44.793+0000 7f8a8fc2e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-20T21:14:45.201 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:44 smithi055 podman[71435]: 2024-05-20 21:14:44.998322962 +0000 UTC m=+0.313746732 container died b34a9406fbc196c32b11a3f9ecd2f6be31538234c88e3a01cd45e66b26614e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-20T21:14:45.586 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:45 smithi055 podman[71435]: 2024-05-20 21:14:45.343811533 +0000 UTC m=+0.659235297 container cleanup b34a9406fbc196c32b11a3f9ecd2f6be31538234c88e3a01cd45e66b26614e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-20T21:14:45.587 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:45 smithi055 bash[71435]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-b 2024-05-20T21:14:45.819 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-20T21:14:45.842 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:45 smithi055 podman[71450]: 2024-05-20 21:14:45.587276371 +0000 UTC m=+0.584196897 container remove b34a9406fbc196c32b11a3f9ecd2f6be31538234c88e3a01cd45e66b26614e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:14:45.842 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:45 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-20T21:14:45.843 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:45 smithi055 systemd[1]: Stopped Ceph mon.b for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T21:14:45.843 INFO:journalctl@ceph.mon.b.smithi055.stdout:May 20 21:14:45 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.b.service: Consumed 15.753s CPU time. 2024-05-20T21:14:45.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:45.881 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-20T21:14:45.882 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-20T21:14:45.882 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.c 2024-05-20T21:14:46.200 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:45 smithi144 systemd[1]: Stopping Ceph mon.c for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-c[35656]: 2024-05-20T21:14:46.200+0000 7f9776c4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:14:46.550 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-c[35656]: 2024-05-20T21:14:46.200+0000 7f9776c4b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-20T21:14:46.551 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:46 smithi144 podman[72772]: 2024-05-20 21:14:46.44550809 +0000 UTC m=+0.366549992 container died 29b7074c07fe226242a44c8dbc1d5343ce89cc51599f5563276361d06bb7abd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:14:47.002 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:46 smithi144 podman[72772]: 2024-05-20 21:14:46.826299026 +0000 UTC m=+0.747340970 container cleanup 29b7074c07fe226242a44c8dbc1d5343ce89cc51599f5563276361d06bb7abd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-20T21:14:47.002 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:46 smithi144 bash[72772]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-c 2024-05-20T21:14:47.252 INFO:journalctl@ceph.mon.c.smithi144.stdout:May 20 21:14:47 smithi144 podman[72787]: 2024-05-20 21:14:47.003137703 +0000 UTC m=+0.553569015 container remove 29b7074c07fe226242a44c8dbc1d5343ce89cc51599f5563276361d06bb7abd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mon-c, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:14:47.257 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-20T21:14:47.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:47.308 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-20T21:14:47.309 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-20T21:14:47.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x 2024-05-20T21:14:47.638 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 21:14:47 smithi026 systemd[1]: Stopping Ceph mgr.x for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:47.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 21:14:47 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T21:14:47.638+0000 7f6025e12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:14:47.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 21:14:47 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x[32220]: 2024-05-20T21:14:47.638+0000 7f6025e12700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-20T21:14:47.952 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 21:14:47 smithi026 podman[142778]: 2024-05-20 21:14:47.669695564 +0000 UTC m=+0.157275467 container died 42c0ddaaf231c8bc23c510eb995f5bc23414f72c677e3264a77c2c8a318bdd89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-20T21:14:48.424 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 21:14:48 smithi026 podman[142778]: 2024-05-20 21:14:48.173653633 +0000 UTC m=+0.661233549 container cleanup 42c0ddaaf231c8bc23c510eb995f5bc23414f72c677e3264a77c2c8a318bdd89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ebd1d5361d457738957dc0455bba90102296634, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T21:14:48.425 INFO:journalctl@ceph.mgr.x.smithi026.stdout:May 20 21:14:48 smithi026 bash[142778]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-x 2024-05-20T21:14:48.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-20T21:14:48.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:48.593 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-20T21:14:48.593 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-20T21:14:48.593 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.y 2024-05-20T21:14:48.935 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:48 smithi055 systemd[1]: Stopping Ceph mgr.y for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:49.201 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:48 smithi055 podman[71559]: 2024-05-20 21:14:48.936603276 +0000 UTC m=+0.144209860 container died b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-20T21:14:49.575 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 podman[71559]: 2024-05-20 21:14:49.34188077 +0000 UTC m=+0.549487340 container cleanup b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-20T21:14:49.576 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 bash[71559]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y 2024-05-20T21:14:49.810 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-20T21:14:49.833 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 podman[71571]: 2024-05-20 21:14:49.576857857 +0000 UTC m=+0.630311467 container remove b53f9546ce24b7891ac9ff07cdcb1d7f3d80ec60f74592172f487282f177d689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-20T21:14:49.833 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-20T21:14:49.833 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-05-20T21:14:49.833 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 systemd[1]: Stopped Ceph mgr.y for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T21:14:49.833 INFO:journalctl@ceph.mgr.y.smithi055.stdout:May 20 21:14:49 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.y.service: Consumed 15.212s CPU time. 2024-05-20T21:14:49.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:49.847 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-20T21:14:49.847 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-20T21:14:49.847 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.z 2024-05-20T21:14:50.193 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 21:14:49 smithi144 systemd[1]: Stopping Ceph mgr.z for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:50.466 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 21:14:50 smithi144 podman[72896]: 2024-05-20 21:14:50.194279182 +0000 UTC m=+0.144067478 container died a2493a2d531cc32cddded5225c796c7e9143fb27996eb8574de6c838a2de9ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-20T21:14:50.800 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 21:14:50 smithi144 podman[72896]: 2024-05-20 21:14:50.557349457 +0000 UTC m=+0.507137733 container cleanup a2493a2d531cc32cddded5225c796c7e9143fb27996eb8574de6c838a2de9ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-20T21:14:50.800 INFO:journalctl@ceph.mgr.z.smithi144.stdout:May 20 21:14:50 smithi144 bash[72896]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mgr-z 2024-05-20T21:14:50.978 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-20T21:14:51.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:51.012 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-20T21:14:51.012 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-20T21:14:51.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.0 2024-05-20T21:14:51.337 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:51 smithi026 systemd[1]: Stopping Ceph osd.0 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:14:51.702 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0[46809]: 2024-05-20T21:14:51.338+0000 7f16b9fa7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:14:51.702 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0[46809]: 2024-05-20T21:14:51.338+0000 7f16b9fa7700 -1 osd.0 84 *** Got signal Terminated *** 2024-05-20T21:14:51.702 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0[46809]: 2024-05-20T21:14:51.338+0000 7f16b9fa7700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:14:57.197 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:56 smithi026 podman[142899]: 2024-05-20 21:14:56.880742583 +0000 UTC m=+5.667237326 container died 0eea9d19d91430f758759b1a1a1514e38095f09de41b78476b6d8f681b40a252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-20T21:14:57.452 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:57 smithi026 podman[142899]: 2024-05-20 21:14:57.388437467 +0000 UTC m=+6.174932200 container cleanup 0eea9d19d91430f758759b1a1a1514e38095f09de41b78476b6d8f681b40a252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-20T21:14:57.452 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:57 smithi026 bash[142899]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0 2024-05-20T21:14:57.952 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:57 smithi026 podman[142916]: 2024-05-20 21:14:57.623548174 +0000 UTC m=+0.740217559 container remove 0eea9d19d91430f758759b1a1a1514e38095f09de41b78476b6d8f681b40a252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-20T21:14:58.360 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:58 smithi026 podman[143007]: 2024-05-20 21:14:57.981810052 +0000 UTC m=+0.017674253 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:14:58.360 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:58 smithi026 podman[143007]: 2024-05-20 21:14:58.099859234 +0000 UTC m=+0.135723427 container create 24e43a4dd4fd08c9f72ed9831b9609b797b2f15204958ea55c01653d256f5731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:14:58.675 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:58 smithi026 podman[143007]: 2024-05-20 21:14:58.361410138 +0000 UTC m=+0.397274339 container init 24e43a4dd4fd08c9f72ed9831b9609b797b2f15204958ea55c01653d256f5731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-20T21:14:58.676 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:58 smithi026 podman[143007]: 2024-05-20 21:14:58.36551494 +0000 UTC m=+0.401379139 container start 24e43a4dd4fd08c9f72ed9831b9609b797b2f15204958ea55c01653d256f5731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-20T21:14:58.676 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:58 smithi026 podman[143007]: 2024-05-20 21:14:58.461207599 +0000 UTC m=+0.497071814 container attach 24e43a4dd4fd08c9f72ed9831b9609b797b2f15204958ea55c01653d256f5731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-20T21:14:58.952 INFO:journalctl@ceph.osd.0.smithi026.stdout:May 20 21:14:58 smithi026 podman[143007]: 2024-05-20 21:14:58.676962141 +0000 UTC m=+0.712826345 container died 24e43a4dd4fd08c9f72ed9831b9609b797b2f15204958ea55c01653d256f5731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-20T21:14:59.385 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-20T21:14:59.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:14:59.433 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-20T21:14:59.433 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-20T21:14:59.433 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.1 2024-05-20T21:14:59.872 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:14:59 smithi055 systemd[1]: Stopping Ceph osd.1 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:00.201 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:14:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1[39775]: 2024-05-20T21:14:59.871+0000 7f29581f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:00.201 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:14:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1[39775]: 2024-05-20T21:14:59.871+0000 7f29581f2700 -1 osd.1 84 *** Got signal Terminated *** 2024-05-20T21:15:00.202 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:14:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1[39775]: 2024-05-20T21:14:59.871+0000 7f29581f2700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:05.689 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:05 smithi055 podman[71699]: 2024-05-20 21:15:05.379688148 +0000 UTC m=+5.736273162 container died 85832347393d6c281c9c85389ce50109718bbcbf5e7b2d0c7cc287534d5e043f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-20T21:15:05.951 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:05 smithi055 podman[71699]: 2024-05-20 21:15:05.781306097 +0000 UTC m=+6.137891116 container cleanup 85832347393d6c281c9c85389ce50109718bbcbf5e7b2d0c7cc287534d5e043f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-20T21:15:05.951 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:05 smithi055 bash[71699]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1 2024-05-20T21:15:06.392 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:06 smithi055 podman[71714]: 2024-05-20 21:15:06.066466979 +0000 UTC m=+0.677387880 container remove 85832347393d6c281c9c85389ce50109718bbcbf5e7b2d0c7cc287534d5e043f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:15:06.652 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:06 smithi055 podman[71802]: 2024-05-20 21:15:06.296758049 +0000 UTC m=+0.018369784 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:06.653 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:06 smithi055 podman[71802]: 2024-05-20 21:15:06.417296212 +0000 UTC m=+0.138907978 container create 8693b6fdc110dd7a20fe927bc4f406d429072876e2116fa41455d31213ec3fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:15:06.653 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:06 smithi055 podman[71802]: 2024-05-20 21:15:06.653410756 +0000 UTC m=+0.375022488 container init 8693b6fdc110dd7a20fe927bc4f406d429072876e2116fa41455d31213ec3fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-20T21:15:06.951 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:06 smithi055 podman[71802]: 2024-05-20 21:15:06.657203556 +0000 UTC m=+0.378815281 container start 8693b6fdc110dd7a20fe927bc4f406d429072876e2116fa41455d31213ec3fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-20T21:15:06.951 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:06 smithi055 podman[71802]: 2024-05-20 21:15:06.786862218 +0000 UTC m=+0.508473961 container attach 8693b6fdc110dd7a20fe927bc4f406d429072876e2116fa41455d31213ec3fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-20T21:15:07.540 INFO:journalctl@ceph.osd.1.smithi055.stdout:May 20 21:15:07 smithi055 podman[71802]: 2024-05-20 21:15:07.251219265 +0000 UTC m=+0.972830992 container died 8693b6fdc110dd7a20fe927bc4f406d429072876e2116fa41455d31213ec3fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-20T21:15:07.835 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-20T21:15:07.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:07.877 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-20T21:15:07.877 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-20T21:15:07.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.2 2024-05-20T21:15:08.203 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:07 smithi144 systemd[1]: Stopping Ceph osd.2 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:08.550 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2[40788]: 2024-05-20T21:15:08.203+0000 7fe27311c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:08.550 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2[40788]: 2024-05-20T21:15:08.203+0000 7fe27311c700 -1 osd.2 84 *** Got signal Terminated *** 2024-05-20T21:15:08.551 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2[40788]: 2024-05-20T21:15:08.203+0000 7fe27311c700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:13.800 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:13 smithi144 podman[73035]: 2024-05-20 21:15:13.543648964 +0000 UTC m=+5.466431073 container died d72e4239d0a0cf72e82a1c0d39c78108b336cfd4f8d49e1c6af4fabc45c3fb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-20T21:15:14.179 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:13 smithi144 podman[73035]: 2024-05-20 21:15:13.928714263 +0000 UTC m=+5.851496362 container cleanup d72e4239d0a0cf72e82a1c0d39c78108b336cfd4f8d49e1c6af4fabc45c3fb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-20T21:15:14.180 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:13 smithi144 bash[73035]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2 2024-05-20T21:15:14.507 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:14 smithi144 podman[73050]: 2024-05-20 21:15:14.180112778 +0000 UTC m=+0.631975662 container remove d72e4239d0a0cf72e82a1c0d39c78108b336cfd4f8d49e1c6af4fabc45c3fb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T21:15:14.800 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:14 smithi144 podman[73139]: 2024-05-20 21:15:14.410563208 +0000 UTC m=+0.017536448 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:14.800 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:14 smithi144 podman[73139]: 2024-05-20 21:15:14.539876871 +0000 UTC m=+0.146850129 container create 2a63a163f9001847d567922867441acc745c8a9cf156bba075df0b66f695bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-20T21:15:15.099 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:14 smithi144 podman[73139]: 2024-05-20 21:15:14.801481618 +0000 UTC m=+0.408454842 container init 2a63a163f9001847d567922867441acc745c8a9cf156bba075df0b66f695bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-20T21:15:15.099 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:14 smithi144 podman[73139]: 2024-05-20 21:15:14.805061117 +0000 UTC m=+0.412034338 container start 2a63a163f9001847d567922867441acc745c8a9cf156bba075df0b66f695bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-20T21:15:15.099 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:14 smithi144 podman[73139]: 2024-05-20 21:15:14.87666408 +0000 UTC m=+0.483637314 container attach 2a63a163f9001847d567922867441acc745c8a9cf156bba075df0b66f695bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T21:15:15.370 INFO:journalctl@ceph.osd.2.smithi144.stdout:May 20 21:15:15 smithi144 podman[73139]: 2024-05-20 21:15:15.100068581 +0000 UTC m=+0.707041806 container died 2a63a163f9001847d567922867441acc745c8a9cf156bba075df0b66f695bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-20T21:15:15.698 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-20T21:15:15.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:15.737 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-20T21:15:15.737 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-20T21:15:15.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.3 2024-05-20T21:15:16.068 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:15 smithi026 systemd[1]: Stopping Ceph osd.3 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:16.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:16 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:16.067+0000 7f7ce728c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:16.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:16 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:16.067+0000 7f7ce728c700 -1 osd.3 84 *** Got signal Terminated *** 2024-05-20T21:15:16.453 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:16 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:16.067+0000 7f7ce728c700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:18.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:17.926+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:18.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:18 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:18.003+0000 7f7cdf878700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.309333+0000 front 2024-05-20T21:14:55.309497+0000 (oldest deadline 2024-05-20T21:15:17.609535+0000) 2024-05-20T21:15:18.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:18 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:18.040+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:18.652 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:18.356+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:19.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:18.652+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:19.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:18 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:18.877+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:19.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:19 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:19.006+0000 7f7cdf878700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.309333+0000 front 2024-05-20T21:14:55.309497+0000 (oldest deadline 2024-05-20T21:15:17.609535+0000) 2024-05-20T21:15:19.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:19 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:19.047+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:19.691 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:19.355+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:20.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:19.691+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:20.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:19 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:19.864+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:20.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:20 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:20.048+0000 7f7cdf878700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.309333+0000 front 2024-05-20T21:14:55.309497+0000 (oldest deadline 2024-05-20T21:15:17.609535+0000) 2024-05-20T21:15:20.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:20 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:20.009+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:20.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:20 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:20.108+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:20.694 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:20.352+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:21.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:20.695+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:21.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:20 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:20.872+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:21.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:21 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3[53727]: 2024-05-20T21:15:21.027+0000 7f7cdf878700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.309333+0000 front 2024-05-20T21:14:55.309497+0000 (oldest deadline 2024-05-20T21:15:17.609535+0000) 2024-05-20T21:15:21.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:21 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:21.059+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:21.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:21 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:21.029+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:21.678 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:21.382+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:21.794 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:21 smithi026 podman[143114]: 2024-05-20 21:15:21.514851674 +0000 UTC m=+5.567268106 container died ffdfcec885ec9cd1a3659e58d7f874c37089f9a3291413c4e921591b12f2cc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:15:22.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:21.678+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:22.068 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:22 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:22.031+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:22.069 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:21 smithi026 podman[143114]: 2024-05-20 21:15:21.985307232 +0000 UTC m=+6.037723651 container cleanup ffdfcec885ec9cd1a3659e58d7f874c37089f9a3291413c4e921591b12f2cc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-20T21:15:22.069 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:21 smithi026 bash[143114]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3 2024-05-20T21:15:22.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:21 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:21.833+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:22.452 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:22 smithi026 podman[143131]: 2024-05-20 21:15:22.303660909 +0000 UTC m=+0.789689689 container remove ffdfcec885ec9cd1a3659e58d7f874c37089f9a3291413c4e921591b12f2cc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T21:15:22.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:22 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:22.068+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:22.654 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:22.337+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:22.655 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:22.396+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:22.936 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:22 smithi026 podman[143217]: 2024-05-20 21:15:22.578581381 +0000 UTC m=+0.017807762 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:22.936 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:22 smithi026 podman[143217]: 2024-05-20 21:15:22.821487443 +0000 UTC m=+0.260713812 container create 4d623b62ffc7338d73ec6993d0899e3fd8719a8f2afdee9b75b6090619d54c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-20T21:15:23.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:22.654+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:23.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:22 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:22.869+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:23.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:22 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:22.988+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:23.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:23 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:23.102+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:23.202 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:23 smithi026 podman[143217]: 2024-05-20 21:15:23.049063816 +0000 UTC m=+0.488290199 container init 4d623b62ffc7338d73ec6993d0899e3fd8719a8f2afdee9b75b6090619d54c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-20T21:15:23.202 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:23 smithi026 podman[143217]: 2024-05-20 21:15:23.052956585 +0000 UTC m=+0.492182955 container start 4d623b62ffc7338d73ec6993d0899e3fd8719a8f2afdee9b75b6090619d54c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-20T21:15:23.202 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:23 smithi026 podman[143217]: 2024-05-20 21:15:23.141012729 +0000 UTC m=+0.580239099 container attach 4d623b62ffc7338d73ec6993d0899e3fd8719a8f2afdee9b75b6090619d54c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-20T21:15:23.659 INFO:journalctl@ceph.osd.3.smithi026.stdout:May 20 21:15:23 smithi026 podman[143217]: 2024-05-20 21:15:23.357851476 +0000 UTC m=+0.797077855 container died 4d623b62ffc7338d73ec6993d0899e3fd8719a8f2afdee9b75b6090619d54c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-20T21:15:23.684 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:23.381+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:23.685 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:23.383+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:24.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-20T21:15:24.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:23.685+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:24.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:24.051 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-20T21:15:24.052 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-20T21:15:24.052 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.4 2024-05-20T21:15:24.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:23 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:23.918+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:24.201 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:24 smithi055 systemd[1]: Stopping Ceph osd.4 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:24.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:23 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:23.950+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:24.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:24 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:24.078+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:24.701 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:24 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T21:15:24.376+0000 7fcdd68fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:24.701 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:24 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T21:15:24.376+0000 7fcdd68fd700 -1 osd.4 84 *** Got signal Terminated *** 2024-05-20T21:15:24.701 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:24 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T21:15:24.376+0000 7fcdd68fd700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:24.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:24.407+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:24.800 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:24.417+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:24.801 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:24.647+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:25.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:24 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:24.923+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:25.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:24 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:24.988+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:25.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:25 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:25.052+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:25.675 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:25.399+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:25.675 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:25.427+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:26.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:25.675+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:26.402 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:25 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:25.955+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:26.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:25 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:25.993+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:26.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:26 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:26.013+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:26.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:26 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:26.013+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:26.697 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:26 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T21:15:26.402+0000 7fcdd2ef1700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.263780+0000 front 2024-05-20T21:15:00.263904+0000 (oldest deadline 2024-05-20T21:15:25.464700+0000) 2024-05-20T21:15:26.724 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:26 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:26.385+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:26.725 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:26 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:26.392+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:26.725 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:26 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:26.392+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:26.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:26 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:26.697+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:27.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:26 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:26.725+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:27.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:26 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:26.725+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:27.407 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:26 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:26.989+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:27.408 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:26 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:26.989+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:27.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:26 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:26.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:27.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:27 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:27.059+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:27.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:27 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:27.059+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:27.701 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:27 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T21:15:27.407+0000 7fcdd2ef1700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.263780+0000 front 2024-05-20T21:15:00.263904+0000 (oldest deadline 2024-05-20T21:15:25.464700+0000) 2024-05-20T21:15:27.704 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:27 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:27.400+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:27.704 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:27 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:27.400+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:27.704 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:27 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:27.430+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:27.704 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:27 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:27.430+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:28.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:27 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:27.704+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:28.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:27 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:27.704+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:28.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:27 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:27.741+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:28.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:27 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:27.965+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:28.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:27 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:27.965+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:28.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:27 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:27.935+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:28.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:27 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:27.935+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:28.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:28.098+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:28.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:28.098+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:28.669 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:28 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:28.395+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:28.669 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:28 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:28.395+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:28.669 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:28 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:28.423+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:28.669 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:28 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:28.423+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:28.700 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:28 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4[46214]: 2024-05-20T21:15:28.417+0000 7fcdd2ef1700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.263780+0000 front 2024-05-20T21:15:00.263904+0000 (oldest deadline 2024-05-20T21:15:25.464700+0000) 2024-05-20T21:15:29.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:28 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:28.669+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:29.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:28 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:28.669+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:29.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:28 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:28.753+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:29.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:28 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:28.957+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:29.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:28 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:28.957+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:29.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:28.960+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:29.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:28 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:28.960+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:29.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:29.055+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:29.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:29.055+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:29.701 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:29 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:29.398+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:29.701 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:29 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:29.398+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:29.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:29 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:29.365+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:29.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:29 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:29.365+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:29.978 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:29 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:29.708+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:29.979 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:29 smithi055 podman[71930]: 2024-05-20 21:15:29.743678267 +0000 UTC m=+5.488223105 container died e2c2821221a665303c91653c7b7cd63bf8ea454c347ca1f7e0da8e47eed2f2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-20T21:15:30.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:29 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:29.701+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:30.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:29 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:29.701+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:30.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:29.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:30.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:29.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:30.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:30.024+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:30.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:30.024+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:30.235 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:29 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:29.977+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:30.235 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:29 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:29.977+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:30.506 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:30 smithi055 podman[71930]: 2024-05-20 21:15:30.236666009 +0000 UTC m=+5.981210848 container cleanup e2c2821221a665303c91653c7b7cd63bf8ea454c347ca1f7e0da8e47eed2f2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-20T21:15:30.506 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:30 smithi055 bash[71930]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4 2024-05-20T21:15:30.685 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:30 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:30.376+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:30.685 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:30 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:30.376+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:30.686 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:30 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:30.445+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:30.686 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:30 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:30.445+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:30.872 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:30 smithi055 podman[71947]: 2024-05-20 21:15:30.506980285 +0000 UTC m=+0.757281867 container remove e2c2821221a665303c91653c7b7cd63bf8ea454c347ca1f7e0da8e47eed2f2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-20T21:15:30.873 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:30 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:30.746+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:31.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:30 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:30.686+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:31.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:30 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:30.686+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:31.144 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:30 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:30.946+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:31.144 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:30 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:30.946+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:31.144 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:30 smithi055 podman[72033]: 2024-05-20 21:15:30.776470857 +0000 UTC m=+0.017830479 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:31.145 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:31 smithi055 podman[72033]: 2024-05-20 21:15:31.022953825 +0000 UTC m=+0.264313433 container create 8ba1b55f21e19717f196e8761dfa71c4f3e4ed9c7a291ba43a055d830b9a9ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:15:31.451 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:31 smithi055 podman[72033]: 2024-05-20 21:15:31.29271417 +0000 UTC m=+0.534073792 container init 8ba1b55f21e19717f196e8761dfa71c4f3e4ed9c7a291ba43a055d830b9a9ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True) 2024-05-20T21:15:31.451 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:31 smithi055 podman[72033]: 2024-05-20 21:15:31.296250422 +0000 UTC m=+0.537610036 container start 8ba1b55f21e19717f196e8761dfa71c4f3e4ed9c7a291ba43a055d830b9a9ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-20T21:15:31.451 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:31 smithi055 podman[72033]: 2024-05-20 21:15:31.384149855 +0000 UTC m=+0.625509497 container attach 8ba1b55f21e19717f196e8761dfa71c4f3e4ed9c7a291ba43a055d830b9a9ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-20T21:15:31.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:30.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:31.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:30 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:30.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:31.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:31.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:31.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:31.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:31.800 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:31 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:31.422+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:31.800 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:31 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:31.422+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:31.801 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:31 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:31.648+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:31.801 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:31 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:31.648+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:31.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:31 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:31.421+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:31.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:31 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:31.421+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:31.911 INFO:journalctl@ceph.osd.4.smithi055.stdout:May 20 21:15:31 smithi055 podman[72033]: 2024-05-20 21:15:31.602228173 +0000 UTC m=+0.843587791 container died 8ba1b55f21e19717f196e8761dfa71c4f3e4ed9c7a291ba43a055d830b9a9ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-20T21:15:31.912 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:31 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:31.782+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:32.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:31 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:31.994+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:32.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:31 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:31.994+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:32.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:31.931+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:32.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:31 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:31.931+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:32.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:32.023+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:32.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:32.023+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:32.254 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-20T21:15:32.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:32.296 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-20T21:15:32.297 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-20T21:15:32.297 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.5 2024-05-20T21:15:32.616 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:32 smithi144 systemd[1]: Stopping Ceph osd.5 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:32.616 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:32.395+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:32.617 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:32.395+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:32.617 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:32.439+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:32.617 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:32.439+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:33.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:32.673+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:33.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:32.673+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:33.051 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:32.616+0000 7ff0224d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:33.051 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:32.616+0000 7ff0224d8700 -1 osd.5 84 *** Got signal Terminated *** 2024-05-20T21:15:33.052 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:32 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:32.616+0000 7ff0224d8700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:33.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:32 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:32.741+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:33.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:32 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:32.975+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:33.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:32 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:32.975+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:33.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:32.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:33.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:32 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:32.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:33.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:33.027+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:33.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:33.027+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:33.800 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:33 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:33.682+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:33.801 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:33 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:33.682+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:33.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:33 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:33.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:33.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:33 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:33.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:33.801 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:33 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:33.436+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:33.802 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:33 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:33.436+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:34.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:33 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:33.735+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:34.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:33 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:33.940+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:34.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:33 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:33.940+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:34.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:33.999+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:34.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:33.999+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:34.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:33.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:34.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:33.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:34.722 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:34.494+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:34.722 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:34.494+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:34.722 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:34.394+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:34.722 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:34.394+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:35.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:34.721+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:35.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:34.721+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:35.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:34 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:34.721+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:35.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:34 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:34.711+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:35.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:34 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:34.948+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:35.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:34 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:34.948+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:35.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:34 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:34.948+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:35.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:34.955+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:35.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:34 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:34.955+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:35.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:35 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:35.045+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:35.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:35 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:35.045+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:35.723 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:35.484+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:35.723 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:35.484+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:35.723 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:35.431+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:35.723 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:35.431+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:36.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:35.723+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:36.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:35.723+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:36.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:35 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:35.723+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:36.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:35 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:35.743+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:36.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:35 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:35.976+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:36.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:35 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:35.976+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:36.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:35 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:35.976+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:36.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:35 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:35.983+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:36.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:35 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:35.983+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:36.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:36.058+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:36.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:36.058+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:36.762 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:36.485+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:36.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:36.485+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:36.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:36.485+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:36.763 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:36.398+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:36.763 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:36.398+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:36.763 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:36.398+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.470627+0000 front 2024-05-20T21:15:11.470606+0000 (oldest deadline 2024-05-20T21:15:36.170736+0000) 2024-05-20T21:15:37.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:36.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:37.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:36.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:37.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:36 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:36.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:37.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:36 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:36.781+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:37.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:36 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:36.995+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:37.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:36 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:36.995+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:37.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:36 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:36.995+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:37.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:36.959+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:37.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:36.959+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:37.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:36.959+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:37.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:37.022+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:37.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:37.022+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:37.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:37.022+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:37.800 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:37.365+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:52.767648+0000 front 2024-05-20T21:14:52.767667+0000 (oldest deadline 2024-05-20T21:15:21.568147+0000) 2024-05-20T21:15:37.801 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:37.365+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.269486+0000 front 2024-05-20T21:15:03.269471+0000 (oldest deadline 2024-05-20T21:15:26.169389+0000) 2024-05-20T21:15:37.801 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5[47243]: 2024-05-20T21:15:37.365+0000 7ff01eacc700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.470627+0000 front 2024-05-20T21:15:11.470606+0000 (oldest deadline 2024-05-20T21:15:36.170736+0000) 2024-05-20T21:15:37.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:37.533+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:37.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:37.533+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:37.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:37.533+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:38.168 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:37.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:38.168 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:37.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:38.168 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:37 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:37.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:38.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:37 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:37.803+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:38.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:38 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:38.019+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:38.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:38 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:38.019+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:38.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:38 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:38.019+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:38.452 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:38 smithi144 podman[73258]: 2024-05-20 21:15:38.168788922 +0000 UTC m=+5.669054611 container died 463642c18c266905488b6e0cf58e3b09da6574edfa4e05a340b1fd3f0ab5736c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-20T21:15:38.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:37.952+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:38.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:37.952+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:38.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:37 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:37.952+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:38.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:38.013+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:38.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:38.013+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:38.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:38.013+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:38.787 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:38 smithi144 podman[73258]: 2024-05-20 21:15:38.562889202 +0000 UTC m=+6.063154872 container cleanup 463642c18c266905488b6e0cf58e3b09da6574edfa4e05a340b1fd3f0ab5736c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-20T21:15:38.787 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:38 smithi144 bash[73258]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5 2024-05-20T21:15:38.787 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:38 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:38.568+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:38.787 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:38 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:38.568+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:38.787 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:38 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:38.568+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:39.050 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:38 smithi144 podman[73273]: 2024-05-20 21:15:38.808239985 +0000 UTC m=+0.635143066 container remove 463642c18c266905488b6e0cf58e3b09da6574edfa4e05a340b1fd3f0ab5736c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:15:39.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:38 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:38.786+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:39.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:38 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:38.786+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:39.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:38 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:38.786+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:39.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:38 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:38.788+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:39.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:38 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:38.788+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:39.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:39.022+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:39.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:39.022+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:39.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:39.022+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:39.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:38.989+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:39.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:38.989+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:39.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:38.989+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:39.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:38.947+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:39.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:38.947+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:39.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:38 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:38.947+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:39.418 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:39 smithi144 podman[73363]: 2024-05-20 21:15:39.038383329 +0000 UTC m=+0.016968975 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:39.419 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:39 smithi144 podman[73363]: 2024-05-20 21:15:39.157379274 +0000 UTC m=+0.135964929 container create f752ac984bd94c81e969b1760ecd9c19b430cb3609d64fc35c6996237fd73297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, org.label-schema.vendor=CentOS) 2024-05-20T21:15:39.419 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:39 smithi144 podman[73363]: 2024-05-20 21:15:39.419097924 +0000 UTC m=+0.397683557 container init f752ac984bd94c81e969b1760ecd9c19b430cb3609d64fc35c6996237fd73297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-20T21:15:39.720 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:39 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:39.570+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:39.721 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:39 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:39.570+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:39.721 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:39 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:39.570+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:39.721 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:39 smithi144 podman[73363]: 2024-05-20 21:15:39.422889424 +0000 UTC m=+0.401475065 container start f752ac984bd94c81e969b1760ecd9c19b430cb3609d64fc35c6996237fd73297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, GIT_CLEAN=True) 2024-05-20T21:15:39.721 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:39 smithi144 podman[73363]: 2024-05-20 21:15:39.504279089 +0000 UTC m=+0.482864722 container attach f752ac984bd94c81e969b1760ecd9c19b430cb3609d64fc35c6996237fd73297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5) 2024-05-20T21:15:40.047 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 20 21:15:39 smithi144 podman[73363]: 2024-05-20 21:15:39.72164051 +0000 UTC m=+0.700226156 container died f752ac984bd94c81e969b1760ecd9c19b430cb3609d64fc35c6996237fd73297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux ) 2024-05-20T21:15:40.047 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:39 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:39.816+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:40.047 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:39 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:39.816+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:40.047 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:39 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:39.816+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:40.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:39.760+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:40.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:39.760+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:40.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:39.977+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:40.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:39.977+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:40.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:39 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:39.977+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:40.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:39 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:39.954+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:40.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:39 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:39.954+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:40.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:39 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:39.954+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:40.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:39 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:39.948+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:40.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:39 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:39.948+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:40.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:39 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:39.948+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:40.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-20T21:15:40.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:40.391 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-20T21:15:40.392 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-20T21:15:40.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.6 2024-05-20T21:15:40.702 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 systemd[1]: Stopping Ceph osd.6 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:40.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:40 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:40.559+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:40.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:40 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:40.559+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:40.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:40 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:40.559+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:40.973 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:40.723+0000 7f8c8ef8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:40.973 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:40.723+0000 7f8c8ef8b700 -1 osd.6 84 *** Got signal Terminated *** 2024-05-20T21:15:40.974 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:40.723+0000 7f8c8ef8b700 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:40.974 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:40.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:40.974 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:40.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:40.974 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:40.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:41.004 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:40 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:40.738+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:41.004 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:40 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:40.738+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:41.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:40 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:40.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:41.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:40 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:40.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:41.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:40 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:40.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:41.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:41 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:41.004+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:41.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:41 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:41.004+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:41.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:41 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:41.004+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:41.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:40.973+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:41.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:40.974+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:41.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:40 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:40.974+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:41.833 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:41.576+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:41.833 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:41.576+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:41.833 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:41.576+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:41.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:41 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:41.693+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:41.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:41 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:41.693+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:42.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:41 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:41.928+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:42.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:41 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:41.928+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:42.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:41 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:41.929+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:42.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:42.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:42.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:42.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:41.833+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:42.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:41.833+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:42.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:41.833+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:42.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:41 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:41.833+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:42.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:42.001+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:42.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:42.001+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:42.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:42.001+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:43.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:42.804+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:43.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:42.804+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:43.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:42.804+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:43.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:42.804+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:43.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:42.624+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:43.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:42.624+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:43.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:42.624+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:43.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:42 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:42.624+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:43.201 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:42.723+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:43.202 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:42.723+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:43.202 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:42.723+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:43.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:42.961+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:43.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:42.961+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:43.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:42 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:42.961+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:43.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:42.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:43.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:42.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:43.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:42.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:43.203 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:42.926+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.744497+0000 front 2024-05-20T21:15:16.744470+0000 (oldest deadline 2024-05-20T21:15:42.044617+0000) 2024-05-20T21:15:43.204 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.966+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:43.204 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.966+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:43.204 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.966+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:43.204 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:42 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:42.966+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:43.983 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:43 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:43.717+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:43.983 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:43 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:43.717+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:43.983 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:43 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:43.717+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:44.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:43.789+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:43.789+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:43.789+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:43.789+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:43.608+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:43.608+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:43.608+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:44.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:43 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:43.608+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:44.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:43 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:43.983+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:44.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:43 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:43.983+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:44.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:43 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:43.983+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:44.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:43 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:43.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:44.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:43 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:43.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:44.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:43 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:43.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:44.452 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:43 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:43.971+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.744497+0000 front 2024-05-20T21:15:16.744470+0000 (oldest deadline 2024-05-20T21:15:42.044617+0000) 2024-05-20T21:15:44.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:44.014+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:44.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:44.014+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:44.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:44.014+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:44.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:44.014+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:44.837 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:44.583+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:44.838 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:44.583+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:44.838 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:44.583+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:44.838 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:44.583+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:44.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:44.679+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:44.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:44.679+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:44.952 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:44.679+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:45.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:44.836+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:45.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:44.836+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:45.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:44.836+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:45.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:44 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:44.836+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:45.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:45.015+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:45.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:45.015+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:45.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:45.015+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:45.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.011+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:45.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.011+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:45.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.011+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:45.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.011+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:45.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:44.957+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:55.740645+0000 front 2024-05-20T21:14:55.740676+0000 (oldest deadline 2024-05-20T21:15:19.240870+0000) 2024-05-20T21:15:45.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:44.957+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.842620+0000 front 2024-05-20T21:15:03.842643+0000 (oldest deadline 2024-05-20T21:15:27.342709+0000) 2024-05-20T21:15:45.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:44.957+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:07.343341+0000 front 2024-05-20T21:15:07.343445+0000 (oldest deadline 2024-05-20T21:15:36.744006+0000) 2024-05-20T21:15:45.453 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:44 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6[60536]: 2024-05-20T21:15:44.957+0000 7f8c8b57f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.744497+0000 front 2024-05-20T21:15:16.744470+0000 (oldest deadline 2024-05-20T21:15:42.044617+0000) 2024-05-20T21:15:45.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:45.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:45.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:45.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:45.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:45.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:45.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:45.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:45.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:45.640+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:45.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:45.640+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:45.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:45 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:45.640+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:46.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:45.881+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:46.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:45.881+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:46.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:45.881+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:46.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:45 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:45.881+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:46.447 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:46 smithi026 podman[143323]: 2024-05-20 21:15:46.148914361 +0000 UTC m=+5.551197178 container died 75de8d0296e13600cbe221ed2f9dcc505a491be2a870fbacb0d8ccad84447f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:15:46.448 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.969+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:46.448 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.969+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:46.448 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.969+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:46.448 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:45 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:45.969+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:46.450 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:46.026+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:46.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:46.026+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:46.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:46.026+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:46.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:46.026+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:46.702 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:46 smithi026 podman[143323]: 2024-05-20 21:15:46.638908497 +0000 UTC m=+6.041191314 container cleanup 75de8d0296e13600cbe221ed2f9dcc505a491be2a870fbacb0d8ccad84447f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-20T21:15:46.702 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:46 smithi026 bash[143323]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6 2024-05-20T21:15:46.865 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:46.575+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:46.865 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:46.575+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:46.865 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:46.575+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:46.866 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:46.575+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:46.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:46.667+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:46.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:46.667+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:46.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:46 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:46.667+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:47.202 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:46 smithi026 podman[143341]: 2024-05-20 21:15:46.898996084 +0000 UTC m=+0.750956306 container remove 75de8d0296e13600cbe221ed2f9dcc505a491be2a870fbacb0d8ccad84447f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-20T21:15:47.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:46.951+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:47.202 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:46.951+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:47.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:46.951+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:47.203 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:46 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:46.951+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:47.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:46.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:47.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:46.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:47.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:46.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:47.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:46 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:46.865+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:47.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:47.027+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:47.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:47.027+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:47.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:47.027+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:47.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:47.027+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:47.551 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:47 smithi026 podman[143430]: 2024-05-20 21:15:47.135176126 +0000 UTC m=+0.018141840 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:47.552 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:47 smithi026 podman[143430]: 2024-05-20 21:15:47.258207919 +0000 UTC m=+0.141173654 container create 73f1d97e2ea465252e82f99a032d73277ee2bfb0beb78aa4d03d977c937e086c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:15:47.552 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:47 smithi026 podman[143430]: 2024-05-20 21:15:47.477891623 +0000 UTC m=+0.360857319 container init 73f1d97e2ea465252e82f99a032d73277ee2bfb0beb78aa4d03d977c937e086c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-20T21:15:47.552 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:47 smithi026 podman[143430]: 2024-05-20 21:15:47.481304522 +0000 UTC m=+0.364270221 container start 73f1d97e2ea465252e82f99a032d73277ee2bfb0beb78aa4d03d977c937e086c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-20T21:15:47.552 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:47 smithi026 podman[143430]: 2024-05-20 21:15:47.552644557 +0000 UTC m=+0.435610273 container attach 73f1d97e2ea465252e82f99a032d73277ee2bfb0beb78aa4d03d977c937e086c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD) 2024-05-20T21:15:47.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:47.628+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:47.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:47.628+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:47.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:47 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:47.628+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:47.952 INFO:journalctl@ceph.osd.6.smithi026.stdout:May 20 21:15:47 smithi026 podman[143430]: 2024-05-20 21:15:47.788423125 +0000 UTC m=+0.671388818 container died 73f1d97e2ea465252e82f99a032d73277ee2bfb0beb78aa4d03d977c937e086c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:15:48.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:47.828+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:48.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:47.828+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:48.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:47.828+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:48.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:47.828+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:48.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:47.590+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:48.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:47.590+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:48.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:47.590+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:48.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:47 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:47.590+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:48.351 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:47 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:47.983+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:48.351 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:47 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:47.983+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:48.351 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:47 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:47.983+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:48.351 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:47 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:47.983+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:48.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:48.076+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:48.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:48.076+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:48.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:48.076+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:48.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:48.076+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:48.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-20T21:15:48.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:48.530 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-20T21:15:48.530 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-20T21:15:48.530 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.7 2024-05-20T21:15:48.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:48.543+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:48.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:48.543+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:48.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:48.543+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:48.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:48.543+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:48.860 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 systemd[1]: Stopping Ceph osd.7 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:48.861 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:48.600+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:48.861 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:48.600+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:48.861 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:48.600+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:48.861 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:48.857+0000 7fa535361700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:48.861 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:48.857+0000 7fa535361700 -1 osd.7 84 *** Got signal Terminated *** 2024-05-20T21:15:49.107 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:48 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:48.857+0000 7fa535361700 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:49.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:48.814+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:49.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:48.814+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:49.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:48.814+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:49.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:48 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:48.814+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:49.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:49.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:49.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:49.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:49.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:49.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:49.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:49.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:49.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:49.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:49.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:49.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:49.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:49.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:49.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:49 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:49.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:49.743 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:49.643+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:49.743 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:49.643+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:49.743 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:49 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:49.643+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:49.800 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:49.780+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:49.800 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:49.780+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:49.801 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:49.780+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:49.801 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:49.780+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:49.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:49.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:49.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:49.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:49.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:49.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:49.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:49 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:49.534+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:50.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:50.100+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:50.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:50.100+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:50.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:50.100+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:50.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:50.100+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:50.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:50 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:50.053+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:50.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:50 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:50.053+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:50.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:50 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:50.053+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:50.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:50 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:50.053+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:50.784 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:50.530+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:50.784 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:50.530+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:50.784 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:50.530+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:50.784 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:50.530+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:50.784 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:50.530+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:50.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:50.619+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:50.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:50.619+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:50.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:50.619+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:50.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:50 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:50.619+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:27.659245+0000 front 2024-05-20T21:15:27.659283+0000 (oldest deadline 2024-05-20T21:15:50.559399+0000) 2024-05-20T21:15:51.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:50.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:51.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:50.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:51.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:50.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:51.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:50 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:50.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:51.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:51.071+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:51.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:51.071+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:51.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:51.071+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:51.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:51.071+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:51.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:51.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:51.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:51.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:51.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:51.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:51.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:51.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:51.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:51 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:51.049+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:51.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:51.495+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:51.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:51.495+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:51.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:51.495+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:51.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:51.495+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:51.784 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:51.495+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:51.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:51.591+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:51.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:51.591+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:51.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:51.591+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:51.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:51 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:51.591+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:27.659245+0000 front 2024-05-20T21:15:27.659283+0000 (oldest deadline 2024-05-20T21:15:50.559399+0000) 2024-05-20T21:15:52.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:51.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:52.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:51.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:52.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:51.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:52.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:51.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:52.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:51 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:51.783+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:52.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:52.072+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:52.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:52.072+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:52.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:52.072+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:52.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:52.072+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:52.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:52.075+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:52.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:52.075+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:52.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:52.075+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:52.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:52.075+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:52.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:52 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:52.075+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:52.550 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:52.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:52.550 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:52.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:52.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:52.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:52.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:52.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:52.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:52.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:52.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:52.569+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:52.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:52.569+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:52.952 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:52.569+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:52.952 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:52 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:52.569+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:27.659245+0000 front 2024-05-20T21:15:27.659283+0000 (oldest deadline 2024-05-20T21:15:50.559399+0000) 2024-05-20T21:15:53.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:52.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:53.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:52.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:53.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:52.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:53.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:52.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:53.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:52 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:52.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:53.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:53.101+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:53.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:53.101+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:53.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:53.101+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:53.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:53.101+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:53.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:53.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:53.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:53.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:53.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:53.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:53.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:53.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:53.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:53 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:53.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:53.717 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:53.431+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:53.717 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:53.431+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:53.718 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:53.431+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:53.718 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:53.431+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:53.718 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:53.431+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:53.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:53.560+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.755672+0000 front 2024-05-20T21:15:03.755628+0000 (oldest deadline 2024-05-20T21:15:26.655759+0000) 2024-05-20T21:15:53.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:53.560+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:12.556844+0000 front 2024-05-20T21:15:12.556861+0000 (oldest deadline 2024-05-20T21:15:38.456872+0000) 2024-05-20T21:15:53.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:53.560+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:18.457606+0000 front 2024-05-20T21:15:18.457519+0000 (oldest deadline 2024-05-20T21:15:41.957411+0000) 2024-05-20T21:15:53.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:53 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7[53135]: 2024-05-20T21:15:53.560+0000 7fa531955700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:27.659245+0000 front 2024-05-20T21:15:27.659283+0000 (oldest deadline 2024-05-20T21:15:50.559399+0000) 2024-05-20T21:15:54.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:53.717+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:54.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:53.717+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:54.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:53.717+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:54.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:53.717+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:54.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:53 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:53.717+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:54.358 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:54.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:54.358 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:54.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:54.359 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:54.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:54.359 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:54.107+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:54.454 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:54.050+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:54.454 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:54.050+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:54.454 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:54.050+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:54.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:54.050+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:54.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:54 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:54.050+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:54.666 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:54 smithi055 podman[72137]: 2024-05-20 21:15:54.359407483 +0000 UTC m=+5.626695052 container died 1c1989aaaff6d05cbc1208cf14e0b89a5964c767aa49f1432b4b8b7577ed59fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True) 2024-05-20T21:15:54.764 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:54.416+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:54.764 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:54.416+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:54.764 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:54.416+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:54.764 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:54.416+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:54.764 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:54.416+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:54.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:54 smithi055 podman[72137]: 2024-05-20 21:15:54.758153881 +0000 UTC m=+6.025441437 container cleanup 1c1989aaaff6d05cbc1208cf14e0b89a5964c767aa49f1432b4b8b7577ed59fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-20T21:15:54.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:54 smithi055 bash[72137]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7 2024-05-20T21:15:55.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:54.763+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:55.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:54.763+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:55.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:54.763+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:55.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:54.763+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:55.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:54 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:54.763+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:55.400 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72166]: 2024-05-20 21:15:55.059886268 +0000 UTC m=+0.701427368 container remove 1c1989aaaff6d05cbc1208cf14e0b89a5964c767aa49f1432b4b8b7577ed59fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-20T21:15:55.401 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:55.086+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:55.401 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:55.086+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:55.401 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:55.086+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:55.401 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:55.086+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:55.402 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:55 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:55.086+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:15:55.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:55.029+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:55.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:55.029+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:55.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:55.029+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:55.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:55.029+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:55.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:55 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:55.029+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:55.654 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72258]: 2024-05-20 21:15:55.305098077 +0000 UTC m=+0.018545210 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:15:55.654 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72258]: 2024-05-20 21:15:55.427461478 +0000 UTC m=+0.140908669 container create 60b5089b4b48481111fc04d0ccff31f6ec81321cc8eb40b6638c9bb87b9d01d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:15:55.654 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72258]: 2024-05-20 21:15:55.654925112 +0000 UTC m=+0.368372238 container init 60b5089b4b48481111fc04d0ccff31f6ec81321cc8eb40b6638c9bb87b9d01d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-20T21:15:55.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:55.465+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:55.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:55.465+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:55.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:55.465+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:55.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:55.465+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:55.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:55.465+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:55.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72258]: 2024-05-20 21:15:55.658381891 +0000 UTC m=+0.371829026 container start 60b5089b4b48481111fc04d0ccff31f6ec81321cc8eb40b6638c9bb87b9d01d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-20T21:15:55.951 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72258]: 2024-05-20 21:15:55.74669988 +0000 UTC m=+0.460147016 container attach 60b5089b4b48481111fc04d0ccff31f6ec81321cc8eb40b6638c9bb87b9d01d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-20T21:15:56.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:55.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:56.300 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:55.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:56.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:55.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:56.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:55.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:56.301 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:55 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:55.812+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:56.424 INFO:journalctl@ceph.osd.7.smithi055.stdout:May 20 21:15:55 smithi055 podman[72258]: 2024-05-20 21:15:55.965123246 +0000 UTC m=+0.678570380 container died 60b5089b4b48481111fc04d0ccff31f6ec81321cc8eb40b6638c9bb87b9d01d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True) 2024-05-20T21:15:56.425 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:56 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:56.075+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:56.425 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:56 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:56.075+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:56.425 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:56 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:56.075+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:56.425 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:56 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:56.075+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:56.425 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:56 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:56.075+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:15:56.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:56.071+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:56.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:56.071+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:56.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:56.071+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:56.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:56.071+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:56.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:56 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:56.071+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:56.779 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-20T21:15:56.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:56.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:56.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:56.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:56.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:56.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:56.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:56.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:56.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:56.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:56.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:15:56.827 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-20T21:15:56.828 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-20T21:15:56.828 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.8 2024-05-20T21:15:57.133 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:56.825+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:57.133 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:56.825+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:57.133 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:56.825+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:57.133 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:56.825+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:57.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:56 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:56.825+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:57.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:56 smithi144 systemd[1]: Stopping Ceph osd.8 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:15:57.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.133+0000 7f897e791700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:15:57.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.133+0000 7f897e791700 -1 osd.8 84 *** Got signal Terminated *** 2024-05-20T21:15:57.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.133+0000 7f897e791700 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:15:57.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:57 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:57.030+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:57.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:57 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:57.030+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:57.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:57 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:57.030+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:57.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:57 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:57.030+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:57.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:57 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:57.030+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:15:57.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:57 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:57.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:57.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:57 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:57.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:57.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:57 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:57.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:57.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:57 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:57.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:57.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:57 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:57.031+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:57.782 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:57.490+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:57.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:57.490+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:57.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:57.490+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:57.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:57.490+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:57.783 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:57.490+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:58.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.782+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:58.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.782+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:58.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.782+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:58.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.782+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:58.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:57 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:57.782+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:58.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:58 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:58.018+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:58.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:58 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:58.018+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:58.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:58 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:58.018+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:58.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:58 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:58.018+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:58.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:58 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:58.018+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:15:58.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:58 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:58.064+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:58.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:58 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:58.064+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:58.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:58 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:58.064+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:58.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:58 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:58.064+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:58.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:58 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:58.064+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:58.777 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:58.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:58.778 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:58.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:58.778 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:58.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:58.778 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:58.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:58.778 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:58.472+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:15:59.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:58.777+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:15:59.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:58.777+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:15:59.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:58.777+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:15:59.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:58.777+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:15:59.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:58 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:58.777+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:15:59.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.014+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:15:59.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.014+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:15:59.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.014+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:15:59.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.014+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:15:59.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.014+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:15:59.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:59 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:59.036+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:15:59.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:59 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:59.036+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:15:59.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:59 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:59.036+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:15:59.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:59 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:59.036+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:15:59.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:15:59 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:15:59.036+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:15:59.750 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:59.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:15:59.751 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:59.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:15:59.751 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:59.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:15:59.751 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:59.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:15:59.751 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:15:59.478+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:00.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:59.750+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:16:00.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:59.750+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:16:00.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:59.750+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:16:00.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:59.750+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:16:00.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:15:59 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:15:59.750+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:16:00.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.969+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:00.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.969+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:00.451 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.969+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:00.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.969+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:00.452 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:15:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:15:59.969+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:00.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.004+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:00.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.004+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:00.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.004+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:00.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.004+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:00.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:00 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.004+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:00.799 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:00.506+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:00.799 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:00.506+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:00.799 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:00.506+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:00.799 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:00.506+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:00.799 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:00.506+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:00.799 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:00.506+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:01.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:00.799+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:16:01.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:00.799+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:16:01.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:00.799+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:16:01.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:00.799+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:16:01.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:00 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:00.799+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:16:01.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:00 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:00.922+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:01.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:00 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:00.922+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:01.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:00 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:00.922+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:01.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:00 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:00.922+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:01.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:00 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:00.922+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:01.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:00 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:00.922+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:01.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:01.473 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:01.473 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:01.473 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:01.473 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:01.473 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:01 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:00.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:01.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:01.469+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:01.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:01.469+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:01.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:01.469+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:01.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:01.469+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:01.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:01.469+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:01.763 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:01.469+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:02.050 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:01.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.987213+0000 front 2024-05-20T21:14:54.987001+0000 (oldest deadline 2024-05-20T21:15:18.487104+0000) 2024-05-20T21:16:02.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:01.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:04.189833+0000 front 2024-05-20T21:15:04.189952+0000 (oldest deadline 2024-05-20T21:15:26.490019+0000) 2024-05-20T21:16:02.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:01.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.991033+0000 front 2024-05-20T21:15:09.991162+0000 (oldest deadline 2024-05-20T21:15:34.091207+0000) 2024-05-20T21:16:02.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:01.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:16.992236+0000 front 2024-05-20T21:15:16.992317+0000 (oldest deadline 2024-05-20T21:15:41.692030+0000) 2024-05-20T21:16:02.051 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:01 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8[54161]: 2024-05-20T21:16:01.762+0000 7f897ad85700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:26.293812+0000 front 2024-05-20T21:15:26.293950+0000 (oldest deadline 2024-05-20T21:15:51.593715+0000) 2024-05-20T21:16:02.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:01.930+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:02.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:01.930+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:02.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:01.930+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:02.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:01.930+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:02.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:01.930+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:02.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:01 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:01.930+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:02.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:01.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:02.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:01.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:02.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:01.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:02.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:01.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:02.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:01.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:02.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:02 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:01.999+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:02.663 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:02 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:02.487+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:02.663 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:02 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:02.487+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:02.663 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:02 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:02.487+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:02.663 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:02 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:02.487+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:02.663 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:02 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:02.487+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:02.663 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:02 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:02.487+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:02.941 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:02 smithi144 podman[73481]: 2024-05-20 21:16:02.663542378 +0000 UTC m=+5.636387496 container died 5e26e79379eda00a93ae623f79e8e5c06318e14bca9647e6474554ba5e272647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-20T21:16:03.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:02 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:02.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:03.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:02 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:02.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:03.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:02 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:02.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:03.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:02 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:02.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:03.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:02 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:02.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:03.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:02 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:02.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:03.225 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 podman[73481]: 2024-05-20 21:16:03.008251195 +0000 UTC m=+5.981096303 container cleanup 5e26e79379eda00a93ae623f79e8e5c06318e14bca9647e6474554ba5e272647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-20T21:16:03.226 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 bash[73481]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8 2024-05-20T21:16:03.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.042+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:03.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.042+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:03.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.042+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:03.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.042+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:03.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.042+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:03.455 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.042+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:03.550 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 podman[73498]: 2024-05-20 21:16:03.22670191 +0000 UTC m=+0.562569319 container remove 5e26e79379eda00a93ae623f79e8e5c06318e14bca9647e6474554ba5e272647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0) 2024-05-20T21:16:03.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:03 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:03.450+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:03.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:03 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:03.450+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:03.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:03 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:03.450+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:03.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:03 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:03.450+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:03.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:03 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:03.450+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:03.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:03 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:03.450+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:03.921 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 podman[73585]: 2024-05-20 21:16:03.669266062 +0000 UTC m=+0.106464012 container create 8255d4f3f5dd6f473166730edc5cb113e7cfcb6c1d1fcd646f319414df88c7d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-20T21:16:03.921 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 podman[73585]: 2024-05-20 21:16:03.580006072 +0000 UTC m=+0.017204047 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:16:03.921 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 podman[73585]: 2024-05-20 21:16:03.92187384 +0000 UTC m=+0.359071800 container init 8255d4f3f5dd6f473166730edc5cb113e7cfcb6c1d1fcd646f319414df88c7d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-20T21:16:04.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:03 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:03.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:04.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:03 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:03.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:04.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:03 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:03.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:04.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:03 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:03.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:04.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:03 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:03.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:04.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:03 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:03.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:04.214 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:03 smithi144 podman[73585]: 2024-05-20 21:16:03.925438187 +0000 UTC m=+0.362636160 container start 8255d4f3f5dd6f473166730edc5cb113e7cfcb6c1d1fcd646f319414df88c7d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-20T21:16:04.215 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:04 smithi144 podman[73585]: 2024-05-20 21:16:04.022064171 +0000 UTC m=+0.459262139 container attach 8255d4f3f5dd6f473166730edc5cb113e7cfcb6c1d1fcd646f319414df88c7d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:16:04.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.993+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:04.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.993+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:04.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.993+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:04.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.993+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:04.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.993+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:04.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:03 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:03.993+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:04.491 INFO:journalctl@ceph.osd.8.smithi144.stdout:May 20 21:16:04 smithi144 podman[73585]: 2024-05-20 21:16:04.215796394 +0000 UTC m=+0.652994356 container died 8255d4f3f5dd6f473166730edc5cb113e7cfcb6c1d1fcd646f319414df88c7d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-05-20T21:16:04.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:04 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:04.491+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:04.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:04 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:04.491+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:04.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:04 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:04.491+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:04.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:04 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:04.491+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:04.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:04 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:04.491+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:04.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:04 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:04.491+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:04.863 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-20T21:16:04.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:04.903 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-20T21:16:04.903 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-20T21:16:04.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.9 2024-05-20T21:16:05.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:04 smithi026 systemd[1]: Stopping Ceph osd.9 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.012+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.012+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.012+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.012+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.012+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.012+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.192+0000 7fd89f2c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:05.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.192+0000 7fd89f2c2700 -1 osd.9 84 *** Got signal Terminated *** 2024-05-20T21:16:05.194 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.192+0000 7fd89f2c2700 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:16:05.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:04 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:04.902+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:05.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:04 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:04.902+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:05.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:04 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:04.902+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:05.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:04 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:04.902+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:05.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:04 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:04.902+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:05.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:04 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:04.902+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:05.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:05 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:05.446+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:05.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:05 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:05.446+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:05.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:05 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:05.446+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:05.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:05 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:05.446+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:05.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:05 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:05.446+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:05.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:05 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:05.446+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:06.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:05.895+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:06.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:05.895+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:06.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:05.895+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:06.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:05.895+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:06.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:05.895+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:06.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:05.895+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:06.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:06.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:06.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:06.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:06.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:06.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:05 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:05.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:06.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:06 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:06.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:06.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:06 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:06.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:06.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:06 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:06.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:06.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:06 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:06.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:06.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:06 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:06.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:06.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:06 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:06.448+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:07.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:06 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:06.936+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:07.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:06 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:06.936+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:07.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:06 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:06.936+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:07.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:06 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:06.936+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:07.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:06 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:06.936+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:07.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:06 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:06.936+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:07.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:07.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:07.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:07.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:07.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:07.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:07.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.003+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:43.198573+0000 front 2024-05-20T21:15:43.198590+0000 (oldest deadline 2024-05-20T21:16:06.698574+0000) 2024-05-20T21:16:07.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:07 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:07.455+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:07.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:07 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:07.455+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:07.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:07 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:07.455+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:07.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:07 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:07.455+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:07.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:07 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:07.455+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:07.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:07 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:07.455+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:08.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:08.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:08.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:08.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:08.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:08.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:08.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:07 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:07.896+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:08.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:08.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:08.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:08.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:08.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:08.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:08.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:07 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:07.977+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:43.198573+0000 front 2024-05-20T21:15:43.198590+0000 (oldest deadline 2024-05-20T21:16:06.698574+0000) 2024-05-20T21:16:08.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:08.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:08.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:08.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:08.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:08.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:08.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:08.438+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:09.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:09.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:09.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:09.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:09.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:09.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:09.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:08 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:08.920+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:09.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:09.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:09.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:09.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:09.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:09.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:09.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:08 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:08.998+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:43.198573+0000 front 2024-05-20T21:15:43.198590+0000 (oldest deadline 2024-05-20T21:16:06.698574+0000) 2024-05-20T21:16:09.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:09.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:09.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:09.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:09.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:09.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:09.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:09 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:09.453+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:10.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:10.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:10.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:10.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:10.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:10.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:10.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:09 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:09.897+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:10.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:54.790341+0000 front 2024-05-20T21:14:54.790397+0000 (oldest deadline 2024-05-20T21:15:17.690517+0000) 2024-05-20T21:16:10.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:01.791956+0000 front 2024-05-20T21:15:01.791775+0000 (oldest deadline 2024-05-20T21:15:25.891924+0000) 2024-05-20T21:16:10.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.693836+0000 front 2024-05-20T21:15:11.693796+0000 (oldest deadline 2024-05-20T21:15:36.393845+0000) 2024-05-20T21:16:10.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:19.294829+0000 front 2024-05-20T21:15:19.294963+0000 (oldest deadline 2024-05-20T21:15:42.794989+0000) 2024-05-20T21:16:10.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:25.696265+0000 front 2024-05-20T21:15:25.696218+0000 (oldest deadline 2024-05-20T21:15:50.396193+0000) 2024-05-20T21:16:10.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:36.297247+0000 front 2024-05-20T21:15:36.297441+0000 (oldest deadline 2024-05-20T21:16:00.297733+0000) 2024-05-20T21:16:10.453 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9[67351]: 2024-05-20T21:16:10.030+0000 7fd89b8b6700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:43.198573+0000 front 2024-05-20T21:15:43.198590+0000 (oldest deadline 2024-05-20T21:16:06.698574+0000) 2024-05-20T21:16:10.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:10.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:10.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:10.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:10.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:10.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:10.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:10.488+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:10.952 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:10 smithi026 podman[143535]: 2024-05-20 21:16:10.676350421 +0000 UTC m=+5.565020534 container died aa9f945a324442255b75770e9388de1caf44c88dc605c2d1a8fb3c4dfed759d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-20T21:16:11.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:11.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:11.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:11.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:11.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:11.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:11.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:10 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:10.890+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:11.326 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:11 smithi026 podman[143535]: 2024-05-20 21:16:11.125803464 +0000 UTC m=+6.014473598 container cleanup aa9f945a324442255b75770e9388de1caf44c88dc605c2d1a8fb3c4dfed759d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-20T21:16:11.327 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:11 smithi026 bash[143535]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9 2024-05-20T21:16:11.702 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:11 smithi026 podman[143557]: 2024-05-20 21:16:11.327341647 +0000 UTC m=+0.642315152 container remove aa9f945a324442255b75770e9388de1caf44c88dc605c2d1a8fb3c4dfed759d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0) 2024-05-20T21:16:11.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:11.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:11.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:11.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:11.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:11.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:11.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:11 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:11.531+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:12.071 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:11 smithi026 podman[143644]: 2024-05-20 21:16:11.663264911 +0000 UTC m=+0.018039453 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:16:12.071 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:11 smithi026 podman[143644]: 2024-05-20 21:16:11.76137509 +0000 UTC m=+0.116149663 container create 313c933da2d92bd4d38920a8cffe63ef8a520910b8111270c1ab4de86030f2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T21:16:12.071 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:11 smithi026 podman[143644]: 2024-05-20 21:16:11.997089351 +0000 UTC m=+0.351863882 container init 313c933da2d92bd4d38920a8cffe63ef8a520910b8111270c1ab4de86030f2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:16:12.071 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:12 smithi026 podman[143644]: 2024-05-20 21:16:12.000599646 +0000 UTC m=+0.355374177 container start 313c933da2d92bd4d38920a8cffe63ef8a520910b8111270c1ab4de86030f2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-05-20T21:16:12.072 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:12 smithi026 podman[143644]: 2024-05-20 21:16:12.07209269 +0000 UTC m=+0.426867232 container attach 313c933da2d92bd4d38920a8cffe63ef8a520910b8111270c1ab4de86030f2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0) 2024-05-20T21:16:12.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:12.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:12.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:12.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:12.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:12.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:12.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:11 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:11.915+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:12.452 INFO:journalctl@ceph.osd.9.smithi026.stdout:May 20 21:16:12 smithi026 podman[143644]: 2024-05-20 21:16:12.310773041 +0000 UTC m=+0.665547572 container died 313c933da2d92bd4d38920a8cffe63ef8a520910b8111270c1ab4de86030f2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True) 2024-05-20T21:16:12.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-20T21:16:12.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:12.907 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-20T21:16:12.908 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-20T21:16:12.908 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.10 2024-05-20T21:16:13.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:13.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:13.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:13.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:13.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:13.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:13.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:12 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:12.580+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:13.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:13.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:13.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:13.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:13.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:13.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:13.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:12.931+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:13.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:12 smithi055 systemd[1]: Stopping Ceph osd.10 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:13.701 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.229+0000 7f1f42bd9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:13.701 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.229+0000 7f1f42bd9700 -1 osd.10 84 *** Got signal Terminated *** 2024-05-20T21:16:13.701 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.229+0000 7f1f42bd9700 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:16:14.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:14.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:14.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:14.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:14.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:14.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:14.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:13.621+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:14.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:14.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:14.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:14.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:14.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:14.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:14.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:13 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:13.883+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:14.953 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:14.953 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:14.953 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:14.953 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:14.953 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:14.953 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:14.954 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:14 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:14.652+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:15.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:15.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:15.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:15.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:15.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:15.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:15.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:14 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:14.919+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:15.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:15.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:15.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:15.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:15.802 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:15.802 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:15.802 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:15 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:15.612+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:16.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:16.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:16.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:16.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:16.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.913+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:16.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.914+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:16.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:15 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:15.914+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:17.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:17.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:17.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:17.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:17.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:17.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:17.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:16 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:16.637+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:17.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.880+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:17.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:17.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:17.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:17.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:17.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:17.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:16 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:16.881+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:18.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:18.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:17 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:17.686+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:18.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.888737+0000 front 2024-05-20T21:14:51.888839+0000 (oldest deadline 2024-05-20T21:15:17.788951+0000) 2024-05-20T21:16:18.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:00.690363+0000 front 2024-05-20T21:15:00.690341+0000 (oldest deadline 2024-05-20T21:15:25.990538+0000) 2024-05-20T21:16:18.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:11.792389+0000 front 2024-05-20T21:15:11.792269+0000 (oldest deadline 2024-05-20T21:15:34.092310+0000) 2024-05-20T21:16:18.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:20.994087+0000 front 2024-05-20T21:15:20.994073+0000 (oldest deadline 2024-05-20T21:15:45.094106+0000) 2024-05-20T21:16:18.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:29.194960+0000 front 2024-05-20T21:15:29.194974+0000 (oldest deadline 2024-05-20T21:15:54.995912+0000) 2024-05-20T21:16:18.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.496917+0000 front 2024-05-20T21:15:35.496988+0000 (oldest deadline 2024-05-20T21:16:00.196820+0000) 2024-05-20T21:16:18.202 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:17 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10[60131]: 2024-05-20T21:16:17.871+0000 7f1f3b1c5700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:44.898084+0000 front 2024-05-20T21:15:44.898146+0000 (oldest deadline 2024-05-20T21:16:07.098273+0000) 2024-05-20T21:16:18.951 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:18 smithi055 podman[72364]: 2024-05-20 21:16:18.607135672 +0000 UTC m=+5.494572173 container died 3666ef2c8f55ee4bd50b6e4fed7b2e2304b79c9b9344f3ef97bd74bc13ef8d4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-20T21:16:19.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:19.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:18.678+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:19.329 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:19 smithi055 podman[72364]: 2024-05-20 21:16:19.070794656 +0000 UTC m=+5.958231155 container cleanup 3666ef2c8f55ee4bd50b6e4fed7b2e2304b79c9b9344f3ef97bd74bc13ef8d4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T21:16:19.330 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:19 smithi055 bash[72364]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10 2024-05-20T21:16:19.673 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:19 smithi055 podman[72378]: 2024-05-20 21:16:19.330194509 +0000 UTC m=+0.722693590 container remove 3666ef2c8f55ee4bd50b6e4fed7b2e2304b79c9b9344f3ef97bd74bc13ef8d4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-20T21:16:19.929 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:19 smithi055 podman[72469]: 2024-05-20 21:16:19.577481383 +0000 UTC m=+0.017599960 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:16:19.929 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:19 smithi055 podman[72469]: 2024-05-20 21:16:19.798291432 +0000 UTC m=+0.238409988 container create 134125dbe6bf98c3ba6f4756d981980c15cc3a91e43a00018dcd851059d7b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-20T21:16:20.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:20.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:20.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:20.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:20.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:20.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:20.052 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:20.053 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:19 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:19.709+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:20.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:20 smithi055 podman[72469]: 2024-05-20 21:16:20.059013031 +0000 UTC m=+0.499131581 container init 134125dbe6bf98c3ba6f4756d981980c15cc3a91e43a00018dcd851059d7b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:16:20.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:20 smithi055 podman[72469]: 2024-05-20 21:16:20.062378401 +0000 UTC m=+0.502496963 container start 134125dbe6bf98c3ba6f4756d981980c15cc3a91e43a00018dcd851059d7b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T21:16:20.201 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:20 smithi055 podman[72469]: 2024-05-20 21:16:20.13390521 +0000 UTC m=+0.574023781 container attach 134125dbe6bf98c3ba6f4756d981980c15cc3a91e43a00018dcd851059d7b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-05-20T21:16:20.653 INFO:journalctl@ceph.osd.10.smithi055.stdout:May 20 21:16:20 smithi055 podman[72469]: 2024-05-20 21:16:20.370683481 +0000 UTC m=+0.810802041 container died 134125dbe6bf98c3ba6f4756d981980c15cc3a91e43a00018dcd851059d7b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-20T21:16:21.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:21.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:20 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:20.668+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:21.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-20T21:16:21.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:21.111 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-20T21:16:21.111 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-20T21:16:21.111 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.11 2024-05-20T21:16:21.411 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 systemd[1]: Stopping Ceph osd.11 for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:21.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.411+0000 7f1cc3a19700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:21.800 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.411+0000 7f1cc3a19700 -1 osd.11 84 *** Got signal Terminated *** 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.411+0000 7f1cc3a19700 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:21.801 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:21.802 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:21.802 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:21 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:21.640+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:23.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:23.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:22 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:22.628+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:24.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:24.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:23 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:23.587+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:25.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:25.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:24 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:24.597+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:26.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-05-20T21:14:51.551116+0000 front 2024-05-20T21:14:51.551037+0000 (oldest deadline 2024-05-20T21:15:17.451139+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6804 osd.1 since back 2024-05-20T21:15:03.054298+0000 front 2024-05-20T21:15:03.054243+0000 (oldest deadline 2024-05-20T21:15:26.554334+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-05-20T21:15:09.955720+0000 front 2024-05-20T21:15:09.955768+0000 (oldest deadline 2024-05-20T21:15:35.855790+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-05-20T21:15:17.556840+0000 front 2024-05-20T21:15:17.556724+0000 (oldest deadline 2024-05-20T21:15:42.256842+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6812 osd.4 since back 2024-05-20T21:15:28.157962+0000 front 2024-05-20T21:15:28.157948+0000 (oldest deadline 2024-05-20T21:15:50.457815+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-05-20T21:15:35.158798+0000 front 2024-05-20T21:15:35.158775+0000 (oldest deadline 2024-05-20T21:15:59.858949+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-05-20T21:15:42.060329+0000 front 2024-05-20T21:15:42.060423+0000 (oldest deadline 2024-05-20T21:16:07.960212+0000) 2024-05-20T21:16:26.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:25 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11[60711]: 2024-05-20T21:16:25.626+0000 7f1cc000d700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.55:6820 osd.7 since back 2024-05-20T21:15:50.861414+0000 front 2024-05-20T21:15:50.861443+0000 (oldest deadline 2024-05-20T21:16:16.761234+0000) 2024-05-20T21:16:27.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:26 smithi144 podman[73700]: 2024-05-20 21:16:26.799302509 +0000 UTC m=+5.479489915 container died 84ea143d80a2e85d210812faad70a0e273a97e15ec5a72a835deaba8c5301f23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-20T21:16:27.437 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 podman[73700]: 2024-05-20 21:16:27.161976069 +0000 UTC m=+5.842163475 container cleanup 84ea143d80a2e85d210812faad70a0e273a97e15ec5a72a835deaba8c5301f23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-20T21:16:27.437 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 bash[73700]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11 2024-05-20T21:16:27.786 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 podman[73714]: 2024-05-20 21:16:27.438094868 +0000 UTC m=+0.630296613 container remove 84ea143d80a2e85d210812faad70a0e273a97e15ec5a72a835deaba8c5301f23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:16:28.050 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 podman[73805]: 2024-05-20 21:16:27.689469957 +0000 UTC m=+0.017246453 image pull 559f9f2978373b56e1ed37e8f2207b076e9eb3d6ccfe36983da65ed32ed7d489 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3 2024-05-20T21:16:28.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 podman[73805]: 2024-05-20 21:16:27.789025875 +0000 UTC m=+0.116802355 container create 4f7a8d02f0a3cbce7740c4343697840be143d4eac178c6a0e7bc0807ecd2dd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:16:28.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 podman[73805]: 2024-05-20 21:16:27.983713956 +0000 UTC m=+0.311490449 container init 4f7a8d02f0a3cbce7740c4343697840be143d4eac178c6a0e7bc0807ecd2dd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5) 2024-05-20T21:16:28.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:27 smithi144 podman[73805]: 2024-05-20 21:16:27.987171674 +0000 UTC m=+0.314948163 container start 4f7a8d02f0a3cbce7740c4343697840be143d4eac178c6a0e7bc0807ecd2dd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T21:16:28.550 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:28 smithi144 podman[73805]: 2024-05-20 21:16:28.078122336 +0000 UTC m=+0.405898821 container attach 4f7a8d02f0a3cbce7740c4343697840be143d4eac178c6a0e7bc0807ecd2dd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-05-20T21:16:28.551 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:28 smithi144 podman[73805]: 2024-05-20 21:16:28.283994026 +0000 UTC m=+0.611770525 container died 4f7a8d02f0a3cbce7740c4343697840be143d4eac178c6a0e7bc0807ecd2dd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-20T21:16:28.838 INFO:journalctl@ceph.osd.11.smithi144.stdout:May 20 21:16:28 smithi144 podman[73832]: 2024-05-20 21:16:28.836683915 +0000 UTC m=+0.547082099 container remove 4f7a8d02f0a3cbce7740c4343697840be143d4eac178c6a0e7bc0807ecd2dd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-20T21:16:28.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-20T21:16:28.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:28.903 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-20T21:16:28.903 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-20T21:16:28.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.a 2024-05-20T21:16:29.202 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 21:16:28 smithi026 systemd[1]: Stopping Ceph mds.a for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:29.453 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 21:16:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a[74558]: 2024-05-20T21:16:29.201+0000 7fcc43e9c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:29.454 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 21:16:29 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a[74558]: 2024-05-20T21:16:29.201+0000 7fcc43e9c700 -1 mds.a *** got signal Terminated *** 2024-05-20T21:16:32.485 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 21:16:32 smithi026 podman[143750]: 2024-05-20 21:16:32.232799649 +0000 UTC m=+3.128683515 container died f0eb87a4777e22387ccff7fdb06962e1c2462c839819d6f1bc1f9067c3c7037e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T21:16:32.952 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 21:16:32 smithi026 podman[143750]: 2024-05-20 21:16:32.635269836 +0000 UTC m=+3.531153744 container cleanup f0eb87a4777e22387ccff7fdb06962e1c2462c839819d6f1bc1f9067c3c7037e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5) 2024-05-20T21:16:32.952 INFO:journalctl@ceph.mds.a.smithi026.stdout:May 20 21:16:32 smithi026 bash[143750]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-a 2024-05-20T21:16:33.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-20T21:16:33.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:33.167 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-20T21:16:33.167 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-20T21:16:33.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.d 2024-05-20T21:16:33.497 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:33 smithi026 systemd[1]: Stopping Ceph mds.d for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:33.952 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d[73330]: 2024-05-20T21:16:33.497+0000 7f2878b9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:33.952 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:33 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d[73330]: 2024-05-20T21:16:33.497+0000 7f2878b9a700 -1 mds.d *** got signal Terminated *** 2024-05-20T21:16:35.323 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 podman[143869]: 2024-05-20 21:16:35.05793492 +0000 UTC m=+1.668394982 container died e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-20T21:16:35.589 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 podman[143869]: 2024-05-20 21:16:35.397483767 +0000 UTC m=+2.007943829 container cleanup e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:16:35.590 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 bash[143869]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d 2024-05-20T21:16:35.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-20T21:16:35.858 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 podman[143883]: 2024-05-20 21:16:35.590510851 +0000 UTC m=+0.529875850 container remove e9bd9787d02a4c6274c8d5dfcd3a859b3b9f0bbb0f14ea2d8726abe0959928b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-20T21:16:35.859 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.d.service: Deactivated successfully. 2024-05-20T21:16:35.859 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 systemd[1]: Stopped Ceph mds.d for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T21:16:35.859 INFO:journalctl@ceph.mds.d.smithi026.stdout:May 20 21:16:35 smithi026 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.d.service: Consumed 1.175s CPU time. 2024-05-20T21:16:35.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:35.871 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-20T21:16:35.871 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-20T21:16:35.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.g 2024-05-20T21:16:36.192 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:35 smithi026 systemd[1]: Stopping Ceph mds.g for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:36.452 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g[74162]: 2024-05-20T21:16:36.192+0000 7f8d83bc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:36.452 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:36 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g[74162]: 2024-05-20T21:16:36.192+0000 7f8d83bc9700 -1 mds.g *** got signal Terminated *** 2024-05-20T21:16:40.467 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:40 smithi026 podman[143991]: 2024-05-20 21:16:40.207162327 +0000 UTC m=+4.113115290 container died 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T21:16:40.827 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:40 smithi026 podman[143991]: 2024-05-20 21:16:40.642977363 +0000 UTC m=+4.548930315 container cleanup 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-05-20T21:16:40.827 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:40 smithi026 bash[143991]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g 2024-05-20T21:16:41.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-20T21:16:41.096 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:40 smithi026 podman[144006]: 2024-05-20 21:16:40.828041066 +0000 UTC m=+0.621386238 container remove 2ef8b14ea257dc97c0d61254aeffc8d4245d8fd3e0942b98f51833bfc7bd1097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-20T21:16:41.096 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:41 smithi026 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-20T21:16:41.096 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:41 smithi026 systemd[1]: Stopped Ceph mds.g for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T21:16:41.097 INFO:journalctl@ceph.mds.g.smithi026.stdout:May 20 21:16:41 smithi026 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.g.service: Consumed 1.160s CPU time. 2024-05-20T21:16:41.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:41.108 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-20T21:16:41.108 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-20T21:16:41.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.j 2024-05-20T21:16:41.446 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:41 smithi026 systemd[1]: Stopping Ceph mds.j for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:41.702 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:41 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-j[72917]: 2024-05-20T21:16:41.446+0000 7f77ad2a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:41.702 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:41 smithi026 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-j[72917]: 2024-05-20T21:16:41.446+0000 7f77ad2a8700 -1 mds.j *** got signal Terminated *** 2024-05-20T21:16:43.702 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:43 smithi026 podman[144112]: 2024-05-20 21:16:43.588484956 +0000 UTC m=+2.248911074 container died 258689f544bbd9bfbdc96f071f790ce3457a8a94dffc45e6365ab476d53111af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5) 2024-05-20T21:16:44.452 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:44 smithi026 podman[144112]: 2024-05-20 21:16:44.189208204 +0000 UTC m=+2.849634317 container cleanup 258689f544bbd9bfbdc96f071f790ce3457a8a94dffc45e6365ab476d53111af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-j, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T21:16:44.452 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:44 smithi026 bash[144112]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-j 2024-05-20T21:16:44.452 INFO:journalctl@ceph.mds.j.smithi026.stdout:May 20 21:16:44 smithi026 podman[144137]: 2024-05-20 21:16:44.390498141 +0000 UTC m=+0.805620456 container remove 258689f544bbd9bfbdc96f071f790ce3457a8a94dffc45e6365ab476d53111af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-20T21:16:44.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-20T21:16:44.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:44.663 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-20T21:16:44.663 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-20T21:16:44.664 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.b 2024-05-20T21:16:44.951 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 21:16:44 smithi055 systemd[1]: Stopping Ceph mds.b for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:45.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 21:16:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b[65759]: 2024-05-20T21:16:44.981+0000 7efc2f87c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:45.451 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 21:16:44 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b[65759]: 2024-05-20T21:16:44.981+0000 7efc2f87c700 -1 mds.b *** got signal Terminated *** 2024-05-20T21:16:53.845 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 21:16:53 smithi055 podman[72587]: 2024-05-20 21:16:53.534980839 +0000 UTC m=+8.666394943 container died cf6c4f297a7567ca4f8e928d10b6074f70ec3bd68d8655096271aef399ccc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-20T21:16:54.201 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 21:16:53 smithi055 podman[72587]: 2024-05-20 21:16:53.997681262 +0000 UTC m=+9.129095354 container cleanup cf6c4f297a7567ca4f8e928d10b6074f70ec3bd68d8655096271aef399ccc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-20T21:16:54.201 INFO:journalctl@ceph.mds.b.smithi055.stdout:May 20 21:16:53 smithi055 bash[72587]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-b 2024-05-20T21:16:54.414 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-20T21:16:54.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:54.452 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-20T21:16:54.452 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-20T21:16:54.452 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.e 2024-05-20T21:16:54.785 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:54 smithi055 systemd[1]: Stopping Ceph mds.e for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:55.201 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e[65137]: 2024-05-20T21:16:54.784+0000 7f8983dff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:55.201 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:54 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e[65137]: 2024-05-20T21:16:54.784+0000 7f8983dff700 -1 mds.e *** got signal Terminated *** 2024-05-20T21:16:58.410 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 podman[72718]: 2024-05-20 21:16:58.056761796 +0000 UTC m=+3.370964567 container died 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:16:58.701 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 podman[72718]: 2024-05-20 21:16:58.510314961 +0000 UTC m=+3.824517720 container cleanup 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-20T21:16:58.701 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 bash[72718]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e 2024-05-20T21:16:58.954 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-20T21:16:58.976 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 podman[72731]: 2024-05-20 21:16:58.72257429 +0000 UTC m=+0.665205777 container remove 19dfeeff83560cd848aae9e5e28f6ea9b69efb2ab0140e226c60e7d4ce2afd01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-20T21:16:58.977 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.e.service: Deactivated successfully. 2024-05-20T21:16:58.977 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 systemd[1]: Stopped Ceph mds.e for 9006c62a-16e9-11ef-bc9a-c7b262605968. 2024-05-20T21:16:58.977 INFO:journalctl@ceph.mds.e.smithi055.stdout:May 20 21:16:58 smithi055 systemd[1]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.e.service: Consumed 7.655s CPU time. 2024-05-20T21:16:58.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:16:58.989 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-20T21:16:58.990 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-20T21:16:58.990 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.h 2024-05-20T21:16:59.447 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:16:59 smithi055 systemd[1]: Stopping Ceph mds.h for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:16:59.701 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:16:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:16:59.447+0000 7fb12fb0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:16:59.701 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:16:59 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h[65442]: 2024-05-20T21:16:59.447+0000 7fb12fb0e700 -1 mds.h *** got signal Terminated *** 2024-05-20T21:17:04.951 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:17:04 smithi055 podman[72842]: 2024-05-20 21:17:04.620186305 +0000 UTC m=+5.408366952 container died bacb3afa4c4ed7dcf666bb29c407dd2f07e3d801efb47cca2252088e1c825e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h, RELEASE=wip-lusov-quiesce-4ebd1d5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-20T21:17:05.356 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:17:05 smithi055 podman[72842]: 2024-05-20 21:17:05.097475389 +0000 UTC m=+5.885656036 container cleanup bacb3afa4c4ed7dcf666bb29c407dd2f07e3d801efb47cca2252088e1c825e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-20T21:17:05.356 INFO:journalctl@ceph.mds.h.smithi055.stdout:May 20 21:17:05 smithi055 bash[72842]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-h 2024-05-20T21:17:05.495 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-20T21:17:05.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:17:05.529 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-20T21:17:05.529 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-20T21:17:05.530 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.k 2024-05-20T21:17:05.701 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:05 smithi055 systemd[1]: Stopping Ceph mds.k for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:17:06.451 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k[64827]: 2024-05-20T21:17:05.951+0000 7f6478f74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:17:06.451 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:05 smithi055 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k[64827]: 2024-05-20T21:17:05.951+0000 7f6478f74700 -1 mds.k *** got signal Terminated *** 2024-05-20T21:17:07.534 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:07 smithi055 podman[72964]: 2024-05-20 21:17:07.255139761 +0000 UTC m=+1.503587570 container died dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-20T21:17:07.876 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:07 smithi055 podman[72964]: 2024-05-20 21:17:07.526331196 +0000 UTC m=+1.774778995 container cleanup dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-20T21:17:07.876 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:07 smithi055 bash[72964]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k 2024-05-20T21:17:07.876 INFO:journalctl@ceph.mds.k.smithi055.stdout:May 20 21:17:07 smithi055 podman[72978]: 2024-05-20 21:17:07.760980068 +0000 UTC m=+0.498848409 container remove dc604ff494df3a9bd3f1c98a48b9df1c2657e2fd325a2c43328c4b41cc15f8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-20T21:17:07.890 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-20T21:17:07.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:17:07.922 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-20T21:17:07.923 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-20T21:17:07.923 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.c 2024-05-20T21:17:08.235 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:17:07 smithi144 systemd[1]: Stopping Ceph mds.c for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:17:08.490 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:17:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: 2024-05-20T21:17:08.235+0000 7f4d73e7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:17:08.490 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:17:08 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c[65953]: 2024-05-20T21:17:08.235+0000 7f4d73e7c700 -1 mds.c *** got signal Terminated *** 2024-05-20T21:17:09.852 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:17:09 smithi144 podman[73911]: 2024-05-20 21:17:09.569594468 +0000 UTC m=+1.446948176 container died a4e344c6d0bd7b067dc8f07eda983f63a25d3fd9ed49cb5e33f6e9c1715d6b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-20T21:17:10.202 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:17:09 smithi144 podman[73911]: 2024-05-20 21:17:09.943943214 +0000 UTC m=+1.821296917 container cleanup a4e344c6d0bd7b067dc8f07eda983f63a25d3fd9ed49cb5e33f6e9c1715d6b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-20T21:17:10.203 INFO:journalctl@ceph.mds.c.smithi144.stdout:May 20 21:17:09 smithi144 bash[73911]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-c 2024-05-20T21:17:10.332 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-20T21:17:10.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:17:10.369 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-20T21:17:10.369 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-20T21:17:10.369 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.f 2024-05-20T21:17:10.550 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:10 smithi144 systemd[1]: Stopping Ceph mds.f for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:17:11.050 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f[65654]: 2024-05-20T21:17:10.723+0000 7f6fedc9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:17:11.050 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:10 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f[65654]: 2024-05-20T21:17:10.723+0000 7f6fedc9b700 -1 mds.f *** got signal Terminated *** 2024-05-20T21:17:12.997 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:12 smithi144 podman[74043]: 2024-05-20 21:17:12.677312442 +0000 UTC m=+2.089231307 container died f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T21:17:12.997 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:12 smithi144 podman[74043]: 2024-05-20 21:17:12.998036787 +0000 UTC m=+2.409955649 container cleanup f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-4ebd1d5) 2024-05-20T21:17:13.300 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:12 smithi144 bash[74043]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f 2024-05-20T21:17:13.300 INFO:journalctl@ceph.mds.f.smithi144.stdout:May 20 21:17:13 smithi144 podman[74056]: 2024-05-20 21:17:13.242459832 +0000 UTC m=+0.558397108 container remove f4802ebc1add3ca5c896952bf839c47d4f4fc3ac95ba455c9fa8e7fefead081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-4ebd1d5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-20T21:17:13.482 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-20T21:17:13.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:17:13.515 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-20T21:17:13.515 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-20T21:17:13.516 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.i 2024-05-20T21:17:13.800 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 21:17:13 smithi144 systemd[1]: Stopping Ceph mds.i for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:17:14.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 21:17:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i[66270]: 2024-05-20T21:17:13.830+0000 7fd832478700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:17:14.300 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 21:17:13 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i[66270]: 2024-05-20T21:17:13.830+0000 7fd832478700 -1 mds.i *** got signal Terminated *** 2024-05-20T21:17:17.934 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 21:17:17 smithi144 podman[74164]: 2024-05-20 21:17:17.670841147 +0000 UTC m=+3.940289619 container died 12ed66c231dd8dbc8729f2a6b642bea2e795c0d9b351f826de1b02e900dfe8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-4ebd1d5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T21:17:18.193 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 21:17:18 smithi144 podman[74164]: 2024-05-20 21:17:18.010184131 +0000 UTC m=+4.279632601 container cleanup 12ed66c231dd8dbc8729f2a6b642bea2e795c0d9b351f826de1b02e900dfe8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-4ebd1d5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-20T21:17:18.194 INFO:journalctl@ceph.mds.i.smithi144.stdout:May 20 21:17:18 smithi144 bash[74164]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-i 2024-05-20T21:17:18.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-20T21:17:18.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:17:18.444 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-20T21:17:18.444 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-20T21:17:18.444 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.l 2024-05-20T21:17:18.780 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:18 smithi144 systemd[1]: Stopping Ceph mds.l for 9006c62a-16e9-11ef-bc9a-c7b262605968... 2024-05-20T21:17:19.050 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-l[65346]: 2024-05-20T21:17:18.780+0000 7f945c399700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T21:17:19.051 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:18 smithi144 ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-l[65346]: 2024-05-20T21:17:18.780+0000 7f945c399700 -1 mds.l *** got signal Terminated *** 2024-05-20T21:17:20.713 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:20 smithi144 podman[74286]: 2024-05-20 21:17:20.448827966 +0000 UTC m=+1.780510286 container died 287aed43d193ad03a40e64134d922289c49f635061613090ab100d930facabf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-l, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T21:17:20.713 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:20 smithi144 podman[74286]: 2024-05-20 21:17:20.714230493 +0000 UTC m=+2.045912805 container cleanup 287aed43d193ad03a40e64134d922289c49f635061613090ab100d930facabf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-l, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-20T21:17:21.050 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:20 smithi144 bash[74286]: ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-l 2024-05-20T21:17:21.050 INFO:journalctl@ceph.mds.l.smithi144.stdout:May 20 21:17:20 smithi144 podman[74300]: 2024-05-20 21:17:20.956797014 +0000 UTC m=+0.500416975 container remove 287aed43d193ad03a40e64134d922289c49f635061613090ab100d930facabf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2eb2a5f16dc82fc2fb58caa939fa48240b0c05b2a3ce0f75aa77a9d2415604e3, name=ceph-9006c62a-16e9-11ef-bc9a-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-4ebd1d5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T21:17:21.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9006c62a-16e9-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-20T21:17:21.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T21:17:21.130 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-20T21:17:21.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-20T21:17:21.332 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T21:17:23.875 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-20T21:17:24.076 INFO:teuthology.orchestra.run.smithi055.stdout:Deleting cluster with fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T21:17:26.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-20T21:17:26.959 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T21:17:29.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T21:17:29.379 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T21:17:29.413 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T21:17:29.439 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-20T21:17:29.442 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi026/crash 2024-05-20T21:17:29.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash -- . 2024-05-20T21:17:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-20T21:17:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-05-20T21:17:29.481 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi055/crash 2024-05-20T21:17:29.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash -- . 2024-05-20T21:17:29.586 INFO:teuthology.orchestra.run.smithi055.stderr:tar: /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-20T21:17:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:tar: Error is not recoverable: exiting now 2024-05-20T21:17:29.589 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi144/crash 2024-05-20T21:17:29.590 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash -- . 2024-05-20T21:17:29.629 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-20T21:17:29.629 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-05-20T21:17:29.631 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-20T21:17:29.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-20T21:17:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:2024-05-20T21:10:05.809733+0000 mds.h (mds.1) 2 : cluster [WRN] Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/39566b0c-c8a4-4314-b820-c3de24137f95/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-05-20T21:17:29.670 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-20T21:17:29.670 INFO:tasks.cephadm:Compressing logs... 2024-05-20T21:17:29.671 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T21:17:29.714 DEBUG:teuthology.orchestra.run.smithi055:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T21:17:29.716 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-20T21:17:29.746 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-20T21:17:29.746 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-05-20T21:17:29.746 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log 2024-05-20T21:17:29.747 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-20T21:17:29.747 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-20T21:17:29.747 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-20T21:17:29.749 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose --/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.f.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-20T21:17:29.749 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-20T21:17:29.749 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-20T21:17:29.749 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-20T21:17:29.750 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.5.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-20T21:17:29.750 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-20T21:17:29.750 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-20T21:17:29.750 INFO:teuthology.orchestra.run.smithi144.stderr: 87.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-20T21:17:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log 2024-05-20T21:17:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-20T21:17:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-20T21:17:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-20T21:17:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.0.log: 90.3% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.a.log: 94.6% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi026.stderr: 94.6%gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-20T21:17:29.757 INFO:teuthology.orchestra.run.smithi144.stderr: 93.4% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-20T21:17:29.757 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-20T21:17:29.757 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-20T21:17:29.758 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.i.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-20T21:17:29.758 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.11.log: 83.2% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-20T21:17:29.759 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.6.log: 94.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-20T21:17:29.759 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-20T21:17:29.759 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-20T21:17:29.759 INFO:teuthology.orchestra.run.smithi026.stderr: 83.3% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-20T21:17:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-20T21:17:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-20T21:17:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log 2024-05-20T21:17:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-20T21:17:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-20T21:17:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-20T21:17:29.763 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-20T21:17:29.764 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-20T21:17:29.764 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-20T21:17:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-20T21:17:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-20T21:17:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-20T21:17:29.768 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-20T21:17:29.770 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-20T21:17:29.778 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-20T21:17:29.780 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-20T21:17:29.807 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.l.log: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-20T21:17:29.810 INFO:teuthology.orchestra.run.smithi144.stderr: 94.7% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-20T21:17:29.811 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.b.log: /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.k.log: 83.3% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-20T21:17:29.868 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-20T21:17:29.892 INFO:teuthology.orchestra.run.smithi055.stderr: 93.2% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-20T21:17:29.971 INFO:teuthology.orchestra.run.smithi055.stderr: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-20T21:17:29.979 INFO:teuthology.orchestra.run.smithi055.stderr: 90.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi055.stderr: 94.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-20T21:17:30.090 INFO:teuthology.orchestra.run.smithi055.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-20T21:17:30.101 INFO:teuthology.orchestra.run.smithi026.stderr: 90.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-20T21:17:30.308 INFO:teuthology.orchestra.run.smithi144.stderr: 91.7% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-20T21:17:30.485 INFO:teuthology.orchestra.run.smithi144.stderr: 95.2% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-20T21:17:31.850 INFO:teuthology.orchestra.run.smithi026.stderr: 90.6% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-20T21:17:32.599 INFO:teuthology.orchestra.run.smithi055.stderr: 91.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-20T21:17:34.586 INFO:teuthology.orchestra.run.smithi144.stderr: 95.0% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-20T21:17:37.952 INFO:teuthology.orchestra.run.smithi026.stderr: 91.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-20T21:17:38.029 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-20T21:17:38.859 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-20T21:17:39.222 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-20T21:17:39.536 INFO:teuthology.orchestra.run.smithi144.stderr: 90.8% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-20T21:17:40.100 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-20T21:17:40.194 INFO:teuthology.orchestra.run.smithi144.stderr: 90.9% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-20T21:17:42.438 INFO:teuthology.orchestra.run.smithi144.stderr: 90.9% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-20T21:17:49.453 INFO:teuthology.orchestra.run.smithi144.stderr: 95.0% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-20T21:17:49.455 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-05-20T21:17:49.455 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m19.724s 2024-05-20T21:17:49.455 INFO:teuthology.orchestra.run.smithi144.stderr:user 1m6.172s 2024-05-20T21:17:49.455 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m1.934s 2024-05-20T21:18:08.755 INFO:teuthology.orchestra.run.smithi055.stderr: 95.0% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-20T21:18:33.223 INFO:teuthology.orchestra.run.smithi055.stderr: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-20T21:18:39.588 INFO:teuthology.orchestra.run.smithi055.stderr: 91.3% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-20T21:19:04.175 INFO:teuthology.orchestra.run.smithi055.stderr: 91.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-20T21:19:06.384 INFO:teuthology.orchestra.run.smithi055.stderr: 90.9% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-20T21:20:12.033 INFO:teuthology.orchestra.run.smithi026.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-20T21:20:12.035 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-05-20T21:20:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:real 2m42.306s 2024-05-20T21:20:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:user 3m14.761s 2024-05-20T21:20:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m5.549s 2024-05-20T21:23:47.756 INFO:teuthology.orchestra.run.smithi055.stderr: 94.4% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-20T21:26:54.598 INFO:teuthology.orchestra.run.smithi055.stderr: 95.1% -- replaced with /var/log/ceph/9006c62a-16e9-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-20T21:26:54.601 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T21:26:54.601 INFO:teuthology.orchestra.run.smithi055.stderr:real 9m24.867s 2024-05-20T21:26:54.601 INFO:teuthology.orchestra.run.smithi055.stderr:user 5m3.706s 2024-05-20T21:26:54.601 INFO:teuthology.orchestra.run.smithi055.stderr:sys 0m12.391s 2024-05-20T21:26:54.602 INFO:tasks.cephadm:Archiving logs... 2024-05-20T21:26:54.602 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi026/log 2024-05-20T21:26:54.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T21:27:23.119 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi055/log 2024-05-20T21:27:23.120 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T21:28:01.111 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi144/log 2024-05-20T21:28:01.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T21:28:10.332 INFO:tasks.cephadm:Removing cluster... 2024-05-20T21:28:10.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --force 2024-05-20T21:28:10.536 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T21:28:10.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --force 2024-05-20T21:28:11.405 INFO:teuthology.orchestra.run.smithi055.stdout:Deleting cluster with fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T21:28:11.954 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9006c62a-16e9-11ef-bc9a-c7b262605968 --force 2024-05-20T21:28:12.154 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 9006c62a-16e9-11ef-bc9a-c7b262605968 2024-05-20T21:28:12.490 INFO:tasks.cephadm:Removing cephadm ... 2024-05-20T21:28:12.490 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-20T21:28:12.510 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-20T21:28:12.537 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-20T21:28:12.553 INFO:tasks.cephadm:Teardown complete 2024-05-20T21:28:12.553 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-20T21:28:12.564 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_4fbe5e8dc899215f216853b8569a107d8be8861d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ebd1d5361d457738957dc0455bba90102296634/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-20T21:28:12.566 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-20T21:28:12.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T21:28:12.569 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T21:28:12.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T21:28:12.678 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-20T21:28:12.689 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-20T21:28:12.689 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T21:28:12.692 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T21:28:12.694 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T21:28:12.707 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-05-20T21:28:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-05-20T21:28:12.708 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-05-20T21:28:12.711 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T21:28:12.712 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-05-20T21:28:12.712 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +1122us[+1135us] +/- 58ms 2024-05-20T21:28:12.712 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -655us[ -655us] +/- 56ms 2024-05-20T21:28:12.712 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -482us[ -469us] +/- 74ms 2024-05-20T21:28:12.712 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T21:28:12.714 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T21:28:12.714 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-05-20T21:28:12.714 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1108us[+1109us] +/- 58ms 2024-05-20T21:28:12.714 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -673us[ -673us] +/- 56ms 2024-05-20T21:28:12.714 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -491us[ -491us] +/- 74ms 2024-05-20T21:28:12.714 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T21:28:12.742 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T21:28:12.742 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-05-20T21:28:12.742 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +1079us[+1077us] +/- 58ms 2024-05-20T21:28:12.742 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -704us[ -706us] +/- 56ms 2024-05-20T21:28:12.742 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -529us[ -532us] +/- 74ms 2024-05-20T21:28:12.742 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T21:28:12.743 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-20T21:28:12.753 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-20T21:28:12.754 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-20T21:28:12.778 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T21:28:12.876 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T21:28:12.996 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T21:28:13.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T21:28:13.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:28:13.095 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T21:28:13.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:28:13.125 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T21:28:13.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:28:13.153 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-20T21:28:13.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-20T21:28:13.178 INFO:teuthology.task.internal:Duration was 3100.431645 seconds 2024-05-20T21:28:13.178 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-20T21:28:13.189 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-20T21:28:13.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T21:28:13.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T21:28:13.195 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T21:28:13.233 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T21:28:13.233 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T21:28:13.358 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T21:28:13.749 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-20T21:28:13.750 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-05-20T21:28:13.750 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T21:28:13.774 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T21:28:13.774 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T21:28:13.799 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-05-20T21:28:13.799 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T21:28:13.820 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-20T21:28:13.821 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T21:28:13.823 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T21:28:13.842 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T21:28:13.888 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-20T21:28:13.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T21:28:13.893 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T21:28:13.910 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T21:28:15.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-20T21:28:15.592 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-20T21:28:15.592 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T21:28:15.618 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T21:28:15.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T21:28:15.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-20T21:28:15.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T21:28:15.705 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T21:28:15.708 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T21:28:15.727 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-05-20T21:28:15.748 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2024-05-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-05-20T21:28:15.787 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T21:28:15.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:28:15.803 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T21:28:15.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:28:15.824 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T21:28:15.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:28:15.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-20T21:28:15.856 INFO:teuthology.task.internal:Transferring archived files... 2024-05-20T21:28:15.857 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi026 2024-05-20T21:28:15.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T21:28:15.933 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi055 2024-05-20T21:28:15.933 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T21:28:15.973 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983/remote/smithi144 2024-05-20T21:28:15.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T21:28:16.012 INFO:teuthology.task.internal:Removing archive directory... 2024-05-20T21:28:16.012 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T21:28:16.014 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T21:28:16.017 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T21:28:16.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-20T21:28:16.077 INFO:teuthology.task.internal:Not uploading archives. 2024-05-20T21:28:16.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-20T21:28:16.090 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-20T21:28:16.090 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T21:28:16.093 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T21:28:16.095 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T21:28:16.106 INFO:teuthology.orchestra.run.smithi026.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 21:28 /home/ubuntu/cephtest 2024-05-20T21:28:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 21:28 /home/ubuntu/cephtest 2024-05-20T21:28:16.120 INFO:teuthology.orchestra.run.smithi144.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 21:28 /home/ubuntu/cephtest 2024-05-20T21:28:16.121 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-20T21:28:16.134 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-20T21:28:16.156 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-20T21:28:16.157 INFO:teuthology.nuke:Checking targets against current locks 2024-05-20T21:28:16.179 DEBUG:teuthology.nuke:shortname: smithi026 2024-05-20T21:28:16.179 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T21:28:16.200 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 20:28:06.300152', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH94DB/ArPbA9slEZyw7UsFPtn415LxD70Ohu53eSzEK2M4tmVNGzI0YTZCJSEbbJvF+paqSGOOu9/GBtbx8XTQ='} 2024-05-20T21:28:16.204 DEBUG:teuthology.nuke:shortname: smithi055 2024-05-20T21:28:16.204 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T21:28:16.226 DEBUG:teuthology.nuke:shortname: smithi144 2024-05-20T21:28:16.226 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T21:28:16.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 20:28:06.297437', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOceghuAt2xCh7SAA4kXy89Vpu7EiGPRy1B8UX+5OL5Ni/AdkLR95xz6wQrjQ0WNt1TUcaPCdspHNMwKawvhso='} 2024-05-20T21:28:16.255 INFO:teuthology.orchestra.console.smithi026:Power off 2024-05-20T21:28:16.256 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-20T21:28:16.272 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-20_18:42:22-fs-wip-lusov-quiesce-distro-default-smithi/7715983', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 20:28:06.298894', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDe6kYwkIY8LgqrD4JJLzygwA2Ng83ubO4qAQSdBwB1ytf8aUDzGlU/eqTp06AvCPdyPSaoTV6httuOjRe7jCJk='} 2024-05-20T21:28:16.280 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2024-05-20T21:28:16.281 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:16.302 INFO:teuthology.orchestra.console.smithi055:Power off 2024-05-20T21:28:16.302 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-20T21:28:16.314 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-05-20T21:28:16.327 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2024-05-20T21:28:16.328 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:16.341 INFO:teuthology.orchestra.console.smithi144:Power off 2024-05-20T21:28:16.341 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-20T21:28:16.358 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-05-20T21:28:16.369 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-05-20T21:28:16.369 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:16.394 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-20T21:28:20.316 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:20.359 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:20.395 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:20.427 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-05-20T21:28:20.470 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-05-20T21:28:20.504 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-20T21:28:24.431 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:24.471 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:24.505 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-20T21:28:24.542 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2024-05-20T21:28:24.582 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-05-20T21:28:24.616 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-05-20T21:28:24.644 INFO:teuthology.orchestra.console.smithi026:Power off completed 2024-05-20T21:28:24.683 INFO:teuthology.orchestra.console.smithi055:Power off completed 2024-05-20T21:28:24.717 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-05-20T21:28:24.824 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 3100.431644678116 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi026 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ebd1d5361d457738957dc0455bba90102296634 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ab11c32662754336bc56741b3b161c7b status: fail success: false 2024-05-20T21:28:24.824 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T21:28:24.914 INFO:teuthology.run:FAIL